SyncQueue reference¶
Experimental feature
This feature is experimental and should be considered unstable and unsupported. API may change at any time and without backwards compatibility.
sheppy.experimental.SyncQueue
¶
SyncQueue(
backend: str | None = None,
name: str | None = None,
*,
backend_kwargs: dict[str, Any] | None = None,
)
Synchronous wrapper around Queue for codebases without asyncio.
A dedicated asyncio event loop runs in a background daemon thread.
Every method submits the corresponding Queue coroutine to that loop
via asyncio.run_coroutine_threadsafe and blocks the calling thread
until the result is ready. The wrapped Queue and its backend always
run on the same loop, so the instance can be shared between threads;
long waits (e.g. wait_for with no timeout) block the calling
thread, not the loop.
Call close() when done to stop the loop and join its thread. The
thread is a daemon, so it does not block interpreter exit.
Unlike Queue, the backend can only be given as a URL string (or via
the SHEPPY_BACKEND_URL environment variable); the constructor does
not accept a Backend instance.
| PARAMETER | DESCRIPTION |
|---|---|
backend
|
Backend URL string, e.g. "redis://localhost:6379" or
"memory://". If None, uses
TYPE:
|
name
|
Name of the queue. Defaults to
TYPE:
|
backend_kwargs
|
Options forwarded to the backend constructor.
TYPE:
|
Example
q = SyncQueue("redis://localhost:6379")
added = q.add(task)
assert added is task
q.close()
Source code in src/sheppy/experimental/_sync_queue.py
51 52 53 54 55 56 57 58 59 60 61 | |
close
¶
close() -> None
Stop the background event loop and join its thread.
Safe to call more than once.
Source code in src/sheppy/experimental/_sync_queue.py
72 73 74 75 76 77 78 79 80 81 | |
add
¶
Add task into the queue. Accept list of tasks for batch add.
| PARAMETER | DESCRIPTION |
|---|---|
task
|
Instance of a Task, or list of Task instances for batch mode. |
| RETURNS | DESCRIPTION |
|---|---|
Task[R] | None | list[Task[R] | None]
|
The added task, or None if its ID already exists. In batch mode, |
Task[R] | None | list[Task[R] | None]
|
returns a list in input order with None for tasks that already exist. |
Source code in src/sheppy/experimental/_sync_queue.py
98 99 100 101 102 103 104 105 106 107 108 109 110 | |
get_task
¶
Get task by id.
| PARAMETER | DESCRIPTION |
|---|---|
task
|
Instance of a Task or its ID, or list of Task instances/IDs for batch mode. |
| RETURNS | DESCRIPTION |
|---|---|
Task[R] | None | dict[UUID, Task[R]]
|
Instance of a Task or None if not found. |
Source code in src/sheppy/experimental/_sync_queue.py
118 119 120 121 122 123 124 125 126 127 | |
get_all_tasks
¶
get_all_tasks() -> list[Task]
Get all tasks, including completed/failed ones.
| RETURNS | DESCRIPTION |
|---|---|
list[Task]
|
List of all tasks |
Source code in src/sheppy/experimental/_sync_queue.py
129 130 131 132 133 134 135 | |
get_pending
¶
get_pending(count: int = 1) -> list[Task]
List pending tasks.
| PARAMETER | DESCRIPTION |
|---|---|
count
|
Number of pending tasks to retrieve.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
list[Task]
|
List of pending tasks |
Source code in src/sheppy/experimental/_sync_queue.py
137 138 139 140 141 142 143 144 145 146 | |
schedule
¶
Schedule task to be processed after certain time.
| PARAMETER | DESCRIPTION |
|---|---|
task
|
Instance of a Task
TYPE:
|
at
|
When to process the task.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
Task[R] | None
|
The scheduled task, or None if its ID already exists. |
Source code in src/sheppy/experimental/_sync_queue.py
148 149 150 151 152 153 154 155 156 157 158 159 160 | |
get_scheduled
¶
get_scheduled() -> list[Task]
List scheduled tasks.
| RETURNS | DESCRIPTION |
|---|---|
list[Task]
|
List of scheduled tasks |
Source code in src/sheppy/experimental/_sync_queue.py
162 163 164 165 166 167 168 | |
retry
¶
retry(
task: Task[R] | UUID | str,
at: datetime | timedelta | None = None,
force: bool = False,
) -> Task[R] | None
Retry failed task.
| PARAMETER | DESCRIPTION |
|---|---|
task
|
Instance of a Task or its ID
TYPE:
|
at
|
When to retry the task.
TYPE:
|
force
|
If True, allows retrying even if task has completed successfully. Defaults to False.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
Task[R] | None
|
The re-queued task, or None if no task with the given ID exists. |
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If task has already completed successfully and force is not set to True. |
TypeError
|
If provided datetime is not offset-aware. |
Source code in src/sheppy/experimental/_sync_queue.py
170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 | |
cancel
¶
Cancel a pending or scheduled task.
| PARAMETER | DESCRIPTION |
|---|---|
task
|
Instance of a Task or its ID.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
Task
|
The updated Task instance with status 'cancelled'. |
| RAISES | DESCRIPTION |
|---|---|
TaskCancellationError
|
If the task cannot be cancelled - either it was already claimed by a worker, it already finished, or it does not exist. |
Source code in src/sheppy/experimental/_sync_queue.py
190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 | |
delete
¶
delete(task: Task | UUID | str) -> bool
Hard-delete a finished task's stored metadata.
Only tasks in a terminal state (completed, failed, crashed, cancelled) can be deleted.
| PARAMETER | DESCRIPTION |
|---|---|
task
|
Instance of a Task or its ID.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
bool
|
True if the task existed and was deleted, False if it was not found. |
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If the task has not finished yet. |
Source code in src/sheppy/experimental/_sync_queue.py
206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 | |
size
¶
size() -> int
Get number of pending tasks in the queue.
| RETURNS | DESCRIPTION |
|---|---|
int
|
Number of pending tasks |
Source code in src/sheppy/experimental/_sync_queue.py
222 223 224 225 226 227 228 | |
clear
¶
clear() -> int
Clear all tasks, including completed ones.
| RETURNS | DESCRIPTION |
|---|---|
int
|
Number of stored task entries removed. |
Source code in src/sheppy/experimental/_sync_queue.py
230 231 232 233 234 235 236 | |
add_cron
¶
Add a cron job to run a task on a schedule.
| PARAMETER | DESCRIPTION |
|---|---|
task
|
Instance of a Task
TYPE:
|
cron
|
Cron expression string (e.g. "*/5 * * * *" to run every 5 minutes)
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
TaskCron | None
|
The added cron, or None if an identical cron already exists. |
Source code in src/sheppy/experimental/_sync_queue.py
238 239 240 241 242 243 244 245 246 247 248 | |
delete_cron
¶
delete_cron(task: Task, cron: str) -> bool
Delete a cron job.
| PARAMETER | DESCRIPTION |
|---|---|
task
|
Instance of a Task
TYPE:
|
cron
|
Cron expression string used when adding the cron job
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
bool
|
Success boolean |
Source code in src/sheppy/experimental/_sync_queue.py
250 251 252 253 254 255 256 257 258 259 260 | |
get_crons
¶
get_crons() -> list[TaskCron]
List all cron jobs.
| RETURNS | DESCRIPTION |
|---|---|
list[TaskCron]
|
List of TaskCron instances |
Source code in src/sheppy/experimental/_sync_queue.py
262 263 264 265 266 267 268 | |
wait_for
¶
Wait for task to complete and return updated task instance.
Blocks the calling thread until the task finishes; use timeout
to bound the wait.
| PARAMETER | DESCRIPTION |
|---|---|
task
|
Instance of a Task or its ID, or list of Task instances/IDs for batch mode. |
timeout
|
Maximum time to wait in seconds. Default is 0 (wait indefinitely).
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
Task[R] | None | dict[UUID, Task[R]]
|
Instance of a Task or None if not found or timeout reached. |
| RAISES | DESCRIPTION |
|---|---|
TimeoutError
|
If timeout is reached and no task is found (only in non-batch mode). |
Source code in src/sheppy/experimental/_sync_queue.py
276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 | |
add_workflow
¶
add_workflow(workflow: Workflow) -> WorkflowResult
Add a workflow into the queue.
Source code in src/sheppy/experimental/_sync_queue.py
297 298 299 | |
wait_for_workflow
¶
Wait for workflow to finish (complete or fail) and return the updated workflow.
| PARAMETER | DESCRIPTION |
|---|---|
workflow
|
Instance of a Workflow or its ID, or list of Workflow instances/IDs for batch mode. |
timeout
|
Maximum time to wait in seconds. Default is 0 (wait indefinitely).
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
Workflow | None | dict[UUID, Workflow]
|
The finished Workflow, or None if not found. |
| RAISES | DESCRIPTION |
|---|---|
TimeoutError
|
If a positive timeout expires before all given workflows have finished. |
Source code in src/sheppy/experimental/_sync_queue.py
307 308 309 310 311 312 313 314 315 316 317 318 319 320 | |
get_workflow
¶
Get workflow by id.
| PARAMETER | DESCRIPTION |
|---|---|
workflow
|
Instance of a Workflow or its ID, or list of Workflow instances/IDs for batch mode. |
| RETURNS | DESCRIPTION |
|---|---|
Workflow | None | dict[UUID, Workflow]
|
Instance of a Workflow or None if not found. |
Source code in src/sheppy/experimental/_sync_queue.py
328 329 330 331 332 333 334 335 336 337 | |
get_all_workflows
¶
get_all_workflows() -> list[Workflow]
Get all stored workflows, including completed/failed ones.
| RETURNS | DESCRIPTION |
|---|---|
list[Workflow]
|
List of all Workflow instances |
Source code in src/sheppy/experimental/_sync_queue.py
339 340 341 342 343 344 345 | |
get_pending_workflows
¶
get_pending_workflows() -> list[Workflow]
Get workflows that are neither completed nor errored.
| RETURNS | DESCRIPTION |
|---|---|
list[Workflow]
|
List of pending Workflow instances |
Source code in src/sheppy/experimental/_sync_queue.py
347 348 349 350 351 352 353 | |
delete_workflow
¶
delete_workflow(workflow: Workflow | UUID | str) -> bool
Delete a workflow.
| PARAMETER | DESCRIPTION |
|---|---|
workflow
|
Instance of a Workflow or its ID
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
bool
|
True if the workflow existed and was deleted. |
Source code in src/sheppy/experimental/_sync_queue.py
355 356 357 358 359 360 361 362 363 364 | |