alphaswarm-worker

Celery orchestration worker pod that drains the light / coordination queues produced by alphaswarm-core.

As of the Phase 4c worker/executor split it has its own slim image (target worker) carrying only the task-dispatch + lineage surface — not the API stage's visualization / dev / Dash deps it used to inherit. Heavy compute (backtest / training / ML / agents / factors / RAG) is offloaded to the sibling alphaswarm-executor. See worker vs executor images for the full rationale and dependency matrix.

Identity

Field	Value
Service id	alphaswarm-worker
Role	worker
Package	alphaswarm/ (tasks under alphaswarm/tasks/*.py)
Image (key)	worker
Built from	alphaswarm_platform/Dockerfile (target worker, multi-arch) or the standalone build/docker/alphaswarm_worker/Dockerfile

Wire

Field	Value
Protocol	none (no HTTP listener)
Health	Celery broker connection probe + Prometheus metrics on :9100 (when enabled)
Public URL	—
Broker	redis://redis:6379/0
Result backend	redis://redis:6379/1

Deployment surfaces

Surface	Where
Compose	service worker in alphaswarm_platform/compose/docker-compose.yml; alphaswarm-worker in deployments/compose/docker-compose.local.yml
Kustomize	deployments/kubernetes/base/alphaswarm-worker/ — Deployment + HPA + PDB
AQP CR	folded into AQPMonolith (spec.workers.queues)
Terraform module	alphaswarm_platform/terraform/modules/faas/ — Celery + KEDA per-queue ScaledObjects

Queue families

The orchestration worker drains the light / coordination queues only. The heavy compute queues (backtest, training, ml, agents, factors, rag) are drained by alphaswarm-executor. KEDA scales each queue family independently. Default queue map:

Queue	Drives	Scale-to-zero	Notes
default	misc tasks, callbacks, lookups	yes	always-on min=1 in prod
paper	paper trading session ticks	no	sub-second latency required
terraform	TerraformRuntime celery wrappers	yes
ingestion	Airbyte / Dagster / connector pulls	no	uses long-lived workers
workflows	WorkflowRuntime orchestration	yes
hft	HFT hot-path event handlers	no	pinned to hft-nodes (compose/legacy)

nota

The faas KEDA module keys per-queue Deployments off local.heavy_queues — heavy queues run the alphaswarm-executor image, everything else runs this alphaswarm-worker image. The two image sets never share a queue.

Dependencies

Upstream:

• redis — broker + result backend.
• postgres — task lookups, ledger writes.
• alphaswarm-core — progress emit callbacks, lookup APIs.
• All data-plane services the alphaswarm-core pod depends on (the same code paths run inside Celery).

Downstream:

• Beat schedules tasks; the worker is the consumer.
• HFT-tagged tasks land on the hft-nodes/ workload (PTP-tuned).

Operations

• Scaling: KEDA ScaledObject per queue; idle queues scale to zero. The per-queue min/max lives in the faas Terraform module.
• Concurrency: the orchestration worker runs concurrency 4 (light, IO-bound dispatch work); 1 for HFT (single-threaded pinning).
• Drain on shutdown: terminationGracePeriodSeconds: 600 so in-flight tasks complete; preStop sends SIGTERM to Celery.
• Audit: WorkloadRuntime actions land workload_runs rows; the worker pod respects the kill-switch Redis key the same way the API does.

See also

• tasks-api.mdc — Celery task progress contract + Redis pub/sub frame shape.
• alphaswarm-executor.md — heavy-compute sibling.
• worker-executor-images.md — image split rationale + dependency matrix.
• alphaswarm-beat.md — sibling scheduler.
• faas Terraform module — KEDA scaling source of truth.