Major Flows

Pair with alphaswarm_docs/architecture.md (system view) and alphaswarm_docs/erd.md (data model). Doc map: alphaswarm_docs/index.md.

End-to-end sequence and state diagrams for the four flows that human and AI contributors most often need to reason about. Each diagram cites the canonical files; if the diagram and the code disagree, the code wins (and the doc is stale — please update).

1. Generic file → Iceberg ingestion

The discovery → director → materialise → verify → annotate pipeline that powers the regulatory-corpus ingest. Canonical doc: alphaswarm_docs/data-catalog.md.

Canonical files:

• alphaswarm/data/pipelines/discovery.py
• alphaswarm/data/pipelines/director.py
• alphaswarm/data/pipelines/materialize.py
• alphaswarm/data/pipelines/annotate.py
• alphaswarm/data/pipelines/runner.py
• alphaswarm/tasks/ingestion_tasks.py
• scripts/ingest_regulatory.py
• scripts/_run_one_source.py

2. Backtest dispatch

Canonical files:

• alphaswarm/api/routes/backtest.py
• alphaswarm/tasks/backtest_tasks.py
• alphaswarm/backtest/engine.py
• alphaswarm/backtest/runner.py
• alphaswarm/strategies/framework.py
• alphaswarm/persistence/ledger.py
• alphaswarm/mlops/autolog.py

3. Agentic crew run

The dual-tier (deep + quick LLM) CrewAI graph used by the TradingAgents-style preset. Files: alphaswarm/tasks/agentic_backtest_tasks.py, alphaswarm/agents/.

Canonical files:

• alphaswarm/api/routes/agentic.py
• alphaswarm/tasks/agentic_backtest_tasks.py
• alphaswarm/agents/
• alphaswarm/llm/providers/router.py

4. Paper trading session

The kill switch is a Redis key (ALPHASWARM_RISK_KILL_SWITCH_KEY, default alphaswarm:kill_switch); set it from anywhere to stop a session.

Canonical files:

• alphaswarm/api/routes/paper.py
• alphaswarm/tasks/paper_tasks.py
• alphaswarm/trading/runner.py
• alphaswarm/trading/session.py
• alphaswarm/risk/

5. (Bonus) Live-data subscription

Browser asks the API for a live data stream; API allocates a Redis pub/sub channel that bridges the broker feed to a WebSocket.

Canonical files:

• alphaswarm/api/routes/market_data_live.py
• alphaswarm/streaming/
• alphaswarm/ws/broker.py

Cross-cutting: progress bus

Every long-running task in AlphaSwarm uses the same progress bus pattern:

API to remember:

• emit(task_id, stage, message, **extras) — publish a progress frame.
• emit_done(task_id, result) — terminal stage="done" + result payload.
• emit_error(task_id, error) — terminal stage="error".

Don't publish to Redis directly from your task code; always go through alphaswarm/tasks/_progress.py so the frame shape stays consistent.