Local platform overlay

AlphaSwarm provides two ways to run the platform locally:

1. alphaswarm-local (Recommended): A standalone packaging of the AlphaSwarm backend that can be linked to the hosted UI via a device-paired reverse tunnel. This is the preferred method for hybrid research and local backtesting.
2. Compose Overlay: A traditional Docker Compose setup using overlays for data and visualization services.

1. alphaswarm-local (Hybrid)

The alphaswarm_local package enables a "Bring Your Own Backend" (BYOB) model. You run the backend locally, and it dials out to the hosted AlphaSwarm platform, allowing you to use the hosted UI with your local data and compute.

See the alphaswarm_local README for installation and linking instructions.

2. Platform overlay (Standalone)

Audience: a developer who wants to run AlphaSwarm standalone, without attaching to the rpi_kubernetes cluster or the hosted platform. The platform overlay (alphaswarm_platform/compose/docker-compose.platform.yml) brings the data + observability services AlphaSwarm code expects into the local compose stack.

The rpi kubernetes/ tree stays untouched — these are copies, not relocations. AlphaSwarm attaches to either the local services or the cluster through the KubernetesAdapter abstraction.

Compose-up matrix

Goal	Command
Just the AlphaSwarm API + workers	docker compose up -d
AlphaSwarm + visualization stack (Trino, Polaris, Superset, Dagster, Dask, Ray)	docker compose -f alphaswarm_platform/compose/docker-compose.yml -f alphaswarm_platform/compose/docker-compose.viz.yml --profile visualization up -d
Full local platform parity (adds Apicurio + real Airbyte + DataHub + Loki + Vector + VictoriaMetrics)	docker compose -f alphaswarm_platform/compose/docker-compose.yml -f alphaswarm_platform/compose/docker-compose.viz.yml -f alphaswarm_platform/compose/docker-compose.platform.yml --profile visualization --profile platform up -d

The platform overlay also activates the visualization profile's services it depends on (Polaris, Trino, Dagster). Don't pass --profile platform alone — the AlphaSwarm webui still depends on Superset from the viz overlay.

Services added by the platform overlay

Service	Container	Default host port	Wires into
apicurio (Schema Registry)	alphaswarm-apicurio	8090 -> 8080	ALPHASWARM_SCHEMA_REGISTRY_URL already supports the URL knob
airbyte-db	alphaswarm-airbyte-db	(internal)	Postgres backing for real Airbyte
airbyte-server-real	alphaswarm-airbyte-server-real	8005 -> 8001	Real Airbyte API (the dev stub at airbyte-server keeps running on :8002)
airbyte-webapp	alphaswarm-airbyte-webapp	8001 -> 80	UI for real Airbyte
datahub-gms	alphaswarm-datahub-gms	8081 -> 8080	ALPHASWARM_DATAHUB_GMS_URL=http://datahub-gms:8080
datahub-frontend	alphaswarm-datahub-frontend	9002 -> 9002	DataHub UI
loki	alphaswarm-loki	3100 -> 3100	Log aggregation; OTel collector + agents push here
vector	alphaswarm-vector	(none)	Tails Docker container logs and ships to Loki
victoriametrics	alphaswarm-victoriametrics	8428 -> 8428	Long-term metrics; scrapes the existing OTel collector + AlphaSwarm API

Sub-profiles (documented but not enabled by default)

The plan keeps these out of the default platform set because the user opted out of "full parity":

• platform-rag — RAGFlow + Milvus stack (heavy; pulls a vector DB).
• platform-jh — JupyterHub.

Add them yourself if needed by extending alphaswarm_platform/compose/docker-compose.platform.yml or shipping an alongside docker-compose.platform.<profile>.yml.

Smoke test sequence

1. docker compose -f alphaswarm_platform/compose/docker-compose.yml -f alphaswarm_platform/compose/docker-compose.viz.yml -f alphaswarm_platform/compose/docker-compose.platform.yml --profile visualization --profile platform up -d
2. curl http://localhost:8428/-/ready — VictoriaMetrics
3. curl http://localhost:3100/ready — Loki
4. curl http://localhost:8081/health — DataHub GMS
5. curl http://localhost:8090/apis — Apicurio
6. curl http://localhost:8005/api/v1/health — real Airbyte
7. docker compose ps — every service should be healthy or running

Where the rpi cluster fits in

When ALPHASWARM_CLUSTER_MGMT_URL is set, the RpiClusterAdapter auto-promotes and AlphaSwarm forwards Kafka admin + Flink session-job + alphavantage stream operations to the homelab management API. Setting both attach paths side-by-side is fine — AlphaSwarm routes the call wherever the active adapter says.

Cleanup

docker compose -f alphaswarm_platform/compose/docker-compose.yml -f alphaswarm_platform/compose/docker-compose.viz.yml -f alphaswarm_platform/compose/docker-compose.platform.yml --profile visualization --profile platform down

Volumes are preserved; pass -v to wipe them.