Vortex Inference — System Architecture

Vortex Inference — System Architecture An architecture diagram generated by Archify. Client · OpenAI SDK · curl · Architecture component Client OpenAI SDK · curl OTLP collector · opt-in · e.g. Jaeger · Architecture component OTLP collector opt-in · e.g. Jaeger Remote LLM APIs · OpenAI-compatible · Anthropic · Architecture component Remote LLM APIs OpenAI-compatible · Anthropic Prometheus · Grafana · Alertmanager · Kubernetes (minikube) · vortex-inference + monitoring namespaces Prometheus Grafana · Alertmanager Vortex API · FastAPI /v1 · KEDA 1–3 pods · Kubernetes (minikube) · vortex-inference + monitoring namespaces Vortex API FastAPI /v1 · KEDA 1–3 pods Inference backend · llama.cpp · remote · fallback · Kubernetes (minikube) · vortex-inference + monitoring namespaces Inference backend llama.cpp · remote · fallback KEDA · ScaledObjects · Kubernetes (minikube) · vortex-inference + monitoring namespaces KEDA ScaledObjects RabbitMQ · vortex.jobs · retry · dead · Kubernetes (minikube) · vortex-inference + monitoring namespaces RabbitMQ vortex.jobs · retry · dead Redis · job records · 24h TTL · Kubernetes (minikube) · vortex-inference + monitoring namespaces Redis job records · 24h TTL vortex-worker · same backend · KEDA 0–N pods · Kubernetes (minikube) · vortex-inference + monitoring namespaces vortex-worker same backend · KEDA 0–N pods REST · SSE · jobs generate · stream remote · fallback OTLP spans scrape active requests queue length publish job job status consume result run job Kubernetes (minikube) · vortex-inference + monitoring namespaces Legend Backend Database Cloud Message bus External