Controller Platform
Eldric Controller is an orchestration platform for running AI models in research, development, and enterprise environments. The solution enables connecting multiple inference servers into a scalable cluster with centralized management.
Core Features
- Cluster management with controller, router, and worker nodes
- Intelligent load balancing across multiple GPU machines
- Knowledge routing with automatic topic detection
- Multi-region deployment for global availability
- Integrated data worker for database connectivity
Research & Development
- Experiment with different models and backends
- A/B testing of different LLMs through the same API
- Quick switching between local and cloud models
- Integration of custom fine-tuned models
- Prototyping without vendor lock-in
Supported Backends (30+)
- Local: Ollama, llama.cpp, MLX (Apple Silicon)
- Enterprise: vLLM, HuggingFace TGI, NVIDIA Triton
- Cloud: OpenAI, Anthropic, Google Vertex AI, Azure
- Specialized: Replicate, Together AI, Groq, xAI, Fireworks
Use Cases
- On-premise AI for privacy-sensitive research
- Hybrid architectures with local and cloud backends
- High-availability AI services with automatic failover
- University and industrial research environments
The platform runs entirely on your own hardware – no data leaves your network. Ideal for research institutions and enterprises that need full control over their AI infrastructure.
Cluster Architecture
Multi-region deployment with edge gateways, intelligent routing, database connectivity, agentic RAG, multi-agent swarm, multimedia processing, and unified messaging
Documentation
Cluster Architecture
Complete reference for distributed AI infrastructure. Multi-region deployment, node types, port assignments, and sample configurations.
Data Worker
Database connectivity for AI workers. Connection pooling, PostgreSQL, MySQL, SQLite, and IBM DB2 (including z/OS mainframes).
Media Worker
Audio/video processing with STT (Whisper), TTS (Piper, ElevenLabs), video transcription, and multimedia RAG indexing.
Communication Worker
Unified messaging across Email, SMS, WhatsApp, Signal, Teams, and XMPP. AI auto-response with approval workflow.
Agent Worker
Multi-tenant agentic RAG with ReAct reasoning. Orchestrate AI agents for knowledge retrieval and tool execution.
Knowledge Routing
AI-powered request routing with theme detection. Route requests to specialized models based on content (medicine, legal, code, etc.).
AI Swarm
Multi-agent orchestration system. Autonomous goal execution with tiered swarm architecture, 6 specialized agents, MCP protocol for custom agents, and emergent behavior detection.
Controller Reference
Controller API documentation. REST endpoints, cluster management, dashboard, licensing, and configuration options.
Science Worker
Scientific research integration with 140+ APIs. NASA, ESA, CERN, genomics, climate data, particle physics, and academic databases.
IoT Worker
Smart home automation with Home Assistant, Matter/Thread, Zigbee, and Z-Wave. Industrial IIoT with OPC-UA, Modbus, and MQTT.
Edge Server
TLS termination, API key auth, rate limiting, embedded web chat, plugin system, and server-synced user settings across devices.
Cloud Worker
Multi-backend cloud inference gateway. Routes to OpenAI, Anthropic, xAI, Groq, Together, DeepSeek, and more with automatic model discovery.
Router
AI-powered load balancing with Swarm LLM ensemble strategies (debate, critique, vote), xLSTM predictor for workload forecasting, and intent detection.
Model Management
Pull, push, delete, and manage models across all cluster workers. Unified model registry with version tracking and deployment automation.
Distributed Inference
Split large LLMs across multiple workers. GGUF layer-level sharding, VRAM-aware scheduling, pipeline parallelism with automated llama.cpp RPC orchestration.
Training Worker
AI model fine-tuning with LoRA, QLoRA, DPO, RLHF. Native xLSTM backend. Multiple engines: Unsloth, TRL, MLX, Axolotl. Training chains and latent reasoning.
Native Inference Worker
Direct GGUF and xLSTM model loading via embedded llama.cpp. Zero cold-start, permanent VRAM residence, pipeline parallelism across nodes.
GUI Client
Native macOS/Windows app with visual workbenches. Training, model merging, RAG, MCP creator, agent designer, and 80+ screenshots.
Mobile Apps
Native iOS & Android apps with voice chat, cluster monitoring, unified messaging inbox, training jobs, and autonomous agent runtime.
DevToolBox
Developer productivity suite with Git forge integration, skills & automations, sandboxed execution, issue triage, and CI/CD monitoring.
Components
eldric-workerd
AI inference worker (Ollama, vLLM, TGI, etc.)
eldric-traind
AI model training & fine-tuning
Quick Start
Supported Databases
SQLite
Built-in, always available
PostgreSQL
via libpq
MySQL
via libmysqlclient
IBM DB2
ODBC + CLI, z/OS support
DevToolBox
Advanced developer tools for Git integration, CI/CD monitoring, and workflow automation
Git Forges
Unified Git integration with GitHub, GitLab, Gitea
/git forges, /git clone
Skills System
Reusable agent workflows for code review, testing, docs
/skill list, /skill run
Automations
Trigger skills on file changes, schedules, webhooks
/automation create
Sandboxes
Isolated Docker containers for safe code execution
/sandbox create
Issue Triage
AI-powered issue analysis, labeling, and fix suggestions
/triage analyze
CI/CD Monitor
Pipeline monitoring with failure analysis and auto-fixes
/cicd analyze
Supported CI: GitHub Actions, GitLab CI, Gitea Actions, Jenkins
Learn More →