Universities · use case
One cluster. Every faculty. No shadow IT.
“Every department wants AI. Every department also wants their AI to not see the other department’s data. Right now the answer is seven vendor subscriptions, zero of which IT actually approved.”
Universities face an awkward mix: a research community that adopts any AI tool that helps (often before asking legal), faculty boundaries that matter for IP and publication reasons, student-privacy rules that vary by jurisdiction, and a budget that can’t absorb seven vendor subscriptions.
Eldric AI OS is the shape of an institution-wide deployment: one cluster, faculty-scoped tenants, real accounts, course-level scopes, audit trail per-department. It runs in the university data centre, it speaks GDPR, and it doesn’t require a contract with anyone outside the institution.
Value propositions
One contract, one DPA, one audit
Replace N vendor subscriptions with one cluster. No per-faculty data-processing agreements. Central IT reviews one audit log.
Faculty tenants as code
Medicine, Engineering, Law, Arts, Business, Sciences — each a tenant. Workgroups for research labs. Projects for specific courses. Identity service enforces boundaries.
Sovereign posture
Open-source, EU-hosted repo, on-prem inference. Matches the EU Digital Education Action Plan and national data-sovereignty mandates.
AI teaching assistants, scoped
Per-course tenant. Students and teaching staff chat with a tool grounded in the course’s own materials — not someone’s training corpus. Same UI whether logged in from library or dorm.
Campus-wide memory
Per-department Matrix Memory. A PhD cohort graduates; the patterns they found don’t leave. Institutional continuity across hires.
Central admin + distributed autonomy
IT runs the cluster; each faculty runs its own tenant. Middle ground between “every dept does its own thing” and “IT does everything”.
AI-driven differentiator
A university isn’t a single enterprise; it’s dozens of semi-autonomous academic units sharing infrastructure. Consumer-AI tools assume a single org; vendor “enterprise AI” tools assume a single cost centre. Eldric’s multi-tenant model was built for this shape from day one. Add sovereign-AI posture and the institution can finally say yes to AI without saying yes to another US data-handling clause.
Scalable use cases
- Research. Per-group tenant. Matrix Memory for institutional recall. Science Worker surface for domain breadth.
- Teaching. Per-course tenant. AI TAs grounded on lecture notes + reading lists. Academic integrity tools as extensions.
- Libraries. Digital archive + chat. Rare-book collections get a chat interface that cites page-level. Scholars don’t need to fly in.
- Administration. Finance, HR, procurement with GDPR-shaped defaults. Audit log is the regulator-ready artefact.
- Student support. Wellbeing, career, admissions — per-service tenants.
session.localposture for students discussing sensitive issues.
Runs on commodity hardware
Eldric AI OS was built to land on small clusters, not on hyperscaler fleets. The whole stack is one binary; the on-prem LLM is embedded llama.cpp. The hardware plan that gets most organisations into production looks like this:
3× RTX 4090 — sweet spot
72 GB total VRAM with tensor-split. Llama 3.3 70B Q4 at 60–80 tok/s, a parallel 8B routing model, and an embedding server concurrently. One-time hardware cost ~€5–7k.
Single RTX 4090 / 4080 — team scale
24 GB. Llama 3.1 8B at 80+ tok/s, 13B comfortable, 32B Q4 possible. Enough for a small department chat with fan-out retrieval.
CPU-only — pilot scale
llama.cpp on 32+ core x86 runs 8B Q4 usefully. Matrix Memory is CPU-memory-bound. A refurbished server from the rack is enough to prove the architecture.
Scale up
Multi-node cluster with H100 / GH200 for research-grade workloads. Same binary, same role modules, topology-aware. See the HPC article.
Institutional baseline
For a 10–20k student university: one 3×4090 node for inference + one NFS-attached data node covers the full deployment. Scale by adding inference-role nodes as adoption grows.
The arithmetic: a €6k workstation displaces a €30–60k-per-year SaaS-AI contract that still leaks IP, still can’t reach your mainframe, and still has a “we may use your data for training” clause hiding somewhere.
What the disk bill looks like
| Artefact | Size | Notes |
|---|---|---|
eldric-aios-5.0.0-3.alpha3.fc43.x86_64.rpm | ~1.4 MB | CPU baseline binary; one RPM, one systemd unit. |
eldric-aios-cuda add-on | ~512 MB | Pulled in automatically via Supplements: cuda-drivers on GPU hosts. Contains GGML_CUDA llama.cpp. |
| Llama 3.1 8B Q4_K_M GGUF | ~4.9 GB | Good default for team-scale chat on a single 4090. |
| Llama 3.3 70B Q4_K_M GGUF | ~40 GB | The sweet spot for 3×4090 tensor-split. Holds a 16k context comfortably. |
| Mixtral 8x22B Q4 GGUF | ~80 GB | Tight on 3×4090; comfortable on 4×4090 or 2×H100. |
| nomic-embed-text (embedding) | ~700 MB | CPU or GPU. One per cluster; handles vector indexing. |
Matrix Memory .emm per domain | 50–500 MB | Depends on rank × dim (see memory article). chat 64/768 ~200 kB; particle_physics 512/1024 ~500 MB. |
| Vector store per 1M chunks | ~6–10 GB | Depends on embedding dim. SQLite backend; FAISS optional. |
| Hash-chained audit log | ~200 MB / 1M calls | JSONL, append-only, rotation at 500 MB files by default. |
Three reference hardware setups
| Pilot / team | Department / BU | Production / enterprise | |
|---|---|---|---|
| CPU | 1× EPYC 7313 (16c) or i9-14900K | 2× EPYC 9354 (32c each) | 2× EPYC 9654 (96c) per node |
| GPU | 1× RTX 4090 (24 GB) | 3× RTX 4090 (72 GB) | 4× H100 (320 GB) or 8× H200 |
| RAM | 128 GB DDR5 | 256 GB DDR5 ECC | 1 TB DDR5 ECC per node |
| Storage | 2× 4 TB NVMe (RAID-1) | 6× 8 TB NVMe (RAID-10) + SSD cache | Tiered: NVMe hot + TB-scale HDD / Lustre |
| Network | 1 GbE OK | 10 GbE with link agg | 25/100 GbE or IB-HDR for multi-node |
| Power | ~1 kW typical / 1.5 kW peak | ~2 kW typical / 3 kW peak | 4–6 kW per node |
| Hardware cost | ~€4–5k | ~€12–15k | €80–250k per node |
| Serves | 8B model, 10–30 concurrent chat users | 70B Q4 at 60–80 tok/s, 200–500 users | Mixtral / Llama-405B, 2k+ users per node |
Network + ops footprint
- Ports. One outward port (443 at the edge). Internally: controller on 8880, data on 8892, inference on 8883, science on 8897, etc. — all behind the edge.
- Storage layout.
${ELDRIC_DATA_DIR}defaults to/data/eldricif writable, else/var/lib/eldric. Subdirs:models/,vectors/,memory/(matrix memory),storage/(file storage),agent/,edge/, and per-module dirs. - Backup. The audit log and
.emmfiles are the two artefacts that matter. Everything else regenerates. Snapshot the data dir nightly; off-site every week. - Updates.
dnf upgrade eldric-aios. Rollback isdnf downgrade. Zero vendor dance. - Ops team. A single systems engineer can run a pilot install. A team of two runs a department deployment. Production enterprise uses your existing Linux sysadmin rota.
SWOT — an honest read
Strengths
- Identity service + multi-tenant shipped real in alpha.3
- Open-source kernel — aligns with academic publishing norms
- GDPR-shaped defaults:
session.local, right-to-erasure, audit log - Science Worker — 140+ APIs useful across faculties out of the box
Weaknesses
- SSO integrations (SAML/OIDC, eduGAIN) still Phase-4 roadmap
- Moodle / Canvas / Blackboard integration requires custom extensions
- Student-information system (SIS) connectors are bespoke today
- Accessibility certifications (WCAG audit) in preparation
Opportunities
- EU Digital Education Action Plan funding streams
- Growing regulatory pressure on shadow-AI in academia
- Open-science publishing reforms favouring sovereign infrastructure
- Recruitment pressure — “we have an AI platform” is a faculty-hiring pitch
Threats
- Microsoft 365 Education + Copilot bundled at discount prices
- Google Workspace EDU with Gemini
- Campus-side AI platforms already sunk costs
- Individual faculty tools embedded in vendor LMS
First entry points — concrete value in 30 / 90 / 180 days
Library pilot
Single-node install at the library. Digital-archive ingest. Open access for staff + a friendly faculty. Collect 100 queries of feedback.
First faculty tenant
Onboard a whole faculty (Sciences or Law are good choices). Per-department Matrix Memory. Teaching assistants for 2-3 courses.
Campus-wide rollout
All faculties live. Central IT runs the cluster; audit log reviewed monthly. Shadow-IT ChatGPT subscriptions decommissioned.