262,144
Token context window · Qwen3.8 native
We are an independent AI research lab in Germany, building language models efficient enough to run on hardware you own. Latest: a full-parameter distill of Qwen3.8-2.4T into 9B / 4B / 2B, and a GDN-aware 27B GGUF at 11.7 GiB. The open-weight Qwythos family has passed one million downloads on Hugging Face.
A full-parameter SFT of Qwen3.8-2.4T-A95B into the Qwen3.5 9B, 4B and 2B architectures. Same curriculum, three capacities: ~70k / 45k / 30k curated teacher traces of dense chain-of-thought — math, code, reasoning, instruction following, tool use. Every answer opens with a <think> block learned from the teacher, not from self-generated rollouts. Native function calling, 262,144-token context, day-one GGUF.
A GDN-aware mixed GGUF of official Qwen3.8-27B: 27B down to 11.73 GiB. State tensors stay Q8_0, Gated-DeltaNet mixers at Q4_K — not a flat 2-bit dump. Native MTP and the BF16 vision tower stay in the file. Wiki PPL 7.82 vs BF16 7.15 (+9.3%).
Hugging Face ↗The 9B that checks its own work. Distilled from 500M+ tokens of frontier traces, then FTPO’d to cut repetition looping from 6.7% to 0.0%. 1M-token context. The v2 GGUF alone is past 500K downloads.
Hugging Face ↗Every release ships day-one quantizations from Q4_K_M to BF16, with MTP draft variants and vision projectors. If you run llama.cpp, Ollama or vLLM, our models work the day they land.
All 25 models ↗A fast terminal agent in one native binary, pointed at any OpenAI-compatible endpoint — a frontier API, or a Qwythos on the machine in front of you. Abacus is where our models get used in anger: proof that small, owned models handle real codebases.
Our Final-Token Preference Optimization method identifies the exact token where a failure mode begins and trains the model to prefer coherent alternatives at that one position. ~2,000 preference tuples took Qwythos looping from 6.7% to 0.0% with no capability loss.
Our trace-generation tool produced the 500M+ tokens of verified chain-of-thought data behind Qwythos, and SFTSuite orders it into curricula. Data quality — not parameter count — is where our models win, and the data pipeline outlives any single release.
An LLM-in-the-loop system that searches architecture space before we commit full training runs — the reason Qwythos ships hybrid linear-attention designs that hold a 1M-token context on consumer hardware.
The same recipe produced Qwythos-9B and Qwythos-27B-v1, then the Qwen3.8 distill at 9B, 4B and 2B. Same teacher, same curriculum, three capacities — MMLU CoT jumps of +20.5 / +19.9 / +26.5. The pipeline scales with compute, which is exactly what this raise buys.
Two next-generation in-house models are in pretraining now, on the infrastructure and data pipelines we already operate. We won't describe them here — no roadmap theatre. We announce models when the weights are ready to download, not before. Every model we've shipped arrived this way; these will too.
Inference is leaving the data centre: a single GPU, an on-prem rack, a laptop. Getting frontier-class reasoning into models that small — models a company can actually own — is the whole of our research.
Qwythos is distilled, not fine-tuned: over 500 million tokens of reasoning traces from rethink, our in-house chain-of-thought tool. When the 9B shipped with a repetition loop, our FTPO method fixed it — 6.7% to 0.0% — with about 2,000 preference pairs, not a retrain.
We have no sales team and no ad budget. Qwythos passed one million downloads because engineers found it, used it and told other engineers — the v2 GGUF alone accounts for over 500,000. Open weights get the work tested, and adopted, at a scale we could not buy.
European enterprises increasingly cannot send regulated data to a foreign API. Qwythos is Apache-2.0, runs on-prem on a single GPU, and can be audited down to the weights. We build in Germany because that constraint is our market — and it is not going away.
Every number on this page is public. Check them: huggingface.co/empero-ai · github.com/empero-org.
A short letter, every other Tuesday: progress on Qwen3.8 and Qwythos, new Abacus releases, and the one thing we got wrong that week. No hype, no roadmap theatre. Cancel from any line.
The weights are on Hugging Face, the code is on GitHub, and the eval transcripts sit in the model cards. If you're running Qwythos and something breaks, if you need a model your auditors can actually read, or if you're wiring Abacus into your team's workflow — write to us.
We are independent and intend to stay that way. Research at this pace takes capital — conversations about that reach the same address.