How We Chose a Base Model for Our Open LLMs (It's Not About the Benchmark)
Everyone asks ‘which model is smartest?’ After benchmarking, a blind bake-off, and one closed-weights dead-end, we ended up asking a different question entirely: which weights can you own, tune, and serve — on the same path?
We chose Qwen3-32B as the shared base for our domain models — not because it tops leaderboards, but because it passed four filters in order: open weights (a model you can’t download is a dependency, not an asset), a permissive Apache-2.0 license, being both tunable and servable on the same infrastructure path, and beating alternatives in a blind bake-off on our actual task. One base now carries multiple LoRA adapters — marketing and finance — hot-swapped on a single deployment.
When we committed to owned domain models, the first decision was the base. The instinct is to ask “what’s the smartest model?” — and it’s the wrong question. These are the research notes from choosing ours, kept honest because we changed our answer once along the way.
Filter 1: weights you can hold
The frontier API models are remarkable — and irrelevant here. A model whose weights you cannot download cannot be an asset on your balance sheet: you can’t audit it, can’t serve it where your data lives, and any fine-tune you buy on it is locked to the vendor. We verified the then-newest headline model (Grok 4.5, released that same month) against live sources rather than assumption: API-only, no published weights — disqualified in one line. So were every OpenAI and Anthropic flagship, for the same reason. This isn’t a criticism; it’s a category difference. Rented intelligence is a dependency. Owned intelligence is an asset.
Filter 2: a license your clients can live with
Open weights aren’t enough — the license has to permit commercial use and redistribution without strings. Apache-2.0 (Qwen’s license) makes our published adapters something a client can run in their own VPC without a legal review cycle.
Filter 3: tunable AND servable — on the same path
This is the filter almost nobody writes about, and it’s the one that changed our answer. We’d initially picked a 14B model: dense (LoRA-friendly), strong for its size, cheap to tune. Then we probed the serving side and found the trap — on our training platform the 14B trained fine but couldn’t be served as a hot-swappable adapter, while the 32B sibling was the smallest strong dense model that was both. A model you can tune but not deploy is a science project. We moved up to 32B for the serving path, not the benchmark.
| Candidate class | Verdict | Why |
|---|---|---|
| Frontier API models (GPT, Claude, Grok) | Disqualified | No weights — can’t own, audit, or self-serve |
| Mixture-of-experts giants | Skipped | Serving cost and complexity out of proportion to the task |
| Dense 7–14B open models | Trap | Tune cheaply, but our serving path couldn’t host their adapters |
| Dense ~32B open (Qwen3-32B) | Chosen | Apache-2.0, strong base, tunable and adapter-servable on one deployment |
Filter 4: a blind bake-off on your task, not a leaderboard
Leaderboards measure everything except your job. We ran candidates blind on our actual task — brief-driven B2B copy, judged without model identities. The result that settled it: the raw candidate base beat our existing fine-tuned model in 62% of head-to-head matchups. An untuned newer base outperforming our tuned older one was the clearest possible signal that base quality had moved — and that our value was never in the base anyway.
One base, many brains
The quiet economics of this choice: LoRA adapters on a shared base mean one deployment serves many domain models. Our marketing brain and our CFO brain are both adapters on the same Qwen3-32B, hot-swapped per request on a single GPU deployment that scales to zero when idle. Every new domain model from here costs a training run, not an infrastructure build-out.
The conclusion we’d defend anywhere: the base model is a commodity — pick it for ownership, license, and the tune-to-serve path. The dataset is the moat. The marketing build log shows what that moat costs to dig.
Frequently asked questions
What's the best base model for fine-tuning in 2026?
The wrong question — the right one is which model passes your filters: open weights you can hold, a license your clients accept, tunability AND servability on the same infrastructure path, and a blind win on your actual task. For our domain adapters that was Qwen3-32B.
Why not fine-tune GPT, Claude, or Grok?
Their weights aren’t published, so a fine-tune on them is a vendor-locked service, not an asset you own. For models you intend to publish, audit, or serve where your data lives, closed weights are disqualifying regardless of capability.
Are bigger base models better for fine-tuning?
Not per rupee. Dense models under ~32B are the LoRA sweet spot: big enough to reason, small enough to tune for tens of dollars and serve on a single GPU. Giant mixture-of-experts models mostly buy you serving complexity your task didn’t ask for.
Can one base model serve multiple fine-tuned adapters?
Yes — that’s the core economics of LoRA. Multiple domain adapters (ours: marketing and finance) attach to one deployment of the shared base and hot-swap per request, so each new domain model adds a training run, not a new server.
Owned models, chosen deliberately.
Both of our domain brains — and the base-model reasoning behind them — are public. See the family, evals attached.
See the models