Foundation-model selection has moved beyond a single question—“which model is smartest?” Modern teams compose a portfolio: a high-capability model for ambiguous reasoning, a lower-cost model for repeated extraction, embedding models for retrieval, and sometimes an open-weight model for controlled environments. Treat the model as a component with observable performance, not as a magical employee.
Assess capability in the context of work
General benchmarks are useful signals but poor procurement decisions when used alone. Build a small evaluation set from de-identified real tasks: support tickets, policy questions, contract clauses, product records, or planning requests. Include easy cases, typical cases and known traps. Grade outputs for factual grounding, formatting, completeness, safety and the amount of human correction required.
Frontier proprietary models generally offer the best broad reasoning, coding and multimodal performance. They are quick to trial and receive frequent improvements, but prices and behaviour can change. Smaller proprietary models often deliver faster responses and materially lower token costs for classification, summarisation and structured extraction. Open-weight models can offer more deployment control and tuning flexibility, but require infrastructure, monitoring and a team that can operate them.
Four practical buying dimensions
1. Quality and reliability
Ask for valid structured output rates, citation behaviour and consistency across repeated runs. A model that produces a polished but unsupported answer is risky in regulated or customer-facing work. Retrieval-augmented generation can improve grounding, but it cannot repair missing source material or a poorly designed retrieval pipeline.
2. Cost and latency
API pricing is normally metered by input and output tokens, while self-hosted systems add GPU, engineering and uptime costs. Model cost is only one part of unit economics: include retrieved context, retries, tool calls and review time. Route straightforward requests to a small model, set output limits, cache stable answers and reserve premium reasoning for work that demonstrates incremental value.
3. Data controls
Confirm retention terms, training-use settings, encryption, regional processing and enterprise identity controls. For sensitive data, minimise what is sent: redact identifiers, retrieve only relevant passages, and keep source-of-truth actions behind authenticated tools. “Private” should be a contract and architecture decision, not a prompt instruction.
4. Integration fit
Check JSON or schema-constrained output, function calling, streaming, rate limits, model version pinning and audit logs. The model must fit the surrounding system. A reliable workflow needs timeouts, fallbacks, clear error states and a way to replay failed jobs.
A sensible model portfolio
Begin with one vendor and two models: a capable default and an economical high-volume option. Add a second provider where resilience, negotiation leverage or a unique capability justifies the operational overhead. Use embeddings selected on retrieval quality for your corpus, not only on published benchmark rank. If local deployment is a requirement, run a proof on representative hardware before committing; quantisation and context length can alter real quality.
Do not fine-tune first. Prompting, retrieval, structured outputs and workflow design solve many early problems. Fine-tuning becomes credible when you have a stable task, hundreds or thousands of high-quality examples, and a measurable baseline to beat. Maintain versioned prompts and evaluation results so a vendor update does not quietly alter production performance.
Decision rule
Choose the lowest-cost model that passes a pre-agreed quality floor on your evaluation set. Re-test monthly and after material model releases. That approach prevents both under-buying capability and overspending on intelligence that the workflow does not need.