> tpl_air_012
Model-Selection ADR and Decision Matrix
Architectural Decision Record (ADR) and quantitative multi-criteria decision matrix evaluating proprietary API frontier models (OpenAI, Anthropic) versus self-hosted open-weights models (Llama, Mistral, Qwen) across data sovereignty, fine-tuning viability, cold-start latency, and total cost of ownership (TCO).
Quantitative ADR and decision matrix comparing proprietary APIs vs self-hosted open models on cost, privacy, and latency.
Important Tech Document Template & Operational Notice
TinyCTO.tv Tech Document Template Notice: This template is a general educational and operational starting point. It is not legal, tax, accounting, investment, procurement, regulatory, security or certification advice. Requirements vary by jurisdiction, organization, contract and risk. Review and adapt it with qualified professionals before relying on it.
Problem Solved
Engineering organizations make irreversible AI architectural bets based on emotional intuition rather than structured decision records, locking themselves into vendor API price hikes or immense self-hosted GPU infrastructure overhead.
When to Use
- •Deciding whether to build on hosted frontier APIs (OpenAI, Anthropic, Bedrock) or self-host open weights (vLLM on Kubernetes/Sovereign Cloud)
- •Conducting formal Architectural Decision Record (ADR) sign-offs for enterprise GenAI platforms and customer-facing copilot services
- •Modeling 3-year TCO inflection points where self-hosted open models become cheaper than commercial token billing
When NOT to Use
- •For broad cloud infrastructure vendor evaluations without AI specialization (use TPL-ARC-007)
- •For commercial software procurement contracts and Master Services Agreements (use TPL-PRC-006)
5 Template Sections & Structural Outline
Framing the decision: business workload requirements, latency thresholds, regulatory constraints, and volume projections.
Evaluating Option A (Proprietary Frontier API), Option B (Self-Hosted Open Weights on K8s), and Option C (Hybrid Gateway Routing).
Scoring candidates across 8 dimensions: Reasoning Capability, Latency/TTFT, Data Sovereignty, Fine-Tuning Ease, Vendor Lock-in, and TCO.
Calculating the token volume inflection point: where fixed GPU cluster hosting costs ($/month) cross variable commercial API token pricing.
Documenting the chosen option, positive/negative consequences, compliance validations, and triggers for re-evaluating the decision in 12 months.
Completion Instructions
Independent Review Checklist
- All mandatory sections completed
- No secrets or passwords included
- Executive sponsor sign-off obtained
Model-Selection ADR and Decision Matrix - Worked Case Study
Fictional Entity: Sovereign Banking Enterprise GenAI Model Selection ADR
Real-world production case study demonstrating complete operational adoption for Sovereign Banking Enterprise GenAI Model Selection ADR.
- •Executed formal MADR 3.0 decision record evaluating Azure OpenAI GPT-4o versus self-hosted Llama 3.3 70B on private OpenShift
- •Identified that regulatory PII restrictions prohibited third-party cloud APIs for core credit dispute adjudication
- •Adopted a hybrid architecture: public GPT-4o for internal marketing copy, self-hosted Llama 3.3 for sovereign banking core
Frequently Asked Questions
When does self-hosting an open-weights model become cheaper than commercial APIs?
The breakeven volume typically occurs when an enterprise processes between 10 million and 25 million tokens per day. Below this threshold, commercial APIs are vastly cheaper because you pay only for active tokens with zero fixed infrastructure or MLOps engineering overhead.
What data privacy guarantees must be legally secured when using proprietary frontier APIs?
Enterprises must mandate a signed Zero Data Retention (ZDR) agreement, explicit contractual commitments that prompts and outputs are never used for model training, and encryption in transit with customer-managed keys (CMEK/BYOK).
Why should enterprises adopt a hybrid LLM routing architecture rather than committing to a single model?
A hybrid gateway routes simple high-volume tasks (data extraction, formatting) to inexpensive, fast open models while reserving expensive frontier models (GPT-4o, Claude 3.5 Sonnet) for high-stakes complex reasoning, cutting total inferencing costs by 60% to 75%.
Download Tech Document Pack
Auth RequiredDownload all blank templates, worked scenarios, and verification manifests in a single verified archive.
Authoritative Sources
- MADR: Markdown Architectural Decision Records (Version 3.0)Architectural Decision Records Community • OFFICIAL REQUIREMENT
- ISO/IEC 42001:2023 Artificial Intelligence Management System — Section 8.2ISO/IEC • OFFICIAL REQUIREMENT
- Cloud FinOps Foundation: Managing Generative AI and LLM CostsFinOps Foundation • OFFICIAL REQUIREMENT