Abstract

How should Large Language Model (LLM) practitioners select the right model for a task without wasting money? We introduce BELLA (Budget-Efficient LLM Selection via Automated skill-profiling), a framework that recommends optimal LLM selection for tasks through interpretable skill-based model selection. Standard benchmarks report aggregate metrics that obscure which specific capabilities a task requires and whether a cheaper model could suffice. BELLA addresses this gap through three stages: (1) decomposing LLM outputs and extract the granular skills required by using critic-based profiling, (2) clustering skills into structured capability matrices, and (3) multi-objective optimization to select the right models to maximize performance while respecting budget constraints. BELLA provides natural-language rationale for recommendations, providing transparency that current black-box routing systems lack. We describe the framework architecture, situate it within the landscape of LLM routing and evaluation, and discuss its application to financial reasoning as a representative domain exhibiting diverse skill requirements and cost-variation across models. Our framework enables practitioners to make principled and cost-performance trade-offs for deploying LLMs.

At a Glance

• Skill-based selection: decomposes model outputs into granular skills via critic-based profiling
• Structured capability matrices from clustered skills, feeding multi-objective optimization
• Natural-language rationale for every recommendation — transparency black-box routers lack
• Demonstrated on financial reasoning, a domain with diverse skill requirements and wide cost variation

Cite This Paper

@article{okamoto2026trust,
  title   = {Trust by Design: Skill Profiles for Transparent, Cost-Aware LLM Routing},
  author  = {Okamoto, Mika and Erol, Ansel Kaplan and Matlin, Glenn},
  year    = {2026},
  journal = {arXiv preprint arXiv:2602.02386},
  note    = {MLSys 2025 Young Professionals Symposium}
}