Model routing
Choosing a suitable model for each request from a defined set of permitted candidates.
Which model should process the request?
Model routing chooses a model for a request from a defined set of permitted candidates. The decision can depend on task type, required capabilities and requirements for response time or operating cost. Fixed rules or a separate selection model can be used. The selected model then processes the request. Routing alone does not train that model. Which candidates may handle the data and task must be defined before selection.
In a fictional project, short subject-line suggestions and detailed report analysis are to be able to use different models. A planned comparison assesses routing against the previous use of one fixed model. Reviewers consider business task success, incorrect routing, response time and actual operating cost. A fallback route after a failure must also use only permitted candidates. The selected route and model version are recorded with the assessment conditions so that errors can be traced to a specific processing step.
Sources
Related terms
- AI inferenceApplying an already trained model to inputs to obtain a result.
- Function callingA language model requests a described function with structured arguments; the connected application executes it.
- LLM evaluationLLM evaluation tests model outputs against defined tasks and criteria. One good example does not establish overall quality.
- AI guardrailsGuardrails constrain and check AI application behaviour. Model instructions and technical enforcement perform different tasks.
- Model quantisationRepresenting model weights and, depending on the method, intermediate values with lower numerical precision.
- Knowledge distillationTraining a student model using behavioural information from a teacher model or an ensemble of models.
Your project with ASES
Describe your project to us. We will get back to you with an initial assessment.