Intelligent Model Routing Guide FinOps & Latency

Intelligent Model Routing allows organizations to dynamically optimize LLM model selection across connected AI agent harnesses. By classifying tasks and routing prompts to the most cost-effective and capable models using adaptive reinforcement learning, Intutic helps you achieve peak performance while minimizing token expenses.

---

How It Works

1. Proxy Interception: Every outbound prompt from your developer tools is routed through the proxy gateway.
2. Gateway Classification: The proxy performs high-speed (sub-5ms) keyword matching to classify the prompt into one of five task types: testing, deployment, review, debugging, or coding.
3. Thompson Sampling Selection: Intutic evaluates historical performance (rewards) for the (Model × SOP Tier × Task Type) combination to select the optimal model.
4. Daily Refinement (LLM-as-a-Judge): A daily background cron job audits a sample of traces, refines classifications, and automatically registers new trigger keywords to continuously improve optimization.

---

Setup & Activation

Step 1: Enable Routing in the Dashboard

1. Open the Compute Metrics Dashboard (e.g., app.intutic.ai or your local console at http://localhost:5174).
2. Navigate to Settings from the sidebar navigation.
3. Click on the Smart Model Routing & Response Cache tab.
4. Check the Enable Intelligent Model Routing option.

---

Step 2: Configure Custom Task Trigger Words

You can customize the words that trigger model redirection to fit your team's tech stack and vocabularies:

1. In the Intelligent Model Routing settings section, locate the keyword configuration fields.
2. Input comma-separated lists of trigger keywords for the following categories:
   • Testing: e.g., test, spec, vitest, jest, unittest, assert
   • Deployment: e.g., deploy, release, kubernetes, docker, gke, pipeline, ci/cd
   • Review: e.g., review, audit, lint, eslint, pr
   • Debugging: e.g., fix, bug, issue, error, crash, debug
3. Click Save Keywords to push the updates to Valkey.

NOTE

Custom keywords are validated at the API layer. Keywords must be alphanumeric strings (or ci/cd) and between 2 and 19 characters long.

---

Step 3: Route Agent Traffic

To route agent traffic, you must ensure your AI agent harnesses are connected to the Intutic proxy gateway:

Option A: Using the CLI (Recommended)

The Intutic CLI sync daemon scans and automatically updates the configurations of all supported harnesses in your local repository:

npm install -g @intutic/cli
intutic login
intutic init
intutic connect

Option B: Standalone Proxy Redirects

For custom agent configurations, point your agent's API base URL environment variables directly to the proxy gateway:

export OPENAI_API_BASE="https://proxy.intutic.ai/v1"
export ANTHROPIC_API_BASE="https://proxy.intutic.ai/v1"

Once connected, your prompts are automatically routed to the most optimal model based on local rules and current learning rates.