Skip to main content

Multi-Provider Arbitrage: Real-Time Price Comparison Routing

Multi-provider arbitrage sounds sophisticated, but the practical idea is simple: stop paying the same price for every request when multiple governed providers can satisfy the workload. Keeptrusts gives you the routing, evidence, and budget controls to do that without turning your stack into a pile of vendor-specific logic.

Use this page when​

  • You want to reduce unit cost by routing requests across multiple providers.
  • You need a governed way to compare price, quality, and resilience at request time.
  • You are planning a provider migration and want to reduce cost before you fully cut over.

Primary audience​

  • Primary: Technical Engineers and Platform Operators
  • Secondary: Technical Leaders and FinOps teams

The problem​

Most organizations negotiate better AI pricing than they actually realize. They may have access to several providers, but application traffic is still pinned to a single upstream target. That means every request pays the same price even when a cheaper or more available option could have handled it.

The obvious response is to build custom routing in the application, but that creates a maintenance burden. Every client has to understand provider-specific differences, fallback rules, and migration logic. Worse, cost optimization becomes invisible. You know you changed the code, but you cannot easily prove whether the blended rate improved or whether the fallback path became more expensive.

Another trap is pretending arbitrage means live market scraping. Keeptrusts does not need that model to be useful. What you actually need is operator-declared pricing, request-time routing logic, and event evidence that shows which target handled the work.

The solution​

Keeptrusts makes arbitrage practical by separating the decision from the application.

You define multiple provider targets, attach pricing metadata where appropriate, and choose a routing strategy that matches the operational goal. If the goal is straight cost discipline, filters such as max_price can exclude expensive targets from consideration. If the goal is cost plus user experience, strategies such as lowest_latency or usage_based let you optimize the blended result rather than simply chasing the lowest nominal price.

Provider budgets then add a second layer of protection. They tell you when a vendor is consuming more budget than expected even if the total monthly spend still looks acceptable. Weighted routing and A/B testing help when you need to compare real production behavior before moving a larger percentage of traffic.

Implementation​

Start with a routing policy that reflects the trade-off you are actually making. If you care only about minimizing the unit price, strict price filters may be enough. If you want a balance of affordability and user experience, use routing that can consider recent performance while respecting cost limits.

This documented pattern is a strong starting point:

providers:
  routing:
    strategy: lowest_latency
    max_price: 2.50
    window_seconds: 180
    min_sample_count: 8
  targets:
    - id: groq-llama
      provider: groq:chat:llama-3.3-70b-versatile
      secret_key_ref:
        env: GROQ_API_KEY
      pricing:
        input_price_per_million: 0.59
        output_price_per_million: 0.79
    - id: openai-mini
      provider: openai:chat:gpt-5.4-mini-mini
      secret_key_ref:
        env: OPENAI_API_KEY
      pricing:
        input_price_per_million: 0.15
        output_price_per_million: 0.60

That example shows an important principle: arbitrage is not just “pick the cheapest.” It is “pick from the set of providers that satisfy the economic and operational rules you declared.”

Next, prove the economics with measured traffic. Use weighted routing or model-routing A/B tests when you want a lower-risk comparison. That lets you compare cost, latency, and outcome quality using governed event evidence instead of intuition. If the cheaper provider performs well enough, you can shift more traffic. If it does not, you have evidence for why the premium path remains justified.

Provider budgets close the loop. Even a well-designed routing strategy can drift if one provider becomes dominant because of latency, availability, or a misconfigured order. A provider budget turns that drift into a visible signal.

The practical rollout sequence is usually straightforward.

  1. Define at least two viable provider targets.
  2. Add pricing metadata and choose a routing strategy.
  3. Start with a measured subset of traffic.
  4. Compare event evidence and spend outcomes.
  5. Set provider budgets so successful arbitrage does not become hidden concentration risk.

Results and impact​

The main benefit of multi-provider arbitrage is a lower blended cost per request, but that is not the only gain. Teams also improve negotiating leverage because they are no longer operationally trapped on one vendor path. Migrations become less risky because you can validate them with weighted or staged routing instead of a hard switch.

Arbitrage also changes how cost conversations happen. Instead of asking whether one provider is “better,” you can ask whether it is better enough for the workloads that matter. That is a more useful question because price, latency, and business impact rarely move together.

Key takeaways​

  • Keeptrusts arbitrage is based on configured pricing and request-time routing, not magic market scraping.
  • max_price, provider pricing, and routing strategy determine which targets are eligible.
  • Weighted routing and A/B tests are the safest way to validate a cheaper path before broad rollout.
  • Provider budgets prevent a successful routing policy from becoming hidden vendor concentration.
  • The goal is a lower blended rate with acceptable quality and resilience, not the cheapest possible request at any cost.

Next steps​