Skip to main content

Together AI

Connect Keeptrusts to the Together AI OpenAI-compatible API. Applications call the local Keeptrusts endpoint; the gateway owns the upstream credential and applies the configured policy chain.

Use this page when​

  • You already have access to Together AI and need to route it through Keeptrusts.
  • You want one explicit provider target that can be linted and reviewed before rollout.
  • You need a stable integration contract without copying mutable prices, context limits, or retirement dates into your config.

Prerequisites​

  • Install the kt CLI.
  • Obtain the upstream credential and an enabled model or endpoint from Together AI.
  • Obtain a Keeptrusts runtime API token for the gateway and a separate gateway key, access key, or personal API token for client requests. --agent uses only the runtime token.

Configure the provider​

Replace the replace-with-... values before starting the gateway. The example uses the current togetherai runtime contract.

pack:
  name: together-ai-integration
  version: 1.0.0
  enabled: true
policies:
  chain:
    - prompt-injection
    - pii-detector
    - audit-logger
providers:
  targets:
    - id: together-ai-primary
      provider: togetherai
      model: "replace-with-together-model-id"
      base_url: https://api.together.ai
      secret_key_ref:
        env: KEEPTRUSTS_TOGETHER_API_KEY

The provider credential is resolved inside the gateway process. It is not the credential that client applications send to Keeptrusts.

:::warning Keep the current base URL explicit

Use https://api.together.ai as shown. The runtime's shorthand-derived Together default still points at the legacy api.together.xyz host, so omitting base_url does not reproduce this verified configuration.

:::

Start and verify​

export KEEPTRUSTS_API_TOKEN="replace-with-keeptrusts-api-token"
export KEEPTRUSTS_TOGETHER_API_KEY="replace-with-upstream-credential"

kt policy lint --file policy-config.yaml
kt gateway run \
  --agent together-ai-integration \
  --listen 127.0.0.1:41002 \
  --policy-config policy-config.yaml

KEEPTRUSTS_API_TOKEN authenticates the gateway runtime and control-plane synchronization. Do not reuse it as the client credential.

In another terminal, export the separate client token, confirm the gateway is healthy, and send one request:

export KEEPTRUSTS_CLIENT_TOKEN="replace-with-separate-client-token"

curl -fsS http://127.0.0.1:41002/healthz
curl -fsS http://127.0.0.1:41002/v1/chat/completions \
  -H "Authorization: Bearer ${KEEPTRUSTS_CLIENT_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{"model":"replace-with-together-model-id","messages":[{"role":"user","content":"Reply with one short sentence."}]}'

Use the same model identifier in the request and target unless you have configured an explicit multi-model route.

Current Keeptrusts contract​

SettingBehavior
providertogetherai
Upstream requestOpenAI-compatible JSON at /v1/chat/completions
Upstream authenticationBearer token from KEEPTRUSTS_TOGETHER_API_KEY
Client endpoint/v1/chat/completions on the Keeptrusts gateway

Use the full model identifier exposed to your Together account. Do not pin prices or model status in this guide.

Model and production checks​

  • Verify the model ID, region, endpoint availability, and account permissions in the official provider surface before rollout.
  • Treat pricing, max_context_tokens, retention metadata, and certifications as operator declarations. Add them only after checking your current contract.
  • Validate streaming, tools, structured output, and other optional request features for the selected request family and model; provider-wide assumptions are unsafe.
  • Keep the upstream credential server-side. Bind production listeners only to the intended interface and protect them with your normal ingress controls.

Next steps​