Skip to main content

Databricks Model Serving

Connect Keeptrusts to a Databricks model-serving endpoint. Applications call the local Keeptrusts endpoint; the gateway owns the upstream credential and applies the configured policy chain.

Use this page when​

  • You already have access to Databricks Model Serving and need to route it through Keeptrusts.
  • You want one explicit provider target that can be linted and reviewed before rollout.
  • You need a stable integration contract without copying mutable prices, context limits, or retirement dates into your config.

Prerequisites​

  • Install the kt CLI.
  • Obtain the upstream credential and an enabled model or endpoint from Databricks Model Serving.
  • Obtain a Keeptrusts runtime API token for the gateway and a separate gateway key, access key, or personal API token for client requests. --agent uses only the runtime token.

Configure the provider​

Replace the replace-with-... values before starting the gateway. The example uses the current databricks runtime contract.

pack:
  name: databricks-integration
  version: 1.0.0
  enabled: true
policies:
  chain:
    - prompt-injection
    - pii-detector
    - audit-logger
providers:
  targets:
    - id: databricks-primary
      provider: databricks
      model: "replace-with-serving-endpoint-name"
      base_url: https://replace-with-workspace-host
      secret_key_ref:
        env: KEEPTRUSTS_DATABRICKS_TOKEN

The provider credential is resolved inside the gateway process. It is not the credential that client applications send to Keeptrusts.

Start and verify​

export KEEPTRUSTS_API_TOKEN="replace-with-keeptrusts-api-token"
export KEEPTRUSTS_DATABRICKS_TOKEN="replace-with-upstream-credential"

kt policy lint --file policy-config.yaml
kt gateway run \
  --agent databricks-integration \
  --listen 127.0.0.1:41002 \
  --policy-config policy-config.yaml

KEEPTRUSTS_API_TOKEN authenticates the gateway runtime and control-plane synchronization. Do not reuse it as the client credential.

In another terminal, export the separate client token, confirm the gateway is healthy, and send one request:

export KEEPTRUSTS_CLIENT_TOKEN="replace-with-separate-client-token"

curl -fsS http://127.0.0.1:41002/healthz
curl -fsS http://127.0.0.1:41002/v1/chat/completions \
  -H "Authorization: Bearer ${KEEPTRUSTS_CLIENT_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{"model":"replace-with-serving-endpoint-name","messages":[{"role":"user","content":"Reply with one short sentence."}]}'

Use the same model identifier in the request and target unless you have configured an explicit multi-model route.

Current Keeptrusts contract​

SettingBehavior
providerdatabricks
Upstream requestDatabricks serving invocation at /serving-endpoints/{model}/invocations
Upstream authenticationBearer token from KEEPTRUSTS_DATABRICKS_TOKEN
Client endpoint/v1/chat/completions on the Keeptrusts gateway

The model value is the Databricks serving endpoint name, not a marketplace model label. The base URL must be the HTTPS workspace host. The current adapter sends the OpenAI chat-completions body to the serving endpoint without converting it to a DataFrame or tensor payload, so the endpoint must accept that request shape.

Model and production checks​

  • Verify the model ID, region, endpoint availability, and account permissions in the official provider surface before rollout.
  • Treat pricing, max_context_tokens, retention metadata, and certifications as operator declarations. Add them only after checking your current contract.
  • Validate streaming, tools, structured output, and other optional request features for the selected request family and model; provider-wide assumptions are unsafe.
  • Keep the upstream credential server-side. Bind production listeners only to the intended interface and protect them with your normal ingress controls.

Next steps​