Skip to main content

Reference for the teleport_inference_model Terraform resource

Report an Issue

This page describes the supported values of the teleport_inference_model resource of the Teleport Terraform provider.

Schema​

Required​

Optional​

  • sub_kind (String) SubKind is the resource sub-kind. Should be empty.
  • version (String) Version is the resource version. Should be set to "v1".

Nested Schema for metadata​

Required:

  • name (String) name is an object name.

Optional:

  • description (String) description is object description.
  • expires (String) expires is a global expiry time header can be set on any resource in the system.
  • labels (Map of String) labels is a set of labels.

Nested Schema for spec​

Optional:

  • bedrock (Attributes) Bedrock indicates that this model uses Amazon Bedrock as the inference provider and specifies Bedrock-specific parameters. (see below for nested schema)
  • max_session_length_bytes (Number) MaxSessionLengthBytes is the maximum session length that can be sent to inference provider. Currently, it's determined by the size of model's context window; future versions of Teleport will allow summarizing larger sessions by splitting them. Inference providers will reject requests that are larger than given model's context window. Since context windows are usually sized in tokens, this value is an approximation. Assuming 2 bytes per input token should be safe. Currently, Teleport will outright reject sessions larger than this limit; future versions will split sessions in chunks, treating this size as a maximum. If unset or set to 0, defaults to 1MB.
  • openai (Attributes) Openai indicates that this model uses OpenAI as the inference provider and specifies OpenAI-specific parameters. (see below for nested schema)

Nested Schema for spec.bedrock​

Optional:

  • bedrock_model_id (String) BedrockModelId specifies a model ID or an inference profile as understood by the Bedrock API.
  • integration (String) Integration is the AWS OIDC Integration name. If unset, Teleport will use AWS credentials available on the Auth Service machine; otherwise, it will use the specified OIDC integration for assuming appropriate role.
  • region (String) Region is the AWS region which will be used for inference.

Nested Schema for spec.openai​

Optional:

  • api_key_secret_ref (String) ApiKeySecretRef is a reference to an InferenceSecret that contains the OpenAI API key.
  • base_url (String) BaseUrl is the OpenAI API base URL. Optional, defaults to the public OpenAI API URL. May be used to point to a custom OpenAI-compatible API, such as LiteLLM. In such case, the api_key_secret_ref must point to a secret that contains the API key for that custom API.
  • openai_model_id (String) OpenaiModelId specifies the model ID, as understood by the OpenAI API.