TeleportInferenceModel
Report an Issue
Is this page helpful?
This guide is a comprehensive reference to the fields in the TeleportInferenceModel
resource, which you can apply after installing the Teleport Kubernetes operator.
resources.teleport.dev/v1
apiVersion: resources.teleport.dev/v1
| Field | Type | Description |
|---|---|---|
| apiVersion | string | APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources |
| kind | string | Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds |
| metadata | object | |
| spec | object | InferenceModel resource definition v1 from Teleport |
spec
| Field | Type | Description |
|---|---|---|
| bedrock | object | Bedrock indicates that this model uses Amazon Bedrock as the inference provider and specifies Bedrock-specific parameters. |
| max_session_length_bytes | integer | MaxSessionLengthBytes is the maximum session length that can be sent to inference provider. Currently, it's determined by the size of model's context window; future versions of Teleport will allow summarizing larger sessions by splitting them. Inference providers will reject requests that are larger than given model's context window. Since context windows are usually sized in tokens, this value is an approximation. Assuming 2 bytes per input token should be safe. Currently, Teleport will outright reject sessions larger than this limit; future versions will split sessions in chunks, treating this size as a maximum. If unset or set to 0, defaults to 1MB. |
| openai | object | Openai indicates that this model uses OpenAI as the inference provider and specifies OpenAI-specific parameters. |
spec.bedrock
| Field | Type | Description |
|---|---|---|
| bedrock_model_id | string | BedrockModelId specifies a model ID or an inference profile as understood by the Bedrock API. |
| integration | string | Integration is the AWS OIDC Integration name. If unset, Teleport will use AWS credentials available on the Auth Service machine; otherwise, it will use the specified OIDC integration for assuming appropriate role. |
| region | string | Region is the AWS region which will be used for inference. |
| temperature | number | Temperature controls the randomness of the model's output. |
spec.openai
| Field | Type | Description |
|---|---|---|
| api_key_secret_ref | string | ApiKeySecretRef is a reference to an InferenceSecret that contains the OpenAI API key. |
| base_url | string | BaseUrl is the OpenAI API base URL. Optional, defaults to the public OpenAI API URL. May be used to point to a custom OpenAI-compatible API, such as LiteLLM. In such case, the api_key_secret_ref must point to a secret that contains the API key for that custom API. |
| openai_model_id | string | OpenaiModelId specifies the model ID, as understood by the OpenAI API. |
| temperature | number | Temperature controls the randomness of the model's output. |
Was this page helpful?