gcp_vertex_ai_chat

Available in: Cloud, Self-Managed

Generates responses to messages in a chat conversation, using the Vertex API AI.

  • Common

  • Advanced

# Common configuration fields, showing default values
label: ""
gcp_vertex_ai_chat:
  project: "" # No default (required)
  credentials_json: "" # No default (optional)
  location: us-central1 # No default (optional)
  model: gemini-1.5-pro-001 # No default (required)
  prompt: "" # No default (optional)
  temperature: 0 # No default (optional)
  max_tokens: 0 # No default (optional)
  response_format: text
# All configuration fields, showing default values
label: ""
gcp_vertex_ai_chat:
  project: "" # No default (required)
  credentials_json: "" # No default (optional)
  location: us-central1 # No default (optional)
  model: gemini-1.5-pro-001 # No default (required)
  prompt: "" # No default (optional)
  system_prompt: "" # No default (optional)
  temperature: 0 # No default (optional)
  max_tokens: 0 # No default (optional)
  response_format: text
  top_p: 0 # No default (optional)
  top_k: 0 # No default (optional)
  stop: [] # No default (optional)
  presence_penalty: 0 # No default (optional)
  frequency_penalty: 0 # No default (optional)

This processor sends prompts to your chosen large language model (LLM) and generates text from the responses, using the Vertex AI API.

For more information, see the Vertex AI documentation.

Fields

attachment

Additional data like an image to send with the prompt to the model. The result of the mapping must be a byte array, and the content type is automatically detected.

Type: string

# Examples:
attachment: root = this.image.decode("base64") # decode base64 encoded image

credentials_json

An optional field to set a Google Service Account Credentials JSON.

This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Manage Secrets before adding it to your configuration.

Type: string

frequency_penalty

Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model’s likelihood to repeat the same line verbatim.

Type: float

history

Historical messages to include in the chat request. The result of the bloblang query should be an array of objects of the form of [{"role": "", "content":""}], where role is "user" or "model".

Type: string

location

Specify the location of a fine tuned model. For base models, you can omit this field.

Type: string

# Examples:
location: us-central1

max_tokens

The maximum number of output tokens to generate per message.

Type: int

max_tool_calls

The maximum number of sequential tool calls.

Type: int

Default: 10

model

The name of the LLM to use. For a full list of models, see the Vertex AI Model Garden.

Type: string

# Examples:
model: gemini-1.5-pro-001
model: gemini-1.5-flash-001

presence_penalty

Positive values penalize new tokens if they appear in the text already, increasing the model’s likelihood to include new topics.

Type: float

project

The GCP project ID to use.

Type: string

prompt

The prompt you want to generate a response for. By default, the processor submits the entire payload as a string. This field supports interpolation functions.

Type: string

response_format

The format of the generated response. You must also prompt the model to output the appropriate response type.

Type: string

Default: text

Options: text, json

stop[]

Sets the stop sequences to use. When this pattern is encountered the LLM stops generating text and returns the final response.

Type: array

system_prompt

The system prompt to submit to the Vertex AI LLM. This field supports interpolation functions.

Type: string

temperature

Controls the randomness of predictions.

Type: float

tools[]

The tools to allow the LLM to invoke. This allows building subpipelines that the LLM can choose to invoke to execute agentic-like actions.

Type: object

Default: []

tools[].description

A description of this tool, the LLM uses this to decide if the tool should be used.

Type: string

tools[].name

The name of this tool.

Type: string

tools[].parameters

The parameters the LLM needs to provide to invoke this tool.

Type: object

tools[].parameters.properties

The properties for the processor’s input data

Type: object

tools[].parameters.properties.description

A description of this parameter.

Type: string

tools[].parameters.properties.enum[]

Specifies that this parameter is an enum and only these specific values should be used.

Type: array

Default: []

tools[].parameters.properties.type

The type of this parameter.

Type: string

tools[].parameters.required[]

The required parameters for this pipeline.

Type: array

Default: []

tools[].processors[]

The pipeline to execute when the LLM uses this tool.

Type: processor

top_k

Enables top-k sampling (optional).

Type: float

top_p

Enables nucleus sampling (optional).

Type: float