couchbase

Type:

Available in: Self-Managed

Perform operations on a Couchbase database for each incoming message. This connector can store or delete data in Couchbase based on the message’s content.

Introduced in version 4.37.0.

  • Common

  • Advanced

# Common configuration fields, showing default values
output:
  label: ""
  couchbase:
    url: couchbase://localhost:11210 # No default (required)
    username: "" # No default (optional)
    password: "" # No default (optional)
    bucket: "" # No default (required)
    id: ${! json("id") } # No default (required)
    content: "" # No default (optional)
    operation: upsert
    max_in_flight: 64
    batching:
      count: 0
      byte_size: 0
      period: ""
      check: ""
# All configuration fields, showing default values
output:
  label: ""
  couchbase:
    url: couchbase://localhost:11210 # No default (required)
    username: "" # No default (optional)
    password: "" # No default (optional)
    bucket: "" # No default (required)
    collection: _default
    transcoder: legacy
    timeout: 15s
    id: ${! json("id") } # No default (required)
    content: "" # No default (optional)
    operation: upsert
    max_in_flight: 64
    batching:
      count: 0
      byte_size: 0
      period: ""
      check: ""
      processors: [] # No default (optional)

Performance

For improved performance, this output sends multiple messages in parallel. You can tune the maximum number of in-flight messages (or message batches), using the max_in_flight field.

You can configure batches at both the input and output level. For more information, see Message Batching.

Fields

batching

Configure a batching policy.

Type: object

# Examples:
batching:
  byte_size: 5000
  count: 0
  period: 1s
batching:
  count: 10
  period: 1s
batching:
  check: this.contains("END BATCH")
  count: 0
  period: 1m

batching.byte_size

The amount of bytes at which the batch is flushed. Set to 0 to disable size-based batching.

Type: int

Default: 0

batching.check

A Bloblang query that returns a boolean value indicating whether a message should end a batch.

Type: string

Default: ""

# Examples:
check: this.type == "end_of_transaction"

batching.count

The number of messages after which the batch is flushed. Set to 0 to disable count-based batching.

Type: int

Default: 0

batching.period

The period of time after which an incomplete batch is flushed regardless of its size.

Type: string

Default: ""

# Examples:
period: 1s
period: 1m
period: 500ms

batching.processors[]

For aggregating and archiving message batches, you can add a list of processors to apply to a batch as it is flushed. All resulting messages are flushed as a single batch even when you configure processors to split the batch into smaller batches.

Type: processor

# Examples:
processors:
  - archive:
      format: concatenate

  - archive:
      format: lines

  - archive:
      format: json_array

bucket

The name of the Couchbase bucket you want the output to access.

Type: string

collection

The name of the specific collection you want the output to write to.

Type: string

content

The document content to update. When inserting, replacing, or upserting documents, you must set a content value.

Type: string

id

The document ID to use.

This field supports interpolation functions.

Type: string

# Examples:
id: ${! json("id") }

max_in_flight

The maximum number of messages to have in flight at a given time. Increase this value to improve throughput.

Type: int

Default: 64

operation

The Couchbase operation to perform.

Type: string

Default: upsert

Option Summary

insert

insert a new document.

remove

delete a document.

replace

replace the contents of a document.

upsert

creates a new document if it does not exist, if it does exist then it updates it.

password

A password to authenticate with the Couchbase cluster.

This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Secrets.

Type: string

scope

Bucket scope.

Type: string

timeout

Operation timeout if this output cannot connect to the Couchbase cluster.

Type: string

Default: 15s

transcoder

Choose from the following Couchbase transcoders to convert the messages' data format before they are written to Couchbase.

Type: string

Default: legacy

Option Summary

json

JSONTranscoder implements the default transcoding behavior and applies JSON transcoding to all values. This will apply the following behavior to the value: binary ([]byte) → error. default → JSON value, JSON Flags.

legacy

LegacyTranscoder implements the behavior for a backward-compatible transcoder. This transcoder implements behavior matching that of gocb v1.This will apply the following behavior to the value: binary ([]byte) → binary bytes, Binary expectedFlags. string → string bytes, String expectedFlags. default → JSON value, JSON expectedFlags.

raw

RawBinaryTranscoder implements passthrough behavior of raw binary data. This transcoder does not apply any serialization. This will apply the following behavior to the value: binary ([]byte) → binary bytes, binary expectedFlags. default → error.

rawjson

RawJSONTranscoder implements passthrough behavior of JSON data. This transcoder does not apply any serialization. It will forward data across the network without incurring unnecessary parsing costs. This will apply the following behavior to the value: binary ([]byte) → JSON bytes, JSON expectedFlags. string → JSON bytes, JSON expectedFlags. default → error.

rawstring

RawStringTranscoder implements passthrough behavior of raw string data. This transcoder does not apply any serialization. This will apply the following behavior to the value: string → string bytes, string expectedFlags. default → error.

url

The Couchbase connection string.

Type: string

# Examples:
url: couchbase://localhost:11210

username

A username to authenticate with the Couchbase cluster.

Type: string