Elasticsearch

NAME	RUNTIME	TYPE	VERSION
elasticsearch	wombat	sink	latest

Publishes messages into an Elasticsearch index. If the index does not exist then it is created with a dynamic mapping.

It's possible to enable AWS connectivity with this output using the aws fields. However, you may need to set sniff and healthcheck to false for connections to succeed.

Configuration

Name	Type	Optional	Secret	Description
urls	[]string	no	no	A list of URLs to connect to. If an item of the list contains commas it will be expanded into multiple URLs.
index	string	no	no	The index to place messages.
action	string	yes	no	The action to take on the document. This field must resolve to one of the following action types: `create`, `index`, `update`, `upsert` or `delete`. The value defaults to `"index"`.
pipeline	string	yes	no	An optional pipeline id to preprocess incoming documents. The value defaults to `""`.
id	string	yes	no	The ID for indexed messages. Interpolation should be used in order to create a unique ID for each message. The value defaults to `"${!counter()}-${!timestamp_unix()}"`.
type	string	yes	no	The document mapping type. This field is required for versions of elasticsearch earlier than 6.0.0, but are invalid for versions 7.0.0 or later. The value defaults to `""`.
routing	string	yes	no	The routing key to use for the document. The value defaults to `""`.
sniff	bool	yes	no	Prompts the connector to sniff for brokers to connect to when establishing a connection. The value defaults to `true`.
healthcheck	bool	yes	no	Whether to enable healthchecks. The value defaults to `true`.
timeout	string	yes	no	The maximum time to wait before abandoning a request (and trying again). The value defaults to `"5s"`.
api_key	string	no	yes	The key to set in the Authorization header if using API keys for authentication.
tls	tls	no	no	Custom TLS settings can be used to override system defaults.
max_in_flight	int	yes	no	The maximum number of messages to have in flight at a given time. Increase this to improve throughput. The value defaults to `64`.
max_retries	int	yes	no	The maximum number of retries before giving up on the request. If set to zero there is no discrete limit. The value defaults to `0`.
backoff	backoff	no	no	Control time intervals between retry attempts.
basic_auth	basic_auth	no	no	Allows you to specify basic authentication.
batching	batching	no	no	Allows you to configure a batching policy.
aws	aws	no	no	Enables and customises connectivity to Amazon Elastic Service.
gzip_compression	bool	yes	no	Enable gzip compression on the request side. The value defaults to `false`.

TLS Configuration

Name	Type	Optional	Secret	Description
enabled	bool	yes	no	Whether custom TLS settings are enabled. The value defaults to `false`.
skip_cert_verify	bool	yes	no	Whether to skip server side certificate verification. The value defaults to `false`.
enable_renegotiation	bool	yes	no	Whether to allow the remote server to repeatedly request renegotiation.
				Enable this option if you're seeing the error message `local error: tls: no renegotiation`. The value defaults to `false`.
root_cas	string	yes	yes	An optional root certificate authority to use.
				This is a string, representing a certificate chain from the parent trusted root certificate,
				to possible intermediate signing certificates, to the host certificate. The value defaults to `""`.
client_certs	[]client_certs	yes	no	A list of client certificates to use. . The value defaults to `[]`.

Backoff

Name	Type	Optional	Secret	Description
initial_interval	string	yes	no	The initial period to wait between retry attempts. The value defaults to `"1s"`.
max_interval	string	yes	no	The maximum period to wait between retry attempts. The value defaults to `"5s"`.
max_elapsed_time	string	yes	no	The maximum period to wait before retry attempts are abandoned. If zero then no limit is used. The value defaults to `"30s"`.

Basic Auth

Name	Type	Optional	Secret	Description
enabled	bool	yes	no	Whether to use basic authentication in requests. The value defaults to `false`.
username	string	yes	no	A username to authenticate as. The value defaults to `""`.
password	string	yes	yes	A password to authenticate with. The value defaults to `""`.

Batching

Name	Type	Optional	Secret	Description
count	int	yes	no	A number of messages at which the batch should be flushed. If `0` disables count based batching. The value defaults to `0`.
byte_size	int	yes	no	An amount of bytes at which the batch should be flushed. If `0` disables size based batching. The value defaults to `0`.
period	string	yes	no	A period in which an incomplete batch should be flushed regardless of its size. The value defaults to `""`.
check	string	yes	no	A Bloblang query that should return a boolean value indicating whether a message should end a batch. The value defaults to `""`.

AWS

Name	Type	Optional	Secret	Description
enabled	bool	yes	no	Whether to connect to Amazon Elastic Service. The value defaults to `false`.
region	string	yes	no	The AWS region to target. The value defaults to `""`.
endpoint	string	yes	no	Allows you to specify a custom endpoint for the AWS API. The value defaults to `""`.
credentials	credentials	no	no	Optional manual configuration of AWS credentials to use. More information can be found in the AWS Guide.