Synadia Connect

AWS Kinesis

NAMERUNTIMETYPEVERSIONSTATUS
aws_kinesiswombatsourcelateststable

Receive messages from one or more Kinesis streams.

Configuration

nametypeoptionalsecretdescription
streams[]stringnonoOne or more Kinesis data streams to consume from. Streams can either be specified by their name or full ARN. Shards of a stream are automatically balanced across consumers by coordinating through the provided DynamoDB table. Multiple comma separated streams can be listed in a single element. Shards are automatically distributed across consumers of a stream by coordinating through the provided DynamoDB table. Alternatively, it's possible to specify an explicit shard to consume from with a colon after the stream name, e.g. foo:0 would consume the shard 0 of the stream foo.
dynamodbdynamodbyesnoDetermines the table used for storing and accessing the latest consumed sequence for shards, and for coordinating balanced consumers of streams.
checkpoint_limitintyesnoThe maximum gap between the in flight sequence versus the latest acknowledged sequence at a given time. Increasing this limit enables parallel processing and batching at the output level to work on individual shards. Any given sequence will not be committed unless all messages under that offset are delivered in order to preserve at least once delivery guarantees. The value defaults to 1024.
auto_replay_nacksboolyesnoWhether messages that are rejected (nacked) at the output level should be automatically replayed indefinitely, eventually resulting in back pressure if the cause of the rejections is persistent. If set to false these messages will instead be deleted. Disabling auto replays can greatly improve memory efficiency of high throughput streams as the original shape of the data can be discarded immediately upon consumption and mutation. The value defaults to true.
commit_periodstringyesnoThe period of time between each update to the checkpoint table. The value defaults to 5s.
rebalance_periodstringyesnoThe period of time between each attempt to rebalance shards across clients. The value defaults to 30s.
lease_periodstringyesnoThe period of time after which a client that has failed to update a shard checkpoint is assumed to be inactive. The value defaults to 30s.
start_from_oldestboolyesnoWhether to consume from the oldest message when a sequence does not yet exist for the stream. The value defaults to true.
regionstringnonoThe AWS region to target.
endpointstringyesnoAllows you to specify a custom endpoint for the AWS API. The value defaults to ``.
credentialscredentialsnonoOptional manual configuration of AWS credentials to use. More information can be found in xref:guides:cloud/aws.adoc[].

DynamoDB

nametypeoptionalsecretdescription
tablestringnonoThe name of the table to access.
createboolyesnoWhether, if the table does not exist, it should be created. The value defaults to false.
billing_modestringyesnoWhen creating the table determines the billing mode. The value defaults to PAY_PER_REQUEST.
read_capacity_unitsintyesnoSet the provisioned read capacity when creating the table with a billing_mode of PROVISIONED. The value defaults to 0.
write_capacity_unitsintnonoSet the provisioned write capacity when creating the table with a billing_mode of PROVISIONED. The value defaults to 0.

Credentials

nametypeoptionalsecretdescription
idstringyesnoThe ID of credentials to use. The value defaults to ``.
secretstringyesnoThe secret for the credentials being used. The value defaults to ``.
tokenstringyesnoThe token for the credentials being used, required when using short term credentials. The value defaults to ``.
Previous
Sources