🚀 Announcing BYOC and the OpenTelemetry Distribution BuilderRead more

AWS S3 Rehydration

Object Format

Rehydrated data must be in OTLP format JSON for correct processing. Ensure your data adheres to this format.

The AWS S3 Destination supports gzip compression. This receiver will gracefully detect and handle if any data is gzipped or uncompressed.

Supported Platforms

PlatformSupported
Linux✓
Windows✓
macOS✓
Kubernetes Cluster*✓

Available in the Bindplane Distro for OpenTelemetry Collector v1.69.0+.

* Note: When deploying with a Kubernetes Cluster agent, select StatefulSet as the deployment method.

Configuration Fields

FieldDescription
RegionThe AWS recognized region string
BucketName of the S3 Bucket to rehydration telemetry from.
Folder PrefixRoot directory of the bucket to rehydration telemetry from.
Poll SizeThe max number of object descriptions to be returned by a single poll against the S3 API.
Batch SizeThe max number of object descriptions to process & rehydrate at once after retrieving from the S3 API.
Starting TimeUTC start time for rehydration in the format YYYY-MM-DDTHH:MM.
Ending TimeUTC end time for rehydration in the format YYYY-MM-DDTHH:MM.
Delete on ReadIf true, objects are deleted after rehydration.
Enable StorageEnable to specify a storage extension for tracking rehydration progress.
Storage DirectoryDirectory for storing rehydration state, useful for maintaining state and resuming operations. (Only relevant if Enable Storage is true)

Usage

Kubernetes

When deploying on Kubernetes, you must configuration authentication to AWS. This can be done by setting the following environment.

Edit the YAML manifest downloaded from Bindplane's agent install page. Add the following environment variables to the opentelemetry-container container.

yaml
1spec:
2  template:
3    spec:
4      containers:
5        - name: opentelemetry-container
6          env:
7            - name: AWS_ACCESS_KEY_ID
8              value: '<your access key>'
9            - name: AWS_SECRET_ACCESS_KEY
10              value: '<your secret key>'

Example Configuration

Basic Configuration

This configuration sets up AWS S3 Rehydration with necessary details such as Region, Bucket, and time range for rehydration.

observIQ docs - AWS S3 Rehydration

Adjust the Region, Bucket, Starting Time, and Ending Time to match your configuration.