Optimize Syslog for Elasticsearch
Optimizes syslog messages for Elasticsearch ingestion via OTLP. Rewrites RFC 3164 timestamps to RFC 5424 format, maps syslog severity and facility to OTLP severity and ECS fields, filters debug-level noise, deduplicates repeated messages, and normalizes field names for the Elastic Common Schema.
Included Processors
Parse Syslog Timestamp
Extracts the RFC 3164 timestamp from the syslog message body into a separate field for timestamp rewriting.
Parse Timestamp to Time Field
Converts the extracted syslog timestamp string into a proper timestamp format for the log record.
Rewrite Timestamp to RFC 5424
Rewrites the syslog timestamp in the log body to RFC 5424 format for consistent Elasticsearch date parsing.
Reassemble Syslog Body
Concatenates the pre-timestamp, rewritten timestamp, and post-timestamp parts back into the syslog body.
Map Syslog Severity to OTLP
Maps syslog severity values (0-7) to OTLP severity_text and severity_number for Elasticsearch severity-based querying. Requires syslog.severity attribute populated by syslog receiver.
Filter Debug Syslog Messages
Drops syslog messages with severity 7 (debug) to reduce log volume while retaining notice-level and above.
Add ECS-Compatible Fields
Adds Elastic Common Schema fields for syslog data, enabling compatibility with built-in Kibana syslog dashboards.
Deduplicate Repeated Messages
Deduplicates repeated syslog messages within a 60-second window.
Batch Telemetry
Batches syslog messages for efficient Elasticsearch bulk indexing.
Delete Empty Fields
Removes null and empty values to reduce storage overhead.