Process Apache Common Logs for Elasticsearch
Optimizes Apache HTTP Server access logs (Common Log Format) for Elasticsearch ingestion via OTLP. Parses CLF fields, filters health check and static asset requests, maps HTTP status codes to OTLP severity levels, masks sensitive data, removes high-cardinality fields, samples successful responses, and deduplicates error bursts.
Included Processors
Parse Apache Common Log Format
Parses Apache Common Log Format extracting remote host, user, timestamp, method, path, status code, and response bytes.
Parse Apache Timestamp
Parses the Apache timestamp into the log record time field.
Filter Health Checks
Drops health check and monitoring endpoint requests.
Filter Static Asset Requests
Drops requests for static assets (CSS, JS, images, fonts).
Map HTTP Status to Severity
Maps HTTP status codes to OTLP severity levels for Elasticsearch severity-based filtering and alerting.
Mask Sensitive Data
Redacts sensitive data from request paths and user fields.
Remove High-Cardinality Fields
Removes ident and high-cardinality client fields.
Add ECS-Compatible Fields
Adds Elastic Common Schema fields for Apache access log data, enabling compatibility with built-in Kibana web server dashboards.
Sample Success Responses
Samples 50% of successful HTTP 2xx response logs to reduce volume.
Deduplicate Error Responses
Deduplicates repeated HTTP error logs within a 30-second window.
Batch Telemetry
Batches Apache logs for efficient Elasticsearch bulk indexing.
Delete Empty Fields
Removes null and empty values to reduce storage overhead.