Process HTTP Logs for ClickHouse
Optimizes HTTP access and application logs for ClickHouse analytics. Normalizes HTTP fields, masks sensitive headers and query parameters, reduces high-cardinality request identifiers, and filters health checks.
Included Processors
Parse JSON Body
Parse JSON HTTP log body into attributes for field processing.
Coalesce HTTP Method Fields
Coalesces common HTTP method field variants into a single standard name.
Filter Health Checks
Drops verbose health check and monitoring endpoint requests to reduce noise.
Map HTTP Status to Severity
Maps HTTP status codes to OTEL-standard severity_text and severity_number for ClickHouse SeverityNumber column queries.
Mask Sensitive Fields
Redacts sensitive data from HTTP headers, queries, and bodies.
Sample Success Logs
Samples 50% of successful HTTP 2xx response logs to reduce volume.
Dedupe Error Logs
Deduplicates repeated HTTP error logs within a 30-second window.
Batch Logs
Batches HTTP logs for efficient ClickHouse ingestion.
Delete Empty Fields
Removes null and empty values to reduce storage overhead.