File
warning
This source has been deprecated and replaced with a new Filelog source. While it will continue to function, it will no longer receive any enhancements and you should migrate to the new source. For more information about the new source, see this documentation.
warning
This source offers a delete_after_read option that can be hazardous. When this option is combined with file globbing, it will delete every file that matches the globbing pattern. Use with caution and care.
Supported Platforms
| Platform | Metrics | Logs | Traces |
|---|---|---|---|
| Linux | ✓ | ||
| Windows | ✓ | ||
| macOS | ✓ |
Configuration Table
| Parameter | Type | Default | Description |
|---|---|---|---|
| file_path* | strings | File or directory paths to tail for logs. | |
| exclude_file_path | strings | "" | File or directory paths to exclude. |
| log_type | string | "file" | A friendly name that will be added to each log entry as an attribute. |
| parse_format | enum | none | Method to use when parsing. Valid values are none, json, and regex. When regex is selected, 'Regex Pattern' must be set. |
| regex_pattern | string | The regex pattern that is used when parsing log entries. | |
| multiline_line_start_pattern | string | Regex pattern that matches the beginning of a log entry for handling multiline logs. | |
| multiline_line_end_pattern | string | Regex pattern that matches the end of a log entry, useful for terminating parsing of multiline logs. | |
| parse_timestamp | bool | false | Whether to parse the timestamp from the log entry. |
| timestamp_field | string | timestamp | The field containing the timestamp in the log entry. |
| parse_timestamp_format | enum | ISO8601 | The format of the timestamp in the log entry. Choose a common format, or specify a custom format. Options include "ISO8601", "RFC3339", "Epoch", and "Manual". |
| epoch_timestamp_format | enum | s | The layout of the epoch-based timestamp. It's required when parse_timestamp_format is set to "Epoch". Options include "s", "ms", "us", "ns", "s.ms", "s.us", "s.ns". |
| manual_timestamp_format | string | '%Y-%m-%dT%H:%M:%S.%f%z' | The strptime layout of the timestamp. It's used when parse_timestamp_format is set to "Manual". |
| timezone | timezone | UTC | The timezone to use if the Timestamp Format doesn't include a timezone. Otherwise, the timezone in the Timestamp Format will be respected. NOTE: This is also required to parse timezone abbreviations due to their ambiguity. |
| parse_severity | bool | false | Whether to parse severity from the log entry. |
| severity_field | string | severity | The field containing the severity in the log entry. |
| encoding | enum | utf-8 | The encoding of the file being read. Valid values are nop, utf-8, utf-16le, utf-16be, ascii, and big5. |
| include_file_name_attribute | bool | true | Whether to add the file name as the attribute log.file.name. |
| include_file_path_attribute | bool | false | Whether to add the file path as the attribute log.file.path. |
| include_file_name_resolved | bool | false | Whether to add the file name after symlinks resolution as the attribute log.file.name_resolved. |
| include_file_path_resolved | bool | false | Whether to add the file path after symlinks resolution as the attribute log.file.path_resolved. |
| delete_after_read | bool | false | Whether to delete the file(s) after reading. Only valid in combination start_at: beginning. |
| offset_storage_dir | string | $OIQ_OTEL_COLLECTOR_HOME/storage | The directory where the offset storage file will be created. It is okay if multiple receivers use the same directory. By default, the Bindplane Distro for OpenTelemetry Collector sets $OIQ_OTEL_COLLECTOR_HOME in its runtime. |
| poll_interval | int | 200 | The duration of time in milliseconds between filesystem polls. |
| max_concurrent_files | int | 1024 | The maximum number of log files from which logs will be read concurrently. If the number of files matched exceeds this number, then files will be processed in batches. |
| parse_to | string | body | The field that the log will be parsed to. Some exporters handle logs favorably when parsed to attributes over body and vice versa. |
| start_at | enum | end | Start reading the file from the 'beginning' or 'end'. |