diff options
Diffstat (limited to '')
| -rw-r--r-- | src/go/plugin/go.d/modules/weblog/integrations/web_server_log_files.md (renamed from src/go/collectors/go.d.plugin/modules/weblog/integrations/web_server_log_files.md) | 108 |
1 files changed, 69 insertions, 39 deletions
diff --git a/src/go/collectors/go.d.plugin/modules/weblog/integrations/web_server_log_files.md b/src/go/plugin/go.d/modules/weblog/integrations/web_server_log_files.md index a433c6dd2..740af5f1d 100644 --- a/src/go/collectors/go.d.plugin/modules/weblog/integrations/web_server_log_files.md +++ b/src/go/plugin/go.d/modules/weblog/integrations/web_server_log_files.md @@ -1,6 +1,6 @@ <!--startmeta -custom_edit_url: "https://github.com/netdata/netdata/edit/master/src/go/collectors/go.d.plugin/modules/weblog/README.md" -meta_yaml: "https://github.com/netdata/netdata/edit/master/src/go/collectors/go.d.plugin/modules/weblog/metadata.yaml" +custom_edit_url: "https://github.com/netdata/netdata/edit/master/src/go/plugin/go.d/modules/weblog/README.md" +meta_yaml: "https://github.com/netdata/netdata/edit/master/src/go/plugin/go.d/modules/weblog/metadata.yaml" sidebar_label: "Web server log files" learn_status: "Published" learn_rel_path: "Collecting Metrics/Web Servers and Web Proxies" @@ -216,20 +216,19 @@ Notes: | exclude_path | Path to exclude. | *.gz | no | | url_patterns | List of URL patterns. | [] | no | | url_patterns.name | Used as a dimension name. | | yes | -| url_patterns.pattern | Used to match against full original request URI. Pattern syntax in [matcher](https://github.com/netdata/netdata/tree/master/src/go/collectors/go.d.plugin/pkg/matcher#supported-format). | | yes | -| parser | Log parser configuration. | | no | -| parser.log_type | Log parser type. | auto | no | -| parser.csv_config | CSV log parser config. | | no | -| parser.csv_config.delimiter | CSV field delimiter. | , | no | -| parser.csv_config.format | CSV log format. | | no | -| parser.ltsv_config | LTSV log parser config. | | no | -| parser.ltsv_config.field_delimiter | LTSV field delimiter. | \t | no | -| parser.ltsv_config.value_delimiter | LTSV value delimiter. | : | no | -| parser.ltsv_config.mapping | LTSV fields mapping to **known fields**. | | yes | -| parser.json_config | JSON log parser config. | | no | -| parser.json_config.mapping | JSON fields mapping to **known fields**. | | yes | -| parser.regexp_config | RegExp log parser config. | | no | -| parser.regexp_config.pattern | RegExp pattern with named groups. | | yes | +| url_patterns.pattern | Used to match against full original request URI. Pattern syntax in [matcher](https://github.com/netdata/netdata/tree/master/src/go/plugin/go.d/pkg/matcher#supported-format). | | yes | +| log_type | Log parser type. | auto | no | +| csv_config | CSV log parser config. | | no | +| csv_config.delimiter | CSV field delimiter. | , | no | +| csv_config.format | CSV log format. | | no | +| ltsv_config | LTSV log parser config. | | no | +| ltsv_config.field_delimiter | LTSV field delimiter. | \t | no | +| ltsv_config.value_delimiter | LTSV value delimiter. | : | no | +| ltsv_config.mapping | LTSV fields mapping to **known fields**. | | yes | +| json_config | JSON log parser config. | | no | +| json_config.mapping | JSON fields mapping to **known fields**. | | yes | +| regexp_config | RegExp log parser config. | | no | +| regexp_config.pattern | RegExp pattern with named groups. | | yes | ##### url_patterns @@ -246,7 +245,7 @@ url_patterns: ``` -##### parser.log_type +##### log_type Weblog supports 5 different log parsers: @@ -261,8 +260,7 @@ Weblog supports 5 different log parsers: Syntax: ```yaml -parser: - log_type: auto +log_type: auto ``` If `log_type` parameter set to `auto` (which is default), weblog will try to auto-detect appropriate log parser and log format using the last line of the log file. @@ -287,43 +285,41 @@ If `log_type` parameter set to `auto` (which is default), weblog will try to aut If you're using the default Apache/NGINX log format, auto-detect will work for you. If it doesn't work you need to set the format manually. -##### parser.csv_config.format +##### csv_config.format -##### parser.ltsv_config.mapping +##### ltsv_config.mapping The mapping is a dictionary where the key is a field, as in logs, and the value is the corresponding **known field**. > **Note**: don't use `$` and `%` prefixes for mapped field names. ```yaml -parser: - log_type: ltsv - ltsv_config: - mapping: - label1: field1 - label2: field2 +log_type: ltsv +ltsv_config: + mapping: + label1: field1 + label2: field2 ``` -##### parser.json_config.mapping +##### json_config.mapping The mapping is a dictionary where the key is a field, as in logs, and the value is the corresponding **known field**. > **Note**: don't use `$` and `%` prefixes for mapped field names. ```yaml -parser: - log_type: json - json_config: - mapping: - label1: field1 - label2: field2 +log_type: json +json_config: + mapping: + label1: field1 + label2: field2 ``` -##### parser.regexp_config.pattern +##### regexp_config.pattern Use pattern with subexpressions names. These names should be **known fields**. @@ -332,10 +328,9 @@ Use pattern with subexpressions names. These names should be **known fields**. Syntax: ```yaml -parser: - log_type: regexp - regexp_config: - pattern: PATTERN +log_type: regexp +regexp_config: + pattern: PATTERN ``` @@ -350,6 +345,8 @@ There are no configuration examples. ### Debug Mode +**Important**: Debug mode is not supported for data collection jobs created via the UI using the Dyncfg feature. + To troubleshoot issues with the `web_log` collector, run the `go.d.plugin` with the debug option enabled. The output should give you clues as to why the collector isn't working. @@ -372,4 +369,37 @@ should give you clues as to why the collector isn't working. ./go.d.plugin -d -m web_log ``` +### Getting Logs + +If you're encountering problems with the `web_log` collector, follow these steps to retrieve logs and identify potential issues: + +- **Run the command** specific to your system (systemd, non-systemd, or Docker container). +- **Examine the output** for any warnings or error messages that might indicate issues. These messages should provide clues about the root cause of the problem. + +#### System with systemd + +Use the following command to view logs generated since the last Netdata service restart: + +```bash +journalctl _SYSTEMD_INVOCATION_ID="$(systemctl show --value --property=InvocationID netdata)" --namespace=netdata --grep web_log +``` + +#### System without systemd + +Locate the collector log file, typically at `/var/log/netdata/collector.log`, and use `grep` to filter for collector's name: + +```bash +grep web_log /var/log/netdata/collector.log +``` + +**Note**: This method shows logs from all restarts. Focus on the **latest entries** for troubleshooting current issues. + +#### Docker Container + +If your Netdata runs in a Docker container named "netdata" (replace if different), use this command: + +```bash +docker logs netdata 2>&1 | grep web_log +``` + |