summaryrefslogtreecommitdiffstats
path: root/docs/source/usage.rst
diff options
context:
space:
mode:
authorDaniel Baumann <daniel.baumann@progress-linux.org>2024-05-04 17:44:55 +0000
committerDaniel Baumann <daniel.baumann@progress-linux.org>2024-05-04 17:44:55 +0000
commit5068d34c08f951a7ea6257d305a1627b09a95817 (patch)
tree08213e2be853396a3b07ce15dbe222644dcd9a89 /docs/source/usage.rst
parentInitial commit. (diff)
downloadlnav-5068d34c08f951a7ea6257d305a1627b09a95817.tar.xz
lnav-5068d34c08f951a7ea6257d305a1627b09a95817.zip
Adding upstream version 0.11.1.upstream/0.11.1upstream
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'docs/source/usage.rst')
-rw-r--r--docs/source/usage.rst291
1 files changed, 291 insertions, 0 deletions
diff --git a/docs/source/usage.rst b/docs/source/usage.rst
new file mode 100644
index 0000000..d43ed52
--- /dev/null
+++ b/docs/source/usage.rst
@@ -0,0 +1,291 @@
+.. _usage:
+
+Usage
+=====
+
+This chapter contains an overview of how to use **lnav**.
+
+
+Basic Controls
+--------------
+
+Like most file viewers, scrolling through files can be done with the usual
+:ref:`hotkeys<hotkeys>`. For non-trivial operations, you can enter the
+:ref:`command<commands>` prompt by pressing :kbd:`:`. To analyze data in a
+log file, you can enter the :ref:`SQL prompt<sql-ext>` by pressing :kbd:`;`.
+
+.. tip::
+
+ Check the bottom right corner of the screen for tips on hotkeys that might
+ be useful in the current context.
+
+ .. figure:: hotkey-tips.png
+ :align: center
+
+ When **lnav** is first open, it suggests using :kbd:`e` and
+ :kbd:`Shift` + :kbd:`e` to jump to error messages.
+
+
+Viewing Files
+-------------
+
+The files to view in **lnav** can be given on the command-line or passed to the
+:ref:`:open<open>` command. A
+`glob pattern <https://en.wikipedia.org/wiki/Glob_(programming)>`_ can be given
+to watch for files with a common name. If the path is a directory, all of the
+files in the directory will be opened and the directory will be monitored for
+files to be added or removed from the view. If the path is an archive or
+compressed file (and lnav was built with libarchive), the archive will be
+extracted to a temporary location and the files within will be loaded. The
+files that are found will be scanned to identify their file format. Files
+that match a log format will be collated by time and displayed in the LOG
+view. Plain text files can be viewed in the TEXT view, which can be accessed
+by pressing :kbd:`t`.
+
+
+Archive Support
+^^^^^^^^^^^^^^^
+
+.. f0:archive
+
+If **lnav** is compiled with `libarchive <https://www.libarchive.org>`_,
+any files to be opened will be examined to see if they are a supported archive
+type. If so, the contents of the archive will be extracted to the
+:code:`$TMPDIR/lnav-user-${UID}-work/archives/` directory. Once extracted, the
+files within will be loaded into lnav. To speed up opening large amounts of
+files, any file that meets the following conditions will be automatically
+hidden and not indexed:
+
+* Binary files
+* Plain text files that are larger than 128KB
+* Duplicate log files
+
+The unpacked files will be left in the temporary directory after exiting
+**lnav** so that opening the same archive again will be faster. Unpacked
+archives that have not been accessed in the past two days will be automatically
+deleted the next time **lnav** is started.
+
+
+.. _remote:
+
+Remote Files
+^^^^^^^^^^^^
+
+Files on remote machines can be viewed and tailed if you have access to the
+machines via SSH. First, make sure you can SSH into the remote machine without
+any interaction by: 1) accepting the host key as known and 2) copying your
+identity's public key to the :file:`.ssh/authorized_keys` file on the remote
+machine. Once the setup is complete, you can open a file on a remote host
+using the same syntax as :command:`scp(1)` where the username and host are
+given, followed by a colon, and then the path to the files, like so::
+
+ [user@]host:/path/to/logs
+
+For example, to open :file:`/var/log/syslog.log` on "host1.example.com" as the
+user "dean", you would write:
+
+.. prompt:: bash
+
+ lnav dean@host1.example.com:/var/log/syslog.log
+
+Remote files can also be opened using the :ref:`:open<open>` command. Opening
+a remote file in the TUI has the advantage that the file path can be
+:kbd:`TAB`-completed and a preview is shown of the first few lines of the
+file.
+
+.. note::
+
+ If lnav is installed from the `snap <https://snapcraft.io/lnav>`_, you will
+ need to connect it to the
+ `ssh-keys plug <https://snapcraft.io/docs/ssh-keys-interface>`_ using the
+ following command:
+
+ .. prompt:: bash
+
+ sudo snap connect lnav:ssh-keys
+
+.. note::
+
+ Remote file access is implemented by transferring an
+ `αcτµαlly pδrταblε εxεcµταblε <https://justine.lol/ape.html>`_ to the
+ destination and invoking it. An APE binary can run on most any x86_64
+ machine and OS (i.e. MacOS, Linux, FreeBSD, Windows). The binary is
+ baked into the lnav executable itself, so there is no extra setup that
+ needs to be done on the remote machine.
+
+ The binary file is named ``tailer.bin.XXXXXX`` where *XXXXXX* is 6 random digits.
+ The file is, under normal circumstancies, deleted immediately.
+
+Searching
+---------
+
+Any log messages that are loaded into **lnav** are indexed by time and log
+level (e.g. error, warning) to make searching quick and easy with
+:ref:`hotkeys<hotkeys>`. For example, pressing :kbd:`e` will jump to the
+next error in the file and pressing :kbd:`Shift` + :kbd:`e` will jump to
+the previous error. Plain text searches can be done by pressing :kbd:`/`
+to enter the search prompt. A regular expression can be entered into the
+prompt to start a search through the current view.
+
+
+.. _filtering:
+
+Filtering
+---------
+
+To reduce the amount of noise in a log file, **lnav** can hide log messages
+that match certain criteria. The following sub-sections explain ways to go
+about that.
+
+
+Regular Expression Match
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+If there are log messages that you are not interested in, you can do a
+"filter out" to hide messages that match a pattern. A filter can be created
+using the interactive editor, the :ref:`:filter-out<filter_out>` command, or
+by doing an :code:`INSERT` into the
+:ref:`lnav_view_filters<table_lnav_view_filters>` table.
+
+If there are log messages that you are only interested in, you can do a
+"filter in" to only show messages that match a pattern. The filter can be
+created using the interactive editor, the :ref:`:filter-in<filter_in>` command,
+or by doing an :code:`INSERT` into the
+:ref:`lnav_view_filters<table_lnav_view_filters>` table.
+
+
+SQLite Expression
+^^^^^^^^^^^^^^^^^
+
+Complex filtering can be done by passing a SQLite expression to the
+:ref:`:filter-expr<filter_expr>` command. The expression will be executed for
+every log message and if it returns true, the line will be shown in the log
+view.
+
+
+Time
+^^^^
+
+To limit log messages to a given time frame, the
+:ref:`:hide-lines-before<hide_lines_before>` and
+:ref:`:hide-lines-after<hide_lines_after>` commands can be used to specify
+the beginning and end of the time frame.
+
+
+Log level
+^^^^^^^^^
+
+To hide messages below a certain log level, you can use the
+:ref:`:set-min-log-level<set_min_log_level>` command.
+
+.. _search_tables:
+
+Search Tables
+-------------
+
+Search tables allow you to access arbitrary data in log messages through
+SQLite virtual tables. If there is some data in a log message that you can
+match with a regular expression, you can create a search-table that matches
+that data and any capture groups will be plumbed through as columns in the
+search table.
+
+Creating a search table can be done interactively using the
+:ref:`:create-search-table<create_search_table>` command or by adding it to
+a :ref:`log format definition<log_formats>`. The main difference between
+the two is that tables defined as part of a format will only search messages
+from log files with that format and the tables will include log message
+columns defined in that format. Whereas a table created with the command
+will search messages from all different formats and no format-specific
+columns will be included in the table.
+
+.. _taking_notes:
+
+Taking Notes
+------------
+
+A few of the columns in the log tables can be updated on a row-by-row basis to
+allow you to take notes. The majority of the columns in a log table are
+read-only since they are backed by the log files themselves. However, the
+following columns can be changed by an :code:`UPDATE` statement:
+
+* **log_part** - The "partition" the log message belongs to. This column can
+ also be changed by the :ref:`:partition-name<partition_name>` command.
+* **log_mark** - Indicates whether the line has been bookmarked.
+* **log_comment** - A free-form text field for storing commentary. This
+ column can also be changed by the :ref:`:comment<comment>` command.
+* **log_tags** - A JSON list of tags associated with the log message. This
+ column can also be changed by the :ref:`:tag<tag>` command.
+
+While these columns can be updated by through other means, using the SQL
+interface allows you to make changes automatically and en masse. For example,
+to bookmark all lines that have the text "something interesting" in the log
+message body, you can execute:
+
+.. code-block:: custsqlite
+
+ ;UPDATE all_logs SET log_mark = 1 WHERE log_body LIKE '%something interesting%'
+
+As a more advanced example of the power afforded by SQL and **lnav**'s virtual
+tables, we will tag log messages where the IP address bound by dhclient has
+changed. For example, if dhclient reports "bound to 10.0.0.1" initially and
+then reports "bound to 10.0.0.2", we want to tag only the messages where the
+IP address was different from the previous message. While this can be done
+with a single SQL statement [#]_, we will break things down into a few steps for
+this example. First, we will use the :ref:`:create-search-table<create_search_table>`
+command to match the dhclient message and extract the IP address:
+
+.. code-block:: lnav
+
+ :create-search-table dhclient_ip bound to (?<ip>[^ ]+)
+
+The above command will create a new table named :code:`dhclient_ip` with the
+standard log columns and an :code:`ip` column that contains the IP address.
+Next, we will create a view over the :code:`dhclient_ip` table that returns
+the log message line number, the IP address from the current row and the IP
+address from the previous row:
+
+.. code-block:: custsqlite
+
+ ;CREATE VIEW IF NOT EXISTS dhclient_ip_changes AS SELECT log_line, ip, lag(ip) OVER (ORDER BY log_line) AS prev_ip FROM dhclient_ip
+
+Finally, the following :code:`UPDATE` statement will concatenate the tag
+"#ipchanged" onto the :code:`log_tags` column for any rows in the view where
+the current IP is different from the previous IP:
+
+.. code-block:: custsqlite
+
+ ;UPDATE syslog_log SET log_tags = json_concat(log_tags, '#ipchanged') WHERE log_line IN (SELECT log_line FROM dhclient_ip_changes WHERE ip != prev_ip)
+
+Since the above can be a lot to type out interactively, you can put these
+commands into a :ref:`script<scripts>` and execute that script with the
+:kbd:`\|` hotkey.
+
+.. [#] The expression :code:`regexp_match('bound to ([^ ]+)', log_body) as ip`
+ can be used to extract the IP address from the log message body.
+
+Sharing Sessions With Others
+----------------------------
+
+After setting up filters, bookmarks, and making notes, you might want to share
+your work with others. If they have access to the same log files, you can
+use the :ref:`:export-session-to<export_session_to>` command to write an
+executable **lnav** script that will recreate the current session state. The
+script contains various SQL statements and **lnav** commands that capture the
+current state. So, you should feel free to modify the script or use it as a
+reference to learn about more advanced uses of lnav.
+
+The script will capture the file paths that were explicitly specified and
+not the files that were actually opened. For example, if you specified
+"/var/log" on the command line, the script will include
+:code:`:open /var/log/*` and not an individual open for each file in that
+directory.
+
+Also, in order to support archives of log files, lnav will try to find the
+directory where the archive was unpacked and use that as the base for the
+:code:`:open` command. Currently, this is done by searching for the top
+"README" file in the directory hierarchy containing the files [1]_. The
+consumer of the session script can then set the :code:`LOG_DIR_0` (or 1, 2,
+...) environment variable to change where the log files will be loaded from.
+
+.. [1] It is assumed a log archive would have a descriptive README file.
+ Other heuristics may be added in the future.