diff options
Diffstat (limited to 'docs')
44 files changed, 673 insertions, 738 deletions
diff --git a/docs/README.md b/docs/README.md index 44b4455..07ac3d2 100755 --- a/docs/README.md +++ b/docs/README.md @@ -6,10 +6,10 @@ # Arista Network Test Automation (ANTA) Framework -| **Code** | [](https://github.com/astral-sh/ruff) [](https://numpydoc.readthedocs.io/en/latest/format.html) [](https://sonarcloud.io/summary/new_code?id=aristanetworks_anta) | +| **Code** | [](https://github.com/astral-sh/ruff) [](https://numpydoc.readthedocs.io/en/latest/format.html) [](https://sonarcloud.io/summary/new_code?id=aristanetworks_anta) [](https://sonarcloud.io/summary/overall?id=aristanetworks_anta) | | :------------: | :-------| | **License** | [](https://github.com/aristanetworks/anta/blob/main/LICENSE) | -| **GitHub** | [](https://github.com/aristanetworks/anta/actions/workflows/code-testing.yml)    [](https://github.com/aristanetworks/anta/releases/) [](https://github.com/aristanetworks/anta/graphs/contributors) | +| **GitHub** | [](https://github.com/aristanetworks/anta/actions/workflows/code-testing.yml)   [](https://github.com/aristanetworks/anta/releases/) [](https://github.com/aristanetworks/anta/graphs/contributors) | | **PyPi** |     | ANTA is Python framework that automates tests for Arista devices. @@ -39,7 +39,9 @@ If you plan to use ANTA only as a CLI tool you can use `pipx` to install it. [`pipx`](https://pipx.pypa.io/stable/) is a tool to install and run python applications in isolated environments. Refer to `pipx` instructions to install on your system. `pipx` installs ANTA in an isolated python environment and makes it available globally. +<!-- markdownlint-disable no-emphasis-as-heading --> **This is not recommended if you plan to contribute to ANTA** +<!-- markdownlint-enable no-emphasis-as-heading --> ```bash # Install ANTA CLI with pipx diff --git a/docs/advanced_usages/as-python-lib.md b/docs/advanced_usages/as-python-lib.md index f8d6734..49c010f 100644 --- a/docs/advanced_usages/as-python-lib.md +++ b/docs/advanced_usages/as-python-lib.md @@ -11,11 +11,11 @@ ANTA is a Python library that can be used in user applications. This section des ## [AntaDevice](../api/device.md#anta.device.AntaDevice) Abstract Class -A device is represented in ANTA as a instance of a subclass of the [AntaDevice](../api/device.md### ::: anta.device.AntaDevice) abstract class. +A device is represented in ANTA as a instance of a subclass of the [AntaDevice](../api/device.md#anta.device.AntaDevice) abstract class. There are few abstract methods that needs to be implemented by child classes: - The [collect()](../api/device.md#anta.device.AntaDevice.collect) coroutine is in charge of collecting outputs of [AntaCommand](../api/models.md#anta.models.AntaCommand) instances. -- The [refresh()](../api/device.md#anta.device.AntaDevice.refresh) coroutine is in charge of updating attributes of the [AntaDevice](../api/device.md### ::: anta.device.AntaDevice) instance. These attributes are used by [AntaInventory](../api/inventory.md#anta.inventory.AntaInventory) to filter out unreachable devices or by [AntaTest](../api/models.md#anta.models.AntaTest) to skip devices based on their hardware models. +- The [refresh()](../api/device.md#anta.device.AntaDevice.refresh) coroutine is in charge of updating attributes of the [AntaDevice](../api/device.md#anta.device.AntaDevice) instance. These attributes are used by [AntaInventory](../api/inventory.md#anta.inventory.AntaInventory) to filter out unreachable devices or by [AntaTest](../api/models.md#anta.models.AntaTest) to skip devices based on their hardware models. The [copy()](../api/device.md#anta.device.AntaDevice.copy) coroutine is used to copy files to and from the device. It does not need to be implemented if tests are not using it. @@ -24,7 +24,7 @@ The [copy()](../api/device.md#anta.device.AntaDevice.copy) coroutine is used to The [AsyncEOSDevice](../api/device.md#anta.device.AsyncEOSDevice) class is an implementation of [AntaDevice](../api/device.md#anta.device.AntaDevice) for Arista EOS. It uses the [aio-eapi](https://github.com/jeremyschulman/aio-eapi) eAPI client and the [AsyncSSH](https://github.com/ronf/asyncssh) library. -- The [collect()](../api/device.md#anta.device.AsyncEOSDevice.collect) coroutine collects [AntaCommand](../api/models.md#anta.models.AntaCommand) outputs using eAPI. +- The [_collect()](../api/device.md#anta.device.AsyncEOSDevice._collect) coroutine collects [AntaCommand](../api/models.md#anta.models.AntaCommand) outputs using eAPI. - The [refresh()](../api/device.md#anta.device.AsyncEOSDevice.refresh) coroutine tries to open a TCP connection on the eAPI port and update the `is_online` attribute accordingly. If the TCP connection succeeds, it sends a `show version` command to gather the hardware model of the device and updates the `established` and `hw_model` attributes. - The [copy()](../api/device.md#anta.device.AsyncEOSDevice.copy) coroutine copies files to and from the device using the SCP protocol. @@ -32,282 +32,26 @@ It uses the [aio-eapi](https://github.com/jeremyschulman/aio-eapi) eAPI client a The [AntaInventory](../api/inventory.md#anta.inventory.AntaInventory) class is a subclass of the standard Python type [dict](https://docs.python.org/3/library/stdtypes.html#dict). The keys of this dictionary are the device names, the values are [AntaDevice](../api/device.md#anta.device.AntaDevice) instances. - [AntaInventory](../api/inventory.md#anta.inventory.AntaInventory) provides methods to interact with the ANTA inventory: -- The [add_device()](../api/inventory.md#anta.inventory.AntaInventory.add_device) method adds an [AntaDevice](../api/device.md### ::: anta.device.AntaDevice) instance to the inventory. Adding an entry to [AntaInventory](../api/inventory.md#anta.inventory.AntaInventory) with a key different from the device name is not allowed. +- The [add_device()](../api/inventory.md#anta.inventory.AntaInventory.add_device) method adds an [AntaDevice](../api/device.md#anta.device.AntaDevice) instance to the inventory. Adding an entry to [AntaInventory](../api/inventory.md#anta.inventory.AntaInventory) with a key different from the device name is not allowed. - The [get_inventory()](../api/inventory.md#anta.inventory.AntaInventory.get_inventory) returns a new [AntaInventory](../api/inventory.md#anta.inventory.AntaInventory) instance with filtered out devices based on the method inputs. - The [connect_inventory()](../api/inventory.md#anta.inventory.AntaInventory.connect_inventory) coroutine will execute the [refresh()](../api/device.md#anta.device.AntaDevice.refresh) coroutines of all the devices in the inventory. - The [parse()](../api/inventory.md#anta.inventory.AntaInventory.parse) static method creates an [AntaInventory](../api/inventory.md#anta.inventory.AntaInventory) instance from a YAML file and returns it. The devices are [AsyncEOSDevice](../api/device.md#anta.device.AsyncEOSDevice) instances. +## Examples -To parse a YAML inventory file and print the devices connection status: +### Parse an ANTA inventory file ```python -""" -Example -""" -import asyncio - -from anta.inventory import AntaInventory - - -async def main(inv: AntaInventory) -> None: - """ - Take an AntaInventory and: - 1. try to connect to every device in the inventory - 2. print a message for every device connection status - """ - await inv.connect_inventory() - - for device in inv.values(): - if device.established: - print(f"Device {device.name} is online") - else: - print(f"Could not connect to device {device.name}") - -if __name__ == "__main__": - # Create the AntaInventory instance - inventory = AntaInventory.parse( - filename="inv.yml", - username="arista", - password="@rista123", - ) - - # Run the main coroutine - res = asyncio.run(main(inventory)) +--8<-- "parse_anta_inventory_file.py" ``` -??? note "How to create your inventory file" +!!! note "How to create your inventory file" Please visit this [dedicated section](../usage-inventory-catalog.md) for how to use inventory and catalog files. -To run an EOS commands list on the reachable devices from the inventory: -```python -""" -Example -""" -# This is needed to run the script for python < 3.10 for typing annotations -from __future__ import annotations - -import asyncio -from pprint import pprint - -from anta.inventory import AntaInventory -from anta.models import AntaCommand - - -async def main(inv: AntaInventory, commands: list[str]) -> dict[str, list[AntaCommand]]: - """ - Take an AntaInventory and a list of commands as string and: - 1. try to connect to every device in the inventory - 2. collect the results of the commands from each device - - Returns: - a dictionary where key is the device name and the value is the list of AntaCommand ran towards the device - """ - await inv.connect_inventory() - - # Make a list of coroutine to run commands towards each connected device - coros = [] - # dict to keep track of the commands per device - result_dict = {} - for name, device in inv.get_inventory(established_only=True).items(): - anta_commands = [AntaCommand(command=command, ofmt="json") for command in commands] - result_dict[name] = anta_commands - coros.append(device.collect_commands(anta_commands)) - - # Run the coroutines - await asyncio.gather(*coros) - - return result_dict - - -if __name__ == "__main__": - # Create the AntaInventory instance - inventory = AntaInventory.parse( - filename="inv.yml", - username="arista", - password="@rista123", - ) - - # Create a list of commands with json output - commands = ["show version", "show ip bgp summary"] - - # Run the main asyncio entry point - res = asyncio.run(main(inventory, commands)) - - pprint(res) -``` - - -## Use tests from ANTA - -All the test classes inherit from the same abstract Base Class AntaTest. The Class definition indicates which commands are required for the test and the user should focus only on writing the `test` function with optional keywords argument. The instance of the class upon creation instantiates a TestResult object that can be accessed later on to check the status of the test ([unset, skipped, success, failure, error]). - -### Test structure - -All tests are built on a class named `AntaTest` which provides a complete toolset for a test: - -- Object creation -- Test definition -- TestResult definition -- Abstracted method to collect data - -This approach means each time you create a test it will be based on this `AntaTest` class. Besides that, you will have to provide some elements: - -- `name`: Name of the test -- `description`: A human readable description of your test -- `categories`: a list of categories to sort test. -- `commands`: a list of command to run. This list _must_ be a list of `AntaCommand` which is described in the next part of this document. - -Here is an example of a hardware test related to device temperature: +### Run EOS commands ```python -from __future__ import annotations - -import logging -from typing import Any, Dict, List, Optional, cast - -from anta.models import AntaTest, AntaCommand - - -class VerifyTemperature(AntaTest): - """ - Verifies device temparture is currently OK. - """ - - # The test name - name = "VerifyTemperature" - # A small description of the test, usually the first line of the class docstring - description = "Verifies device temparture is currently OK" - # The category of the test, usually the module name - categories = ["hardware"] - # The command(s) used for the test. Could be a template instead - commands = [AntaCommand(command="show system environment temperature", ofmt="json")] - - # Decorator - @AntaTest.anta_test - # abstract method that must be defined by the child Test class - def test(self) -> None: - """Run VerifyTemperature validation""" - command_output = cast(Dict[str, Dict[Any, Any]], self.instance_commands[0].output) - temperature_status = command_output["systemStatus"] if "systemStatus" in command_output.keys() else "" - if temperature_status == "temperatureOk": - self.result.is_success() - else: - self.result.is_failure(f"Device temperature is not OK, systemStatus: {temperature_status }") +--8<-- "run_eos_commands.py" ``` - -When you run the test, object will automatically call its `anta.models.AntaTest.collect()` method to get device output for each command if no pre-collected data was given to the test. This method does a loop to call `anta.inventory.models.InventoryDevice.collect()` methods which is in charge of managing device connection and how to get data. - -??? info "run test offline" - You can also pass eos data directly to your test if you want to validate data collected in a different workflow. An example is provided below just for information: - - ```python - test = VerifyTemperature(device, eos_data=test_data["eos_data"]) - asyncio.run(test.test()) - ``` - -The `test` function is always the same and __must__ be defined with the `@AntaTest.anta_test` decorator. This function takes at least one argument which is a `anta.inventory.models.InventoryDevice` object. -In some cases a test would rely on some additional inputs from the user, for instance the number of expected peers or some expected numbers. All parameters __must__ come with a default value and the test function __should__ validate the parameters values (at this stage this is the only place where validation can be done but there are future plans to make this better). - -```python -class VerifyTemperature(AntaTest): - ... - @AntaTest.anta_test - def test(self) -> None: - pass - -class VerifyTransceiversManufacturers(AntaTest): - ... - @AntaTest.anta_test - def test(self, manufacturers: Optional[List[str]] = None) -> None: - # validate the manufactures parameter - pass -``` - -The test itself does not return any value, but the result is directly available from your AntaTest object and exposes a `anta.result_manager.models.TestResult` object with result, name of the test and optional messages: - - -- `name` (str): Device name where the test has run. -- `test` (str): Test name runs on the device. -- `categories` (List[str]): List of categories the TestResult belongs to, by default the AntaTest categories. -- `description` (str): TestResult description, by default the AntaTest description. -- `results` (str): Result of the test. Can be one of ["unset", "success", "failure", "error", "skipped"]. -- `message` (str, optional): Message to report after the test if any. -- `custom_field` (str, optional): Custom field to store a string for flexibility in integrating with ANTA - -```python -from anta.tests.hardware import VerifyTemperature - -test = VerifyTemperature(device, eos_data=test_data["eos_data"]) -asyncio.run(test.test()) -assert test.result.result == "success" -``` - -### Classes for commands - -To make it easier to get data, ANTA defines 2 different classes to manage commands to send to devices: - -#### [AntaCommand](../api/models.md#anta.models.AntaCommand) Class - -Represent a command with following information: - -- Command to run -- Output format expected -- eAPI version -- Output of the command - -Usage example: - -```python -from anta.models import AntaCommand - -cmd1 = AntaCommand(command="show zerotouch") -cmd2 = AntaCommand(command="show running-config diffs", ofmt="text") -``` - -!!! tip "Command revision and version" - * Most of EOS commands return a JSON structure according to a model (some commands may not be modeled hence the necessity to use `text` outformat sometimes. - * The model can change across time (adding feature, ... ) and when the model is changed in a non backward-compatible way, the __revision__ number is bumped. The initial model starts with __revision__ 1. - * A __revision__ applies to a particular CLI command whereas a __version__ is global to an eAPI call. The __version__ is internally translated to a specific __revision__ for each CLI command in the RPC call. The currently supported __version__ values are `1` and `latest`. - * A __revision takes precedence over a version__ (e.g. if a command is run with version="latest" and revision=1, the first revision of the model is returned) - * By default, eAPI returns the first revision of each model to ensure that when upgrading, integrations with existing tools are not broken. This is done by using by default `version=1` in eAPI calls. - - By default, ANTA uses `version="latest"` in AntaCommand, but when developing tests, the revision MUST be provided when the outformat of the command is `json`. As explained earlier, this is to ensure that the eAPI always returns the same output model and that the test remains always valid from the day it was created. For some commands, you may also want to run them with a different revision or version. - - For instance, the `VerifyBFDPeersHealth` test leverages the first revision of `show bfd peers`: - - ``` - # revision 1 as later revision introduce additional nesting for type - commands = [AntaCommand(command="show bfd peers", revision=1)] - ``` - -#### [AntaTemplate](../api/models.md#anta.models.AntaTemplate) Class - -Because some command can require more dynamic than just a command with no parameter provided by user, ANTA supports command template: you define a template in your test class and user provide parameters when creating test object. - -```python - -class RunArbitraryTemplateCommand(AntaTest): - """ - Run an EOS command and return result - Based on AntaTest to build relevant output for pytest - """ - - name = "Run aributrary EOS command" - description = "To be used only with anta debug commands" - template = AntaTemplate(template="show interfaces {ifd}") - categories = ["debug"] - - @AntaTest.anta_test - def test(self) -> None: - errdisabled_interfaces = [interface for interface, value in response["interfaceStatuses"].items() if value["linkStatus"] == "errdisabled"] - ... - - -params = [{"ifd": "Ethernet2"}, {"ifd": "Ethernet49/1"}] -run_command1 = RunArbitraryTemplateCommand(device_anta, params) -``` - -In this example, test waits for interfaces to check from user setup and will only check for interfaces in `params` diff --git a/docs/advanced_usages/caching.md b/docs/advanced_usages/caching.md index ce4a787..8b089ce 100644 --- a/docs/advanced_usages/caching.md +++ b/docs/advanced_usages/caching.md @@ -10,7 +10,7 @@ ANTA is a streamlined Python framework designed for efficient interaction with n By default, ANTA utilizes [aiocache](https://github.com/aio-libs/aiocache)'s memory cache backend, also called [`SimpleMemoryCache`](https://aiocache.aio-libs.org/en/v0.12.2/caches.html#simplememorycache). This library aims for simplicity and supports asynchronous operations to go along with Python `asyncio` used in ANTA. -The `_init_cache()` method of the [AntaDevice](../advanced_usages/as-python-lib.md#antadevice-abstract-class) abstract class initializes the cache. Child classes can override this method to tweak the cache configuration: +The `_init_cache()` method of the [AntaDevice](../api/device.md#anta.device.AntaDevice) abstract class initializes the cache. Child classes can override this method to tweak the cache configuration: ```python def _init_cache(self) -> None: @@ -29,7 +29,7 @@ The cache is initialized per `AntaDevice` and uses the following cache key desig `<device_name>:<uid>` -The `uid` is an attribute of [AntaCommand](../advanced_usages/as-python-lib.md#antacommand-class), which is a unique identifier generated from the command, version, revision and output format. +The `uid` is an attribute of [AntaCommand](../api/models.md#anta.models.AntaCommand), which is a unique identifier generated from the command, version, revision and output format. Each UID has its own asyncio lock. This design allows coroutines that need to access the cache for different UIDs to do so concurrently. The locks are managed by the `self.cache_locks` dictionary. @@ -44,10 +44,13 @@ Caching is enabled by default in ANTA following the previous configuration and m There might be scenarios where caching is not wanted. You can disable caching in multiple ways in ANTA: 1. Caching can be disabled globally, for **ALL** commands on **ALL** devices, using the `--disable-cache` global flag when invoking anta at the [CLI](../cli/overview.md#invoking-anta-cli): + ```bash anta --disable-cache --username arista --password arista nrfu table ``` -2. Caching can be disabled per device, network or range by setting the `disable_cache` key to `True` when defining the ANTA [Inventory](../usage-inventory-catalog.md#create-an-inventory-file) file: + +2. Caching can be disabled per device, network or range by setting the `disable_cache` key to `True` when defining the ANTA [Inventory](../usage-inventory-catalog.md#device-inventory) file: + ```yaml anta_inventory: hosts: @@ -69,9 +72,10 @@ There might be scenarios where caching is not wanted. You can disable caching in end: 172.22.22.19 disable_cache: True ``` + This approach effectively disables caching for **ALL** commands sent to devices targeted by the `disable_cache` key. -3. For tests developers, caching can be disabled for a specific [`AntaCommand`](../advanced_usages/as-python-lib.md#antacommand-class) or [`AntaTemplate`](../advanced_usages/as-python-lib.md#antatemplate-class) by setting the `use_cache` attribute to `False`. That means the command output will always be collected on the device and therefore, never use caching. +3. For tests developers, caching can be disabled for a specific [`AntaCommand`](../api/models.md#anta.models.AntaCommand) or [`AntaTemplate`](../api/models.md#anta.models.AntaTemplate) by setting the `use_cache` attribute to `False`. That means the command output will always be collected on the device and therefore, never use caching. ### Disable caching in a child class of `AntaDevice` diff --git a/docs/advanced_usages/custom-tests.md b/docs/advanced_usages/custom-tests.md index c6a2fa8..d79fe50 100644 --- a/docs/advanced_usages/custom-tests.md +++ b/docs/advanced_usages/custom-tests.md @@ -4,7 +4,7 @@ ~ that can be found in the LICENSE file. --> -!!! info "" +!!! info This documentation applies for both creating tests in ANTA or creating your own test package. ANTA is not only a Python library with a CLI and a collection of built-in tests, it is also a framework you can extend by building your own tests. @@ -64,16 +64,13 @@ Full AntaTest API documentation is available in the [API documentation section]( - `name` (`str`): Name of the test. Used during reporting. - `description` (`str`): A human readable description of your test. - `categories` (`list[str]`): A list of categories in which the test belongs. -- `commands` (`[list[AntaCommand | AntaTemplate]]`): A list of command to collect from devices. This list __must__ be a list of [AntaCommand](../api/models.md#anta.models.AntaCommand) or [AntaTemplate](../api/models.md#anta.models.AntaTemplate) instances. Rendering [AntaTemplate](../api/models.md#anta.models.AntaTemplate) instances will be discussed later. +- `commands` (`[list[AntaCommand | AntaTemplate]]`): A list of command to collect from devices. This list **must** be a list of [AntaCommand](../api/models.md#anta.models.AntaCommand) or [AntaTemplate](../api/models.md#anta.models.AntaTemplate) instances. Rendering [AntaTemplate](../api/models.md#anta.models.AntaTemplate) instances will be discussed later. !!! info All these class attributes are mandatory. If any attribute is missing, a `NotImplementedError` exception will be raised during class instantiation. ### Instance Attributes -!!! info - You can access an instance attribute in your code using the `self` reference. E.g. you can access the test input values using `self.inputs`. - ::: anta.models.AntaTest options: show_docstring_attributes: true @@ -87,7 +84,6 @@ Full AntaTest API documentation is available in the [API documentation section]( show_root_toc_entry: false heading_level: 10 - !!! note "Logger object" ANTA already provides comprehensive logging at every steps of a test execution. The [AntaTest](../api/models.md#anta.models.AntaTest) class also provides a `logger` attribute that is a Python logger specific to the test instance. See [Python documentation](https://docs.python.org/3/library/logging.html) for more information. @@ -140,8 +136,8 @@ Full `ResultOverwrite` model documentation is available in [API documentation se ### Methods -- [test(self) -> None](../api/models.md#anta.models.AntaTest.test): This is an abstract method that __must__ be implemented. It contains the test logic that can access the collected command outputs using the `instance_commands` instance attribute, access the test inputs using the `inputs` instance attribute and __must__ set the `result` instance attribute accordingly. It must be implemented using the `AntaTest.anta_test` decorator that provides logging and will collect commands before executing the `test()` method. -- [render(self, template: AntaTemplate) -> list[AntaCommand]](../api/models.md#anta.models.AntaTest.render): This method only needs to be implemented if [AntaTemplate](../api/models.md#anta.models.AntaTemplate) instances are present in the `commands` class attribute. It will be called for every [AntaTemplate](../api/models.md#anta.models.AntaTemplate) occurrence and __must__ return a list of [AntaCommand](../api/models.md#anta.models.AntaCommand) using the [AntaTemplate.render()](../api/models.md#anta.models.AntaTemplate.render) method. It can access test inputs using the `inputs` instance attribute. +- [test(self) -> None](../api/models.md#anta.models.AntaTest.test): This is an abstract method that **must** be implemented. It contains the test logic that can access the collected command outputs using the `instance_commands` instance attribute, access the test inputs using the `inputs` instance attribute and **must** set the `result` instance attribute accordingly. It must be implemented using the `AntaTest.anta_test` decorator that provides logging and will collect commands before executing the `test()` method. +- [render(self, template: AntaTemplate) -> list[AntaCommand]](../api/models.md#anta.models.AntaTest.render): This method only needs to be implemented if [AntaTemplate](../api/models.md#anta.models.AntaTemplate) instances are present in the `commands` class attribute. It will be called for every [AntaTemplate](../api/models.md#anta.models.AntaTemplate) occurrence and **must** return a list of [AntaCommand](../api/models.md#anta.models.AntaCommand) using the [AntaTemplate.render()](../api/models.md#anta.models.AntaTemplate.render) method. It can access test inputs using the `inputs` instance attribute. ## Test execution @@ -199,6 +195,22 @@ class <YourTestName>(AntaTest): ] ``` +!!! tip "Command revision and version" + * Most of EOS commands return a JSON structure according to a model (some commands may not be modeled hence the necessity to use `text` outformat sometimes. + * The model can change across time (adding feature, ... ) and when the model is changed in a non backward-compatible way, the **revision** number is bumped. The initial model starts with **revision** 1. + * A **revision** applies to a particular CLI command whereas a **version** is global to an eAPI call. The **version** is internally translated to a specific **revision** for each CLI command in the RPC call. The currently supported **version** values are `1` and `latest`. + * A **revision takes precedence over a version** (e.g. if a command is run with version="latest" and revision=1, the first revision of the model is returned) + * By default, eAPI returns the first revision of each model to ensure that when upgrading, integrations with existing tools are not broken. This is done by using by default `version=1` in eAPI calls. + + By default, ANTA uses `version="latest"` in AntaCommand, but when developing tests, the revision MUST be provided when the outformat of the command is `json`. As explained earlier, this is to ensure that the eAPI always returns the same output model and that the test remains always valid from the day it was created. For some commands, you may also want to run them with a different revision or version. + + For instance, the `VerifyBFDPeersHealth` test leverages the first revision of `show bfd peers`: + + ``` + # revision 1 as later revision introduce additional nesting for type + commands = [AntaCommand(command="show bfd peers", revision=1)] + ``` + ### Inputs definition If the user needs to provide inputs for your test, you need to define a [pydantic model](https://docs.pydantic.dev/latest/usage/models/) that defines the schema of the test inputs: @@ -261,6 +273,7 @@ class <YourTestName>(AntaTest): ``` The logic usually includes the following different stages: + 1. Parse the command outputs using the `self.instance_commands` instance attribute. 2. If needed, access the test inputs using the `self.inputs` instance attribute and write your conditional logic. 3. Set the `result` instance attribute to reflect the test result by either calling `self.result.is_success()` or `self.result.is_failure("<FAILURE REASON>")`. Sometimes, setting the test result to `skipped` using `self.result.is_skipped("<SKIPPED REASON>")` can make sense (e.g. testing the OSPF neighbor states but no neighbor was found). However, you should not need to catch any exception and set the test result to `error` since the error handling is done by the framework, see below. @@ -341,6 +354,7 @@ anta_custom.dc_project: - VerifyFeatureX: minimum: 1 ``` + And now you can run your NRFU tests with the CLI: ```bash diff --git a/docs/api/catalog.md b/docs/api/catalog.md index fc719ea..44cc4df 100644 --- a/docs/api/catalog.md +++ b/docs/api/catalog.md @@ -5,6 +5,7 @@ --> ### ::: anta.catalog.AntaCatalog + options: filters: ["!^_[^_]", "!__str__"] diff --git a/docs/api/csv_reporter.md b/docs/api/csv_reporter.md new file mode 100644 index 0000000..0432c45 --- /dev/null +++ b/docs/api/csv_reporter.md @@ -0,0 +1,13 @@ +--- +anta_title: CSV Reporter +--- +<!-- + ~ Copyright (c) 2023-2024 Arista Networks, Inc. + ~ Use of this source code is governed by the Apache License 2.0 + ~ that can be found in the LICENSE file. + --> + +::: anta.reporter.csv_reporter + options: + show_root_heading: false + show_root_toc_entry: false diff --git a/docs/api/device.md b/docs/api/device.md index 03cff19..136fec7 100644 --- a/docs/api/device.md +++ b/docs/api/device.md @@ -6,20 +6,20 @@ # AntaDevice base class -## UML representation + - +## ::: anta.device.AntaDevice -### ::: anta.device.AntaDevice options: - filters: ["!^_[^_]", "!__(eq|rich_repr)__"] + filters: ["!^_[^_]", "!__(eq|rich_repr)__", "_collect"] # Async EOS device class -## UML representation + - +<!-- _collect must be last to be kept --> + +## ::: anta.device.AsyncEOSDevice -### ::: anta.device.AsyncEOSDevice options: - filters: ["!^_[^_]", "!__(eq|rich_repr)__"] + filters: ["!^_[^_]", "!__(eq|rich_repr)__", "_collect"] diff --git a/docs/api/inventory.md b/docs/api/inventory.md index 5e4400c..b826b9f 100644 --- a/docs/api/inventory.md +++ b/docs/api/inventory.md @@ -5,6 +5,7 @@ --> ### ::: anta.inventory.AntaInventory + options: filters: ["!^_[^_]", "!__str__"] diff --git a/docs/api/md_reporter.md b/docs/api/md_reporter.md new file mode 100644 index 0000000..7fa1a15 --- /dev/null +++ b/docs/api/md_reporter.md @@ -0,0 +1,13 @@ +--- +anta_title: Markdown Reporter +--- +<!-- + ~ Copyright (c) 2023-2024 Arista Networks, Inc. + ~ Use of this source code is governed by the Apache License 2.0 + ~ that can be found in the LICENSE file. + --> + +::: anta.reporter.md_reporter + options: + show_root_heading: false + show_root_toc_entry: false diff --git a/docs/api/models.md b/docs/api/models.md index b0c1e91..1b360de 100644 --- a/docs/api/models.md +++ b/docs/api/models.md @@ -6,20 +6,18 @@ # Test definition -## UML Diagram + - +## ::: anta.models.AntaTest -### ::: anta.models.AntaTest options: filters: ["!^_[^_]", "!__init_subclass__", "!update_progress"] # Command definition -## UML Diagram + - -### ::: anta.models.AntaCommand +## ::: anta.models.AntaCommand !!! warning CLI commands are protected to avoid execution of critical commands such as `reload` or `write erase`. @@ -30,8 +28,6 @@ # Template definition -## UML Diagram + - - -### ::: anta.models.AntaTemplate +## ::: anta.models.AntaTemplate diff --git a/docs/api/report_manager.md b/docs/api/reporters.md index f0e3818..a72e107 100644 --- a/docs/api/report_manager.md +++ b/docs/api/reporters.md @@ -4,4 +4,7 @@ ~ that can be found in the LICENSE file. --> -### ::: anta.reporter.ReportTable +::: anta.reporter + options: + show_root_heading: false + show_root_toc_entry: false diff --git a/docs/api/result_manager.md b/docs/api/result_manager.md index 72e05aa..9fc978a 100644 --- a/docs/api/result_manager.md +++ b/docs/api/result_manager.md @@ -6,10 +6,9 @@ # Result Manager definition -## UML Diagram + - +## ::: anta.result_manager.ResultManager -### ::: anta.result_manager.ResultManager options: filters: ["!^_[^_]", "!^__len__"] diff --git a/docs/api/result_manager_models.md b/docs/api/result_manager_models.md index 096bd03..42e2648 100644 --- a/docs/api/result_manager_models.md +++ b/docs/api/result_manager_models.md @@ -6,10 +6,9 @@ # Test Result model -## UML Diagram + - +## ::: anta.result_manager.models.TestResult -### ::: anta.result_manager.models.TestResult options: filters: ["!^_[^_]", "!__str__"] diff --git a/docs/api/runner.md b/docs/api/runner.md index 27fbaa2..a2de007 100644 --- a/docs/api/runner.md +++ b/docs/api/runner.md @@ -5,5 +5,6 @@ --> ### ::: anta.runner + options: filters: ["!^_[^_]", "!__str__"] diff --git a/docs/api/tests.flow_tracking.md b/docs/api/tests.flow_tracking.md new file mode 100644 index 0000000..0df0b1d --- /dev/null +++ b/docs/api/tests.flow_tracking.md @@ -0,0 +1,20 @@ +--- +anta_title: ANTA catalog for flow tracking tests +--- +<!-- + ~ Copyright (c) 2023-2024 Arista Networks, Inc. + ~ Use of this source code is governed by the Apache License 2.0 + ~ that can be found in the LICENSE file. + --> + +::: anta.tests.flow_tracking + options: + show_root_heading: false + show_root_toc_entry: false + show_bases: false + merge_init_into_class: false + anta_hide_test_module_description: true + show_labels: true + filters: + - "!test" + - "!render" diff --git a/docs/api/tests.md b/docs/api/tests.md index b11f0ba..7dd74c1 100644 --- a/docs/api/tests.md +++ b/docs/api/tests.md @@ -1,11 +1,12 @@ +--- +anta_title: ANTA Tests Landing Page +--- <!-- ~ Copyright (c) 2023-2024 Arista Networks, Inc. ~ Use of this source code is governed by the Apache License 2.0 ~ that can be found in the LICENSE file. --> -# ANTA Tests Landing Page - This section describes all the available tests provided by the ANTA package. ## Available Tests @@ -17,7 +18,8 @@ Here are the tests that we currently provide: - [BFD](tests.bfd.md) - [Configuration](tests.configuration.md) - [Connectivity](tests.connectivity.md) -- [Field Notice](tests.field_notices.md) +- [Field Notices](tests.field_notices.md) +- [Flow Tracking](tests.flow_tracking.md) - [GreenT](tests.greent.md) - [Hardware](tests.hardware.md) - [Interfaces](tests.interfaces.md) @@ -30,6 +32,7 @@ Here are the tests that we currently provide: - [Router Path Selection](tests.path_selection.md) - [Routing Generic](tests.routing.generic.md) - [Routing BGP](tests.routing.bgp.md) +- [Routing ISIS](tests.routing.isis.md) - [Routing OSPF](tests.routing.ospf.md) - [Security](tests.security.md) - [Services](tests.services.md) diff --git a/docs/api/tests.routing.bgp.md b/docs/api/tests.routing.bgp.md index 5c54678..4537ec2 100644 --- a/docs/api/tests.routing.bgp.md +++ b/docs/api/tests.routing.bgp.md @@ -18,3 +18,4 @@ anta_title: ANTA catalog for BGP tests filters: - "!test" - "!render" + - "!^_[^_]" diff --git a/docs/api/tests.routing.isis.md b/docs/api/tests.routing.isis.md index b545f33..bf50c72 100644 --- a/docs/api/tests.routing.isis.md +++ b/docs/api/tests.routing.isis.md @@ -18,3 +18,4 @@ anta_title: ANTA catalog for IS-IS tests filters: - "!test" - "!render" + - "!^_[^_]" diff --git a/docs/api/tests.routing.ospf.md b/docs/api/tests.routing.ospf.md index a01eb50..2fd0cd4 100644 --- a/docs/api/tests.routing.ospf.md +++ b/docs/api/tests.routing.ospf.md @@ -18,3 +18,4 @@ anta_title: ANTA catalog for OSPF tests filters: - "!test" - "!render" + - "!^_[^_]" diff --git a/docs/api/types.md b/docs/api/types.md index 806ab63..a633e04 100644 --- a/docs/api/types.md +++ b/docs/api/types.md @@ -5,6 +5,7 @@ --> ### ::: anta.custom_types + options: show_if_no_docstring: true show_root_full_path: true diff --git a/docs/cli/check.md b/docs/cli/check.md index d7dea62..c230722 100644 --- a/docs/cli/check.md +++ b/docs/cli/check.md @@ -1,11 +1,12 @@ +--- +anta_title: ANTA check commands +--- <!-- ~ Copyright (c) 2023-2024 Arista Networks, Inc. ~ Use of this source code is governed by the Apache License 2.0 ~ that can be found in the LICENSE file. --> -# ANTA check commands - The ANTA check command allow to execute some checks on the ANTA input files. Only checking the catalog is currently supported. @@ -27,10 +28,12 @@ Commands: ```bash Usage: anta check catalog [OPTIONS] - Check that the catalog is valid + Check that the catalog is valid. Options: - -c, --catalog FILE Path to the test catalog YAML file [env var: - ANTA_CATALOG; required] - --help Show this message and exit. + -c, --catalog FILE Path to the test catalog file [env var: + ANTA_CATALOG; required] + --catalog-format [yaml|json] Format of the catalog file, either 'yaml' or + 'json' [env var: ANTA_CATALOG_FORMAT] + --help Show this message and exit. ``` diff --git a/docs/cli/debug.md b/docs/cli/debug.md index db5f496..4c864db 100644 --- a/docs/cli/debug.md +++ b/docs/cli/debug.md @@ -1,11 +1,12 @@ +--- +anta_title: ANTA debug commands +--- <!-- ~ Copyright (c) 2023-2024 Arista Networks, Inc. ~ Use of this source code is governed by the Apache License 2.0 ~ that can be found in the LICENSE file. --> -# ANTA debug commands - The ANTA CLI includes a set of debugging tools, making it easier to build and test ANTA content. This functionality is accessed via the `debug` subcommand and offers the following options: - Executing a command on a device from your inventory and retrieving the result. @@ -14,7 +15,7 @@ The ANTA CLI includes a set of debugging tools, making it easier to build and te These tools are especially helpful in building the tests, as they give a visual access to the output received from the eAPI. They also facilitate the extraction of output content for use in unit tests, as described in our [contribution guide](../contribution.md). !!! warning - The `debug` tools require a device from your inventory. Thus, you MUST use a valid [ANTA Inventory](../usage-inventory-catalog.md#create-an-inventory-file). + The `debug` tools require a device from your inventory. Thus, you must use a valid [ANTA Inventory](../usage-inventory-catalog.md#device-inventory). ## Executing an EOS command @@ -52,8 +53,6 @@ Options: ANTA_DISABLE_CACHE] -i, --inventory FILE Path to the inventory YAML file. [env var: ANTA_INVENTORY; required] - --tags TEXT List of tags using comma as separator: - tag1,tag2,tag3. [env var: ANTA_TAGS] --ofmt [json|text] EOS eAPI format to use. can be text or json -v, --version [1|latest] EOS eAPI version -r, --revision INTEGER eAPI command revision @@ -97,8 +96,9 @@ Usage: anta debug run-template [OPTIONS] PARAMS... Takes a list of arguments (keys followed by a value) to build a dictionary used as template parameters. - Example: ------- anta debug run-template -d leaf1a -t 'show vlan {vlan_id}' - vlan_id 1 + Example + ------- + anta debug run-template -d leaf1a -t 'show vlan {vlan_id}' vlan_id 1 Options: -u, --username TEXT Username to connect to EOS [env var: @@ -125,8 +125,6 @@ Options: ANTA_DISABLE_CACHE] -i, --inventory FILE Path to the inventory YAML file. [env var: ANTA_INVENTORY; required] - --tags TEXT List of tags using comma as separator: - tag1,tag2,tag3. [env var: ANTA_TAGS] --ofmt [json|text] EOS eAPI format to use. can be text or json -v, --version [1|latest] EOS eAPI version -r, --revision INTEGER eAPI command revision @@ -161,11 +159,12 @@ Run templated command 'show vlan {vlan_id}' with {'vlan_id': '10'} on DC1-LEAF1A 'sourceDetail': '' } ``` -!!! warning - If multiple arguments of the same key are provided, only the last argument value will be kept in the template parameters. ### Example of multiple arguments +!!! warning + If multiple arguments of the same key are provided, only the last argument value will be kept in the template parameters. + ```bash anta -log DEBUG debug run-template --template "ping {dst} source {src}" dst "8.8.8.8" src Loopback0 --device DC1-SPINE1 > {'dst': '8.8.8.8', 'src': 'Loopback0'} diff --git a/docs/cli/exec.md b/docs/cli/exec.md index 061a19e..2eb12ee 100644 --- a/docs/cli/exec.md +++ b/docs/cli/exec.md @@ -1,14 +1,16 @@ +--- +anta_title: Executing Commands on Devices +--- <!-- ~ Copyright (c) 2023-2024 Arista Networks, Inc. ~ Use of this source code is governed by the Apache License 2.0 ~ that can be found in the LICENSE file. --> -# Executing Commands on Devices - ANTA CLI provides a set of entrypoints to facilitate remote command execution on EOS devices. -### EXEC Command overview +## EXEC command overview + ```bash anta exec --help Usage: anta exec [OPTIONS] COMMAND [ARGS]... @@ -133,6 +135,7 @@ json_format: text_format: - show bfd peers ``` + ### Example ```bash diff --git a/docs/cli/get-inventory-information.md b/docs/cli/get-inventory-information.md index 1831484..ab1bebc 100644 --- a/docs/cli/get-inventory-information.md +++ b/docs/cli/get-inventory-information.md @@ -1,14 +1,61 @@ +--- +anta_title: Retrieving Inventory Information +--- <!-- ~ Copyright (c) 2023-2024 Arista Networks, Inc. ~ Use of this source code is governed by the Apache License 2.0 ~ that can be found in the LICENSE file. --> -# Retrieving Inventory Information +The ANTA CLI offers multiple commands to access data from your local inventory. + +## List devices in inventory + +This command will list all devices available in the inventory. Using the `--tags` option, you can filter this list to only include devices with specific tags (visit [this page](tag-management.md) to learn more about tags). The `--connected` option allows to display only the devices where a connection has been established. + +### Command overview -The ANTA CLI offers multiple entrypoints to access data from your local inventory. +```bash +Usage: anta get inventory [OPTIONS] -## Inventory used of examples + Show inventory loaded in ANTA. + +Options: + -u, --username TEXT Username to connect to EOS [env var: + ANTA_USERNAME; required] + -p, --password TEXT Password to connect to EOS that must be + provided. It can be prompted using '--prompt' + option. [env var: ANTA_PASSWORD] + --enable-password TEXT Password to access EOS Privileged EXEC mode. + It can be prompted using '--prompt' option. + Requires '--enable' option. [env var: + ANTA_ENABLE_PASSWORD] + --enable Some commands may require EOS Privileged EXEC + mode. This option tries to access this mode + before sending a command to the device. [env + var: ANTA_ENABLE] + -P, --prompt Prompt for passwords if they are not + provided. [env var: ANTA_PROMPT] + --timeout FLOAT Global API timeout. This value will be used + for all devices. [env var: ANTA_TIMEOUT; + default: 30.0] + --insecure Disable SSH Host Key validation. [env var: + ANTA_INSECURE] + --disable-cache Disable cache globally. [env var: + ANTA_DISABLE_CACHE] + -i, --inventory FILE Path to the inventory YAML file. [env var: + ANTA_INVENTORY; required] + --tags TEXT List of tags using comma as separator: + tag1,tag2,tag3. [env var: ANTA_TAGS] + --connected / --not-connected Display inventory after connection has been + created + --help Show this message and exit. +``` + +!!! tip + By default, `anta get inventory` only provides information that doesn't rely on a device connection. If you are interested in obtaining connection-dependent details, like the hardware model, use the `--connected` option. + +### Example Let's consider the following inventory: @@ -65,123 +112,15 @@ anta_inventory: tags: ["BL", "DC2"] ``` -## Obtaining all configured tags - -As most of ANTA's commands accommodate tag filtering, this particular command is useful for enumerating all tags configured in the inventory. Running the `anta get tags` command will return a list of all tags that have been configured in the inventory. - -### Command overview - -```bash -Usage: anta get tags [OPTIONS] - - Get list of configured tags in user inventory. - -Options: - -u, --username TEXT Username to connect to EOS [env var: ANTA_USERNAME; - required] - -p, --password TEXT Password to connect to EOS that must be provided. It - can be prompted using '--prompt' option. [env var: - ANTA_PASSWORD] - --enable-password TEXT Password to access EOS Privileged EXEC mode. It can - be prompted using '--prompt' option. Requires '-- - enable' option. [env var: ANTA_ENABLE_PASSWORD] - --enable Some commands may require EOS Privileged EXEC mode. - This option tries to access this mode before sending - a command to the device. [env var: ANTA_ENABLE] - -P, --prompt Prompt for passwords if they are not provided. [env - var: ANTA_PROMPT] - --timeout FLOAT Global API timeout. This value will be used for all - devices. [env var: ANTA_TIMEOUT; default: 30.0] - --insecure Disable SSH Host Key validation. [env var: - ANTA_INSECURE] - --disable-cache Disable cache globally. [env var: - ANTA_DISABLE_CACHE] - -i, --inventory FILE Path to the inventory YAML file. [env var: - ANTA_INVENTORY; required] - --tags TEXT List of tags using comma as separator: - tag1,tag2,tag3. [env var: ANTA_TAGS] - --help Show this message and exit. -``` - -### Example - -To get the list of all configured tags in the inventory, run the following command: - -```bash -anta get tags -Tags found: -[ - "BL", - "DC1", - "DC2", - "LEAF", - "SPINE" -] - -* note that tag all has been added by anta -``` - -!!! note - Even if you haven't explicitly configured the `all` tag in the inventory, it is automatically added. This default tag allows to execute commands on all devices in the inventory when no tag is specified. - -## List devices in inventory - -This command will list all devices available in the inventory. Using the `--tags` option, you can filter this list to only include devices with specific tags. The `--connected` option allows to display only the devices where a connection has been established. - -### Command overview - -```bash -Usage: anta get inventory [OPTIONS] - - Show inventory loaded in ANTA. - -Options: - -u, --username TEXT Username to connect to EOS [env var: - ANTA_USERNAME; required] - -p, --password TEXT Password to connect to EOS that must be - provided. It can be prompted using '--prompt' - option. [env var: ANTA_PASSWORD] - --enable-password TEXT Password to access EOS Privileged EXEC mode. - It can be prompted using '--prompt' option. - Requires '--enable' option. [env var: - ANTA_ENABLE_PASSWORD] - --enable Some commands may require EOS Privileged EXEC - mode. This option tries to access this mode - before sending a command to the device. [env - var: ANTA_ENABLE] - -P, --prompt Prompt for passwords if they are not - provided. [env var: ANTA_PROMPT] - --timeout FLOAT Global API timeout. This value will be used - for all devices. [env var: ANTA_TIMEOUT; - default: 30.0] - --insecure Disable SSH Host Key validation. [env var: - ANTA_INSECURE] - --disable-cache Disable cache globally. [env var: - ANTA_DISABLE_CACHE] - -i, --inventory FILE Path to the inventory YAML file. [env var: - ANTA_INVENTORY; required] - --tags TEXT List of tags using comma as separator: - tag1,tag2,tag3. [env var: ANTA_TAGS] - --connected / --not-connected Display inventory after connection has been - created - --help Show this message and exit. -``` - - -!!! tip - In its default mode, `anta get inventory` provides only information that doesn't rely on a device connection. If you are interested in obtaining connection-dependent details, like the hardware model, please use the `--connected` option. - -### Example - To retrieve a comprehensive list of all devices along with their details, execute the following command. It will provide all the data loaded into the ANTA inventory from your [inventory file](../usage-inventory-catalog.md). ```bash -anta get inventory --tags SPINE +$ anta get inventory --tags SPINE Current inventory content is: { 'DC1-SPINE1': AsyncEOSDevice( name='DC1-SPINE1', - tags=['SPINE', 'DC1'], + tags={'DC1-SPINE1', 'DC1', 'SPINE'}, hw_model=None, is_online=False, established=False, @@ -189,13 +128,12 @@ Current inventory content is: host='172.20.20.101', eapi_port=443, username='arista', - enable=True, - enable_password='arista', + enable=False, insecure=False ), 'DC1-SPINE2': AsyncEOSDevice( name='DC1-SPINE2', - tags=['SPINE', 'DC1'], + tags={'DC1', 'SPINE', 'DC1-SPINE2'}, hw_model=None, is_online=False, established=False, @@ -203,12 +141,12 @@ Current inventory content is: host='172.20.20.102', eapi_port=443, username='arista', - enable=True, + enable=False, insecure=False ), 'DC2-SPINE1': AsyncEOSDevice( name='DC2-SPINE1', - tags=['SPINE', 'DC2'], + tags={'DC2', 'DC2-SPINE1', 'SPINE'}, hw_model=None, is_online=False, established=False, @@ -216,12 +154,12 @@ Current inventory content is: host='172.20.20.201', eapi_port=443, username='arista', - enable=True, + enable=False, insecure=False ), 'DC2-SPINE2': AsyncEOSDevice( name='DC2-SPINE2', - tags=['SPINE', 'DC2'], + tags={'DC2', 'DC2-SPINE2', 'SPINE'}, hw_model=None, is_online=False, established=False, @@ -229,7 +167,7 @@ Current inventory content is: host='172.20.20.202', eapi_port=443, username='arista', - enable=True, + enable=False, insecure=False ) } diff --git a/docs/cli/inv-from-ansible.md b/docs/cli/inv-from-ansible.md index b2672e2..6bbaca9 100644 --- a/docs/cli/inv-from-ansible.md +++ b/docs/cli/inv-from-ansible.md @@ -1,14 +1,15 @@ +--- +anta_title: Create an Inventory from Ansible inventory +--- <!-- ~ Copyright (c) 2023-2024 Arista Networks, Inc. ~ Use of this source code is governed by the Apache License 2.0 ~ that can be found in the LICENSE file. --> -# Create an Inventory from Ansible inventory - In large setups, it might be beneficial to construct your inventory based on your Ansible inventory. The `from-ansible` entrypoint of the `get` command enables the user to create an ANTA inventory from Ansible. -### Command overview +## Command overview ```bash $ anta get from-ansible --help @@ -32,9 +33,8 @@ Options: !!! warning - `anta get from-ansible` does not support inline vaulted variables, comment them out to generate your inventory. - If the vaulted variable is necessary to build the inventory (e.g. `ansible_host`), it needs to be unvaulted for `from-ansible` command to work." - + `anta get from-ansible` does not support inline vaulted variables, comment them out to generate your inventory. + If the vaulted variable is necessary to build the inventory (e.g. `ansible_host`), it needs to be unvaulted for `from-ansible` command to work." The output is an inventory where the name of the container is added as a tag for each host: @@ -54,8 +54,7 @@ anta_inventory: By default, if user does not provide `--output` file, anta will save output to configured anta inventory (`anta --inventory`). If the output file has content, anta will ask user to overwrite when running in interactive console. This mechanism can be controlled by triggers in case of CI usage: `--overwrite` to force anta to overwrite file. If not set, anta will exit - -### Command output +## Command output `host` value is coming from the `ansible_host` key in your inventory while `name` is the name you defined for your host. Below is an ansible inventory example used to generate previous inventory: diff --git a/docs/cli/inv-from-cvp.md b/docs/cli/inv-from-cvp.md index a37af62..9717870 100644 --- a/docs/cli/inv-from-cvp.md +++ b/docs/cli/inv-from-cvp.md @@ -1,17 +1,18 @@ +--- +anta_title: Create an Inventory from CloudVision +--- <!-- ~ Copyright (c) 2023-2024 Arista Networks, Inc. ~ Use of this source code is governed by the Apache License 2.0 ~ that can be found in the LICENSE file. --> -# Create an Inventory from CloudVision - In large setups, it might be beneficial to construct your inventory based on CloudVision. The `from-cvp` entrypoint of the `get` command enables the user to create an ANTA inventory from CloudVision. !!! info The current implementation only works with on-premises CloudVision instances, not with CloudVision as a Service (CVaaS). -### Command overview +## Command overview ```bash Usage: anta get from-cvp [OPTIONS] @@ -54,7 +55,7 @@ anta_inventory: !!! warning The current implementation only considers devices directly attached to a specific container when using the `--cvp-container` option. -### Creating an inventory from multiple containers +## Creating an inventory from multiple containers If you need to create an inventory from multiple containers, you can use a bash command and then manually concatenate files to create a single inventory file: diff --git a/docs/cli/nrfu.md b/docs/cli/nrfu.md index 90b4a40..0f2b425 100644 --- a/docs/cli/nrfu.md +++ b/docs/cli/nrfu.md @@ -1,17 +1,20 @@ +--- +anta_title: Execute Network Readiness For Use (NRFU) Testing +--- <!-- ~ Copyright (c) 2023-2024 Arista Networks, Inc. ~ Use of this source code is governed by the Apache License 2.0 ~ that can be found in the LICENSE file. --> -# Execute Network Readiness For Use (NRFU) Testing - ANTA provides a set of commands for performing NRFU tests on devices. These commands are under the `anta nrfu` namespace and offer multiple output format options: -- [Text view](#performing-nrfu-with-text-rendering) -- [Table view](#performing-nrfu-with-table-rendering) -- [JSON view](#performing-nrfu-with-json-rendering) -- [Custom template view](#performing-nrfu-with-custom-reports) +- [Text report](#performing-nrfu-with-text-rendering) +- [Table report](#performing-nrfu-with-table-rendering) +- [JSON report](#performing-nrfu-with-json-rendering) +- [Custom template report](#performing-nrfu-with-custom-reports) +- [CSV report](#performing-nrfu-and-saving-results-in-a-csv-file) +- [Markdown report](#performing-nrfu-and-saving-results-in-a-markdown-file) ## NRFU Command overview @@ -28,16 +31,7 @@ All commands under the `anta nrfu` namespace require a catalog yaml file specifi ### Tag management -The `--tags` option can be used to target specific devices in your inventory and run only tests configured with this specific tags from your catalog. The default tag is set to `all` and is implicit. Expected behaviour is provided below: - -| Command | Description | -| ------- | ----------- | -| `none` | Run all tests on all devices according `tag` definition in your inventory and test catalog. And tests with no tag are executed on all devices| -| `--tags leaf` | Run all tests marked with `leaf` tag on all devices configured with `leaf` tag.<br/> All other tags are ignored | -| `--tags leaf,spine` | Run all tests marked with `leaf` tag on all devices configured with `leaf` tag.<br/>Run all tests marked with `spine` tag on all devices configured with `spine` tag.<br/> All other tags are ignored | - -!!! info - [More examples](tag-management.md) available on this dedicated page. +The `--tags` option can be used to target specific devices in your inventory and run only tests configured with this specific tags from your catalog. Refer to the [dedicated page](tag-management.md) for more information. ### Device and test filtering @@ -45,7 +39,7 @@ Options `--device` and `--test` can be used to target one or multiple devices an ### Hide results -Option `--hide` can be used to hide test results in the output based on their status. The option can be repeated. Example: `anta nrfu --hide error --hide skipped`. +Option `--hide` can be used to hide test results in the output or report file based on their status. The option can be repeated. Example: `anta nrfu --hide error --hide skipped`. ## Performing NRFU with text rendering @@ -67,6 +61,7 @@ Options: ```bash anta nrfu --device DC1-LEAF1A text ``` + { loading=lazy width="1600" } ## Performing NRFU with table rendering @@ -92,6 +87,7 @@ The `--group-by` option show a summarized view of the test results per host or p ```bash anta nrfu --tags LEAF table ``` + { loading=lazy width="1600" } For larger setups, you can also group the results by host or test to get a summarized view: @@ -99,11 +95,13 @@ For larger setups, you can also group the results by host or test to get a summa ```bash anta nrfu table --group-by device ``` + { loading=lazy width="1600" } ```bash anta nrfu table --group-by test ``` + { loading=lazy width="1600" } To get more specific information, it is possible to filter on a single device or a single test: @@ -111,16 +109,18 @@ To get more specific information, it is possible to filter on a single device or ```bash anta nrfu --device spine1 table ``` + { loading=lazy width="1600" } ```bash anta nrfu --test VerifyZeroTouch table ``` + { loading=lazy width="1600" } ## Performing NRFU with JSON rendering -The JSON rendering command in NRFU testing is useful in generating a JSON output that can subsequently be passed on to another tool for reporting purposes. +The JSON rendering command in NRFU testing will generate an output of all test results in JSON format. ### Command overview @@ -131,20 +131,66 @@ Usage: anta nrfu json [OPTIONS] ANTA command to check network state with JSON result. Options: - -o, --output FILE Path to save report as a file [env var: + -o, --output FILE Path to save report as a JSON file [env var: ANTA_NRFU_JSON_OUTPUT] --help Show this message and exit. ``` -The `--output` option allows you to save the JSON report as a file. +The `--output` option allows you to save the JSON report as a file. If specified, no output will be displayed in the terminal. This is useful for further processing or integration with other tools. ### Example ```bash anta nrfu --tags LEAF json ``` + { loading=lazy width="1600" } +## Performing NRFU and saving results in a CSV file + +The `csv` command in NRFU testing is useful for generating a CSV file with all tests result. This file can be easily analyzed and filtered by operator for reporting purposes. + +### Command overview + +```bash +anta nrfu csv --help +Usage: anta nrfu csv [OPTIONS] + + ANTA command to check network states with CSV result. + +Options: + --csv-output FILE Path to save report as a CSV file [env var: + ANTA_NRFU_CSV_CSV_OUTPUT] + --help Show this message and exit. +``` + +### Example + +{ loading=lazy width="1600" } + +## Performing NRFU and saving results in a Markdown file + +The `md-report` command in NRFU testing generates a comprehensive Markdown report containing various sections, including detailed statistics for devices and test categories. + +### Command overview + +```bash +anta nrfu md-report --help + +Usage: anta nrfu md-report [OPTIONS] + + ANTA command to check network state with Markdown report. + +Options: + --md-output FILE Path to save the report as a Markdown file [env var: + ANTA_NRFU_MD_REPORT_MD_OUTPUT; required] + --help Show this message and exit. +``` + +### Example + +{ loading=lazy width="1600" } + ## Performing NRFU with custom reports ANTA offers a CLI option for creating custom reports. This leverages the Jinja2 template system, allowing you to tailor reports to your specific needs. @@ -164,6 +210,7 @@ Options: ANTA_NRFU_TPL_REPORT_OUTPUT] --help Show this message and exit. ``` + The `--template` option is used to specify the Jinja2 template file for generating the custom report. The `--output` option allows you to choose the path where the final report will be saved. @@ -173,6 +220,7 @@ The `--output` option allows you to choose the path where the final report will ```bash anta nrfu --tags LEAF tpl-report --template ./custom_template.j2 ``` + { loading=lazy width="1600" } The template `./custom_template.j2` is a simple Jinja2 template: @@ -183,7 +231,7 @@ The template `./custom_template.j2` is a simple Jinja2 template: {% endfor %} ``` -The Jinja2 template has access to all `TestResult` elements and their values, as described in this [documentation](../api/result_manager_models.md#testresult-entry). +The Jinja2 template has access to all `TestResult` elements and their values, as described in this [documentation](../api/result_manager_models.md#anta.result_manager.models.TestResult). You can also save the report result to a file using the `--output` option: diff --git a/docs/cli/overview.md b/docs/cli/overview.md index 107976a..f1247b7 100644 --- a/docs/cli/overview.md +++ b/docs/cli/overview.md @@ -1,11 +1,12 @@ +--- +anta_title: Overview of ANTA's Command-Line Interface (CLI) +--- <!-- ~ Copyright (c) 2023-2024 Arista Networks, Inc. ~ Use of this source code is governed by the Apache License 2.0 ~ that can be found in the LICENSE file. --> -# Overview of ANTA's Command-Line Interface (CLI) - ANTA provides a powerful Command-Line Interface (CLI) to perform a wide range of operations. This document provides a comprehensive overview of ANTA CLI usage and its commands. ANTA can also be used as a Python library, allowing you to build your own tools based on it. Visit this [page](../advanced_usages/as-python-lib.md) for more details. @@ -35,7 +36,7 @@ To set them as environment variables: export ANTA_USERNAME=admin export ANTA_PASSWORD=arista123 export ANTA_INVENTORY=inventory.yml -export ANTA_INVENTORY=tests.yml +export ANTA_CATALOG=tests.yml ``` Then, run the CLI without options: diff --git a/docs/cli/tag-management.md b/docs/cli/tag-management.md index 8c043d7..ad5ccf3 100644 --- a/docs/cli/tag-management.md +++ b/docs/cli/tag-management.md @@ -4,162 +4,250 @@ ~ that can be found in the LICENSE file. --> -# Tag management +ANTA commands can be used with a `--tags` option. This option **filters the inventory** with the specified tag(s) when running the command. -## Overview +Tags can also be used to **restrict a specific test** to a set of devices when using `anta nrfu`. -Some of the ANTA commands like `anta nrfu` command come with a `--tags` option. +## Defining tags -For `nrfu`, this allows users to specify a set of tests, marked with a given tag, to be run on devices marked with the same tag. For instance, you can run tests dedicated to leaf devices on your leaf devices only and not on other devices. +### Device tags -Tags are string defined by the user and can be anything considered as a string by Python. A [default one](#default-tags) is present for all tests and devices. +Device tags can be defined in the inventory: -The next table provides a short summary of the scope of tags using CLI +```yaml +anta_inventory: + hosts: + - name: leaf1 + host: leaf1.anta.arista.com + tags: ["leaf"] + - name: leaf2 + host: leaf2.anta.arista.com + tags: ["leaf"] + - name: spine1 + host: spine1.anta.arista.com + tags: ["spine"] +``` + +Each device also has its own name automatically added as a tag: + +```bash +$ anta get inventory +Current inventory content is: +{ + 'leaf1': AsyncEOSDevice( + name='leaf1', + tags={'leaf', 'leaf1'}, <-- + [...] + host='leaf1.anta.arista.com', + [...] + ), + 'leaf2': AsyncEOSDevice( + name='leaf2', + tags={'leaf', 'leaf2'}, <-- + [...] + host='leaf2.anta.arista.com', + [...] + ), + 'spine1': AsyncEOSDevice( + name='spine1', + tags={'spine1', 'spine'}, <-- + [...] + host='spine1.anta.arista.com', + [...] + ) +} +``` + +### Test tags + +Tags can be defined in the test catalog to restrict tests to tagged devices: + +```yaml +anta.tests.system: + - VerifyUptime: + minimum: 10 + filters: + tags: ['spine'] + - VerifyUptime: + minimum: 9 + filters: + tags: ['leaf'] + - VerifyReloadCause: + filters: + tags: ['spine', 'leaf'] + - VerifyCoredump: + - VerifyAgentLogs: + - VerifyCPUUtilization: + - VerifyMemoryUtilization: + - VerifyFileSystemUtilization: + - VerifyNTP: + +anta.tests.mlag: + - VerifyMlagStatus: + filters: + tags: ['leaf'] + +anta.tests.interfaces: + - VerifyL3MTU: + mtu: 1500 + filters: + tags: ['spine'] +``` + +> A tag used to filter a test can also be a device name + +!!! tip "Use different input values for a specific test" + Leverage tags to define different input values for a specific test. See the `VerifyUptime` example above. + +## Using tags | Command | Description | | ------- | ----------- | -| `none` | Run all tests on all devices according `tag` definition in your inventory and test catalog. And tests with no tag are executed on all devices| -| `--tags leaf` | Run all tests marked with `leaf` tag on all devices configured with `leaf` tag.<br/> All other tags are ignored | -| `--tags leaf,spine` | Run all tests marked with `leaf` tag on all devices configured with `leaf` tag.<br/>Run all tests marked with `spine` tag on all devices configured with `spine` tag.<br/> All other tags are ignored | - -## Inventory and Catalog for tests - -All commands in this page are based on the following inventory and test catalog. - -=== "Inventory" - - ```yaml - --- - anta_inventory: - hosts: - - host: 192.168.0.10 - name: spine01 - tags: ['fabric', 'spine'] - - host: 192.168.0.11 - name: spine02 - tags: ['fabric', 'spine'] - - host: 192.168.0.12 - name: leaf01 - tags: ['fabric', 'leaf'] - - host: 192.168.0.13 - name: leaf02 - tags: ['fabric', 'leaf'] - - host: 192.168.0.14 - name: leaf03 - tags: ['fabric', 'leaf'] - - host: 192.168.0.15 - name: leaf04 - tags: ['fabric', 'leaf' - ``` - -=== "Test Catalog" - - ```yaml - anta.tests.system: - - VerifyUptime: - minimum: 10 - filters: - tags: ['fabric'] - - VerifyReloadCause: - tags: ['leaf', spine'] - - VerifyCoredump: - - VerifyAgentLogs: - - VerifyCPUUtilization: - filters: - tags: ['spine', 'leaf'] - - VerifyMemoryUtilization: - - VerifyFileSystemUtilization: - - VerifyNTP: - - anta.tests.mlag: - - VerifyMlagStatus: - - - anta.tests.interfaces: - - VerifyL3MTU: - mtu: 1500 - filters: - tags: ['demo'] - ``` - -## Default tags - -By default, ANTA uses a default tag for both devices and tests. This default tag is `all` and it can be explicit if you want to make it visible in your inventory and also implicit since the framework injects this tag if it is not defined. - -So this command will run all tests from your catalog on all devices. With a mapping for `tags` defined in your inventory and catalog. If no `tags` configured, then tests are executed against all devices. +| No `--tags` option | Run all tests on all devices according to the `tag` definitions in your inventory and test catalog.<br/> Tests without tags are executed on all devices. | +| `--tags leaf` | Run all tests marked with the `leaf` tag on all devices configured with the `leaf` tag.<br/> All other tests are ignored. | +| `--tags leaf,spine` | Run all tests marked with the `leaf` tag on all devices configured with the `leaf` tag.<br/>Run all tests marked with the `spine` tag on all devices configured with the `spine` tag.<br/> All other tests are ignored. | + +### Examples + +The following examples use the inventory and test catalog defined above. + +#### No `--tags` option + +Tests without tags are run on all devices. +Tests with tags will only run on devices with matching tags. ```bash -$ anta nrfu -c .personal/catalog-class.yml table --group-by device +$ anta nrfu table --group-by device +╭────────────────────── Settings ──────────────────────╮ +│ - ANTA Inventory contains 3 devices (AsyncEOSDevice) │ +│ - Tests catalog contains 11 tests │ +╰──────────────────────────────────────────────────────╯ + +--- ANTA NRFU Run Information --- +Number of devices: 3 (3 established) +Total number of selected tests: 27 +--------------------------------- + Summary per device +┏━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ +┃ Device ┃ # of success ┃ # of skipped ┃ # of failure ┃ # of errors ┃ List of failed or error test cases ┃ +┡━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ +│ leaf1 │ 9 │ 0 │ 0 │ 0 │ │ +├────────┼──────────────┼──────────────┼──────────────┼─────────────┼────────────────────────────────────┤ +│ leaf2 │ 7 │ 1 │ 1 │ 0 │ VerifyAgentLogs │ +├────────┼──────────────┼──────────────┼──────────────┼─────────────┼────────────────────────────────────┤ +│ spine1 │ 9 │ 0 │ 0 │ 0 │ │ +└────────┴──────────────┴──────────────┴──────────────┴─────────────┴────────────────────────────────────┘ +``` + +#### Single tag +With a tag specified, only tests matching this tag will be run on matching devices. + +```bash +$ anta nrfu --tags leaf text ╭────────────────────── Settings ──────────────────────╮ -│ Running ANTA tests: │ -│ - ANTA Inventory contains 6 devices (AsyncEOSDevice) │ -│ - Tests catalog contains 10 tests │ +│ - ANTA Inventory contains 3 devices (AsyncEOSDevice) │ +│ - Tests catalog contains 11 tests │ ╰──────────────────────────────────────────────────────╯ -┏━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ -┃ Device ┃ # of success ┃ # of skipped ┃ # of failure ┃ # of errors ┃ List of failed or error test cases ┃ -┡━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ -│ spine01 │ 5 │ 1 │ 1 │ 0 │ ['VerifyCPUUtilization'] │ -│ spine02 │ 5 │ 1 │ 1 │ 0 │ ['VerifyCPUUtilization'] │ -│ leaf01 │ 6 │ 0 │ 1 │ 0 │ ['VerifyCPUUtilization'] │ -│ leaf02 │ 6 │ 0 │ 1 │ 0 │ ['VerifyCPUUtilization'] │ -│ leaf03 │ 6 │ 0 │ 1 │ 0 │ ['VerifyCPUUtilization'] │ -│ leaf04 │ 6 │ 0 │ 1 │ 0 │ ['VerifyCPUUtilization'] │ -└─────────┴──────────────┴──────────────┴──────────────┴─────────────┴────────────────────────────────────┘ +--- ANTA NRFU Run Information --- +Number of devices: 3 (2 established) +Total number of selected tests: 6 +--------------------------------- + +leaf1 :: VerifyReloadCause :: SUCCESS +leaf1 :: VerifyUptime :: SUCCESS +leaf1 :: VerifyMlagStatus :: SUCCESS +leaf2 :: VerifyReloadCause :: SUCCESS +leaf2 :: VerifyUptime :: SUCCESS +leaf2 :: VerifyMlagStatus :: SKIPPED (MLAG is disabled) ``` -## Use a single tag in CLI +In this case, only `leaf` devices defined in the inventory are used to run tests marked with the `leaf` in the test catalog. -The most used approach is to use a single tag in your CLI to filter tests & devices configured with this one. +#### Multiple tags -In such scenario, ANTA will run tests marked with `$tag` only on devices marked with `$tag`. All other tests and devices will be ignored +It is possible to use multiple tags using the `--tags tag1,tag2` syntax. ```bash -$ anta nrfu -c .personal/catalog-class.yml --tags leaf text +$ anta nrfu --tags leaf,spine text ╭────────────────────── Settings ──────────────────────╮ -│ Running ANTA tests: │ -│ - ANTA Inventory contains 6 devices (AsyncEOSDevice) │ -│ - Tests catalog contains 10 tests │ +│ - ANTA Inventory contains 3 devices (AsyncEOSDevice) │ +│ - Tests catalog contains 11 tests │ ╰──────────────────────────────────────────────────────╯ -leaf01 :: VerifyUptime :: SUCCESS -leaf01 :: VerifyReloadCause :: SUCCESS -leaf01 :: VerifyCPUUtilization :: SUCCESS -leaf02 :: VerifyUptime :: SUCCESS -leaf02 :: VerifyReloadCause :: SUCCESS -leaf02 :: VerifyCPUUtilization :: SUCCESS -leaf03 :: VerifyUptime :: SUCCESS -leaf03 :: VerifyReloadCause :: SUCCESS -leaf03 :: VerifyCPUUtilization :: SUCCESS -leaf04 :: VerifyUptime :: SUCCESS -leaf04 :: VerifyReloadCause :: SUCCESS -leaf04 :: VerifyCPUUtilization :: SUCCESS +--- ANTA NRFU Run Information --- +Number of devices: 3 (3 established) +Total number of selected tests: 15 +--------------------------------- + +leaf1 :: VerifyReloadCause :: SUCCESS +leaf1 :: VerifyMlagStatus :: SUCCESS +leaf1 :: VerifyUptime :: SUCCESS +leaf1 :: VerifyL3MTU :: SUCCESS +leaf1 :: VerifyUptime :: SUCCESS +leaf2 :: VerifyReloadCause :: SUCCESS +leaf2 :: VerifyMlagStatus :: SKIPPED (MLAG is disabled) +leaf2 :: VerifyUptime :: SUCCESS +leaf2 :: VerifyL3MTU :: SUCCESS +leaf2 :: VerifyUptime :: SUCCESS +spine1 :: VerifyReloadCause :: SUCCESS +spine1 :: VerifyMlagStatus :: SUCCESS +spine1 :: VerifyUptime :: SUCCESS +spine1 :: VerifyL3MTU :: SUCCESS +spine1 :: VerifyUptime :: SUCCESS ``` -In this case, only `leaf` devices defined in your [inventory](#inventory-and-catalog-for-tests) are used to run tests marked with `leaf` in your [test catalog](#inventory-and-catalog-for-tests) +## Obtaining all configured tags + +As most ANTA commands accommodate tag filtering, this command is useful for enumerating all tags configured in the inventory. Running the `anta get tags` command will return a list of all tags configured in the inventory. -## Use multiple tags in CLI +### Command overview + +```bash +Usage: anta get tags [OPTIONS] + + Get list of configured tags in user inventory. + +Options: + -u, --username TEXT Username to connect to EOS [env var: ANTA_USERNAME; + required] + -p, --password TEXT Password to connect to EOS that must be provided. It + can be prompted using '--prompt' option. [env var: + ANTA_PASSWORD] + --enable-password TEXT Password to access EOS Privileged EXEC mode. It can + be prompted using '--prompt' option. Requires '-- + enable' option. [env var: ANTA_ENABLE_PASSWORD] + --enable Some commands may require EOS Privileged EXEC mode. + This option tries to access this mode before sending + a command to the device. [env var: ANTA_ENABLE] + -P, --prompt Prompt for passwords if they are not provided. [env + var: ANTA_PROMPT] + --timeout FLOAT Global API timeout. This value will be used for all + devices. [env var: ANTA_TIMEOUT; default: 30.0] + --insecure Disable SSH Host Key validation. [env var: + ANTA_INSECURE] + --disable-cache Disable cache globally. [env var: + ANTA_DISABLE_CACHE] + -i, --inventory FILE Path to the inventory YAML file. [env var: + ANTA_INVENTORY; required] + --tags TEXT List of tags using comma as separator: + tag1,tag2,tag3. [env var: ANTA_TAGS] + --help Show this message and exit. +``` -A more advanced usage of the tag feature is to list multiple tags in your CLI using `--tags $tag1,$tag2` syntax. +### Example -In such scenario, all devices marked with `$tag1` will be selected and ANTA will run tests with `$tag1`, then devices with `$tag2` will be selected and will be tested with tests marked with `$tag2` +To get the list of all configured tags in the inventory, run the following command: ```bash -anta nrfu -c .personal/catalog-class.yml --tags leaf,fabric text - -spine01 :: VerifyUptime :: SUCCESS -spine02 :: VerifyUptime :: SUCCESS -leaf01 :: VerifyUptime :: SUCCESS -leaf01 :: VerifyReloadCause :: SUCCESS -leaf01 :: VerifyCPUUtilization :: SUCCESS -leaf02 :: VerifyUptime :: SUCCESS -leaf02 :: VerifyReloadCause :: SUCCESS -leaf02 :: VerifyCPUUtilization :: SUCCESS -leaf03 :: VerifyUptime :: SUCCESS -leaf03 :: VerifyReloadCause :: SUCCESS -leaf03 :: VerifyCPUUtilization :: SUCCESS -leaf04 :: VerifyUptime :: SUCCESS -leaf04 :: VerifyReloadCause :: SUCCESS -leaf04 :: VerifyCPUUtilization :: SUCCESS +$ anta get tags +Tags found: +[ + "leaf", + "leaf1", + "leaf2", + "spine", + "spine1" +] ``` diff --git a/docs/contribution.md b/docs/contribution.md index ac5d026..88f09c1 100644 --- a/docs/contribution.md +++ b/docs/contribution.md @@ -1,11 +1,12 @@ +--- +anta_title: How to contribute to ANTA +--- <!-- ~ Copyright (c) 2023-2024 Arista Networks, Inc. ~ Use of this source code is governed by the Apache License 2.0 ~ that can be found in the LICENSE file. --> -# How to contribute to ANTA - Contribution model is based on a fork-model. Don't push to aristanetworks/anta directly. Always do a branch in your forked repository and create a PR. To help development, open your PR as soon as possible even in draft mode. It helps other to know on what you are working on and avoid duplicate PRs. @@ -28,7 +29,7 @@ $ pip install -e .[dev,cli] $ pip list -e Package Version Editable project location ------- ------- ------------------------- -anta 1.0.0 /mnt/lab/projects/anta +anta 1.1.0 /mnt/lab/projects/anta ``` Then, [`tox`](https://tox.wiki/) is configured with few environments to run CI locally: @@ -39,10 +40,10 @@ default environments: clean -> Erase previous coverage reports lint -> Check the code style type -> Check typing -py38 -> Run pytest with py38 py39 -> Run pytest with py39 py310 -> Run pytest with py310 py311 -> Run pytest with py311 +py312 -> Run pytest with py312 report -> Generate coverage report ``` @@ -51,21 +52,22 @@ report -> Generate coverage report ```bash tox -e lint [...] -lint: commands[0]> black --check --diff --color . -All done! ✨ 🍰 ✨ -104 files would be left unchanged. -lint: commands[1]> isort --check --diff --color . -Skipped 7 files -lint: commands[2]> flake8 --max-line-length=165 --config=/dev/null anta -lint: commands[3]> flake8 --max-line-length=165 --config=/dev/null tests -lint: commands[4]> pylint anta +lint: commands[0]> ruff check . +All checks passed! +lint: commands[1]> ruff format . --check +158 files already formatted +lint: commands[2]> pylint anta -------------------------------------------------------------------- Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00) -.pkg: _exit> python /Users/guillaumemulocher/.pyenv/versions/3.8.13/envs/anta/lib/python3.8/site-packages/pyproject_api/_backend.py True setuptools.build_meta - lint: OK (19.26=setup[5.83]+cmd[1.50,0.76,1.19,1.20,8.77] seconds) - congratulations :) (19.56 seconds) +lint: commands[3]> pylint tests + +-------------------------------------------------------------------- +Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00) + + lint: OK (22.69=setup[2.19]+cmd[0.02,0.02,9.71,10.75] seconds) + congratulations :) (22.72 seconds) ``` ### Code Typing @@ -75,10 +77,11 @@ tox -e type [...] type: commands[0]> mypy --config-file=pyproject.toml anta -Success: no issues found in 52 source files -.pkg: _exit> python /Users/guillaumemulocher/.pyenv/versions/3.8.13/envs/anta/lib/python3.8/site-packages/pyproject_api/_backend.py True setuptools.build_meta - type: OK (46.66=setup[24.20]+cmd[22.46] seconds) - congratulations :) (47.01 seconds) +Success: no issues found in 68 source files +type: commands[1]> mypy --config-file=pyproject.toml tests +Success: no issues found in 82 source files + type: OK (31.15=setup[14.62]+cmd[6.05,10.48] seconds) + congratulations :) (31.18 seconds) ``` > NOTE: Typing is configured quite strictly, do not hesitate to reach out if you have any questions, struggles, nightmares. @@ -92,7 +95,7 @@ All submodule should have its own pytest section under `tests/units/anta_tests/< ### How to write a unit test for an AntaTest subclass The Python modules in the `tests/units/anta_tests` folder define test parameters for AntaTest subclasses unit tests. -A generic test function is written for all unit tests in `tests.lib.anta` module. +A generic test function is written for all unit tests in `tests.units.anta_tests` module. The `pytest_generate_tests` function definition in `conftest.py` is called during test collection. @@ -102,21 +105,20 @@ See https://docs.pytest.org/en/7.3.x/how-to/parametrize.html#basic-pytest-genera The `DATA` structure is a list of dictionaries used to parametrize the test. The list elements have the following keys: - - `name` (str): Test name as displayed by Pytest. - - `test` (AntaTest): An AntaTest subclass imported in the test module - e.g. VerifyUptime. - - `eos_data` (list[dict]): List of data mocking EOS returned data to be passed to the test. - - `inputs` (dict): Dictionary to instantiate the `test` inputs as defined in the class from `test`. - - `expected` (dict): Expected test result structure, a dictionary containing a key +- `name` (str): Test name as displayed by Pytest. +- `test` (AntaTest): An AntaTest subclass imported in the test module - e.g. VerifyUptime. +- `eos_data` (list[dict]): List of data mocking EOS returned data to be passed to the test. +- `inputs` (dict): Dictionary to instantiate the `test` inputs as defined in the class from `test`. +- `expected` (dict): Expected test result structure, a dictionary containing a key `result` containing one of the allowed status (`Literal['success', 'failure', 'unset', 'skipped', 'error']`) and optionally a key `messages` which is a list(str) and each message is expected to be a substring of one of the actual messages in the TestResult object. - In order for your unit tests to be correctly collected, you need to import the generic test function even if not used in the Python module. Test example for `anta.tests.system.VerifyUptime` AntaTest. ``` python # Import the generic test function -from tests.lib.anta import test # noqa: F401 +from tests.units.anta_tests import test # Import your AntaTest from anta.tests.system import VerifyUptime @@ -157,20 +159,21 @@ pre-commit install When running a commit or a pre-commit check: ``` bash -❯ echo "import foobaz" > test.py && git add test.py ❯ pre-commit -pylint...................................................................Failed -- hook id: pylint -- exit code: 22 - -************* Module test -test.py:1:0: C0114: Missing module docstring (missing-module-docstring) -test.py:1:0: E0401: Unable to import 'foobaz' (import-error) -test.py:1:0: W0611: Unused import foobaz (unused-import) +trim trailing whitespace.................................................Passed +fix end of files.........................................................Passed +check for added large files..............................................Passed +check for merge conflicts................................................Passed +Check and insert license on Python files.................................Passed +Check and insert license on Markdown files...............................Passed +Run Ruff linter..........................................................Passed +Run Ruff formatter.......................................................Passed +Check code style with pylint.............................................Passed +Checks for common misspellings in text files.............................Passed +Check typing with mypy...................................................Passed +Check Markdown files style...............................................Passed ``` -> NOTE: It could happen that pre-commit and tox disagree on something, in that case please open an issue on Github so we can take a look.. It is most probably wrong configuration on our side. - ## Configure MYPYPATH In some cases, mypy can complain about not having `MYPYPATH` configured in your shell. It is especially the case when you update both an anta test and its unit test. So you can configure this environment variable with: @@ -229,4 +232,4 @@ muffet -c 2 --color=always http://127.0.0.1:8000 -e fonts.gstatic.com -b 8192 ## Continuous Integration -GitHub actions is used to test git pushes and pull requests. The workflows are defined in this [directory](https://github.com/aristanetworks/anta/tree/main/.github/workflows). We can view the results [here](https://github.com/aristanetworks/anta/actions). +GitHub actions is used to test git pushes and pull requests. The workflows are defined in this [directory](https://github.com/aristanetworks/anta/tree/main/.github/workflows). The results can be viewed [here](https://github.com/aristanetworks/anta/actions). diff --git a/docs/faq.md b/docs/faq.md index a699c84..7a58663 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -1,5 +1,6 @@ --- -toc_depth: 2 +toc_depth: 3 +anta_title: Frequently Asked Questions (FAQ) --- <!-- ~ Copyright (c) 2023-2024 Arista Networks, Inc. @@ -7,7 +8,7 @@ toc_depth: 2 ~ that can be found in the LICENSE file. --> <style> - .md-typeset h2 { + .md-typeset h3 { visibility: hidden; font-size: 0em; height: 0em; @@ -21,9 +22,8 @@ toc_depth: 2 } </style> -# Frequently Asked Questions (FAQ) - ## A local OS error occurred while connecting to a device + ???+ faq "A local OS error occurred while connecting to a device" When running ANTA, you can receive `A local OS error occurred while connecting to <device>` errors. The underlying [`OSError`](https://docs.python.org/3/library/exceptions.html#OSError) exception can have various reasons: `[Errno 24] Too many open files` or `[Errno 16] Device or resource busy`. @@ -38,13 +38,13 @@ toc_depth: 2 You can get the current hard limit for a user using the command `ulimit -n -H` while logged in. Create the file `/etc/security/limits.d/10-anta.conf` with the following content: ``` - <user> hard nofile <value> + <user> hard nofile <value> ``` The `user` is the one with which the ANTA process is started. The `value` is the new hard limit. The maximum value depends on the system. A hard limit of 16384 should be sufficient for ANTA to run in most high scale scenarios. After creating this file, log out the current session and log in again. - ## `Timeout` error in the logs + ???+ faq "`Timeout` error in the logs" When running ANTA, you can receive `<Foo>Timeout` errors in the logs (could be ReadTimeout, WriteTimeout, ConnectTimeout or PoolTimeout). More details on the timeouts of the underlying library are available here: https://www.python-httpx.org/advanced/timeouts. @@ -63,8 +63,8 @@ toc_depth: 2 The timeout is increased to 50s to allow ANTA to wait for API calls a little longer. ## `ImportError` related to `urllib3` -???+ faq "`ImportError` related to `urllib3` when running ANTA" +???+ faq "`ImportError` related to `urllib3` when running ANTA" When running the `anta --help` command, some users might encounter the following error: @@ -90,9 +90,9 @@ toc_depth: 2 As per the [urllib3 v2 migration guide](https://urllib3.readthedocs.io/en/latest/v2-migration-guide.html), the root cause of this error is an incompatibility with older OpenSSL versions. For example, users on RHEL7 might consider upgrading to RHEL8, which supports the required OpenSSL version. -##`AttributeError: module 'lib' has no attribute 'OpenSSL_add_all_algorithms'` -???+ faq "`AttributeError: module 'lib' has no attribute 'OpenSSL_add_all_algorithms'` when running ANTA" +## `AttributeError: module 'lib' has no attribute 'OpenSSL_add_all_algorithms'` +???+ faq "`AttributeError: module 'lib' has no attribute 'OpenSSL_add_all_algorithms'` when running ANTA" When running the `anta` commands after installation, some users might encounter the following error: @@ -111,8 +111,8 @@ toc_depth: 2 ``` ## `__NSCFConstantString initialize` error on OSX -???+ faq "`__NSCFConstantString initialize` error on OSX" +???+ faq "`__NSCFConstantString initialize` error on OSX" This error occurs because of added security to restrict multithreading in macOS High Sierra and later versions of macOS. https://www.wefearchange.org/2018/11/forkmacos.rst.html diff --git a/docs/getting-started.md b/docs/getting-started.md index 39b270c..aac88c6 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -4,8 +4,6 @@ ~ that can be found in the LICENSE file. --> -# Getting Started - This section shows how to use ANTA with basic configuration. All examples are based on Arista Test Drive (ATD) topology you can access by reaching out to your preferred SE. ## Installation @@ -72,7 +70,7 @@ anta_inventory: tags: ['fabric', 'leaf'] ``` -> You can read more details about how to build your inventory [here](usage-inventory-catalog.md#create-an-inventory-file) +> You can read more details about how to build your inventory [here](usage-inventory-catalog.md#device-inventory) ## Test Catalog @@ -257,7 +255,7 @@ $ anta nrfu \ ] ``` -You can find more information under the __usage__ section of the website +You can find more information under the **usage** section of the website ### Basic usage in a Python script diff --git a/docs/imgs/anta-nrfu-md-report-output.png b/docs/imgs/anta-nrfu-md-report-output.png Binary files differnew file mode 100644 index 0000000..984e76b --- /dev/null +++ b/docs/imgs/anta-nrfu-md-report-output.png diff --git a/docs/imgs/anta_nrfu_csv.png b/docs/imgs/anta_nrfu_csv.png Binary files differnew file mode 100644 index 0000000..5e4129a --- /dev/null +++ b/docs/imgs/anta_nrfu_csv.png diff --git a/docs/overrides/main.html b/docs/overrides/main.html index ad3693f..d70fb98 100644 --- a/docs/overrides/main.html +++ b/docs/overrides/main.html @@ -16,6 +16,8 @@ {{app}} {% endblock %} +{# Keeping this for future announcement if required {% block announce %} ANTA code has moved to a new house in aristanetworks organization and so has the documentation. <strong>Please update your bookmark to use <a href="https://anta.arista.com">anta.arista.com<a/></strong> {% endblock %} +#} diff --git a/docs/requirements-and-installation.md b/docs/requirements-and-installation.md index ffcb9aa..1b35758 100644 --- a/docs/requirements-and-installation.md +++ b/docs/requirements-and-installation.md @@ -4,8 +4,6 @@ ~ that can be found in the LICENSE file. --> -# ANTA Requirements - ## Python version Python 3 (`>=3.9`) is required: @@ -21,7 +19,6 @@ This installation will deploy tests collection, scripts and all their Python req The ANTA package and the cli require some packages that are not part of the Python standard library. They are indicated in the [pyproject.toml](https://github.com/aristanetworks/anta/blob/main/pyproject.toml) file, under dependencies. - ### Install library from Pypi server ```bash @@ -31,14 +28,12 @@ pip install anta !!! Warning * This command alone **will not** install the ANTA CLI requirements. - * When using ANTA mode in [AVD](https://avd.arista.com) `eos_validate` role, (currently in preview), ensure you install the documented supported ANTA version for your AVD version.</br> - The latest documented version can be found at: https://avd.arista.com/stable/roles/eos_validate_state/ANTA-Preview.html ### Install ANTA CLI as an application with `pipx` [`pipx`](https://pipx.pypa.io/stable/) is a tool to install and run python applications in isolated environments. If you plan to use ANTA only as a CLI tool you can use `pipx` to install it. `pipx` installs ANTA in an isolated python environment and makes it available globally. -``` +```bash pipx install anta[cli] ``` @@ -46,7 +41,6 @@ pipx install anta[cli] Please take the time to read through the installation instructions of `pipx` before getting started. - ### Install CLI from Pypi server Alternatively, pip install with `cli` extra is enough to install the ANTA CLI. @@ -57,7 +51,6 @@ pip install anta[cli] ### Install ANTA from github - ```bash pip install git+https://github.com/aristanetworks/anta.git pip install git+https://github.com/aristanetworks/anta.git#egg=anta[cli] @@ -93,7 +86,7 @@ which anta ```bash # Check ANTA version anta --version -anta, version v1.0.0 +anta, version v1.1.0 ``` ## EOS Requirements diff --git a/docs/scripts/generate_svg.py b/docs/scripts/generate_svg.py index e6bf87a..f017b24 100644 --- a/docs/scripts/generate_svg.py +++ b/docs/scripts/generate_svg.py @@ -24,6 +24,7 @@ from unittest.mock import patch from rich.console import Console from rich.logging import RichHandler +from rich.progress import Progress from anta.cli.console import console from anta.cli.nrfu.utils import anta_progress_bar @@ -37,7 +38,7 @@ root.addHandler(r) OUTPUT_DIR = pathlib.Path(__file__).parent.parent / "imgs" -def custom_progress_bar() -> None: +def custom_progress_bar() -> Progress: """Set the console of progress_bar to main anta console. Caveat: this capture all steps of the progress bar.. diff --git a/docs/snippets/anta_nrfu_help.txt b/docs/snippets/anta_nrfu_help.txt index 68cb4b8..cb23fa7 100644 --- a/docs/snippets/anta_nrfu_help.txt +++ b/docs/snippets/anta_nrfu_help.txt @@ -29,8 +29,10 @@ Options: ANTA_INVENTORY; required] --tags TEXT List of tags using comma as separator: tag1,tag2,tag3. [env var: ANTA_TAGS] - -c, --catalog FILE Path to the test catalog YAML file [env - var: ANTA_CATALOG; required] + -c, --catalog FILE Path to the test catalog file [env var: + ANTA_CATALOG; required] + --catalog-format [yaml|json] Format of the catalog file, either 'yaml' or + 'json' [env var: ANTA_CATALOG_FORMAT] -d, --device TEXT Run tests on a specific device. Can be provided multiple times. -t, --test TEXT Run a specific test. Can be provided @@ -41,7 +43,8 @@ Options: or 1 if any test failed. [env var: ANTA_NRFU_IGNORE_ERROR] --hide [success|failure|error|skipped] - Group result by test or device. + Hide results by type: success / failure / + error / skipped'. --dry-run Run anta nrfu command but stop before starting to execute the tests. Considers all devices as connected. [env var: @@ -49,7 +52,9 @@ Options: --help Show this message and exit. Commands: - json ANTA command to check network state with JSON result. - table ANTA command to check network states with table result. - text ANTA command to check network states with text result. + csv ANTA command to check network state with CSV report. + json ANTA command to check network state with JSON results. + md-report ANTA command to check network state with Markdown report. + table ANTA command to check network state with table results. + text ANTA command to check network state with text results. tpl-report ANTA command to check network state with templated report. diff --git a/docs/stylesheets/extra.material.css b/docs/stylesheets/extra.material.css index 1724da9..2476f8c 100644 --- a/docs/stylesheets/extra.material.css +++ b/docs/stylesheets/extra.material.css @@ -126,36 +126,29 @@ line-height: 1em; font-size: 1.3rem; margin: 1em 0; - /* font-weight: 700; */ letter-spacing: -.01em; color: var(--md-default-fg-color--light); - text-transform: capitalize; font-style: normal; font-weight: bold; } .md-typeset h4 { - font-size: 0.9rem; + font-size: 1.1rem; margin: 1em 0; font-weight: 700; letter-spacing: -.01em; line-height: 1em; color: var(--md-default-fg-color--light); font-style: italic; - text-transform: capitalize; } .md-typeset h5, .md-typeset h6 { font-size: 0.9rem; margin: 1em 0; - /* font-weight: 700; */ letter-spacing: -.01em; - /* line-height: 2em; */ color: var(--md-default-fg-color--light); font-style: italic; - text-transform: capitalize; - text-decoration: underline; } .md-typeset table:not([class]) th { @@ -163,17 +156,13 @@ padding: .6rem .8rem; color: var(--md-default-fg-color); vertical-align: top; - /* background-color: var(--md-accent-bg-color); */ text-align: left; - /* min-width: 100%; */ - /* display: table; */ } .md-typeset table:not([class]) td { /* padding: .9375em 1.25em; */ border-collapse: collapse; vertical-align: center; text-align: left; - /* border-bottom: 1px solid var(--md-default-fg-color--light); */ } .md-typeset code { padding: 0 .2941176471em; @@ -250,3 +239,7 @@ div.doc-contents { padding-left: 25px; border-left: .05rem solid var(--md-typeset-table-color); } +h5.doc-heading { + /* Avoid to capitalize h5 headers for mkdocstrings */ + text-transform: none; +} diff --git a/docs/templates/python/material/anta_test.html b/docs/templates/python/material/anta_test.html.jinja index ade0ba6..a40d86a 100644 --- a/docs/templates/python/material/anta_test.html +++ b/docs/templates/python/material/anta_test.html.jinja @@ -31,7 +31,7 @@ {% endif %} {% with heading_level = heading_level + extra_level %} {% for attribute in attributes|order_members(config.members_order, members_list) %} - {% if members_list is not none or attribute.is_public(check_name=False) %} + {% if members_list is not none or attribute.is_public %} {% include attribute|get_template with context %} {% endif %} {% endfor %} @@ -60,7 +60,7 @@ {% include "attributes_table.html" with context %} {% set obj = old_obj %} {% else %} - {% if members_list is not none or class.is_public(check_name=False) %} + {% if members_list is not none or class.is_public %} {% include class|get_template with context %} {% endif %} {% endif %} @@ -82,7 +82,7 @@ {% with heading_level = heading_level + extra_level %} {% for function in functions|order_members(config.members_order, members_list) %} {% if not (obj.kind.value == "class" and function.name == "__init__" and config.merge_init_into_class) %} - {% if members_list is not none or function.is_public(check_name=False) %} + {% if members_list is not none or function.is_public %} {% include function|get_template with context %} {% endif %} {% endif %} @@ -104,7 +104,7 @@ {% endif %} {% with heading_level = heading_level + extra_level %} {% for module in modules|order_members(config.members_order.alphabetical, members_list) %} - {% if members_list is not none or module.is_public(check_name=False) %} + {% if members_list is not none or module.is_public %} {% include module|get_template with context %} {% endif %} {% endfor %} @@ -129,7 +129,7 @@ {% if not (obj.is_class and child.name == "__init__" and config.merge_init_into_class) %} - {% if members_list is not none or child.is_public(check_name=False) %} + {% if members_list is not none or child.is_public %} {% if child.is_attribute %} {% with attribute = child %} {% include attribute|get_template with context %} diff --git a/docs/templates/python/material/class.html b/docs/templates/python/material/class.html.jinja index 940103b..1c1173c 100644 --- a/docs/templates/python/material/class.html +++ b/docs/templates/python/material/class.html.jinja @@ -1,4 +1,4 @@ -{% extends "_base/class.html" %} +{% extends "_base/class.html.jinja" %} {% set anta_test = namespace(found=false) %} {% for base in class.bases %} {% set basestr = base | string %} @@ -10,7 +10,7 @@ {% if anta_test.found %} {% set root = False %} {% set heading_level = heading_level + 1 %} - {% include "anta_test.html" with context %} + {% include "anta_test.html.jinja" with context %} {# render source after children - TODO make add flag to respect disabling it.. though do we want to disable?#} <details class="quote"> <summary>Source code in <code> diff --git a/docs/templates/python/material/docstring.html b/docs/templates/python/material/docstring.html.jinja index b12ae7e..b12ae7e 100644 --- a/docs/templates/python/material/docstring.html +++ b/docs/templates/python/material/docstring.html.jinja diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 40fc07f..25b061c 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -4,8 +4,6 @@ ~ that can be found in the LICENSE file. --> -# Troubleshooting ANTA - A couple of things to check when hitting an issue with ANTA: ```mermaid @@ -26,18 +24,18 @@ flowchart LR click B "../faq" "FAQ" click E "https://github.com/aristanetworks/anta/issues" click H "https://github.com/aristanetworks/anta/issues" - style A stroke:#f00,stroke-width:2px + style A stroke:#f00,stroke-width:2px ``` ## Capturing logs -To help document the issue in Github, it is important to capture some logs so the developers can understand what is affecting your system. No logs mean that the first question asked on the issue will probably be _"Can you share some logs please?"_. +To help document the issue in Github, it is important to capture some logs so the developers can understand what is affecting your system. No logs mean that the first question asked on the issue will probably be *"Can you share some logs please?"*. ANTA provides very verbose logs when using the `DEBUG` level. When using DEBUG log level with a log file, the DEBUG logging level is not sent to stdout, but only to the file. !!! danger - On real deployments, do not use DEBUG logging level without setting a log file at the same time. + On real deployments, do not use DEBUG logging level without setting a log file at the same time. To save the logs to a file called `anta.log`, use the following flags: @@ -54,19 +52,17 @@ See `anta --help` for more information. These have to precede the `nrfu` cmd. so the `-l` and `--log-file` MUST be between `anta` and the `ANTA_COMMAND`. similarly, all the `nrfu` options MUST be set between the `nrfu` and the `ANTA_NRFU_SUBCOMMAND` (`json`, `text`, `table` or `tpl-report`). - As an example, for the `nrfu` command, it would look like: ```bash anta -l DEBUG --log-file anta.log nrfu --enable --username username --password arista --inventory inventory.yml -c nrfu.yml text ``` - ### `ANTA_DEBUG` environment variable -??? warning +!!! warning - Do not use this if you do not know why. This produces a lot of logs and can create confusion if you do not know what to look for. + Do not use this if you do not know why. This produces a lot of logs and can create confusion if you do not know what to look for. The environment variable `ANTA_DEBUG=true` enable ANTA Debug Mode. @@ -83,6 +79,7 @@ ANTA_DEBUG=true anta -l DEBUG --log-file anta.log nrfu --enable --username usern ANTA is using a specific ID in eAPI requests towards EOS. This allows for easier eAPI requests debugging on the device using EOS configuration `trace CapiApp setting UwsgiRequestContext/4,CapiUwsgiServer/4` to set up CapiApp agent logs. Then, you can view agent logs using: + ```bash bash tail -f /var/log/agents/CapiApp-* diff --git a/docs/usage-inventory-catalog.md b/docs/usage-inventory-catalog.md index f469933..e41321a 100644 --- a/docs/usage-inventory-catalog.md +++ b/docs/usage-inventory-catalog.md @@ -4,9 +4,10 @@ ~ that can be found in the LICENSE file. --> -# Inventory and Catalog +The ANTA framework needs 2 important inputs from the user to run: -The ANTA framework needs 2 important inputs from the user to run: a **device inventory** and a **test catalog**. +1. A **device inventory** +2. A **test catalog**. Both inputs can be defined in a file or programmatically. @@ -78,7 +79,8 @@ A test catalog is an instance of the [AntaCatalog](./api/catalog.md#anta.catalog In addition to the inventory file, you also have to define a catalog of tests to execute against your devices. This catalog list all your tests, their inputs and their tags. -A valid test catalog file must have the following structure: +A valid test catalog file must have the following structure in either YAML or JSON: + ```yaml --- <Python module>: @@ -86,6 +88,16 @@ A valid test catalog file must have the following structure: <AntaTest.Input compliant dictionary> ``` +```json +{ + "<Python module>": [ + { + "<AntaTest subclass>": <AntaTest.Input compliant dictionary> + } + ] +} +``` + ### Example ```yaml @@ -108,7 +120,45 @@ anta.tests.connectivity: custom_field: "Test run by John Doe" ``` +or equivalent in JSON: + +```json +{ + "anta.tests.connectivity": [ + { + "VerifyReachability": { + "result_overwrite": { + "description": "Test with overwritten description", + "categories": [ + "Overwritten category 1" + ], + "custom_field": "Test run by John Doe" + }, + "filters": { + "tags": [ + "leaf" + ] + }, + "hosts": [ + { + "destination": "1.1.1.1", + "source": "Management0", + "vrf": "MGMT" + }, + { + "destination": "8.8.8.8", + "source": "Management0", + "vrf": "MGMT" + } + ] + } + } + ] +} +``` + It is also possible to nest Python module definition: + ```yaml anta.tests: connectivity: @@ -157,7 +207,6 @@ anta.tests.system: All tests available as part of the ANTA framework are defined under the `anta.tests` Python module and are categorised per family (Python submodule). The complete list of the tests and their respective inputs is available at the [tests section](api/tests.md) of this website. - To run test to verify the EOS software version, you can do: ```yaml @@ -165,7 +214,7 @@ anta.tests.software: - VerifyEOSVersion: ``` -It will load the test `VerifyEOSVersion` located in `anta.tests.software`. But since this test has mandatory inputs, we need to provide them as a dictionary in the YAML file: +It will load the test `VerifyEOSVersion` located in `anta.tests.software`. But since this test has mandatory inputs, we need to provide them as a dictionary in the YAML or JSON file: ```yaml anta.tests.software: @@ -176,6 +225,21 @@ anta.tests.software: - 4.26.1F ``` +```json +{ + "anta.tests.software": [ + { + "VerifyEOSVersion": { + "versions": [ + "4.25.4M", + "4.31.1F" + ] + } + } + ] +} +``` + The following example is a very minimal test catalog: ```yaml @@ -247,29 +311,11 @@ Once you run `anta nrfu table`, you will see following output: ### Example script to merge catalogs -The following script reads all the files in `intended/test_catalogs/` with names `<device_name>-catalog.yml` and merge them together inside one big catalog `anta-catalog.yml`. +The following script reads all the files in `intended/test_catalogs/` with names `<device_name>-catalog.yml` and merge them together inside one big catalog `anta-catalog.yml` using the new `AntaCatalog.merge_catalogs()` class method. ```python -#!/usr/bin/env python -from anta.catalog import AntaCatalog - -from pathlib import Path -from anta.models import AntaTest - - -CATALOG_SUFFIX = '-catalog.yml' -CATALOG_DIR = 'intended/test_catalogs/' - -if __name__ == "__main__": - catalog = AntaCatalog() - for file in Path(CATALOG_DIR).glob('*'+CATALOG_SUFFIX): - c = AntaCatalog.parse(file) - device = str(file).removesuffix(CATALOG_SUFFIX).removeprefix(CATALOG_DIR) - print(f"Merging test catalog for device {device}") - # Apply filters to all tests for this device - for test in c.tests: - test.inputs.filters = AntaTest.Input.Filters(tags=[device]) - catalog = catalog.merge(c) - with open(Path('anta-catalog.yml'), "w") as f: - f.write(catalog.dump().yaml()) +--8<-- "merge_catalogs.py" ``` + +!!! warning + The `AntaCatalog.merge()` method is deprecated and will be removed in ANTA v2.0. Please use the `AntaCatalog.merge_catalogs()` class method instead. |
