diff options
Diffstat (limited to '')
6 files changed, 3010 insertions, 0 deletions
diff --git a/ansible_collections/amazon/aws/docs/docsite/extra-docs.yml b/ansible_collections/amazon/aws/docs/docsite/extra-docs.yml new file mode 100644 index 000000000..aeb152847 --- /dev/null +++ b/ansible_collections/amazon/aws/docs/docsite/extra-docs.yml @@ -0,0 +1,14 @@ +--- +sections: + - title: Changelog + toctree: + - CHANGELOG + - title: Scenario Guide + toctree: + - guide_aws + - title: Module Development Guidelines + toctree: + - dev_guidelines + - title: Dynamic Inventory Plugin Guide + toctree: + - aws_ec2_guide diff --git a/ansible_collections/amazon/aws/docs/docsite/links.yml b/ansible_collections/amazon/aws/docs/docsite/links.yml new file mode 100644 index 000000000..ce667b367 --- /dev/null +++ b/ansible_collections/amazon/aws/docs/docsite/links.yml @@ -0,0 +1,41 @@ +--- +# based on https://github.com/ansible-collections/collection_template/blob/main/docs/docsite/links.yml +# +# This will make sure that plugin and module documentation gets Edit on GitHub links +# that allow users to directly create a PR for this plugin or module in GitHub's UI. +# Remove this section if the collection repository is not on GitHub, or if you do not want this +# functionality for your collection. +edit_on_github: + repository: ansible-collections/amazon.aws + branch: main + # If your collection root (the directory containing galaxy.yml) does not coincide with your + # repository's root, you have to specify the path to the collection root here. For example, + # if the collection root is in a subdirectory ansible_collections/community/REPO_NAME + # in your repository, you have to set path_prefix to 'ansible_collections/community/REPO_NAME'. + path_prefix: '' + +# Here you can add arbitrary extra links. Please keep the number of links down to a +# minimum! Also please keep the description short, since this will be the text put on +# a button. +# +# Also note that some links are automatically added from information in galaxy.yml. +# The following are automatically added: +# 1. A link to the issue tracker (if `issues` is specified); +# 2. A link to the homepage (if `homepage` is specified and does not equal the +# `documentation` or `repository` link); +# 3. A link to the collection's repository (if `repository` is specified). + +# extra_links: +# - description: <DESC> +# url: <URL> + +# Specify communication channels for your collection. We suggest to not specify more +# than one place for communication per communication tool to avoid confusion. +communication: + matrix_rooms: + - topic: General usage and support questions + room: '#aws:ansible.im' + irc_channels: + - topic: General usage and support questions + network: Libera + channel: '#ansible-aws' diff --git a/ansible_collections/amazon/aws/docs/docsite/rst/CHANGELOG.rst b/ansible_collections/amazon/aws/docs/docsite/rst/CHANGELOG.rst new file mode 100644 index 000000000..6e07527c1 --- /dev/null +++ b/ansible_collections/amazon/aws/docs/docsite/rst/CHANGELOG.rst @@ -0,0 +1,1013 @@ +======================== +amazon.aws Release Notes +======================== + +.. contents:: Topics + + +v5.5.1 +====== + +Release Summary +--------------- + +This release brings few bugfixes. + + +Bugfixes +-------- + +- autoscaling_group - fix ValidationError when describing an autoscaling group that has more than 20 target groups attached to it by breaking the request into chunks (https://github.com/ansible-collections/amazon.aws/pull/1593). +- autoscaling_group_info - fix ValidationError when describing an autoscaling group that has more than 20 target groups attached to it by breaking the request into chunks (https://github.com/ansible-collections/amazon.aws/pull/1593). +- aws_account_attribute - raise correct ``AnsibleLookupError`` rather than ``AnsibleError`` (https://github.com/ansible-collections/amazon.aws/issues/1528). +- aws_secret - raise correct ``AnsibleLookupError`` rather than ``AnsibleError`` (https://github.com/ansible-collections/amazon.aws/issues/1528). +- aws_service_ip_ranges raise correct ``AnsibleLookupError`` rather than ``AnsibleError`` (https://github.com/ansible-collections/amazon.aws/issues/1528). +- aws_ssm - raise correct ``AnsibleLookupError`` rather than ``AnsibleError`` (https://github.com/ansible-collections/amazon.aws/issues/1528). +- ec2_instance - fix check_mode issue when adding network interfaces (https://github.com/ansible-collections/amazon.aws/issues/1403). +- elb_application_lb - fix missing attributes on creation of ALB. The ``create_or_update_alb()`` was including ALB-specific attributes when updating an existing ALB but not when creating a new ALB (https://github.com/ansible-collections/amazon.aws/issues/1510). + +v5.5.0 +====== + +Release Summary +--------------- + +This release contains a number of bugfixes, new features and new modules. This is the last planned minor release prior to the release of version 6.0.0. + + +Minor Changes +------------- + +- Add connectivity_type to ec2_vpc_nat_gateway module (https://github.com/ansible-collections/amazon.aws/pull/1267). +- cloudwatch - Add metrics and extended_statistic keys to cloudwatch module (https://github.com/ansible-collections/amazon.aws/pull/1133). +- ec2_ami - add support for BootMode, TpmSupport, UefiData params (https://github.com/ansible-collections/amazon.aws/pull/1037). +- ec2_metadata_facts - added support to query instance tags in metadata (https://github.com/ansible-collections/amazon.aws/pull/1186). +- kms_key - Add multi_region option to create_key (https://github.com/ansible-collections/amazon.aws/pull/1290). +- lambda - add support for function layers when creating or updating lambda function (https://github.com/ansible-collections/amazon.aws/pull/1118). +- lambda_event - Added support to set FunctionResponseTypes when creating lambda event source mappings (https://github.com/ansible-collections/amazon.aws/pull/1209). +- module_utils/elbv2 - removed compatibility code for ``botocore < 1.10.30`` (https://github.com/ansible-collections/amazon.aws/pull/1477). +- rds_cluster - New ``engine_mode`` parameter (https://github.com/ansible-collections/amazon.aws/pull/941). +- rds_cluster - add new options (e.g., ``db_cluster_instance_class``, ``allocated_storage``, ``storage_type``, ``iops``) (https://github.com/ansible-collections/amazon.aws/pull/1191). +- rds_cluster - update list of supported engines with ``mysql`` and ``postgres`` (https://github.com/ansible-collections/amazon.aws/pull/1191). +- s3_bucket - ensure ``public_access`` is configured before updating policies (https://github.com/ansible-collections/amazon.aws/pull/1511). + +Bugfixes +-------- + +- cloudwatch_metric_alarm - Don't consider ``StateTransitionedTimestamp`` in change detection. (https://github.com/ansible-collections/amazon.aws/pull/1440). +- ec2_instance - Pick up ``app_callback -> set_password`` rather than ``app_callback -> set_passwd`` (https://github.com/ansible-collections/amazon.aws/issues/1449). +- lambda_info - Do not convert environment variables to snake_case when querying lambda config. (https://github.com/ansible-collections/amazon.aws/pull/1457). +- rds_instance - fix type of ``promotion_tier`` as passed to the APIs (https://github.com/ansible-collections/amazon.aws/pull/1475). + +New Modules +----------- + +- lambda_layer - Creates an AWS Lambda layer or deletes an AWS Lambda layer version +- lambda_layer_info - List lambda layer or lambda layer versions + +v5.4.0 +====== + +Release Summary +--------------- + +This minor release brings bugfixes and minor new features. + +Minor Changes +------------- + +- ec2_spot_instance - add parameter ``terminate_instances`` to support terminate instances associated with spot requests. (https://github.com/ansible-collections/amazon.aws/pull/1402). +- route53_health_check - added support for enabling Latency graphs (MeasureLatency) during creation of a Route53 Health Check. (https://github.com/ansible-collections/amazon.aws/pull/1201). + +Bugfixes +-------- + +- ec2_metadata_facts - fix ``AttributeError`` when running the ec2_metadata_facts module on Python 2 managed nodes (https://github.com/ansible-collections/amazon.aws/issues/1358). +- ec2_vol - handle ec2_vol.tags when the associated instance already exists (https://github.com/ansible-collections/amazon.aws/pull/1071). +- rds_instance - Fixed ``TypeError`` when tagging RDS DB with storage type ``gp3`` (https://github.com/ansible-collections/amazon.aws/pull/1437). +- route53_info - Add new return key ``health_check_observations`` for health check operations (https://github.com/ansible-collections/amazon.aws/pull/1419). +- route53_info - Fixed ``Key Error`` when getting status or failure_reason of a health check (https://github.com/ansible-collections/amazon.aws/pull/1419). + +v5.3.0 +====== + +Release Summary +--------------- + +This release brings some minor changes, bugfixes, and deprecated features. + +Minor Changes +------------- + +- ec2_instance - more consistently return ``instances`` information (https://github.com/ansible-collections/amazon.aws/pull/964). +- ec2_instance - remove unused import (https://github.com/ansible-collections/amazon.aws/pull/1350). +- ec2_key - Add unit-tests coverage (https://github.com/ansible-collections/amazon.aws/pull/1288). +- ec2_vpc_nat_gateway - ensure allocation_id is defined before potential access (https://github.com/ansible-collections/amazon.aws/pull/1350). +- route53_zone - added support for associating multiple VPCs to route53 hosted zones (https://github.com/ansible-collections/amazon.aws/pull/1300). +- s3_bucket - add option to support creation of buckets with object lock enabled (https://github.com/ansible-collections/amazon.aws/pull/1372). + +Deprecated Features +------------------- + +- support for passing both profile and security tokens through a mix of environment variables and parameters has been deprecated and support will be removed in release 6.0.0. After release 6.0.0 it will only be possible to pass either a profile or security tokens, regardless of mechanism used to pass them. To explicitly block a parameter coming from an environment variable pass an empty string as the parameter value. Support for passing profile and security tokens together was originally deprecated in release 1.2.0, however only partially implemented in release 5.0.0 (https://github.com/ansible-collections/amazon.aws/pull/1355). + +Bugfixes +-------- + +- cloudtrail - support to disabling encryption using ``kms_key_id`` (https://github.com/ansible-collections/amazon.aws/pull/1384). +- ec2_key - fix issue when trying to update existing key pair with the same key material (https://github.com/ansible-collections/amazon.aws/pull/1383). +- module_utils/elbv2 - fix change detection by adding default values for ``Scope`` and ``SessionTimeout`` parameters in ``authenticate-oidc`` rules (https://github.com/ansible-collections/amazon.aws/pull/1270). +- module_utils/elbv2 - respect ``UseExistingClientSecret`` parameter in ``authenticate-oidc`` rules (https://github.com/ansible-collections/amazon.aws/pull/1270). +- revert breaking change introduced in 5.2.0 when passing credentials through a mix of environment variables and parameters (https://github.com/ansible-collections/amazon.aws/issues/1353). +- s3_bucket - empty bucket policy was throwing a JSONDecodeError - deal with it gracefully instead (https://github.com/ansible-collections/amazon.aws/pull/1368) + +v5.2.0 +====== + +Release Summary +--------------- + +A minor release containing bugfixes for the ``ec2_eni_info`` module and the ``aws_rds`` inventory plugin, as well as improvements to the ``rds_instance`` module. + + +Minor Changes +------------- + +- amazon.aws collection - refacterization of code to use argument specification ``fallback`` when falling back to environment variables for security credentials and AWS connection details (https://github.com/ansible-collections/amazon.aws/pull/1174). +- rds_instance - Split up the integration test-suite in a series of smaller tests (https://github.com/ansible-collections/amazon.aws/pull/1185). +- rds_instance - add support for gp3 storage type (https://github.com/ansible-collections/amazon.aws/pull/1266). + +Bugfixes +-------- + +- aws_rds - fixes bug in RDS inventory plugin where config file was ignored (https://github.com/ansible-collections/amazon.aws/issues/1304). +- lambda - fix flaky integration test which assumes there are no other lambdas in the account (https://github.com/ansible-collections/amazon.aws/issues/1277) + +v5.1.0 +====== + +Release Summary +--------------- + +This release brings some minor changes, bugfixes, security fixes and deprecated features. + +Minor Changes +------------- + +- amazon.aws collection - The ``aws_access_key`` parameter has been renamed to ``access_key``, ``access_key`` was previously an alias for this parameter and ``aws_access_key`` remains as an alias. This change should have no observable effect for users outside the module/plugin documentation. (https://github.com/ansible-collections/amazon.aws/pull/1172). +- amazon.aws collection - The ``aws_secret_key`` parameter has been renamed to ``secret_key``, ``secret_key`` was previously an alias for this parameter and ``aws_secret_key`` remains as an alias. This change should have no observable effect for users outside the module/plugin documentation. (https://github.com/ansible-collections/amazon.aws/pull/1172). +- amazon.aws collection - The ``security_token`` parameter has been renamed to ``session_token``, ``security_token`` was previously an alias for this parameter and ``security_token`` remains as an alias. This change should have no observable effect for users outside the module/plugin documentation. (https://github.com/ansible-collections/amazon.aws/pull/1172). +- aws_account_attribute lookup plugin - use ``missing_required_lib`` for more consistent error message when boto3/botocore is not available (https://github.com/ansible-collections/amazon.aws/pull/1152). +- aws_ec2 inventory - minor linting fixes (https://github.com/ansible-collections/amazon.aws/pull/1181). +- aws_ec2 inventory plugin - use ``missing_required_lib`` for more consistent error message when boto3/botocore is not available (https://github.com/ansible-collections/amazon.aws/pull/1152). +- aws_rds inventory plugin - use ``missing_required_lib`` for more consistent error message when boto3/botocore is not available (https://github.com/ansible-collections/amazon.aws/pull/1152). +- aws_secret lookup plugin - use ``missing_required_lib`` for more consistent error message when boto3/botocore is not available (https://github.com/ansible-collections/amazon.aws/pull/1152). +- aws_ssm lookup plugin - use ``missing_required_lib`` for more consistent error message when boto3/botocore is not available (https://github.com/ansible-collections/amazon.aws/pull/1152). +- ec2_instance - minor fix for launching an instance in specified AZ when ``vpc_subnet_id`` is not provided (https://github.com/ansible-collections/amazon.aws/pull/1150). +- ec2_instance - refacter ``tower_callback`` code to handle parameter validation as part of the argument specification (https://github.com/ansible-collections/amazon.aws/pull/1199). +- ec2_instance - the ``instance_role`` parameter has been renamed to ``iam_instance_profile`` to better reflect what it is, ``instance_role`` remains as an alias (https://github.com/ansible-collections/amazon.aws/pull/1151). +- ec2_instance - the ``tower_callback`` parameter has been renamed to ``aap_callback``, ``tower_callback`` remains as an alias. This change should have no observable effect for users outside the module documentation (https://github.com/ansible-collections/amazon.aws/pull/1199). +- s3_object_info - minor linting fixes (https://github.com/ansible-collections/amazon.aws/pull/1181). + +Deprecated Features +------------------- + +- amazon.aws collection - Support for the ``EC2_ACCESS_KEY`` environment variable has been deprecated and will be removed in a release after 2024-12-01. Please use the ``access_key`` parameter or ``AWS_ACCESS_KEY_ID`` environment variable instead (https://github.com/ansible-collections/amazon.aws/pull/1172). +- amazon.aws collection - Support for the ``EC2_REGION`` environment variable has been deprecated and will be removed in a release after 2024-12-01. Please use the ``region`` parameter or ``AWS_REGION`` environment variable instead (https://github.com/ansible-collections/amazon.aws/pull/1172). +- amazon.aws collection - Support for the ``EC2_SECRET_KEY`` environment variable has been deprecated and will be removed in a release after 2024-12-01. Please use the ``secret_key`` parameter or ``AWS_SECRET_ACCESS_KEY`` environment variable instead (https://github.com/ansible-collections/amazon.aws/pull/1172). +- amazon.aws collection - Support for the ``EC2_SECURITY_TOKEN`` environment variable has been deprecated and will be removed in a release after 2024-12-01. Please use the ``session_token`` parameter or ``AWS_SESSION_TOKEN`` environment variable instead (https://github.com/ansible-collections/amazon.aws/pull/1172). +- amazon.aws collection - Support for the ``EC2_URL`` and ``S3_URL`` environment variables has been deprecated and will be removed in a release after 2024-12-01. Please use the ``endpoint_url`` parameter or ``AWS_ENDPOINT_URL`` environment variable instead (https://github.com/ansible-collections/amazon.aws/pull/1172). +- amazon.aws collection - The ``access_token`` alias for the ``session_token`` parameter has been deprecated and will be removed in a release after 2024-12-01. Please use the ``session_token`` name instead (https://github.com/ansible-collections/amazon.aws/pull/1172). +- amazon.aws collection - The ``access_token`` alias for the ``session_token`` parameter has been deprecated and will be removed in a release after 2024-12-01. Please use the ``session_token`` name instead (https://github.com/ansible-collections/amazon.aws/pull/1172). +- amazon.aws collection - The ``aws_security_token`` alias for the ``session_token`` parameter has been deprecated and will be removed in a release after 2024-12-01. Please use the ``session_token`` name instead (https://github.com/ansible-collections/amazon.aws/pull/1172). +- amazon.aws collection - The ``ec2_access_key`` alias for the ``access_key`` parameter has been deprecated and will be removed in a release after 2024-12-01. Please use the ``access_key`` name instead (https://github.com/ansible-collections/amazon.aws/pull/1172). +- amazon.aws collection - The ``ec2_region`` alias for the ``region`` parameter has been deprecated and will be removed in a release after 2024-12-01. Please use the ``region`` name instead (https://github.com/ansible-collections/amazon.aws/pull/1172). +- amazon.aws collection - The ``ec2_secret_key`` alias for the ``secret_key`` parameter has been deprecated and will be removed in a release after 2024-12-01. Please use the ``secret_key`` name instead (https://github.com/ansible-collections/amazon.aws/pull/1172). +- amazon.aws collection - The ``security_token`` alias for the ``session_token`` parameter has been deprecated and will be removed in a release after 2024-12-01. Please use the ``session_token`` name instead (https://github.com/ansible-collections/amazon.aws/pull/1172). +- ec2_security_group - support for passing nested lists to ``cidr_ip`` and ``cidr_ipv6`` has been deprecated. Nested lists can be passed through the ``flatten`` filter instead ``cidr_ip: '{{ my_cidrs | flatten }}'`` (https://github.com/ansible-collections/amazon.aws/pull/1213). +- module_utils.url - ``ansible_collections.amazon.aws.module_utils.urls`` is believed to be unused and has been deprecated and will be removed in release 7.0.0. + +Security Fixes +-------------- + +- ec2_instance - fixes leak of password into logs when using ``tower_callback.windows=True`` and ``tower_callback.set_password`` (https://github.com/ansible-collections/amazon.aws/pull/1199). + +Bugfixes +-------- + +- ec2_instance - fixes ``Invalid type for parameter TagSpecifications, value None`` error when tags aren't specified (https://github.com/ansible-collections/amazon.aws/issues/1148). +- module_utils.transformations - ensure that ``map_complex_type`` still returns transformed items if items exists that are not in the type_map (https://github.com/ansible-collections/amazon.aws/pull/1163). + +v5.0.2 +====== + +Bugfixes +-------- + +- ec2_metadata_facts - fixed ``AttributeError`` (https://github.com/ansible-collections/amazon.aws/issues/1134). + +v5.0.1 +====== + +Bugfixes +-------- + +- ec2_vpc_net_info - fix KeyError (https://github.com/ansible-collections/amazon.aws/pull/1109). +- ec2_vpc_net_info - remove hardcoded ``ClassicLinkEnabled`` parameter when request for ``ClassicLinkDnsSupported`` failed (https://github.com/ansible-collections/amazon.aws/pull/1109). +- s3_object - be more defensive when checking the results of ``s3.get_bucket_ownership_controls`` (https://github.com/ansible-collections/amazon.aws/issues/1115). + +v5.0.0 +====== + +Release Summary +--------------- + +In this release we promoted many community modules to Red Hat supported status. Those modules have been moved from the commuity.aws to amazon.aws collection. This release also brings some new features, bugfixes, breaking changes and deprecated features. The amazon.aws collection has dropped support for ``botocore<1.21.0`` and ``boto3<1.18.0``. Support for ``ansible-core<2.11`` has also been dropped. + +Major Changes +------------- + +- autoscaling_group - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.autoscaling_group``. +- autoscaling_group_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.autoscaling_group_info``. +- cloudtrail - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.cloudtrail``. +- cloudwatch_metric_alarm - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.cloudwatch_metric_alarm``. +- cloudwatchevent_rule - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.cloudwatchevent_rule``. +- cloudwatchlogs_log_group - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.cloudwatchlogs_log_group``. +- cloudwatchlogs_log_group_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.cloudwatchlogs_log_group_info``. +- cloudwatchlogs_log_group_metric_filter - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.cloudwatchlogs_log_group_metric_filter``. +- ec2_eip - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_eip``. +- ec2_eip_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_eip_info``. +- elb_application_lb - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.elb_application_lb``. +- elb_application_lb_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.elb_application_lb_info``. +- execute_lambda - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.execute_lambda``. +- iam_policy - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.iam_policy``. +- iam_policy_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.iam_policy_info``. +- iam_user - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.iam_user``. +- iam_user_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.iam_user_info``. +- kms_key - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.kms_key``. +- kms_key_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.kms_key_info``. +- lambda - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.lambda``. +- lambda_alias - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.lambda_alias``. +- lambda_event - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.lambda_event``. +- lambda_execute - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.lambda_execute``. +- lambda_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.lambda_info``. +- lambda_policy - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.lambda_policy``. +- rds_cluster - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.rds_cluster``. +- rds_cluster_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.rds_cluster_info``. +- rds_cluster_snapshot - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.rds_cluster_snapshot``. +- rds_instance - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.rds_instance``. +- rds_instance_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.rds_instance_info``. +- rds_instance_snapshot - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.rds_instance_snapshot``. +- rds_option_group - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.rds_option_group``. +- rds_option_group_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.rds_option_group_info``. +- rds_param_group - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.rds_param_group``. +- rds_snapshot_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.rds_snapshot_info``. +- rds_subnet_group - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.rds_subnet_group``. +- route53 - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.route53``. +- route53_health_check - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.route53_health_check``. +- route53_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.route53_info``. +- route53_zone - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.route53_zone``. + +Minor Changes +------------- + +- Ability to record and replay the API interaction of a module for testing purpose. Show case the feature with an example (https://github.com/ansible-collections/amazon.aws/pull/998). +- Remove the empty __init__.py file from the distribution, they were not required anymore (https://github.com/ansible-collections/amazon.aws/pull/1018). +- amazon.aws modules - the ``ec2_url`` parameter has been renamed to ``endpoint_url`` for consistency, ``ec2_url`` remains as an alias (https://github.com/ansible-collections/amazon.aws/pull/992). +- aws_caller_info - minor linting fixes (https://github.com/ansible-collections/amazon.aws/pull/968). +- aws_ec2 - introduce the ``allow_duplicated_hosts`` configuration key (https://github.com/ansible-collections/amazon.aws/pull/1026). +- cloudformation - avoid catching ``Exception``, catch more specific errors instead (https://github.com/ansible-collections/amazon.aws/pull/968). +- cloudwatch_metric_alarm_info - Added a new module that describes the cloudwatch metric alarms (https://github.com/ansible-collections/amazon.aws/pull/988). +- ec2_group - The ``ec2_group`` module has been renamed to ``ec2_security_group``, ``ec2_group`` remains as an alias (https://github.com/ansible-collections/amazon.aws/pull/897). +- ec2_group_info - The ``ec2_group_info`` module has been renamed to ``ec2_security_group_info``, ``ec2_group_info`` remains as an alias (https://github.com/ansible-collections/amazon.aws/pull/897). +- ec2_instance - Add hibernation_options and volumes->ebs->encrypted keys to support stop-hibernate instance (https://github.com/ansible-collections/amazon.aws/pull/972). +- ec2_instance - expanded the use of the automatic retries to ``InsuffienctInstanceCapacity`` (https://github.com/ansible-collections/amazon.aws/issues/1038). +- ec2_metadata_facts - avoid catching ``Exception``, catch more specific errors instead (https://github.com/ansible-collections/amazon.aws/pull/968). +- ec2_security_group - minor linting fixes (https://github.com/ansible-collections/amazon.aws/pull/968). +- ec2_vpc_endpoint - avoid catching ``Exception``, catch more specific errors instead (https://github.com/ansible-collections/amazon.aws/pull/968). +- ec2_vpc_nat_gateway - minor linting fixes (https://github.com/ansible-collections/amazon.aws/pull/968). +- ec2_vpc_net_info - handle classic link check for shared VPCs by throwing a warning instead of an error (https://github.com/ansible-collections/amazon.aws/pull/984). +- module_utils/acm - Move to jittered backoff (https://github.com/ansible-collections/amazon.aws/pull/946). +- module_utils/elbv2 - ensures that ``ip_address_type`` is set on creation rather than re-setting it after creation (https://github.com/ansible-collections/amazon.aws/pull/940). +- module_utils/elbv2 - uses new waiters with retries for temporary failures (https://github.com/ansible-collections/amazon.aws/pull/940). +- module_utils/waf - Move to jittered backoff (https://github.com/ansible-collections/amazon.aws/pull/946). +- module_utils/waiters - Add waiters to manage eks_nodegroup module (https://github.com/ansible-collections/community.aws/pull/1415). +- s3_bucket - ``rgw`` was added as an alias for the ``ceph`` parameter for consistency with the ``s3_object`` module (https://github.com/ansible-collections/amazon.aws/pull/994). +- s3_bucket - the ``s3_url`` parameter was merged into the ``endpoint_url`` parameter, ``s3_url`` remains as an alias (https://github.com/ansible-collections/amazon.aws/pull/994). +- s3_object - added the ``sig_v4`` paramater, enbling the user to opt in to signature version 4 for download/get operations. (https://github.com/ansible-collections/amazon.aws/pull/1014) +- s3_object - minor linting fixes (https://github.com/ansible-collections/amazon.aws/pull/968). +- s3_object - the ``rgw`` parameter was renamed to ``ceph`` for consistency with the ``s3_bucket`` module, ``rgw`` remains as an alias (https://github.com/ansible-collections/amazon.aws/pull/994). +- s3_object - the ``s3_url`` parameter was merged into the ``endpoint_url`` parameter, ``s3_url`` remains as an alias (https://github.com/ansible-collections/amazon.aws/pull/994). +- s3_object - updated module to add support for handling file upload to a bucket with ACL disabled (https://github.com/ansible-collections/amazon.aws/pull/921). +- s3_object_info - Added a new module that describes S3 Objects (https://github.com/ansible-collections/amazon.aws/pull/977). + +Breaking Changes / Porting Guide +-------------------------------- + +- amazon.aws collection - Support for ansible-core < 2.11 has been dropped (https://github.com/ansible-collections/amazon.aws/pull/1087). +- amazon.aws collection - The amazon.aws collection has dropped support for ``botocore<1.21.0`` and ``boto3<1.18.0``. Most modules will continue to work with older versions of the AWS SDK, however compatibility with older versions of the SDK is not guaranteed and will not be tested. When using older versions of the SDK a warning will be emitted by Ansible (https://github.com/ansible-collections/amazon.aws/pull/934). +- doc_fragments - remove minimum collection requirements from doc_fragments/aws.py and allow pulling those from doc_fragments/aws_boto3.py instead (https://github.com/ansible-collections/amazon.aws/pull/985). +- ec2_ami - the default value for ``purge_tags`` has been changed from ``False`` to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/916). +- ec2_ami - the parameter aliases ``DeviceName``, ``VirtualName`` and ``NoDevice`` were previously deprecated and have been removed, please use ``device_name``, ``virtual_name`` and ``no_device`` instead (https://github.com/ansible-collections/amazon.aws/pull/913). +- ec2_eni_info - the mutual exclusivity of the ``eni_id`` and ``filters`` parameters is now enforced, previously ``filters`` would be ignored if ``eni_id`` was set (https://github.com/ansible-collections/amazon.aws/pull/954). +- ec2_instance - the default value for ``purge_tags`` has been changed from ``False`` to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/916). +- ec2_key - the default value for ``purge_tags`` has been changed from ``False`` to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/916). +- ec2_vol - the default value for ``purge_tags`` has been changed from ``False`` to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/916). +- ec2_vpc_dhcp_option_info - the parameter aliases ``DhcpOptionIds`` and ``DryRun`` were previously deprecated and have been removed, please use ``dhcp_options_ids`` and ``no_device`` instead (https://github.com/ansible-collections/amazon.aws/pull/913). +- ec2_vpc_endpoint - the default value for ``purge_tags`` has been changed from ``False`` to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/916). +- ec2_vpc_net - the default value for ``purge_tags`` has been changed from ``False`` to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/916). +- ec2_vpc_route_table - the default value for ``purge_tags`` has been changed from ``False`` to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/916). +- s3_bucket - the previously deprecated alias ``S3_URL`` for the ``s3_url`` parameter has been removed. Playbooks shuold be updated to use ``s3_url`` (https://github.com/ansible-collections/amazon.aws/pull/908). +- s3_object - the previously deprecated alias ``S3_URL`` for the ``s3_url`` parameter has been removed. Playbooks should be updated to use ``s3_url`` (https://github.com/ansible-collections/amazon.aws/pull/908). + +Deprecated Features +------------------- + +- amazon.aws collection - due to the AWS SDKs announcing the end of support for Python less than 3.7 (https://aws.amazon.com/blogs/developer/python-support-policy-updates-for-aws-sdks-and-tools/) support for Python less than 3.7 by this collection has been deprecated and will be removed in a release after 2023-05-31 (https://github.com/ansible-collections/amazon.aws/pull/935). +- inventory/aws_ec2 - the ``include_extra_api_calls`` is now deprecated, its value is silently ignored (https://github.com/ansible-collections/amazon.aws/pull/1097). + +Bugfixes +-------- + +- aws_ec2 - address a regression introduced in 4.1.0 (https://github.com/ansible-collections/amazon.aws/pull/862) that cause the presnse of duplicated hosts in the inventory. +- cloudtrail - Fix key error TagList to TagsList (https://github.com/ansible-collections/amazon.aws/issues/1088). +- ec2_instance - Only show the deprecation warning for the default value of ``instance_type`` when ``count`` or ``exact_count`` are specified (https://github.com//issues/980). +- ec2_metadata_facts - fix ``'NoneType' object is not callable`` exception when using Ansible 2.13+ (https://github.com/ansible-collections/amazon.aws/issues/942). +- module_utils/botocore - fix ``object has no attribute 'fail'`` error in error handling (https://github.com/ansible-collections/amazon.aws/pull/1045). +- module_utils/elbv2 - fixes ``KeyError`` when using ``UseExistingClientSecret`` rather than ``ClientSecret`` (https://github.com/ansible-collections/amazon.aws/pull/940). +- module_utils/elbv2 - improvements to idempotency when comparing listeners (https://github.com/ansible-collections/community.aws/issues/604). +- s3_object - also use ``ignore_nonexistent_bucket`` when listing a bucket (https://github.com/ansible-collections/amazon.aws/issues/966). + +New Modules +----------- + +- cloudtrail_info - Gather information about trails in AWS Cloud Trail. +- cloudwatch_metric_alarm_info - Gather information about the alarms for the specified metric +- s3_object_info - Gather information about objects in S3 + +v4.3.0 +====== + +Release Summary +--------------- + +The amazon.aws 4.3.0 release includes a number of minor bug fixes and improvements. +Following the release of amazon.aws 5.0.0, backports to the 4.x series will be limited to +security issues and bugfixes. + + +Minor Changes +------------- + +- ec2_instance - expanded the use of the automatic retries to ``InsuffienctInstanceCapacity`` (https://github.com/ansible-collections/amazon.aws/issues/1038). + +Bugfixes +-------- + +- ec2_metadata_facts - fix ``'NoneType' object is not callable`` exception when using Ansible 2.13+ (https://github.com/ansible-collections/amazon.aws/issues/942). +- module_utils/cloud - Fix ``ValueError: ansible_collections.amazon.aws.plugins.module_utils.core.__spec__ is None`` error on Ansible 2.9 (https://github.com/ansible-collections/amazon.aws/issues/1083). + +v4.2.0 +====== + +Minor Changes +------------- + +- ec2_security_group - set type as ``list`` for rules->group_name as it can accept both ``str`` and ``list`` (https://github.com/ansible-collections/amazon.aws/pull/971). +- various modules - linting fixups (https://github.com/ansible-collections/amazon.aws/pull/953). + +Deprecated Features +------------------- + +- module_utils.cloud - removal of the ``CloudRetry.backoff`` has been delayed until release 6.0.0. It is recommended to update custom modules to use ``jittered_backoff`` or ``exponential_backoff`` instead (https://github.com/ansible-collections/amazon.aws/pull/951). + +v4.1.0 +====== + +Minor Changes +------------- + +- ec2_instance - expanded the use of the automatic retries on temporary failures (https://github.com/ansible-collections/amazon.aws/issues/927). +- s3_bucket - updated module to enable support for setting S3 Bucket Keys for SSE-KMS (https://github.com/ansible-collections/amazon.aws/pull/882). + +Deprecated Features +------------------- + +- amazon.aws collection - due to the AWS SDKs announcing the end of support for Python less than 3.7 (https://aws.amazon.com/blogs/developer/python-support-policy-updates-for-aws-sdks-and-tools/) support for Python less than 3.7 by this collection has been deprecated and will be removed in a release after 2023-05-31 (https://github.com/ansible-collections/amazon.aws/pull/935). + +Bugfixes +-------- + +- aws_ec2 - ensure the correct number of hosts are returned when tags as hostnames are used (https://github.com/ansible-collections/amazon.aws/pull/862). +- elb_application_lb - fix ``KeyError`` when balancing across two Target Groups (https://github.com/ansible-collections/community.aws/issues/1089). +- elb_classic_lb - fix ``'NoneType' object has no attribute`` bug when creating a new ELB in check mode with a health check (https://github.com/ansible-collections/amazon.aws/pull/915). +- elb_classic_lb - fix ``'NoneType' object has no attribute`` bug when creating a new ELB using security group names (https://github.com/ansible-collections/amazon.aws/issues/914). + +v4.0.0 +====== + +Major Changes +------------- + +- amazon.aws collection - The amazon.aws collection has dropped support for ``botocore<1.20.0`` and ``boto3<1.17.0``. Most modules will continue to work with older versions of the AWS SDK, however compatibility with older versions of the SDK is not guaranteed and will not be tested. When using older versions of the SDK a warning will be emitted by Ansible (https://github.com/ansible-collections/amazon.aws/pull/574). + +Minor Changes +------------- + +- aws_s3 - Add ``validate_bucket_name`` option, to control bucket name validation (https://github.com/ansible-collections/amazon.aws/pull/615). +- aws_s3 - The ``aws_s3`` module has been renamed to ``s3_object`` (https://github.com/ansible-collections/amazon.aws/pull/869). +- aws_s3 - ``resource_tags`` has been added as an alias for the ``tags`` parameter (https://github.com/ansible-collections/amazon.aws/pull/845). +- ec2_eni - Change parameter ``device_index`` data type to string when passing to ``describe_network_inter`` api call (https://github.com/ansible-collections/amazon.aws/pull/877). +- ec2_eni - ``resource_tags`` has been added as an alias for the ``tags`` parameter (https://github.com/ansible-collections/amazon.aws/pull/845). +- ec2_group - add ``egress_rules`` as an alias for ``rules_egress`` (https://github.com/ansible-collections/amazon.aws/pull/878). +- ec2_group - add ``purge_egress_rules`` as an alias for ``purge_rules_egress`` (https://github.com/ansible-collections/amazon.aws/pull/878). +- ec2_instance - Add missing ``metadata_options`` parameters (https://github.com/ansible-collections/amazon.aws/pull/715). +- ec2_key - ``resource_tags`` has been added as an alias for the ``tags`` parameter (https://github.com/ansible-collections/amazon.aws/pull/845). +- ec2_vpc_net - add support for managing VPCs by ID (https://github.com/ansible-collections/amazon.aws/pull/848). +- ec2_vpc_subnet - add support for OutpostArn param (https://github.com/ansible-collections/amazon.aws/pull/598). +- elb_classic_lb - ``resource_tags`` has been added as an alias for the ``tags`` parameter (https://github.com/ansible-collections/amazon.aws/pull/845). +- s3_bucket - Add ``validate_bucket_name`` option, to control bucket name validation (https://github.com/ansible-collections/amazon.aws/pull/615). +- s3_bucket - ``resource_tags`` has been added as an alias for the ``tags`` parameter (https://github.com/ansible-collections/amazon.aws/pull/845). + +Breaking Changes / Porting Guide +-------------------------------- + +- Tags beginning with ``aws:`` will not be removed when purging tags, these tags are reserved by Amazon and may not be updated or deleted (https://github.com/ansible-collections/amazon.aws/issues/817). +- amazon.aws collection - the ``profile`` parameter is now mutually exclusive with the ``aws_access_key``, ``aws_secret_key`` and ``security_token`` parameters (https://github.com/ansible-collections/amazon.aws/pull/834). +- aws_az_info - the module alias ``aws_az_facts`` was deprecated in Ansible 2.9 and has now been removed (https://github.com/ansible-collections/amazon.aws/pull/832). +- aws_s3 - the default value for ``ensure overwrite`` has been changed to ``different`` instead of ``always`` so that the module is idempotent by default (https://github.com/ansible-collections/amazon.aws/issues/811). +- aws_ssm - on_denied and on_missing now both default to error, for consistency with both aws_secret and the base Lookup class (https://github.com/ansible-collections/amazon.aws/issues/617). +- ec2 - The ``ec2`` module has been removed in release 4.0.0 and replaced by the ``ec2_instance`` module (https://github.com/ansible-collections/amazon.aws/pull/630). +- ec2_vpc_igw_info - The default value for ``convert_tags`` has been changed to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/835). +- elb_classic_lb - the ``ec2_elb`` fact has been removed (https://github.com/ansible-collections/amazon.aws/pull/827). +- module_utils - Support for the original AWS SDK aka ``boto`` has been removed, including all relevant helper functions. All modules should now use the ``boto3``/``botocore`` AWS SDK (https://github.com/ansible-collections/amazon.aws/pull/630) + +Deprecated Features +------------------- + +- aws_s3 - The ``S3_URL`` alias for the s3_url option has been deprecated and will be removed in release 5.0.0 (https://github.com/ansible-collections/community.aws/pull/795). +- ec2_ami - The ``DeviceName`` alias for the device_name option has been deprecated and will be removed in release 5.0.0 (https://github.com/ansible-collections/community.aws/pull/795). +- ec2_ami - The ``NoDevice`` alias for the no_device option has been deprecated and will be removed in release 5.0.0 (https://github.com/ansible-collections/community.aws/pull/795). +- ec2_ami - The ``VirtualName`` alias for the virtual_name option has been deprecated and will be removed in release 5.0.0 (https://github.com/ansible-collections/community.aws/pull/795). +- ec2_ami - the current default value of ``False`` for ``purge_tags`` has been deprecated and will be updated in release 5.0.0 to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/846). +- ec2_instance - The default value for ```instance_type``` has been deprecated, in the future release you must set an instance_type or a launch_template (https://github.com/ansible-collections/amazon.aws/pull/587). +- ec2_instance - the current default value of ``False`` for ``purge_tags`` has been deprecated and will be updated in release 5.0.0 to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/849). +- ec2_key - the current default value of ``False`` for ``purge_tags`` has been deprecated and will be updated in release 5.0.0 to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/846). +- ec2_vol - the current default value of ``False`` for ``purge_tags`` has been deprecated and will be updated in release 5.0.0 to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/846). +- ec2_vpc_dhcp_option_info - The ``DhcpOptionIds`` alias for the dhcp_option_ids option has been deprecated and will be removed in release 5.0.0 (https://github.com/ansible-collections/community.aws/pull/795). +- ec2_vpc_dhcp_option_info - The ``DryRun`` alias for the dry_run option has been deprecated and will be removed in release 5.0.0 (https://github.com/ansible-collections/community.aws/pull/795). +- ec2_vpc_endpoint - the current default value of ``False`` for ``purge_tags`` has been deprecated and will be updated in release 5.0.0 to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/846). +- ec2_vpc_net - the current default value of ``False`` for ``purge_tags`` has been deprecated and will be updated in release 5.0.0 to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/848). +- ec2_vpc_route_table - the current default value of ``False`` for ``purge_tags`` has been deprecated and will be updated in release 5.0.0 to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/846). +- s3_bucket - The ``S3_URL`` alias for the s3_url option has been deprecated and will be removed in release 5.0.0 (https://github.com/ansible-collections/community.aws/pull/795). +- s3_object - Support for creation and deletion of S3 buckets has been deprecated. Please use the ``amazon.aws.s3_bucket`` module to create and delete buckets (https://github.com/ansible-collections/amazon.aws/pull/869). + +Removed Features (previously deprecated) +---------------------------------------- + +- cloudformation - the ``template_format`` option has been removed. It has been ignored by the module since Ansible 2.3 (https://github.com/ansible-collections/amazon.aws/pull/833). +- ec2_key - the ``wait_timeout`` option had no effect, was deprecated in release 1.0.0, and has now been removed (https://github.com/ansible-collections/amazon.aws/pull/830). +- ec2_key - the ``wait`` option had no effect, was deprecated in release 1.0.0, and has now been removed (https://github.com/ansible-collections/amazon.aws/pull/830). +- ec2_tag - the previously deprecated state ``list`` has been removed. To list tags on an EC2 resource the ``ec2_tag_info`` module can be used (https://github.com/ansible-collections/amazon.aws/pull/829). +- ec2_vol - the previously deprecated state ``list`` has been removed. To list volumes the ``ec2_vol_info`` module can be used (https://github.com/ansible-collections/amazon.aws/pull/828). +- module_utils.batch - the class ``ansible_collections.amazon.aws.plugins.module_utils.batch.AWSConnection`` has been removed. Please use ``AnsibleAWSModule.client()`` instead (https://github.com/ansible-collections/amazon.aws/pull/831). + +Bugfixes +-------- + +- ec2_group - fix uncaught exception when running with ``--diff`` and ``--check`` to create a new security group (https://github.com/ansible-collections/amazon.aws/issues/440). +- ec2_instance - Add a condition to handle default ```instance_type``` value for fix breaking on instance creation with launch template (https://github.com/ansible-collections/amazon.aws/pull/587). +- ec2_instance - raise an error when missing permission to stop instance when ``state`` is set to ``rebooted``` (https://github.com/ansible-collections/amazon.aws/pull/671). +- ec2_vpc_igw - use gateway_id rather than filters to paginate if possible to fix 'NoneType' object is not subscriptable error (https://github.com/ansible-collections/amazon.aws/pull/766). +- ec2_vpc_net - fix a bug where CIDR configuration would be updated in check mode (https://github.com/ansible/ansible/issues/62678). +- ec2_vpc_net - fix a bug where the module would get stuck if DNS options were updated in check mode (https://github.com/ansible/ansible/issues/62677). +- elb_classic_lb - modify the return value of _format_listeners method to resolve a failure creating https listeners (https://github.com/ansible-collections/amazon.aws/pull/860). + +v3.5.0 +====== + +Release Summary +--------------- + +Following the release of amazon.aws 5.0.0, 3.5.0 is a bugfix release and the final planned release for the 3.x series. + + +Minor Changes +------------- + +- ec2_security_group - set type as ``list`` for rules->group_name as it can accept both ``str`` and ``list`` (https://github.com/ansible-collections/amazon.aws/pull/971). + +Bugfixes +-------- + +- ec2_metadata_facts - fix ``'NoneType' object is not callable`` exception when using Ansible 2.13+ (https://github.com/ansible-collections/amazon.aws/issues/942). + +v3.4.0 +====== + +Minor Changes +------------- + +- ec2_instance - expanded the use of the automatic retries on temporary failures (https://github.com/ansible-collections/amazon.aws/issues/927). + +Bugfixes +-------- + +- elb_application_lb - fix ``KeyError`` when balancing across two Target Groups (https://github.com/ansible-collections/community.aws/issues/1089). +- elb_classic_lb - fix ``'NoneType' object has no attribute`` bug when creating a new ELB in check mode with a health check (https://github.com/ansible-collections/amazon.aws/pull/915). +- elb_classic_lb - fix ``'NoneType' object has no attribute`` bug when creating a new ELB using security group names (https://github.com/ansible-collections/amazon.aws/issues/914). + +v3.3.1 +====== + +v3.3.0 +====== + +Minor Changes +------------- + +- aws_ec2 inventory - Allow for literal strings in hostname that don't match filter parameters in ec2 describe-instances (https://github.com/ansible-collections/amazon.aws/pull/826). +- aws_ssm - Add support for ``endpoint`` parameter (https://github.com/ansible-collections/amazon.aws/pull/837). +- module.utils.rds - add retry_codes to get_rds_method_attribute return data to use in call_method and add unit tests (https://github.com/ansible-collections/amazon.aws/pull/776). +- module.utils.rds - refactor to utilize get_rds_method_attribute return data (https://github.com/ansible-collections/amazon.aws/pull/776). +- module_utils - add new aliases ``aws_session_token`` and ``session_token`` to the ``security_token`` parameter to be more in-line with the boto SDK (https://github.com/ansible-collections/amazon.aws/pull/631). +- module_utils.rds - Add support and unit tests for addition/removal of IAM roles to/from a db instance in module_utils.rds with waiters (https://github.com/ansible-collections/amazon.aws/pull/714). + +Bugfixes +-------- + +- Include ``PSF-license.txt`` file for ``plugins/module_utils/_version.py``. +- aws_account_attribute lookup plugin - fix linting errors in documentation data (https://github.com/ansible-collections/amazon.aws/pull/701). +- aws_ec2 inventory plugin - fix linting errors in documentation data (https://github.com/ansible-collections/amazon.aws/pull/701). +- aws_rds inventory plugin - fix linting errors in documentation data (https://github.com/ansible-collections/amazon.aws/pull/701). +- aws_resource_actions callback plugin - fix linting errors in documentation data (https://github.com/ansible-collections/amazon.aws/pull/701). +- aws_secret lookup plugin - fix linting errors in documentation data (https://github.com/ansible-collections/amazon.aws/pull/701). +- aws_service_ip_ranges lookup plugin - fix linting errors in documentation data (https://github.com/ansible-collections/amazon.aws/pull/701). +- aws_ssm - Fix environment variables for client configuration (e.g., AWS_PROFILE, AWS_ACCESS_KEY_ID) (https://github.com/ansible-collections/amazon.aws/pull/837). +- aws_ssm lookup plugin - fix linting errors in documentation data (https://github.com/ansible-collections/amazon.aws/pull/701). +- ec2_instance - ec2_instance module broken in Python 3.8 - dict keys modified during iteration (https://github.com/ansible-collections/amazon.aws/issues/709). +- module.utils.rds - Add waiter for promoting read replica to fix idempotency issue (https://github.com/ansible-collections/amazon.aws/pull/714). +- module.utils.rds - Catch InvalidDBSecurityGroupStateFault when modifying a db instance (https://github.com/ansible-collections/amazon.aws/pull/776). +- module.utils.s3 - Update validate_bucket_name minimum length to 3 (https://github.com/ansible-collections/amazon.aws/pull/802). + +v3.2.0 +====== + +Minor Changes +------------- + +- aws_secret - add pagination for ``bypath`` functionality (https://github.com/ansible-collections/amazon.aws/pull/591). +- ec2_instance - Fix scope of deprecation warning to not show warning when ``state`` in ``absent`` (https://github.com/ansible-collections/amazon.aws/pull/719). +- ec2_vpc_route_table - support associating internet gateways (https://github.com/ansible-collections/amazon.aws/pull/690). +- module_utils.elbv2 - Add support for alb specific attributes and compare_elb_attributes method to support check_mode in module_utils.elbv2 (https://github.com/ansible-collections/amazon.aws/pull/696). +- s3_bucket - Add support for enforced bucket owner object ownership (https://github.com/ansible-collections/amazon.aws/pull/694). + +Bugfixes +-------- + +- aws_ec2 inventory - use the iam_role_arn configuration parameter to assume the role before trying to call DescribeRegions if the regions configuration is not set and AWS credentials provided without enough privilege to perform the DescribeRegions action. (https://github.com/ansible-collections/amazon.aws/issues/566). +- ec2_vol - changing a volume from a type that does not support IOPS (like ``standard``) to a type that does (like ``gp3``) fails (https://github.com/ansible-collections/amazon.aws/issues/626). +- ec2_vpc_igw - fix 'NoneType' object is not subscriptable error (https://github.com/ansible-collections/amazon.aws/pull/691). +- ec2_vpc_igw - use paginator for describe internet gateways and add retry to fix NoneType object is not subscriptable error (https://github.com/ansible-collections/amazon.aws/pull/695). +- ec2_vpc_net - In check mode, ensure the module does not change the configuration. Handle case when Amazon-provided ipv6 block is enabled, then disabled, then enabled again. Do not disable IPv6 CIDR association (using Amazon pool) if ipv6_cidr property is not present in the task. If the VPC already exists and ipv6_cidr property, retain the current config (https://github.com/ansible-collections/amazon.aws/pull/631). + +v3.1.1 +====== + +Minor Changes +------------- + +- bump the release version of the amazon.aws collection from 3.1.0 to 3.1.1 because of a bug that occurred while uploading to Galaxy. + +v3.1.0 +====== + +Minor Changes +------------- + +- add new parameters hostvars_prefix and hostvars_suffix for inventory plugins aws_ec2 and aws_rds (https://github.com/ansible-collections/amazon.aws/issues/535). +- aws_s3 - Add ``validate_bucket_name`` option, to control bucket name validation (https://github.com/ansible-collections/amazon.aws/pull/615). +- aws_s3 - add latest choice on ``overwrite`` parameter to get latest object on S3 (https://github.com/ansible-collections/amazon.aws/pull/595). +- ec2_vol - add support for OutpostArn param (https://github.com/ansible-collections/amazon.aws/pull/597). +- ec2_vol - tag volume on creation (https://github.com/ansible-collections/amazon.aws/pull/603). +- ec2_vpc_route_table - add support for IPv6 in creating route tables (https://github.com/ansible-collections/amazon.aws/pull/601). +- s3_bucket - Add ``validate_bucket_name`` option, to control bucket name validation (https://github.com/ansible-collections/amazon.aws/pull/615). + +Deprecated Features +------------------- + +- ec2_instance - The default value for ```instance_type``` has been deprecated, in the future release you must set an instance_type or a launch_template (https://github.com/ansible-collections/amazon.aws/pull/587). + +Bugfixes +-------- + +- Various modules and plugins - use vendored version of ``distutils.version`` instead of the deprecated Python standard library ``distutils`` (https://github.com/ansible-collections/amazon.aws/pull/599). +- aws_acm - No longer raising ResourceNotFound exception while retrieving ACM certificates. +- aws_s3 - fix exception raised when using module to copy from source to destination and key is missing from source (https://github.com/ansible-collections/amazon.aws/issues/602). +- ec2_instance - Add a condition to handle default ```instance_type``` value for fix breaking on instance creation with launch template (https://github.com/ansible-collections/amazon.aws/pull/587). +- ec2_key - add support for ED25519 key type (https://github.com/ansible-collections/amazon.aws/issues/572). +- ec2_vol - Sets the Iops value in req_obj even if the iops value has not changed, to allow modifying volume types that require passing an iops value to boto. (https://github.com/ansible-collections/amazon.aws/pull/606) +- elb_classic_lb - handle security_group_ids when providing security_group_names and fix broken tasks in integration test (https://github.com/ansible-collections/amazon.aws/pull/592). +- s3_bucket - Enable the management of bucket-level ACLs (https://github.com/ansible-collections/amazon.aws/issues/573). + +v3.0.0 +====== + +Major Changes +------------- + +- amazon.aws collection - The amazon.aws collection has dropped support for ``botocore<1.19.0`` and ``boto3<1.16.0``. Most modules will continue to work with older versions of the AWS SDK, however compatibility with older versions of the SDK is not guaranteed and will not be tested. When using older versions of the SDK a warning will be emitted by Ansible (https://github.com/ansible-collections/amazon.aws/pull/574). + +Minor Changes +------------- + +- ec2_instance - add count parameter support (https://github.com/ansible-collections/amazon.aws/pull/539). + +Breaking Changes / Porting Guide +-------------------------------- + +- aws_caller_facts - Remove deprecated ``aws_caller_facts`` alias. Please use ``aws_caller_info`` instead. +- cloudformation_facts - Remove deprecated ``cloudformation_facts`` alias. Please use ``cloudformation_info`` instead. +- ec2_ami_facts - Remove deprecated ``ec2_ami_facts`` alias. Please use ``ec2_ami_info`` instead. +- ec2_eni_facts - Remove deprecated ``ec2_eni_facts`` alias. Please use ``ec2_eni_info`` instead. +- ec2_group_facts - Remove deprecated ``ec2_group_facts`` alias. Please use ``ec2_group_info`` instead. +- ec2_instance_facts - Remove deprecated ``ec2_instance_facts`` alias. Please use ``ec2_instance_info`` instead. +- ec2_snapshot_facts - Remove deprecated ``ec2_snapshot_facts`` alias. Please use ``ec2_snapshot_info`` instead. +- ec2_vol_facts - Remove deprecated ``ec2_vol_facts`` alias. Please use ``ec2_vol_info`` instead. +- ec2_vpc_dhcp_option_facts - Remove deprecated ``ec2_vpc_dhcp_option_facts`` alias. Please use ``ec2_vpc_dhcp_option_info`` instead. +- ec2_vpc_endpoint_facts - Remove deprecated ``ec2_vpc_endpoint_facts`` alias. Please use ``ec2_vpc_endpoint_info`` instead. +- ec2_vpc_igw_facts - Remove deprecated ``ec2_vpc_igw_facts`` alias. Please use ``ec2_vpc_igw_info`` instead. +- ec2_vpc_nat_gateway_facts - Remove deprecated ``ec2_vpc_nat_gateway_facts`` alias. Please use ``ec2_vpc_nat_gateway_info`` instead. +- ec2_vpc_net_facts - Remove deprecated ``ec2_vpc_net_facts`` alias. Please use ``ec2_vpc_net_info`` instead. +- ec2_vpc_route_table_facts - Remove deprecated ``ec2_vpc_route_table_facts`` alias. Please use ``ec2_vpc_route_table_info`` instead. +- ec2_vpc_subnet_facts - Remove deprecated ``ec2_vpc_subnet_facts`` alias. Please use ``ec2_vpc_subnet_info`` instead. + +Deprecated Features +------------------- + +- module_utils - support for the original AWS SDK ``boto`` has been deprecated in favour of the ``boto3``/``botocore`` SDK. All ``boto`` based modules have either been deprecated or migrated to ``botocore``, and the remaining support code in module_utils will be removed in release 4.0.0 of the amazon.aws collection. Any modules outside of the amazon.aws and community.aws collections based on the ``boto`` library will need to be migrated to the ``boto3``/``botocore`` libraries (https://github.com/ansible-collections/amazon.aws/pull/575). + +v2.2.0 +====== + +Minor Changes +------------- + +- ec2_instance - add count parameter support (https://github.com/ansible-collections/amazon.aws/pull/539). + +Bugfixes +-------- + +- aws_ec2 inventory - use the iam_role_arn configuration parameter to assume the role before trying to call DescribeRegions if the regions configuration is not set and AWS credentials provided without enough privilege to perform the DescribeRegions action. (https://github.com/ansible-collections/amazon.aws/issues/566). +- ec2_vol - Sets the Iops value in req_obj even if the iops value has not changed, to allow modifying volume types that require passing an iops value to boto. (https://github.com/ansible-collections/amazon.aws/pull/606) +- ec2_vol - changing a volume from a type that does not support IOPS (like ``standard``) to a type that does (like ``gp3``) fails (https://github.com/ansible-collections/amazon.aws/issues/626). +- ec2_vpc_igw - fix 'NoneType' object is not subscriptable error (https://github.com/ansible-collections/amazon.aws/pull/691). +- ec2_vpc_igw - use paginator for describe internet gateways and add retry to fix NoneType object is not subscriptable error (https://github.com/ansible-collections/amazon.aws/pull/695). +- elb_classic_lb - handle security_group_ids when providing security_group_names and fix broken tasks in integration test (https://github.com/ansible-collections/amazon.aws/pull/592). + +v2.1.0 +====== + +Minor Changes +------------- + +- aws_service_ip_ranges - add new option ``ipv6_prefixes`` to get only IPV6 addresses and prefixes for Amazon services (https://github.com/ansible-collections/amazon.aws/pull/430) +- cloudformation - fix detection when there are no changes. Sometimes when there are no changes, the change set will have a status FAILED with StatusReason No updates are to be performed (https://github.com/ansible-collections/amazon.aws/pull/507). +- ec2_ami - add check_mode support (https://github.com/ansible-collections/amazon.aws/pull/516). +- ec2_ami - use module_util helper for tagging AMIs (https://github.com/ansible-collections/amazon.aws/pull/520). +- ec2_ami - when creating an AMI from an instance pass the tagging options at creation time (https://github.com/ansible-collections/amazon.aws/pull/551). +- ec2_elb_lb - module renamed to ``elb_classic_lb`` (https://github.com/ansible-collections/amazon.aws/pull/377). +- ec2_eni - add check mode support (https://github.com/ansible-collections/amazon.aws/pull/534). +- ec2_eni - use module_util helper for tagging ENIs (https://github.com/ansible-collections/amazon.aws/pull/522). +- ec2_instance - use module_util helpers for tagging (https://github.com/ansible-collections/amazon.aws/pull/527). +- ec2_key - add support for tagging key pairs (https://github.com/ansible-collections/amazon.aws/pull/548). +- ec2_snapshot - add check_mode support (https://github.com/ansible-collections/amazon.aws/pull/512). +- ec2_vol - add check_mode support (https://github.com/ansible-collections/amazon.aws/pull/509). +- ec2_vpc_dhcp_option - use module_util helpers for tagging (https://github.com/ansible-collections/amazon.aws/pull/531). +- ec2_vpc_endpoint - added ``vpc_endpoint_security_groups`` parameter to support defining the security group attached to an interface endpoint (https://github.com/ansible-collections/amazon.aws/pull/544). +- ec2_vpc_endpoint - added ``vpc_endpoint_subnets`` parameter to support defining the subnet attached to an interface or gateway endpoint (https://github.com/ansible-collections/amazon.aws/pull/544). +- ec2_vpc_endpoint - use module_util helper for tagging (https://github.com/ansible-collections/amazon.aws/pull/525). +- ec2_vpc_endpoint - use module_util helpers for tagging (https://github.com/ansible-collections/amazon.aws/pull/531). +- ec2_vpc_igw - use module_util helper for tagging (https://github.com/ansible-collections/amazon.aws/pull/523). +- ec2_vpc_igw - use module_util helpers for tagging (https://github.com/ansible-collections/amazon.aws/pull/531). +- ec2_vpc_nat_gateway - use module_util helper for tagging (https://github.com/ansible-collections/amazon.aws/pull/524). +- ec2_vpc_nat_gateway - use module_util helpers for tagging (https://github.com/ansible-collections/amazon.aws/pull/531). +- elb_classic_lb - added retries on common AWS temporary API failures (https://github.com/ansible-collections/amazon.aws/pull/377). +- elb_classic_lb - added support for check_mode (https://github.com/ansible-collections/amazon.aws/pull/377). +- elb_classic_lb - added support for wait during creation (https://github.com/ansible-collections/amazon.aws/pull/377). +- elb_classic_lb - added support for wait during instance addition and removal (https://github.com/ansible-collections/amazon.aws/pull/377). +- elb_classic_lb - migrated to boto3 SDK (https://github.com/ansible-collections/amazon.aws/pull/377). +- elb_classic_lb - various error messages changed due to refactor (https://github.com/ansible-collections/amazon.aws/pull/377). +- module_utils.ec2 - moved generic tagging helpers into module_utils.tagging (https://github.com/ansible-collections/amazon.aws/pull/527). +- module_utils.tagging - add new helper to generate TagSpecification lists (https://github.com/ansible-collections/amazon.aws/pull/527). + +Deprecated Features +------------------- + +- ec2_classic_lb - setting of the ``ec2_elb`` fact has been deprecated and will be removed in release 4.0.0 of the collection. The module now returns ``elb`` which can be accessed using the register keyword (https://github.com/ansible-collections/amazon.aws/pull/552). + +Bugfixes +-------- + +- AWS action group - added missing ``ec2_instance_facts`` entry (https://github.com/ansible-collections/amazon.aws/issues/557) +- ec2_ami - fix problem when creating an AMI from an instance with ephemeral volumes (https://github.com/ansible-collections/amazon.aws/issues/511). +- ec2_instance - ensure that ec2_instance falls back to the tag(Name) parameter when no filter and no name parameter is passed (https://github.com/ansible-collections/amazon.aws/issues/526). +- s3_bucket - update error handling to better support DigitalOcean Space (https://github.com/ansible-collections/amazon.aws/issues/508). + +v2.0.0 +====== + +Major Changes +------------- + +- amazon.aws collection - Due to the AWS SDKs announcing the end of support for Python less than 3.6 (https://boto3.amazonaws.com/v1/documentation/api/1.17.64/guide/migrationpy3.html) this collection now requires Python 3.6+ (https://github.com/ansible-collections/amazon.aws/pull/298). +- amazon.aws collection - The amazon.aws collection has dropped support for ``botocore<1.18.0`` and ``boto3<1.15.0``. Most modules will continue to work with older versions of the AWS SDK, however compatibility with older versions of the SDK is not guaranteed and will not be tested. When using older versions of the SDK a warning will be emitted by Ansible (https://github.com/ansible-collections/amazon.aws/pull/502). +- ec2_instance - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_instance``. +- ec2_instance_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_instance_info``. +- ec2_vpc_endpoint - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_vpc_endpoint``. +- ec2_vpc_endpoint_facts - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_vpc_endpoint_info``. +- ec2_vpc_endpoint_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_vpc_endpoint_info``. +- ec2_vpc_endpoint_service_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_vpc_endpoint_service_info``. +- ec2_vpc_igw - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_vpc_igw``. +- ec2_vpc_igw_facts - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_vpc_igw_facts``. +- ec2_vpc_igw_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_vpc_igw_info``. +- ec2_vpc_nat_gateway - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_vpc_nat_gateway``. +- ec2_vpc_nat_gateway_facts - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_vpc_nat_gateway_info``. +- ec2_vpc_nat_gateway_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_vpc_nat_gateway_info``. +- ec2_vpc_route_table - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_vpc_route_table``. +- ec2_vpc_route_table_facts - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_vpc_route_table_facts``. +- ec2_vpc_route_table_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_vpc_route_table_info``. + +Minor Changes +------------- + +- aws_ec2 - use a generator rather than list comprehension (https://github.com/ansible-collections/amazon.aws/pull/465). +- aws_s3 - Tests for compatibility with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442). +- aws_s3 - Tests for compatibility with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442). +- aws_s3 - add ``tags`` and ``purge_tags`` features for an S3 object (https://github.com/ansible-collections/amazon.aws/pull/335) +- aws_s3 - new mode to copy existing on another bucket (https://github.com/ansible-collections/amazon.aws/pull/359). +- aws_secret - added support for gracefully handling deleted secrets (https://github.com/ansible-collections/amazon.aws/pull/455). +- aws_ssm - add "on_missing" and "on_denied" option (https://github.com/ansible-collections/amazon.aws/pull/370). +- cloudformation - Tests for compatibility with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442). +- cloudformation - Tests for compatibility with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442). +- ec2_ami - ensure tags are propagated to the snapshot(s) when creating an AMI (https://github.com/ansible-collections/amazon.aws/pull/437). +- ec2_eni - fix idempotency when ``security_groups`` attribute is specified (https://github.com/ansible-collections/amazon.aws/pull/337). +- ec2_eni - timeout increased when waiting for ENIs to finish detaching (https://github.com/ansible-collections/amazon.aws/pull/501). +- ec2_group - Tests for compatibility with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442). +- ec2_group - Tests for compatibility with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442). +- ec2_group - use a generator rather than list comprehension (https://github.com/ansible-collections/amazon.aws/pull/465). +- ec2_group - use system ipaddress module, available with Python >= 3.3, instead of vendored copy (https://github.com/ansible-collections/amazon.aws/pull/461). +- ec2_instance - Tests for compatibility with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442). +- ec2_instance - Tests for compatibility with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442). +- ec2_instance - add ``throughput`` parameter for gp3 volume types (https://github.com/ansible-collections/amazon.aws/pull/433). +- ec2_instance - add support for controlling metadata options (https://github.com/ansible-collections/amazon.aws/pull/414). +- ec2_instance - remove unnecessary raise when exiting with a failure (https://github.com/ansible-collections/amazon.aws/pull/460). +- ec2_instance_info - Tests for compatibility with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442). +- ec2_instance_info - Tests for compatibility with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442). +- ec2_snapshot - migrated to use the boto3 python library (https://github.com/ansible-collections/amazon.aws/pull/356). +- ec2_spot_instance_info - Added a new module that describes the specified Spot Instance requests (https://github.com/ansible-collections/amazon.aws/pull/487). +- ec2_vol - add parameter ``multi_attach`` to support Multi-Attach on volume creation/update (https://github.com/ansible-collections/amazon.aws/pull/362). +- ec2_vol - relax the boto3/botocore requirements and only require botocore 1.19.27 for modifying the ``throughput`` parameter (https://github.com/ansible-collections/amazon.aws/pull/346). +- ec2_vpc_dhcp_option - Now also returns a boto3-style resource description in the ``dhcp_options`` result key. This includes any tags for the ``dhcp_options_id`` and has the same format as the current return value of ``ec2_vpc_dhcp_option_info``. (https://github.com/ansible-collections/amazon.aws/pull/252) +- ec2_vpc_dhcp_option_info - Now also returns a user-friendly ``dhcp_config`` key that matches the historical ``new_config`` key from ec2_vpc_dhcp_option, and alleviates the need to use ``items2dict(key_name='key', value_name='values')`` when parsing the output of the module. (https://github.com/ansible-collections/amazon.aws/pull/252) +- ec2_vpc_subnet - Tests for compatibility with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442). +- ec2_vpc_subnet - Tests for compatibility with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442). +- integration tests - remove dependency with collection ``community.general`` (https://github.com/ansible-collections/amazon.aws/pull/361). +- module_utils/waiter - add RDS cluster ``cluster_available`` waiter (https://github.com/ansible-collections/amazon.aws/pull/464). +- module_utils/waiter - add RDS cluster ``cluster_deleted`` waiter (https://github.com/ansible-collections/amazon.aws/pull/464). +- module_utils/waiter - add Route53 ``resource_record_sets_changed`` waiter (https://github.com/ansible-collections/amazon.aws/pull/350). +- s3_bucket - Tests for compatibility with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442). +- s3_bucket - Tests for compatibility with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442). +- s3_bucket - add new option ``object_ownership`` to configure object ownership (https://github.com/ansible-collections/amazon.aws/pull/311) +- s3_bucket - updated to use HeadBucket instead of ListBucket when testing for bucket existence (https://github.com/ansible-collections/amazon.aws/pull/357). + +Breaking Changes / Porting Guide +-------------------------------- + +- ec2_instance - instance wait for state behaviour has changed. If plays require the old behavior of waiting for the instance monitoring status to become ``OK`` when launching a new instance, the action will need to specify ``state: started`` (https://github.com/ansible-collections/amazon.aws/pull/481). +- ec2_snapshot - support for waiting indefinitely has been dropped, new default is 10 minutes (https://github.com/ansible-collections/amazon.aws/pull/356). +- ec2_vol_info - return ``attachment_set`` is now a list of attachments with Multi-Attach support on disk. (https://github.com/ansible-collections/amazon.aws/pull/362). +- ec2_vpc_dhcp_option - The module has been refactored to use boto3. Keys and value types returned by the module are now consistent, which is a change from the previous behaviour. A ``purge_tags`` option has been added, which defaults to ``True``. (https://github.com/ansible-collections/amazon.aws/pull/252) +- ec2_vpc_dhcp_option_info - Now preserves case for tag keys in return value. (https://github.com/ansible-collections/amazon.aws/pull/252) +- module_utils.core - The boto3 switch has been removed from the region parameter (https://github.com/ansible-collections/amazon.aws/pull/287). +- module_utils/compat - vendored copy of ipaddress removed (https://github.com/ansible-collections/amazon.aws/pull/461). +- module_utils/core - updated the ``scrub_none_parameters`` function so that ``descend_into_lists`` is set to ``True`` by default (https://github.com/ansible-collections/amazon.aws/pull/297). + +Deprecated Features +------------------- + +- ec2 - the boto based ``ec2`` module has been deprecated in favour of the boto3 based ``ec2_instance`` module. The ``ec2`` module will be removed in release 4.0.0 (https://github.com/ansible-collections/amazon.aws/pull/424). +- ec2_vpc_dhcp_option - The ``new_config`` return key has been deprecated and will be removed in a future release. It will be replaced by ``dhcp_config``. Both values are returned in the interim. (https://github.com/ansible-collections/amazon.aws/pull/252) + +Bugfixes +-------- + +- aws_s3 - Fix upload permission when an S3 bucket ACL policy requires a particular canned ACL (https://github.com/ansible-collections/amazon.aws/pull/318) +- ec2_ami - Fix ami issue when creating an ami with no_device parameter (https://github.com/ansible-collections/amazon.aws/pull/386) +- ec2_instance - ``ec2_instance`` was waiting on EC2 instance monitoring status to be ``OK`` when launching a new instance. This could cause a play to wait multiple minutes for AWS's monitoring to complete status checks (https://github.com/ansible-collections/amazon.aws/pull/481). +- ec2_snapshot - Fix snapshot issue when capturing a snapshot of a volume without tags (https://github.com/ansible-collections/amazon.aws/pull/383) +- ec2_vol - Fixes ``changed`` status when ``modify_volume`` is used, but no new disk is being attached. The module incorrectly reported that no change had occurred even when disks had been modified (iops, throughput, type, etc.). (https://github.com/ansible-collections/amazon.aws/issues/482). +- ec2_vol - fix iops setting and enforce iops/throughput parameters usage (https://github.com/ansible-collections/amazon.aws/pull/334) +- inventory - ``include_filters`` won't be ignored anymore if ``filters`` is not set (https://github.com/ansible-collections/amazon.aws/issues/457). +- s3_bucket - Fix error handling when attempting to set a feature that is not implemented (https://github.com/ansible-collections/amazon.aws/pull/391). +- s3_bucket - Gracefully handle ``NotImplemented`` exceptions when fetching encryption settings (https://github.com/ansible-collections/amazon.aws/issues/390). + +New Modules +----------- + +- ec2_spot_instance - request, stop, reboot or cancel spot instance +- ec2_spot_instance_info - Gather information about ec2 spot instance requests + +v1.5.0 +====== + +Minor Changes +------------- + +- AWS inventory plugins - use shared HAS_BOTO3 helper rather than copying code (https://github.com/ansible-collections/amazon.aws/pull/288). +- AWS lookup plugins - use shared HAS_BOTO3 helper rather than copying code (https://github.com/ansible-collections/amazon.aws/pull/288). +- aws_account_attribute - add retries on common AWS failures (https://github.com/ansible-collections/amazon.aws/pull/295). +- aws_ec2 inventory - expose a new configuration key ``use_contrib_script_compatible_ec2_tag_keys`` to reproduce a behavior of the old ``ec2.py`` inventory script. With this option enabled, each tag is exposed using a ``ec2_tag_TAGNAME`` key (https://github.com/ansible-collections/amazon.aws/pull/331). +- aws_ec2 inventory - expose to new keys called ``include_filters`` and ``exclude_filters`` to give the user the ability to compose an inventory with multiple queries (https://github.com/ansible-collections/amazon.aws/pull/328). +- aws_ec2 inventory plugin - Added support for using Jinja2 templates in the authentication fields (https://github.com/ansible-collections/amazon.aws/pull/57). +- cloudformation - added support for StackPolicyDuringUpdateBody (https://github.com/ansible-collections/amazon.aws/pull/155). +- ec2_metadata_facts - add support for IMDSv2 (https://github.com/ansible-collections/amazon.aws/pull/43). +- ec2_snapshot_info - add the ``max_results`` along with ``next_token_id`` option (https://github.com/ansible-collections/amazon.aws/pull/321). +- ec2_tag - use common code for tagging resources (https://github.com/ansible-collections/amazon.aws/pull/309). +- ec2_tag_info - use common code for tagging resources (https://github.com/ansible-collections/amazon.aws/pull/309). +- ec2_vol - add the ``purge_tags`` option (https://github.com/ansible-collections/amazon.aws/pull/242). +- ec2_vol - use common code for tagging resources (https://github.com/ansible-collections/amazon.aws/pull/309). +- ec2_vpc_net - use a custom waiter which can handle API rate limiting (https://github.com/ansible-collections/amazon.aws/pull/270). +- ec2_vpc_subnet - use AWSRetry decorator to more consistently handle API rate limiting (https://github.com/ansible-collections/amazon.aws/pull/270). +- ec2_vpc_subnet - use common code for tagging resources (https://github.com/ansible-collections/amazon.aws/pull/309). +- module_utils.cloudfront_facts - linting cleanup (https://github.com/ansible-collections/amazon.aws/pull/291). +- module_utils.ec2 - linting cleanup (https://github.com/ansible-collections/amazon.aws/pull/291). +- module_utils/core - add a helper function ``normalize_boto3_result`` (https://github.com/ansible-collections/amazon.aws/pull/271). +- module_utils/core - add parameter ``descend_into_lists`` to ``scrub_none_parameters`` helper function (https://github.com/ansible-collections/amazon.aws/pull/262). +- module_utils/ec2 - added additional helper functions for tagging EC2 resources (https://github.com/ansible-collections/amazon.aws/pull/309). +- sanity tests - add ignore.txt for 2.12 (https://github.com/ansible-collections/amazon.aws/pull/315). + +Bugfixes +-------- + +- ec2_vol - create or update now preserves the existing tags, including Name (https://github.com/ansible-collections/amazon.aws/issues/229) +- ec2_vol - fix exception when platform information isn't available (https://github.com/ansible-collections/amazon.aws/issues/305). + +v1.4.1 +====== + +Minor Changes +------------- + +- module_utils - the ipaddress module utility has been vendored into this collection. This eliminates the collection dependency on ansible.netcommon (which had removed the library in its 2.0 release). The ipaddress library is provided for internal use in this collection only. (https://github.com/ansible-collections/amazon.aws/issues/273)- + +v1.4.0 +====== + +Minor Changes +------------- + +- aws_ec2 - Add hostname options concatenation +- aws_ec2 inventory plugin - avoid a superfluous import of ``ansible.utils.display.Display`` (https://github.com/ansible-collections/amazon.aws/pull/226). +- aws_ec2 module - Replace inverse aws instance-state-name filters !terminated, !shutting-down in favor of postive filters pending, running, stopping, stopped. Issue 235. (https://github.com/ansible-collections/amazon.aws/pull/237) +- aws_secret - add ``bypath`` functionality (https://github.com/ansible-collections/amazon.aws/pull/192). +- ec2_key - add AWSRetry decorator to automatically retry on common temporary failures (https://github.com/ansible-collections/amazon.aws/pull/213). +- ec2_vol - Add support for gp3 volumes and support for modifying existing volumes (https://github.com/ansible-collections/amazon.aws/issues/55). +- module_utils/elbv2 - add logic to compare_rules to suit Values list nested within dicts unique to each field type. Fixes issue (https://github.com/ansible-collections/amazon.aws/issues/187) +- various AWS plugins and module_utils - Cleanup unused imports (https://github.com/ansible-collections/amazon.aws/pull/217). + +Bugfixes +-------- + +- ec2_vol - a creation or update now returns a structure with an up to date list of tags (https://github.com/ansible-collections/amazon.aws/pull/241). + +v1.3.0 +====== + +Minor Changes +------------- + +- aws_caller_info - add AWSRetry decorator to automatically retry on common temporary failures (https://github.com/ansible-collections/amazon.aws/pull/208) +- aws_s3 - Add support for uploading templated content (https://github.com/ansible-collections/amazon.aws/pull/20). +- aws_secret - add "on_missing" and "on_denied" option (https://github.com/ansible-collections/amazon.aws/pull/122). +- ec2_ami - Add retries for ratelimiting related errors (https://github.com/ansible-collections/amazon.aws/pull/195). +- ec2_ami - fixed and streamlined ``max_attempts`` logic when waiting for AMI creation to finish (https://github.com/ansible-collections/amazon.aws/pull/194). +- ec2_ami - increased default ``wait_timeout`` to 1200 seconds (https://github.com/ansible-collections/amazon.aws/pull/194). +- ec2_ami_info - Add retries for ratelimiting related errors (https://github.com/ansible-collections/amazon.aws/pull/195). +- ec2_eni - Improve reliability of the module by adding waiters and performing lookups by ENI ID rather than repeated searches (https://github.com/ansible-collections/amazon.aws/pull/180). +- ec2_eni_info - Improve reliability of the module by adding waiters and performing lookups by ENI ID rather than repeated searches (https://github.com/ansible-collections/amazon.aws/pull/180). +- ec2_group - add AWSRetry decorator to automatically retry on common temporary failures (https://github.com/ansible-collections/amazon.aws/pull/207) +- ec2_group_info - add AWSRetry decorator to automatically retry on common temporary failures (https://github.com/ansible-collections/amazon.aws/pull/207) +- ec2_snapshot_info - add AWSRetry decorator to automatically retry on common temporary failures (https://github.com/ansible-collections/amazon.aws/pull/208) +- ec2_vol - Add automatic retries on AWS rate limit errors (https://github.com/ansible-collections/amazon.aws/pull/199). +- ec2_vol - ported ec2_vol to use boto3 (https://github.com/ansible-collections/amazon.aws/pull/53). +- ec2_vpc_dhcp_option_info - add AWSRetry decorator to automatically retry on common temporary failures (https://github.com/ansible-collections/amazon.aws/pull/208) +- module_utils/core - add helper function ``scrub_none_parameters`` to remove params set to ``None`` (https://github.com/ansible-collections/community.aws/issues/251). +- module_utils/waiters - Add retries to our waiters for the same failure codes that we retry with AWSRetry (https://github.com/ansible-collections/amazon.aws/pull/185) +- s3_bucket - Add support for managing the ``public_access`` settings (https://github.com/ansible-collections/amazon.aws/pull/171). + +Bugfixes +-------- + +- ec2 - Code fix so module can create ec2 instances with ``ec2_volume_iops`` option (https://github.com/ansible-collections/amazon.aws/pull/177). +- ec2 - ignore terminated instances and instances that are shutting down when starting and stopping (https://github.com/ansible-collections/amazon.aws/issues/146). +- ec2_group - Fixes error handling during tagging failures (https://github.com/ansible-collections/amazon.aws/issues/210). +- ec2_group_info - Code fix so module works with Python 3.8 (make dict immutable in loop) (https://github.com/ansible-collections/amazon.aws/pull/181) + +v1.2.1 +====== + +Minor Changes +------------- + +- ec2_eni - Add support for tagging. +- ec2_eni - Port ec2_eni module to boto3 and add an integration test suite. +- ec2_eni_info - Add retries on transient AWS failures. +- ec2_eni_info - Add support for providing an ENI ID. + +v1.2.0 +====== + +Minor Changes +------------- + +- ec2 module_utils - Update ``ec2_connect`` (boto2) behaviour so that ``ec2_url`` overrides ``region``. +- module_utils.core - Support passing arbitrary extra keys to fail_json_aws, matching capabilities of fail_json. + +Deprecated Features +------------------- + +- All AWS Modules - ``aws_access_key``, ``aws_secret_key`` and ``security_token`` will be made mutually exclusive with ``profile`` after 2022-06-01. + +Bugfixes +-------- + +- ec2 module_utils - Ensure boto3 verify parameter isn't overridden by setting a profile (https://github.com/ansible-collections/amazon.aws/issues/129) +- s3_bucket - Ceph compatibility: treat error code NoSuchTagSetError used by Ceph synonymously to NoSuchTagSet used by AWS + +v1.1.0 +====== + +Major Changes +------------- + +- ec2 module_utils - The ``AWSRetry`` decorator no longer catches ``NotFound`` exceptions by default. ``NotFound`` exceptions need to be explicitly added using ``catch_extra_error_codes``. Some AWS modules may see an increase in transient failures due to AWS''s eventual consistency model. + +Minor Changes +------------- + +- Add ``aws_security_token``, ``aws_endpoint_url`` and ``endpoint_url`` aliases to improve AWS module parameter naming consistency. +- Add support for ``aws_ca_bundle`` to boto3 based AWS modules +- Add support for configuring boto3 profiles using ``AWS_PROFILE`` and ``AWS_DEFAULT_PROFILE`` +- Added check_mode support to aws_az_info +- Added check_mode support to ec2_eni_info +- Added check_mode support to ec2_snapshot_info +- ansible_dict_to_boto3_filter_list - convert integers and bools to strings before using them in filters. +- aws_direct_connect_virtual_interface - add direct_connect_gateway_id parameter. This field is only applicable in private VIF cases (public=False) and is mutually exclusive to virtual_gateway_id. +- cloudformation - Return change_set_id in the cloudformation output if a change set was created. +- ec2 - deprecate allowing both group and group_id - currently we ignore group_id if both are passed. +- ec2_ami_info - allow integer and bool values for filtering images (https://github.com/ansible/ansible/issues/43570). +- ec2_asg - Add support for Max Instance Lifetime +- ec2_asg - Add the ability to use mixed_instance_policy in launch template driven autoscaling groups +- ec2_asg - Migrated to AnsibleAWSModule +- ec2_placement_group - make ``name`` a required field. +- ec2_vol_info - Code cleanup and use of the AWSRetry decorator to improve stability +- ec2_vpc_net - Enable IPv6 CIDR assignment + +Breaking Changes / Porting Guide +-------------------------------- + +- aws_s3 - can now delete versioned buckets even when they are not empty - set mode to delete to delete a versioned bucket and everything in it. + +Deprecated Features +------------------- + +- cloudformation - The ``template_format`` option had no effect since Ansible 2.3 and will be removed after 2022-06-01 +- cloudformation - the ``template_format`` option has been deprecated and will be removed in a later release. It has been ignored by the module since Ansible 2.3. +- data_pipeline - The ``version`` option had no effect and will be removed in after 2022-06-01 +- ec2 - in a later release, the ``group`` and ``group_id`` options will become mutually exclusive. Currently ``group_id`` is ignored if you pass both. +- ec2_ami - The ``no_device`` alias ``NoDevice`` has been deprecated and will be removed after 2022-06-01 +- ec2_ami - The ``virtual_name`` alias ``VirtualName`` has been deprecated and will be removed after 2022-06-01 +- ec2_eip - The ``wait_timeout`` option had no effect and will be removed after 2022-06-01 +- ec2_key - The ``wait_timeout`` option had no effect and will be removed after 2022-06-01 +- ec2_key - The ``wait`` option had no effect and will be removed after 2022-06-01 +- ec2_key - the ``wait_timeout`` option has been deprecated and will be removed in a later release. It has had no effect since Ansible 2.5. +- ec2_key - the ``wait`` option has been deprecated and will be removed in a later release. It has had no effect since Ansible 2.5. +- ec2_lc - The ``associate_public_ip_address`` option had no effect and will be removed after 2022-06-01 +- ec2_tag - deprecate the ``list`` option in favor of ec2_tag_info +- ec2_tag - support for ``list`` as a state has been deprecated and will be removed in a later release. The ``ec2_tag_info`` can be used to fetch the tags on an EC2 resource. + +Bugfixes +-------- + +- aws_ec2 - fix idempotency when managing tags +- aws_ec2 - fix idempotency when metrics are enable +- aws_s3 - Delete objects and delete markers so versioned buckets can be removed. +- aws_s3 - Try to wait for the bucket to exist before setting the access control list. +- cloudformation_info - Fix a KeyError returning information about the stack(s). +- ec2_asg - Ensure "wait" is honored during replace operations +- ec2_launch_template - Update output to include latest_version and default_version, matching the documentation +- ec2_transit_gateway - Use AWSRetry before ClientError is handled when describing transit gateways +- ec2_transit_gateway - fixed issue where auto_attach set to yes was not being honored (https://github.com/ansible/ansible/issues/61907) +- ec2_vol - fix filtering bug +- s3_bucket - Accept XNotImplemented response to support NetApp StorageGRID. diff --git a/ansible_collections/amazon/aws/docs/docsite/rst/aws_ec2_guide.rst b/ansible_collections/amazon/aws/docs/docsite/rst/aws_ec2_guide.rst new file mode 100644 index 000000000..3891aec2e --- /dev/null +++ b/ansible_collections/amazon/aws/docs/docsite/rst/aws_ec2_guide.rst @@ -0,0 +1,590 @@ +.. _ansible_collections.amazon.aws.docsite.dynamic_inventory: + + +Dynamic Inventory Plugin +======================== + +A dynamic inventory plugin allows users to point at data sources to compile the inventory of hosts that Ansible uses to target tasks, either via the ``-i /path/to/file`` and/or ``-i 'host1, host2'`` command line parameters or from other configuration sources. + +When using Ansible with AWS, inventory file maintenance will be a hectic task as AWS frequently changes IPs, autoscaling instances, and more. +Once your AWS EC2 hosts are spun up, you'll probably want to talk to them again. +With a cloud setup, it's best not to maintain a static list of cloud hostnames in text files. +Rather, the best way to handle this is to use the ``aws_ec2`` dynamic inventory plugin. + +The ``aws_ec2`` dynamic inventory plugin makes API calls to AWS to get a list of inventory hosts from Amazon Web Services EC2 in the run time. +It gives the EC2 instance details dynamically to manage the AWS infrastructure. + +The plugin will also return instances that were created outside of Ansible and allow Ansible to manage them. + +To start using the ``aws_ec2`` dynamic inventory plugin with a YAML configuration source, create a file with the accepted filename schema documented for the plugin (a YAML configuration file that ends with ``aws_ec2.(yml|yaml)``, e.g., ``demo.aws_ec2.yml``), then add ``plugin: amazon.aws.aws_ec2``. Use the fully qualified name if the plugin is in a collection. + +.. _ansible_collections.amazon.aws.docsite.using_inventory_plugin: + +Authentication +============== + +If your Ansible controller is not in AWS, authentication is handled by either +specifying your access and secret key as ENV variables or inventory plugin arguments. + +For environment variables: + +.. code-block:: bash + + export AWS_ACCESS_KEY_ID='AK123' + export AWS_SECRET_ACCESS_KEY='abc123' + +The ``AWS_SECURITY_TOKEN`` environment variable can also be used, but is only supported for backward compatibility. +The ``AWS_SECURITY_TOKEN`` is a replacement for ``AWS_SESSION_TOKEN`` and it is only needed when you are using temporary credentials. + +Or you can set ``aws_access_key``, ``aws_secret_key``, and ``security_token`` inside the inventory configuration file. + +.. code-block:: yaml + + # demo.aws_ec2.yml + plugin: amazon.aws.aws_ec2 + + # The access key for your AWS account. + aws_access_key: <YOUR-AWS-ACCESS-KEY-HERE> + # The secret access key for your AWS account. + aws_secret_key: <YOUR-AWS-SECRET-KEY-HERE> + +If you use different credentials for different tools or applications, you can use profiles. + +The ``profile`` argument is mutually exclusive with the ``aws_access_key``, ``aws_secret_key`` and ``security_token`` options. +When no credentials are explicitly provided then the AWS SDK (boto3) which Ansible uses will fall back to its configuration files (typically ``~/.aws/credentials``). +The shared credentials file has a default location of ``~/.aws/credentials``. +You can change the location of the shared credentials file by setting the ``AWS_SHARED_CREDENTIALS_FILE`` environment variable. + +.. code-block:: yaml + + # demo.aws_ec2.yml + plugin: amazon.aws.aws_ec2 + + # Attach the default AWS profile + aws_profile: default + + # You could use Jinja2 to attach the AWS profile from the environment variable. + aws_profile: "{{ lookup('env', 'AWS_PROFILE') | default('dev-profile', true) }}" + +You can also set your AWS profile as an ENV variable: + +.. code-block:: bash + + export AWS_PROFILE='test-profile' + + +If your Ansible controller is running on an EC2 instance with an assigned IAM Role, the credential may be omitted. +See the documentation for the controller `for more details <https://docs.ansible.com/ansible-tower/latest/html/userguide/inventories.html#ug-source-ec2>`_. + +You can also use the ARN of the IAM role to assume to perform the inventory lookup. +This can be useful for connecting across different accounts, or to limit user access. +To do so, you should specify the ``iam_role_arn``. +You should still provide AWS credentials with enough privilege to perform the AssumeRole action. + +.. code-block:: yaml + + # demo.aws_ec2.yml + plugin: amazon.aws.aws_ec2 + + iam_role_arn: arn:aws:iam::1234567890:role/assumed-ansible + + +Minimal Example +=============== + +Fetch all hosts in us-east-1, the hostname is the public DNS if it exists, otherwise the private IP address. + +.. code-block:: yaml + + # demo.aws_ec2.yml + plugin: amazon.aws.aws_ec2 + + # This sets the region. If empty (the default) default this will include all regions, except possibly + # restricted ones like us-gov-west-1 and cn-north-1. + regions: + - us-east-1 + +After providing any required options, you can view the populated inventory with ``ansible-inventory -i demo.aws_ec2.yml --graph``: + +.. code-block:: text + + @all: + |--@aws_ec2: + | |--ip-10-210-0-189.ec2.internal + | |--ip-10-210-0-195.ec2.internal + |--@ungrouped: + + +Allowed Options +=============== + +Some of the ``aws_ec2`` dynamic inventory plugin options are explained in detail below. For a full list see `the plugin documentation <https://docs.ansible.com/ansible/latest/collections/amazon/aws/aws_ec2_inventory.html#id3>`_. + +``hostnames`` +------------- + +``hostnames`` option provides different settings to choose how the hostname will be displayed. + +Some examples are shown below: + +.. code-block:: yaml + + hostnames: + # This option allows displaying the public ip addresses. + - ip-address + + # This option allows displaying the private ip addresses using `tag:Name` as a prefix. + # `name` can be one of the options specified in http://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instances.html#options. + - name: 'private-ip-address' + separator: '_' + prefix: 'tag:Name' + + # Using literal values for hostname + # # Hostname will be aws-test_literal + - name: 'test_literal' + separator: '-' + prefix: 'aws' + + # To use tags as hostnames use the syntax `tag:Name=Value` to use the hostname `Name_Value`, or + # `tag:Name` to use the value of the Name tag. If value provided does not exist in the above options, + # it will be used as a literal string. + - name: 'tag:Tag1=Test1,Tag2=Test2' + + # Use dns-name attribute as hostname + - dns-name + + # You can also specify a list in order of precedence for hostname variables. + - ip-address + - dns-name + - tag:Name + - private-ip-address + +By default, the inventory will only return the first match one of the ``hostnames`` entries. +You may want to get all the potential matches in your inventory, this also implies you will get +duplicated entries. To switch to this behavior, set the ``allow_duplicated_hosts`` configuration key to ``True``. + +``keyed_groups`` +---------------- + +You can create dynamic groups using host variables with the ``keyed_groups`` option. ``keyed_groups`` comes in a prefix and a key format. +The prefix will be the name of the host group that is to be concatenated with the key. + +Some examples are shown below: + +.. code-block:: yaml + + keyed_groups: + # This creates host groups based on architecture. + - prefix: arch + key: architecture + + # This creates host groups based on `x86_64` architecture. + - prefix: arch + key: architecture + value: + 'x86_64' + + # This creates host groups based on availability zone. + - prefix: az + key: placement.availability_zone + + # If the EC2 tag Name had the value `redhat` the tag variable would be: `tag_Name_redhat`. + # Similarly, if a tag existed for an AWS EC2 instance as `Applications` with the value of `nodejs` the + # variable would be: `tag_Applications_nodejs`. + - prefix: tag + key: tags + + # This creates host groups using instance_type, e.g., `instance_type_z3_tiny`. + - prefix: instance_type + key: instance_type + + # This creates host groups using security_groups id, e.g., `security_groups_sg_abcd1234` group for each security group. + - key: 'security_groups|json_query("[].group_id")' + prefix: 'security_groups' + + # This creates a host group for each value of the Application tag. + - key: tags.Application + separator: '' + + # This creates a host group per region e.g., `aws_region_us_east_2`. + - key: placement.region + prefix: aws_region + + # This creates host groups based on the value of a custom tag `Role` and adds them to a metagroup called `project`. + - key: tags['Role'] + prefix: foo + parent_group: "project" + + # This creates a common parent group for all EC2 availability zones. + - key: placement.availability_zone + parent_group: all_ec2_zones + + # This creates a group per distro (distro_CentOS, distro_Debian) and assigns the hosts that have matching values to it, + # using the default separator "_". + - prefix: distro + key: ansible_distribution + + +``groups`` +---------- + +It is also possible to create groups using the ``groups`` option. + +Some examples are shown below: + +.. code-block:: yaml + + groups: + # This created two groups - `Production` and `PreProduction` based on tags + # These conditionals are expressed using Jinja2 syntax. + redhat: "'Production' in tags.Environment" + ubuntu: "'PreProduction' in tags.Environment" + + # This created a libvpc group based on specific condition on `vpc_id`. + libvpc: vpc_id == 'vpc-####' + + +``compose`` +----------- + +``compose`` creates and modifies host variables from Jinja2 expressions. + +.. code-block:: yaml + + compose: + # This sets the ansible_host variable to connect with the private IP address without changing the hostname. + ansible_host: private_ip_address + + # This sets location_vars variable as a dictionary with location as a key. + location_vars: + location: "east_coast" + server_type: "ansible_hostname | regex_replace ('(.{6})(.{2}).*', '\\2')" + + # This sets location variable. + location: "'east_coast'" + + # This lets you connect over SSM to the instance id. + ansible_host: instance_id + ansible_connection: 'community.aws.aws_ssm' + + # This defines combinations of host servers, IP addresses, and related SSH private keys. + ansible_host: private_ip_address + ansible_user: centos + ansible_ssh_private_key_file: /path/to/private_key_file + + # This sets the ec2_security_group_ids variable. + ec2_security_group_ids: security_groups | map(attribute='group_id') | list | join(',') + + # Host variables that are strings need to be wrapped with two sets of quotes. + # See https://docs.ansible.com/ansible/latest/plugins/inventory.html#using-inventory-plugins for details. + ansible_connection: '"community.aws.aws_ssm"' + ansible_user: '"ssm-user"' + + +``include_filters`` and ``exclude_filters`` +------------------------------------------- + +``include_filters`` and ``exclude_filters`` options give you the ability to compose the inventory with several queries (see `available filters <http://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instances.html#options>`_). + +.. code-block:: yaml + + include_filters: + # This includes everything in the inventory that has the following tags. + - tag:Project: + - 'planets' + - tag:Environment: + - 'demo' + + # This excludes everything from the inventory that has the following tag:Name. + exclude_filters: + - tag:Name: + - '{{ resource_prefix }}_3' + + +``filters`` +----------- + +``filters`` are used to filter out AWS EC2 instances based on conditions (see `available filters <http://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instances.html#options>`_). + +.. code-block:: yaml + + filters: + # This selects only running instances with tag `Environment` tag set to `dev`. + tag:Environment: dev + instance-state-name : running + + # This selects only instances with tag `Environment` tag set to `dev` and `qa` and specific security group id. + tag:Environment: + - dev + - qa + instance.group-id: sg-xxxxxxxx + + # This selects only instances with tag `Name` fulfilling specific conditions. + - tag:Name: + - dev-* + - share-resource + - hotfix + + +``use_contrib_script_compatible_ec2_tag_keys`` and ``use_contrib_script_compatible_sanitization`` +------------------------------------------------------------------------------------------------- + +``use_contrib_script_compatible_ec2_tag_keys`` exposes the host tags with ec2_tag_TAGNAME keys like the old ec2.py inventory script when it's True. + +By default the ``aws_ec2`` plugin is using a general group name sanitization to create safe and usable group names for use in Ansible. + +``use_contrib_script_compatible_ec2_tag_keys`` allows you to override that, in efforts to allow migration from the old inventory script and matches the sanitization of groups when the script's replace_dash_in_groups option is set to False. +To replicate behavior of replace_dash_in_groups = True with constructed groups, you will need to replace hyphens with underscores via the regex_replace filter for those entries. + +For this to work you should also turn off the TRANSFORM_INVALID_GROUP_CHARS setting, otherwise the core engine will just use the standard sanitization on top. + +This is not the default as such names break certain functionality as not all characters are valid Python identifiers which group names end up being used as. + +The use of this feature is discouraged and we advise to migrate to the new tags structure. + +.. code-block:: yaml + + # demo.aws_ec2.yml + plugin: amazon.aws.aws_ec2 + regions: + - us-east-1 + filters: + tag:Name: + - 'instance-*' + hostnames: + - tag:Name + use_contrib_script_compatible_sanitization: True + use_contrib_script_compatible_ec2_tag_keys: True + +After providing any required options, you can view the populated inventory with ``ansible-inventory -i demo.aws_ec2.yml --list``: + +.. code-block:: text + + { + "_meta": { + "hostvars": { + "instance-01": { + "aws_ami_launch_index_ec2": 0, + "aws_architecture_ec2": "x86_64", + ... + "ebs_optimized": false, + "ec2_tag_Environment": "dev", + "ec2_tag_Name": "instance-01", + "ec2_tag_Tag1": "Test1", + "ec2_tag_Tag2": "Test2", + "ena_support": true, + "enclave_options": { + "enabled": false + }, + ... + }, + "instance-02": { + ... + "ebs_optimized": false, + "ec2_tag_Environment": "dev", + "ec2_tag_Name": "instance-02", + "ec2_tag_Tag1": "Test3", + "ec2_tag_Tag2": "Test4", + "ena_support": true, + "enclave_options": { + "enabled": false + }, + ... + } + } + }, + all": { + "children": [ + "aws_ec2", + "ungrouped" + ] + }, + "aws_ec2": { + "hosts": [ + "instance-01", + "instance-02" + ] + } + } + + +``hostvars_prefix`` and ``hostvars_suffix`` +------------------------------------------- + +``hostvars_prefix`` and ``hostvars_sufix`` allow to set up a prefix and suffix for host variables. + +.. code-block:: yaml + + # demo.aws_ec2.yml + plugin: amazon.aws.aws_ec2 + regions: + - us-east-1 + filters: + tag:Name: + - 'instance-*' + hostvars_prefix: 'aws_' + hostvars_suffix: '_ec2' + hostnames: + - tag:Name + +Now the output of ``ansible-inventory -i demo.aws_ec2.yml --list``: + +.. code-block:: text + + { + "_meta": { + "hostvars": { + "instance-01": { + "aws_ami_launch_index_ec2": 0, + "aws_architecture_ec2": "x86_64", + "aws_block_device_mappings_ec2": [ + { + "device_name": "/dev/sda1", + "ebs": { + "attach_time": "2022-06-27T09:04:57+00:00", + "delete_on_termination": true, + "status": "attached", + "volume_id": "vol-06e065bca44e6eae5" + } + } + ], + "aws_capacity_reservation_specification_ec2": { + "capacity_reservation_preference": "open" + } + ..., + }, + "instance-02": { + ..., + } + } + }, + all": { + "children": [ + "aws_ec2", + "ungrouped" + ] + }, + "aws_ec2": { + "hosts": [ + "instance-01", + "instance-02" + ] + } + } + + +``strict`` and ``strict_permissions`` +------------------------------------- + +``strict: False`` will skip instead of producing an error if there are missing facts. + +``strict_permissions: False`` will ignore 403 errors rather than failing. + + +``cache`` +--------- + +``aws_ec2`` inventory plugin support caching can use the general settings for the fact cache defined in the ``ansible.cfg`` file's ``[defaults]`` section or define inventory-specific settings in the ``[inventory]`` section. +You can can define plugin-specific cache settings in the config file: + +.. code-block:: yaml + + # demo.aws_ec2.yml + plugin: aws_ec2 + # This enables cache. + cache: yes + # Plugin to be used. + cache_plugin: jsonfile + cache_timeout: 7200 + # Location where files are stored in the cache. + cache_connection: /tmp/aws_inventory + cache_prefix: aws_ec2 + +Here is an example of setting inventory caching with some fact caching defaults for the cache plugin used and the timeout in an ``ansible.cfg`` file: + +.. code-block:: ini + + [defaults] + fact_caching = ansible.builtin.jsonfile + fact_caching_connection = /tmp/ansible_facts + cache_timeout = 3600 + + [inventory] + cache = yes + cache_connection = /tmp/ansible_inventory + + +Complex Example +=============== + +Here is an ``aws_ec2`` complex example utilizing some of the previously listed options: + +.. code-block:: yaml + + # demo.aws_ec2.yml + plugin: amazon.aws.aws_ec2 + regions: + - us-east-1 + - us-east-2 + keyed_groups: + # add hosts to tag_Name_value groups for each aws_ec2 host's tags.Name variable. + - key: tags.Name + prefix: tag_Name_ + separator: "" + groups: + # add hosts to the group dev if any of the dictionary's keys or values is the word 'dev'. + development: "'dev' in (tags|list)" + filters: + tag:Name: + - 'instance-01' + - 'instance-03' + include_filters: + - tag:Name: + - 'instance-02' + - 'instance-04' + exclude_filters: + - tag:Name: + - 'instance-03' + - 'instance-04' + hostnames: + # You can also specify a list in order of precedence for hostname variables. + - ip-address + - dns-name + - tag:Name + - private-ip-address + compose: + # This sets the `ansible_host` variable to connect with the private IP address without changing the hostname. + ansible_host: private_ip_address + +If a host does not have the variables in the configuration above (i.e. ``tags.Name``, ``tags``, ``private_ip_address``), the host will not be added to groups other than those that the inventory plugin creates and the ``ansible_host`` host variable will not be modified. + +Now the output of ``ansible-inventory -i demo.aws_ec2.yml --graph``: + +.. code-block:: text + + @all: + |--@aws_ec2: + | |--instance-01 + | |--instance-02 + |--@tag_Name_instance_01: + | |--instance-01 + |--@tag_Name_instance_02: + | |--instance-02 + |--@ungrouped: + + +Using Dynamic Inventory Inside Playbook +======================================= + +If you want to use dynamic inventory inside the playbook, you just need to mention the group name in the hosts variable as shown below. + +.. code-block:: yaml + + --- + - name: Ansible Test Playbook + gather_facts: false + hosts: tag_Name_instance_02 + + tasks: + - name: Run Shell Command + command: echo "Hello World" diff --git a/ansible_collections/amazon/aws/docs/docsite/rst/dev_guidelines.rst b/ansible_collections/amazon/aws/docs/docsite/rst/dev_guidelines.rst new file mode 100644 index 000000000..f105cc78a --- /dev/null +++ b/ansible_collections/amazon/aws/docs/docsite/rst/dev_guidelines.rst @@ -0,0 +1,1050 @@ +.. _ansible_collections.amazon.aws.docsite.dev_guide_intro: + +**************************************************** +Guidelines for Ansible Amazon AWS module development +**************************************************** + +The Ansible AWS collection (on `Galaxy <https://galaxy.ansible.com/community/aws>`_, source code `repository <https://github.com/ansible-collections/community.aws>`_) is maintained by the Ansible AWS Working Group. For further information see the `AWS working group community page <https://github.com/ansible/community/wiki/aws>`_. If you are planning to contribute AWS modules to Ansible then getting in touch with the working group is a good way to start, especially because a similar module may already be under development. + +.. contents:: + :local: + +.. _ansible_collections.amazon.aws.docsite.dev_python: + +Requirements +============ + +Python Compatibility +-------------------- + +AWS content in Ansible 2.9 and 1.x collection releases supported Python 2.7 and newer. + +Starting with the 2.0 releases of both collections, Python 2.7 support will be ended in accordance with AWS' `end of Python 2.7 support <https://aws.amazon.com/blogs/developer/announcing-end-of-support-for-python-2-7-in-aws-sdk-for-python-and-aws-cli-v1/>`_. Contributions to both collections that target the 2.0 or later collection releases can be written to support Python 3.6+ syntax. + +SDK Version Support +------------------- + +Starting with the 2.0 releases of both collections, it is generally the policy to support the versions of botocore and boto3 that were released 12 months prior to the most recent major collection release, following semantic versioning (for example, 2.0.0, 3.0.0). + +Features and functionality that require newer versions of the SDK can be contributed provided they are noted in the module documentation: + +.. code-block:: yaml + + DOCUMENTATION = ''' + --- + module: ec2_vol + options: + throughput: + description: + - Volume throughput in MB/s. + - This parameter is only valid for gp3 volumes. + - Valid range is from 125 to 1000. + - Requires at least botocore version 1.19.27. + type: int + version_added: 1.4.0 + +And handled using the ``botocore_at_least`` helper method: + +.. code-block:: python + + if module.params.get('throughput'): + if not module.botocore_at_least("1.19.27"): + module.fail_json(msg="botocore >= 1.19.27 is required to set the throughput for a volume") + +Starting with the 4.0 releases of both collections, all support for the original boto SDK has been dropped. AWS Modules must be written using the botocore and boto3 SDKs. + +.. _ansible_collections.amazon.aws.docsite.dev_module_maint: + +Maintaining existing modules +============================ + +Changelogs +---------- + +A changelog fragment must be added to any PR that changes functionality or fixes +a bug. More information about changelog fragments can be found in the +`Making your PR merge-worthy section of the Ansible Development Cycle documentation<community_changelogs>` + +Breaking Changes +---------------- + +Changes that are likely to break existing playbooks using the AWS collections should be +avoided, should only be made in a major release, and where practical should be +preceeded by a deprecation cycle of at least 1 full major release. Deprecations +may be backported to the stable branches. + +For example: +- A deprecation added in release 3.0.0 may be removed in release 4.0.0. +- A deprecation added in release 1.2.0 may be removed in release 3.0.0. + +Breaking changes include: +- Removing a parameter. +- Making a parameter ``required``. +- Updating the default value of a parameter. +- Changing or removing an existing return value. + +Adding new features +------------------- + +Try to keep backward compatibility with versions of boto3/botocore that are at least a year old. +This means that if you want to implement functionality that uses a new feature of boto3/botocore, +it should only fail if that feature is explicitly used, with a message stating the missing feature +and minimum required version of botocore. (Feature support is usually defined in botocore and then +used by boto3) + +.. code-block:: python + + module = AnsibleAWSModule( + argument_spec=argument_spec, + ... + ) + + if module.params.get('scope') == 'managed': + module.require_botocore_at_least('1.23.23', reason='to list managed rules') + +.. _ansible_collections.amazon.aws.docsite.dev_backports: + +Release policy and backporting merged PRs +----------------------------------------- + +All amazon.aws and community.aws PRs must be merged to the ``main`` branch first. After a PR has +been accepted and merged to the ``main`` branch they can be backported to the stable branches. + +The ``main`` branch is a staging location for the next major version (X+1) of the collections and +may include breaking changes. + +General backport policy: + +- New features, deprecations and minor changes can be backported to the latest stable release. +- Bugfixes can be backported to the 2 latest stable releases. +- Security fixes should be backported to at least the 2 latest stable releases. + +Where necessary, additional CI related changes may be introduced to older stable branches to +ensure CI continues to function. + +The simplest mechanism for backporting PRs is by adding the ``backport-Y`` label to a PR. Once the +PR has been merged the patchback bot will attempt to automatically create a backport PR. + +.. _ansible_collections.amazon.aws.docsite.dev_module_create: + +Creating new AWS modules +======================== + +When writing a new module it is important to think about the scope of the module. In general, try +to do one thing and do it well. + +Where the Amazon APIs provide a distinction between dependent resources, such as S3 buckets and S3 +objects, this is often a good divider between modules. Additionally, resources which have a +many-to-many relationship with another resource, such as IAM managed policies and IAM roles, are +often best managed by two separate modules. + +While it's possible to write an ``s3`` module which manages all things related to S3, thoroughly +testing and maintaining such a module is difficult. Similarly, while it would be possible to +write a module that manages the base EC2 security group resource, and a second module to manage the +rules on the security group, this would be contrary to what users of the module might anticipate. + +There is no hard and fast right answer, but it's important to think about it, and Amazon have often +done this work for you when designing their APIs. + +Naming your module +------------------ + +Module names should include the name of the resource being managed and be prefixed with the AWS API +that the module is based on. Where examples of a prefix don't already exist a good rule of thumb is +to use whatever client name you use with boto3 as a starting point. + +Unless something is a well known abbreviation of a major component of AWS (for example, VPC or ELB) +avoid further abbreviating names and don't create new abbreviations independently. + +Where an AWS API primarily manages a single resource, the module managing this resource can be +named as just the name of the API. However, consider using ``instance`` or ``cluster`` for clarity +if Amazon refers to them using these names. + +Examples: + +- ``ec2_instance`` +- ``s3_object`` (previously named ``aws_s3``, but is primarily for manipulating S3 objects) +- ``elb_classic_lb`` (previously ``ec2_elb_lb``, but is part of the ELB API, not EC2) +- ``networkfirewall_rule_group`` +- ``networkfirewall`` (while this could be called ``networkfirewall_firewall`` the second firewall is redundant and the API is focused around creating these firewall resources) + +Note: Prior to the collections being split from Ansible Core, it was common to use ``aws_`` as a +prefix to disambiguate services with a generic name, such as ``aws_secret``. This is no longer +necessary, and the ``aws_`` prefix is reserved for services with a very broad effect where +referencing the AWS API might cause confusion. For example, ``aws_region_info``, which +connects to EC2 but provides global information about the regions enabled in an account for all +services. + +Use boto3 and AnsibleAWSModule +------------------------------- + +All new AWS modules must use boto3/botocore and ``AnsibleAWSModule``. + +``AnsibleAWSModule`` greatly simplifies exception handling and library +management, reducing the amount of boilerplate code. If you cannot +use ``AnsibleAWSModule`` as a base, you must document the reason and request an exception to this rule. + +Importing botocore and boto3 +---------------------------- + +The ``ansible_collections.amazon.aws.plugins.module_utils.ec2`` module and +``ansible_collections.amazon.aws.plugins.module_utils.core`` modules both +automatically import boto3 and botocore. If boto3 is missing from the system then the variable +``HAS_BOTO3`` will be set to false. Normally, this means that modules don't need to import +boto3 directly. There is no need to check ``HAS_BOTO3`` when using AnsibleAWSModule +as the module does that check: + +.. code-block:: python + + from ansible_collections.amazon.aws.plugins.module_utils.core import AnsibleAWSModule + try: + import botocore + except ImportError: + pass # handled by AnsibleAWSModule + +or: + +.. code-block:: python + + from ansible.module_utils.basic import AnsibleModule + from ansible_collections.amazon.aws.plugins.module_utils.ec2 import HAS_BOTO3 + try: + import botocore + except ImportError: + pass # handled by imported HAS_BOTO3 + + def main(): + + if not HAS_BOTO3: + module.fail_json(msg='boto3 and botocore are required for this module') + +Supporting Module Defaults +-------------------------- + +The existing AWS modules support using :ref:`module_defaults <module_defaults>` for common +authentication parameters. To do the same for your new module, add an entry for it in +``meta/runtime.yml``. These entries take the form of: + +.. code-block:: yaml + + action_groups: + aws: + ... + aws_example_module + +Module behavior +--------------- + +To reduce the chance of breaking changes occurring when new features are added, +the module should avoid modifying the resource attribute when a parameter is +not explicitly set in a task. + +By convention, when a parameter is explicitly set in a task, the module should +set the resource attribute to match what was set in the task. In some cases, +such as tags or associations, it can be helpful to add an additional parameter +which can be set to change the behavior from replacive to additive. However, the +default behavior should still be replacive rather than additive. + +See the `Dealing with tags<ansible_collections.amazon.aws.docsite.dev_tags>` +section for an example with ``tags`` and ``purge_tags``. + +.. _ansible_collections.amazon.aws.docsite.dev_module_connection: + +Connecting to AWS +================= + +AnsibleAWSModule provides the ``resource`` and ``client`` helper methods for obtaining boto3 connections. +These handle some of the more esoteric connection options, such as security tokens and boto profiles. + +If using the basic AnsibleModule then you should use ``get_aws_connection_info`` and then ``boto3_conn`` +to connect to AWS as these handle the same range of connection options. + +These helpers also check for missing profiles or a region not set when it needs to be, so you don't have to. + +An example of connecting to ec2 is shown below. Note that unlike boto there is no ``NoAuthHandlerFound`` +exception handling like in boto. Instead, an ``AuthFailure`` exception will be thrown when you use the +connection. To ensure that authorization, parameter validation and permissions errors are all caught, +you should catch ``ClientError`` and ``BotoCoreError`` exceptions with every boto3 connection call. +See exception handling: + +.. code-block:: python + + module.client('ec2') + +or for the higher level ec2 resource: + +.. code-block:: python + + module.resource('ec2') + + +An example of the older style connection used for modules based on AnsibleModule rather than AnsibleAWSModule: + +.. code-block:: python + + region, ec2_url, aws_connect_params = get_aws_connection_info(module, boto3=True) + connection = boto3_conn(module, conn_type='client', resource='ec2', region=region, endpoint=ec2_url, **aws_connect_params) + +.. code-block:: python + + region, ec2_url, aws_connect_params = get_aws_connection_info(module, boto3=True) + connection = boto3_conn(module, conn_type='client', resource='ec2', region=region, endpoint=ec2_url, **aws_connect_params) + + +Common Documentation Fragments for Connection Parameters +-------------------------------------------------------- + +There are four :ref:`common documentation fragments <module_docs_fragments>` +that should be included into almost all AWS modules: + +* ``aws`` - contains the common boto3 connection parameters +* ``ec2`` - contains the common region parameter required for many AWS modules +* ``boto3`` - contains the minimum requirements for the collection +* ``tags`` - contains the common tagging parameters used by many AWS modules + +These fragments should be used rather than re-documenting these properties to ensure consistency +and that the more esoteric connection options are documented. For example: + +.. code-block:: python + + DOCUMENTATION = ''' + module: my_module + # some lines omitted here + extends_documentation_fragment: + - amazon.aws.aws + - amazon.aws.ec2 + - amazon.aws.boto3 + ''' + +.. _ansible_collections.amazon.aws.docsite.dev_exceptions: + +Handling exceptions +=================== + +You should wrap any boto3 or botocore call in a try block. If an exception is thrown, then there +are a number of possibilities for handling it. + +* Catch the general ``ClientError`` or look for a specific error code with + ``is_boto3_error_code``. +* Use ``aws_module.fail_json_aws()`` to report the module failure in a standard way +* Retry using AWSRetry +* Use ``fail_json()`` to report the failure without using ``ansible_collections.amazon.aws.plugins.module_utils.core`` +* Do something custom in the case where you know how to handle the exception + +For more information on botocore exception handling see the `botocore error documentation <https://botocore.readthedocs.io/en/latest/client_upgrades.html#error-handling>`_. + +Using is_boto3_error_code +------------------------- + +To use ``ansible_collections.amazon.aws.plugins.module_utils.core.is_boto3_error_code`` to catch a single +AWS error code, call it in place of ``ClientError`` in your except clauses. In +this example, *only* the ``InvalidGroup.NotFound`` error code will be caught here, +and any other error will be raised for handling elsewhere in the program. + +.. code-block:: python + + try: + info = connection.describe_security_groups(**kwargs) + except is_boto3_error_code('InvalidGroup.NotFound'): + pass + do_something(info) # do something with the info that was successfully returned + +Using fail_json_aws() +--------------------- + +In the AnsibleAWSModule there is a special method, ``module.fail_json_aws()`` for nice reporting of +exceptions. Call this on your exception and it will report the error together with a traceback for +use in Ansible verbose mode. + +You should use the AnsibleAWSModule for all new modules, unless not possible. + +.. code-block:: python + + from ansible_collections.amazon.aws.plugins.module_utils.core import AnsibleAWSModule + + # Set up module parameters + # module params code here + + # Connect to AWS + # connection code here + + # Make a call to AWS + name = module.params.get['name'] + try: + result = connection.describe_frooble(FroobleName=name) + except (botocore.exceptions.BotoCoreError, botocore.exceptions.ClientError) as e: + module.fail_json_aws(e, msg="Couldn't obtain frooble %s" % name) + +Note that it should normally be acceptable to catch all normal exceptions here, however if you +expect anything other than botocore exceptions you should test everything works as expected. + +If you need to perform an action based on the error boto3 returned, use the error code and the +``is_boto3_error_code()`` helper. + +.. code-block:: python + + # Make a call to AWS + name = module.params.get['name'] + try: + result = connection.describe_frooble(FroobleName=name) + except is_boto3_error_code('FroobleNotFound'): + workaround_failure() # This is an error that we can work around + except (botocore.exceptions.BotoCoreError, botocore.exceptions.ClientError) as e: # pylint: disable=duplicate-except + module.fail_json_aws(e, msg="Couldn't obtain frooble %s" % name) + +using fail_json() and avoiding ansible_collections.amazon.aws.plugins.module_utils.core +--------------------------------------------------------------------------------------- + +Boto3 provides lots of useful information when an exception is thrown so pass this to the user +along with the message. + +.. code-block:: python + + from ansible.module_utils.ec2 import HAS_BOTO3 + try: + import botocore + except ImportError: + pass # caught by imported HAS_BOTO3 + + # Connect to AWS + # connection code here + + # Make a call to AWS + name = module.params.get['name'] + try: + result = connection.describe_frooble(FroobleName=name) + except botocore.exceptions.ClientError as e: + module.fail_json(msg="Couldn't obtain frooble %s: %s" % (name, str(e)), + exception=traceback.format_exc(), + **camel_dict_to_snake_dict(e.response)) + +Note: we use ``str(e)`` rather than ``e.message`` as the latter doesn't +work with python3 + +If you need to perform an action based on the error boto3 returned, use the error code. + +.. code-block:: python + + # Make a call to AWS + name = module.params.get['name'] + try: + result = connection.describe_frooble(FroobleName=name) + except botocore.exceptions.ClientError as e: + if e.response['Error']['Code'] == 'FroobleNotFound': + workaround_failure() # This is an error that we can work around + else: + module.fail_json(msg="Couldn't obtain frooble %s: %s" % (name, str(e)), + exception=traceback.format_exc(), + **camel_dict_to_snake_dict(e.response)) + except botocore.exceptions.BotoCoreError as e: + module.fail_json_aws(e, msg="Couldn't obtain frooble %s" % name) + +.. _ansible_collections.amazon.aws.docsite.dev_ratelimits: + +API throttling (rate limiting) and pagination +============================================= + +For methods that return a lot of results, boto3 often provides +`paginators <https://boto3.readthedocs.io/en/latest/guide/paginators.html>`_. If the method +you're calling has ``NextToken`` or ``Marker`` parameters, you should probably +check whether a paginator exists (the top of each boto3 service reference page has a link +to Paginators, if the service has any). To use paginators, obtain a paginator object, +call ``paginator.paginate`` with the appropriate arguments and then call ``build_full_result``. + +Any time that you are calling the AWS API a lot, you may experience API throttling, +and there is an ``AWSRetry`` decorator that can be used to ensure backoff. Because +exception handling could interfere with the retry working properly (as AWSRetry needs to +catch throttling exceptions to work correctly), you'd need to provide a backoff function +and then put exception handling around the backoff function. + +You can use ``exponential_backoff`` or ``jittered_backoff`` strategies - see +the cloud ``module_utils`` ()/lib/ansible/module_utils/cloud.py) +and `AWS Architecture blog <https://www.awsarchitectureblog.com/2015/03/backoff.html>`_ for more details. + +The combination of these two approaches is then: + +.. code-block:: python + + @AWSRetry.jittered_backoff(retries=5, delay=5) + def describe_some_resource_with_backoff(client, **kwargs): + paginator = client.get_paginator('describe_some_resource') + return paginator.paginate(**kwargs).build_full_result()['SomeResource'] + + def describe_some_resource(client, module): + filters = ansible_dict_to_boto3_filter_list(module.params['filters']) + try: + return describe_some_resource_with_backoff(client, Filters=filters) + except botocore.exceptions.ClientError as e: + module.fail_json_aws(e, msg="Could not describe some resource") + + +Prior to Ansible 2.10 if the underlying ``describe_some_resources`` API call threw +a ``ResourceNotFound`` exception, ``AWSRetry`` would take this as a cue to retry until +it is not thrown (this is so that when creating a resource, we can just retry until it +exists). This default was changed and it is now necessary to explicitly request +this behaviour. This can be done by using the ``catch_extra_error_codes`` +argument on the decorator. + +.. code-block:: python + + @AWSRetry.jittered_backoff(retries=5, delay=5, catch_extra_error_codes=['ResourceNotFound']) + def describe_some_resource_retry_missing(client, **kwargs): + return client.describe_some_resource(ResourceName=kwargs['name'])['Resources'] + + def describe_some_resource(client, module): + name = module.params.get['name'] + try: + return describe_some_resource_with_backoff(client, name=name) + except (botocore.exceptions.BotoCoreError, botocore.exceptions.ClientError) as e: + module.fail_json_aws(e, msg="Could not describe resource %s" % name) + + +To make use of AWSRetry easier, it can now be wrapped around a client returned +by ``AnsibleAWSModule``. any call from a client. To add retries to a client, +create a client: + +.. code-block:: python + + module.client('ec2', retry_decorator=AWSRetry.jittered_backoff(retries=10)) + +Any calls from that client can be made to use the decorator passed at call-time +using the ``aws_retry`` argument. By default, no retries are used. + +.. code-block:: python + + ec2 = module.client('ec2', retry_decorator=AWSRetry.jittered_backoff(retries=10)) + ec2.describe_instances(InstanceIds=['i-123456789'], aws_retry=True) + + # equivalent with normal AWSRetry + @AWSRetry.jittered_backoff(retries=10) + def describe_instances(client, **kwargs): + return ec2.describe_instances(**kwargs) + + describe_instances(module.client('ec2'), InstanceIds=['i-123456789']) + +The call will be retried the specified number of times, so the calling functions +don't need to be wrapped in the backoff decorator. + +You can also use customization for ``retries``, ``delay`` and ``max_delay`` parameters used by +``AWSRetry.jittered_backoff`` API using module params. You can take a look at +the ``cloudformation <cloudformation_module>`` module for example. + +To make all Amazon modules uniform, prefix the module param with ``backoff_``, so ``retries`` becomes ``backoff_retries`` + and likewise with ``backoff_delay`` and ``backoff_max_delay``. + +.. _ansible_collections.amazon.aws.docsite.dev_return: + +Returning Values +================ + +When you make a call using boto3, you will probably get back some useful information that you +should return in the module. As well as information related to the call itself, you will also have +some response metadata. It is OK to return this to the user as well as they may find it useful. + +Boto3 returns most keys in CamelCase. Ansible adopts python standards for naming variables and usage. +There is a useful helper function called ``camel_dict_to_snake_dict`` that allows for an easy conversion +of the boto3 response to snake_case. It resides in ``module_utils/common/dict_transformations``. + +You should use this helper function and avoid changing the names of values returned by Boto3. +E.g. if boto3 returns a value called 'SecretAccessKey' do not change it to 'AccessKey'. + +There is an optional parameter, ``ignore_list``, which is used to avoid converting a sub-tree +of a dict. This is particularly useful for tags, where keys are case-sensitive. + +.. code-block:: python + + # Make a call to AWS + resource = connection.aws_call() + + # Convert resource response to snake_case + snaked_resource = camel_dict_to_snake_dict(resource, ignore_list=['Tags']) + + # Return the resource details to the user without modifying tags + module.exit_json(changed=True, some_resource=snaked_resource) + +Note: The returned key representing the details of the specific resource (``some_resource`` above) +should be a sensible approximation of the resource name. For example, ``volume`` for ``ec2_vol``, +``volumes`` for ``ec2_vol_info``. + +Tags +---- + +Tags should be returned as a dictionary of key: value pairs, with each key being the tag's +key and value being the tag's value. It should be noted, however, that boto3 often returns tags +as a list of dictionaries. + +There is a helper function in module_utils/ec2.py called ``boto3_tag_list_to_ansible_dict`` +(discussed in detail below in the "Helper Functions" section) that allows for an easy conversion +from boto3's returned tag list to the desired dictionary of tags to be returned by the module. + +Below is a full example of getting the result of an AWS call and returning the expected values: + +.. code-block:: python + + # Make a call to AWS + result = connection.aws_call() + + # Make result snake_case without modifying tags + snaked_result = camel_dict_to_snake_dict(result, ignore_list=['Tags']) + + # Convert boto3 list of dict tags to just a dict of tags + snaked_result['tags'] = boto3_tag_list_to_ansible_dict(result.get('tags', [])) + + # Return the result to the user + module.exit_json(changed=True, **snaked_result) + +Info modules +------------ + +Info modules that can return information on multiple resources should return a list of +dictionaries, with each dictionary containing information about that particular resource +(i.e. ``security_groups`` in ``ec2_group_info``). + +In cases where the _info module only returns information on a singular resource +(i.e. ``ec2_tag_info``), a singular dictionary should be returned as opposed to a list +of dictionaries. + +In cases where the _info module returns no instances, an empty list '[]' should be returned. + +Keys in the returned dictionaries should follow the guidelines above and use snake_case. +If a return value can be used as a parameter for its corresponding main module, the key should +match either the parameter name itself, or an alias of that parameter. + +The following is an example of improper usage of a sample info module with its respective main module: + +.. code-block:: yaml + + "security_groups": { + { + "description": "Created by ansible integration tests", + "group_id": "sg-050dba5c3520cba71", + "group_name": "ansible-test-87988625-unknown5c5f67f3ad09-icmp-1", + "ip_permissions": [], + "ip_permissions_egress": [], + "owner_id": "721066863947", + "tags": [ + { + "Key": "Tag_One" + "Value": "Tag_One_Value" + }, + ], + "vpc_id": "vpc-0cbc2380a326b8a0d" + } + } + +The sample output above shows a few mistakes in the sample security group info module: +* ``security_groups`` is a dict of dict, not a list of dicts. +* ``tags`` appears to be directly returned from boto3, since they're a list of dicts. + +The following is what the sample output would look like, with the mistakes corrected. + +.. code-block:: yaml + + "security_groups": [ + { + "description": "Created by ansible integration tests", + "group_id": "sg-050dba5c3520cba71", + "group_name": "ansible-test-87988625-unknown5c5f67f3ad09-icmp-1", + "ip_permissions": [], + "ip_permissions_egress": [], + "owner_id": "721066863947", + "tags": { + "Tag_One": "Tag_One_Value", + }, + "vpc_id": "vpc-0cbc2380a326b8a0d" + } + ] + +Deprecating return values +------------------------- + +If changes need to be made to current return values, the new/"correct" keys should be +returned **in addition to** the existing keys to preserve compability with existing +playbooks. A deprecation should be added to the return values being replaced, initially +placed at least 2 years out, on the 1st of a month. + +For example: + +.. code-block:: python + + # Deprecate old `iam_user` return key to be replaced by `user` introduced on 2022-04-10 + module.deprecate("The 'iam_user' return key is deprecated and will be replaced by 'user'. Both values are returned for now.", + date='2024-05-01', collection_name='community.aws') + +.. _ansible_collections.amazon.aws.docsite.dev_policies: + +Dealing with IAM JSON policy +============================ + +If your module accepts IAM JSON policies then set the type to 'json' in the module spec. For +example: + +.. code-block:: python + + argument_spec.update( + dict( + policy=dict(required=False, default=None, type='json'), + ) + ) + +Note that AWS is unlikely to return the policy in the same order that is was submitted. Therefore, +use the ``compare_policies`` helper function which handles this variance. + +``compare_policies`` takes two dictionaries, recursively sorts and makes them hashable for comparison +and returns True if they are different. + +.. code-block:: python + + from ansible_collections.amazon.aws.plugins.module_utils.ec2 import compare_policies + + import json + + # some lines skipped here + + # Get the policy from AWS + current_policy = json.loads(aws_object.get_policy()) + user_policy = json.loads(module.params.get('policy')) + + # Compare the user submitted policy to the current policy ignoring order + if compare_policies(user_policy, current_policy): + # Update the policy + aws_object.set_policy(user_policy) + else: + # Nothing to do + pass + +.. _ansible_collections.amazon.aws.docsite.dev_tags: + +Dealing with tags +================= + +AWS has a concept of resource tags. Usually the boto3 API has separate calls +for tagging and untagging a resource. For example, the EC2 API has +``create_tags`` and ``delete_tags`` calls. + +When adding tagging support, Ansible AWS modules should add a ``tags`` parameter +that defaults to ``None`` and a ``purge_tags`` parameter that defaults to +``True``. + + +.. code-block:: python + + argument_spec.update( + dict( + tags=dict(type='dict', required=False, default=None), + purge_tags=dict(type='bool', required=False, default=True), + ) + ) + +When the ``purge_tags`` parameter is set to ``True`` **and** the ``tags`` +parameter is explicitly set in the task, then any tags not explicitly set in +``tags`` should be removed. + +If the ``tags`` parameter is not set then tags should not be modified, even if +``purge_tags`` is set to ``True``. This means that removing all tags requires +``tags`` be explicitly set to an empty dictionary ``{}`` in the Ansible task. + + +There is a helper function ``compare_aws_tags`` to ease dealing with tags. It +compares two dictionaries, the current tags and the desired tags, and returns +the tags to set and the tags to delete. See the Helper function section below +for more detail. + +There is also a documentation fragment ``amazon.aws.tags`` which should be +included when adding tagging support. + +.. _ansible_collections.amazon.aws.docsite.dev_helpers: + +Helper functions +================ + +Along with the connection functions in Ansible ec2.py module_utils, there are some other useful +functions detailed below. + +camel_dict_to_snake_dict +------------------------ + +boto3 returns results in a dict. The keys of the dict are in CamelCase format. In keeping with +Ansible format, this function will convert the keys to snake_case. + +``camel_dict_to_snake_dict`` takes an optional parameter called ``ignore_list`` which is a list of +keys not to convert (this is usually useful for the ``tags`` dict, whose child keys should remain with +case preserved) + +Another optional parameter is ``reversible``. By default, ``HTTPEndpoint`` is converted to ``http_endpoint``, +which would then be converted by ``snake_dict_to_camel_dict`` to ``HttpEndpoint``. +Passing ``reversible=True`` converts HTTPEndpoint to ``h_t_t_p_endpoint`` which converts back to ``HTTPEndpoint``. + +snake_dict_to_camel_dict +------------------------ + +``snake_dict_to_camel_dict`` converts snake cased keys to camel case. By default, because it was +first introduced for ECS purposes, this converts to dromedaryCase. An optional +parameter called ``capitalize_first``, which defaults to ``False``, can be used to convert to CamelCase. + +ansible_dict_to_boto3_filter_list +--------------------------------- + +Converts a an Ansible list of filters to a boto3 friendly list of dicts. This is useful for any +boto3 ``_facts`` modules. + +boto_exception +-------------- + +Pass an exception returned from boto or boto3, and this function will consistently get the message from the exception. + +Deprecated: use ``AnsibleAWSModule``'s ``fail_json_aws`` instead. + + +boto3_tag_list_to_ansible_dict +------------------------------ + +Converts a boto3 tag list to an Ansible dict. Boto3 returns tags as a list of dicts containing keys +called 'Key' and 'Value' by default. This key names can be overridden when calling the function. +For example, if you have already camel_cased your list of tags you may want to pass lowercase key +names instead, in other words, 'key' and 'value'. + +This function converts the list in to a single dict where the dict key is the tag key and the dict +value is the tag value. + +ansible_dict_to_boto3_tag_list +------------------------------ + +Opposite of above. Converts an Ansible dict to a boto3 tag list of dicts. You can again override +the key names used if 'Key' and 'Value' is not suitable. + +get_ec2_security_group_ids_from_names +------------------------------------- + +Pass this function a list of security group names or combination of security group names and IDs +and this function will return a list of IDs. You should also pass the VPC ID if known because +security group names are not necessarily unique across VPCs. + +compare_policies +---------------- + +Pass two dicts of policies to check if there are any meaningful differences and returns true +if there are. This recursively sorts the dicts and makes them hashable before comparison. + +This method should be used any time policies are being compared so that a change in order +doesn't result in unnecessary changes. + +compare_aws_tags +---------------- + +Pass two dicts of tags and an optional purge parameter and this function will return a dict +containing key pairs you need to modify and a list of tag key names that you need to remove. Purge +is True by default. If purge is False then any existing tags will not be modified. + +This function is useful when using boto3 ``add_tags`` and ``remove_tags`` functions. Be sure to use the +other helper function ``boto3_tag_list_to_ansible_dict`` to get an appropriate tag dict before +calling this function. Since the AWS APIs are not uniform (for example, EC2 is different from Lambda) this will work +without modification for some (Lambda) and others may need modification before using these values +(such as EC2, with requires the tags to unset to be in the form ``[{'Key': key1}, {'Key': key2}]``). + +.. _ansible_collections.amazon.aws.docsite.dev_tests: + +Integration Tests for AWS Modules +================================= + +All new AWS modules should include integration tests to ensure that any changes in AWS APIs that +affect the module are detected. At a minimum this should cover the key API calls and check the +documented return values are present in the module result. + +For general information on running the integration tests see the :ref:`Integration Tests page of the +Module Development Guide <testing_integration>`, especially the section on configuration for cloud tests. + +The integration tests for your module should be added in ``test/integration/targets/MODULE_NAME``. + +You must also have a aliases file in ``test/integration/targets/MODULE_NAME/aliases``. This file serves +two purposes. First indicates it's in an AWS test causing the test framework to make AWS credentials +available during the test run. Second putting the test in a test group causing it to be run in the +continuous integration build. + +Tests for new modules should be added to the ``cloud/aws`` group. In general just copy +an existing aliases file such as the `aws_s3 tests aliases file <https://github.com/ansible-collections/amazon.aws/blob/master/tests/integration/targets/aws_s3/aliases>`_. + + +Custom SDK versions for Integration Tests +----------------------------------------- + +By default integration tests will run against the earliest supported version of +the AWS SDK. The current supported versions can be found in +``tests/integration/constraints.txt`` and should not be updated. Where a module +needs access to a later version of the SDK this can be installed by depending on +the ``setup_botocore_pip`` role and setting the ``botocore_version`` variable in +the ``meta/main.yml`` file for your tests. + +.. code-block:: yaml + + dependencies: + - role: setup_botocore_pip + vars: + botocore_version: "1.20.24" + + +Creating EC2 instances in Integration Tests +------------------------------------------- + +When started, the integration tests will be passed ``aws_region`` as an extra var. +Any resources created should be created in in this region, this includes EC2 +instances. Since AMIs are region specific there is a role which can be +included which will query the APIs for an AMI to use and set the ``ec2_ami_id`` +fact. This role can be included by adding the ``setup_ec2_facts`` role as a +dependency in the ``meta/main.yml`` file for your tests. + + +.. code-block:: yaml + + dependencies: + - role: setup_ec2_facts + +The ``ec2_ami_id`` fact can then be used in the tests. + +.. code-block:: yaml + + - name: Create launch configuration 1 + community.aws.ec2_lc: + name: '{{ resource_prefix }}-lc1' + image_id: '{{ ec2_ami_id }}' + assign_public_ip: yes + instance_type: '{{ ec2_instance_type }}' + security_groups: '{{ sg.group_id }}' + volumes: + - device_name: /dev/xvda + volume_size: 10 + volume_type: gp2 + delete_on_termination: true + +To improve test result reproducability across regions, tests should use this +role and the fact it provides to chose an AMI to use. + + +Resource naming in Integration Tests +------------------------------------ + +AWS has a range of limitations for the name of resources. Where possible, +resource names should include a string which makes the resource names unique +to the test. + +The ``ansible-test`` tool used for running the integration tests provides two +helpful extra vars: ``resource_prefix`` and ``tiny_prefix`` which are unique to the +test set, and should generally used as part of the name. ``resource_prefix`` will generate a prefix based on the host the test is being run on. Sometimes this may result in a resource name that exceeds the character limit allowed by AWS. In these cases, ``tiny_prefix`` will provide a 12-character randomly generated prefix. + +AWS Credentials for Integration Tests +------------------------------------- + +The testing framework handles running the test with appropriate AWS credentials, these are made available +to your test in the following variables: + +* ``aws_region`` +* ``aws_access_key`` +* ``aws_secret_key`` +* ``security_token`` + +So all invocations of AWS modules in the test should set these parameters. To avoid duplicating these +for every call, it's preferable to use :ref:`module_defaults <module_defaults>`. For example: + +.. code-block:: yaml + + - name: set connection information for aws modules and run tasks + module_defaults: + group/aws: + aws_access_key: "{{ aws_access_key }}" + aws_secret_key: "{{ aws_secret_key }}" + security_token: "{{ security_token | default(omit) }}" + region: "{{ aws_region }}" + + block: + + - name: Do Something + ec2_instance: + ... params ... + + - name: Do Something Else + ec2_instance: + ... params ... + +AWS Permissions for Integration Tests +------------------------------------- + +As explained in the :ref:`Integration Test guide <testing_integration>` +there are defined IAM policies in `mattclay/aws-terminator <https://github.com/mattclay/aws-terminator>`_ that contain the necessary permissions +to run the AWS integration test. + +If your module interacts with a new service or otherwise requires new permissions, tests will fail when you submit a pull request and the +`Ansibullbot <https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md>`_ will tag your PR as needing revision. +We do not automatically grant additional permissions to the roles used by the continuous integration builds. +You will need to raise a Pull Request against `mattclay/aws-terminator <https://github.com/mattclay/aws-terminator>`_ to add them. + +If your PR has test failures, check carefully to be certain the failure is only due to the missing permissions. If you've ruled out other sources of failure, add a comment with the ``ready_for_review`` +tag and explain that it's due to missing permissions. + +Your pull request cannot be merged until the tests are passing. If your pull request is failing due to missing permissions, +you must collect the minimum IAM permissions required to +run the tests. + +There are two ways to figure out which IAM permissions you need for your PR to pass: + +* Start with the most permissive IAM policy, run the tests to collect information about which resources your tests actually use, then construct a policy based on that output. This approach only works on modules that use ``AnsibleAWSModule``. +* Start with the least permissive IAM policy, run the tests to discover a failure, add permissions for the resource that addresses that failure, then repeat. If your module uses ``AnsibleModule`` instead of ``AnsibleAWSModule``, you must use this approach. + +To start with the most permissive IAM policy: + +1) `Create an IAM policy <https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create.html#access_policies_create-start>`_ that allows all actions (set ``Action`` and ``Resource`` to ``*``). +2) Run your tests locally with this policy. On AnsibleAWSModule-based modules, the ``debug_botocore_endpoint_logs`` option is automatically set to ``yes``, so you should see a list of AWS ACTIONS after the PLAY RECAP showing all the permissions used. If your tests use a boto/AnsibleModule module, you must start with the least permissive policy (see below). +3) Modify your policy to allow only the actions your tests use. Restrict account, region, and prefix where possible. Wait a few minutes for your policy to update. +4) Run the tests again with a user or role that allows only the new policy. +5) If the tests fail, troubleshoot (see tips below), modify the policy, run the tests again, and repeat the process until the tests pass with a restrictive policy. +6) Open a pull request proposing the minimum required policy to the `CI policies <https://github.com/mattclay/aws-terminator/tree/master/aws/policy>`_. + +To start from the least permissive IAM policy: + +1) Run the integration tests locally with no IAM permissions. +2) Examine the error when the tests reach a failure. + a) If the error message indicates the action used in the request, add the action to your policy. + b) If the error message does not indicate the action used in the request: + - Usually the action is a CamelCase version of the method name - for example, for an ec2 client the method ``describe_security_groups`` correlates to the action ``ec2:DescribeSecurityGroups``. + - Refer to the documentation to identify the action. + c) If the error message indicates the resource ARN used in the request, limit the action to that resource. + d) If the error message does not indicate the resource ARN used: + - Determine if the action can be restricted to a resource by examining the documentation. + - If the action can be restricted, use the documentation to construct the ARN and add it to the policy. +3) Add the action or resource that caused the failure to `an IAM policy <https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create.html#access_policies_create-start>`_. Wait a few minutes for your policy to update. +4) Run the tests again with this policy attached to your user or role. +5) If the tests still fail at the same place with the same error you will need to troubleshoot (see tips below). If the first test passes, repeat steps 2 and 3 for the next error. Repeat the process until the tests pass with a restrictive policy. +6) Open a pull request proposing the minimum required policy to the `CI policies <https://github.com/mattclay/aws-terminator/tree/master/aws/policy>`_. + +Troubleshooting IAM policies +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +- When you make changes to a policy, wait a few minutes for the policy to update before re-running the tests. +- Use the `policy simulator <https://policysim.aws.amazon.com/>`_ to verify that each action (limited by resource when applicable) in your policy is allowed. +- If you're restricting actions to certain resources, replace resources temporarily with ``*``. If the tests pass with wildcard resources, there is a problem with the resource definition in your policy. +- If the initial troubleshooting above doesn't provide any more insight, AWS may be using additional undisclosed resources and actions. +- Examine the AWS FullAccess policy for the service for clues. +- Re-read the AWS documentation, especially the list of `Actions, Resources and Condition Keys <https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_actions-resources-contextkeys.html>`_ for the various AWS services. +- Look at the `cloudonaut <https://iam.cloudonaut.io>`_ documentation as a troubleshooting cross-reference. +- Use a search engine. +- Ask in the #ansible-aws chat channel (using Matrix at ansible.im or using IRC at `irc.libera.chat <https://libera.chat/>`_). + +Unsupported Integration tests +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +There are a limited number of reasons why it may not be practical to run integration +tests for a module within CI. Where these apply you should add the keyword +``unsupported`` to the aliases file in ``test/integration/targets/MODULE_NAME/aliases``. + +Some cases where tests should be marked as unsupported: +1) The tests take longer than 10 or 15 minutes to complete +2) The tests create expensive resources +3) The tests create inline policies +4) The tests require the existence of external resources +5) The tests manage Account level security policies such as the password policy or AWS Organizations. + +Where one of these reasons apply you should open a pull request proposing the minimum required policy to the +`unsupported test policies <https://github.com/mattclay/aws-terminator/tree/master/hacking/aws_config/test_policies>`_. + +Unsupported integration tests will not be automatically run by CI. However, the +necessary policies should be available so that the tests can be manually run by +someone performing a PR review or writing a patch. diff --git a/ansible_collections/amazon/aws/docs/docsite/rst/guide_aws.rst b/ansible_collections/amazon/aws/docs/docsite/rst/guide_aws.rst new file mode 100644 index 000000000..a05d21221 --- /dev/null +++ b/ansible_collections/amazon/aws/docs/docsite/rst/guide_aws.rst @@ -0,0 +1,302 @@ +.. _ansible_collections.amazon.aws.docsite.aws_intro: + +************************* +Amazon Web Services Guide +************************* + +The ``amazon.aws`` collection contains a number of modules and plugins for controlling Amazon Web Services (AWS). This guide explains how to use the modules and inventory scripts to automate your AWS resources with Ansible. + +.. contents:: + :local: + +Requirements for the AWS modules are minimal. + +All of the modules require and are tested against recent versions of botocore and boto3. Starting with the 2.0 AWS collection releases, it is generally the policy of the collections to support the versions of these libraries released 12 months prior to the most recent major collection revision. Individual modules may require a more recent library version to support specific features or may require the boto library, check the module documentation for the minimum required version for each module. You must have the boto3 Python module installed on your control machine. You can install these modules from your OS distribution or using the python package installer: ``pip install boto3``. + +Starting with the 2.0 releases of both collections, Python 2.7 support will be ended in accordance with AWS' `end of Python 2.7 support <https://aws.amazon.com/blogs/developer/announcing-end-of-support-for-python-2-7-in-aws-sdk-for-python-and-aws-cli-v1/>`_ and Python 3.6 or greater will be required. + +Whereas classically Ansible will execute tasks in its host loop against multiple remote machines, most cloud-control steps occur on your local machine with reference to the regions to control. + +In your playbook steps we'll typically be using the following pattern for provisioning steps:: + + - hosts: localhost + gather_facts: False + tasks: + - ... + +.. _ansible_collections.amazon.aws.docsite.aws_authentication: + +Authentication +`````````````` + +Authentication with the AWS-related modules is handled by either +specifying your access and secret key as ENV variables or module arguments. + +For environment variables:: + + export AWS_ACCESS_KEY_ID='AK123' + export AWS_SECRET_ACCESS_KEY='abc123' + +For storing these in a vars_file, ideally encrypted with ansible-vault:: + + --- + aws_access_key: "--REMOVED--" + aws_secret_key: "--REMOVED--" + +Note that if you store your credentials in vars_file, you need to refer to them in each AWS-module. For example:: + + - amazon.aws.ec2_instance: + aws_access_key: "{{ aws_access_key }}" + aws_secret_key: "{{ aws_secret_key }}" + key_name: "example-ssh-key" + image_id: "..." + +Or they can be specified using "module_defaults" at the top of a playbook.:: + + # demo_setup.yml + + - hosts: localhost + module_defaults: + group/aws: + aws_access_key: '{{ aws_access_key }}' + aws_secret_key: '{{ aws_secret_key }}' + region: '{{ region }}' + tasks: + - amazon.aws.ec2_instance: + key_name: "example-ssh-key" + image_id: "..." + +Credentials can also be accessed from a `Credentials Profile <https://docs.aws.amazon.com/sdk-for-php/v3/developer-guide/guide_credentials_profiles.html>`_.:: + + - amazon.aws.ec2_instance: + aws_profile: default + key_name: "example-ssh-key" + image_id: "..." + +.. _ansible_collections.amazon.aws.docsite.aws_provisioning: + +Provisioning +```````````` + +The ec2_instance module provisions and de-provisions instances within EC2. + +An example of creating an instance with a public IP assigned follows. + +The "name" parameter will create a "tag:Name" on the instance. Additional tags can be specified with the "tags" parameter.:: + + # demo_setup.yml + + - hosts: localhost + gather_facts: False + + tasks: + + - name: Provision an EC2 instance with a public IP address + amazon.aws.ec2_instance: + name: Demo + key_name: "example-ssh-key" + vpc_subnet_id: subnet-5ca1ab1e + instance_type: c5.large + security_group: default + network: + assign_public_ip: true + image_id: ami-123456 + tags: + Environment: Testing + register: result + +The data about the instance that has been created is being saved by the "register" keyword in the variable named "result". + +From this, we'll use the add_host module to dynamically create a host group consisting of these new instances. This facilitates performing configuration actions on the hosts immediately in a subsequent task.:: + + # demo_setup.yml + + - hosts: localhost + gather_facts: False + + tasks: + + - name: Provision an EC2 instance with a public IP address + amazon.aws.ec2_instance: + name: Demo + key_name: "example-ssh-key" + vpc_subnet_id: subnet-5ca1ab1e + instance_type: c5.large + security_group: default + network: + assign_public_ip: true + image_id: ami-123456 + tags: + Environment: Testing + register: result + + - name: Add all instance public IPs to host group + add_host: hostname={{ item.public_ip }} groups=ec2hosts + loop: "{{ result.instances }}" + +With the host group now created, a second play at the bottom of the same provisioning playbook file might now have some configuration steps:: + + # demo_setup.yml + + - name: Provision a set of instances + hosts: localhost + # ... AS ABOVE ... + + - hosts: ec2hosts + name: configuration play + user: ec2-user + gather_facts: true + + tasks: + + - name: Check NTP service + service: name=ntpd state=started + +.. _ansible_collections.amazon.aws.docsite.aws_security_groups: + +Security Groups +``````````````` + +Security groups on AWS are stateful. The response of a request from your instance is allowed to flow in regardless of inbound security group rules and vice-versa. +In case you only want allow traffic with AWS S3 service, you need to fetch the current IP ranges of AWS S3 for one region and apply them as an egress rule.:: + + - name: fetch raw ip ranges for aws s3 + set_fact: + raw_s3_ranges: "{{ lookup('aws_service_ip_ranges', region='eu-central-1', service='S3', wantlist=True) }}" + + - name: prepare list structure for ec2_group module + set_fact: + s3_ranges: "{{ s3_ranges | default([]) + [{'proto': 'all', 'cidr_ip': item, 'rule_desc': 'S3 Service IP range'}] }}" + loop: "{{ raw_s3_ranges }}" + + - name: set S3 IP ranges to egress rules + ec2_group: + name: aws_s3_ip_ranges + description: allow outgoing traffic to aws S3 service + region: eu-central-1 + state: present + vpc_id: vpc-123456 + purge_rules: true + purge_rules_egress: true + rules: [] + rules_egress: "{{ s3_ranges }}" + tags: + Name: aws_s3_ip_ranges + +.. _ansible_collections.amazon.aws.docsite.aws_host_inventory: + +Host Inventory +`````````````` + +Once your nodes are spun up, you'll probably want to talk to them again. With a cloud setup, it's best to not maintain a static list of cloud hostnames +in text files. Rather, the best way to handle this is to use the aws_ec2 inventory plugin. See :ref:`dynamic_inventory`. + +The plugin will also return instances that were created outside of Ansible and allow Ansible to manage them. + +.. _ansible_collections.amazon.aws.docsite.aws_tags_and_groups: + +Tags And Groups And Variables +````````````````````````````` + +When using the inventory plugin, you can configure extra inventory structure based on the metadata returned by AWS. + +For instance, you might use ``keyed_groups`` to create groups from instance tags:: + + plugin: amazon.aws.aws_ec2 + keyed_groups: + - prefix: tag + key: tags + + +You can then target all instances with a "class" tag where the value is "webserver" in a play:: + + - hosts: tag_class_webserver + tasks: + - ping + +You can also use these groups with 'group_vars' to set variables that are automatically applied to matching instances. + +.. _ansible_collections.amazon.aws.docsite.aws_pull: + +Autoscaling with Ansible Pull +````````````````````````````` + +Amazon Autoscaling features automatically increase or decrease capacity based on load. There are also Ansible modules shown in the cloud documentation that +can configure autoscaling policy. + +When nodes come online, it may not be sufficient to wait for the next cycle of an ansible command to come along and configure that node. + +To do this, pre-bake machine images which contain the necessary ansible-pull invocation. Ansible-pull is a command line tool that fetches a playbook from a git server and runs it locally. + +One of the challenges of this approach is that there needs to be a centralized way to store data about the results of pull commands in an autoscaling context. +For this reason, the autoscaling solution provided below in the next section can be a better approach. + +Read :ref:`ansible-pull` for more information on pull-mode playbooks. + +.. _ansible_collections.amazon.aws.docsite.aws_autoscale: + +Autoscaling with Ansible Automation Platform +```````````````````````````````````````````` + +`Ansible Automation Platform (AAP) <https://access.redhat.com/documentation/en-us/red_hat_ansible_automation_platform/>`_ +also contains a very nice feature for auto-scaling use cases. In this mode, a simple curl script can call +a defined URL and the server will "dial out" to the requester and configure an instance that is spinning up. This can be a great way +to reconfigure ephemeral nodes. See the install and product documentation for more details. + +A benefit of using the callback in AAP over pull mode is that job results are still centrally recorded and less information has to be shared +with remote hosts. + +.. _ansible_collections.amazon.aws.docsite.aws_cloudformation_example: + +Ansible With (And Versus) CloudFormation +```````````````````````````````````````` + +CloudFormation is a Amazon technology for defining a cloud stack as a JSON or YAML document. + +Ansible modules provide an easier to use interface than CloudFormation in many examples, without defining a complex JSON/YAML document. +This is recommended for most users. + +However, for users that have decided to use CloudFormation, there is an Ansible module that can be used to apply a CloudFormation template +to Amazon. + +When using Ansible with CloudFormation, typically Ansible will be used with a tool like Packer to build images, and CloudFormation will launch +those images, or ansible will be invoked through user data once the image comes online, or a combination of the two. + +Please see the examples in the Ansible CloudFormation module for more details. + +.. _ansible_collections.amazon.aws.docsite.aws_image_build: + +AWS Image Building With Ansible +``````````````````````````````` + +Many users may want to have images boot to a more complete configuration rather than configuring them entirely after instantiation. To do this, +one of many programs can be used with Ansible playbooks to define and upload a base image, which will then get its own AMI ID for usage with +the ec2 module or other Ansible AWS modules such as ec2_asg or the cloudformation module. Possible tools include Packer, aminator, and Ansible's +ec2_ami module. + +Generally speaking, we find most users using Packer. + +See the Packer documentation of the `Ansible local Packer provisioner <https://www.packer.io/docs/provisioners/ansible/ansible-local>`_ and `Ansible remote Packer provisioner <https://www.packer.io/docs/provisioners/ansible/ansible>`_. + +If you do not want to adopt Packer at this time, configuring a base-image with Ansible after provisioning (as shown above) is acceptable. + +.. _ansible_collections.amazon.aws.docsite.aws_next_steps: + +Next Steps: Explore Modules +``````````````````````````` + +Ansible ships with lots of modules for configuring a wide array of EC2 services. Browse the "Cloud" category of the module +documentation for a full list with examples. + +.. seealso:: + + :ref:`list_of_collections` + Browse existing collections, modules, and plugins + :ref:`working_with_playbooks` + An introduction to playbooks + :ref:`playbooks_delegation` + Delegation, useful for working with loud balancers, clouds, and locally executed steps. + `User Mailing List <https://groups.google.com/group/ansible-devel>`_ + Have a question? Stop by the google group! + `irc.libera.chat <https://libera.chat/>`_ + #ansible IRC chat channel |