* Resolve static actions when the FQCN is already known or demanded by a callback plugin
shorthand syntax (e.g. "- ping:") is resolved by ModuleArgsParser
action/local_action syntax (e.g. "- action: ping") is resolved on demand
* Emit a warning if a callback plugin accesses the property when it's None. This is expected if action/local_action is a template and a callback plugin uses this value too early (like in v2_playbook_on_task_start) or late (like in v2_runner_on_ok for a task with a loop).
* service_facts: Handle KeyError while processing service name
As a part of follow up review,
* Handle KeyError with exception handling
* Warn user about the missing service name in the given service details
Signed-off-by: Abhijeet Kasurde <Akasurde@redhat.com>
* remove internal collections earlier to ignore consistently for different sub-commands
* remove internal collection handling from the dependency resolver
* add a test to ensure ansible._protomatter is not in the output of ansible-galaxy collection list
* fix existing test to ensure an error is given if no valid collection path is configured
* changelog
* Add correct empty venv creation handling
Makes the module exit with changed=True
when an empty venv is created with correct
module output
* Add tests for creating an empty venv
* Use display.error_as_warning instead of self.warning
* Use display.error_as_warning instead of display.warning_as_error.
Signed-off-by: Abhijeet Kasurde <Akasurde@redhat.com>
* Prevents callback handle_exception from displaying the captured exception when the task is not failed.
* Added tests.
Co-authored-by: Matt Clay <matt@mystile.com>
Fix ssh plugin host variables. Variable names should conform to Python variable naming rules.
, and not contain characters like "[" or "]".
Update unit test
* Mark dependent packages as auto which are installed as part of
deb file installation
Fixes: #78123
Co-authored-by: Matt Clay <matt@mystile.com>
Signed-off-by: Abhijeet Kasurde <Akasurde@redhat.com>
* ansible-doc: print where the description is missing
Signed-off-by: Thomas Sjögren <konstruktoid@users.noreply.github.com>
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
* Limit askpass prompts to single attempt
OpenSSH client option NumberOfPasswordPrompts defaults to 3 so in case
an incorrect connection password is provided it is excessively tried 3
times. Not only that but running the `_ssh_askpass` entry point multiple
times (via ssh) results in `json.decoder.JSONDecodeError` as after the
first run the shared memory is zero'd and the subsequent runs end up
calling `json.loads` on empty data.
`json.decoder.JSONDecodeError` does not happen prior to Python 3.13 as
the share memory is unlinked automatically on `.close()` and the
`_ssh_askpass` entry point exits with return code 1 before attempting to
load zero'd memory.
Fixes#85359
* changelog and tests
* Update changelogs/fragments/85359-askpass-incorrect-password-retries.yml
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
* Update lib/ansible/cli/_ssh_askpass.py
Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
* Avoid race condition in second unlink
---------
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
Co-authored-by: Matt Clay <matt@mystile.com>
* deprecate invalid variable names in inventory
* was previously a hard error for `Host`
* added missing check to `Group`
* swapped blanket Python keyword prohibition with Jinja singleton and `not` check
Co-authored-by: Matt Clay <matt@mystile.com>
* fix invalid variable name test
---------
Co-authored-by: Matt Clay <matt@mystile.com>
* added ansible-test integration env/set directive for aliases
* applied to module-serialization-profiles test to enable feature flag
Co-authored-by: Matt Clay <matt@mystile.com>
* Fix handling FileExistsError, instead of only handling OSError when the human-readable error message is "File exists".
Co-authored-by: Matt Clay <matt@mystile.com>
* Add test for a handler including tasks from a variable filename
* Add FieldAttributeBase attribute to indicate if the object should be post validated
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
* always allow Marker args to pass through
* always disable pre-emptive trip-on-retrieval for Macro JinjaCallContext
* add macro-callable template expression result test cases
Co-authored-by: Matt Clay <matt@mystile.com>
* deprecate DEFAULT_MANAGED_STR and prevent masking of ansible_managed var
* adjust public API behavior
* restore backward-compatible behavior on existing public API
* doc - Dynamically document jinja builtins
This change has `ansible-doc` dynamically generate the documentation for
any Jinja builtin filter and test plugins. These dynamic stubs will
point to the official Jinja documentation pages for more information.
* Fix sanity issues
* Add tests
* Update Jinja builtin doc gen
Co-authored-by: Matt Clay <matt@mystile.com>
---------
Co-authored-by: Matt Davis <nitzmahone@redhat.com>
Co-authored-by: Matt Clay <matt@mystile.com>
* Clean up TE error handling, wrap sigalrm handler
* Preserve error detail on AnsibleAction and Connection exceptions.
* Remove multiple layers of unreachable or redundant error handling.
* Wrap manual alarm signal/timeout handling into a context manager, add tests.
Co-authored-by: Matt Clay <matt@mystile.com>
* update error message check in test
* update test timeout message assertions
---------
Co-authored-by: Matt Clay <matt@mystile.com>
async_status and async_wrrapper now use booleans instead of 0/1 for 'thruthyness'
gather_facts also updated
Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Matt Clay <matt@mystile.com>
* Add support for Windows App Control/WDAC
Adds preview support for Windows App Control, formerly known as WDAC.
This is a tech preview feature and is designed to test out improvements
needed in future versions of Ansible.
* Use psd1 and parse it through the Ast to avoid any unexpected execution results
* Add tests for various manifest permutations
* Ignore test shebang failure
* Apply suggestions from code review
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
* Use more flexible test expectations
* Add type annotations for shell functions
---------
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
* ansible-test - Improve deprecated checking type inference
Also disabled the ``bad-super-call`` pylint rule due to false positives.
* Add type comment support
* Try without using register_transform
Fix the logic for a constructable inventory `keyed_groups` entry to use
the `default_value` if the ``key` expression result is `None` or `omit`
and not just an empty string. This bug was introduced with the changes
in data tagging and goes back to the original behaviour.
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
* template module - render `None` as empty string
* Update changelogs/fragments/template-none.yml
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
---------
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
* remove dead config comment noise
* update `list` typed config defaults to be lists
* fix tag preservation/propagation in config
* numerous other ensure_type bugfixes
* 100% unit test coverage of ensure_type
* emit warnings on template_default failures
* fix unhandled exception in convert_bool on unhashable inputs
Co-authored-by: Matt Clay <matt@mystile.com>
---------
Co-authored-by: Matt Clay <matt@mystile.com>
* See changelog fragment for most changes.
* Defer early config warnings until display is functioning, eliminating related fallback display logic.
* Added more type annotations and docstrings.
* ansible-test - pylint sanity for deprecations improved.
* Refactored inline legacy resolutions in PluginLoader.
Co-authored-by: Matt Clay <matt@mystile.com>
* Implement TaskResult backward compatibility for callbacks
* general API cleanup
* misc deprecations
Co-authored-by: Matt Clay <matt@mystile.com>
* fix v2_on_any deprecation exclusion for base
---------
Co-authored-by: Matt Clay <matt@mystile.com>
* Added support for URL-safe decoding into b64decode
* Added support for URL-safe encoding into b64encode
Fixes: #84147
Signed-off-by: Abhijeet Kasurde <Akasurde@redhat.com>
* Add ssh-agent launching, and ssh-agent python client
* Move things around, is this better??
* docs
* postpone creating dir after bin lookup
* fix method name
* changelog ssh agent
* address reviews
* fix typing
* do not redefine public_key
* typing
* more typing
* Catch OSError when starting ssh agent
* likely copy pasted old code
* var type fix
* why is this needed?
ci_complete
* ignoring the change for now
* write out pub key file atomically
* defensive timeout for the socket
* _populate_agent docstring
* do not allow setting these in config
* check expected length before slicing blobs
* test all key types
* remove lock/unlock functionality
* docstring
* private _ssh_agent
* .
* launch agent in cli and ansible_ssh_*
* additional info for ssh-agent comment
* Add tests for remove and remove_all
* comment on os.rename
* hopefully mitigate agent startup/delays problems
* exceptions
* unused import
* fix sanity
* perf
---------
Co-authored-by: Matt Martz <matt@sivel.net>
* Use shallow copy of os.environ to update PYTHONPATH value
instead of using '|' operator
Fixes: #84954
Signed-off-by: Abhijeet Kasurde <Akasurde@redhat.com>
Moved check to connection as it should be the final decider
Added property to become plugins to indicate support
Also removed hardcoded su exception
Added tty detection logic for ssh (pipelining won't work if tty is needed or forced)
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
* Git allows embedding username and password in repo URL for
https authentication. This may lead to exposing the user sensitive
information to logs and unautheticated users.
Adding no_log will partially solve this.
* Added documentation warning user about URL embedded with username
and password.
* Added logic to remember user sensitive information for later sanitization
Fixes: #84557
Signed-off-by: Abhijeet Kasurde <Akasurde@redhat.com>
* Clean up interpreter discovery
- Deprecated `auto_legacy` and `auto_legacy_silent`
- Removed obsolete platform fallback config and logic
- Replaced unit tests with integration tests
- Increased test coverage
* Support download_url without a scheme:host. Fixes#84213
* Support properly rebuilding pagination links. #84765
* Don't permit non absolute pagination links
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
Updates the Windows exec runner in preparation for the WDAC changes.
This new process is designed to improve the way modules are run by
Windows and expose common functionality to run PowerShell code in a
common environment. It also includes futher changes to improve the error
handling to make it easier to see where an error occurred in the running
code.
* Improve performance of including tasks into the play
PlayIterator.add_tasks is used to insert tasks from an include into the
play for particular host. It makes a copy of the current block including
the tasks within the block and inserts the new tasks from the include
into the copied block. But there is no need to make copies of tasks
within the block, what we want is a "shallow" copy of the block. This
PR changes that to copy the block excluding the tasks within.
On a contrived playbook with 50 include_role tasks, each role has 1 task,
running on 10 hosts the running time is reduced from ~55s to ~44s in my
environment.
ci_complete
* Add changelog
* skip 'no such file or directory' error code while files and
directories and report them.
Fixes: #84873
Signed-off-by: Abhijeet Kasurde <Akasurde@redhat.com>
* uri: form location correctly from relative redirect
Previously, the original URL would be combined with the relative location incorrectly, especially for URL of any complexity.
Add simple tests demonstrating the problem that fail without the fix
* fix pylint error, import the method similar to other uri methods
* add changelog fragment
Signed-off-by: Abhijeet Kasurde <Akasurde@redhat.com>
Refactor the async wrapper and watchdog scripts for Windows. This
attempts to avoid WMI on connection plugins that allow breaking away
from a job like winrm and ssh as an optimisation and changes how WMI is
used so that we can get the error details on a failed process more
easily.
These changes are being made also in preparation for the WDAC
implementation that requires this new execution model where input needs
to be provided through stdin.
This patch marks the `ansible.module_utils.compat.datetime` module as deprecated, including `UTC`, `utcfromtimestamp()` and `utcnow` shims that it provides, scheduling its removal for v2.21.
It also replaces any uses of the compatibility helpers with non-deprecated calls to CPython stdlib.
PR #81874
Co-authored-by: Matt Clay <matt@mystile.com>
Prior to this change, when the dependency resolver started looping over multiple versions of the same collection due to backtracking, it might take a lot of time to consider and disregard tens or hundreds of versions. But to the end-user, it looks like “nothing is happening, the program is *stuck*”. Even worse if such a time-consuming backtracking hits multiple collections and it “hangs” for longer cumulative period of time.
This patch improves the perceived responsiveness by printing out informational messages with the current status whenever the backtracking for a collection happens for the first, the eighth and the thirteenth times. The last message also reminds them that they can interrupt the process and attempt to adjust the constraints.
In debug mode, it also shows what caused conflicts leading up to candidate rejections. But this is only available with `resolvelib >= 0.9.0`.
The improvement is heavily inspired by https://github.com/pypa/pip/commit/9731131.
PR #81709
Co-Authored-By: Jordan Borean <jborean93@gmail.com>
Removes the use of pexpect in the winrm connection plugin and rely on
just subprocess. In the past pexpect was used for macOS compatibility so
that it could handle the TTY prompt but after testing it seems like
subprocess with `start_new_session=True` is enough to get it reading
from stdin on all platforms. This simplifies the code as there's no
longer an optional library changing how things are called and will work
out of the box.
Fix up bug when attempting to run any module on a Windows host that has
been configured with WDAC and Dynamic Code Security in audit mode. This
does not enable WDAC support with signed scripts so Ansible will still
not pass the audit events but it no longer fails to run.
* Ensure implicit flush_handlers have a parent block
To avoid getting tracebacks when calling methods like ``get_play()`` on
them.
* task needs to be copied
* copy only when necessary
No the file name that caused the error will be apparent
---------
Signed-off-by: Abhijeet Kasurde <Akasurde@redhat.com>
Co-authored-by: Abhijeet Kasurde <Akasurde@redhat.com>
* Fixed various become-related issues in `local` connection plugin.
* Fixed various issues in `sudo` and `su` become plugins.
* Added unit and integration test coverage.
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Davis <nitzmahone@redhat.com>
Previously, requirement version specs starting with `!=` were
incorrectly considered as pinned release requests because the
comparison was being made against a one-char string while the
operator is two-char. This patch changes the check to test against `!`
which is enough to detect this case.
* rc-status commands returns unwanted lines with service names
and their status. Skip such lines while parsing service names
Fixes: #84512
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* apt-key module updates due to debian removal
Still kept for now for backwards compat, but removing from testing when not present
And adding more explicit mesasges to errors to point to new module
* added docs and error msg
* clog
* aslkdfj
* no docs to document doc changes
* also add warning to apt_repository
* clog on apt_repo too
* fix string concat
* Apply suggestions from code review
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
---------
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Improves the logic for parsing CLIXML values in the stderr returned by
SSH. This fixes encoding problems by having a fallback in case the
output is not valid UTF-8. It also can now extract embedded CLIXML
sequences in all of stderr rather than just at the start.
When a user doesn't exist and user module is used to create the user and the
homedir, adduser is called which parses HOME_MODE from /etc/login.defs, and when
not set calculates the mode from UMASK from the same file.
When a user already exists without homedir, and the user module is used to add a
home dir, it incorrectly ignores HOME_MODE, resulting in a world-readable home
dir when UMASK is not set. This is for example the case in Debian trixie and
later, and likely Ubuntu 25.04 and later.
Signed-off-by: Lee Garrett <lgarrett@rocketjump.eu>
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
prompt now only errors if stdin is specifically triggered and not due to lack of other args
fixes#84489
---------
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
* fix[doc.py]: path will be undefined and a direct reference will throw an UnboundLocalError.
If none of the files in files exists, path will be undefined and a direct reference will throw an UnboundLocalError.
Repair function file parameter type annotation is inaccurate
* Update changelogs/fragments/fix-cli-doc-path_undefined.yaml
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
---------
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
Fixes:
Traceback (most recent call last):
File "/home/shertel/ansible/lib/ansible/plugins/vars/host_group_vars.py", line 139, in get_vars
FOUND[key] = found_files = loader.find_vars_files(opath, entity_name)
^^^
UnboundLocalError: cannot access local variable 'key' where it is not associated with a value
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/shertel/ansible/lib/ansible/plugins/vars/host_group_vars.py", line 151, in get_vars
raise AnsibleParserError(to_native(e))
ansible.errors.AnsibleParserError: cannot access local variable 'key' where it is not associated with a value
Also remove redundant check from tqm
Now covers module output (set_fact/include_vars)
Includes play objects at any stage (tasks that error were not covered)
Added tests, moved them to role structure
Some versions and architectures come with `ca_root_nss` pre-installed.
However, at least FreeBSD 13.4 on aarch64 does not.
This change ensures the certificates will always be installed.
* ansible-test - Replace Fedora 40 with 41
Also update other distro containers to remove unnecessary packages:
* apache2
* subversion
* ruby
* Fix subversion test on Ubuntu
Also remove obsolete vars files from subversion test.
* Skip dnf test on Fedora 41+
The dnf5 test is all that is needed.
* Support dnf5 for the package test
* Extend unix-chkpwd work-around for Fedora
gather_facts, fix network_os and smart logic and defaults
setup will be default for smart only if network_os is not set, now you get warnings and errors when missing a valid facts module for a network os
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
* simplify redundancy with AnsibleModule set_*_if_different methods
* simplify copying a source directory to a dest directory without modifying behavior
