Commit graph

11192 commits

Author SHA1 Message Date
Brad Warren
9afb6415b8
fix up acme & certbot standalone code (#10293)
certbot's standalone code contains confusing references to things like
`SSLSocket` which we were hoping to deprecate in
https://github.com/certbot/certbot/issues/10284. are they relevant?
they're sure not!

certbot's standalone plugin only supports HTTP-01 so comments about
things like `ACMETLSServer` and the completely unused `certs` variable
can be deleted

furthermore, the type of the different variables named things like
`http_01_resources` were wrong in multiple places. as can be seen in
certbot's standalone code, the type is
`Set[acme_standalone.HTTP01RequestHandler.HTTP01Resource]`. this is also
[the type used in acme.standalone's
tests](723fe64d4d/acme/src/acme/_internal/tests/standalone_test.py (L78-L81))
despite the file's type annotations saying it takes a different type. i
think the incorrect type annotations were never caught because mypy
can't fully make sense of our overly complex server classes here

finally, `from __future__ import annotations` was added to make [forward
references in type
annotations](https://mypy.readthedocs.io/en/stable/cheat_sheet_py3.html#forward-references)
easier
2025-06-12 11:09:12 -07:00
ohemorange
680d998597
Pass in dict of acme clients instead of acme so we can wait to initialize in some cases (#10337)
Regression test fails on main with commit "add regression test"
cherry-picked onto it

```
$ pytest   certbot/src/certbot/_internal/tests/renewal_test.py 
======================================================================= test session starts =======================================================================
platform darwin -- Python 3.12.8, pytest-8.3.5, pluggy-1.5.0
rootdir: /Users/erica/certbot
configfile: pytest.ini
plugins: anyio-4.9.0, cov-6.1.1, xdist-3.6.1
collected 27 items                                                                                                                                                

certbot/src/certbot/_internal/tests/renewal_test.py .....F.....................                                                                             [100%]

============================================================================ FAILURES =============================================================================
___________________________________________________________ RenewalTest.test_no_network_if_no_autorenew ___________________________________________________________

self = <certbot._internal.tests.renewal_test.RenewalTest testMethod=test_no_network_if_no_autorenew>
mock_autorenewal_enabled = <MagicMock name='autorenewal_is_enabled' id='4378096224'>, mock_client_network_get = <MagicMock name='get' id='4378087008'>
unused_mock_display = <certbot.tests.util.FreezableMock object at 0x104eb4f50>

>   ???
E   AssertionError: assert 1 == 0
E    +  where 1 = <MagicMock name='get' id='4378087008'>.call_count

certbot/src/certbot/_internal/tests/renewal_test.py:260: AssertionError
===================================================================== short test summary info =====================================================================
FAILED certbot/src/certbot/_internal/tests/renewal_test.py::RenewalTest::test_no_network_if_no_autorenew - AssertionError: assert 1 == 0
================================================================== 1 failed, 26 passed in 0.30s ===================================================================

```
2025-06-12 11:02:22 -07:00
ohemorange
31599bad83
Reduce logging level of ARI failure to info (#10335)
This is a feature people didn't have before and won't miss if it fails.
We can always raise it later, but let's reduce it for now to stop people
worrying about the big red warning.
2025-06-12 16:16:57 +00:00
ohemorange
b682687449
Avoid ari mismatch problem during dry-run (#10332)
This is one solution to https://github.com/certbot/certbot/issues/10327.
It won't test an ARI check during a dry run, since it will just avoid
the mismatch problem by checking for dry run first and returning before
checking ARI. This PR will make the big error (actually a warning, but
red and scary) go away though.
2025-06-12 08:05:57 -07:00
ohemorange
2e827c5da6
Improve changelog entry (#10331)
I thought https://github.com/certbot/certbot/pull/9804/ was abandoned
but the author just missed my comment. I would like to accept that PR to
get it in, but in the process of updating the PR I wrote a nicer
changelog entry, so I would like to add that.
2025-06-11 16:37:43 -07:00
✨ Q (it/its) ✨
8e9d867447
Print error details when an IssuanceError is thrown (#9804)
When a CA fails to issue a certificate after finalisation Certbot
currently prints the following unhelpful message:

```
An unexpected error occurred:
acme.errors.IssuanceError
```

This PR makes Certbot print the ACME error object from the order, as
such

```
An unexpected error occurred:
CAA error :: Invalid CAA: CAA prohibits issuance
```

## Pull Request Checklist

- [ ] The Certbot team has recently expressed interest in reviewing a PR
for this. If not, this PR may be closed due our limited resources and
need to prioritize how we spend them.
- [x] If the change being made is to a [distributed
component](https://certbot.eff.org/docs/contributing.html#code-components-and-layout),
edit the `master` section of `certbot/CHANGELOG.md` to include a
description of the change being made.
- [x] Add or update any documentation as needed to support the changes
in this PR.
- [x] Include your name in `AUTHORS.md` if you like.
2025-06-11 15:15:45 -07:00
Brad Warren
1e8c09c05f
Release 4.1.0 (#10326) 2025-06-11 07:32:03 -07:00
Erica Portnoy
4a1a136fcb Bump version to 5.0.0 2025-06-10 14:43:36 -07:00
Erica Portnoy
42789114b3 Add contents to certbot/CHANGELOG.md for next version 2025-06-10 14:43:36 -07:00
Erica Portnoy
9a08102f43 Remove built packages from git 2025-06-10 14:43:36 -07:00
Erica Portnoy
6a72811a39
Release 4.1.0 2025-06-10 14:43:35 -07:00
Erica Portnoy
f417f24998 Update changelog for 4.1.0 release 2025-06-10 14:43:05 -07:00
ohemorange
10b019b3b8
moving to src layout means we need to cd into src as well to grab version number for changelog (#10325) 2025-06-10 21:40:53 +00:00
ohemorange
47b44a6751
Add a changelog entry describing the impacts of ARI on short renew_before_expiry (#10323)
Fixes #10312. This is perhaps overly detailed, but I was hoping that by
giving a viable path forward it would forestall requests to change it
back, add a flag to ignore ari, or otherwise change the behavior. Very
open to suggestions on wording/content/length/etc.

---------

Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
2025-06-09 20:11:19 +00:00
ohemorange
4c5492fbec
Use ubuntu-latest for mattermost-notify on azure (#10324)
There's no reason to be using a specific vmImage here; set it to
`ubuntu-latest` so we don't have to regularly update this. Fixes
https://github.com/certbot/certbot/issues/10322.
2025-06-09 12:45:53 -07:00
Jacob Hoffman-Andrews
1d9fc8dccf
renewal: use lineage-specific server for ARI (#10307)
Previously, we were constructing an ACME client for ARI checking that
used the global value for `server`, not the one recorded in a lineage's
renewal file.

This resulted in errors in the logs and failure to observe ARI for
lineages that used a non-default `--server` (e.g. staging or non-Let's
Encrypt CAs).

---------

Co-authored-by: ohemorange <ebportnoy@gmail.com>
2025-06-09 11:44:04 -07:00
Jacob Hoffman-Andrews
a75057042f
integration: add test for early renewal from ARI (#10311)
This depends on a pending Pebble pull request and so will fail
integration tests until/unless that lands:
https://github.com/letsencrypt/pebble/pull/501

However, I'd appreciate some eyes on this PR in this regard: is the
interface we're using in Pebble useful and appropriate? If not, we can
adjust the Pebble PR.

Inspired based on conversation on
https://github.com/certbot/certbot/pull/10307, but note that this just
tests the general case; it does not test the "default server differs
from lineage server" case yet; when I try adding that I get some bugs
that may reflect a problem in #10307 I need to fix (or may reflect that
I need to inhibit the `--server` flag rather than trying to override it
late in the command line).
2025-06-06 14:39:10 -07:00
Brad Warren
95a70e98c2
don't check ARI for expired certs (#10317)
fixes https://github.com/certbot/certbot/issues/10308

my thinking here was if the spec forbids checking ARI for expired certs,
this check should happen directly in the renewal_time function. if we do
that, what's its most useful response? error? return None? return a
datetime in the past?

i feel the latter is most helpful. tell the caller to renew now rather
than erroring out or giving it no suggestion about when it should renew

it probably doesn't matter much, but i think this would be nice to have
for 4.1.0 as it fixes a (minor) spec compliance issue in our ARI
implementation that is being released
2025-06-06 10:52:54 -07:00
ohemorange
48f34938c6
Change acme.renewal_time to only check ARI, and not also a default time. Separate out default check and use that in should_autorenew instead. (#10309)
Fixes https://github.com/certbot/certbot/issues/10298. Replacement for
#10301.

---------

Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
2025-06-04 14:48:44 -07:00
ohemorange
3cbe1288c9
Clean up renew_before_expiry default behavior (#10306)
[Recent changes](https://github.com/certbot/certbot/pull/10272/) to
`renewal.should_autorenew` assumed that if
`RenewableCert.configuration.renew_before_expiry` was set, that means
the user set it. That's wasn't true; we were throwing in a default value
if the user didn't set it. But there's no reason for that, especially
since we now set the default renewal time dynamically. Also, we were
writing out a commented `# renew_before_expiry = 30 days` without any
further documentation, in a file that we tell users they [shouldn't
really be
editing](https://eff-certbot.readthedocs.io/en/latest/using.html#modifying-the-renewal-configuration-file).
We now do neither of those things.
2025-06-02 14:19:31 -07:00
Brad Warren
e873874752
update developer OS dependency list (#10304)
this is a follow up from https://github.com/certbot/certbot/pull/10286
and related to https://github.com/certbot/certbot/issues/10302

sorry i initially missed this! in #10286 our tests were just yelling at
me about the different augeas package needed, but python headers and a
compiler are also needed for things to work with an updated version of
python-augeas

i don't believe we need this change in our macOS instructions because:

1. homebrew doesn't split up python packages the way many linux distros
do. there is no equivalent python-dev package
2. if you're using homebrew, you already have a compiler because
[homebrew requires command line tools for
xcode](https://docs.brew.sh/Installation#macos-requirements)
3. "it works on my machine"
2025-06-02 12:54:33 -07:00
Jacob Hoffman-Andrews
dbd0c6fce8
Deprecate parameter enforce_openssl_binary_usage (#10300)
Part of https://github.com/certbot/certbot/issues/10291
2025-05-29 13:28:48 -07:00
ohemorange
7a27a67cdb
Respect Retry-After header when polling for order finalization (#10288)
Fixes #10273.

---------

Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
2025-05-15 16:24:52 +00:00
Brad Warren
5d03191493
deprecate SSLSocket and TLSServer (#10294)
fixes https://github.com/certbot/certbot/issues/10284
2025-05-15 09:06:18 -07:00
Jacob Hoffman-Andrews
723fe64d4d
Add ARI support to acme module and to Certbot (#10272)
Follow-up to #10241. The acme module code is mostly the same, except the
switch to return a tuple containing Retry-After.

This includes the CLI-side work to call out to the new `renewal_time`
method when checking for renewal.

I moved `should_autorenew` from `storage.py` into `renewal.py`, where it
fits better (and also this solves an import cycle problem). To make the
edits more visible I split this into one commit for the move and [one
commit for the subsequent
edits](4e137d9b00 (diff-fad906e31304c767d620bfd243f4c7adf1e63a3420fd634ee57a0f6651c182cf)).

This does not yet attempt to store the Retry-After info, or failure
retries, in renewal configs. I figured since that's a pretty big chunk
of work and design on its own, I wanted to get interim feedback as is. I
think this PR would be okay to land with the current default crons /
systemd timers that run twice a day. I think we should implement storage
of retry information before increasing the frequency of runs. And if the
team would like to hold off on landing any ARI until that storage is
done, I'm good with that too. 👍🏻
2025-05-13 10:34:19 -07:00
Brad Warren
c5686e6653
fix mac dev augeas setup (#10287)
it appears these changes are also needed to work with python-augeas
1.2.0. i didn't catch this in
https://github.com/certbot/certbot/pull/10286 because the problem only
affects ARM macs and it appears [our CI only offers intel
macs](https://learn.microsoft.com/en-us/azure/devops/release-notes/roadmap/macos-agents-apple-silicon)

the issue here is described in homebrew issues like
https://github.com/Homebrew/brew/issues/13481 and
https://github.com/orgs/Homebrew/discussions/868. essentially, homebrew
on intel macs puts files in /usr/local which is then found by other
software by default while on arm macs it uses /opt/homebrew meaning we
have to set additional flags for things like C compilers to find headers
and libraries installed through homebrew. there was a little discussion
in https://github.com/Homebrew/brew/issues/13481 of having homebrew
fixup environment variables like `CFLAGS` by default on ARM systems, but
the issue was closed ☹️

in the meantime, this PR should fix things for certbot devs and removes
the need for the ~/lib symlinks with both new and old versions of
python-augeas
2025-05-12 11:26:40 -07:00
Brad Warren
fde359f4da
fixup http01_example.py (#10285)
it looks like https://github.com/certbot/certbot/pull/10098 introduced a
couple bugs into this file:

1.
[RSAPrivateKeys](https://cryptography.io/en/latest/hazmat/primitives/asymmetric/rsa/#cryptography.hazmat.primitives.asymmetric.rsa.RSAPrivateKey)
don't have a `public_bytes` method
2. `cryptography.x509` wasn't imported and
[load_pem_x509_certificate](https://cryptography.io/en/latest/x509/reference/#cryptography.x509.load_pem_x509_certificate)
takes bytes, not a string

i think avoiding this is unfortunately difficult as this file has no
tests, but it was useful for me just now when testing
https://github.com/certbot/certbot/pull/10283 so i wanted to fix it up

i also changed the script to initially create the account without an
email address as the fake@example.com email causes registration with
LE's staging server to fail early in execution

with the changes in this PR changes, if you:

1. change the value of
[DOMAIN](0075104805/acme/examples/http01_example.py (L57))
to a domain pointing at your machine
2. as root, activate your certbot dev environment, and run `python
acme/examples/http01_example.py `

it will fail late in the script with:
```
Traceback (most recent call last):
  File "/home/brad/certbot/acme/examples/http01_example.py", line 237, in <module>
    example_http()
    ~~~~~~~~~~~~^^
  File "/home/brad/certbot/acme/examples/http01_example.py", line 223, in example_http
    regr = client_acme.update_registration(
        regr.update(
    ...<3 lines>...
        )
    )
  File "/home/brad/certbot/acme/src/acme/client.py", line 101, in update_registration
    updated_regr = self._send_recv_regr(regr, body=body)
  File "/home/brad/certbot/acme/src/acme/client.py", line 373, in _send_recv_regr
    response = self._post(regr.uri, body)
  File "/home/brad/certbot/acme/src/acme/client.py", line 392, in _post
    return self.net.post(*args, **kwargs)
           ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/brad/certbot/acme/src/acme/client.py", line 766, in post
    return self._post_once(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/brad/certbot/acme/src/acme/client.py", line 781, in _post_once
    response = self._check_response(response, content_type=content_type)
  File "/home/brad/certbot/acme/src/acme/client.py", line 630, in _check_response
    raise messages.Error.from_json(jobj)
acme.messages.Error: urn:ietf:params:acme:error:invalidContact :: The provided contact URI was invalid :: Unable to update account :: invalid contact: contact email has forbidden domain "example.org"
```
if you also change [this email
variable](0075104805/acme/examples/http01_example.py (L223))
to a valid email address, the script will run successfully
2025-05-08 15:43:37 -07:00
Brad Warren
10747555ae
upgrade python-augeas (#10286)
a couple weeks ago, [python-augeas
1.2.0](https://pypi.org/project/python-augeas/#history) was uploaded to
pypi. unfortunately, this broke things for us

the first major change was from
https://github.com/hercules-team/python-augeas/pull/49 where
python-augeas now needs the new OS packages described in the initial
comment there

the second change was from
https://github.com/hercules-team/python-augeas/pull/51 which added a
python interface to augeas functions that weren't introduced until
[augeas
1.13.0](af2aa88ab3/NEWS (L65-L66)).
this isn't ideal, but i don't think it's a big deal for us. augeas
1.13.0 is over three years old and [ubuntu
20.04](https://ubuntu.com/blog/ubuntu-20-04-eol-for-devicesional) and
[debian bullseye](https://www.debian.org/releases/) which have older
versions than that are technically EOL'd

regardless of how we feel about these changes, our tests don't currently
work with an updated version of python-augeas and this PR fixes it. i'm
also tracking https://github.com/certbot/certbot/issues/10282 to update
certbot.eff.org to list the newly required OS packages
2025-05-08 13:03:31 -07:00
Will Greenberg
0fc755fe08
Fix 10260 (#10283)
Builds off of https://github.com/certbot/certbot/pull/7066 to stringify
these validation errors

Fixes #10260

---------

Co-authored-by: George Daramouskas <gdaramouskas@therp.nl>
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
2025-05-08 09:13:04 -07:00
Jacob Hoffman-Andrews
dcdfdacf75
store preferred/required_profile in renewal config (#10280)
This ensures that renewals of certificates will use the same profile
settings.

Fixes #10271
2025-05-07 16:32:48 -07:00
Jacob Hoffman-Andrews
0075104805
acme.ClientNetwork: JWK becomes optional (#10275)
This results in a ClientNetwork that can .get() but not .post(). Useful
for fetching ARI, which does not require authentication.
2025-05-06 12:34:50 -07:00
Alexis
2cf6cda1fa
[REPO] Update SECURITY.md (#10253)
Add supported versions

---------

Signed-off-by: zoracon <zoracon@pm.me>
2025-05-06 10:57:50 -07:00
Brad Warren
6418ee32e5
upgrade certbot compatibility test images (#10277)
i need this for some other upgrades i'm working on. using these debian
buster images which were [EOL'd ages
ago](https://www.debian.org/releases/) is giving me problems

while i was at it, i fixed up up the following warnings docker was
printing at me:

*
https://docs.docker.com/reference/build-checks/legacy-key-value-format/
* https://docs.docker.com/reference/build-checks/maintainer-deprecated/
2025-05-06 10:57:10 -07:00
Brad Warren
cc08242abc
update pinned dependencies (#10278)
this fixes the security alerts those with access can see at
https://github.com/certbot/certbot/security/dependabot

i based what needed to be done to drop python < 3.9.2 support on
https://github.com/certbot/certbot/pull/10077 and concluded we only
really needed to update `python_requires`. we could do a deprecation
period for this, but i think it's not necessary. cryptography didn't
(it's not even in mentioned in [their
changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst))
and none of the major LTS distros use python 3.9.0 or 3.9.1
2025-05-06 10:56:19 -07:00
ohemorange
62361dac44
Catch and ignore orderNotReady response when posting a request to begin finalization and poll until ready (#10239)
Fixes #9766.
2025-05-02 12:08:46 -07:00
Brad Warren
5dcfd32a11
remove unneeded cryptography req (#10276)
since https://github.com/certbot/certbot/pull/10130, we depend on much
newer versions of cryptography so this restraint is no longer needed
2025-04-30 11:47:35 -07:00
ohemorange
6ac951e146
Deprecate TLS-ALPN support in ACME (#10268)
Fixes #10266.

See example deprecation in
https://github.com/certbot/josepy/pull/207/files

I can add stacklevel=2, though I find that usually I just look at the
whole stack anyway when debugging, myself, so it doesn't really matter.
2025-04-28 15:09:15 -07:00
ohemorange
508fba1da6
Why do we have this? It means warnings aren't filtered in acme tests (#10267)
This was added in https://github.com/certbot/certbot/pull/6091 to make
tests pass in EPEL and older ubuntus, 7 years ago. It is probably no
longer needed.

| pytest version | min. Python version |
|---------------|---------------------|
|8.0+ | 3.8+|
| 7.1+ | 3.7+ |
| 6.2 - 7.0 | 3.6+ |
| 5.0 - 6.1 | 3.5+ |
| 3.3 - 4.6 | 2.7, 3.4+ |

That version is [no longer
supported](https://docs.pytest.org/en/stable/backwards-compatibility.html).

Probably therefore we can just get rid of this.
2025-04-25 20:53:31 +00:00
ohemorange
2da39317b2
Replace pyparsing error that usually misdirects people with a more helpful message (#10265)
Addresses #10264, though I could not actually find a way to fix that
particular issue. So, fixes #10264 is not actually accurate, but I would
like github to link them.
2025-04-25 13:11:28 -07:00
ohemorange
16f858547f
Add --use-pep517 flag to pip to silence warning in tools/venv.py, and switch codebase to src-layout (#10249)
Fixes #10252.

See further discussion here: https://github.com/pypa/pip/issues/11457

We are doing option:

> Alternatively, enable the --use-pep517 pip option, possibly with
--no-build-isolation. The --use-pip517 flag will force pip to use the
modern mechanism for editable installs. --no-build-isolation may be
needed if your project has build-time requirements beyond setuptools and
wheel. By passing this flag, you are responsible for making sure your
environment already has the required dependencies to build your package.
Once the legacy mechanism is removed, --use-pep517 will have no effect
and will essentially be enabled by default in this context.

Major changes made here include:
- Add `--use-pep517` to use the modern mechanism, which will be the only
mechanism in future pip releases
- Change to `/src` layout to appease mypy, and because for editable
installs that really is the normal way these days.
  - `cd acme && mkdir src && mv acme src/` etc.
- add `where='src'` argument to `find_packages` and add
`package_dir={'': 'src'},` in `setup.py`s
  - update `MANIFEST.in` files with new path locations 
- Update our many hardcoded filepaths
- Update `importlib-metadata` requirement to fix
double-plugin-entry-point problem in oldest tests
2025-04-11 19:30:33 +00:00
mirchicap
6de7570af0
Add certbot-dns-cdmon to third-party plugins list (#10258)
This PR adds `certbot-dns-cdmon` to the list of third-party plugins in
the documentation.

`certbot-dns-cdmon` enables DNS-01 challenge automation for domains
managed with cdmon's DNS.

PyPI: https://pypi.org/project/certbot-dns-cdmon/
2025-04-10 14:38:02 -07:00
Brad Warren
9bc9e3412e
Merge pull request #10261 from certbot/candidate-4.0.0 2025-04-08 10:25:34 -07:00
Will Greenberg
f822602fff Bump version to 4.1.0 2025-04-07 15:04:01 -07:00
Will Greenberg
52ad3e80bd Add contents to certbot/CHANGELOG.md for next version 2025-04-07 15:04:01 -07:00
Will Greenberg
d9e3d7b2d2 Remove built packages from git 2025-04-07 15:04:01 -07:00
Will Greenberg
d95a389c3f
Release 4.0.0 2025-04-07 15:04:00 -07:00
Will Greenberg
f7a0df3461 Update changelog for 4.0.0 release 2025-04-07 15:03:33 -07:00
Jonathan Vanasco
cb5d579a84
cloudtest some warnings (#10255) 2025-04-03 14:38:54 -07:00
Jacob Hoffman-Andrews
45626e88e2
Add --preferred-profile and --required-profile (#10230)
Fixes #10194
2025-04-03 06:48:52 +09:00
Will Greenberg
15024aabd3
Repin dependencies for josepy 2.0 (#10254) 2025-04-02 18:17:44 +00:00