Only map mbuf when a policy is looked up and indicates that IPSEC needs
to transform the packet. If IPSEC is inline offloaded, it is up to the
interface driver to request remap if needed.
Fetch the IP header using m_copydata() instead of using mtod() to select
policy/SA.
Reviewed by: markj
Sponsored by: NVidia networking
Differential revision: https://reviews.freebsd.org/D48265
Besides not doing any sufficient check that the length of a parsed
message is not bigger than the actual allocated buffer, kdebug_sadb()
incorrectly compares ext->sadb_ext_len, the extension payload size in 8
byte chunks, with tlen, which is the full message payload size in bytes.
This should compare PFKEY_UNUNIT64(ext->sadb_ext_len) with tlen instead.
PR: 277456
MFC after: 2 weeks
Properly initialize setdf variable in ipsec_encap().
It is used for AF_INET6 case when IPv6 datagram is going to be
encapsulated into IPv4 datagram.
PR: 282535
Fixes: 4046178557
MFC after: 1 week
sys/netinet and sys/netipsec are both part of the 'blessed' netstack, so
can access struct ifnet directly. With this structure becoming private
very soon, the necessary files need to get direct access.
Sponsored by: Juniper Networks, Inc.
Re-apply commit e196b12f4d. This was reverted by commit 28294dc924
because it could trigger a deadlock, but the underlying problem there
was fixed in commit f76826b892.
Reported by: KASAN
Reviewed by: kib
Fixes: ef2a572bf6 ("ipsec_offload: kernel infrastructure")
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D46483
Using global taskqueue_thread XXX with the vnet tasks scheduled during
VNET destruction. VNET shutdown needs to wait for all vnet-scoped
SAs/SPs to be handled, and doing that from taskqueue_thread task
deadlocks because the same thread proceeds the removals.
Reviewed by: markj
Sponsored by: NVidia networking
Differential revision: https://reviews.freebsd.org/D46494
This change can cause a deadlock in some cases, since it's possible for
VNET teardown to happen in the context of taskqueue_thread, and
ipsec_accel_sync() drains taskqueue_thread's work queue.
This reverts commit e196b12f4d.
The ipsec_offload code in some cases releases object references in an
asynchronous context where it needs to set the current VNET. Make sure
that all such work completes before the VNET is actually destroyed,
otherwise a use-after-free is possible.
Reported by: KASAN
Reviewed by: kib
Fixes: ef2a572bf6 ("ipsec_offload: kernel infrastructure")
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D46483
Extend what we did for netinet counters in 60d8dbbef0 (netinet: add a probe
point for IP, IP6, ICMP, ICMP6, UDP and TCP stats counters, 2024-01-18) to the
IPsec code.
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D46416
Specifically, ipsec_accel_on_ifdown() and ipsec_accel_drv_sa_lifetime_update()
should be present in kernel for future mlx5en driver to be statically
linkable into the kernel built with IPSEC_HOOKS + IPSEC_OFFLOAD.
Sponsored by: NVIDIA networking
Inline IPSEC offload moves almost whole IPSEC processing from the
CPU/MCU and possibly crypto accelerator, to the network card.
The transmitted packet content is not touched by CPU during TX
operations, kernel only does the required policy and security
association lookups to find out that given flow is offloaded, and then
packet is transmitted as plain text to the card. For driver convenience,
a metadata is attached to the packet identifying SA which must process
the packet. Card does encryption of the payload, padding, calculates
authentication, and does the reformat according to the policy.
Similarly, on receive, card does the decapsulation, decryption, and
authentification. Kernel receives the identifier of SA that was
used to process the packet, together with the plain-text packet.
Overall, payload octets are only read or written by card DMA engine,
removing a lot of memory subsystem overhead, and saving CPU time because
IPSEC algos calculations are avoided.
If driver declares support for inline IPSEC offload (with the
IFCAP2_IPSEC_OFFLOAD capability set and registering method table struct
if_ipsec_accel_methods), kernel offers the SPD and SAD to driver.
Driver decides which policies and SAs can be offloaded based on
hardware capacity, and acks/nacks each SA for given interface to
kernel. Kernel needs to keep this information to make a decision to
skip software processing on TX, and to assume processing already done
on RX. This shadow SPD/SAD database of offloads is rooted from
policies (struct secpolicy accel_ifps, struct ifp_handle_sp) and SAs
(struct secasvar accel_ipfs, struct ifp_handle_sav).
Some extensions to the PF_KEY socket allow to limit interfaces for
which given SP/SA could be offloaded (proposed for offload). Also,
additional statistics extensions allow to observe allocation/octet/use
counters for specific SA.
Since SPs and SAs are typically instantiated in non-sleepable context,
while offloading them into card is expected to require costly async
manipulations of the card state, calls to the driver for offload and
termination are executed in the threaded taskqueue. It also solves
the issue of allocating resources needed for the offload database.
Neither ipf_handle_sp nor ipf_handle_sav do not add reference to the
owning SP/SA, the offload must be terminated before last reference is
dropped. ipsec_accel only adds transient references to ensure safe
pointer ownership by taskqueue.
Maintaining the SA counters for hardware-accelerated packets is the
duty of the driver. The helper ipsec_accel_drv_sa_lifetime_update()
is provided to hide accel infrastructure from drivers which would use
expected callout to query hardware periodically for updates.
Reviewed by: rscheff (transport, stack integration), np
Sponsored by: NVIDIA networking
Differential revision: https://reviews.freebsd.org/D44219
Similarly, mtu is needed to decide inline IPSEC offloiad for the driver.
Sponsored by: NVIDIA networking
Differential revision: https://reviews.freebsd.org/D44224
The information about the interface is needed to coordinate inline
offloading of IPSEC processing with corresponding driver.
Sponsored by: NVIDIA networking
Differential revision: https://reviews.freebsd.org/D44223
- add const to the appropriate places in the libipsec public API and the
relevant internal functions needed to support that.
- replace caddr_t with c_caddr_t in ipsec_dump_policy()
- update the ipsec_dump_policy manpage to use c_caddr_t (this manpage
was already wrong as it had "char *" instead of caddr_t previously).
While here, update pfkeyv2.h to not cast away const in the PFKEY_*()
macros.
This should not cause any ABI changes as the actual types have not
changed.
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/1099
The functions ipsec_kmod_udp_input() and ipsec_kmod_udp_pcbctl() are
used by netinet6 for IPSEC_SUPPORT, but are guarded behind #ifdef INET.
Since neither of these require INET, remove the guard so they're built
even without INET.
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/1158
It is possible that SA was removed while processing packed, in which
case it is changed to the DEAD state and it index is removed from the
tree. Dereferencing sav->sah then touches freed memory.
Reviewed by: ae
Sponsored by: NVIDIA networking
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D44079
This patch provides UDP encapsulation of ESP packets over IPv6.
Ports the IPv4 code to IPv6 and adds support for IPv6 in udpencap.c
As required by the RFC and unlike in IPv4 encapsulation,
UDP checksums are calculated.
Co-authored-by: Aurelien Cazuc <aurelien.cazuc.external@stormshield.eu>
Sponsored-by: Stormshield
Sponsored-by: Wiktel
Sponsored-by: Klara, Inc.
My failure to run all kinds of kernel builds lead to missing the keysock
and incorrectly assuming SDP as not having a shutdown method.
Fixes: 5bba272807
Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.
Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/
Sponsored by: Netflix
Skip the UDP header for the computation. This is similar to
skipping IPv6 extension headers.
Reviewed by: cc, rscheff
MFC after: 3 days
Sponsored by: Netflix, Inc.
Differential Revision: https://reviews.freebsd.org/D40596
The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.
Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix
to limit cache-line bouncing.
Note that as there is no atomic_store we are hoping the compiler wont
speculatively do the store. It is not employed because the size depends
on target arch.
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D38433
Summary:
In preparation of making if_t completely opaque outside of the netstack,
explicitly include the header. <net/if_var.h> will stop including the
header in the future.
Sponsored by: Juniper Networks, Inc.
Reviewed by: glebius, melifaro
Differential Revision: https://reviews.freebsd.org/D38200
Various handlers for SADB messages will allocate a new mbuf and populate
some structures in it. Some of these structures, such as struct
sadb_supported, contain small reserved fields that are not initialized
and are thus leaked to userspace.
Fix the problem by adding a helper to allocate zeroed mbufs. This
reduces code duplication and the overhead of zeroing these messages
isn't harmful.
Reviewed by: zlei, melifaro
Reported by: KMSAN
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D38068
For the TCP protocol inpcb storage specify allocation size that would
provide space to most of the data a TCP connection needs, embedding
into struct tcpcb several structures, that previously were allocated
separately.
The most import one is the inpcb itself. With embedding we can provide
strong guarantee that with a valid TCP inpcb the tcpcb is always valid
and vice versa. Also we reduce number of allocs/frees per connection.
The embedded inpcb is placed in the beginning of the struct tcpcb,
since in_pcballoc() requires that. However, later we may want to move
it around for cache line efficiency, and this can be done with a little
effort. The new intotcpcb() macro is ready for such move.
The congestion algorithm data, the TCP timers and osd(9) data are
also embedded into tcpcb, and temprorary struct tcpcb_mem goes away.
There was no extra allocation here, but we went through extra pointer
every time we accessed this data.
One interesting side effect is that now TCP data is allocated from
SMR-protected zone. Potentially this allows the TCP stacks or other
TCP related modules to utilize that for their own synchronization.
Large part of the change was done with sed script:
s/tp->ccv->/tp->t_ccv./g
s/tp->ccv/\&tp->t_ccv/g
s/tp->cc_algo/tp->t_cc/g
s/tp->t_timers->tt_/tp->tt_/g
s/CCV\(ccv, osd\)/\&CCV(ccv, t_osd)/g
Dependency side effect is that code that needs to know struct tcpcb
should also know struct inpcb, that added several <netinet/in_pcb.h>.
Differential revision: https://reviews.freebsd.org/D37127
Saves on work in a common case of checking both directions.
Note further work in the area is impending to elide these in the common
case to begin with.
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D36485
