Commit graph

258 commits

Author SHA1 Message Date
Mark Johnston
27211b7998 mlx5: Remove a less than helpful debug print
Reviewed by:	khng
Fixes:	e23731db48 ("mlx5en: add IPSEC_OFFLOAD support")
Differential Revision:	https://reviews.freebsd.org/D46273
2024-08-12 23:06:01 +00:00
Konstantin Belousov
e23731db48 mlx5en: add IPSEC_OFFLOAD support
Right now, only IPv4 transport mode, with aes-gcm ESP, is supported.
Driver also cooperates with NAT-T, and obeys socket policies, which
makes IKEd like StrongSwan working.

Sponsored by:	NVIDIA networking
2024-07-30 18:00:04 +03:00
Kristof Provost
7deadea2eb mlx5: handle vlan PF restrictions
Reviewed by:	kib
MFC after:	2 weeks
Sponsored by:	Orange Business Services
Differential Revision:	https://reviews.freebsd.org/D45427
2024-06-17 10:50:16 +02:00
Mark Johnston
47a6fb9d5a mlx5: Zero DMA memory mlx5_alloc_cmd_msg() and alloc_cmd_page()
These functions may map more memory for DMA than is actually used, since
the allocator operates on multiples of a 4KB page size.  Thus,
bus_dmamap_sync() can trigger KMSAN reports when the unused portion of
a page is not zero-ed.

Reported by:	KMSAN
Reviewed by:	kib
MFC after:	2 weeks
Sponsored by:	Klara, Inc.
Sponsored by:	Juniper Networks, Inc.
Differential Revision:	https://reviews.freebsd.org/D43133
2024-01-18 16:52:17 -05:00
Konstantin Belousov
987446fa39 mlx5(4): only detach IOV children if iov was successfully initialized
Reported by:	jwd
Sponsored by:	NVidia networking
MFC after:	1 week
2024-01-05 06:52:39 +02:00
Patrisious Haddad
0cd90ee598 mlx5: Fix HCA cap 2 query
Previously we were trying to set hca_cap_2 without checking if
sw_vhca_id_valid max value, which is the only settable value inside
hca_cap_2, and seeing that we dont have driver support for sw_vhca_id
yet there is no need to set hca_cap_2 at all, it is enough to query it.

Fixes: 7b959396ca ("mlx5: Introduce new destination type TABLE_TYPE")
MFC after:	3 days
2023-12-03 10:21:44 +02:00
Warner Losh
fdafd315ad sys: Automated cleanup of cdefs and other formatting
Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by:		Netflix
2023-11-26 22:24:00 -07:00
Martin Matuska
a592812327 mlx5_core: fix deadlock when using RXTLS
If removing a node of type FS_TYPE_FLOW_DEST we lock the flow group too
late. This can lead to a deadlock with fs_add_dst_fg().

PR:		274715
MFC after:	1 week
Reviewed by:	kib
Tested by:	mm
Differential Revision: https://reviews.freebsd.org/D42368
2023-11-16 12:17:41 +01:00
Patrisious Haddad
7b959396ca mlx5: Introduce new destination type TABLE_TYPE
This new destination type supports flow transition between different
table types, e.g. from NIC_RX to RDMA_RX or from RDMA_TX to NIC_TX.

In addition add driver support to be able to query the capability for
this new destination type.

Signed-off-by: Patrisious Haddad <phaddad@nvidia.com>
Sponsored by:	NVidia networking
MFC after:	1 week
2023-11-16 01:08:17 +02:00
Mark Bloch
ad74454131 mlx5: add ability to attach flow counter to steering rule
Expose a way to attach a counter to a flow rule.

Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Sponsored by:	NVidia networking
MFC after:	1 week
2023-11-16 01:08:17 +02:00
Raed Salem
35bbcf0916 mlx5: add fs_counters
Signed-off-by: Raed Salem <raeds@nvidia.com>
Sponsored by:	NVidia networking
MFC after:	1 week
2023-11-16 01:08:17 +02:00
Mark Bloch
6a6af22b6e mlx5: Add a no-append flow insertion mode
This allows to insert a rule and make sure it doesn't get
combined by the steering layer with any other rule.

Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Sponsored by:	NVidia networking
MFC after:	1 week
2023-11-16 01:08:16 +02:00
Mark Bloch
0a5db6bb3a net/mlx5: Allow creating autogroups with reserved entries
Exclude the last n entries for an autogrouped flow table.

Reserving entries at the end of the FT will ensure that this FG will be
the last to be evaluated. This will be used in the next patch to create

Linux upstream commit: 79cdb0aaea8b5478db34afa1d4d5ecc808689a67
Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Sponsored by:	NVidia networking
MFC after:	1 week
2023-11-16 01:08:16 +02:00
Mark Bloch
04db54fe43 net/mlx5: Fix auto group size calculation
Once all the large flow groups (defined by the user when the flow table
is created - max_num_groups) were created, then all the following new
flow groups will have only one flow table entry, even though the flow table
has place to larger groups.
Fix the condition to prefer large flow group.

Upstream Linux commit: 97fd8da281f80e7e69e0114bc906575734d4dfaf
Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Sponsored by:	NVidia networking
MFC after:	1 week
2023-11-16 01:08:16 +02:00
Mark Bloch
76ed99ed8a mlx5: Use software enum in APIs instead of PRM
Users of the steering APIs shouldn't use the PRM directly.
Create an software enum to be used instead.

Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Sponsored by:	NVidia networking
MFC after:	1 week
2023-11-16 01:08:16 +02:00
Mark Bloch
45e2e55df6 mlx5: Add packet reformat support to flow rules
Allow attaching a packet reformat action to a flow rule.

Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Sponsored by:	NVidia networking
MFC after:	1 week
2023-11-16 01:08:16 +02:00
Mark Bloch
847542c60c mlx5: Add modify header support to flow rules
Allow attaching a modify header to a flow rule.

Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Sponsored by:	NVidia networking
MFC after:	1 week
2023-11-16 01:08:16 +02:00
Mark Bloch
cb054a493a mlx5: Refactor flow actions into a struct
Create a struct to hold flow actions to be used when creating
a flow rule.

Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Sponsored by:	NVidia networking
MFC after:	1 week
2023-11-16 01:08:16 +02:00
Mark Bloch
bb4645b95b mlx5: Add packet reformat allocation support
Add support to allocating a packet reformat context.

Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Sponsored by:	NVidia networking
MFC after:	1 week
2023-11-16 01:08:16 +02:00
Mark Bloch
e4f84168b3 mlx5: Add modify header support
Add support to allocating a modify header context.

Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Sponsored by:	NVidia networking
MFC after:	1 week
2023-11-16 01:08:15 +02:00
Warner Losh
685dc743dc sys: Remove $FreeBSD$: one-line .c pattern
Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/
2023-08-16 11:54:36 -06:00
Warner Losh
95ee2897e9 sys: Remove $FreeBSD$: two-line .h pattern
Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/
2023-08-16 11:54:11 -06:00
Hans Petter Selasky
aa7bbdabde mlx5: Implement diagostic counters as sysctl(8) nodes.
MFC after:	1 week
Sponsored by:	NVIDIA Networking
2023-04-18 15:01:07 +02:00
Hans Petter Selasky
95bf70a4bf mlx5: Don't give zero number of pages to the firmware.
Can happen when using virtual mlx5_core<N> functions, VFs.

MFC after:	1 week
Sponsored by:	NVIDIA Networking
2023-04-18 15:01:06 +02:00
Hans Petter Selasky
273bfac08f mlx5: Implement mlx5_core_modify_cq_by_mask().
Implement one CQ modify function supporting all firmware versions,
instead of having more variants of CQ modify.

MFC after:	1 week
Sponsored by:	NVIDIA Networking
2023-04-18 15:01:06 +02:00
Hans Petter Selasky
2f7e9a8a21 mlx5: Fix duplicate free of default flow rule in error case.
MFC after:	1 week
Sponsored by:	NVIDIA Networking
2023-04-18 15:01:06 +02:00
Hans Petter Selasky
b0b87d9151 mlx5: Make mlx5_del_flow_rule() NULL safe.
This change factors out repeated NULL checks.

No functional change intended.

MFC after:	1 week
Sponsored by:	NVIDIA Networking
2023-04-18 15:01:06 +02:00
Hans Petter Selasky
3bb3e4768f mlx5: Make MLX5_COMP_EQ_SIZE tunable.
When using hardware pacing, this value can be increased, because more SQ's
means more EQ events aswell. Make it tunable, hw.mlx5.comp_eq_size .

MFC after:	1 week
Sponsored by:	NVIDIA Networking
2023-04-18 15:01:06 +02:00
John Baldwin
f49fd63a6a kmem_malloc/free: Use void * instead of vm_offset_t for kernel pointers.
Reviewed by:	kib, markj
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D36549
2022-09-22 15:09:19 -07:00
Gordon Bergling
e1a40dd294 mlx5en(4): Correct a typo in a kernel error message
- s/ouput/output

MFC after:	5 days
2022-09-03 19:29:33 +02:00
Dimitry Andric
6332ad8673 Fix unused variable warning in mlx5_fs_tree.c
With clang 15, the following -Werror warning is produced:

    sys/dev/mlx5/mlx5_core/mlx5_fs_tree.c:1408:15: error: variable 'candidate_group_num' set but not used [-Werror,-Wunused-but-set-variable]
            unsigned int candidate_group_num = 0;
                         ^

The 'candidate_group_num' variable appears to have been a debugging aid
that has never been used for anything, so remove it.

MFC after:	3 days
2022-07-25 00:40:13 +02:00
John Baldwin
75095cd38f mlx5: fs_tcp is only used for INET or INET6. 2022-04-13 16:08:21 -07:00
John Baldwin
834533b9c5 mlx5: Remove unused variables.
Reviewed by:	hselasky
Differential Revision:	https://reviews.freebsd.org/D34827
2022-04-08 17:25:13 -07:00
John Baldwin
ebb16d5e93 mlx5: Pass the correct data pointer to the add_dst_cb instead of NULL.
Reported by:	-Wunused-but-set-variable
Reviewed by:	hselasky
Differential Revision:	https://reviews.freebsd.org/D34812
2022-04-07 10:46:48 -07:00
John Baldwin
6301649e0e mlx5: Remove write-only variables. 2022-04-06 16:45:28 -07:00
Hans Petter Selasky
b18c510844 mlx5/mlx4: Bump driver version to 3.7.1
MFC after:	1 week
Sponsored by:	NVIDIA Networking
2022-03-08 13:12:03 +01:00
Hans Petter Selasky
eb16e362d6 mlx5core: Add PCI IDs for ConnectX-8.
MFC after:	1 week
Sponsored by:	NVIDIA Networking
2022-02-21 09:35:19 +01:00
Hans Petter Selasky
ea8aacc523 mlx5core: Add PCI IDs for ConnectX-7.
Linux commits:
505a7f5478062c6cd11e22022d9f1bf64cd8eab3
dd8595eabeb486d41ad9994e6cece36e0e25e313

MFC after:	1 week
Sponsored by:	NVIDIA Networking
2022-02-21 09:35:19 +01:00
Hans Petter Selasky
e6d7ac1d03 mlx5core: Set driver version into firmware.
If the driver_version capability bit is enabled, send the driver
version to firmware after the init HCA command, for display purposes.

Example of driver version: "FreeBSD,mlx5_core,14.0.0,3.x-xxx"

Linux commits:
012e50e109fd27ff989492ad74c50ca7ab21e6a1

MFC after:	1 week
Sponsored by:	NVIDIA Networking
2022-02-01 16:21:17 +01:00
Hans Petter Selasky
ea00d7e8ca mlx5: Add raw ethernet local loopback support.
Currently, unicast/multicast loopback raw ethernet (non-RDMA) packets
are sent back to the vport.  A unicast loopback packet is the packet
with destination MAC address the same as the source MAC address.  For
multicast, the destination MAC address is in the vport's multicast
filter list.

Moreover, the local loopback is not needed if there is one or none
user space context.

After this patch, the raw ethernet unicast and multicast local
loopback are disabled by default. When there is more than one user
space context, the local loopback is enabled.

Note that when local loopback is disabled, raw ethernet packets are
not looped back to the vport and are forwarded to the next routing
level (eswitch, or multihost switch, or out to the wire depending on
the configuration).

Linux commits:
c85023e153e3824661d07307138fdeff41f6d86a
8978cc921fc7fad3f4d6f91f1da01352aeeeff25

MFC after:	1 week
Sponsored by:	NVIDIA Networking
2022-02-01 16:21:16 +01:00
Hans Petter Selasky
c1b76119cb mlx5: Implement mlx5_nic_vport_update_local_lb()
MFC after:	1 week
Sponsored by:	NVIDIA Networking
2022-02-01 16:21:16 +01:00
Hans Petter Selasky
2c0ade806a mlx5: Implement flow steering helper functions for TCP sockets.
This change adds convenience functions to setup a flow steering rule based on
a TCP socket. The helper function gets all the address information from the
socket and returns a steering rule, to be used with HW TLS RX offload.

MFC after:	1 week
Sponsored by:	NVIDIA Networking
2022-02-01 16:21:16 +01:00
Hans Petter Selasky
0ee1b09eaa mlx5: Implement offloads flowtable namespace.
This namespace will be used for TCP offloads, like hardware decryption
of TLS TCP data.

MFC after:	1 week
Sponsored by:	NVIDIA Networking
2022-02-01 16:21:16 +01:00
Hans Petter Selasky
266c81aae3 mlx5/mlx5en: Add SQ remap support
Add support to map an SQ to a specific schedule queue using a
special WQE as performance enhancement.

SQ remap operation is handled by a privileged internal queue, IQ,
and the mapping is enabled from one rate to another.

The transition from paced to non-paced should however always go
through FW.

MFC after:	1 week
Sponsored by:	NVIDIA Networking
2022-02-01 16:21:15 +01:00
Hans Petter Selasky
21228c67ab mlx5en: Implement helper functions to open and close TLS TIR context.
MFC after:	1 week
Sponsored by:	NVIDIA Networking
2022-02-01 16:21:15 +01:00
Hans Petter Selasky
12c56d7dc4 mlx5: idiomatic use of preprocessor, in particular paths
MFC after:	1 week
Sponsored by:	NVIDIA Networking
2022-02-01 14:45:57 +02:00
Konstantin Belousov
ee9d634bd3 mlx5: normalize use of the opt_*.h files
MFC after:	1 week
Sponsored by:	NVIDIA Networking
2022-02-01 14:45:57 +02:00
Warner Losh
c6df6f5322 Create wrapper for Giant taken for newbus
Create a wrapper for newbus to take giant and for busses to take it too.
bus_topo_lock() should be called before interacting with newbus routines
and unlocked with bus_topo_unlock(). If you need the topology lock for
some reason, bus_topo_mtx() will provide that.

Sponsored by:		Netflix
Reviewed by:		mav
Differential Revision:	https://reviews.freebsd.org/D31831
2021-12-09 17:04:45 -07:00
Hans Petter Selasky
d2cbfbc57b mlx5/mlx4: Bump driver version to 3.7
While at it only output driver version to dmesg(8) when hardware is present.

Differential Revision:	https://reviews.freebsd.org/D29100
MFC after:	1 week
Reviewed by:	kib and markj
Sponsored by:	NVIDIA Networking
2021-07-28 13:47:05 +02:00
Hans Petter Selasky
b633e08c70 ibcore: Kernel space update based on Linux 5.7-rc1.
Overview:

This is the first stage of a RDMA stack upgrade introducing kernel
changes only based on Linux 5.7-rc1.

This patch is based on about four main areas of work:
- Update of the IB uobjects system:
  - The memory holding so-called AH, CQ, PD, SRQ and UCONTEXT objects
    is now managed by ibcore. This also require some changes in the
    kernel verbs API. The updated verbs changes are typically about
    initialize and deinitialize objects, and remove allocation and
    free of memory.

- Update of the uverbs IOCTL framework:
  - The parsing and handling of user-space commands has been
    completely refactored to integrate with the updated IB uobjects
    system.

- Various changes and updates to the generic uverbs interfaces in
  device drivers including the new uAPI surface.

- The mlx5_ib_devx.c in mlx5ib and related mlx5 core changes.

Dependencies:

- The mlx4ib driver code has been updated with the minimum changes
needed.

- The mlx5ib driver code has been updated with the minimum changes
needed including DV support.

Compatibility:

- All user-space facing APIs are backwards compatible after this
  change.

- All kernel-space facing RDMA APIs are backwards compatible after
  this change, with exception of ib_create_ah() and ib_destroy_ah()
  which takes a new flag.

- The "ib_device_ops" structure exist, but only contains the driver ID
  and some structure sizes.

Differences from Linux:

- Infiniband drivers must use the INIT_IB_DEVICE_OPS() macro to set
  the sizes needed for allocating various IB objects, when adding
  IB device instances.

Security:

- PRIV_NET_RAW is needed to use raw ethernet transmit features.
- PRIV_DRIVER is needed to use other privileged operations.

Based on upstream Linux, Torvalds (5.7-rc1):
8632e9b5645bbc2331d21d892b0d6961c1a08429

MFC after:	1 week
Reviewed by:	kib
Differential Revision:	https://reviews.freebsd.org/D31149
Sponsored by:	NVIDIA Networking
2021-07-28 13:28:29 +02:00