Commit graph

526 commits

Author SHA1 Message Date
Hans Petter Selasky
16635c7b21 mlx5en: Make sure the NIC IP addresses are written to firmware on link up.
Fixes e059c120b4 .

PR:		261746
Sponsored by:	NVIDIA Networking

(cherry picked from commit 04f407a3e5)
2022-02-11 11:15:00 +01:00
Hans Petter Selasky
7e5b40d818 mlx5core: Set driver version into firmware.
If the driver_version capability bit is enabled, send the driver
version to firmware after the init HCA command, for display purposes.

Example of driver version: "FreeBSD,mlx5_core,14.0.0,3.x-xxx"

Linux commits:
012e50e109fd27ff989492ad74c50ca7ab21e6a1

Sponsored by:	NVIDIA Networking

(cherry picked from commit e6d7ac1d03)
2022-02-08 16:08:54 +01:00
Hans Petter Selasky
5ddb1a584d mlx5en: Implement one RQT object per channel.
These objects will eventually be used to switch TLS RX traffic.

Sponsored by:	NVIDIA Networking

(cherry picked from commit 8e332232a5)
2022-02-08 16:08:54 +01:00
Hans Petter Selasky
823bcb3a13 mlx5: Add raw ethernet local loopback support.
Currently, unicast/multicast loopback raw ethernet (non-RDMA) packets
are sent back to the vport.  A unicast loopback packet is the packet
with destination MAC address the same as the source MAC address.  For
multicast, the destination MAC address is in the vport's multicast
filter list.

Moreover, the local loopback is not needed if there is one or none
user space context.

After this patch, the raw ethernet unicast and multicast local
loopback are disabled by default. When there is more than one user
space context, the local loopback is enabled.

Note that when local loopback is disabled, raw ethernet packets are
not looped back to the vport and are forwarded to the next routing
level (eswitch, or multihost switch, or out to the wire depending on
the configuration).

Linux commits:
c85023e153e3824661d07307138fdeff41f6d86a
8978cc921fc7fad3f4d6f91f1da01352aeeeff25

Sponsored by:	NVIDIA Networking

(cherry picked from commit ea00d7e8ca)
2022-02-08 16:08:54 +01:00
Hans Petter Selasky
7fb8dd15fa mlx5: Implement mlx5_nic_vport_update_local_lb()
Sponsored by:	NVIDIA Networking

(cherry picked from commit c1b76119cb)
2022-02-08 16:08:54 +01:00
Hans Petter Selasky
952d47e5cc mlx5en: Create TIRs before flowtables.
Because flowtables may redirect traffic to TIRs.

Sponsored by:	NVIDIA Networking

(cherry picked from commit 5381f93647)
2022-02-08 16:08:54 +01:00
Hans Petter Selasky
0c4de0f986 mlx5en: Create flowtables in correct order.
Because it affects how the flow tables may re-direct traffic.

Sponsored by:	NVIDIA Networking

(cherry picked from commit 001106f807)
2022-02-08 16:08:53 +01:00
Hans Petter Selasky
26b0e75d64 mlx5: Implement flow steering helper functions for TCP sockets.
This change adds convenience functions to setup a flow steering rule based on
a TCP socket. The helper function gets all the address information from the
socket and returns a steering rule, to be used with HW TLS RX offload.

Sponsored by:	NVIDIA Networking

(cherry picked from commit 2c0ade806a)
2022-02-08 16:08:53 +01:00
Hans Petter Selasky
513a6d84e9 mlx5: Implement offloads flowtable namespace.
This namespace will be used for TCP offloads, like hardware decryption
of TLS TCP data.

Sponsored by:	NVIDIA Networking

(cherry picked from commit 0ee1b09eaa)
2022-02-08 16:08:53 +01:00
Hans Petter Selasky
f42aaf43d3 mlx5en: Create and destroy all flow tables and rules when the network interface attaches and detaches.
Previously flow steering tables and rules were only created and destroyed
at link up and down events, respectivly. Due to new requirements for adding
TLS RX flow tables and rules, the main flow steering table must always be
available as there are permanent redirections from the TLS RX flow table
to the vlan flow table.

Sponsored by:	NVIDIA Networking

(cherry picked from commit e059c120b4)
2022-02-08 16:08:53 +01:00
Hans Petter Selasky
e441376f45 mlx5en: Add race protection for SQ remap
Add a refcount for posted WQEs to avoid a race between
post WQE and FW command flows.

Sponsored by:	NVIDIA Networking

(cherry picked from commit a8e715d21b)
2022-02-08 16:08:53 +01:00
Hans Petter Selasky
75312cafe7 mlx5en: Properly account for no-checksum on tunneled packets.
Sponsored by:	NVIDIA Networking

(cherry picked from commit aabca1034c)
2022-02-08 16:08:53 +01:00
Hans Petter Selasky
428331a8bf mlx5en: Force all packets through the indirection table.
All packets must go through the indirection table, RQT,
because it is not possible to modify the RQN of the TIR
for direct dispatchment after it is created, typically
when the link goes up and down.

Sponsored by:	NVIDIA Networking

(cherry picked from commit 06c2bd1872)
2022-02-08 16:08:53 +01:00
Hans Petter Selasky
c7ec839e30 mlx5/mlx5en: Add SQ remap support
Add support to map an SQ to a specific schedule queue using a
special WQE as performance enhancement.

SQ remap operation is handled by a privileged internal queue, IQ,
and the mapping is enabled from one rate to another.

The transition from paced to non-paced should however always go
through FW.

Sponsored by:	NVIDIA Networking

(cherry picked from commit 266c81aae3)
2022-02-08 16:08:53 +01:00
Hans Petter Selasky
57b86395fe mlx5: Properly define the reg_umr_sq networking offload capability bit.
Sponsored by:	NVIDIA Networking

(cherry picked from commit 1c407d0494)
2022-02-08 16:08:52 +01:00
Hans Petter Selasky
95854550de mlx5en: Only delete installed VxLAN rules.
Sponsored by:	NVIDIA Networking

(cherry picked from commit 9680b1ba71)
2022-02-08 16:08:52 +01:00
Hans Petter Selasky
2c8132accd mlx5en: Fix inverted logical assignment.
Sponsored by:	NVIDIA Networking

(cherry picked from commit 6176a5e338)
2022-02-08 16:08:52 +01:00
Hans Petter Selasky
e53c3826c0 mlx5en: Implement support for internal queues, IQ.
Internal send queues are regular sendqueues which are reserved for WQE commands
towards the hardware and firmware. These queues typically carry resync
information for ongoing TLS RX connections and when changing schedule queues
for rate limited connections.

The internal queue, IQ, code is more or less a stripped down copy
of the existing SQ managing code with exception of:

1) An optional single segment memory buffer which can be read or
   written as a whole by the hardware, may be provided.
2) An optional completion callback for all transmit operations, may
   be provided.
3) Does not support mbufs.

Sponsored by:	NVIDIA Networking

(cherry picked from commit 694263572f)
2022-02-08 16:08:52 +01:00
Hans Petter Selasky
4deb6e4fab mlx5en: Implement helper functions to open and close TLS TIR context.
Sponsored by:	NVIDIA Networking

(cherry picked from commit 21228c67ab)
2022-02-08 16:08:52 +01:00
Hans Petter Selasky
9aca511703 mlx5en: Share DEK objects with TLS RX.
The TLS RX support also needs to be able to allocate DEK objects.
Share the available objects 1:1.

Sponsored by:	NVIDIA Networking

(cherry picked from commit 75767cb889)
2022-02-08 16:08:52 +01:00
Hans Petter Selasky
b5b957a8db mlx5en: Add missing TLS structure prototype.
Sponsored by:	NVIDIA Networking

(cherry picked from commit fad4b7d1f2)
2022-02-08 16:08:52 +01:00
Hans Petter Selasky
b324932190 mlx5en: Remove unused hardware TLS field.
Sponsored by:	NVIDIA Networking

(cherry picked from commit 3a1bf85503)
2022-02-08 16:08:52 +01:00
Hans Petter Selasky
3954e5380a mlx5en: Make the receive packet indirection table, RQT, static instead of dynamic.
Allocate the RQT once, pointing all initial entries to the drop RQN.
When opening the channels simplify modify the RQT, directing all traffic
to the new RQNs. Similarly when closing the channels point all RQT entries
back to the so-called drop RQN.

Sponsored by:	NVIDIA Networking

(cherry picked from commit 33a6a7a72a)
2022-02-08 16:08:52 +01:00
Hans Petter Selasky
0c8b126da5 mlx5en: Set CQN in RQ parameters for drop RQ.
Else creating the drop RQ fails.

Sponsored by:	NVIDIA Networking

(cherry picked from commit 7800af352a)
2022-02-08 16:08:51 +01:00
Hans Petter Selasky
e2c02a4920 mlx5en: Set channel pointer for drop receive queue.
A valid channel pointer is needed to get the priv pointer during init.

Sponsored by:	NVIDIA Networking

(cherry picked from commit 03567b0dfa)
2022-02-08 16:08:51 +01:00
Hans Petter Selasky
e5b00e9b64 mlx5en: Print error code when opening drop RQ fails.
Sponsored by:	NVIDIA Networking

(cherry picked from commit 4e40e984da)
2022-02-08 16:08:51 +01:00
Hans Petter Selasky
62283a6345 mlx5en: Implement dummy receive queue, RQ, for dropping packets.
What is a drop RQ and why is it needed?

The RSS indirection table, also called the RQT, selects the
destination RQ based on the receive queue number, RQN. The RQT is
frequently referred to by flow steering rules to distribute traffic
among multiple RQs. The problem is that the RQs cannot be destroyed
before the RQT referring them is destroyed too. Further, TLS RX
rules may still be referring to the RQT even if the link went
down. Because there is no magic RQN for dropping packets, we create
a dummy RQ, also called drop RQ, which sole purpose is to drop all
received packets. When the link goes down this RQN is filled in all
RQT entries, of the main RQT, so the real RQs which are about to be
destroyed can be released and the TLS RX rules can be sustained.

Sponsored by:	NVIDIA Networking

(cherry picked from commit 27b778ae55)
2022-02-08 16:08:51 +01:00
Hans Petter Selasky
7dedd4ba03 mlx5en: Make the hw_lro parameter read only tunable.
This prevents the so-called TIR context from changing during runtime.

Sponsored by:	NVIDIA Networking

(cherry picked from commit a60f953424)
2022-02-08 16:08:51 +01:00
Hans Petter Selasky
ecf32f2daa mlx5: Remove support for FreeBSD 10 and older.
Sponsored by:	NVIDIA Networking

(cherry picked from commit 788e9e7478)
2022-02-08 16:08:51 +01:00
Hans Petter Selasky
3be2f639a8 mlx5en: Patch to inhibit transmit doorbell writes during packet reception.
During packet reception the network stack frequently transmit data in
response to TCP window updates. To reduce the number of transmit doorbells
needed, inhibit all transmit doorbells designated for the same channel until
after the reception of packets for the given channel is completed.

While at it slightly refactor the mlx5e_tx_notify_hw() function:

1) The doorbell information is always stored into sq->doorbell.d64 .
No need to pass a separate pointer to this variable.

2) Move checks for skipping doorbell writes inside this function.

Sponsored by:	NVIDIA Networking

(cherry picked from commit 2d5e5a0d75)
2022-02-08 16:08:51 +01:00
Konstantin Belousov
19371d5334 mlx5ib: idiomatic use of preprocessor, in particular paths
(cherry picked from commit 028130b8e4)
2022-02-08 08:42:07 +02:00
Konstantin Belousov
0330cf6769 mlx5ib: normalize use of the opt_*.h files
(cherry picked from commit 7060097908)
2022-02-08 08:42:07 +02:00
Konstantin Belousov
fe68072877 mlx5en: idiomatic use of preprocessor, in particular paths
(cherry picked from commit 89918a2375)
2022-02-08 08:42:07 +02:00
Konstantin Belousov
6f8ed9c125 mlx5en: normalize use of the opt_*.h files
(cherry picked from commit b984b95693)
2022-02-08 08:42:07 +02:00
Hans Petter Selasky
c91a6860a3 mlx5: idiomatic use of preprocessor, in particular paths
(cherry picked from commit 12c56d7dc4)
2022-02-08 08:42:07 +02:00
Konstantin Belousov
63a7bd998c mlx5: normalize use of the opt_*.h files
(cherry picked from commit ee9d634bd3)
2022-02-08 08:42:06 +02:00
Elyes HAOUAS
27fceea98c Fix typo on "Celsius"
"Celcius" --> "Celsius"

Signed-off-by: Elyes HAOUAS <ehaouas@noos.fr>
Pull Request: https://github.com/freebsd/freebsd-src/pull/551/files
(cherry picked from commit 9097ac9af4)
2021-12-06 08:55:55 -07:00
Gordon Bergling
8460767ded mlx5: Correct a typo in a sysctl description
- s/parameteres/parameters/

(cherry picked from commit c937fb286f)
2021-12-05 10:08:07 +01:00
Konstantin Belousov
9a1f5e104e mlx5_en: fix use of CALLOUT_DIRECT
(cherry picked from commit 0eb2dcd32e)
2021-10-12 10:00:10 +03:00
Hans Petter Selasky
929e8e5605 mlx5/mlx4: Bump driver version to 3.7
While at it only output driver version to dmesg(8) when hardware is present.

Differential Revision:	https://reviews.freebsd.org/D29100
MFC after:	1 week
Reviewed by:	kib and markj
Sponsored by:	NVIDIA Networking

(cherry picked from commit d2cbfbc57b)
2021-09-22 14:59:31 +02:00
Hans Petter Selasky
5b803f6820 mlx5: Numa domain improvements.
Properly allocate all mlx5en(4) structures from correct numa domain.

While at it cleanup unused numa domain integers deriving from the
Linux version of mlx5en(4).

Reviewed by:	kib
Sponsored by:	Mellanox Technologies // NVIDIA Networking

(cherry picked from commit 7c3eff94bd)
2021-07-26 18:04:33 +02:00
Hans Petter Selasky
2b4db9bbc2 mlx5: Fix for uninitialized "uid" field.
Make sure the "uid" field gets properly set when destroying DCT and QP
objects by making a copy of the field when creating such objects.

Reviewed by:	kib
Sponsored by:	Mellanox Technologies // NVIDIA Networking

(cherry picked from commit cbf6911e10)
2021-07-26 18:04:33 +02:00
Hans Petter Selasky
fdab56d1f6 mlx5en: Allow binding channels to CPUs when RSS is not enabled.
Submitted by:	Netflix
Reviewed by:	kib
Sponsored by:	Mellanox Technologies // NVIDIA Networking

(cherry picked from commit c8d16d1e08)
2021-07-26 18:04:33 +02:00
Hans Petter Selasky
857966b357 mlx5en: Allocate per-channel doorbells.
To avoid congestion on the same PCI memory register space when
traffic consists mostly of small packets.

Reviewed by:	kib
Sponsored by:	Mellanox Technologies // NVIDIA Networking

(cherry picked from commit 9dfa21486e)
2021-07-26 18:04:33 +02:00
Hans Petter Selasky
0ebff67cab mlx5en: Wait for all TLS connections to terminate when unloading driver.
The driver expects all TLS tags to be returned to the driver before
it can free the UMA zone where the TLS tags reside.

Reviewed by:	kib
Sponsored by:	Mellanox Technologies // NVIDIA Networking

(cherry picked from commit 3a934ba7a3)
2021-07-26 18:04:33 +02:00
Hans Petter Selasky
cd08d4c537 mlx4ib and mlx5ib: Set slid to zero in Ethernet completion struct
IB spec says that a lid should be ignored when link layer is Ethernet,
for example when building or parsing a CM request message (CA17-34).
However, since ib_lid_be16() and ib_lid_cpu16()  validates the slid,
not only when link layer is IB, we set the slid to zero to prevent
false warnings in the kernel log.

Linux commit:
65389322b28f81cc137b60a41044c2d958a7b950

Reviewed by:	kib
Sponsored by:	Mellanox Technologies // NVIDIA Networking

(cherry picked from commit 30416d4e82)
2021-07-26 18:04:33 +02:00
Hans Petter Selasky
1554e2673b mlx5en: Configure relaxed PCI read and write ordering for ethernet.
This may improve performance in some configurations.

Reviewed by:	kib
Sponsored by:	Mellanox Technologies // NVIDIA Networking

(cherry picked from commit de2437f199)
2021-07-26 18:04:32 +02:00
Hans Petter Selasky
c41403c5ca mlx5en: Check for pci_channel_offline() when draining sendqueue.
This speeds up detach in hypervisor environments.

Reviewed by:	kib
Sponsored by:	Mellanox Technologies // NVIDIA Networking

(cherry picked from commit 4692d9808e)
2021-07-26 18:04:32 +02:00
Hans Petter Selasky
763d239db2 mlx5ib: Implement support for enabling and disabling RoCE ECN.
RoCE is short for Remote direct memory access over Converged Ethernet.
ECN is short for Explicit Congestion Notification.

Reviewed by:	kib
Sponsored by:	Mellanox Technologies // NVIDIA Networking

(cherry picked from commit 8abf5ac0e6)
2021-07-26 18:04:32 +02:00
Hans Petter Selasky
526c54f54b mlx5ib: Extend parameter macros so that more arguments may be added.
Reviewed by:	kib
Sponsored by:	Mellanox Technologies // NVIDIA Networking

(cherry picked from commit 42f719d611)
2021-07-26 18:04:32 +02:00