Change POOL_NEXT_SIZE define value from 0 to BIT(30), since this define
is used to request the available maximum sized flow table, and zero doesn't
make sense for it, whereas many places in the driver use zero explicitly
expecting the smallest table size possible but instead due to this
define they end up allocating the biggest table size unawarely.
Sponsored by: NVidia networking
Align the code of fdb steering with flow steering core
and add missing parts in namespace initialization and
in prio logic
PR: 281714
Sponsored by: NVidia networking
Formally, there are 12 bits for TCP header flags.
Use the accessor functions in more (kernel) places.
No functional change.
Reviewed By: cc, #transport, cy, glebius, #iflib, kbowling
Sponsored by: NetApp, Inc.
Differential Revision: https://reviews.freebsd.org/D47063
Remove the array of port module status and instead save module status
and module number.
At boot, for each PCI function driver get event from fw about module
status. The event contains module number and module status. Driver
stores module number and module status.. When user (ifconfig) ask for
modules information, for each pci function driver first queries fw to
get module number of current pci function, then driver compares the
module number to the module number it stored before and if it matches
and module status is "plugged and enabled" then driver queries fw for
the eprom information of that module number and return it to the
caller.
In fact fw could have concluded that required module number of the
current pci function, but fw is not implemented this way. current
design of PRM/FW is that MCIA register handling is only aware of
modules, not the pci function->module connections. FW is designed to
take the module number written to MCIA and write/read the content
to/from the associated module's EPROM.
So, based on current FW design, we must supply the module num so fw
can find the corresponding I2C interface of the module to write/read.
Sponsored by: NVidia networking
MFC after: 1 week
Ensure all allocated tags have a hardware context associated.
The hardware context allocation is moved into the zone import
routine, as suggested by kib. This is safe because these zone
allocations are always done in a sleepable context.
I have removed the now pointless num_resources tracking,
and added sysctls / tunables to control UMA zone limits
for these tls tags, as well as a tunable to let the
driver pre-allocate tags at boot.
MFC after: 2 weeks
Only ever set the capabilities bits if kernel options are enabled.
Check for hardware capabilities before setting software bits.
Sponsored by: NVidia networking
MFC after: 1 week
Under massive connection thrashing (web server restarting), we see
long periods where the web server blocks when enabling ktls offload
when NIC ktls offload is enabled.
It turns out the driver uses a single-threaded linux work queue to
serialize the commands that must be sent to the nic to allocate and
free tls resources. When freeing sessions, this work is handled
asynchronously. However, when allocating sessions, the work is handled
synchronously and the driver waits for the work to complete before
returning. When under massive connection thrashing, the work queue is
first filled by TLS sessions closing. Then when new sessions arrive,
the web server enables kTLS and blocks while the tens or hundreds of
thousands of sessions closes queued up are processed by the NIC.
Rather than using the work queue to open a TLS session on the NIC,
switch to doing the open directly. This allows use to cut in front of
all those sessions that are waiting to close, and minimize the amount
of time the web server blocks. The risk is that the NIC may be out of
resources because it has not processed all of those session frees. So
if we fail to open a session directly, we fall back to using the work
queue.
Differential Revision: https://reviews.freebsd.org/D47260
Sponsored by: Netflix
Reviewed by: kib
Rather than compute ilog2(roundup_pow_of_two(x)), which invokes ilog2
twice, just use order_base_2 once. And employ that optimization
twice.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D46838
Add vlan tag match to RX FS SA and policy rules
and report SA lifetime counter on vlan interface
in case SA was installed on vlan interface
Existing code didn't have the net tag id as part of
the FS matching rules. This can cause applying
ipsec offload to the wrong interface.
This commit add tag id as part of FS matchers
and treat tag value 0 as no tag
Sponsored by: NVidia networking
Do not prepend ipsec tags into mbuf head when preparing rx wqe, store it
separately. Only prepend (and clear the store) when received packed was
indeed offloaded by ipsec engine. Then we do not need to refill tags
for slots that received non-ipsec packets.
This should solve some minimal degradation of the rx CPU usage due to
unneeded tag allocation for each packet.
Sponsored by: NVidia networking
When the kernel is compiled with options RATELIMIT, the
mlx5en driver cannot detach. It gets stuck waiting for all
kernel users of its rates to drop to zero before finally calling
ether_ifdetach.
The tcp ratelimit code has an eventhandler for ifnet departure
which causes rates to be released. However, this is called as an
ifnet departure eventhandler, which is invoked as part of
ifdetach(), via either_ifdetach(). This means that the tcp
ratelimit code holds down many hw rates when the mlx5en driver
is waiting for the rate count to go to 0. Thus devctl detach
will deadlock on mlx5 with this stack:
mi_switch+0xcf sleepq_timedwait+0x2f _sleep+0x1a3 pause_sbt+0x77 mlx5e_destroy_ifp+0xaf mlx5_remove_device+0xa7 mlx5_unregister_device+0x78 mlx5_unload_one+0x10a remove_one+0x1e linux_pci_detach_device+0x36 linux_pci_detach+0x24 device_detach+0x180 devctl2_ioctl+0x3dc devfs_ioctl+0xbb vn_ioctl+0xca devfs_ioctl_f+0x1e kern_ioctl+0x1c3 sys_ioctl+0x10a
To fix this, provide an explicit API for a driver to call the tcp
ratelimit code telling it to detach itself from an ifnet. This
allows the mlx5 driver to unload cleanly. I considered adding an
ifnet pre-departure eventhandler. However, that would need to be
invoked by the driver, so a simple function call seemed better.
The mlx5en driver has been updated to call this function.
Reviewed by: kib, rrs
Differential Revision: https://reviews.freebsd.org/D46221
Sponsored by: Netflix
Right now, only IPv4 transport mode, with aes-gcm ESP, is supported.
Driver also cooperates with NAT-T, and obeys socket policies, which
makes IKEd like StrongSwan working.
Sponsored by: NVIDIA networking
Change 4787572d05 made if_alloc_domain() never fail, then also do the
wrappers if_alloc(), if_alloc_dev(), and if_gethandle().
No functional change intended.
Reviewed by: kp, imp, glebius, stevek
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D45740
In several places, a loop tests for powers of two, or iterates through
powers of two. In those places, replace the loop with an invocation
of fls or ilog2 without changing the meaning of the code.
Reviewed by: alc, markj, kib, np, erj, avg (previous version)
Differential Revision: https://reviews.freebsd.org/D45494
mlx4 and mlx5 are Ethernet devices and ether_ifattach() does an
unconditional bpfattach(). From commit 16d878cc99 [1] and on, we
should not check ifp->if_bpf to tell us whether or not we have any bpf
peers that might be interested in receiving packets. And since commit
2b9600b449 [2], ifp->if_bpf can not be NULL even after the network
interface has been detached.
No functional change intended.
1. 16d878cc99 Fix the following bpf(4) race condition which can result in a panic
2. 2b9600b449 Add dead_bpf_if structure, that should be used as fake bpf_if during ifnet detach
Reviewed by: kp, kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D45196
These functions may map more memory for DMA than is actually used, since
the allocator operates on multiples of a 4KB page size. Thus,
bus_dmamap_sync() can trigger KMSAN reports when the unused portion of
a page is not zero-ed.
Reported by: KMSAN
Reviewed by: kib
MFC after: 2 weeks
Sponsored by: Klara, Inc.
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D43133
Previously we were trying to set hca_cap_2 without checking if
sw_vhca_id_valid max value, which is the only settable value inside
hca_cap_2, and seeing that we dont have driver support for sw_vhca_id
yet there is no need to set hca_cap_2 at all, it is enough to query it.
Fixes: 7b959396ca ("mlx5: Introduce new destination type TABLE_TYPE")
MFC after: 3 days
Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.
Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/
Sponsored by: Netflix
If removing a node of type FS_TYPE_FLOW_DEST we lock the flow group too
late. This can lead to a deadlock with fs_add_dst_fg().
PR: 274715
MFC after: 1 week
Reviewed by: kib
Tested by: mm
Differential Revision: https://reviews.freebsd.org/D42368
This new destination type supports flow transition between different
table types, e.g. from NIC_RX to RDMA_RX or from RDMA_TX to NIC_TX.
In addition add driver support to be able to query the capability for
this new destination type.
Signed-off-by: Patrisious Haddad <phaddad@nvidia.com>
Sponsored by: NVidia networking
MFC after: 1 week
This allows to insert a rule and make sure it doesn't get
combined by the steering layer with any other rule.
Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Sponsored by: NVidia networking
MFC after: 1 week
Exclude the last n entries for an autogrouped flow table.
Reserving entries at the end of the FT will ensure that this FG will be
the last to be evaluated. This will be used in the next patch to create
Linux upstream commit: 79cdb0aaea8b5478db34afa1d4d5ecc808689a67
Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Sponsored by: NVidia networking
MFC after: 1 week
Once all the large flow groups (defined by the user when the flow table
is created - max_num_groups) were created, then all the following new
flow groups will have only one flow table entry, even though the flow table
has place to larger groups.
Fix the condition to prefer large flow group.
Upstream Linux commit: 97fd8da281f80e7e69e0114bc906575734d4dfaf
Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Sponsored by: NVidia networking
MFC after: 1 week
Users of the steering APIs shouldn't use the PRM directly.
Create an software enum to be used instead.
Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Sponsored by: NVidia networking
MFC after: 1 week
Create a struct to hold flow actions to be used when creating
a flow rule.
Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Sponsored by: NVidia networking
MFC after: 1 week
