Chelsio has always been recording a timestamp in the mbuf (rcv_tstmp) but
not setting the M_TSTMP bit in the mbuf flags. This is because the timestamp
was just the free running 60bit clock. This change fixes that so that
we keep a synchronization by periodically (every 30 seconds after startup)
getting the timestamp and the current nanosecond time. We always keep
several sets around and the current one we always keep the current pair
and the previous pair of timestamps. This allows us to setup a ratio
between the two so we can correctly translate the time. Note that
we use special care to split the timestamp into seconds (per the clock tick)
and nanoseconds otherwise 64bit math would overflow.
Reviewed by: np
Sponsored by: Netflix Inc
Differential Revision: https://reviews.freebsd.org/D36315
(cherry picked from commit e398922eaf)
Remove a few more remnants from the old pre-KTLS support and instead
assume that each work request sends a single TLS record.
Sponsored by: Chelsio Communications
(cherry picked from commit c6b3a3772c)
If an adapter advertises support for TLS keys but an empty TLS key
storage area in on-board memory, fail the request rather than invoking
vmem_alloc on an uninitialized vmem.
Sponsored by: Chelsio Communications
(cherry picked from commit 1ca4f45ea8)
The NIC TLS and TOE TLS modes in cxgbe(4) both work with TLS key
contexts. Previously, TOE TLS supported TLS key contexts created by
two different methods, and NIC TLS had a separate bit of code copied
from NIC TLS but specific to KTLS. Now that TOE TLS only supports
KTLS, pull common code for creating TLS key contexts and programming
them into on-card memory into t4_keyctx.c.
Sponsored by: Chelsio Communications
(cherry picked from commit 18c69734e9)
TOE TLS offload was first supported via a customized OpenSSL developed
by Chelsio with proprietary socket options prior to KTLS being present
either in FreeBSD or upstream OpenSSL. With the addition of KTLS in
both places, cxgbe's TOE driver was extended to support TLS offload
via KTLS as well. This change removes the older interface leaving
only the KTLS bindings for TOE TLS.
Since KTLS was added to TOE TLS second, it was somehat shoe-horned
into the existing code. In addition to removing the non-KTLS TLS
offload, refactor and simplify the code to assume KTLS, e.g. not
copying keys into a helper structure that mimic'ed the non-KTLS mode,
but using the KTLS session object directly when constructing key
contexts.
This also removes some unused code to send TX keys inline in work
requests for TOE TLS. This code was never enabled, and was arguably
sending the wrong thing (it was not sending the raw key context as we
do for NIC TLS when using inline keys).
Sponsored by: Chelsio Communications
(cherry picked from commit 789f2d4b3f)
cxgbei needs the ability to return different limits based on the
connection (e.g. if the connection is over a T5 adapter or a T6
adapter as well as factoring in the MTU).
This change plumbs through the changes in the ioctls without changing
any of the backends. The limits callback passed to icl_register now
accepts a second socket argument which holds the integer file
descriptor. To support ABI compatiblity for old binaries, the
callback should return "global" values if the socket fd is zero.
The CTL_ISCSI_LIMITS argument used with CTL_ISCSI by ctld(8) now
accepts the socket fd in a field that was previously part of a
reserved spare field. Old binaries zero this request which results in
passing a socket fd of 0 to the limits callback.
The ISCSIDREQUEST ioctl no longer returns limits. Instead, iscsid(8)
invokes a new ISCSIDLIMITS ioctl after establishing the connection via
connect(2). For ABI compat, if the old ISCSIDREQUEST is invoked, the
global limits are still fetched (with a socket fd of 0) and returned.
Reviewed by: mav
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D34928
(cherry picked from commit 7b02c1e8c6)
This doesn't matter for real traffic but fixes failures in the KTLS
unit tests that use unusual record types.
Sponsored by: Chelsio Communications
(cherry picked from commit 782db2881b)
Round down the maximim ISO payload by the current MSS. Otherwise the
round up by MSS when calculating the 16-bit maximum payload len passed
along in the FLOWC work request can overflow.
Discussed with: np
Sponsored by: Chelsio Communications
(cherry picked from commit 8020c05683)
With clang 15, the following -Werror warnings are produced:
sys/dev/cxgbe/t4_tracer.c:234:18: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
t4_tracer_modload()
^
void
sys/dev/cxgbe/t4_tracer.c:243:20: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
t4_tracer_modunload()
^
void
This is because t4_tracer_modload() and t4_tracer_modunload() are
declared with a (void) argument list, but defined with an empty argument
list. Make the definitions match the declarations.
MFC after: 3 days
(cherry picked from commit 54e5efb264)
Clang 15 warns:
sys/dev/cxgbe/cudbg/cudbg_lib.c:2949:6: error: variable 'i' set but not used [-Werror,-Wunused-but-set-variable]
int i = 0;
^
Apparently 'i' was meant as the current retry counter, but '1' was used
in the while loop comparison instead, making the loop potentially
infinite, if 'busy' never gets reset.
MFC after: 3 days
Reviewed by: np
Differential Revision: https://reviews.freebsd.org/D35834
(cherry picked from commit fb0493d559)
Otherwise we end up copying one uninitialized byte into the socket
buffer.
Reported by: KMSAN
Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
(cherry picked from commit 6be8944d96)
'command' is too generic for something specific to the kernel debugger;
change this so it is less likely to collide with local variable names.
Also rename struct command_table to struct db_command_table.
Reviewed by: markj
MFC after: 1 week
Sponsored by: Juniper Networks, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D35367
(cherry picked from commit 4ef7db5a7e)
- Add icl_pdu_append_bio and icl_pdu_get_bio methods.
- Add new page pod routines for allocating and writing page pods for
unmapped bio requests. Use these new routines for setting up DDP
for iSCSI tasks with a SCSI I/O CCB which uses CAM_DATA_BIO.
- When ICL_NOCOPY is used to append data from an unmapped I/O request
to a PDU, construct unmapped mbufs from the relevant pages backing
the struct bio. This also requires changes in the t4_push_pdus path
to support unmapped mbufs.
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D34383
(cherry picked from commit 2beaefe884)
Having a single pool of worker threads adds extra complexity and
overhead. The software backend also uses per-connection kthreads.
Sponsored by: Chelsio Communications
(cherry picked from commit 511b83b167)
Previously the driver was called to send PDUs to the NIC synchronously
from the icl_conn_pdu_queue_cb callback. However, this performed a
fair bit of work while holding the icl connection lock. Instead,
change the callback to add sent PDUs to a STAILQ and defer dispatching
of PDUs to the NIC to a helper thread similar to the scheme used in
the TCP iSCSI backend.
- Replace rx_flags int and the sole RXF_ACTIVE flag with a simple
rx_active bool.
- Add a pool of transmit worker threads for cxgbei.
- Fix worker thread exit to depend on the wakeup in kthread_exit()
to fix a race with module unload.
Reported by: mav
Sponsored by: Chelsio Communications
(cherry picked from commit fd8f61d6e9)
sbcut() returns mbufs in reverse order so is not suitable for reading
data from the socket buffer. Instead, check for already-received data
in the receive worker thread before passing offload PDUs up to the
iSCSI layer. This uses soreceive() to read data from the socket and
is also to use M_WAITOK since it now runs from a worker thread instead
of an interrupt thread.
Also, fix decoding of the data segment length for pre-offload PDUs.
Reported by: Jithesh Arakkan @ Chelsio
Fixes: a8c4147edc cxgbei: Parse all PDUs received prior to enabling offload mode.
Sponsored by: Chelsio Communications
(cherry picked from commit 74fea8eb4f)
Previously this would only handle a single PDU that did not contain
any data. This should now handle an arbitrary number of PDUs.
While here check for these PDUs in the T6-specific CPL_RX_ISCSI_CMP
handler in addition to CPL_RX_ISCSI_DDP.
Reported by: Jithesh Arakkan @ Chelsio
Sponsored by: Chelsio Communications
(cherry picked from commit a8c4147edc)
Previously the driver duplicated code from cryptosoft.c to handle
certain edge case AES-CCM and AES-GCM requests. However, this
approach has a few downsides:
1) It only uses "plain" software and not accelerated software since it
uses enc_xform directly.
2) It performs the operation synchronously even though the caller
believes it is invoking an async driver. This was fine for the
original use case of requests with only AAD and no payload that
execute quickly, but is a bit more disingenuous for large requests
which fall back due to exceeding the size of a firmware work
request (e.g. due to large scatter/gather lists).
3) It has required several updates since ccr(4) was added to the tree.
Instead, allocate a software session for AES-CCM and AES-GCM sessions
and dispatch a cloned request asynchronusly to the software session.
Reviewed by: markj
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D33608
(cherry picked from commit e43cf698d9)
When the initiator is reconnecting to the target, the connection may
temporarily be marked disconnected or not have an associated socket.
New I/O requests received by the initiator in this state should not
fail with ECONNRESET as that results in an I/O error back to userland.
Instead, they need to still succeed so that CAM can queue the requests
and send them once the connection is re-established.
Setting up DDP for zero-copy receive requires a socket, so just punt
on using DDP for these transfers.
Reported by: Jithesh Arakkan @ Chelsio
Sponsored by: Chelsio Communications
(cherry picked from commit 752e211e64)
ccr(4) can handle requests for AES-CTR (a stream cipher), not just
block ciphers, so make the function and structure names more generic.
Sponsored by: Chelsio Communications
(cherry picked from commit 762f1dcb1c)
This centralizes the check for valid nonce lengths for AES-GCM.
While here, remove some duplicate checks for valid AES-GCM tag lengths
from ccp(4) and ccr(4).
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33194
(cherry picked from commit 6e17a2e00d)
A recent change caused iq flags, like LRO, to be set before
init_iq(). However, init_iq() clears those flags, so they
became effectively impossible to set. This change moves
the initializion of these flags to after the call to init_iq().
This fixes LRO.
Differential Revision: https://reviews.freebsd.org/D30460
Reviewed by: np, rrs
Sponsored by: Netflix
Fixes: 43bbae1948
(cherry picked from commit df8437a93d)
The firmware doesn't report FORCE_FEC in pcaps if the transceiver
plugged in at that time does not support a speed that may use FEC. It
is incorrect for the driver to assume that the FORCE_FEC value it read
during attach (in init_link_config) is permanent. Instead, it should
check pcaps just before issuing the L1CFG command.
Sponsored by: Chelsio Communications
(cherry picked from commit 231f211240)
This has been safe since e9e7bc8250, which moved parts of error
handling from the ithread to a taskqueue.
Sponsored by: Chelsio Communications
(cherry picked from commit 41c4e1c7f7)
* New error_flags that can be used from the error ithread and elsewhere
without a synch_op.
* Stop the adapter immediately in t4_fatal_err but defer most of the
rest of the handling to a task. The task is allowed to sleep, unlike
the ithread. Remove async_event_task as it is no longer needed.
* Dump the devlog, CIMLA, and PCIE_FW exactly once on any fatal error
involving the firmware or the CIM block. While here, dump some
additional info (see dump_cim_regs) for these errors.
* If both reset_on_fatal_err and panic_on_fatal_err are set then attempt
a reset first and do not panic the system if it is successful.
Sponsored by: Chelsio Communications
(cherry picked from commit e9e7bc8250)
cxgbe_refresh_stats takes into account VI_SKIP_STATS but not
VI_INIT_DONE when deciding whether to read the hardware stats. But
before this change VI_SKIP_STATS was set only for VIs with VI_INIT_DONE.
That meant that cxgbe_refresh_stats always accessed the hardware for
uninitialized VIs, and this is a problem if the adapter is suspended or
in the middle of a reset.
Fix this by setting VI_SKIP_STATS on all VIs during suspend. While
here, ignore VI_INIT_DONE in vi_refresh_stats too to be consistent with
cxgbe_refresh_stats.
Sponsored by: Chelsio Communications
(cherry picked from commit 08c7dc7fd4)
The hardware is unavailable when the device is suspended or in the
middle of a reset.
Sponsored by: Chelsio Communications
(cherry picked from commit 39a36707bd)
The default sysctl context setup by newbus for a device is eventually
freed by device_sysctl_fini, which runs after the device driver's detach
routine. sysctl nodes associated with this context must not use any
resources (like driver locks, hardware access, counters, etc.) that are
released by driver detach.
There are a lot of sysctl nodes like this in cxgbe(4) and the fix is to
hang them off a context that is explicitly freed by the driver before it
releases any resource that might be used by a sysctl.
This fixes panics when running "sysctl dev.t6nex dev.cc" in a tight loop
and loading/unloading the driver in parallel.
Reported by: Suhas Lokesha
Sponsored by: Chelsio Communications
(cherry picked from commit a727d9531a)
This ensures that the driver reports an error instead of failing
silently when an invalid media is requested.
Reported by: Suhas Lokesha @ Chelsio
Sponsored by: Chelsio Communications
(cherry picked from commit cdd7fe04cb)
This eliminates error messages like this from the driver when running at
50Gbps with 100G cables:
[3726] cc0: l1cfg failed: 71
[4407] cc0: l1cfg failed: 71
Note that link comes up anyway with or without this change.
Reported by: Suhas Lokesha @ Chelsio
Sponsored by: Chelsio Communications
(cherry picked from commit f3c2987f2f)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
CHANGES
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Version : 1.26.6.0
Date : 01/03/2022
================================================================================
Fixes
-----
BASE:
- Fixed one module eeprom read failure.
- Fixed an issue with speed selection when 40G and 25G are advertised and
supported.
- Fixed a random traffic hang when T5 receives invalid ets BW in dcbx
messages from a switch.
- Fixed very long link up time with few switches.
================================================================================
Obtained from: Chelsio Communications
Sponsored by: Chelsio Communications
(cherry picked from commit 3b76242433)
