This provides better semantics as a standalone DEVMETHOD for
device_attach as bus drivers should remove child devices they created
as part of detach cleanup. The implementation calls
bus_detach_children() first to permit child devices an opportunity to
veto the detach operation. If that succeeds, device_delete_children()
is used to delete the child devices.
This requires fixing various drivers that were deleting devices
explicitly (via a device_t pointer cached in the softc) after calling
bus_generic_detach to stop doing that and just rely on
bus_generic_detach to remove child devices.
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D47959
These drivers perform additional teardown steps in between detaching
child devices and deleting child devices.
Differential Revision: https://reviews.freebsd.org/D47957
The commit to introduce TCP_USE_DDP support had a couple of bugs that
broke support for zerocopy receive via aio_read(). First, the length
and offset arguments to mk_update_tcb_for_ddp() were reversed which
prevented DDP from working. Second, the AIO state in the toep was
initialized too late when the first aio_read() request was queued.
Reported by: Harshavardhan Tanneru @ Chelsio
Fixes: eba13bbc37 cxgbe: Support TCP_USE_DDP on offloaded TOE connections
MFC after: 1 week
Sponsored by: Chelsio Communications
- Only two of these tunables are used for RATELIMIT without
TCP_OFFLOAD.
- Mark t4_tmr_idx_ofld and t4_pktc_idx_ofld static.
- Move hw.cxgbe.cop_managed_offloading under hw.cxgbe.toe since it is
specific to TOE.
Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D47765
Normally the reply to destroy_server() releases the listen context but
it is not called when the adapter is suspended. Release the context
right away in that case.
MFC after: 1 week
Sponsored by: Chelsio Communications
Formally, there are 12 bits for TCP header flags.
Use the accessor functions in more (kernel) places.
No functional change.
Reviewed By: cc, #transport, cy, glebius, #iflib, kbowling
Sponsored by: NetApp, Inc.
Differential Revision: https://reviews.freebsd.org/D47063
Some M_EXTPG mbufs are read-only (e.g. those backing sendfile
requests), but others are not. Add a flags argument to
mb_alloc_ext_pgs that can be used to set M_RDONLY when needed rather
than setting it unconditionally. Update mb_unmapped_to_ext to
preserve M_RDONLY from the unmapped mbuf.
Reviewed by: gallatin
Differential Revision: https://reviews.freebsd.org/D46783
A hardware IPv6 server needs 2 consecutive stids (server tids) starting
from a 2-aligned stid whereas an IPv4 server needs only 1 stid without
any constraint. The allocator used to grab the first free stid(s) for
both but this can fragment the stid space leaving nothing suitable for
IPv6 even when lots of stids are available. Change the allocator to
prefer stids for IPv4 from the ones that cannot be used for IPv6.
Reviewed by: jhb
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D47042
adapter->flags are guarded by a synch_op, as noted in the comment in
adapter.h where the flags are defined.
Fixes: 5241b210a4 cxgbe(4): Basic infrastructure for ULDs to participate in adapter reset.
MFC after: 1 week
Sponsored by: Chelsio Communications
* Disable IFCAP_TOE automatically on all ifnets on all adapters during
unload. This is user-friendly and avoids panics due to stale ifnet
state after t4_tom is unloaded.
* Do not allow unload if tids are in use by the TOE on any adapter.
Reported by: Bimal Abraham @ Chelsio
MFC after: 1 week
Sponsored by: Chelsio Communications
Tracers have to be recreated after a restart but that's okay given that
they are used for debugging only.
MFC after: 1 week
Sponsored by: Chelsio Communications
An L2 entry in the driver's hash was marked STALE unconditionally if it
changed in the kernel when its driver refcount is 0. Fix the driver to
do this for VALID entries only.
MFC after: 1 week
Sponsored by: Chelsio Communications
1. Mark the L2T entry valid only if t4_write_l2e succeeds, which won't
happen if the adapter is stopped. This prevents L2T entries from
sometimes getting (re)promoted to VALID on Tx activity during stop.
2. Discard a work request immediately instead of enqueueing it to the
arp queue if the adapter is stopped.
Fixes: c1c524852f cxgbe/t4_tom: Implement uld_stop and uld_restart for ULD_TOM.
MFC after: 1 week
Sponsored by: Chelsio Communications
The destination queue for tracing filters is destroyed during stop or
suspend and the software state needs to reflect this. A new destination
queue will be setup when the adapter resumes operation.
MFC after: 1 week
Sponsored by: Chelsio Communications
The STALE state means the L2T entry is valid in hardware but needs to be
refreshed (ARP/NDP) in software. But stop/suspend wipes the hardware
L2T and STALE entries need to be updated just like VALID entries to match
actual hardware state.
Fixes: c1c524852f cxgbe/t4_tom: Implement uld_stop and uld_restart for ULD_TOM.
MFC after: 1 week
Sponsored by: Chelsio Communications
This fixes a panic where the peer's ack to the synack arrives on a
different queue and do_pass_establish tries to remove the synqe from
synqe_list before it has been added by do_pass_accept_req.
Reported by: Sony Arpita Das @ Chelsio
Fixes: 283333c0e3 cxgbe/t4_tom: Track all synq entries in a per-adapter list.
MFC after: 1 week
Sponsored by: Chelsio Communications
This fixes a regression in 5241b210a4 where the driver stopped
reporting link down after a fatal error unless t4_reset_on_fatal_err was
also set.
Fixes: 5241b210a4 cxgbe(4): Basic infrastructure for ULDs to participate in adapter reset.
MFC after: 1 week
Sponsored by: Chelsio Communications
Stop allocating new resources when the RNIC is stopped but continue to
allow previously allocated resources to be freed. Note that t4_tom's
uld_stop tears down all TOE connections, including those being used for
iWARP, and that triggers the cleanup of iWARP resources.
Fail post_send/post_recv early too to avoid the SQ doorbell.
MFC after: 1 week
Sponsored by: Chelsio Communications
backlog is an int and not a string. While here, fix an adjacent CTR
that was spread over two lines even though it fits in one.
MFC after: 1 week
Sponsored by: Chelsio Communications
Now that suspend/resume is supported by the base driver, a fatal error
isn't the only reason that the RNIC can stop abruptly. Also, this state
is no longer permanent as it's possible to resume operations after a
stop. Rename the flag and associated routines to match the new state of
affairs.
MFC after: 1 week
Sponsored by: Chelsio Communications
Add a reset_adapter wrapper that picks the most suitable reset routine
internally. Use it in the fatal error handler as well as the sysctl
based reset.
MFC after: 1 week
Sponsored by: Chelsio Communications
Clear the EQ_HW_ALLOCATED flag with the wrq lock held and discard all
work requests, pending or new, when it's not set.
MFC after: 1 week
Sponsored by: Chelsio Communications
1. Remove toepcb from the toep_list on active open failure.
2. Purge the wr_list for an L2T entry on an adpater stop.
Fixes: c1c524852f cxgbe/t4_tom: Implement uld_stop and uld_restart for ULD_TOM.
MFC after: 2 weeks
Sponsored by: Chelsio Communications
As of 9e6544dd6e contigfree(9) is no longer
needed and should not be used anymore. We leave a wrapper for 3rd party
code in at least 15.x but remove (almost) all other cases from the tree.
This leaves one use of contigfree(9) untouched; that was the original
trigger for 9e6544dd6e and is handled in D45813 (to be committed
seperately later).
Sponsored by: The FreeBSD Foundation
Reviewed by: markj, kib
Tested by: pho (10h stress test run)
Differential Revision: https://reviews.freebsd.org/D46099
This allows the adapter to be suspended or reset even when stateful TOE is
active, in some limited configurations.
The LLD has already stopped the adapter hardware and all its queues by the time
these ULD routines get called. The general approach in t4_tom is to purge the
lookup tables immediately so that they are ready for operation by the time the
adapter resumes, and park all the resources left hanging by the stopped hardware
into separate "stranded" queues that can be dealt with at leisure.
Outstanding active opens, live connections, and synq entries (for connections in
the middle of the 3-way handshake) are all treated as if the hardware had
reported an abrupt error for the tid. The servers/listeners are a bit different
in that no error is reported. They're just noted as non-functional when the
hardware stops and are recreated by the driver during restart.
MFC after: 1 month
Sponsored by: Chelsio Communications
Live tid entries in tid_tab are either full fledged connections or synq
entries. toep_list tracks the connections already and this change adds
a synqe_list to track the synq entries. These two lists can be used to
enumerate and iterate over all live tids.
MFC after: 1 week
Sponsored by: Chelsio Communications
L2T entries are used by both filters and TOE and the L2T is shared
between the base driver (LLD) and the TOM ULD. Add a flag to indicate
that the L2T is stopped, which means:
* t4_alloc_l2e and t4_l2t_alloc_switching will not allocate new entries.
* t4_tom will ignore all ARP/NDP updates from the kernel.
* Previously allocated L2T entries can still be freed.
MFC after: 1 month
Sponsored by: Chelsio Communications
atids are used by both filters and TOE and the atid table is in the base
driver (LLD). New atids cannot be allocated when the allocator is
stopped but existing ones can still be freed. It is expected that the
owners of outstanding atids will release them in their own stop
processing, before the adapter is restarted.
MFC after: 1 month
Sponsored by: Chelsio Communications
The suspend/resume/reset implementation in the base driver (LLD)
currently works when only stateless features are in use. This commit
adds basic infrastructure for stateful upper layer drivers (ULDs) to
participate in suspend/resume/reset.
* Add a uld_restart to indicate that the adapter has been restarted
after a stop and the ULD should resume operations.
* Move the existing functionality in t4_suspend/t4_resume to stop_lld and
restart_lld. Use these and the new uld restart routines everywhere the
adapter has to be stopped abruptly and restarted, namely:
1. PCIE bus suspend/resume/reset methods invoked by the kernel.
2. Manual internal-reset using driver sysctl.
3. Automatic internal-reset on a fatal error.
* Implement an alternate internal-reset for use in VMs and for testing.
Typical reset sequence is:
stop_adapter(sc);
stop_lld(sc);
stop_all_uld(sc);
set_adapter_hwstatus(sc, false);
/* hw reset takes place here. */
restart_adapter(sc);
restart_lld(sc);
set_adapter_hwstatus(sc, true);
restart_all_uld(sc);
MFC after: 1 month
Sponsored by: Chelsio Communications
* Convert t4_uld_list to an array. There will be at most 3 items in the
list and it's simpler to track them in an array with a fixed slot for
each ULD.
* There is no need to refcount ULDs so stop doing that.
* Add uld_ prefix to all members of uld_info.
* Rename async_event to uld_stop to match its actual purpose. Call it
for all ULDs and not just ULD_IWARP.
Reviewed by: jhb
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D46029
Do not assume that the table starts at index 0 and is typically 4K in
size. The only thing the driver needs to verify is that its use of
F_SYNC_WR doesn't collide with the L2T hwidx range.
Reviewed by: jhb
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D46028
Final CPL means the tid is done in the hardware and other resources
associated with it can be freed right away. There is no need to wait
for the kernel to detach the toepcb.
Reviewed by: jhb
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D45991
The kernel used to call tod_pcb_detach when entering TIME_WAIT but that
seems to have changed, likely with the TIME_WAIT overhaul in the kernel
some time ago. Catch up by having the driver perform the detach.
The hardware does not handle TIME_WAIT so it's important to detach and
let the kernel arm the 2MSL timer to deal with it.
Reported by: Sony Arpita Das @ Chelsio
Reviewed by: jhb
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D45990
Align the comment for if_t with the rest and remove the declaration of
t4_l2t_set_switching, which was removed in 061bbaf7e7.
No functional change.
MFC after: 3 days
Sponsored by: Chelsio Communications
Previously, when transmitting short runs of packets via cxgbe_nm_tx(),
we would wait until a large number of packets were buffered before
scheduling a task to clean transmit buffers.
Obtained from: np
This fixes a panic when multiple VIs are configured on an interface and
only the non-primary VI is up at the time of driver detach. The problem
was that the driver would queue a link state change notification for an
interface about to be freed.
To reproduce the panic, add "hw.cxgbe.num_vis=2" to loader.conf and
# kldload if_cxgbe
# ifconfig vcc0 up
# devctl detach t6nex0
trap 0x9, rip = 0xffffffff8107db70, rsp = 0xfffffe0055263d60, rbp = 0xfffffe0055263dd0
taskqueue_run_locked() at taskqueue_run_locked+0x2a0/frame 0xfffffe0055263dd0
taskqueue_run() at taskqueue_run+0x72/frame 0xfffffe0055263df0
taskqueue_swi_run() at taskqueue_swi_run+0x18/frame 0xfffffe0055263e10
intr_event_execute_handlers() at intr_event_execute_handlers+0x249/frame 0xfffffe0055263e50
ithread_execute_handlers() at ithread_execute_handlers+0x9e/frame 0xfffffe0055263e70
Reviewed by: jhb
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D45864
