Remove some uses of PHOLD which were there only to prevent the process'
threads from being swapped out.
Tested by: pho
Reviewed by: imp, kib
Differential Revision: https://reviews.freebsd.org/D46118
- Remove unnecessary CHN_[UN]LOCKs.
- Improve device_printf() messages.
- Remove redundant checks.
- Remove 0 assignments, since the channel is allocated with M_ZERO.
- Re-organize sections in a more coherent way.
- Remove "out1" label, just return NULL directly.
- Rename "out2" to "fail" and simplify logic.
- Do not check the return value of dsp_unit2name(), as it is guaranteed
not to fail (we pass a valid channel type).
Sponsored by: The FreeBSD Foundation
MFC after: 2 days
Reviewed by: dev_submerge.ch
Differential Revision: https://reviews.freebsd.org/D45985
No functional change intended.
Sponsored by: The FreeBSD Foundation
MFC after: 2 days
Reviewed by: dev_submerge.ch, markj
Differential Revision: https://reviews.freebsd.org/D45984
In case chn_init() fails, we are leaking memory by not freeing the
feeder.
Sponsored by: The FreeBSD Foundation
MFC after: 2 days
Reviewed by: dev_submerge.ch, markj
Differential Revision: https://reviews.freebsd.org/D45979
Assignment taken from dsp_oss_engineinfo().
Sponsored by: The FreeBSD Foundation
MFC after: 2 days
Reviewed by: dev_submerge.ch
Differential Revision: https://reviews.freebsd.org/D46166
The pre-freebsd7 stuff is labeled _old right now. Relabel it as
_freebsd6 instead. No functional change intended. I selected freebsd6
because that's the last version to have the ioctls, although I know
there were changes in the interface for 4, 5 and 6. This broadly batches
what we do with system calls that accumulate changes until we break and
use a new one, we use the last major the system call was in.
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D45867
As of 9e6544dd6e contigfree(9) is no longer
needed and should not be used anymore. We leave a wrapper for 3rd party
code in at least 15.x but remove (almost) all other cases from the tree.
This leaves one use of contigfree(9) untouched; that was the original
trigger for 9e6544dd6e and is handled in D45813 (to be committed
seperately later).
Sponsored by: The FreeBSD Foundation
Reviewed by: markj, kib
Tested by: pho (10h stress test run)
Differential Revision: https://reviews.freebsd.org/D46099
This sysctl sets a cap on the maximum payload of transmitted data PDUs
including both C2H_DATA and H2C_DATA PDUs, not just C2H_DATA PDUs.
Sponsored by: Chelsio Communications
D28743 was commited, reverted and then f689cb23b2 landed before it
was recommitted. However, D28743 included an extra length check. Redo
that functionality so we check both the number of entries as well as the
length checks for wacky data.
Sponsored by: Netflix
Reviewed by: gallatin
Differential Revision: https://reviews.freebsd.org/D45763
Issue a warning if we have system interrupt issues. If you get this
warning, then we submitted a request, it timed out without an interrupt
being posted, but when we polled the card's completion, we found
completion events. This indicates that we're missing interrupts, and to
date all the times I've helped people track issues like this down it has
been a system issue, not an NVMe driver isseue.
Sponsored by: Netflix
Reviewed by: gallatin
Differential Revision: https://reviews.freebsd.org/D46031
Optimize timeout code based on three observations.
(1) The tr queues are sorted in order of submission, so the first one
that could time out is the first "real" one on the list.
(2) Timeouts for a given queue are all the same length (well, except
at startup, where timeout doesn't matter, and when you change it
at runtime, where timeouts will still happen eventually and the
difference isn't worth optimizing for).
(3) Calling the ISR races the real ISR and we should avoid that better.
So now, after checking to see if the card is there and working, the
timeout routine scans the pending tracker list until it finds a non-AER
tracker. If the deadline hasn't passed, we return, doing nothing
further. Otherwise, we call poll completions and then process the list
looking for timed out items.
This should move the timeout routine to touching hardware only when it's
really necessary. It thus avoids racing the normal ISR, while still
timig out stuck transactions quickly enough.
There was also some minor code motion to make all of the above flow more
nicely for the reader.
When interrupts aren't working at all, then this will increase latency
somewhat. But when interrupts aren't working at all, there's bigger
problems and we should poll quite often in that case. That will be
handled in future commits.
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D46026
When processing an abort completion command, we have to lock. But we
have to lock the qpair of the original transaction (not the abort we're
completing). We do this to avoid races with checking the completion id
to tr mapping array, as well as to manually complete it.
Note: we don't handle the completion status of 'Asked to abort too many
transactions at once.' That will be fixed on subsequent commits. Add a
note to that effect for now since it's a harder problem to solve.
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D46025
When we lose a race with the timeout code, shift towards waiting for
that timeout code to complete so we can acquire the lock. This way we
can make sure we're in 'normal' mode before processing I/O
completions. If we're not in 'normal' mode, then we're resetting and we
should avoid completions.
Sponsored by: Netflix
Reviewed by: gallatin
Differential Revision: https://reviews.freebsd.org/D46024
Users may interpret the message as a possible hardware error, but the
issue is in fact unimplemented functionality. Reword the message to
avoid implying it is an error.
Reviewed by: wulf
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D46092
This uart has the requirement for 32-bit sized and aligned memory
accesses. It is also described in the Serial Port Console Redirection
Table (SPCR) with a different interface type value.
Reviewed by: imp
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D45834
If a PDU (such as a Command Capsule PDU) on a connection that has
enabled data digests does not have a data section, it will not have
the the PDU data digest flag set. The previous check was requiring
this flag to be present on all PDU types that support data sections
even if no data was included in the PDU.
Sponsored by: Chelsio Communications
This allows the adapter to be suspended or reset even when stateful TOE is
active, in some limited configurations.
The LLD has already stopped the adapter hardware and all its queues by the time
these ULD routines get called. The general approach in t4_tom is to purge the
lookup tables immediately so that they are ready for operation by the time the
adapter resumes, and park all the resources left hanging by the stopped hardware
into separate "stranded" queues that can be dealt with at leisure.
Outstanding active opens, live connections, and synq entries (for connections in
the middle of the 3-way handshake) are all treated as if the hardware had
reported an abrupt error for the tid. The servers/listeners are a bit different
in that no error is reported. They're just noted as non-functional when the
hardware stops and are recreated by the driver during restart.
MFC after: 1 month
Sponsored by: Chelsio Communications
Live tid entries in tid_tab are either full fledged connections or synq
entries. toep_list tracks the connections already and this change adds
a synqe_list to track the synq entries. These two lists can be used to
enumerate and iterate over all live tids.
MFC after: 1 week
Sponsored by: Chelsio Communications
L2T entries are used by both filters and TOE and the L2T is shared
between the base driver (LLD) and the TOM ULD. Add a flag to indicate
that the L2T is stopped, which means:
* t4_alloc_l2e and t4_l2t_alloc_switching will not allocate new entries.
* t4_tom will ignore all ARP/NDP updates from the kernel.
* Previously allocated L2T entries can still be freed.
MFC after: 1 month
Sponsored by: Chelsio Communications
atids are used by both filters and TOE and the atid table is in the base
driver (LLD). New atids cannot be allocated when the allocator is
stopped but existing ones can still be freed. It is expected that the
owners of outstanding atids will release them in their own stop
processing, before the adapter is restarted.
MFC after: 1 month
Sponsored by: Chelsio Communications
The suspend/resume/reset implementation in the base driver (LLD)
currently works when only stateless features are in use. This commit
adds basic infrastructure for stateful upper layer drivers (ULDs) to
participate in suspend/resume/reset.
* Add a uld_restart to indicate that the adapter has been restarted
after a stop and the ULD should resume operations.
* Move the existing functionality in t4_suspend/t4_resume to stop_lld and
restart_lld. Use these and the new uld restart routines everywhere the
adapter has to be stopped abruptly and restarted, namely:
1. PCIE bus suspend/resume/reset methods invoked by the kernel.
2. Manual internal-reset using driver sysctl.
3. Automatic internal-reset on a fatal error.
* Implement an alternate internal-reset for use in VMs and for testing.
Typical reset sequence is:
stop_adapter(sc);
stop_lld(sc);
stop_all_uld(sc);
set_adapter_hwstatus(sc, false);
/* hw reset takes place here. */
restart_adapter(sc);
restart_lld(sc);
set_adapter_hwstatus(sc, true);
restart_all_uld(sc);
MFC after: 1 month
Sponsored by: Chelsio Communications
Make nvme_qpair_manual_complete_request take dnr as well as a
print_on_error action. Make the status word computation common between
it and nvme_qpair_manual_complete_tracker. And print the error when
we are cancelling the I/O on failure, but not when we're filtering
the I/O after we've failed. Make it private again to nvme_qpair.c.
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D46049
When the drive is failed, we can't send passthrough commands to the
card, so fail them right away. Rearrange the comments to reflect the
current failure paths in the driver.
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D46048
* Convert t4_uld_list to an array. There will be at most 3 items in the
list and it's simpler to track them in an array with a fixed slot for
each ULD.
* There is no need to refcount ULDs so stop doing that.
* Add uld_ prefix to all members of uld_info.
* Rename async_event to uld_stop to match its actual purpose. Call it
for all ULDs and not just ULD_IWARP.
Reviewed by: jhb
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D46029
Do not assume that the table starts at index 0 and is typically 4K in
size. The only thing the driver needs to verify is that its use of
F_SYNC_WR doesn't collide with the L2T hwidx range.
Reviewed by: jhb
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D46028
Final CPL means the tid is done in the hardware and other resources
associated with it can be freed right away. There is no need to wait
for the kernel to detach the toepcb.
Reviewed by: jhb
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D45991
The kernel used to call tod_pcb_detach when entering TIME_WAIT but that
seems to have changed, likely with the TIME_WAIT overhaul in the kernel
some time ago. Catch up by having the driver perform the detach.
The hardware does not handle TIME_WAIT so it's important to detach and
let the kernel arm the 2MSL timer to deal with it.
Reported by: Sony Arpita Das @ Chelsio
Reviewed by: jhb
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D45990
A recent change dropped the only use of fls64() in axgbe, so drop its
definition.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D46030
Drop an implementation of __rounddown_pow_of_two, which triggers
Coverity warnings, and use the libkern implementation of
rounddown_pow_of_two instead.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D46022
All architectures define PCI_RES_BUS unconditionally now that only
NEW_PCIB is supported, so we no longer need the legacy code.
Reviewed by: jhb, andrew
Differential Revision: https://reviews.freebsd.org/D32955
All architectures enable NEW_PCIB in DEFAULTS (arm being the most recent
to do so in 121be55599 (arm: Set NEW_PCIB in DEFAULTS rather than a
subset of kernel configs")), so it's time we removed the legacy code
that no longer sees much testing and has a significant maintenance
burden.
Reviewed by: jhb, andrew, emaste
Differential Revision: https://reviews.freebsd.org/D32954
Align the comment for if_t with the rest and remove the declaration of
t4_l2t_set_switching, which was removed in 061bbaf7e7.
No functional change.
MFC after: 3 days
Sponsored by: Chelsio Communications
This got missed as it's not built in a normal tinderbox build.
This fixes building arm.armv7 ARMADAXP (marked NO_UNIVERSE).
Fixes: 9dbf5b0e68 ("new-bus: Remove the 'rid' and 'type' arguments from BUS_RELEASE_RESOURCE")
Commit 9a7bf07ccd from 2016 introduced a workaround for some broken
BIOSes that specified active-lo instead of active-hi polarity for ISA
IRQs for UARTs. The workaround assumed that edge-sensitive ISA IRQs
on x86 should always be active-hi. However, some recent AMD systems
actually use active-lo edge-sensitive ISA IRQs (and not just for
UARTs, but also for the keyboard and PS/2 mouse devices) and the
override causes interrupts to be dropped resulting in boot time hangs,
non-working keyboards, etc.
Add a hw.acpi.override_isa_irq_polarity tunable (readable as a sysctl
post-boot) to control this quirk. It can be set to 1 to force enable
the override and 0 to disable it. The log of original message
mentions an Intel motherboard as the sample case, so default the
tunable to 1 on systems with an Intel CPU and 0 otherwise.
Special thanks to Matthias Lanter <freebsd@lanter-it.ch> for tracking
down boot time issues on recent AMD systems to mismatched interrupt
polarity.
PR: 270707
Reported by: aixdroix_OSS@protonmail.com, Michael Dexter
Reported by: mfw_burn@pm.me, Hannes Hfauswedell <h2+fbsdports@fsfe.org>
Reported by: Matthias Lanter <freebsd@lanter-it.ch>
Reported by: William Bulley <web@umich.edu>
Reviewed by: imp, emaste
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D45554
