Rename isp_rd_2400_nvram to isp_rd_2xxx_flash.
Cleanup some leftovers.
Hide all output regarding FLT parsing behind ISP_LOGDEBUG0.
Thanks to imp@ and mav@ for reviewing and commenting.
PR: 271062
Reviewed by: imp, mav
Sponsored by: Technical University of Munich
Pull Request: https://github.com/freebsd/freebsd-src/pull/726
The ISP26xx based HBAs are left as is for now with static NVRAM addressing.
Those HBAs are known as 83xx (2031 and 8031 for real) and need special handling.
This is left for further investigation for now.
Cosmetics:
- rename functions and defines as they are no longer specific to 28xx
- set reasonable log levels
- sort FLT and NVRAM functions (in the order they are used)
Tested and approved to work on real hardware with:
- Qlogic ISP 2532 (QLogic QLE2562 8Gb 2Port FC Adapter)
- Qlogic ISP 2722 (QLogic QLE2690 16Gb FC Adapter)
- Qlogic ISP 2812 (QLogic QLE2772 32Gbit 2Port FC Adapter)
PR: 271062
Reviewed by: imp, mav
Sponsored by: Technical University of Munich
Pull Request: https://github.com/freebsd/freebsd-src/pull/726
The FLT is like a TOC for the flash area and contains entries for every flash
region with start/end address, size and flags.
Start using NVRAM addresses from FLT instead of hardcoded ones for ISP28xx
based HBAs.
The FLT should be available on earlier HBAs too, probably since ISP24xx based.
This needs further investigation and testing.
PR: 271062
Reviewed by: imp, mav
Sponsored by: Technical University of Munich
Pull Request: https://github.com/freebsd/freebsd-src/pull/726
Use correct NVRAM address for ISP28xx based HBAs to read NVRAM contents.
WWPN/WWNN and framesize are correctly read from NVRAM now.
PR: 271062
Reviewed by: imp, mav
Sponsored by: Technical University of Munich
Pull Request: https://github.com/freebsd/freebsd-src/pull/726
This covers the following HBAs:
ISP2812-based 64/32G Fibre Channel to PCIe Controller:
QLE2770 Single Port 32GFC PCIe Gen4 x8 Adapter
QLE2772 Dual Port 32GFC PCIe Gen4 x8 Adapter
QLE2870 Single Port 64GFC PCIe Gen4 x8 Adapter
QLE2872 Dual Port 64GFC PCIe Gen4 x8 Adapter
ISP2814-based 64/32G Fibre Channel to PCIe Controller:
QLE2774 Quad Port 32GFC PCIe Gen4 x16 Adapter
QLE2874 Quad Port 64GFC PCIe Gen4 x16 Adapter
While here, add required bits to support 64GB FC.
Default framesize is set to 2048 for ISP28xx based HBAs for now.
PR: 271062
Reviewed by: imp, mav
Sponsored by: Technical University of Munich
Pull Request: https://github.com/freebsd/freebsd-src/pull/726
The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.
Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix
It appears at least on QLE2694L cards 3rd and 4th ports follow the
same NVRAM addressing logic as the first two. In lack of proper
documentation this guess is as good as it can be.
MFC after: 1 week
Sponsored by: iXsystems, Inc.
- Remove code duplication by adding two new functions to execute prepared
queue entry via either mbox or request queue and wait for result.
- Since the new function executing via request queue sleeps any way, make
it sleep also in case of overflows or handle shortages. It should make it
more reliable and less affecting other less flexible request queue users.
- Turn isp_target_put_entry() into not target-specific isp_send_entry().
- Make handling of responses with control handles more universal.
- Move RQSTYPE_RPT_ID_ACQ handling into new function.
- Inline isp_handle_other_response(), becoming trivial after above.
- Clean the list of IOCBs from pre-24xx ones.
It was broken by design and unused for years due to conflicts between
different threads, fighting for the same set of mailbox registers, not
designed for multiple requests at a time. So either request has to be
synchronous and spin under the lock, or it should be sent asynchronously
through the queues as Mailbox Command IOCB or some other way.
This removes any OS specifics from the wait code, so it can be inlined.
Before this change in case of request queue overflow driver just froze the
device queue for 100ms to retry after. It was pretty bad for performance.
This change introduces SIM queue freezing when free space on the request
queue drops below 255 entries (worst case of maximum I/O size S/G list),
checking for a chance to release it on I/O completion. If the queue still
get overflowed somehow, the old mechanism is still in place, just with
delay reduced to 10ms.
With the earlier queue length increase overflows should not happen often,
but it is still easily reachable on synthetic tests.
- Allocate 256 handlers more than payload commands for management purposes.
- Increase maximum number of handlers from 8K to 16K by tuning the format.
- Just to be safe limit the number of payload commands to 16K - 256.
- Limit number of target exchanges in mixed mode to the number of atpds.
- If we still somehow get out of atpds -- return BUSY, since we really are.
Qlogic chips store S/G lists in the same queue as requests themselves. In
the worst case 1MB I/O may require up to 52 IOCBs, that means queue of 1024
IOCBs can store only 19 of such requests. The increase reduces chances of
overflow, while we should be able to afford additional 512KB of RAM per HBA.
The Linux driver uses comparable numbers.
While there, decouple ATIO queue size from response queue size. There is
no reason for them to be equal.
- Make isp_start() to set all the IOCB fields aside of S/G list, removing
extra information from isp_send_cmd(), now only doing S/G lists and sending.
- Turn DMA setup/free from being card and PCI-specific into OS-specific,
instead add new card-specific method for isp_send_cmd(). Previously this
function was a monster handling all the cards.
- Remove double error code translation.
This removes 288KB (36%) of the driver code and zillions of hacks and
workarounds, making single driver uniformly support several different
generations of hardware interfaces, not counting minor card variations.
After years of the hopeless fight, I don't think it worth to continue
support for hardware obsolete for 15-20 years. Instead much cleaner
now code should allow to move forward toward better locking, multiple
queues and other cool features.
All the remaining Qlogic cards starting from 4Gb 24xx to 32Gb 27xx use
the same hardware/firmware interface with minor incremental improvements,
so it seems to be a good new starting point. Except one PCI-X model all
all of them are PCIe and so still usable in modern systems.
Discussed with: ken, scottl, jpaetzel, imp
Relnotes: yes
SAM-3 specification introduced concept of Task Priority, that was renamed
to Command Priority in SAM-4, and supported by all modern SCSI transports.
It provides 15 levels of relative priorities: 1 - highest, 15 - lowest and
0 - default. SAT specification for SATA devices translates priorities 1-3
into NCQ high priority.
This change adds new "priority" field into empty spots of struct ccb_scsiio
and struct ccb_accept_tio of CAM and struct ctl_scsiio of CTL. Respective
support is added into iscsi(4), isp(4), mpr(4), mps(4) and ocs_fc(4) drivers
for both initiator and where applicable target roles. Minimal support was
added to CTL to receive the priority value from different frontends, pass it
between HA controllers and report in few places.
This patch does not add consumers of this functionality, so nothing should
really change yet, since the field is still set to 0 (default) on initiator
and not actively used on target. Those are to be implemented separately.
I've confirmed priority working on WD Red SATA disks connected via mpr(4)
and properly transferred to CTL target via iscsi(4), isp(4) and ocs_fc(4).
While there, added missing tag_action support to ocs_fc(4) initiator role.
MFC after: 1 month
Relnotes: yes
Sponsored by: iXsystems, Inc.
The point of r345008 was to reset the Command Reference Number (CRN)
in some situations where a device stayed in the topology, but had
changed somehow.
This can include moving from a switch connection to a direct
connection or vice versa, or a device that temporarily goes away
and comes back. (e.g. moving to a different switch port)
There were a couple of bugs in that change:
- We were reporting that a device had not changed whenever the
Establish Image Pair bit was not set. That is not quite correct.
Instead, if the Establish Image Pair bit stays the same (set or
not), the device hasn't changed in that way.
- We weren't setting PRLI Word0 in the port database when a new
device arrived, so comparisons with the old value for the
Establish Image Pair bit weren't really possible. So, make sure
PRLI Word0 is set in the port database for new devices.
- We were resetting the CRN whenever the Establish Image Pair bit
was set for a device, even when the device had stayed the same
and the value of the bit hadn't changed. Now, only reset the
CRN for devices that have changed, not devices that sayed the
same.
The result of all of this was that if we had a single FC device on
an FC port and it went away and came back, we would wind up
correctly resetting the CRN.
But, if we had multiple devices connected via a switch, and there
was any change in one or more of those devices, all of the devices
that stayed the same would also have their CRN values reset.
The result, from a user standpoint, is that the tape drives, etc.
would all start to time out commands and the initiator would send
aborts.
sys/dev/isp/isp.c:
In isp_pdb_add_update(), look at whether the Establish
Image Pair bit has changed as part of the check to
determine whether a device is still the same. This was
causing erroneous change notifications. Also, when
creating a new port database entry, initialize the
PRLI Word 0 values.
sys/dev/isp/isp_freebsd.c:
In isp_async(), in the changed/stayed case, instead of
looking at the Establish Image Pair bit to determine
whether to reset the CRN, look at the command value.
(Changed vs. Stayed.) Only reset the CRN for devices
that have changed.
Sponsored by: Spectra Logic
MFC after: 3 days
The Command Reference Number (CRN) is part of the FC-Tape features
that we enable when talking to tape drives. It starts at 1, and
goes to 255 and wraps around to 1. There are a number of reset
type conditions that result in the CRN getting reset to 1. These
are detailed in section 4.10 (table 8) of the FCP-4r02b specification.
One of the conditions is when a PRLI (Process Login) is sent by
the initiator, and the Establish Image Pair bit is set in Word 0
of the PRLI.
Previously, the isp(4) driver core sent a notification via
isp_async() that the target had changed or stayed in place, but
there was no indication of whether a PRLI was sent and whether the
Establish Image Pair bit was set.
The result of this was that in some situations, notably
switching back and forth between a direct connection and a switch
connection to a tape drive, the isp(4) driver would fail to reset
the CRN in situations that require it according to the spec. When
the CRN isn't reset in a situation that requires it, the tape drive
then rejects every subsequent command that is sent to the drive.
It is assuming that the commands are being sent out of order.
So, modify the isp(4) driver to include Word 0 of the PRLI command
when it sends isp_async() notifications of target changes. Look at
the Establish Image Pair bit, and reset the CRN if that bit is set.
With this change, I am able to switch a tape drive back and forth
between a direct connection and a switch connection, and the isp(4)
driver resets the CRN when it should.
sys/dev/isp_stds.h:
Add bit definitions for PRLI Word 0.
sys/dev/ispmbox.h:
Add PRLI Word 0 to the port database type, isp_pdb_t.
sys/dev/ispvar.h
Add PRLI Word 0 to fcportdb_t.
sys/dev/isp.c:
Populate the new prli_word0 parameter in the port database.
In isp_pdb_add_update(), add a check to see if the
Establish Image Pair bit is set in PRLI Word 0. If it is,
then that is an additional reason to create a change
notification.
sys/dev/isp_freebsd.c:
In isp_async(), if the device changed or stayed, look at
PRLI Word 0 to see if the Establish Image Pair bit is set.
If it is, reset the CRN if we haven't already.
MFC after: 1 week
Sponsored by: Spectra Logic
Differential Revision: https://reviews.freebsd.org/D19472
It seems default timeout of 100ms is not enough for my 2694L card,
while it was perfectly fine for others, even for full-height 2694.
MFC after: 1 week
Sponsored by: iXsystems, Inc.
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
This largely reverts FreeBSD SVN change 289937 from October 25th, 2015.
The intent of that change was to keep loop IDs persistent across
chip reinits.
The problem is that the change turned on the PREVLOOP /
PREV_ADDRESS bit (bit 7 in Firmware Options 2), which tells the
Qlogic chip to not participate in the loop if it can't get the
requested loop address. It also turned off soft addressing on 2400
(4Gb) and newer controllers.
The isp(4) driver defaults to loop address 0, and the tape drives
I have tested default to loop address 0 if hard addressing is turned
on. So when hard loop addressing is turned on on the drive, the isp(4)
driver just refuses to participate in the loop.
The solution is to largely revert that change. I left some elements
in place that are related to virtual ports, since they were new.
This does work with IBM tape drives with hard and soft addressing
turned on. I have tested it with 4Gb, 8Gb, and 16Gb controllers.
sys/dev/isp.c:
Largely revert FreeBSD SVN change 289937. I left the
ispmbox.h changes in place.
Don't use the PREV_ADDRESS bit on initialization. It tells
the chip to not participate if it can't get the requested
loop ID.
Do use soft addressing on 2400 and newer chips.
Use hard addressing when the user has requested a specific
initiator ID. (hint.isp.X.iid=N in /boot/loader.conf)
Leave some of the virtual port options from that change in
place, but don't turn on the PREV_ADDRESS bit.
Reviewed by: mav
MFC after: 3 days
Sponsored by: Spectra Logic
Instead of using GID_FT SNS request to get list of registered FCP ports,
use GID_PT to get list of all Nx_Ports, and then use GFF_ID and/or GFT_ID
requests to find whether they are FCP and target capable.
The problem with old approach is that GID_FT does not report ports without
FC-4 type registered. In particular it was impossible to boot OS from
FreeBSD FC target using QLogic FC BIOS, since one does not register FC-4
type even on new cards and so ignored by old code as incompatible.
As a side bonus this allows initiator to skip pointless logins to other
initiators by fetching that information from SNS instead.
In case some switches do not implement GFF_ID/GFT_ID correctly, add sysctls
to disable that functionality. I handled broken GFF_ID of my Brocade 200E,
but there may be other switches with different bugs.
Linux also uses GID_PT, but GFF_ID is disabled by default there, and GFT_ID
is not supported.
Sponsored by: iXsystems, Inc.
Prior to this change, the CRN (Command Reference Number) is reset on any
firmware LIP, LOOP DOWN, or LOOP RESET event in violation of FCP-4 which
specifies that the CRN should only be reset in response to a LIP Reset
(LIPyx) primitive. FCP-4 also indicates PLOGI/LOGO and PRLI/PRLO ELS
actions as conditions for resetting the CRN for the associated initiator
port.
These violations manifest themselves when the HBA is removed from the
loop, or a target device is removed (especially during an outstanding
command) without power cycling. If the HBA and and the target device
determine upon re-establishing the loop that no PLOGI or PRLI is
required, and the target does not issue a LIPxy to the initiator, the
CRN for the target will have been improperly reset by the isp driver. As
a result, the target port will silently ignore all FCP commands issued
during the device probe (which will time out) preventing the device from
attaching.
This change corrects thie CRN reset behavior in response to loop state
changes, also introduces CRN resets for the above mentioned ELS actions
as encountered through async PDB change events.
This change also adds cleanup of outstanding commands in isp_loop_dead()
that was previously missing.
sys/dev/isp/isp.c
Add the last login state to debug output when syncing the pdb
sys/dev/isp/isp_freebsd.c
Replace binary statement setting aborted ccb status in
isp_watchdog() with the XS_SETERR macro used elsewhere
In isp_loop_dead(), abort or complete pending commands as done
in isp_watchdog()
In isp_async(), segregate the ISPASYNC_LOOP_RESET action from
ISPASYNC_LIP, ISPASYNC_LOOP_DOWN, and ISPASYNC_LOOP_UP
fallthroughs, and only reset the CRN in the RESET case. Also add
checks to handle false LOOP RESET actions that do not have a
proper associated LIP primitive, and log the primitive in the
debug messages
In isp_async(), remove the goto from ISP_ASYNC_DEV_STAYED, and
only reset the CRN in the DEV_CHANGED action
In isp_async(), when processing an ISPASYNC_CHANGE_PDB status,
reset CRN(s) for the associated nphdl (or all ports) if the
change reason is some form of ELS login/logout. Also remove
assignment to fc since it is not used in the scope
sys/dev/isp/ispmbox.h
Add macro definition for the global N-Port handle, and correct a
macro typo 'PDB24XX_AE_PRLI_DONJE'
sys/dev/isp/ispvar.h
Add macros FCP_AL_DA_ALL, FCP_AL_PA, and FCP_IS_DEST_ALPD for
more legible code when determining if an AL_PD port matches the
portid for a given struct fcparam* by value or by virtue of the
AL_PD port being 0xFF
Submitted by: Reid Linnemann
Sponsored by: Spectra Logic
MFC after: 1 week
Let firmware do its best first, and if it can't, try software recovery.
I would remove software timeout handler completely, but found bunch of
complains on command timeout on sparc64 mailing list few years ago, so
better be safe in case of interrupt loss.
MFC after: 2 weeks
For 24xx and above use 2 vectors (default and response queue).
For 26xx and above use 3 vectors (default, response and ATIO queues).
Due to global lock interrupt hardlers never run simultaneously now, but
at least this allows to save one regitster read per interrupt.
MFC after: 2 weeks
Since we support RQSTYPE_RPT_ID_ACQ, that functionality is only useful
in loop mode, which probably doesn't worth having this hack in 2017.
MFC after: 2 weeks
Instead of single isp_intr() function doing all possible magic, introduce
four different functions to handle mailbox operation completions, async
events, response and ATIO queues. The goal is to isolate different code
paths to make code more readable, and to make easier support for multiple
interrupt vectors. Even oldest hardware in many cases can identify what
code path it should run on interrupt. Contemporary hardware can assign
them to different interrupt vectors.
MFC after: 2 weeks
It was implemented to reduce context switches when uploading firmware to
card's RAM. But this mechanism is not used last 10 years since all mbox
operations are now polled, and it was never used for cards produced in
last 15 years. Newer cards can use DMA to upload firmware.
MFC after: 2 weeks
This change fixes DMA resource leak on driver unload. Also it removes
DMA resources allocation for hardcoded number of requests before fetching
the real number from firmware. Also it prepares ground for more flexible
IRQs allocation according to firmware capabilities.
MFC after: 2 weeks
