mirror of
https://github.com/opnsense/src.git
synced 2026-05-19 08:25:22 -04:00
Upgrade ENA to v2.4.0
ena: change ENA C++-style comment into C-style According to man style(9), only C-style comments should be used. Submitted by: Michal Krawczyk <mk@semihalf.com> Obtained from: Semihalf MFC after: 2 weeks Sponsored by: Amazon, Inc. (cherry picked from commit438c9e3cf8) ena: add support for the large LLQ headers in ENA Default LLQ (Low-latency queue) maximum header size is 96 bytes and can be too small for some types of packets - like IPv6 packets with multiple extension. This can be fixed, by using large LLQ headers. If the device supports larger LLQ headers, the user can activate this feature by setting sysctl tunable 'hw.ena.force_large_llq_header' to '1' in the /boot/loader.conf file. In case the device isn't supporting this feature, the default value (96B) will be used. Submitted by: Michal Krawczyk <mk@semihalf.com> Obtained from: Semihalf MFC after: 2 weeks Sponsored by: Amazon, Inc. (cherry picked from commitbeaadec9ea) ena: remove surplus NULL checks when freeing ENA resources Calling free on a NULL pointer is valid, as appropriate check is already done internally: /* free(NULL, ...) does nothing */ if (addr == NULL) return; Submitted by: Artur Rojek <ar@semihalf.com> Obtained from: Semihalf MFC after: 2 weeks Sponsored by: Amazon, Inc. (cherry picked from commitddec69e6a7) ena: hide sysctl nodes for unused ENA queues IO queue related attributes are registered statically at driver attach with the rest of the ENA specific sysctl nodes. However, the number of queues can be changed at runtime via the `ena_sysctl_io_queues_nb` request, leading to a potential exposure of attributes for non-existing queues. Introduce a new `ena_sysctl_update_queue_node_nb` function, which updates the sysctl nodes after the number of queues is altered. This happens by either registering or unregistering node specific oids, based on a delta between the previous and current queue count. NOTE: All unregistered oids must be registered again before the driver detach, e.g. by another call to this function. Submitted by: Artur Rojek <ar@semihalf.com> Obtained from: Semihalf MFC after: 2 weeks Sponsored by: Amazon, Inc. (cherry picked from commit0e7d31f63b) Merge tag 'vendor/ena-com/2.4.0' Update the driver in order not to break its compilation and make use of the new ENA logging system Migrate platform code to the new logging system provided by ena_com layer. Make ENA_INFO the new default log level. Remove all explicit use of `device_printf`, all new logs requiring one of the log macros to be used. (cherry picked from commit3fc5d816f8) Update ENA driver man page Bring the obsolete man page up to date: * update diagnostic error messages * add documentation of loader tunables * document netmap support * add a driver history section * update the contact information Submitted by: Artur Rojek <ar@semihalf.com> Submitted by: Michal Krawczyk <mk@semihalf.com> Obtained from: Semihalf MFC after: 2 weeks Sponsored by: Amazon, Inc. (cherry picked from commite34856a2c4) Update ENA version to v2.4.0 Some of the changes in this release: * Large LLQ headers, * Bug/stability fixes, * Change of the README/Documentation. Submitted by: Michal Krawczyk <mk@semihalf.com> Obtained from: Semihalf MFC after: 2 weeks Sponsored by: Amazon, Inc. (cherry picked from commit93f0df457b)
This commit is contained in:
Marcin Wojtas
parent
fb5eed2c43
commit
87ffe59470
14 changed files with 743 additions and 368 deletions
|
|
@ -29,7 +29,7 @@
|
|||
.\"
|
||||
.\" $FreeBSD$
|
||||
.\"
|
||||
.Dd August 16, 2017
|
||||
.Dd June 4, 2021
|
||||
.Dt ENA 4
|
||||
.Os
|
||||
.Sh NAME
|
||||
|
|
@ -86,7 +86,13 @@ debug logs.
|
|||
.Pp
|
||||
Some of the ENA devices support a working mode called Low-latency
|
||||
Queue (LLQ), which saves several more microseconds.
|
||||
This feature will be implemented for driver in future releases.
|
||||
.Pp
|
||||
Support for the
|
||||
.Xr netmap 4
|
||||
framework is provided by the
|
||||
.Nm
|
||||
driver.
|
||||
Kernel must be built with the DEV_NETMAP option to be able to use this feature.
|
||||
.Sh HARDWARE
|
||||
Supported PCI vendor ID/device IDs:
|
||||
.Pp
|
||||
|
|
@ -100,8 +106,172 @@ Supported PCI vendor ID/device IDs:
|
|||
.It
|
||||
1d0f:ec21 - ENA VF with LLQ support
|
||||
.El
|
||||
.Sh LOADER TUNABLES
|
||||
The
|
||||
.Nm
|
||||
driver's behavior can be changed using run-time or boot-time sysctl
|
||||
arguments.
|
||||
The boot-time arguments can be set at the
|
||||
.Xr loader 8
|
||||
prompt before booting the kernel, or stored in the
|
||||
.Xr loader.conf 5 .
|
||||
The run-time arguments can be set using the
|
||||
.Xr sysctl 8
|
||||
command.
|
||||
.Pp
|
||||
Boot-time tunables:
|
||||
.Bl -tag -width indent
|
||||
.It Va hw.ena.enable_9k_mbufs
|
||||
Use 9k mbufs for the Rx descriptors.
|
||||
The default is 0.
|
||||
If the node value is set to 1, 9k mbufs will be used for the Rx buffers.
|
||||
If set to 0, page size mbufs will be used instead.
|
||||
.Pp
|
||||
Using 9k buffers for Rx can improve Rx throughput, but in low memory conditions
|
||||
it might increase allocation time, as the system has to look for 3 contiguous
|
||||
pages.
|
||||
This can further lead to OS instability, together with ENA driver reset and NVMe
|
||||
timeouts.
|
||||
If network performance is critical and memory capacity is sufficient, the 9k
|
||||
mbufs can be used.
|
||||
.It Va hw.ena.force_large_llq_headers
|
||||
Force the driver to use large LLQ headers (224 bytes).
|
||||
The default is 0.
|
||||
If the node value is set to 0, the regular size LLQ header will be used, which
|
||||
is 96B.
|
||||
In some cases, the packet header can be bigger than this (for example -
|
||||
IPv6 with multiple extensions).
|
||||
In such a situation, the large LLQ headers should be used by setting this node
|
||||
value to 1.
|
||||
This will take effect only if the device supports both LLQ and large LLQ
|
||||
headers.
|
||||
Otherwise, it will fallback to the no LLQ mode or regular header size.
|
||||
.Pp
|
||||
Increasing LLQ header size reduces the size of the Tx queue by half, so it may
|
||||
affect the number of dropped Tx packets.
|
||||
.El
|
||||
.Pp
|
||||
Run-time tunables:
|
||||
.Bl -tag -width indent
|
||||
.It Va hw.ena.log_level
|
||||
Controls extra logging verbosity of the driver.
|
||||
The default is 2.
|
||||
The higher the logging level, the more logs will be printed out. 0 means all
|
||||
extra logs are disabled and only error logs will be printed out.
|
||||
Default value (2) reports errors, warnings and is verbose about driver
|
||||
operation.
|
||||
.Pp
|
||||
The possible flags are:
|
||||
.Pp
|
||||
.Bl -bullet -compact
|
||||
.It
|
||||
0 - ENA_ERR - Enable driver error messages and ena_com error logs.
|
||||
.It
|
||||
1 - ENA_WARN - Enable logs for non-critical errors.
|
||||
.It
|
||||
2 - ENA_INFO - Make the driver more verbose about its actions.
|
||||
.It
|
||||
3 - ENA_DBG - Enable debug logs.
|
||||
.El
|
||||
.Pp
|
||||
NOTE: In order to enable logging on the Tx/Rx data path, driver must be compiled
|
||||
with ENA_LOG_IO_ENABLE compilation flag.
|
||||
.Pp
|
||||
Example:
|
||||
To enable logs for errors and warnings, the following command should be used:
|
||||
.Bd -literal -offset indent
|
||||
sysctl hw.ena.log_level=1
|
||||
.Ed
|
||||
.It Va dev.ena.X.io_queues_nb
|
||||
Number of the currently allocated and used IO queues.
|
||||
The default is max_num_io_queues.
|
||||
Controls the number of IO queue pairs (Tx/Rx). As this call has to reallocate
|
||||
the queues, it will reset the interface and restart all the queues - this means
|
||||
that everything, which was currently held in the queue, will be lost, leading to
|
||||
potential packet drops.
|
||||
.Pp
|
||||
This call can fail if the system isn't able to provide the driver with enough
|
||||
resources.
|
||||
In that situation, the driver will try to revert the previous number of the IO
|
||||
queues.
|
||||
If this also fails, the device reset will be triggered.
|
||||
.Pp
|
||||
Example:
|
||||
To use only 2 Tx and Rx queues for the device ena1, the following command should
|
||||
be used:
|
||||
.Bd -literal -offset indent
|
||||
sysctl dev.ena.1.io_queues_nb=2
|
||||
.Ed
|
||||
.It Va dev.ena.X.rx_queue_size
|
||||
Size of the Rx queue.
|
||||
The default is 1024.
|
||||
Controls the number of IO descriptors for each Rx queue.
|
||||
The user may want to increase the Rx queue size if they observe a high number of
|
||||
Rx drops in the driver's statistics.
|
||||
For performance reasons, the Rx queue size must be a power of 2.
|
||||
.Pp
|
||||
This call can fail if the system isn't able to provide the driver with enough
|
||||
resources.
|
||||
In that situation, the driver will try to revert to the previous number of the
|
||||
descriptors.
|
||||
If this also fails, the device reset will be triggered.
|
||||
.Pp
|
||||
Example:
|
||||
To increase Rx ring size to 8K descriptors for the device ena0, the following
|
||||
command should be used:
|
||||
.Bd -literal -offset indent
|
||||
sysctl dev.ena.0.rx_queue_size=8192
|
||||
.Ed
|
||||
.It Va dev.ena.X.buf_ring_size
|
||||
Size of the Tx buffer ring (drbr).
|
||||
The default is 4096.
|
||||
Input must be a power of 2.
|
||||
Controls the number of mbufs that can be held in the Tx buffer ring.
|
||||
The drbr is used as a multiple-producer, single-consumer lockless ring for
|
||||
buffering extra mbufs coming from the stack in case the Tx procedure is busy
|
||||
sending the packets, or the Tx ring is full.
|
||||
Increasing the size of the buffer ring may reduce the number of Tx packets being
|
||||
dropped in case of a big Tx burst, which cannot be handled by the IO queue
|
||||
immediately.
|
||||
Each Tx queue has its own drbr.
|
||||
.Pp
|
||||
It is recommended to keep the drbr with at least the default value, but in case
|
||||
the system lacks the resources, it can be reduced.
|
||||
This call can fail if the system is not able to provide the driver with enough
|
||||
resources.
|
||||
In that situation, the driver will try to revert to the previous number of the
|
||||
drbr and trigger the device reset.
|
||||
.Pp
|
||||
Example:
|
||||
To set drbr size for interface ena0 to 2048, the following command should
|
||||
be used:
|
||||
.Bd -literal -offset indent
|
||||
sysctl dev.ena.0.buf_ring_size=2048
|
||||
.Ed
|
||||
.It Va dev.ena.X.eni_metrics.sample_interval
|
||||
Interval in seconds for updating ENI metrics.
|
||||
The default is 0.
|
||||
Determines how often (if ever) the ENI metrics should be updated.
|
||||
The ENI metrics are being updated asynchronously in a timer service in order to
|
||||
avoid admin queue overload by sysctl node reading.
|
||||
The value in this node controls the interval between issuing admin commands to
|
||||
the device, which will update the ENI metrics values.
|
||||
.Pp
|
||||
If some application is periodically monitoring the eni_metrics, then the ENI
|
||||
metrics interval can be adjusted accordingly.
|
||||
Value 0 turns off the update completely.
|
||||
Value 1 is the minimum interval and is equal to 1 second.
|
||||
The maximum allowed update interval is 1 hour.
|
||||
.Pp
|
||||
Example:
|
||||
To update ENI metrics for the device ena1 every 10 seconds, the following
|
||||
command should be used:
|
||||
.Bd -literal -offset indent
|
||||
sysctl dev.ena.1.eni_metrics.sample_interval=10
|
||||
.Ed
|
||||
.El
|
||||
.Sh DIAGNOSTICS
|
||||
.Ss Device initialization phase:
|
||||
.Ss Device initialization phase
|
||||
.Bl -diag
|
||||
.It ena%d: failed to init mmio read less
|
||||
.Pp
|
||||
|
|
@ -116,7 +286,7 @@ Device may not be responding or is already during reset.
|
|||
Version of the controller is too old and it is not supported by the driver.
|
||||
.It ena%d: Invalid dma width value %d
|
||||
.Pp
|
||||
The controller is able to request dma transaction width.
|
||||
The controller is unable to request dma transaction width.
|
||||
.br
|
||||
Device stopped responding or it demanded invalid value.
|
||||
.It ena%d: Can not initialize ena admin queue with device
|
||||
|
|
@ -132,33 +302,30 @@ Failed to get attributes of the device from the controller.
|
|||
.Pp
|
||||
Errors occurred when trying to configure AENQ groups.
|
||||
.El
|
||||
.Ss Driver initialisation/shutdown phase:
|
||||
.Ss Driver initialization/shutdown phase
|
||||
.Bl -diag
|
||||
.It ena%d: PCI resource allocation failed!
|
||||
.It ena%d: allocating ena_dev failed
|
||||
.It ena%d: failed to pmap registers bar
|
||||
.It ena%d: Error while setting up bufring
|
||||
.It ena%d: Error with initialization of IO rings
|
||||
.It ena%d: can not allocate ifnet structure
|
||||
.It ena%d: Error with network interface setup
|
||||
.It ena%d: Failed to enable and set the admin interrupts
|
||||
.It ena%d: Failed to allocate %d, vectors %d
|
||||
.It ena%d: Error, MSI-X is already enabled
|
||||
.It ena%d: Failed to enable MSIX, vectors %d rc %d
|
||||
.It ena%d: Not enough number of MSI-X allocated: %d
|
||||
.It ena%d: Error with MSI-X enablement
|
||||
.It ena%d: could not allocate irq vector: %d
|
||||
.It ena%d: Unable to allocate bus resource: registers
|
||||
.It ena%d: unable to allocate bus resource: registers!
|
||||
.It ena%d: unable to allocate bus resource: msix!
|
||||
.Pp
|
||||
Resource allocation failed when initializing the device.
|
||||
.br
|
||||
Driver will not be attached.
|
||||
.It ena%d: ENA device init failed (err: %d)
|
||||
.It ena%d: Cannot initialize device
|
||||
.Pp
|
||||
Device initialization failed.
|
||||
.br
|
||||
Driver will not be attached.
|
||||
.It ena%d: could not activate irq vector: %d
|
||||
.Pp
|
||||
Error occurred when trying to activate interrupt vectors for Admin Queue.
|
||||
.It ena%d: failed to register interrupt handler for irq %ju: %d
|
||||
.Pp
|
||||
Error occurred when trying to register Admin Queue interrupt handler.
|
||||
|
|
@ -181,8 +348,7 @@ VLANs must be detached first and then detach routine have to be called again.
|
|||
.It ena%d: Unmapped TX DMA tag associations
|
||||
.Pp
|
||||
Error occurred when trying to destroy RX/TX DMA tag.
|
||||
.It ena%d: Cannot init RSS
|
||||
.It ena%d: Cannot fill indirect table
|
||||
.It ena%d: Cannot init indirect table
|
||||
.It ena%d: Cannot fill indirect table
|
||||
.It ena%d: Cannot fill hash function
|
||||
.It ena%d: Cannot fill hash control
|
||||
|
|
@ -192,20 +358,30 @@ Error occurred during initialization of one of RSS resources.
|
|||
.br
|
||||
The device will work with reduced performance because all RX packets will be
|
||||
passed to queue 0 and there will be no hash information.
|
||||
.It ena%d: LLQ is not supported. Fallback to host mode policy.
|
||||
.It ena%d: Failed to configure the device mode. Fallback to host mode policy.
|
||||
.It ena%d: unable to allocate LLQ bar resource. Fallback to host mode policy.
|
||||
.Pp
|
||||
Error occured during Low-latency Queue mode setup.
|
||||
.br
|
||||
The device will work, but without the LLQ performance gain.
|
||||
.It ena%d: failed to enable write combining.
|
||||
.Pp
|
||||
Error occured while setting the Write Combining mode, required for the LLQ.
|
||||
.It ena%d: failed to tear down irq: %d
|
||||
.It ena%d: dev has no parent while releasing res for irq: %d
|
||||
Release of the interrupts failed.
|
||||
.El
|
||||
.Ss Additional diagnostic:
|
||||
.Ss Additional diagnostic
|
||||
.Bl -diag
|
||||
.It ena%d: Cannot get attribute for ena device
|
||||
.Pp
|
||||
This message appears when trying to change MTU and driver is unable to get
|
||||
attributes from the device.
|
||||
.It ena%d: Invalid MTU setting. new_mtu: %d
|
||||
.It ena%d: Invalid MTU setting. new_mtu: %d max_mtu: %d min mtu: %d
|
||||
.Pp
|
||||
Requested MTU value is not supported and will not be set.
|
||||
.It ena%d: keep alive watchdog timeout
|
||||
.It ena%d: Failed to set MTU to %d
|
||||
.Pp
|
||||
This message appears when either MTU change feature is not supported, or device
|
||||
communication error has occured.
|
||||
.It ena%d: Keep alive watchdog timeout.
|
||||
.Pp
|
||||
Device stopped responding and will be reset.
|
||||
.It ena%d: Found a Tx that wasn't completed on time, qid %d, index %d.
|
||||
|
|
@ -215,18 +391,35 @@ Packet was pushed to the NIC but not sent within given time limit.
|
|||
It may be caused by hang of the IO queue.
|
||||
.It ena%d: The number of lost tx completion is aboce the threshold (%d > %d). Reset the device
|
||||
.Pp
|
||||
If too many Tx wasn't completed on time the device is going to be reset.
|
||||
If too many Tx weren't completed on time the device is going to be reset.
|
||||
.br
|
||||
It may be caused by hanged queue or device.
|
||||
.It ena%d: trigger reset is on
|
||||
.It ena%d: Trigger reset is on
|
||||
.Pp
|
||||
Device will be reset.
|
||||
.br
|
||||
Reset is triggered either by watchdog or if too many TX packets were not
|
||||
completed on time.
|
||||
.It ena%d: invalid value recvd
|
||||
.It ena%d: device reset scheduled but trigger_reset is off
|
||||
.Pp
|
||||
Link status received from the device in the AENQ handler is invalid.
|
||||
Reset task has been triggered, but the driver did not request it.
|
||||
.br
|
||||
Device reset will not be performed.
|
||||
.It ena%d: Device reset failed
|
||||
.Pp
|
||||
Error occured while trying to reset the device.
|
||||
.It ena%d: Cannot initialize device
|
||||
.It ena%d: Error, mac address are different
|
||||
.It ena%d: Error, device max mtu is smaller than ifp MTU
|
||||
.It ena%d: Validation of device parameters failed
|
||||
.It ena%d: Enable MSI-X failed
|
||||
.It ena%d: Failed to create I/O queues
|
||||
.It ena%d: Reset attempt failed. Can not reset the device
|
||||
.Pp
|
||||
Error occured while trying to restore the device after reset.
|
||||
.It ena%d: Device reset completed successfully, Driver info: %s
|
||||
.Pp
|
||||
Device has been correctly restored after reset and is ready to use.
|
||||
.It ena%d: Allocation for Tx Queue %u failed
|
||||
.It ena%d: Allocation for Rx Queue %u failed
|
||||
.It ena%d: Unable to create Rx DMA map for buffer %d
|
||||
|
|
@ -234,7 +427,6 @@ Link status received from the device in the AENQ handler is invalid.
|
|||
.It ena%d: Failed to get TX queue handlers. TX queue num %d rc: %d
|
||||
.It ena%d: Failed to create io RX queue[%d] rc: %d
|
||||
.It ena%d: Failed to get RX queue handlers. RX queue num %d rc: %d
|
||||
.It ena%d: failed to request irq
|
||||
.It ena%d: could not allocate irq vector: %d
|
||||
.It ena%d: failed to register interrupt handler for irq %ju: %d
|
||||
.Pp
|
||||
|
|
@ -246,12 +438,20 @@ Interface will not be brought up.
|
|||
Initialization of the LRO for the RX ring failed.
|
||||
.It ena%d: failed to alloc buffer for rx queue
|
||||
.It ena%d: failed to add buffer for rx queue %d
|
||||
.It ena%d: refilled rx queue %d with %d pages only
|
||||
.It ena%d: refilled rx qid %d with only %d mbufs (from %d)
|
||||
.Pp
|
||||
Allocation of resources used on RX path failed.
|
||||
.br
|
||||
If happened during initialization of the IO queue, the interface will not be
|
||||
brought up.
|
||||
.It ena%d: NULL mbuf in rx_info
|
||||
.Pp
|
||||
Error occured while assembling mbuf from descriptors.
|
||||
.It ena%d: tx_info doesn't have valid mbuf
|
||||
.It ena%d: Invalid req_id: %hu
|
||||
.It ena%d: failed to prepare tx bufs
|
||||
.Pp
|
||||
Error occured while preparing a packet for transmission.
|
||||
.It ena%d: ioctl promisc/allmulti
|
||||
.Pp
|
||||
IOCTL request for the device to work in promiscuous/allmulti mode.
|
||||
|
|
@ -259,22 +459,23 @@ IOCTL request for the device to work in promiscuous/allmulti mode.
|
|||
See
|
||||
.Xr ifconfig 8
|
||||
for more details.
|
||||
.It ena%d: too many fragments. Last fragment: %d!
|
||||
.Pp
|
||||
Packet with unsupported number of segments was queued for sending to the
|
||||
device.
|
||||
.br
|
||||
Packet will be dropped.
|
||||
.El
|
||||
.Sh SUPPORT
|
||||
If an issue is identified with the released source code with a supported
|
||||
adapter, please email the specific information related to the issue to
|
||||
.Aq Mt mk@semihalf.com
|
||||
.Aq Mt mk@semihalf.com ,
|
||||
.Aq Mt ar@semihalf.com
|
||||
and
|
||||
.Aq Mt mw@semihalf.com .
|
||||
.Sh SEE ALSO
|
||||
.Xr netmap 4 ,
|
||||
.Xr vlan 4 ,
|
||||
.Xr ifconfig 8
|
||||
.Sh HISTORY
|
||||
The
|
||||
.Nm
|
||||
driver first appeared in
|
||||
.Fx 11.1 .
|
||||
.Sh AUTHORS
|
||||
The
|
||||
.Nm
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
/*-
|
||||
* SPDX-License-Identifier: BSD-3-Clause
|
||||
*
|
||||
* Copyright (c) 2015-2020 Amazon.com, Inc. or its affiliates.
|
||||
* Copyright (c) 2015-2021 Amazon.com, Inc. or its affiliates.
|
||||
* All rights reserved.
|
||||
*
|
||||
* Redistribution and use in source and binary forms, with or without
|
||||
|
|
@ -694,7 +694,7 @@ static int ena_com_config_llq_info(struct ena_com_dev *ena_dev,
|
|||
} else {
|
||||
ena_trc_err(ena_dev, "Invalid header location control, supported: 0x%x\n",
|
||||
supported_feat);
|
||||
return -EINVAL;
|
||||
return ENA_COM_INVAL;
|
||||
}
|
||||
|
||||
if (likely(llq_info->header_location_ctrl == ENA_ADMIN_INLINE_HEADER)) {
|
||||
|
|
@ -709,7 +709,7 @@ static int ena_com_config_llq_info(struct ena_com_dev *ena_dev,
|
|||
} else {
|
||||
ena_trc_err(ena_dev, "Invalid desc_stride_ctrl, supported: 0x%x\n",
|
||||
supported_feat);
|
||||
return -EINVAL;
|
||||
return ENA_COM_INVAL;
|
||||
}
|
||||
|
||||
ena_trc_err(ena_dev, "Default llq stride ctrl is not supported, performing fallback, default: 0x%x, supported: 0x%x, used: 0x%x\n",
|
||||
|
|
@ -738,7 +738,7 @@ static int ena_com_config_llq_info(struct ena_com_dev *ena_dev,
|
|||
} else {
|
||||
ena_trc_err(ena_dev, "Invalid entry_size_ctrl, supported: 0x%x\n",
|
||||
supported_feat);
|
||||
return -EINVAL;
|
||||
return ENA_COM_INVAL;
|
||||
}
|
||||
|
||||
ena_trc_err(ena_dev, "Default llq ring entry size is not supported, performing fallback, default: 0x%x, supported: 0x%x, used: 0x%x\n",
|
||||
|
|
@ -752,7 +752,7 @@ static int ena_com_config_llq_info(struct ena_com_dev *ena_dev,
|
|||
*/
|
||||
ena_trc_err(ena_dev, "Illegal entry size %d\n",
|
||||
llq_info->desc_list_entry_size);
|
||||
return -EINVAL;
|
||||
return ENA_COM_INVAL;
|
||||
}
|
||||
|
||||
if (llq_info->desc_stride_ctrl == ENA_ADMIN_MULTIPLE_DESCS_PER_ENTRY)
|
||||
|
|
@ -776,7 +776,7 @@ static int ena_com_config_llq_info(struct ena_com_dev *ena_dev,
|
|||
} else {
|
||||
ena_trc_err(ena_dev, "Invalid descs_num_before_header, supported: 0x%x\n",
|
||||
supported_feat);
|
||||
return -EINVAL;
|
||||
return ENA_COM_INVAL;
|
||||
}
|
||||
|
||||
ena_trc_err(ena_dev, "Default llq num descs before header is not supported, performing fallback, default: 0x%x, supported: 0x%x, used: 0x%x\n",
|
||||
|
|
@ -1409,16 +1409,17 @@ int ena_com_execute_admin_command(struct ena_com_admin_queue *admin_queue,
|
|||
comp_ctx = ena_com_submit_admin_cmd(admin_queue, cmd, cmd_size,
|
||||
comp, comp_size);
|
||||
if (IS_ERR(comp_ctx)) {
|
||||
if (comp_ctx == ERR_PTR(ENA_COM_NO_DEVICE))
|
||||
ret = PTR_ERR(comp_ctx);
|
||||
if (ret == ENA_COM_NO_DEVICE)
|
||||
ena_trc_dbg(admin_queue->ena_dev,
|
||||
"Failed to submit command [%ld]\n",
|
||||
PTR_ERR(comp_ctx));
|
||||
"Failed to submit command [%d]\n",
|
||||
ret);
|
||||
else
|
||||
ena_trc_err(admin_queue->ena_dev,
|
||||
"Failed to submit command [%ld]\n",
|
||||
PTR_ERR(comp_ctx));
|
||||
"Failed to submit command [%d]\n",
|
||||
ret);
|
||||
|
||||
return (int)PTR_ERR(comp_ctx);
|
||||
return ret;
|
||||
}
|
||||
|
||||
ret = ena_com_wait_and_process_admin_cq(comp_ctx, admin_queue);
|
||||
|
|
@ -2034,7 +2035,7 @@ int ena_com_get_dev_attr_feat(struct ena_com_dev *ena_dev,
|
|||
return rc;
|
||||
|
||||
if (get_resp.u.max_queue_ext.version != ENA_FEATURE_MAX_QUEUE_EXT_VER)
|
||||
return -EINVAL;
|
||||
return ENA_COM_INVAL;
|
||||
|
||||
memcpy(&get_feat_ctx->max_queue_ext, &get_resp.u.max_queue_ext,
|
||||
sizeof(get_resp.u.max_queue_ext));
|
||||
|
|
@ -2363,7 +2364,7 @@ done:
|
|||
}
|
||||
#endif
|
||||
|
||||
int ena_com_set_dev_mtu(struct ena_com_dev *ena_dev, int mtu)
|
||||
int ena_com_set_dev_mtu(struct ena_com_dev *ena_dev, u32 mtu)
|
||||
{
|
||||
struct ena_com_admin_queue *admin_queue;
|
||||
struct ena_admin_set_feat_cmd cmd;
|
||||
|
|
@ -2381,7 +2382,7 @@ int ena_com_set_dev_mtu(struct ena_com_dev *ena_dev, int mtu)
|
|||
cmd.aq_common_descriptor.opcode = ENA_ADMIN_SET_FEATURE;
|
||||
cmd.aq_common_descriptor.flags = 0;
|
||||
cmd.feat_common.feature_id = ENA_ADMIN_MTU;
|
||||
cmd.u.mtu.mtu = (u32)mtu;
|
||||
cmd.u.mtu.mtu = mtu;
|
||||
|
||||
ret = ena_com_execute_admin_command(admin_queue,
|
||||
(struct ena_admin_aq_entry *)&cmd,
|
||||
|
|
@ -2792,7 +2793,7 @@ int ena_com_indirect_table_set(struct ena_com_dev *ena_dev)
|
|||
return ret;
|
||||
}
|
||||
|
||||
cmd.control_buffer.length = (u32)(1ULL << rss->tbl_log_size) *
|
||||
cmd.control_buffer.length = (1ULL << rss->tbl_log_size) *
|
||||
sizeof(struct ena_admin_rss_ind_table_entry);
|
||||
|
||||
ret = ena_com_execute_admin_command(admin_queue,
|
||||
|
|
@ -2814,7 +2815,7 @@ int ena_com_indirect_table_get(struct ena_com_dev *ena_dev, u32 *ind_tbl)
|
|||
u32 tbl_size;
|
||||
int i, rc;
|
||||
|
||||
tbl_size = (u32)(1ULL << rss->tbl_log_size) *
|
||||
tbl_size = (1ULL << rss->tbl_log_size) *
|
||||
sizeof(struct ena_admin_rss_ind_table_entry);
|
||||
|
||||
rc = ena_com_get_feature_ex(ena_dev, &get_resp,
|
||||
|
|
@ -3098,7 +3099,7 @@ int ena_com_config_dev_mode(struct ena_com_dev *ena_dev,
|
|||
|
||||
if (unlikely(ena_dev->tx_max_header_size == 0)) {
|
||||
ena_trc_err(ena_dev, "The size of the LLQ entry is smaller than needed\n");
|
||||
return -EINVAL;
|
||||
return ENA_COM_INVAL;
|
||||
}
|
||||
|
||||
ena_dev->tx_mem_queue_type = ENA_ADMIN_PLACEMENT_POLICY_DEV;
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
/*-
|
||||
* SPDX-License-Identifier: BSD-3-Clause
|
||||
*
|
||||
* Copyright (c) 2015-2020 Amazon.com, Inc. or its affiliates.
|
||||
* Copyright (c) 2015-2021 Amazon.com, Inc. or its affiliates.
|
||||
* All rights reserved.
|
||||
*
|
||||
* Redistribution and use in source and binary forms, with or without
|
||||
|
|
@ -51,8 +51,6 @@
|
|||
#define ADMIN_CQ_SIZE(depth) ((depth) * sizeof(struct ena_admin_acq_entry))
|
||||
#define ADMIN_AENQ_SIZE(depth) ((depth) * sizeof(struct ena_admin_aenq_entry))
|
||||
|
||||
#define ENA_CDESC_RING_SIZE_ALIGNMENT (1 << 12) /* 4K */
|
||||
|
||||
/*****************************************************************************/
|
||||
/*****************************************************************************/
|
||||
/* ENA adaptive interrupt moderation settings */
|
||||
|
|
@ -645,7 +643,7 @@ int ena_com_get_eni_stats(struct ena_com_dev *ena_dev,
|
|||
*
|
||||
* @return: 0 on Success and negative value otherwise.
|
||||
*/
|
||||
int ena_com_set_dev_mtu(struct ena_com_dev *ena_dev, int mtu);
|
||||
int ena_com_set_dev_mtu(struct ena_com_dev *ena_dev, u32 mtu);
|
||||
|
||||
/* ena_com_get_offload_settings - Retrieve the device offloads capabilities
|
||||
* @ena_dev: ENA communication layer struct
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
/*-
|
||||
* SPDX-License-Identifier: BSD-3-Clause
|
||||
*
|
||||
* Copyright (c) 2015-2020 Amazon.com, Inc. or its affiliates.
|
||||
* Copyright (c) 2015-2021 Amazon.com, Inc. or its affiliates.
|
||||
* All rights reserved.
|
||||
*
|
||||
* Redistribution and use in source and binary forms, with or without
|
||||
|
|
@ -853,7 +853,8 @@ enum ena_admin_os_type {
|
|||
ENA_ADMIN_OS_FREEBSD = 4,
|
||||
ENA_ADMIN_OS_IPXE = 5,
|
||||
ENA_ADMIN_OS_ESXI = 6,
|
||||
ENA_ADMIN_OS_GROUPS_NUM = 6,
|
||||
ENA_ADMIN_OS_MACOS = 7,
|
||||
ENA_ADMIN_OS_GROUPS_NUM = 7,
|
||||
};
|
||||
|
||||
struct ena_admin_host_info {
|
||||
|
|
@ -902,7 +903,9 @@ struct ena_admin_host_info {
|
|||
* 2 : interrupt_moderation
|
||||
* 3 : rx_buf_mirroring
|
||||
* 4 : rss_configurable_function_key
|
||||
* 31:5 : reserved
|
||||
* 5 : reserved
|
||||
* 6 : rx_page_reuse
|
||||
* 31:7 : reserved
|
||||
*/
|
||||
uint32_t driver_supported_features;
|
||||
};
|
||||
|
|
@ -1092,8 +1095,6 @@ enum ena_admin_aenq_group {
|
|||
};
|
||||
|
||||
enum ena_admin_aenq_notification_syndrome {
|
||||
ENA_ADMIN_SUSPEND = 0,
|
||||
ENA_ADMIN_RESUME = 1,
|
||||
ENA_ADMIN_UPDATE_HINTS = 2,
|
||||
};
|
||||
|
||||
|
|
@ -1228,6 +1229,8 @@ struct ena_admin_ena_mmio_req_read_less_resp {
|
|||
#define ENA_ADMIN_HOST_INFO_RX_BUF_MIRRORING_MASK BIT(3)
|
||||
#define ENA_ADMIN_HOST_INFO_RSS_CONFIGURABLE_FUNCTION_KEY_SHIFT 4
|
||||
#define ENA_ADMIN_HOST_INFO_RSS_CONFIGURABLE_FUNCTION_KEY_MASK BIT(4)
|
||||
#define ENA_ADMIN_HOST_INFO_RX_PAGE_REUSE_SHIFT 6
|
||||
#define ENA_ADMIN_HOST_INFO_RX_PAGE_REUSE_MASK BIT(6)
|
||||
|
||||
/* feature_rss_ind_table */
|
||||
#define ENA_ADMIN_FEATURE_RSS_IND_TABLE_ONE_ENTRY_UPDATE_MASK BIT(0)
|
||||
|
|
@ -1689,6 +1692,16 @@ static inline void set_ena_admin_host_info_rss_configurable_function_key(struct
|
|||
p->driver_supported_features |= (val << ENA_ADMIN_HOST_INFO_RSS_CONFIGURABLE_FUNCTION_KEY_SHIFT) & ENA_ADMIN_HOST_INFO_RSS_CONFIGURABLE_FUNCTION_KEY_MASK;
|
||||
}
|
||||
|
||||
static inline uint32_t get_ena_admin_host_info_rx_page_reuse(const struct ena_admin_host_info *p)
|
||||
{
|
||||
return (p->driver_supported_features & ENA_ADMIN_HOST_INFO_RX_PAGE_REUSE_MASK) >> ENA_ADMIN_HOST_INFO_RX_PAGE_REUSE_SHIFT;
|
||||
}
|
||||
|
||||
static inline void set_ena_admin_host_info_rx_page_reuse(struct ena_admin_host_info *p, uint32_t val)
|
||||
{
|
||||
p->driver_supported_features |= (val << ENA_ADMIN_HOST_INFO_RX_PAGE_REUSE_SHIFT) & ENA_ADMIN_HOST_INFO_RX_PAGE_REUSE_MASK;
|
||||
}
|
||||
|
||||
static inline uint8_t get_ena_admin_feature_rss_ind_table_one_entry_update(const struct ena_admin_feature_rss_ind_table *p)
|
||||
{
|
||||
return p->flags & ENA_ADMIN_FEATURE_RSS_IND_TABLE_ONE_ENTRY_UPDATE_MASK;
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
/*-
|
||||
* SPDX-License-Identifier: BSD-3-Clause
|
||||
*
|
||||
* Copyright (c) 2015-2020 Amazon.com, Inc. or its affiliates.
|
||||
* Copyright (c) 2015-2021 Amazon.com, Inc. or its affiliates.
|
||||
* All rights reserved.
|
||||
*
|
||||
* Redistribution and use in source and binary forms, with or without
|
||||
|
|
@ -30,5 +30,5 @@
|
|||
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
*/
|
||||
#define ENA_GEN_DATE "Fri Sep 18 17:09:00 IDT 2020"
|
||||
#define ENA_GEN_COMMIT "0f80d82"
|
||||
#define ENA_GEN_DATE "Tue Jan 19 12:45:09 STD 2021"
|
||||
#define ENA_GEN_COMMIT "f023ae8f"
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
/*-
|
||||
* SPDX-License-Identifier: BSD-3-Clause
|
||||
*
|
||||
* Copyright (c) 2015-2020 Amazon.com, Inc. or its affiliates.
|
||||
* Copyright (c) 2015-2021 Amazon.com, Inc. or its affiliates.
|
||||
* All rights reserved.
|
||||
*
|
||||
* Redistribution and use in source and binary forms, with or without
|
||||
|
|
@ -178,7 +178,7 @@ static int ena_com_close_bounce_buffer(struct ena_com_io_sq *io_sq)
|
|||
return ENA_COM_OK;
|
||||
|
||||
/* bounce buffer was used, so write it and get a new one */
|
||||
if (pkt_ctrl->idx) {
|
||||
if (likely(pkt_ctrl->idx)) {
|
||||
rc = ena_com_write_bounce_buffer_to_dev(io_sq,
|
||||
pkt_ctrl->curr_bounce_buf);
|
||||
if (unlikely(rc)) {
|
||||
|
|
|
|||
74
sys/contrib/ena-com/ena_fbsd_log.h
Normal file
74
sys/contrib/ena-com/ena_fbsd_log.h
Normal file
|
|
@ -0,0 +1,74 @@
|
|||
/*-
|
||||
* SPDX-License-Identifier: BSD-3-Clause
|
||||
*
|
||||
* Copyright (c) 2015-2021 Amazon.com, Inc. or its affiliates.
|
||||
* All rights reserved.
|
||||
*
|
||||
* Redistribution and use in source and binary forms, with or without
|
||||
* modification, are permitted provided that the following conditions
|
||||
* are met:
|
||||
*
|
||||
* * Redistributions of source code must retain the above copyright
|
||||
* notice, this list of conditions and the following disclaimer.
|
||||
* * Redistributions in binary form must reproduce the above copyright
|
||||
* notice, this list of conditions and the following disclaimer in
|
||||
* the documentation and/or other materials provided with the
|
||||
* distribution.
|
||||
* * Neither the name of copyright holder nor the names of its
|
||||
* contributors may be used to endorse or promote products derived
|
||||
* from this software without specific prior written permission.
|
||||
*
|
||||
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
*/
|
||||
|
||||
#ifndef ENA_FBSD_LOG_H
|
||||
#define ENA_FBSD_LOG_H
|
||||
|
||||
enum ena_log_t {
|
||||
ENA_ERR = 0,
|
||||
ENA_WARN,
|
||||
ENA_INFO,
|
||||
ENA_DBG,
|
||||
};
|
||||
|
||||
extern int ena_log_level;
|
||||
|
||||
#define ena_log(dev, level, fmt, args...) \
|
||||
do { \
|
||||
if (ENA_ ## level <= ena_log_level) \
|
||||
device_printf((dev), fmt, ##args); \
|
||||
} while (0)
|
||||
|
||||
#define ena_log_raw(level, fmt, args...) \
|
||||
do { \
|
||||
if (ENA_ ## level <= ena_log_level) \
|
||||
printf(fmt, ##args); \
|
||||
} while (0)
|
||||
|
||||
#define ena_log_unused(dev, level, fmt, args...) \
|
||||
do { \
|
||||
(void)(dev); \
|
||||
} while (0)
|
||||
|
||||
#ifdef ENA_LOG_IO_ENABLE
|
||||
#define ena_log_io(dev, level, fmt, args...) \
|
||||
ena_log((dev), level, fmt, ##args)
|
||||
#else
|
||||
#define ena_log_io(dev, level, fmt, args...) \
|
||||
ena_log_unused((dev), level, fmt, ##args)
|
||||
#endif
|
||||
|
||||
#define ena_log_nm(dev, level, fmt, args...) \
|
||||
ena_log((dev), level, "[nm] " fmt, ##args)
|
||||
|
||||
#endif /* !(ENA_FBSD_LOG_H) */
|
||||
|
|
@ -1,7 +1,7 @@
|
|||
/*-
|
||||
* SPDX-License-Identifier: BSD-3-Clause
|
||||
*
|
||||
* Copyright (c) 2015-2020 Amazon.com, Inc. or its affiliates.
|
||||
* Copyright (c) 2015-2021 Amazon.com, Inc. or its affiliates.
|
||||
* All rights reserved.
|
||||
*
|
||||
* Redistribution and use in source and binary forms, with or without
|
||||
|
|
@ -91,22 +91,12 @@ __FBSDID("$FreeBSD$");
|
|||
#include <dev/pci/pcivar.h>
|
||||
#include <dev/pci/pcireg.h>
|
||||
|
||||
#include "ena_fbsd_log.h"
|
||||
|
||||
extern struct ena_bus_space ebs;
|
||||
|
||||
/* Levels */
|
||||
#define ENA_ALERT (1 << 0) /* Alerts are providing more error info. */
|
||||
#define ENA_WARNING (1 << 1) /* Driver output is more error sensitive. */
|
||||
#define ENA_INFO (1 << 2) /* Provides additional driver info. */
|
||||
#define ENA_DBG (1 << 3) /* Driver output for debugging. */
|
||||
/* Detailed info that will be printed with ENA_INFO or ENA_DEBUG flag. */
|
||||
#define ENA_TXPTH (1 << 4) /* Allows TX path tracing. */
|
||||
#define ENA_RXPTH (1 << 5) /* Allows RX path tracing. */
|
||||
#define ENA_RSC (1 << 6) /* Goes with TXPTH or RXPTH, free/alloc res. */
|
||||
#define ENA_IOQ (1 << 7) /* Detailed info about IO queues. */
|
||||
#define ENA_ADMQ (1 << 8) /* Detailed info about admin queue. */
|
||||
#define ENA_NETMAP (1 << 9) /* Detailed info about netmap. */
|
||||
|
||||
#define DEFAULT_ALLOC_ALIGNMENT 8
|
||||
#define ENA_CDESC_RING_SIZE_ALIGNMENT (1 << 12) /* 4K */
|
||||
|
||||
extern int ena_log_level;
|
||||
|
||||
|
|
@ -116,27 +106,18 @@ extern int ena_log_level;
|
|||
(type *)((uintptr_t)__p - offsetof(type, member)); \
|
||||
})
|
||||
|
||||
#define ena_trace_raw(ctx, level, fmt, args...) \
|
||||
do { \
|
||||
((void)(ctx)); \
|
||||
if (((level) & ena_log_level) != (level)) \
|
||||
break; \
|
||||
printf(fmt, ##args); \
|
||||
} while (0)
|
||||
|
||||
#define ena_trace(ctx, level, fmt, args...) \
|
||||
ena_trace_raw(ctx, level, "%s() [TID:%d]: " \
|
||||
ena_log((ctx)->dmadev, level, "%s() [TID:%d]: " \
|
||||
fmt, __func__, curthread->td_tid, ##args)
|
||||
|
||||
|
||||
#define ena_trc_dbg(ctx, format, arg...) \
|
||||
ena_trace(ctx, ENA_DBG, format, ##arg)
|
||||
ena_trace(ctx, DBG, format, ##arg)
|
||||
#define ena_trc_info(ctx, format, arg...) \
|
||||
ena_trace(ctx, ENA_INFO, format, ##arg)
|
||||
ena_trace(ctx, INFO, format, ##arg)
|
||||
#define ena_trc_warn(ctx, format, arg...) \
|
||||
ena_trace(ctx, ENA_WARNING, format, ##arg)
|
||||
ena_trace(ctx, WARN, format, ##arg)
|
||||
#define ena_trc_err(ctx, format, arg...) \
|
||||
ena_trace(ctx, ENA_ALERT, format, ##arg)
|
||||
ena_trace(ctx, ERR, format, ##arg)
|
||||
|
||||
#define unlikely(x) __predict_false(!!(x))
|
||||
#define likely(x) __predict_true(!!(x))
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
|
|
@ -40,8 +40,8 @@
|
|||
#include "ena-com/ena_eth_com.h"
|
||||
|
||||
#define DRV_MODULE_VER_MAJOR 2
|
||||
#define DRV_MODULE_VER_MINOR 3
|
||||
#define DRV_MODULE_VER_SUBMINOR 1
|
||||
#define DRV_MODULE_VER_MINOR 4
|
||||
#define DRV_MODULE_VER_SUBMINOR 0
|
||||
|
||||
#define DRV_MODULE_NAME "ena"
|
||||
|
||||
|
|
@ -215,6 +215,7 @@ struct ena_que {
|
|||
|
||||
uint32_t id;
|
||||
int cpu;
|
||||
struct sysctl_oid *oid;
|
||||
};
|
||||
|
||||
struct ena_calc_queue_size_ctx {
|
||||
|
|
|
|||
|
|
@ -76,7 +76,7 @@ ena_cleanup(void *arg, int pending)
|
|||
if (unlikely((if_getdrvflags(ifp) & IFF_DRV_RUNNING) == 0))
|
||||
return;
|
||||
|
||||
ena_trace(NULL, ENA_DBG, "MSI-X TX/RX routine\n");
|
||||
ena_log_io(adapter->pdev, DBG, "MSI-X TX/RX routine\n");
|
||||
|
||||
tx_ring = que->tx_ring;
|
||||
rx_ring = que->rx_ring;
|
||||
|
|
@ -194,11 +194,11 @@ validate_tx_req_id(struct ena_ring *tx_ring, uint16_t req_id)
|
|||
tx_info = &tx_ring->tx_buffer_info[req_id];
|
||||
if (tx_info->mbuf != NULL)
|
||||
return (0);
|
||||
device_printf(adapter->pdev,
|
||||
ena_log(adapter->pdev, ERR,
|
||||
"tx_info doesn't have valid mbuf\n");
|
||||
}
|
||||
|
||||
device_printf(adapter->pdev, "Invalid req_id: %hu\n", req_id);
|
||||
ena_log(adapter->pdev, ERR, "Invalid req_id: %hu\n", req_id);
|
||||
counter_u64_add(tx_ring->tx_stats.bad_req_id, 1);
|
||||
|
||||
/* Trigger device reset */
|
||||
|
|
@ -267,7 +267,7 @@ ena_tx_cleanup(struct ena_ring *tx_ring)
|
|||
bus_dmamap_unload(adapter->tx_buf_tag,
|
||||
tx_info->dmamap);
|
||||
|
||||
ena_trace(NULL, ENA_DBG | ENA_TXPTH, "tx: q %d mbuf %p completed\n",
|
||||
ena_log_io(adapter->pdev, DBG, "tx: q %d mbuf %p completed\n",
|
||||
tx_ring->qid, mbuf);
|
||||
|
||||
m_freem(mbuf);
|
||||
|
|
@ -292,8 +292,8 @@ ena_tx_cleanup(struct ena_ring *tx_ring)
|
|||
|
||||
work_done = TX_BUDGET - budget;
|
||||
|
||||
ena_trace(NULL, ENA_DBG | ENA_TXPTH, "tx: q %d done. total pkts: %d\n",
|
||||
tx_ring->qid, work_done);
|
||||
ena_log_io(adapter->pdev, DBG, "tx: q %d done. total pkts: %d\n",
|
||||
tx_ring->qid, work_done);
|
||||
|
||||
/* If there is still something to commit update ring state */
|
||||
if (likely(commit != TX_COMMIT)) {
|
||||
|
|
@ -408,22 +408,24 @@ ena_rx_mbuf(struct ena_ring *rx_ring, struct ena_com_rx_buf_info *ena_bufs,
|
|||
struct mbuf *mbuf;
|
||||
struct ena_rx_buffer *rx_info;
|
||||
struct ena_adapter *adapter;
|
||||
device_t pdev;
|
||||
unsigned int descs = ena_rx_ctx->descs;
|
||||
uint16_t ntc, len, req_id, buf = 0;
|
||||
|
||||
ntc = *next_to_clean;
|
||||
adapter = rx_ring->adapter;
|
||||
pdev = adapter->pdev;
|
||||
|
||||
len = ena_bufs[buf].len;
|
||||
req_id = ena_bufs[buf].req_id;
|
||||
rx_info = &rx_ring->rx_buffer_info[req_id];
|
||||
if (unlikely(rx_info->mbuf == NULL)) {
|
||||
device_printf(adapter->pdev, "NULL mbuf in rx_info");
|
||||
ena_log(pdev, ERR, "NULL mbuf in rx_info");
|
||||
return (NULL);
|
||||
}
|
||||
|
||||
ena_trace(NULL, ENA_DBG | ENA_RXPTH, "rx_info %p, mbuf %p, paddr %jx\n",
|
||||
rx_info, rx_info->mbuf, (uintmax_t)rx_info->ena_buf.paddr);
|
||||
ena_log_io(pdev, DBG, "rx_info %p, mbuf %p, paddr %jx\n", rx_info,
|
||||
rx_info->mbuf, (uintmax_t)rx_info->ena_buf.paddr);
|
||||
|
||||
bus_dmamap_sync(adapter->rx_buf_tag, rx_info->map,
|
||||
BUS_DMASYNC_POSTREAD);
|
||||
|
|
@ -431,17 +433,16 @@ ena_rx_mbuf(struct ena_ring *rx_ring, struct ena_com_rx_buf_info *ena_bufs,
|
|||
mbuf->m_flags |= M_PKTHDR;
|
||||
mbuf->m_pkthdr.len = len;
|
||||
mbuf->m_len = len;
|
||||
// Only for the first segment the data starts at specific offset
|
||||
/* Only for the first segment the data starts at specific offset */
|
||||
mbuf->m_data = mtodo(mbuf, ena_rx_ctx->pkt_offset);
|
||||
ena_trace(NULL, ENA_DBG | ENA_RXPTH,
|
||||
"Mbuf data offset=%u\n", ena_rx_ctx->pkt_offset);
|
||||
ena_log_io(pdev, DBG, "Mbuf data offset=%u\n", ena_rx_ctx->pkt_offset);
|
||||
mbuf->m_pkthdr.rcvif = rx_ring->que->adapter->ifp;
|
||||
|
||||
/* Fill mbuf with hash key and it's interpretation for optimization */
|
||||
ena_rx_hash_mbuf(rx_ring, ena_rx_ctx, mbuf);
|
||||
|
||||
ena_trace(NULL, ENA_DBG | ENA_RXPTH, "rx mbuf 0x%p, flags=0x%x, len: %d\n",
|
||||
mbuf, mbuf->m_flags, mbuf->m_pkthdr.len);
|
||||
ena_log_io(pdev, DBG, "rx mbuf 0x%p, flags=0x%x, len: %d\n", mbuf,
|
||||
mbuf->m_flags, mbuf->m_pkthdr.len);
|
||||
|
||||
/* DMA address is not needed anymore, unmap it */
|
||||
bus_dmamap_unload(rx_ring->adapter->rx_buf_tag, rx_info->map);
|
||||
|
|
@ -461,7 +462,7 @@ ena_rx_mbuf(struct ena_ring *rx_ring, struct ena_com_rx_buf_info *ena_bufs,
|
|||
rx_info = &rx_ring->rx_buffer_info[req_id];
|
||||
|
||||
if (unlikely(rx_info->mbuf == NULL)) {
|
||||
device_printf(adapter->pdev, "NULL mbuf in rx_info");
|
||||
ena_log(pdev, ERR, "NULL mbuf in rx_info");
|
||||
/*
|
||||
* If one of the required mbufs was not allocated yet,
|
||||
* we can break there.
|
||||
|
|
@ -480,12 +481,12 @@ ena_rx_mbuf(struct ena_ring *rx_ring, struct ena_com_rx_buf_info *ena_bufs,
|
|||
BUS_DMASYNC_POSTREAD);
|
||||
if (unlikely(m_append(mbuf, len, rx_info->mbuf->m_data) == 0)) {
|
||||
counter_u64_add(rx_ring->rx_stats.mbuf_alloc_fail, 1);
|
||||
ena_trace(NULL, ENA_WARNING, "Failed to append Rx mbuf %p\n",
|
||||
ena_log_io(pdev, WARN, "Failed to append Rx mbuf %p\n",
|
||||
mbuf);
|
||||
}
|
||||
|
||||
ena_trace(NULL, ENA_DBG | ENA_RXPTH,
|
||||
"rx mbuf updated. len %d\n", mbuf->m_pkthdr.len);
|
||||
ena_log_io(pdev, DBG, "rx mbuf updated. len %d\n",
|
||||
mbuf->m_pkthdr.len);
|
||||
|
||||
/* Free already appended mbuf, it won't be useful anymore */
|
||||
bus_dmamap_unload(rx_ring->adapter->rx_buf_tag, rx_info->map);
|
||||
|
|
@ -508,6 +509,7 @@ static inline void
|
|||
ena_rx_checksum(struct ena_ring *rx_ring, struct ena_com_rx_ctx *ena_rx_ctx,
|
||||
struct mbuf *mbuf)
|
||||
{
|
||||
device_t pdev = rx_ring->adapter->pdev;
|
||||
|
||||
/* if IP and error */
|
||||
if (unlikely((ena_rx_ctx->l3_proto == ENA_ETH_IO_L3_PROTO_IPV4) &&
|
||||
|
|
@ -515,7 +517,7 @@ ena_rx_checksum(struct ena_ring *rx_ring, struct ena_com_rx_ctx *ena_rx_ctx,
|
|||
/* ipv4 checksum error */
|
||||
mbuf->m_pkthdr.csum_flags = 0;
|
||||
counter_u64_add(rx_ring->rx_stats.bad_csum, 1);
|
||||
ena_trace(NULL, ENA_DBG, "RX IPv4 header checksum error\n");
|
||||
ena_log_io(pdev, DBG, "RX IPv4 header checksum error\n");
|
||||
return;
|
||||
}
|
||||
|
||||
|
|
@ -526,7 +528,7 @@ ena_rx_checksum(struct ena_ring *rx_ring, struct ena_com_rx_ctx *ena_rx_ctx,
|
|||
/* TCP/UDP checksum error */
|
||||
mbuf->m_pkthdr.csum_flags = 0;
|
||||
counter_u64_add(rx_ring->rx_stats.bad_csum, 1);
|
||||
ena_trace(NULL, ENA_DBG, "RX L4 checksum error\n");
|
||||
ena_log_io(pdev, DBG, "RX L4 checksum error\n");
|
||||
} else {
|
||||
mbuf->m_pkthdr.csum_flags = CSUM_IP_CHECKED;
|
||||
mbuf->m_pkthdr.csum_flags |= CSUM_IP_VALID;
|
||||
|
|
@ -542,6 +544,7 @@ static int
|
|||
ena_rx_cleanup(struct ena_ring *rx_ring)
|
||||
{
|
||||
struct ena_adapter *adapter;
|
||||
device_t pdev;
|
||||
struct mbuf *mbuf;
|
||||
struct ena_com_rx_ctx ena_rx_ctx;
|
||||
struct ena_com_io_cq* io_cq;
|
||||
|
|
@ -561,6 +564,7 @@ ena_rx_cleanup(struct ena_ring *rx_ring)
|
|||
#endif /* DEV_NETMAP */
|
||||
|
||||
adapter = rx_ring->que->adapter;
|
||||
pdev = adapter->pdev;
|
||||
ifp = adapter->ifp;
|
||||
qid = rx_ring->que->id;
|
||||
ena_qid = ENA_IO_RXQ_IDX(qid);
|
||||
|
|
@ -573,7 +577,7 @@ ena_rx_cleanup(struct ena_ring *rx_ring)
|
|||
return (0);
|
||||
#endif /* DEV_NETMAP */
|
||||
|
||||
ena_trace(NULL, ENA_DBG, "rx: qid %d\n", qid);
|
||||
ena_log_io(pdev, DBG, "rx: qid %d\n", qid);
|
||||
|
||||
do {
|
||||
ena_rx_ctx.ena_bufs = rx_ring->ena_bufs;
|
||||
|
|
@ -601,7 +605,7 @@ ena_rx_cleanup(struct ena_ring *rx_ring)
|
|||
if (unlikely(ena_rx_ctx.descs == 0))
|
||||
break;
|
||||
|
||||
ena_trace(NULL, ENA_DBG | ENA_RXPTH, "rx: q %d got packet from ena. "
|
||||
ena_log_io(pdev, DBG, "rx: q %d got packet from ena. "
|
||||
"descs #: %d l3 proto %d l4 proto %d hash: %x\n",
|
||||
rx_ring->qid, ena_rx_ctx.descs, ena_rx_ctx.l3_proto,
|
||||
ena_rx_ctx.l4_proto, ena_rx_ctx.hash);
|
||||
|
|
@ -654,8 +658,8 @@ ena_rx_cleanup(struct ena_ring *rx_ring)
|
|||
do_if_input = 0;
|
||||
}
|
||||
if (do_if_input != 0) {
|
||||
ena_trace(NULL, ENA_DBG | ENA_RXPTH,
|
||||
"calling if_input() with mbuf %p\n", mbuf);
|
||||
ena_log_io(pdev, DBG, "calling if_input() with mbuf %p\n",
|
||||
mbuf);
|
||||
(*ifp->if_input)(ifp, mbuf);
|
||||
}
|
||||
|
||||
|
|
@ -835,7 +839,7 @@ ena_tx_map_mbuf(struct ena_ring *tx_ring, struct ena_tx_buffer *tx_info,
|
|||
rc = bus_dmamap_load_mbuf_sg(adapter->tx_buf_tag, tx_info->dmamap, mbuf,
|
||||
segs, &nsegs, BUS_DMA_NOWAIT);
|
||||
if (unlikely((rc != 0) || (nsegs == 0))) {
|
||||
ena_trace(NULL, ENA_WARNING,
|
||||
ena_log_io(adapter->pdev, WARN,
|
||||
"dmamap load failed! err: %d nsegs: %d\n", rc, nsegs);
|
||||
goto dma_error;
|
||||
}
|
||||
|
|
@ -867,9 +871,8 @@ ena_tx_map_mbuf(struct ena_ring *tx_ring, struct ena_tx_buffer *tx_info,
|
|||
counter_u64_add(tx_ring->tx_stats.llq_buffer_copy, 1);
|
||||
}
|
||||
|
||||
ena_trace(NULL, ENA_DBG | ENA_TXPTH,
|
||||
"mbuf: %p header_buf->vaddr: %p push_len: %d\n",
|
||||
mbuf, *push_hdr, *header_len);
|
||||
ena_log_io(adapter->pdev, DBG, "mbuf: %p ""header_buf->vaddr: %p "
|
||||
"push_len: %d\n", mbuf, *push_hdr, *header_len);
|
||||
|
||||
/* If packet is fitted in LLQ header, no need for DMA segments. */
|
||||
if (mbuf->m_pkthdr.len <= tx_ring->tx_max_header_size) {
|
||||
|
|
@ -926,6 +929,7 @@ static int
|
|||
ena_xmit_mbuf(struct ena_ring *tx_ring, struct mbuf **mbuf)
|
||||
{
|
||||
struct ena_adapter *adapter;
|
||||
device_t pdev;
|
||||
struct ena_tx_buffer *tx_info;
|
||||
struct ena_com_tx_ctx ena_tx_ctx;
|
||||
struct ena_com_dev *ena_dev;
|
||||
|
|
@ -940,17 +944,18 @@ ena_xmit_mbuf(struct ena_ring *tx_ring, struct mbuf **mbuf)
|
|||
|
||||
ena_qid = ENA_IO_TXQ_IDX(tx_ring->que->id);
|
||||
adapter = tx_ring->que->adapter;
|
||||
pdev = adapter->pdev;
|
||||
ena_dev = adapter->ena_dev;
|
||||
io_sq = &ena_dev->io_sq_queues[ena_qid];
|
||||
|
||||
rc = ena_check_and_collapse_mbuf(tx_ring, mbuf);
|
||||
if (unlikely(rc != 0)) {
|
||||
ena_trace(NULL, ENA_WARNING,
|
||||
"Failed to collapse mbuf! err: %d\n", rc);
|
||||
ena_log_io(pdev, WARN, "Failed to collapse mbuf! err: %d\n",
|
||||
rc);
|
||||
return (rc);
|
||||
}
|
||||
|
||||
ena_trace(NULL, ENA_DBG | ENA_TXPTH, "Tx: %d bytes\n", (*mbuf)->m_pkthdr.len);
|
||||
ena_log_io(pdev, DBG, "Tx: %d bytes\n", (*mbuf)->m_pkthdr.len);
|
||||
|
||||
next_to_use = tx_ring->next_to_use;
|
||||
req_id = tx_ring->free_tx_ids[next_to_use];
|
||||
|
|
@ -959,7 +964,7 @@ ena_xmit_mbuf(struct ena_ring *tx_ring, struct mbuf **mbuf)
|
|||
|
||||
rc = ena_tx_map_mbuf(tx_ring, tx_info, *mbuf, &push_hdr, &header_len);
|
||||
if (unlikely(rc != 0)) {
|
||||
ena_trace(NULL, ENA_WARNING, "Failed to map TX mbuf\n");
|
||||
ena_log_io(pdev, WARN, "Failed to map TX mbuf\n");
|
||||
return (rc);
|
||||
}
|
||||
memset(&ena_tx_ctx, 0x0, sizeof(struct ena_com_tx_ctx));
|
||||
|
|
@ -974,7 +979,7 @@ ena_xmit_mbuf(struct ena_ring *tx_ring, struct mbuf **mbuf)
|
|||
|
||||
if (tx_ring->acum_pkts == DB_THRESHOLD ||
|
||||
ena_com_is_doorbell_needed(tx_ring->ena_com_io_sq, &ena_tx_ctx)) {
|
||||
ena_trace(NULL, ENA_DBG | ENA_TXPTH,
|
||||
ena_log_io(pdev, DBG,
|
||||
"llq tx max burst size of queue %d achieved, writing doorbell to send burst\n",
|
||||
tx_ring->que->id);
|
||||
ena_com_write_sq_doorbell(tx_ring->ena_com_io_sq);
|
||||
|
|
@ -986,11 +991,10 @@ ena_xmit_mbuf(struct ena_ring *tx_ring, struct mbuf **mbuf)
|
|||
rc = ena_com_prepare_tx(io_sq, &ena_tx_ctx, &nb_hw_desc);
|
||||
if (unlikely(rc != 0)) {
|
||||
if (likely(rc == ENA_COM_NO_MEM)) {
|
||||
ena_trace(NULL, ENA_DBG | ENA_TXPTH,
|
||||
"tx ring[%d] if out of space\n", tx_ring->que->id);
|
||||
ena_log_io(pdev, DBG, "tx ring[%d] is out of space\n",
|
||||
tx_ring->que->id);
|
||||
} else {
|
||||
device_printf(adapter->pdev,
|
||||
"failed to prepare tx bufs\n");
|
||||
ena_log(pdev, ERR, "failed to prepare tx bufs\n");
|
||||
}
|
||||
counter_u64_add(tx_ring->tx_stats.prepare_ctx_err, 1);
|
||||
goto dma_error;
|
||||
|
|
@ -1019,8 +1023,7 @@ ena_xmit_mbuf(struct ena_ring *tx_ring, struct mbuf **mbuf)
|
|||
*/
|
||||
if (unlikely(!ena_com_sq_have_enough_space(tx_ring->ena_com_io_sq,
|
||||
adapter->max_tx_sgl_size + 2))) {
|
||||
ena_trace(NULL, ENA_DBG | ENA_TXPTH, "Stop queue %d\n",
|
||||
tx_ring->que->id);
|
||||
ena_log_io(pdev, DBG, "Stop queue %d\n", tx_ring->que->id);
|
||||
|
||||
tx_ring->running = false;
|
||||
counter_u64_add(tx_ring->tx_stats.queue_stop, 1);
|
||||
|
|
@ -1072,8 +1075,8 @@ ena_start_xmit(struct ena_ring *tx_ring)
|
|||
io_sq = &adapter->ena_dev->io_sq_queues[ena_qid];
|
||||
|
||||
while ((mbuf = drbr_peek(adapter->ifp, tx_ring->br)) != NULL) {
|
||||
ena_trace(NULL, ENA_DBG | ENA_TXPTH, "\ndequeued mbuf %p with flags %#x and"
|
||||
" header csum flags %#jx\n",
|
||||
ena_log_io(adapter->pdev, DBG,
|
||||
"\ndequeued mbuf %p with flags %#x and header csum flags %#jx\n",
|
||||
mbuf, mbuf->m_flags, (uint64_t)mbuf->m_pkthdr.csum_flags);
|
||||
|
||||
if (unlikely(!tx_ring->running)) {
|
||||
|
|
|
|||
|
|
@ -88,7 +88,7 @@ ena_netmap_attach(struct ena_adapter *adapter)
|
|||
{
|
||||
struct netmap_adapter na;
|
||||
|
||||
ena_trace(NULL, ENA_NETMAP, "netmap attach\n");
|
||||
ena_log_nm(adapter->pdev, INFO, "netmap attach\n");
|
||||
|
||||
bzero(&na, sizeof(na));
|
||||
na.na_flags = NAF_MOREFRAG;
|
||||
|
|
@ -126,31 +126,31 @@ ena_netmap_alloc_rx_slot(struct ena_adapter *adapter,
|
|||
nm_i = kring->nr_hwcur;
|
||||
head = kring->rhead;
|
||||
|
||||
ena_trace(NULL, ENA_NETMAP | ENA_DBG, "nr_hwcur: %d, nr_hwtail: %d, "
|
||||
ena_log_nm(adapter->pdev, DBG, "nr_hwcur: %d, nr_hwtail: %d, "
|
||||
"rhead: %d, rcur: %d, rtail: %d\n", kring->nr_hwcur,
|
||||
kring->nr_hwtail, kring->rhead, kring->rcur, kring->rtail);
|
||||
|
||||
if ((nm_i == head) && rx_ring->initialized) {
|
||||
ena_trace(NULL, ENA_NETMAP, "No free slots in netmap ring\n");
|
||||
ena_log_nm(adapter->pdev, ERR, "No free slots in netmap ring\n");
|
||||
return (ENOMEM);
|
||||
}
|
||||
|
||||
ring = kring->ring;
|
||||
if (ring == NULL) {
|
||||
device_printf(adapter->pdev, "Rx ring %d is NULL\n", qid);
|
||||
ena_log_nm(adapter->pdev, ERR, "Rx ring %d is NULL\n", qid);
|
||||
return (EFAULT);
|
||||
}
|
||||
slot = &ring->slot[nm_i];
|
||||
|
||||
addr = PNMB(na, slot, &paddr);
|
||||
if (addr == NETMAP_BUF_BASE(na)) {
|
||||
device_printf(adapter->pdev, "Bad buff in slot\n");
|
||||
ena_log_nm(adapter->pdev, ERR, "Bad buff in slot\n");
|
||||
return (EFAULT);
|
||||
}
|
||||
|
||||
rc = netmap_load_map(na, adapter->rx_buf_tag, rx_info->map, addr);
|
||||
if (rc != 0) {
|
||||
ena_trace(NULL, ENA_WARNING, "DMA mapping error\n");
|
||||
ena_log_nm(adapter->pdev, WARN, "DMA mapping error\n");
|
||||
return (rc);
|
||||
}
|
||||
bus_dmamap_sync(adapter->rx_buf_tag, rx_info->map, BUS_DMASYNC_PREREAD);
|
||||
|
|
@ -179,19 +179,19 @@ ena_netmap_free_rx_slot(struct ena_adapter *adapter,
|
|||
|
||||
na = NA(adapter->ifp);
|
||||
if (na == NULL) {
|
||||
device_printf(adapter->pdev, "netmap adapter is NULL\n");
|
||||
ena_log_nm(adapter->pdev, ERR, "netmap adapter is NULL\n");
|
||||
return;
|
||||
}
|
||||
|
||||
if (na->rx_rings == NULL) {
|
||||
device_printf(adapter->pdev, "netmap rings are NULL\n");
|
||||
ena_log_nm(adapter->pdev, ERR, "netmap rings are NULL\n");
|
||||
return;
|
||||
}
|
||||
|
||||
qid = rx_ring->qid;
|
||||
kring = na->rx_rings[qid];
|
||||
if (kring == NULL) {
|
||||
device_printf(adapter->pdev,
|
||||
ena_log_nm(adapter->pdev, ERR,
|
||||
"netmap kernel ring %d is NULL\n", qid);
|
||||
return;
|
||||
}
|
||||
|
|
@ -210,7 +210,7 @@ ena_netmap_free_rx_slot(struct ena_adapter *adapter,
|
|||
|
||||
slot = &kring->ring->slot[nm_i];
|
||||
|
||||
ENA_WARN(slot->buf_idx != 0, NULL, "Overwrite slot buf\n");
|
||||
ENA_WARN(slot->buf_idx != 0, adapter->ena_dev, "Overwrite slot buf\n");
|
||||
slot->buf_idx = rx_info->netmap_buf_idx;
|
||||
slot->flags = NS_BUF_CHANGED;
|
||||
|
||||
|
|
@ -252,7 +252,7 @@ ena_netmap_reset_ring(struct ena_adapter *adapter, int qid, enum txrx x)
|
|||
return;
|
||||
|
||||
netmap_reset(NA(adapter->ifp), x, qid, 0);
|
||||
ena_trace(NULL, ENA_NETMAP, "%s ring %d is in netmap mode\n",
|
||||
ena_log_nm(adapter->pdev, INFO, "%s ring %d is in netmap mode\n",
|
||||
(x == NR_TX) ? "Tx" : "Rx", qid);
|
||||
}
|
||||
|
||||
|
|
@ -273,6 +273,7 @@ ena_netmap_reg(struct netmap_adapter *na, int onoff)
|
|||
{
|
||||
struct ifnet *ifp = na->ifp;
|
||||
struct ena_adapter* adapter = ifp->if_softc;
|
||||
device_t pdev = adapter->pdev;
|
||||
struct netmap_kring *kring;
|
||||
enum txrx t;
|
||||
int rc, i;
|
||||
|
|
@ -282,7 +283,7 @@ ena_netmap_reg(struct netmap_adapter *na, int onoff)
|
|||
ena_down(adapter);
|
||||
|
||||
if (onoff) {
|
||||
ena_trace(NULL, ENA_NETMAP, "netmap on\n");
|
||||
ena_log_nm(pdev, INFO, "netmap on\n");
|
||||
for_rx_tx(t) {
|
||||
for (i = 0; i <= nma_get_nrings(na, t); i++) {
|
||||
kring = NMR(na, t)[i];
|
||||
|
|
@ -293,7 +294,7 @@ ena_netmap_reg(struct netmap_adapter *na, int onoff)
|
|||
}
|
||||
nm_set_native_flags(na);
|
||||
} else {
|
||||
ena_trace(NULL, ENA_NETMAP, "netmap off\n");
|
||||
ena_log_nm(pdev, INFO, "netmap off\n");
|
||||
nm_clear_native_flags(na);
|
||||
for_rx_tx(t) {
|
||||
for (i = 0; i <= nma_get_nrings(na, t); i++) {
|
||||
|
|
@ -307,7 +308,7 @@ ena_netmap_reg(struct netmap_adapter *na, int onoff)
|
|||
|
||||
rc = ena_up(adapter);
|
||||
if (rc != 0) {
|
||||
ena_trace(NULL, ENA_WARNING, "ena_up failed with rc=%d\n", rc);
|
||||
ena_log_nm(pdev, WARN, "ena_up failed with rc=%d\n", rc);
|
||||
adapter->reset_reason = ENA_REGS_RESET_DRIVER_INVALID_STATE;
|
||||
nm_clear_native_flags(na);
|
||||
ena_destroy_device(adapter, false);
|
||||
|
|
@ -401,7 +402,7 @@ ena_netmap_tx_frame(struct ena_netmap_ctx *ctx)
|
|||
|
||||
adapter = ctx->adapter;
|
||||
if (ena_netmap_count_slots(ctx) > adapter->max_tx_sgl_size) {
|
||||
ena_trace(NULL, ENA_WARNING, "Too many slots per packet\n");
|
||||
ena_log_nm(adapter->pdev, WARN, "Too many slots per packet\n");
|
||||
return (EINVAL);
|
||||
}
|
||||
|
||||
|
|
@ -415,7 +416,7 @@ ena_netmap_tx_frame(struct ena_netmap_ctx *ctx)
|
|||
rc = ena_netmap_tx_map_slots(ctx, tx_info, &push_hdr, &header_len,
|
||||
&packet_len);
|
||||
if (unlikely(rc != 0)) {
|
||||
device_printf(adapter->pdev, "Failed to map Tx slot\n");
|
||||
ena_log_nm(adapter->pdev, ERR, "Failed to map Tx slot\n");
|
||||
return (rc);
|
||||
}
|
||||
|
||||
|
|
@ -438,10 +439,10 @@ ena_netmap_tx_frame(struct ena_netmap_ctx *ctx)
|
|||
rc = ena_com_prepare_tx(ctx->io_sq, &ena_tx_ctx, &nb_hw_desc);
|
||||
if (unlikely(rc != 0)) {
|
||||
if (likely(rc == ENA_COM_NO_MEM)) {
|
||||
ena_trace(NULL, ENA_NETMAP | ENA_DBG | ENA_TXPTH,
|
||||
ena_log_nm(adapter->pdev, DBG,
|
||||
"Tx ring[%d] is out of space\n", tx_ring->que->id);
|
||||
} else {
|
||||
device_printf(adapter->pdev,
|
||||
ena_log_nm(adapter->pdev, ERR,
|
||||
"Failed to prepare Tx bufs\n");
|
||||
}
|
||||
counter_u64_add(tx_ring->tx_stats.prepare_ctx_err, 1);
|
||||
|
|
@ -528,17 +529,20 @@ static int
|
|||
ena_netmap_map_single_slot(struct netmap_adapter *na, struct netmap_slot *slot,
|
||||
bus_dma_tag_t dmatag, bus_dmamap_t dmamap, void **vaddr, uint64_t *paddr)
|
||||
{
|
||||
device_t pdev;
|
||||
int rc;
|
||||
|
||||
pdev = ((struct ena_adapter *)na->ifp->if_softc)->pdev;
|
||||
|
||||
*vaddr = PNMB(na, slot, paddr);
|
||||
if (unlikely(vaddr == NULL)) {
|
||||
ena_trace(NULL, ENA_ALERT, "Slot address is NULL\n");
|
||||
ena_log_nm(pdev, ERR, "Slot address is NULL\n");
|
||||
return (EINVAL);
|
||||
}
|
||||
|
||||
rc = netmap_load_map(na, dmatag, dmamap, *vaddr);
|
||||
if (unlikely(rc != 0)) {
|
||||
ena_trace(NULL, ENA_ALERT, "Failed to map slot %d for DMA\n",
|
||||
ena_log_nm(pdev, ERR, "Failed to map slot %d for DMA\n",
|
||||
slot->buf_idx);
|
||||
return (EINVAL);
|
||||
}
|
||||
|
|
@ -599,7 +603,7 @@ ena_netmap_tx_map_slots(struct ena_netmap_ctx *ctx,
|
|||
if (likely(push_len <= slot_head_len)) {
|
||||
*push_hdr = NMB(ctx->na, slot);
|
||||
if (unlikely(push_hdr == NULL)) {
|
||||
device_printf(adapter->pdev,
|
||||
ena_log_nm(adapter->pdev, ERR,
|
||||
"Slot vaddress is NULL\n");
|
||||
return (EINVAL);
|
||||
}
|
||||
|
|
@ -615,7 +619,7 @@ ena_netmap_tx_map_slots(struct ena_netmap_ctx *ctx,
|
|||
push_len,
|
||||
tx_ring->push_buf_intermediate_buf);
|
||||
if (unlikely(rc)) {
|
||||
device_printf(adapter->pdev,
|
||||
ena_log_nm(adapter->pdev, ERR,
|
||||
"Failed to copy data from slots to push_buf\n");
|
||||
return (EINVAL);
|
||||
}
|
||||
|
|
@ -626,7 +630,7 @@ ena_netmap_tx_map_slots(struct ena_netmap_ctx *ctx,
|
|||
delta = push_len - slot_head_len;
|
||||
}
|
||||
|
||||
ena_trace(NULL, ENA_NETMAP | ENA_DBG | ENA_TXPTH,
|
||||
ena_log_nm(adapter->pdev, DBG,
|
||||
"slot: %d header_buf->vaddr: %p push_len: %d\n",
|
||||
slot->buf_idx, *push_hdr, push_len);
|
||||
|
||||
|
|
@ -642,7 +646,7 @@ ena_netmap_tx_map_slots(struct ena_netmap_ctx *ctx,
|
|||
&vaddr,
|
||||
&paddr);
|
||||
if (unlikely(rc != 0)) {
|
||||
device_printf(adapter->pdev,
|
||||
ena_log_nm(adapter->pdev, ERR,
|
||||
"DMA mapping error\n");
|
||||
return (rc);
|
||||
}
|
||||
|
|
@ -695,7 +699,7 @@ ena_netmap_tx_map_slots(struct ena_netmap_ctx *ctx,
|
|||
&vaddr,
|
||||
&paddr);
|
||||
if (unlikely(rc != 0)) {
|
||||
device_printf(adapter->pdev,
|
||||
ena_log_nm(adapter->pdev, ERR,
|
||||
"DMA mapping error\n");
|
||||
goto error_map;
|
||||
}
|
||||
|
|
@ -744,7 +748,7 @@ ena_netmap_tx_map_slots(struct ena_netmap_ctx *ctx,
|
|||
&vaddr,
|
||||
&paddr);
|
||||
if (unlikely(rc != 0)) {
|
||||
device_printf(adapter->pdev,
|
||||
ena_log_nm(adapter->pdev, ERR,
|
||||
"DMA mapping error\n");
|
||||
goto error_map;
|
||||
}
|
||||
|
|
@ -860,8 +864,8 @@ ena_netmap_tx_clean_one(struct ena_netmap_ctx *ctx, uint16_t req_id)
|
|||
/* Next, retain the sockets back to the userspace */
|
||||
for (n = 0; n < nm_info->sockets_used; n++) {
|
||||
ctx->nm_i = nm_next(ctx->nm_i, ctx->lim);
|
||||
ENA_WARN(ctx->slots[ctx->nm_i].buf_idx != 0, NULL,
|
||||
"Tx idx is not 0.\n");
|
||||
ENA_WARN(ctx->slots[ctx->nm_i].buf_idx != 0,
|
||||
ctx->adapter->ena_dev, "Tx idx is not 0.\n");
|
||||
ctx->slots[ctx->nm_i].buf_idx = nm_info->socket_buf_idx[n];
|
||||
ctx->slots[ctx->nm_i].flags = NS_BUF_CHANGED;
|
||||
nm_info->socket_buf_idx[n] = 0;
|
||||
|
|
@ -882,7 +886,7 @@ validate_tx_req_id(struct ena_ring *tx_ring, uint16_t req_id)
|
|||
if (likely(req_id < tx_ring->ring_size))
|
||||
return (0);
|
||||
|
||||
ena_trace(NULL, ENA_WARNING, "Invalid req_id: %hu\n", req_id);
|
||||
ena_log_nm(adapter->pdev, WARN, "Invalid req_id: %hu\n", req_id);
|
||||
counter_u64_add(tx_ring->tx_stats.bad_req_id, 1);
|
||||
|
||||
ena_trigger_reset(adapter, ENA_REGS_RESET_INV_TX_REQ_ID);
|
||||
|
|
@ -932,7 +936,7 @@ ena_netmap_rx_frames(struct ena_netmap_ctx *ctx)
|
|||
/* In case of multiple frames, it is not an error. */
|
||||
rc = 0;
|
||||
if (frames_counter > ENA_MAX_FRAMES) {
|
||||
device_printf(ctx->adapter->pdev,
|
||||
ena_log_nm(ctx->adapter->pdev, ERR,
|
||||
"Driver is stuck in the Rx loop\n");
|
||||
break;
|
||||
}
|
||||
|
|
@ -960,7 +964,7 @@ ena_netmap_rx_frame(struct ena_netmap_ctx *ctx)
|
|||
|
||||
rc = ena_com_rx_pkt(ctx->io_cq, ctx->io_sq, &ena_rx_ctx);
|
||||
if (unlikely(rc != 0)) {
|
||||
ena_trace(NULL, ENA_ALERT,
|
||||
ena_log_nm(ctx->adapter->pdev, ERR,
|
||||
"Failed to read pkt from the device with error: %d\n", rc);
|
||||
if (rc == ENA_COM_NO_SPACE) {
|
||||
counter_u64_add(ctx->ring->rx_stats.bad_desc_num, 1);
|
||||
|
|
@ -975,7 +979,8 @@ ena_netmap_rx_frame(struct ena_netmap_ctx *ctx)
|
|||
if (unlikely(ena_rx_ctx.descs == 0))
|
||||
return (ENA_NETMAP_NO_MORE_FRAMES);
|
||||
|
||||
ena_trace(NULL, ENA_NETMAP | ENA_DBG, "Rx: q %d got packet from ena. descs #:"
|
||||
ena_log_nm(ctx->adapter->pdev, DBG,
|
||||
"Rx: q %d got packet from ena. descs #:"
|
||||
" %d l3 proto %d l4 proto %d hash: %x\n", ctx->ring->qid,
|
||||
ena_rx_ctx.descs, ena_rx_ctx.l3_proto, ena_rx_ctx.l4_proto,
|
||||
ena_rx_ctx.hash);
|
||||
|
|
@ -1031,7 +1036,7 @@ ena_netmap_rx_load_desc(struct ena_netmap_ctx *ctx, uint16_t buf, int *len)
|
|||
BUS_DMASYNC_POSTREAD);
|
||||
netmap_unload_map(ctx->na, ctx->adapter->rx_buf_tag, rx_info->map);
|
||||
|
||||
ENA_WARN(ctx->slots[ctx->nm_i].buf_idx != 0, NULL,
|
||||
ENA_WARN(ctx->slots[ctx->nm_i].buf_idx != 0, ctx->adapter->ena_dev,
|
||||
"Rx idx is not 0.\n");
|
||||
|
||||
ctx->slots[ctx->nm_i].buf_idx = rx_info->netmap_buf_idx;
|
||||
|
|
@ -1044,7 +1049,7 @@ ena_netmap_rx_load_desc(struct ena_netmap_ctx *ctx, uint16_t buf, int *len)
|
|||
ctx->slots[ctx->nm_i].len = ctx->ring->ena_bufs[buf].len;
|
||||
*len += ctx->slots[ctx->nm_i].len;
|
||||
ctx->ring->free_rx_ids[ctx->nt] = req_id;
|
||||
ena_trace(NULL, ENA_DBG, "rx_info %p, buf_idx %d, paddr %jx, nm: %d\n",
|
||||
ena_log_nm(ctx->adapter->pdev, DBG, "rx_info %p, buf_idx %d, paddr %jx, nm: %d\n",
|
||||
rx_info, ctx->slots[ctx->nm_i].buf_idx,
|
||||
(uintmax_t)rx_info->ena_buf.paddr, ctx->nm_i);
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
/*-
|
||||
* SPDX-License-Identifier: BSD-2-Clause
|
||||
*
|
||||
* Copyright (c) 2015-2020 Amazon.com, Inc. or its affiliates.
|
||||
* Copyright (c) 2015-2021 Amazon.com, Inc. or its affiliates.
|
||||
* All rights reserved.
|
||||
*
|
||||
* Redistribution and use in source and binary forms, with or without
|
||||
|
|
@ -28,6 +28,7 @@
|
|||
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
*/
|
||||
#include <sys/cdefs.h>
|
||||
#include <sys/param.h>
|
||||
__FBSDID("$FreeBSD$");
|
||||
|
||||
#include "ena_sysctl.h"
|
||||
|
|
@ -50,7 +51,7 @@ static SYSCTL_NODE(_hw, OID_AUTO, ena, CTLFLAG_RD | CTLFLAG_MPSAFE, 0,
|
|||
/*
|
||||
* Logging level for changing verbosity of the output
|
||||
*/
|
||||
int ena_log_level = ENA_ALERT | ENA_WARNING;
|
||||
int ena_log_level = ENA_INFO;
|
||||
SYSCTL_INT(_hw_ena, OID_AUTO, log_level, CTLFLAG_RWTUN,
|
||||
&ena_log_level, 0, "Logging level indicating verbosity of the logs");
|
||||
|
||||
|
|
@ -69,6 +70,19 @@ int ena_enable_9k_mbufs = 0;
|
|||
SYSCTL_INT(_hw_ena, OID_AUTO, enable_9k_mbufs, CTLFLAG_RDTUN,
|
||||
&ena_enable_9k_mbufs, 0, "Use 9 kB mbufs for Rx descriptors");
|
||||
|
||||
/*
|
||||
* Force the driver to use large LLQ (Low Latency Queue) header. Defaults to
|
||||
* false. This option may be important for platforms, which often handle packet
|
||||
* headers on Tx with total header size greater than 96B, as it may
|
||||
* reduce the latency.
|
||||
* It also reduces the maximum Tx queue size by half, so it may cause more Tx
|
||||
* packet drops.
|
||||
*/
|
||||
bool ena_force_large_llq_header = false;
|
||||
SYSCTL_BOOL(_hw_ena, OID_AUTO, force_large_llq_header, CTLFLAG_RDTUN,
|
||||
&ena_force_large_llq_header, 0,
|
||||
"Increases maximum supported header size in LLQ mode to 224 bytes, while reducing the maximum Tx queue size by half.\n");
|
||||
|
||||
void
|
||||
ena_sysctl_add_nodes(struct ena_adapter *adapter)
|
||||
{
|
||||
|
|
@ -175,6 +189,8 @@ ena_sysctl_add_stats(struct ena_adapter *adapter)
|
|||
namebuf, CTLFLAG_RD | CTLFLAG_MPSAFE, NULL, "Queue Name");
|
||||
queue_list = SYSCTL_CHILDREN(queue_node);
|
||||
|
||||
adapter->que[i].oid = queue_node;
|
||||
|
||||
/* TX specific stats */
|
||||
tx_node = SYSCTL_ADD_NODE(ctx, queue_list, OID_AUTO,
|
||||
"tx_ring", CTLFLAG_RD | CTLFLAG_MPSAFE, NULL, "TX ring");
|
||||
|
|
@ -383,6 +399,42 @@ ena_sysctl_add_tuneables(struct ena_adapter *adapter)
|
|||
}
|
||||
|
||||
|
||||
/*
|
||||
* ena_sysctl_update_queue_node_nb - Register/unregister sysctl queue nodes.
|
||||
*
|
||||
* Whether the nodes are registered or unregistered depends on a delta between
|
||||
* the `old` and `new` parameters, representing the number of queues.
|
||||
*
|
||||
* This function is used to hide sysctl attributes for queue nodes which aren't
|
||||
* currently used by the HW (e.g. after a call to `ena_sysctl_io_queues_nb`).
|
||||
*
|
||||
* NOTE:
|
||||
* All unregistered nodes must be registered again at detach, i.e. by a call to
|
||||
* this function.
|
||||
*/
|
||||
void
|
||||
ena_sysctl_update_queue_node_nb(struct ena_adapter *adapter, int old, int new)
|
||||
{
|
||||
device_t dev;
|
||||
struct sysctl_oid *oid;
|
||||
int min, max, i;
|
||||
|
||||
dev = adapter->pdev;
|
||||
min = MIN(old, new);
|
||||
max = MIN(MAX(old, new), adapter->max_num_io_queues);
|
||||
|
||||
for (i = min; i < max; ++i) {
|
||||
oid = adapter->que[i].oid;
|
||||
|
||||
sysctl_wlock();
|
||||
if (old > new)
|
||||
sysctl_unregister_oid(oid);
|
||||
else
|
||||
sysctl_register_oid(oid);
|
||||
sysctl_wunlock();
|
||||
}
|
||||
}
|
||||
|
||||
static int
|
||||
ena_sysctl_buf_ring_size(SYSCTL_HANDLER_ARGS)
|
||||
{
|
||||
|
|
@ -400,20 +452,20 @@ ena_sysctl_buf_ring_size(SYSCTL_HANDLER_ARGS)
|
|||
return (error);
|
||||
|
||||
if (!powerof2(val) || val == 0) {
|
||||
device_printf(adapter->pdev,
|
||||
ena_log(adapter->pdev, ERR,
|
||||
"Requested new Tx buffer ring size (%u) is not a power of 2\n",
|
||||
val);
|
||||
return (EINVAL);
|
||||
}
|
||||
|
||||
if (val != adapter->buf_ring_size) {
|
||||
device_printf(adapter->pdev,
|
||||
ena_log(adapter->pdev, INFO,
|
||||
"Requested new Tx buffer ring size: %d. Old size: %d\n",
|
||||
val, adapter->buf_ring_size);
|
||||
|
||||
error = ena_update_buf_ring_size(adapter, val);
|
||||
} else {
|
||||
device_printf(adapter->pdev,
|
||||
ena_log(adapter->pdev, ERR,
|
||||
"New Tx buffer ring size is the same as already used: %u\n",
|
||||
adapter->buf_ring_size);
|
||||
}
|
||||
|
|
@ -438,7 +490,7 @@ ena_sysctl_rx_queue_size(SYSCTL_HANDLER_ARGS)
|
|||
return (error);
|
||||
|
||||
if (val < ENA_MIN_RING_SIZE || val > adapter->max_rx_ring_size) {
|
||||
device_printf(adapter->pdev,
|
||||
ena_log(adapter->pdev, ERR,
|
||||
"Requested new Rx queue size (%u) is out of range: [%u, %u]\n",
|
||||
val, ENA_MIN_RING_SIZE, adapter->max_rx_ring_size);
|
||||
return (EINVAL);
|
||||
|
|
@ -446,21 +498,21 @@ ena_sysctl_rx_queue_size(SYSCTL_HANDLER_ARGS)
|
|||
|
||||
/* Check if the parameter is power of 2 */
|
||||
if (!powerof2(val)) {
|
||||
device_printf(adapter->pdev,
|
||||
ena_log(adapter->pdev, ERR,
|
||||
"Requested new Rx queue size (%u) is not a power of 2\n",
|
||||
val);
|
||||
return (EINVAL);
|
||||
}
|
||||
|
||||
if (val != adapter->requested_rx_ring_size) {
|
||||
device_printf(adapter->pdev,
|
||||
ena_log(adapter->pdev, INFO,
|
||||
"Requested new Rx queue size: %u. Old size: %u\n",
|
||||
val, adapter->requested_rx_ring_size);
|
||||
|
||||
error = ena_update_queue_size(adapter,
|
||||
adapter->requested_tx_ring_size, val);
|
||||
} else {
|
||||
device_printf(adapter->pdev,
|
||||
ena_log(adapter->pdev, ERR,
|
||||
"New Rx queue size is the same as already used: %u\n",
|
||||
adapter->requested_rx_ring_size);
|
||||
}
|
||||
|
|
@ -475,7 +527,7 @@ static int
|
|||
ena_sysctl_io_queues_nb(SYSCTL_HANDLER_ARGS)
|
||||
{
|
||||
struct ena_adapter *adapter = arg1;
|
||||
uint32_t tmp = 0;
|
||||
uint32_t old_num_queues, tmp = 0;
|
||||
int error;
|
||||
|
||||
error = sysctl_wire_old_buffer(req, sizeof(tmp));
|
||||
|
|
@ -487,7 +539,7 @@ ena_sysctl_io_queues_nb(SYSCTL_HANDLER_ARGS)
|
|||
return (error);
|
||||
|
||||
if (tmp == 0) {
|
||||
device_printf(adapter->pdev,
|
||||
ena_log(adapter->pdev, ERR,
|
||||
"Requested number of IO queues is zero\n");
|
||||
return (EINVAL);
|
||||
}
|
||||
|
|
@ -500,21 +552,26 @@ ena_sysctl_io_queues_nb(SYSCTL_HANDLER_ARGS)
|
|||
* device reset (`ena_destroy_device()` + `ena_restore_device()`).
|
||||
*/
|
||||
if (tmp > (adapter->msix_vecs - ENA_ADMIN_MSIX_VEC)) {
|
||||
device_printf(adapter->pdev,
|
||||
ena_log(adapter->pdev, ERR,
|
||||
"Requested number of IO queues is higher than maximum "
|
||||
"allowed (%u)\n", adapter->msix_vecs - ENA_ADMIN_MSIX_VEC);
|
||||
return (EINVAL);
|
||||
}
|
||||
if (tmp == adapter->num_io_queues) {
|
||||
device_printf(adapter->pdev,
|
||||
ena_log(adapter->pdev, ERR,
|
||||
"Requested number of IO queues is equal to current value "
|
||||
"(%u)\n", adapter->num_io_queues);
|
||||
} else {
|
||||
device_printf(adapter->pdev,
|
||||
ena_log(adapter->pdev, INFO,
|
||||
"Requested new number of IO queues: %u, current value: "
|
||||
"%u\n", tmp, adapter->num_io_queues);
|
||||
|
||||
old_num_queues = adapter->num_io_queues;
|
||||
error = ena_update_io_queue_nb(adapter, tmp);
|
||||
if (error != 0)
|
||||
return (error);
|
||||
|
||||
ena_sysctl_update_queue_node_nb(adapter, old_num_queues, tmp);
|
||||
}
|
||||
|
||||
return (error);
|
||||
|
|
@ -536,18 +593,18 @@ ena_sysctl_eni_metrics_interval(SYSCTL_HANDLER_ARGS)
|
|||
return (error);
|
||||
|
||||
if (interval > ENI_METRICS_MAX_SAMPLE_INTERVAL) {
|
||||
device_printf(adapter->pdev,
|
||||
ena_log(adapter->pdev, ERR,
|
||||
"ENI metrics update interval is out of range - maximum allowed value: %d seconds\n",
|
||||
ENI_METRICS_MAX_SAMPLE_INTERVAL);
|
||||
return (EINVAL);
|
||||
}
|
||||
|
||||
if (interval == 0) {
|
||||
device_printf(adapter->pdev,
|
||||
ena_log(adapter->pdev, INFO,
|
||||
"ENI metrics update is now turned off\n");
|
||||
bzero(&adapter->eni_metrics, sizeof(adapter->eni_metrics));
|
||||
} else {
|
||||
device_printf(adapter->pdev,
|
||||
ena_log(adapter->pdev, INFO,
|
||||
"ENI metrics update interval is set to: %"PRIu16" seconds\n",
|
||||
interval);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
/*-
|
||||
* SPDX-License-Identifier: BSD-2-Clause
|
||||
*
|
||||
* Copyright (c) 2015-2020 Amazon.com, Inc. or its affiliates.
|
||||
* Copyright (c) 2015-2021 Amazon.com, Inc. or its affiliates.
|
||||
* All rights reserved.
|
||||
*
|
||||
* Redistribution and use in source and binary forms, with or without
|
||||
|
|
@ -40,8 +40,13 @@
|
|||
#include "ena.h"
|
||||
|
||||
void ena_sysctl_add_nodes(struct ena_adapter *adapter);
|
||||
void ena_sysctl_update_queue_node_nb(struct ena_adapter *adapter, int old,
|
||||
int new);
|
||||
|
||||
extern int ena_enable_9k_mbufs;
|
||||
#define ena_mbuf_sz (ena_enable_9k_mbufs ? MJUM9BYTES : MJUMPAGESIZE)
|
||||
|
||||
/* Force the driver to use large LLQ (Low Latency Queue) headers. */
|
||||
extern bool ena_force_large_llq_header;
|
||||
|
||||
#endif /* !(ENA_SYSCTL_H) */
|
||||
|
|
|
|||
Loading…
Reference in a new issue