When targeting Armv8.1 we can assume FEAT_LSE is available and can use
the atomic instructions this provides without needing to check for
support first.
Reviewed by: imp, markj, emaste
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46159
I've change enough of this file to add Arm as a copyright holder.
Add it after the "All rights reserved" line as that's not needed.
Reviewed by: imp
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46157
When enqueueing on an architecture with a weak memory model ensure
loading br->br_prod_head and br->br_cons_tail are ordered correctly.
If br_cons_tail is loaded first then other threads may perform a
dequeue and enqueue before br_prod_head is loaded. This will mean the
tail is one less than it should be and the code under the
prod_next == cons_tail check could incorrectly be skipped.
buf_ring_dequeue_mc has the same issue with br->br_prod_tail and
br->br_cons_head so needs the same fix.
Reported by: Ali Saidi <alisaidi@amazon.com>
Co-developed by: Ali Saidi <alisaidi@amazon.com>
Reviewed by: imp, kib, markj
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46155
As with br_cons_tail use an atomic load acquire to read br_prod_tail
in buf_ring_dequeue_mc and buf_ring_peek*.
On dequeue we need to ensure we don't read the entry from the buf_ring
until it is available and prod_tail has updated. There is already an
appropriate store in the enqueue path and an appropriate load in the
single consumer dequeue, we just need one in the other functions that
read from the buf_ring.
Reviewed by: imp, markj
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46154
In the single consumer dequeue the consumer thread controls
br_cons_head. As such no ordering between this and other data are
required.
Reviewed by: alc, imp, kib, markj
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46153
Use an atomic operation with a memory barrier loading br_cons_tail
from the producer thread and storing to it in the consumer thread.
On dequeue we need to read the pointer value from the buf_ring before
moving the consumer tail as that indicates the entry is available to be
used. The store release atomic operation guarantees this.
In the enqueueing thread we then need to use a load acquire atomic
operation to ensure writing to this entry can only happen after the
tail has been read and checked.
Reported by: Ali Saidi <alisaidi@amazon.com>
Co-developed by: Ali Saidi <alisaidi@amazon.com>
Reviewed by: markj
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46152
If a thread reads the head but then sleeps for long enough that
another thread fills the ring and leaves the new head with the
expected value then the cmpset can pass when it should have failed.
To work around this keep the full head and tail value and use the
upper bits as a generation count.
Reviewed by: kib
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46151
We are operating on uint32_t values, use uint32_t atomic functions.
Reviewed by: alc, imp, kib, markj
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46150
The only part of DEBUG_BUFRING we don't support in userspace is the
mutex checks. Add _KERNEL checks around these so we can enable the
extra debugging.
Reviewed by: alc, imp, kib, markj
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46149
I'm not able to find anything in the tree that ever defined it. Remove
as it's unused so is untested.
Reviewed by: alc, imp, kib, markj
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46148
sys/netinet/libalias/alias_db.c has internal static function UseLink()
that passes a link to CleanupLink() to verify if the link has expired.
If so, UseLink() may return NULL.
_FindLinkIn()'s usage of UseLink() is not quite correct.
Assume there is "redirect_port udp" configured to forward incoming
traffic for specific port to some internal address.
Such a rule creates partially specified permanent link.
After first such packet libalias creates new fully specifiled
temporary LINK_UDP with default timeout 60 seconds.
Also, in case of low traffic libalias may assign "timestamp"
for this new temporary link way in the past because
LibAliasTime is updated seldom and can keep old value
for tens of seconds, and it will be used for the temporary link.
It may happen that next incoming packet for redirected port
passed to _FindLinkIn() results in a call to UseLink()
that returns NULL due to detected expiration.
Immediate return of NULL results in broken translation:
either a packet is dropped (deny_incoming mode) or delivered to
original destination address instead of internal one.
Fix it with additional check for NULL to proceed with a search
for original partially specified link. In case of UDP,
it also recreates temporary fully specified link
with a call to ReLink().
Practical examples are "redirect_port udp" rules for unidirectional
SYSLOG protocol (port 514) or some low volume VPN encapsulated in UDP.
Thanks to Peter Much for initial analysis and first version of a patch.
Reported by: Peter Much <pmc@citylink.dinoex.sub.org>
PR: 269770
MFC after: 1 week
Clear the EQ_HW_ALLOCATED flag with the wrq lock held and discard all
work requests, pending or new, when it's not set.
MFC after: 1 week
Sponsored by: Chelsio Communications
RFC8275 defines a new attribute as an extension to NFSv4.2
called MODE_UMASK. This patch adds the attribute number
to nfsproto.h.
Future patches will add optional support for the attribute.
This patch does not cause any semantics change.
MFC after: 2 weeks
When route_to() processes a packet without state, pf_map_addr() is called for
each packet. Pf_map_addr() will search for a source node and will find none
since those are created only in pf_create_state(). Thus sticky address,
even though requested in rule definition, will never work.
Raise an error when a stateless filter rule uses sticky address to avoid
confusion and to keep ruleset limitations in sync with what the pf code
really does.
Reviewed by: kp
Differential Revision: https://reviews.freebsd.org/D46310
When it's a I/O failure, we can still send admin commands. Separate out
the admin failures and flag them as such so that we can still send admin
commands on half-failed drives.
Fixes: 9229b3105d (nvme: Fail passthrough commands right away in failed state)
Sponsored by: Netflix
When it's a I/O failure, we can still send admin commands. Separate out
the admin failures and flag them as such so that we can still send admin
commands on half-failed drives.
Fixes: 9229b3105d (nvme: Fail passthrough commands right away in failed state)
Sponsored by: Netflix
tmpfs_mem_percent is an int not a long, so on a 64-bit system this
writes 4 bytes past the end of the variable. The read above is correct,
so this was likely a copy paste error from sysctl_mem_reserved.
Found by: CHERI
Fixes: 636592343c ("tmpfs: increase memory reserve to a percent of available memory + swap")
Both acl_copy_oldacl_into_acl() and acl_copy_acl_into_oldacl() may fail
in some circumstances (e.g., acl.acl_cnt exceeding the capacity of
OLDACL_MAX_ENTRIES). This change marks both routines with
__result_use_check, enforcing check for errors by the caller.
Suggested by: markj
Reviewed by: markj, emaste
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D46254
Commit 099b595154 ("Improve loading of multipage aligned buffers.")
modified bounce_bus_dmamap_load_buffer() with the assumption that busdma
memory allocations are physically contiguous, which is not always true:
bounce_bus_dmamem_alloc() will allocate memory with
kmem_alloc_attr_domainset() in some cases, and this function is not
guaranteed to return contiguous memory.
The damage seems to have been mitigated for most consumers by clamping
the segment size to maxsegsz, but this was removed in commit
a77e1f0f81 ("busdma: better handling of small segment bouncing"); in
practice, it seems busdma memory is often allocated with maxsegsz ==
PAGE_SIZE. In particular, after commit a77e1f0f81 I see occasional
random kernel memory corruption when benchmarking TCP through mlx5
interfaces.
Fix the problem by using separate flags for contiguous and
non-contiguous busdma memory allocations, and using that to decide
whether to clamp.
Fixes: 099b595154 ("Improve loading of multipage aligned buffers.")
Fixes: a77e1f0f81 ("busdma: better handling of small segment bouncing")
Sponsored by: Klara, Inc.
Sponsored by: Stormshield
Differential Revision: https://reviews.freebsd.org/D46238
The struct pf_addr_wrap when used inside of kernel operates on pointers to
tables or interfaces. When reading a ruleset the struct must contain
counters calculated from the aforementioned tables and interfaces. Both the
pointers and the resulting counters are stored in an union and thus can't be
present in the struct at the same time.
The original ioctl code handles this by making a copy of struct pf_addr_wrap
for pool addresses, accessing the table or interface structures by their
pointers, calculating the counter values and storing them in place of those
pointers in the copy. Then this copy is sent over ioctl.
Use this mechanism for netlink too. Create a copy of src/dst addresses. Use
the existing function pf_addr_copyout() to convert pointers to counters both
for src/dst and pool addresses.
Reviewed by: kp
Differential Revision: https://reviews.freebsd.org/D46291
Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Reviewed-by: Rob Norris <robn@despairlabs.com>
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
Closes#16431
(cherry picked from commit 244ea5c488)
(e.g. traceroute with icmp)
ok henning, jsing
Also extend the test case to cover this scenario.
PR: 280701
Obtained from: OpenBSD
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
While it is easy enough to bounce over to nvme.c from nvme_ctrlr.c to
find this out, I've had to do that several times, so a little bit of
context is quite helpful.
Sponsored by: Netflix
APIC ID 255 and above require x2APIC and DMAR interrupt remapping.
FreeBSD is starting to be tested on high core count Intel systems that
meet this criteria.
Reviewed by: kib, jhb
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D42435
APIC ID 255 and above require x2APIC and DMAR interrupt remapping.
FreeBSD is starting to be tested on high core count Intel systems that
meet this criteria. We're going to enable DMAR by default to support
this, so default hw.iommu.dma to 0 to avoid a significant performance
regression.
Reviewed by: kib, jhb
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D42435
In 534ee17e6 pf state checking for ICMP(v6) was made stricter. This change
failed to correctly set the pf_pdesc for ICMP-in-ICMP lookups, resulting in ICMP
error packets potentially being dropped incorrectly.
Specially, it copied the ICMP header into a separate variable, not into the
pf_pdesc.
Populate the required pf_pdesc fields for the embedded ICMP packet's state lookup.
PR: 280701
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
Some SCTP implementations will abort connections and then later re-use the same
port numbers (i.e. both src and dst) for a new connection, before pf has fully
purged the old connection.
Apply the same hack we already have for similarly misbehaving TCP
implementations and forcibly remove the old state so we can create a new one.
MFC after: 2 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
A lock needs to be held to ensure that the socket does not become a
listening socket while sotoxsocket() is loading fields from the socket
buffers, as the memory backing the socket buffers is repurposed when
transitioning to a listening socket.
MFC after: 1 week
Sponsored by: Klara, Inc.
Sponsored by: Stormshield
bitstring.h includes a definition of bit_foreach, for iterating over
the set bits of a bitstring. axgbe implements its own version of this
for bitstrings. Drop it, and use the bitstring method.
Reviewed by: des
Differential Revision: https://reviews.freebsd.org/D46037
Make the system call honor `AT_SYMLINK_NOFOLLOW`.
Also enable this from `linux_faccessat2` where the issue arised the first time.
Update manual pages accordingly.
PR: 275295
Reported by: kenrap@kennethraplee.com
Approved by: kib@
Differential Revision: https://reviews.freebsd.org/D46267
Notable upstream pull request merges:
#158175536c0dee Sync AUX label during pool import
#15889c7ada64bb ddt: dedup table quota enforcement
#1589062e7d3c89 ddt: add support for prefetching tables into the ARC
#15894e26b3771e spa_preferred_class: pass the entire zio
#15894d54d0fff3 dnode: allow storage class to be overridden by object type
#1619755427add3 Several improvements to ARC shrinking
#16217 -multiple JSON output for various zfs and zpool subcommands
#1624824e6585e7 libzfs.h: Set ZFS_MAXPROPLEN and ZPOOL_MAXPROPLEN
to ZAP_MAXVALUELEN
#162649dfc5c4a0 Fix long_free_dirty accounting for small files
#16268ed0db1cc8 Make txg_wait_synced conditional in zfsvfs_teardown,
for FreeBSD
#16288d60debbf5 Fix sa_add_projid to lookup and update SA_ZPL_DXATTR
#16308ec580bc52 zfs: add bounds checking to zil_parse
#16310c21dc56ea Fix zdb_dump_block for little endian
#163157ddc1f737 zil: add stats for commit failure/fallback
#16326b0bf14cdb abd: lift ABD zero scan from zio_compress_data()
to abd_cmp_zero()
#16337c8184d714 Block cloning conditionally destroy ARC buffer
#16338dbe07928b Add support for multiple lines to the sharenfs property
for FreeBSD
#163741a3e32e6a Cleanup DB_DNODE() macros usage
#16374ed87d456e Skip dnode handles use when not needed
#16346fb6d8cf22 Add some missing vdev properties
#16364670147be5 zvol: ensure device minors are properly cleaned up
#16382dea8fabf7 FreeBSD: Fix RLIMIT_FSIZE handling for block cloning
#16387aef452f10 Improve zfs_blkptr_verify()
#16395cbcb52243 Fix the names of some FreeBSD sysctls in
include/tunables.cfg
#164015b9f3b766 Soften pruning threshold on not evictable metadata
#16404cdd53fea1 FreeBSD: Add missing memory reclamation accounting
#164041fdcb653b Once more refactor arc_summary output
#164191f5bf91a8 Fix memory corruption during parallel zpool import
with -o cachefile
#16426cf6e8b218 zstream: remove duplicate highbit64 definition
Obtained from: OpenZFS
OpenZFS commit: 9c56b8ec78
The maximum cluster number was calculated based on the number of data
cluters that fit in the givem partition size and the size of the FAT
area. This limit did not take into account that the highest 10 cluster
numbers are reserved and must not be used for files.
PR: 280347
MFC after: 3 days
Reported by: pho@FreeBSD.org
acl_copyin did not validate the return value of acl_copy_oldacl_into_acl
which could lead to uninitialized acl structure memory reads.
Reported by: Synacktiv
Reviewed by: markj, emaste
Sponsored by: The Alpha-Omega Project
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D46125
Initialize V_ts_offset_secret for each vnet, not only for the
default vnet, since it is vnet specific.
Reviewed by: Peter Lei
MFC after: 3 days
Sponsored by: Netflix, Inc.
Differential Revision: https://reviews.freebsd.org/D46246
