Add the Branch Target Identification (BTI) note to libc assembly
sources. As all obect files need the note for the library to have it
we need to insert it in all asm files.
Reviewed by: emaste, markj
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D42228
(cherry picked from commit fd5aaf2ea0178b03aa93c35245053247e5d3840c)
The section INTERNET ADDRESSES describes the acceptance of dotted
values with varying number of parts in multiple bases. This applies
to inet_aton and inet_addr, but not to inet_pton. Clarify this
section by listing the functions to which this applies. Move the
description of what inet_pton accepts into this section from STANDARDS,
where it is easily missed. Rename the section to clarify that it
applies only to IPv4. (inet_pton also works with IPv6.)
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D43537
(cherry picked from commit 9231c42127bf8e47588169ecc395f57cae0e15fb)
The code in this file runs before the sanitizer can initialize its
shadow map.
Fixes: ad2fac552c ("lib/libc/amd64: add archlevel-based simd dispatch framework")
(cherry picked from commit 4dedcb1bb54cbbe8043c79ad733f966b6ffc6972)
The scalar implementation is fairly simplistic and only performs
slightly better than the generic C implementation. It could be
improved by using the same algorithm as for memchr, but it would
have been a lot more complicated.
The baseline implementation is similar to timingsafe_memcmp. It's
slightly slower than memchr() due to the more complicated main
loop, but I don't think that can be significantly improved.
Tested by: developers@, exp-run
Approved by: mjg
MFC after: 1 month
MFC to: stable/14
PR: 275785
Differential Revision: https://reviews.freebsd.org/D42925
(cherry picked from commit fb197a4f7751bb4e116989e57ba7fb12a981895f)
The "values" test case is specifically crafted to detect the off-by-one
error previous discovered in the scalar strchrnul implementation.
Tested by: developers@, exp-run
Approved by: mjg
MFC after: 1 month
MFC to: stable/14
PR: 275785
Differential Revision: https://reviews.freebsd.org/D42925
(cherry picked from commit 691ff1832e09a6ccbc8f5b04c88cafc7452b3ce6)
POSIX.1-2004 and the upcoming C23 agree that memccpy()'s arguments
are restrict qualified and must not overlap. In 2002, restrict
qualifiers were added to <string.h>'s declaration of the function.
Make things official and document that the arguments must not
overlap.
See also: 61b60edfd3
Approved by: kib
MFC after: 1 month
MFC to: stable/14
(cherry picked from commit e0d4f419ac41aa91b862f3ceadc32a86abf08572)
Based on the strlcpy code from D42863, this patch adds a SIMD-enhanced
implementation of memccpy for amd64. A scalar implementation calling
into memchr and memcpy to do the job is provided, too.
Please note that this code does not behave exactly the same as the C
implementation of memccpy for overlapping inputs. However, overlapping
inputs are not allowed for this function by ISO/IEC 9899:1999 and neither
has the C implementation any code to deal with the possibility. It just
proceeds byte-by-byte, which may or may not do the expected thing for
some overlaps. We do not document whether overlapping inputs are
supported in memccpy(3).
Tested by: developers@, exp-run
Approved by: mjg
MFC after: 1 month
MFC to: stable/14
PR: 275785
Differential Revision: https://reviews.freebsd.org/D42902
(cherry picked from commit fc0e38a7a67a6d43095efb00cf19ee5f95dcf710)
This should pick up our optimised memchr(), strlen(), and strlcpy()
when strlcat() is called.
Tested by: developers@, exp-run
Approved by: mjg
MFC after: 1 month
MFC to: stable/14
PR: 275785
Differential Revision: https://reviews.freebsd.org/D42863
(cherry picked from commit 2b7b03b7ae179db465c1ef19a5007f729874916a)
Somewhat similar to stpncpy, but different in that we need to compute
the full source length even if the buffer is shorter than the source.
strlcat is implemented as a simple wrapper around strlcpy. The scalar
implementation of strlcpy just calls into strlen() and memcpy() to do
the job.
Perf-wise we're very close to stpncpy. The code is slightly slower as
it needs to carry on with finding the source string length even if the
buffer ends before the string.
Sponsored by: The FreeBSD Foundation
Tested by: developers@, exp-run
Approved by: mjg
MFC after: 1 month
MFC to: stable/14
PR: 275785
Differential Revision: https://reviews.freebsd.org/D42863
(cherry picked from commit 74d6cfad54d676299ee5e4695139461876dfd757)
A straightforward derivation from the stpncpy unit test.
Sponsored by: The FreeBSD Foundation
Tested by: developers@, exp-run
Approved by: mjg
MFC after: 1 month
MFC to: stable/14
PR: 275785
Differential Revision: https://reviews.freebsd.org/D42863
(cherry picked from commit f7098b8659923873a7c60b64cb68182e470786f9)
strcat has a bespoke scalar assembly implementation we
inherited from NetBSD. While it performs well, it is
better to call into our SIMD implementations if any SIMD
features are available at all. So do that and implement
strcat() by calling into strlen() and strcpy() if these
are available.
Sponsored by: The FreeBSD Foundation
Tested by: developers@, exp-run
Approved by: mjg
MFC after: 1 month
MFC to: stable/14
PR: 275785
Differential Reviison: https://reviews.freebsd.org/D42600
(cherry picked from commit aff9143a242c0012b0195b3666e03fa3b7cd33e8)
This was surprisingly annoying to get right, despite being such a simple
function. A scalar implementation is also provided, it just calls into
our optimised memchr(), memcpy(), and memset() routines to carry out its
job.
I'm quite happy with the performance. glibc only beats us for very long
strings, likely due to the use of AVX-512. The scalar implementation
just calls into our optimised memchr(), memcpy(), and memset() routines,
so it has a high overhead to begin with but then performs ok for the
amount of effort that went into it. Still beats the old C code, except
for very short strings.
Sponsored by: The FreeBSD Foundation
Tested by: developers@, exp-run
Approved by: mjg
MFC after: 1 month
MFC to: stable/14
PR: 275785
Differential Revision: https://reviews.freebsd.org/D42519
(cherry picked from commit 90253d49db09a9b1490c448d05314f3e4bbfa468)
This adds additional unit tests validating the function for
All possible alignment offsets of source and destination.
Also extend the test to allow testing of an external stpncpy
implementation, which greatly simplifies the development of
custom implementations.
Sponsored by: The FreeBSD Foundation
Tested by: developers@, exp-run
Approved by: mjg
MFC after: 1 month
MFC to: stable/14
PR: 275785
Differential Revision: https://reviews.freebsd.org/D42519
(cherry picked from commit 6fa9e7d8737548ef93c573387ce62402c368d486)
The strsep() function is basically strcspn() with extra steps.
On amd64, we now have an optimised implementation of strcspn(),
so instead of implementing the inner loop manually, just call
into the optimised routine.
Sponsored by: The FreeBSD Foundation
Tested by: developers@, exp-run
Approved by: mjg
MFC after: 1 month
MFC to: stable/14
PR: 275785
Differential Revision: https://reviews.freebsd.org/D42346
(cherry picked from commit fd2ecd91aeeeab579c769c9a39f90b4bd4a493a9)
The baseline implementation is very straightforward, while the scalar
implementation suffers from register pressure and the need to use SWAR
techniques similar to those used for strchr().
Sponsored by: The FreeBSD Foundation
Tested by: developers@, exp-run
Approved by: mjg
MFC after: 1 month
MFC to: stable/14
PR: 275785
Differential Revision: https://reviews.freebsd.org/D42217
(cherry picked from commit 2ed514a220edbac6ca5ec9f40a3e0b3f2804796d)
The scalar implementation is fairly straightforward and merely unrolled
four times. The baseline implementation closely follows D41971 with
appropriate extensions and extra code paths to pay attention to string
length.
Performance is quite good. We beat both glibc (except for very long
strings, but they likely use AVX which we don't) and Bionic (except for
medium-sized aligned strings, where we are still in the same ballpark).
Sponsored by: The FreeBSD Foundation
Tested by: developers@, exp-run
Approved by: mjg
MFC after: 1 month
MFC to: stable/14
PR: 275785
Differential Revision: https://reviews.freebsd.org/D42122
(cherry picked from commit 14289e973f5c941e4502cc2b11265e4b3072839a)
These are patterned after the previously added (D41970)
strcmp tests, but are extended to check for various length
conditions.
Sponsored by: The FreeBSD Foundation
Tested by: developers@, exp-run
Approved by: mjg
MFC after: 1 month
MFC to: stable/14
PR: 275785
Differential Revision: https://reviews.freebsd.org/D42122
(cherry picked from commit 459ddefcc9dcc010f6f7445285e61e2b6780379c)
This lets us use our optimised strcspn() routine for strpbrk() calls.
Sponsored by: The FreeBSD Foundation
Tested by: developers@, exp-run
Approved by: mjg
MFC after: 1 month
MFC to: stable/14
PR: 275785
Differential Revision: https://reviews.freebsd.org/D41980
(cherry picked from commit f4fc317c364f2c81ad3d36763d8e5a60393ddbd1)
This is the most complicated one so far. The basic idea is to process
the bulk of the string in aligned blocks of 16 bytes such that one
string runs ahead and the other runs behind. The string that runs ahead
is checked for NUL bytes, the one that runs behind is compared with the
corresponding chunk of the string that runs ahead. This trades an extra
load per iteration for the very complicated block-reassembly needed in
the other implementations (bionic, glibc). On the flip side, we need
two code paths depending on the relative alignment of the two buffers.
The initial part of the string is compared directly if it is known not
to cross a page boundary. Otherwise, a complex slow path to avoid
crossing into unmapped memory commences.
Performance-wise we beat bionic for misaligned strings (i.e. the strings
do not share an alignment offset) and reach comparable performance for
aligned strings. glibc is a bit better as it has a special kernel for
AVX-512, where this stuff is a bit easier to do.
Sponsored by: The FreeBSD Foundation
Tested by: developers@, exp-run
Approved by: mjg
MFC after: 1 month
MFC to: stable/14
PR: 275785
Differential Revision: https://reviews.freebsd.org/D41971
(cherry picked from commit bca25680b91b3bea7faef615765806a04634eb23)
We don't support it, so there's no need to tell readers what would
happen if we did. Also, don't remind the user that a certain field is
ignored by aio_read. Mentioning every ignored field would make the man
pages too verbose.
Sponsored by: Axcient
Reviewed by: Pau Amma <pauamma@gundo.com>
Differential Revision: https://reviews.freebsd.org/D42622
(cherry picked from commit 18e2c4175f78f1aaa648dd7fb7530220aed23671)
Add a required include to resolv.h for sockaddr_in. This should reduce
patching required when porting code written with Linux or NetBSD in mind.
PR: 182466
MFC after: 1 week
(cherry picked from commit 58cf91d3b72a01777bacf72d66a648a744ae3143)
PR#273962 reported that copy_file_range(2) did not work
on shared memory objects and returned EINVAL.
Although the reporter felt this was incorrect, it is what
the Linux copy_file_range(2) syscall does.
Since there was no collective agreement that the FreeBSD
semantics should be changed to no longer be Linux compatible,
copy_file_range(2) still works on regular files only.
This man page update clarifies that. If, someday, copy_file_range(2)
is changed to support non-regular files, then the man page will
need to be updated to reflect that.
PR: 273962
(cherry picked from commit 84b4342c0d7ac8a3187309a978d41e6765154cc1)
Use boolean evaluation of :M matches and a single if statement.
Reviewed by: imp, kib
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D42915
(cherry picked from commit fc0288993cdad8a559fcd2c2166cf95f1fa43745)
For architectures where vfork.S was named Ovfork.S this was needed, but
it was always pointless here as an entry in either MDASM or NOASM is
equivalent.
Reviewed by: kib
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D42914
(cherry picked from commit ec27c0bb3eea73be4db6cd2f275db6c516e12d00)
While this has been Ovfork.S forever on i386 it differs from other
syscalls that require wrappers for no obvious reason so fix that.
Reviewed by: kib
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D42909
(cherry picked from commit 0ea469bcd548d29bbbc970325e4fa851d0e4c022)
The actual implementation of sbrk(2) is on top of the undocumented
break(2) system call. On powerpc* this means we don't build _sbrk and
__sys_sbrk which were neither used nor exposed for linkage. Otherwise
it is a no-op.
The addition to lib/libc/sys/Makefile.inc is a direct commit to
stable/14 in lieu of merging the removal of the sbrk and sstk syscalls.
(cherry picked from commit 7893419d492c40ca82b68fca3dcc0f5f7047d39b)
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D43159
