To allow for a dynamic page size on arm64 have the runtime linker
query the kernel for the currentl page size.
Reviewed by: kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34765
- Include <machine/tls.h> in MD rtld_machdep.h headers.
- Remove local definitions of TLS_* constants from rtld_machdep.h
headers and libc using the values from <machine/tls.h> instead.
- Use _tcb_set() instead of inlined versions in MD
allocate_initial_tls() routines in rtld. The one exception is amd64
whose _tcb_set() invokes the amd64_set_fsbase ifunc. rtld cannot
use ifuncs, so amd64 inlines the logic to optionally write to fsbase
directly.
- Use _tcb_set() instead of _set_tp() in libc.
- Use '&_tcb_get()->tcb_dtv' instead of _get_tp() in both rtld and libc.
This permits removing _get_tp.c from rtld.
- Use TLS_TCB_SIZE and TLS_TCB_ALIGN with allocate_tls() in MD
allocate_initial_tls() routines in rtld.
Reviewed by: kib, jrtc27 (earlier version)
Differential Revision: https://reviews.freebsd.org/D33353
This fixes a regression in d36d681615, where the call to
__tls_get_address() was performed under rtld_bind_lock write-locked.
Instead use tls_get_addr_slow() directly, with locked = true.
Reported by: jkim, many others
Tested by: jkim, bdragon (powerpc), mhorne (riscv)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D29623
Parentheses added to HASZERO macro to avoid a GCC warning, and formatted
with clang-format as we have adopted these and don't consider them
'contrib' code.
Obtained from: musl (snapshot at commit 4d0a82170a25)
Reviewed by: kib (libc integration), mjg (both earlier)
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D17630
Statically link rtld-elf with libcompiler_rt on all architectures so
that we don't need to try to pick and choose the bits we need from it
for each architecture (we now leave that to the linker). Compilers may
emit calls to support functions in this library, but because of the use
of the linker flag -nostdlib for rtld's special needs, the library is
not linked as normal.
Previously we had two different solutions. On some architectures, we
were able to extract reimplementations of the necessary builtin
functions from our special build of libc. On ARM, we just linked
libcompiler_rt.
This is motivated by the same issue as D26199 and D27665, but should be
a simpler solution that will apply to all architectures.
Reviewed by: arichardson, kib
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D27736
This fixes the gcc9 build of rtld-elf32 on amd64, which needed an
implementation of udivmoddi4.
rtld-elf uses certain functions normally found in libc, and so it
includes certain files from libc in its own build. It has two
mechanisms to include files from libc: one that rebuilds source files in
the rtld-elf environment, and one that extracts object files from a
purpose-built no-SSP PIC archive.
In addition to libc functions, rtld-elf may need to link functions
normally found in libcompiler_rt (formerly libgcc). Now, add an ability
to rebuild libcompiler_rt source files in the rtld-elf environment. We
don't yet have a need for an object file extraction mechanism.
libcompiler_rt could also supply udivdi3 and umoddi3, but leave them
alone for now.
Reviewed by: arichardson, kib
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D27665
ar cr is an update of an archive, not a creation of a new one. During
incremental builds (e.g. with meta mode) the archive was not getting
cleaned, and so could retain now-deleted objects from previous builds.
Now, delete the archive before creating/updating it.
Reviewed by: arichardson, bdrewery, kib
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D27663
Given that we have converted to ELFv2 for BE already, endianness is the only
difference between the two ARCHs.
As such, there is no need to differentiate LIBC_ARCH between the two.
Combining them like this lets us avoid needing to have two copies of several
bits for no good reason.
Sponsored by: Tag1 Consulting, Inc.
Rtld itself is a shared object which does not have vendor note, so
after the direct exec of ld-elf.so.1 process has p_osrel set to zero.
This affects the ABI of syscalls.
Set osrel to the __FreeBSD_version value at compile time right after
rtld identified direct exec mode. Then, switch to the osrel read from
the binary note or zero if no note, right before starting calling
ifunc resolvers, which is the first byte of the user code.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
realpath(3) is used a lot e.g., by clang and is a major source of getcwd
and fstatat calls. This can be done more efficiently in the kernel.
This works by performing a regular lookup while saving the name and found
parent directory. If the terminal vnode is a directory we can resolve it using
usual means. Otherwise we can use the name saved by lookup and resolve the
parent.
See the review for sample syscall counts.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D23574
This allows for rtld to not issue two sigprocmask(2) syscalls for each
symbol binding operation in single-threaded processes. Rtld needs to
block signals as part of locking to ensure signal safety of the bind
process, because signal handlers might need to lazily resolve symbol
references.
As result, number of syscalls issued on startup by simple programs not
using libthr, is typically reduced 2x. For instance, for hello world,
I see:
non-sigfastblock
# (truss ./hello > /dev/null) |& wc -l
63
sigfastblock
# (truss ./hello > /dev/null) |& wc -l
37
Tested by: pho
Disscussed with: cem, emaste, jilles
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D12773
They were introduced to take care of ifunc, but right now no architecture
provides ifunc'ed variants. Since rtld uses memset extensively this results in
a pessmization. Should someone want to use ifunc here they should provide a
mandatory symbol (e.g., rtld_memset).
See the review for profiling data.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D23176
I found this on one of the CheriBSD Jenkins builders. Using
beforelinking instead of ${PROG} should fix the dependency for the
DEBUG_FILES case.
Reviewed by: brooks
Currently RTLD is linked against libc_nossp_pic which means that any libc
symbol used in rtld can pull in a lot of depedencies. This was causing
symbol such as __libc_interposing and all the pthread stubs to be included
in RTLD even though they are not required. It turns out most of these
dependencies can easily be avoided by providing overrides inside of rtld.
This change is motivated by CHERI, where we have an experimental ABI that
requires additional relocation processing to allow the use of function
pointers inside of rtld. Instead of adding this self-relocation code to
RTLD I attempted to remove most function pointers from RTLD and discovered
that most of them came from the libc dependencies instead of being actually
used inside rtld.
A nice side-effect of this change is that rtld is now 22% smaller on amd64.
text data bss dec hex filename
0x21eb6 0xce0 0xe60 145910 239f6 /home/alr48/ld-elf-x86.before.so.1
0x1a6ed 0x728 0xdd8 113645 1bbed /home/alr48/ld-elf-x86.after.so.1
The number of R_X86_64_RELATIVE relocations that need to be processed on
startup has also gone down from 368 to 187 (almost 50% less).
Reviewed By: kib
Differential Revision: https://reviews.freebsd.org/D20663
