Strictly speaking, SYS V Aarch64 ABI does not states that lr should be
cleared, however gdb relies on lr and tryes to unwind stack further.
Reviewed by:
Differential Revision: https://reviews.freebsd.org/D40841
Use of compiler builtin ffs/ctz functions will result in optimized
instruction sequences when possible, and fall back to calling a function
provided by the compiler run-time library. We have slowly shifted our
platforms to take advantage of these builtins in 60645781d6 (arm64),
1c76d3a9fb (arm), 9e319462a0 (powerpc, partial).
Some platforms still rely on the libkern implementations of these
functions provided by libkern, namely riscv, powerpc (ffs*, flsll), and
i386 (ffsll and flsll). These routines are slow, as they perform a
linear search for the bit in question. Even on platforms lacking
dedicated bit-search instructions, such as riscv, the compiler library
will provide better-optimized routines, e.g. by using binary search.
Consolidate all definitions of these functions (whether currently using
builtins or not) to libkern.h. This should result in equivalent or
better performing routines in all cases.
One wart in all of this is the existing HAVE_INLINE_F*** macros, which
we use in a few places to conditionally avoid the slow libkern routines.
These aren't easily removed in one commit. For now, provide these
defines unconditionally, but marked for removal after subsequent
cleanup.
Removal of the now unused libkern routines will follow in the next
commit.
Reviewed by: dougm, imp (previous version)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40698
This will be used by bhyve and will allow the size to change, e.g. for SVE.
Reviewed by: markj
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D40131
In get_kernel_reg_masked we use update_lower_register to get the lower
value of two registers for a given field. It will return the entire
register value with just the single field updated.
Because of this get_kernel_reg_masked needs to use the returned value
directly rather than ORing each field together. Fix this by updating
the mask and returning that from get_kernel_reg_masked.
Reviewed by: markj
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D40846
The setting of VM_NFREEORDER and the comment describing it were copied
from sparc64 where both the page size and the number of page table
entries that fit in a cache line are different from arm64.
Reviewed by: andrew, kib, markj
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D40782
No functional change intended.
Reviewed by: andrew
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc. (hardware)
Differential Revision: https://reviews.freebsd.org/D40502
This lets consumers fetch the value of a system register and apply a
mask over individual fields. That is, each field in the returned value
will be the "smaller" of the two provided by "mask" and the value saved
in kern_cpu_desc. This will be used by vmm to sanitize host system
register fields.
Reviewed by: andrew
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc. (hardware)
Differential Revision: https://reviews.freebsd.org/D40500
Since pmap_ps_enabled() is true by default, check it inside of
pmap_promote_l2() instead of at every call site.
Modify pmap_promote_l2() to return true if the promotion succeeded and
false otherwise.
(A similar change was applied to the amd64 pmap in 0d2f98c2f092.)
Reviewed by: kib, markj
Differential Revision: https://reviews.freebsd.org/D40781
Not making it explicit that we're printing values in hex can be
misleading when the number doesn't contain hex-only symbols (a-f). A
good example of this is print_gp_register(), where we print "(func +
offset)"; if the offset doesn't contain a-f symbols, it's not
immediately clear if that value is in decimal or hex. Using '%#' instead
of '0x%' also isn't a better option, it doesn't print '0x' if the value
is 0, and it also messes up column alignment.
Reviewed by: imp, markj
Approved by: markj (mentor)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40583
Add disassembly support for the following shifted register instructions:
* mvn
* orn
* orr
* and
* ands
* bic
* bics
* eon
* eor
* tst
According to Arm64 documenation, operational pseuducode of shifted
register instruction must return `UNDEFINED` if shift type is `RESERVED`
('11'). Hence, removed "rsv" from `shift_2` array and add "ror". In case
of shift type is 3 and this type is `RESERVED`, we will return
`undefined`.
Reviewed by: mhorne
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D40386
Replace several sequential searches for a segment that contains a
phyiscal address with a call to a function that does it by binary
search. In vm_page_reclaim_contig_domain_ext, find the first segment
to reclaim from, and reclaim from each subsequent appropriate segment.
Eliminate vm_phys_scan_contig.
Reviewed by: alc, markj
Differential Revision: https://reviews.freebsd.org/D40058
It doesn't seem particularly useful to have it enabled by default, and
it spits several dozen lines into the dmesg on a test system.
Reviewed by: andrew
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D40499
It was used in one place and was added specifically to support dtrace
stack unwinding code. Write an equivalent expression using struct
unwind_state instead. No functional change intended.
Reviewed by: andrew
MFC after: 1 week
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D40538
Stop requiring all of the PTEs to have the accessed bit set for superpage
promotion to occur. Given that change, add support for promotion to
pmap_enter_quick(), which does not set the accessed bit in the PTE that
it creates.
Since the final mapping within a superpage-aligned and sized region of a
memory-mapped file is typically created by a call to pmap_enter_quick(),
we now achieve promotions in circumstances where they did not occur
before, for example, the X server's read-only mapping of libLLVM-15.so.
See also https://www.usenix.org/system/files/atc20-zhu-weixi_0.pdf
Reviewed by: kib, markj
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D40478
This is a mapping from ID register value to offset in struct cpu_desc.
These registers may be needed with future architecture revisions either
by userspace or by bhyve.
Sponsored by: Arm Ltd
We only need this during boot. Allocate the array before starting CPUs
to reduce the memory usage.
Reviewed by: Zach Leaf <zachary.leaf@arm.com>
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D40433
To help moving to a dynamically allocated cpu_desc array reduce the
places we use it directly and create a pointer that is passed in to
functions that read it.
Reviewed by: Zach Leaf <zachary.leaf@arm.com>
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D40392
To find the redistributor registers use the resource we have already
found and add an offset. This removed the need to create a
per-redistributor resource as it can now be a pointer to the resource
found in attach.
While here check the offset is within the bounds of the resource. Some
ACPI tables list each redistributor as a separate memory range, even
if they are physically contiguous. In this case we may not have each
resource virtually contiguous with neighbouring resources. This can
lead to a data abort when reading past the resource range.
Reviewed by: kevans
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D40263
The prior implementation of VM_PAGE_TO_PV_LIST_LOCK() performed a
linear-time search of the vm_phys_segs[] array. However, in contrast to
PHYS_TO_PV_LIST_LOCK(), that search is unnecessary because every (non-
fictitious) vm_page contains the index of the vm_phys_seg in which it
resides.
Change most of the remaining uses of CHANGE_PV_LIST_LOCK_TO_PHYS() and
PHYS_TO_PV_LIST_LOCK() to CHANGE_PV_LIST_LOCK_TO_VM_PAGE() and
VM_PAGE_TO_PV_LIST_LOCK(), respectively.
Collectively, these changes also reduce the size of a GENERIC-NODEBUG
kernel's pmap.
Before:
text data bss dec hex filename
70144 3200 2248 75592 0x12748 pmap.o
After:
text data bss dec hex filename
69192 3200 2248 74640 0x12390 pmap.o
Reviewed by: kib, markj
Differential Revision: https://reviews.freebsd.org/D40306
The sysentvec sv_imgact_try was used by kern_exec() to allow
non-native ABI to fixup shell path according to ABI root directory.
Since the non-native ABI can now specify its root directory directly
to namei() via pwd_altroot() call this facility is not needed anymore.
Differential Revision: https://reviews.freebsd.org/D40092
MFC after: 2 month
The Barndinfo emul_path was used by the Elf image activator to fixup
interpreter file name according to ABI root directory. Since the
non-native ABI can now specify its root directory directly to namei()
via pwd_altroot() call this facility is not needed anymore.
Differential Revision: https://reviews.freebsd.org/D40091
MFC after: 2 month
The global variable "pmap_last_pa" was copied from the amd64 pmap as a
part of commit c15085278c "arm64 pmap: implement per-superpage locks"
but it is neither used nor needed by the arm64 pmap.
This avoids bloating the kernel image when MAXCPU is large.
A follow-up patch for kgdb and other kernel debuggers is needed since
the stoppcbs symbol is now a pointer. Bump __FreeBSD_version so that
debuggers can use osreldate to figure out how to handle stoppcbs.
PR: 269572
MFC after: never
Reviewed by: mjg, emaste
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39806
Add disassembly support for the following shifted register instructions:
* adds
* subs
* sub
* neg
* negs
* cmp
* cmn
The 'Mandatory Tokens' checks are relaxed to allow for the alias
instructions (e.g. cmp) which hard-code one or more registers as xzr.
Reviewed by: mhorne
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D40006
Update the few uppercase fields fields to be consistent with others.
Reviewed by: mhorne
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D39820
The GICv3 redistributor register access macros take arguments that are
used as variables. Add brackets around them as is the style, and for
safety.
Sponsored by: Arm Ltd
When checking if the spinlock count is correct in a data abort we can
panic without printing the registers. These are useful to debug the
abort, e.g. by giving the fault address register.
Reviewed by: kib
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D40228
We don't print leading zeros for other registers, so do the same with
ESR_EL1.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D40129
These will be used by bhyve to manage the host VFP registers, e.g.
saving the host state before entering a guest.
Reviewed by: markj
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D40130
These will be used by bhyve
Reviewed by: markj
Sponsored by: Arm Ltd
Sponsored by: Innovate UK
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40128
Introduce macro for PHYS_TO_PTE, setting the groundwork for future
support of various Arm VMSA extensions.
For extensions such as 52-bit VA/PA (FEAT_LPA2), the representation of
an address between a PTE and PA are not equivalent. This macro will
allow converting between the different representations.
Currently PHYS_TO_PTE is a NOP. Replace all instances where we go from
PA to PTE with new PHYS_TO_PTE macro.
Reviewed by: markj
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D39828
Introduce macro for PTE_TO_PHYS, setting the groundwork for future
support of various Arm VMSA extensions.
For extensions such as 52-bit VA/PA (FEAT_LPA2), the representation of
an address between a PTE and PA are not equivalent. This macro will
allow converting between the different representations.
Currently going from PTE to PA is achieved by masking off the upper and
lower attributes. Retain this behaviour but replace all instances with
the new macro instead.
Reviewed by: alc, markj
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D39827
For all PPIs setup earlier than SI_SUB_SMP, PIC_INIT_SECONDARY ends up
cleaning these up for each AP as it comes online. Once they're online,
we don't currently do anything to make sure they're configured for other
APs. Fix it by using smp_rendezvous for the meaty bits of configuring a
PPI, which will just do single-thread behavior before APs are online but
do the right thing for other CPUs after.
While we're here, make sure redistributor config is correct for other
APs as they come online in gic_v3_init_secondary.
Reported/Tested by: Souradeep Chakrabarti (Microsoft/Hyper-V)
Reviewed by: andrew (before slight refactor)
Differential Revision: https://reviews.freebsd.org/D40112
