Needed by the forthcoming RISC-V and ARM64 ports.
Reviewed by: markj
Approved by: markj (mentor)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40872
Use unprivileged loads to access user memory. Without this, the
accesses trap and various dtrace actions such as ustack() fail.
Reviewed by: andrew
MFC after: 1 week
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D40540
It was used in one place and was added specifically to support dtrace
stack unwinding code. Write an equivalent expression using struct
unwind_state instead. No functional change intended.
Reviewed by: andrew
MFC after: 1 week
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D40538
The forthcoming RISC-V and ARM64 ports of kinst introduce a new field
named "t_kinst_curprobe", so "t_kinst" (which points to a trampoline)
becomes a misleading name.
No functional change intended.
Reviewed by: markj
Approved by: markj (mentor)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40507
match_opcode() is defined in FBT, kinst, and dtrace_subr.c. The function
prologue-checking functions are defined in FBT and kinst.
Reviewed by: markj
Approved by: markj (mentor)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40335
kinst uses this function as well, but because it is not exported, it
implements its own copy of it. The patch also exposes the function to
userland, so programs that need to use dtrace_disx86() can use this
function instead of rolling their own copies.
Reviewed by: markj
Approved by: markj (mentor)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39871
This brings in the following commits:
commit 584b574a3b16c6772c8204ec1d1c957c56f22a87
12174 i86pc: variable may be used uninitialized
Author: Toomas Soome <tsoome@me.com>
Reviewed by: John Levon <john.levon@joyent.com>
Reviewed by: Andrew Stormont <astormont@racktopsystems.com>
Approved by: Dan McDonald <danmcd@joyent.com>
commit a25e615d76804404e5fc63897a9196d4f92c3f5e
12371 dis x86 EVEX prefix mishandled
12372 dis EVEX encoding SIB mishandled
12373 dis support for EVEX vaes instructions
12374 dis support for EVEX vpclmulqdq instructions
12375 dis support for gfni instructions
Author: Robert Mustacchi <rm@fingolfin.org>
Reviewed by: Jerry Jelinek <jerry.jelinek@joyent.com>
Approved by: Joshua M. Clulow <josh@sysmgr.org>
commit c1e9bf00765d7ac9cf1986575e4489dd8710d9b1
12369 dis WBNOINVD support
Author: Robert Mustacchi <rm@joyent.com>
Reviewed by: Hans Rosenfeld <hans.rosenfeld@joyent.com>
Reviewed by: Jerry Jelinek <jerry.jelinek@joyent.com>
Reviewed by: Andy Fiddaman <andy@omniosce.org>
Reviewed by: Toomas Soome <tsoome@me.com>
Approved by: Dan McDonald <danmcd@joyent.com>
commit e4f6ce7088a7dd335b9edf4774325f888692e5fb
10893 Need support for new Cascade Lake Instructions
Author: Robert Mustacchi <rm@joyent.com>
Reviewed by: Hans Rosenfeld <hans.rosenfeld@joyent.com>
Reviewed by: Dan McDonald <danmcd@joyent.com>
Reviewed by: Richard Lowe <richlowe@richlowe.net>
Approved by: Gordon Ross <gwr@nexenta.com>
commit cff040f3ef42d16ae655969398f5a5e6e700b85e
10226 Need support for new EPYC ISA extensions
Author: Robert Mustacchi <rm@joyent.com>
Reviewed by: Hans Rosenfeld <hans.rosenfeld@joyent.com>
Reviewed by: Jason King <jason.king@joyent.com>
Reviewed by: Richard Lowe <richlowe@richlowe.net>
Approved by: Dan McDonald <danmcd@joyent.com>
commit d242cdf5288b86d9070d88791c8ee696612becdc
8492 AVX512 dis - legacy logical instructions
Author: Jerry Jelinek <jerry.jelinek@joyent.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Reviewed by: Gordon Ross <gordon.w.ross@gmail.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
commit 81b505b772ab015c588c56bb116239ee549b6eee
8384 AVX512 dis - EVEX prefix support
8385 32-bit avx dis test mishandles EVEX prefix
8386 32-bit bound dis is incorrect
Author: Jerry Jelinek <jerry.jelinek@joyent.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Reviewed by: Gordon Ross <gordon.w.ross@gmail.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
commit 92381362ae635a3bea638d87b7119f1623b6212e
8319 dis support for new xsave instructions
Author: Jerry Jelinek <jerry.jelinek@joyent.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Reviewed by: Gordon Ross <gordon.w.ross@gmail.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
commit a4e73d5d60e566669c550027fae2b1d87b4be2b4
8240 AVX512 dis - opmask instruction support
Author: Jerry Jelinek <jerry.jelinek@joyent.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Reviewed by: Toomas Soome <tsoome@me.com>
Approved by: Gordon Ross <gordon.w.ross@gmail.com>
959b2dfd39979fe8a9a315a52741d009eb168822
7825 want avx dis tests
7826 PCLMULQDQ psuedo-ops aren't properly described in dis
7827 dis tests for f16c, movbe, cpuid, msr, tsc, fence instrs
7828 sysenter and sysexit dis should be allowed in 64-bit x86
Author: Robert Mustacchi <rm@joyent.com>
Reviewed by: Jerry Jelinek <jerry.jelinek@joyent.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
MFC after: 2 weeks
dtrace_instr_size() is needed by the forthcoming RISC-V port of kinst,
as well as by libdtrace in D38825 for both amd64 and RISC-V.
Reviewed by: markj, mhorne
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39489
Callers are specifying uint8_t anyway and this slightly reduces
dependencies on compatibility typedefs. No functional change intended.
Reviewed by: markj, mhorne
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39490
This will be used by a forthcoming port of the kinst provider.
Reviewed by: markj
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39481
In function boundary tracing the link register is not yet saved to the
save stack location, so the save point contains whatever the previous
'lr' save was, or even garbage, at the time the trap is taken. Address
this by explicitly loading the link register from the trap frame instead
of the stack, and propagate that out.
Pretty trivial following other implementations. The existing
dtrace_getustack_common() does most of the work.
Reviewed by: markj
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D38303
The unwind logic was copied from AArch64 which follows the peculiar
AACPS (where, unlike typical RISC architectures, its frame pointer
follows an x86/stack machine-like convention where the frame pointer
points at the bottom of the frame record, not the top). Delete the
pointless riscv_frame struct and fix this.
Reviewed by: mhorne
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D28054
We must detect the correct amount to increment sepc, as it may have been
a compressed instruction that triggered the fault.
Reviewed by: markj
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D38299
In order to read or write userspace memory without generating an access
fault, we must first enable the SUM bit in the sstatus CSR.
Reviewed by: markj
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D38298
Backtraces for fbt probes are missing the caller's frame. Despite what
the inherited comment claims, we do need to insert this manually on
riscv. In fbt_invop(), set cpu_dtrace_caller to be the return address,
not addr.
We should not increment aframes within this function, since we begin the
main loop by unwinding past the current frame.
Plus some very small comment/style tweaks.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D37661
These are fixed, so having upstream's version is not especially useful,
and the duplicated definitions make for confusing reading. No
functional change intended.
MFC after: 1 week
This is a new DTrace provider which allows arbitrary kernel instructions
to be traced. Currently it is implemented only for amd64.
kinst probes are created on demand by libdtrace, and there is a probe
for each kernel instruction. Probes are named
kinst:<module>:<function>:<offset>, where "offset" is the offset of the
target instruction relative to the beginning of the function. Omitting
"offset" causes all instructions in the function to be traced.
kinst works similarly to FBT in that it places a breakpoint on the
target instruction and hooks into the kernel breakpoint handler.
Because kinst has to be able to trace arbitrary instructions, it does
not emulate most of them in software but rather causes the traced thread
to execute a copy of the instruction before returning to the original
code.
The provider is quite low-level and as-is will be useful mostly only to
kernel developers. However, it provides a great deal of visibility into
kernel code execution and could be used as a building block for
higher-level tooling which can in some sense translate between C sources
and generated machine code. In particular, the "regs" variable recently
added to D allows the CPU's register file to be accessed from kinst
probes.
kinst is experimental and should not be used on production systems for
now.
In collaboration with: markj
Sponsored by: Google, Inc. (GSoC 2022)
MFC after: 3 months
Differential Revision: https://reviews.freebsd.org/D36851
This allows invop-based providers (i.e., fbt and kinst) to expose the
register file of the CPU at the point where the probe fired. It does
not work for SDT providers because their probes are implemented as plain
function calls and so don't save registers. It's not clear what
semantics "regs" should have for them anyway.
This is akin to "uregs", which nominally provides access to the
userspace registers. In fact, DIF already had a DIF_VAR_REGS variable
defined, it was simply unimplemented.
Usage example: print the contents of %rdi upon each call to
amd64_syscall():
fbt::amd64_syscall:entry {printf("%x", regs[R_RDI]);}
Note that the R_* constants are defined in /usr/lib/dtrace/regs_x86.d.
Currently there are no similar definitions for non-x86 platforms.
Reviewed by: christos
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D36799
With clang 15, the following -Werror warning is produced:
sys/cddl/dev/dtrace/riscv/dtrace_subr.c:165:17: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
dtrace_gethrtime()
^
void
This is because dtrace_gethrtime() is declared with a (void) argument
list, but defined with an empty argument list. Make the definition match
the declaration.
MFC after: 3 days
With clang 15, the following -Werror warning is produced:
sys/cddl/dev/dtrace/powerpc/dtrace_subr.c:237:17: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
dtrace_gethrtime()
^
void
This is because dtrace_gethrtime() is declared with a (void) argument
list, but defined with an empty argument list. Make the definition match
the declaration.
MFC after: 3 days
With clang 15, the following -Werror warning is produced:
sys/cddl/dev/dtrace/arm/dtrace_subr.c:174:17: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
dtrace_gethrtime()
^
void
This is because dtrace_gethrtime() is declared with a (void) argument
list, but defined with an empty argument list. Make the definition match
the declaration.
MFC after: 3 days
Here, the provider is responsible for updating the trapframe to redirect
control flow and for computing the return address. Once software-saved
registers are restored, the emulation shifts the remaining context down
on the stack to make space for the return address, then copies the
address provided by the invop handler. dtrace_invop() is modified to
allocate temporary storage space on the stack for use by the provider to
return the return address.
This is to support a new provider for amd64 which can instrument
arbitrary instructions, not just function entry and exit instructions as
FBT does.
In collaboration with: christos
Sponsored by: Google, Inc. (GSoC 2022)
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
With clang 15, the following -Werror warnings is produced:
In file included from sys/cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c:18440:
sys/cddl/dev/dtrace/dtrace_unload.c:26:14: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
dtrace_unload()
^
void
This is because dtrace_unload() is declared with a (void) argument list,
but defined with an empty argument list. Make the definition match the
declaration.
MFC after: 3 days
We had a hardcoded limit of 1/128-th of physical memory that was further
subdivided between all CPUs as principal buffers are allocated on the
per-CPU basis. Actually, the buffers could use up 1/64-th of the
memmory because with the default switch policy there are two buffers per
CPU.
This commit allows to change that limit.
Note that the discussed limit is per dtrace command invocation.
The idea is to limit the size of a single malloc(9) call, not the total
memory size used by DTrace buffers.
Reviewed by: markj
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D33648
As with arm and riscv fix return fbt probes on arm64. arg0 should be
the offset within the function of the return instruction and arg1
should be the return value.
Reviewed by: kp, markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33440
Move the common kernel function signatures from machine/reg.h to a new
sys/reg.h. This is in preperation for adding PT_GETREGSET to ptrace(2).
Reviewed by: imp, markj
Sponsored by: DARPA, AFRL (original work)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D19830
To trace leaf asm functions we can insert a single nop instruction as
the first instruction in a function and trigger off this.
Reviewed by: gnn
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D28132
high-resolution nanosecond timestamp used for the DTrace 'timestamp'
built-in variable. The new implementation uses the EL0 cycle
counter and frequency registers in ARMv8-A. This replaces a
previous implementation that relied on an instrumentation-safe
implementation of getnanotime(), which provided only timer
resolution.
MFC after: 3 days
Reviewed by: andrew, bsdimp (older version)
Useful comments appreciated: jrtc27, emaste
The existing implementation relies on each trap handler saving a normal
stack frame record, which is a waste of time and space when we're
already saving a trapframe to the stack. It's also wrong as it currently
saves LR not ELR.
Instead of patching it up, rewrite it based on the RISC-V implementation
with inspiration from the amd64 implementation for how to handle
vectored traps to provide an improved implementation. This includes
compressing the information down to one line like other architectures
rather than the highly-verbose old form that repeats itself by printing
LR and FP in one frame only to print them as PC and SP in the next. It
also includes printing out actually useful information about the traps
that occurred, though FAR is not saved in the trapframe so we cannot
print it (in general it can be clobbered between when the trap happened
and now), only ESR.
The AAPCS also allows the stack frame record to be located anywhere in
the frame, not just the top, so the caller's SP is not at a fixed offset
from the callee's FP like on almost all other architectures in
existence. This means there is no way to derive the caller's SP in the
unwinder, and so we have to drop that bit of (unused) state everywhere.
Reviewed by: jhb, markj
Differential Revision: https://reviews.freebsd.org/D28026
A more complete fix for this function is being worked on in D28054. Fix
the uninitialized variable error so that builds can at least proceed.
Reported by: several
Some stack frames are too large for a store pair instruction we already
detect in the arm64 fbt code. Add support for handling subtracting the
stack pointer directly.
Sponsored by: Innovate UK
- Implement a dtrace_getnanouptime(), matching the existing
dtrace_getnanotime(), to avoid DTrace calling out to a potentially
instrumentable function.
(These should probably both be under KDTRACE_HOOKS. Also, it's not clear
to me that they are correct implementations for the DTrace thread time
functions they are used in .. fixes for another commit.)
- Don't allow FBT to instrument functions involved in EL1 exception handling
that are involved in FBT trap processing: handle_el1h_sync() and
do_el1h_sync().
- Don't allow FBT to instrument DDB and KDB functions, as that makes it
rather harder to debug FBT problems.
Prior to these changes, use of FBT on FreeBSD/arm64 rapidly led to kernel
panics due to recursion in DTrace.
Reliable FBT on FreeBSD/arm64 is reliant on another change from @andrew to
have the aarch64 instrumentor more carefully check that instructions it
replaces are against the stack pointer, which can otherwise lead to memory
corruption. That change remains under review.
MFC after: 2 weeks
Reviewed by: andrew, kp, markj (earlier version), jrtc27 (earlier version)
Differential revision: https://reviews.freebsd.org/D27766
This same check is used on other architectures. Previously this would
permit a stack frame to unwind into any arbitrary kernel address
(including unmapped addresses).
Reviewed by: andrew, markj
Obtained from: CheriBSD
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D27362
- Push the kstack_contains check down into unwind_frame() so that it
is honored by DDB and DTrace.
- Check that the trapframe for an exception frame is contained in the
traced thread's kernel stack for DDB traces.
Reviewed by: markj
Obtained from: CheriBSD
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D27357
