Set the number of artificial frames to 5:
1. cpu_exception_handler_supervisor()
2. do_trap_supervisor()
3. dtrace_invop_start()
4. dtrace_invop()
5. fbt_invop()
Reviewed by: markj
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D37663
Experimentation shows this is the correct value; the dtrace/interrupt
handler frames are omitted, while the backtrace of the active thread is
recorded in its entirety.
Reviewed by: markj
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D37662
Backtraces for fbt probes are missing the caller's frame. Despite what
the inherited comment claims, we do need to insert this manually on
riscv. In fbt_invop(), set cpu_dtrace_caller to be the return address,
not addr.
We should not increment aframes within this function, since we begin the
main loop by unwinding past the current frame.
Plus some very small comment/style tweaks.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D37661
In the common case, kinst emulates a traced instruction by copying it to
a trampoline, where it is followed by a jump back to the original code,
and pointing the interrupted thread's %rip at the trampoline. In
particular, the trampoline is executed with the same CPU context as the
original instruction, so if interrupts are enabled at the point where
the probe fires, they will be enabled when the trampoline is
subsequently executed.
It can happen that an interrupt is raised while a thread is executing a
kinst trampoline. In that case, it is possible that the interrupt
handler will trigger a kinst probe, so we must ensure that the thread
does not recurse and overwrite its trampoline before it is finished
executing the original contents, otherwise an attempt to trace code
called from interrupt handlers can crash the kernel.
To that end, add a per-CPU trampoline, used when the probe fired with
interrupts disabled. Note that this is not quite complete since it does
not handle the possibility of kinst probes firing while executing an NMI
handler.
Also ensure that we do not trace instructions which set IF, since in
that case it is not clear which trampoline (the per-thread trampoline or
the per-CPU trampoline) we should use, and since such instructions are
rare.
Reported and tested by: Domagoj Stolfa
Reviewed by: christos
Fixes: f0bc4ed144 ("kinst: Initial revision")
Differential Revision: https://reviews.freebsd.org/D37619
When a breakpoint exception is raised, the saved value of %rip points to
the instruction following the breakpoint. However, when fetching the
value of %rip using regs[], it's more natural to provide the address of
the breakpoint itself, so modify the kinst and fbt providers accordingly.
Reported by: khng
Reviewed by: christos, khng
MFC after: 2 months
Differential Revision: https://reviews.freebsd.org/D37218
These are fixed, so having upstream's version is not especially useful,
and the duplicated definitions make for confusing reading. No
functional change intended.
MFC after: 1 week
This is a new DTrace provider which allows arbitrary kernel instructions
to be traced. Currently it is implemented only for amd64.
kinst probes are created on demand by libdtrace, and there is a probe
for each kernel instruction. Probes are named
kinst:<module>:<function>:<offset>, where "offset" is the offset of the
target instruction relative to the beginning of the function. Omitting
"offset" causes all instructions in the function to be traced.
kinst works similarly to FBT in that it places a breakpoint on the
target instruction and hooks into the kernel breakpoint handler.
Because kinst has to be able to trace arbitrary instructions, it does
not emulate most of them in software but rather causes the traced thread
to execute a copy of the instruction before returning to the original
code.
The provider is quite low-level and as-is will be useful mostly only to
kernel developers. However, it provides a great deal of visibility into
kernel code execution and could be used as a building block for
higher-level tooling which can in some sense translate between C sources
and generated machine code. In particular, the "regs" variable recently
added to D allows the CPU's register file to be accessed from kinst
probes.
kinst is experimental and should not be used on production systems for
now.
In collaboration with: markj
Sponsored by: Google, Inc. (GSoC 2022)
MFC after: 3 months
Differential Revision: https://reviews.freebsd.org/D36851
This allows invop-based providers (i.e., fbt and kinst) to expose the
register file of the CPU at the point where the probe fired. It does
not work for SDT providers because their probes are implemented as plain
function calls and so don't save registers. It's not clear what
semantics "regs" should have for them anyway.
This is akin to "uregs", which nominally provides access to the
userspace registers. In fact, DIF already had a DIF_VAR_REGS variable
defined, it was simply unimplemented.
Usage example: print the contents of %rdi upon each call to
amd64_syscall():
fbt::amd64_syscall:entry {printf("%x", regs[R_RDI]);}
Note that the R_* constants are defined in /usr/lib/dtrace/regs_x86.d.
Currently there are no similar definitions for non-x86 platforms.
Reviewed by: christos
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D36799
With clang 15, the following -Werror warning is produced:
sys/cddl/dev/dtrace/riscv/dtrace_subr.c:165:17: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
dtrace_gethrtime()
^
void
This is because dtrace_gethrtime() is declared with a (void) argument
list, but defined with an empty argument list. Make the definition match
the declaration.
MFC after: 3 days
With clang 15, the following -Werror warning is produced:
sys/cddl/dev/dtrace/powerpc/dtrace_subr.c:237:17: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
dtrace_gethrtime()
^
void
This is because dtrace_gethrtime() is declared with a (void) argument
list, but defined with an empty argument list. Make the definition match
the declaration.
MFC after: 3 days
With clang 15, the following -Werror warning is produced:
sys/cddl/dev/dtrace/arm/dtrace_subr.c:174:17: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
dtrace_gethrtime()
^
void
This is because dtrace_gethrtime() is declared with a (void) argument
list, but defined with an empty argument list. Make the definition match
the declaration.
MFC after: 3 days
Here, the provider is responsible for updating the trapframe to redirect
control flow and for computing the return address. Once software-saved
registers are restored, the emulation shifts the remaining context down
on the stack to make space for the return address, then copies the
address provided by the invop handler. dtrace_invop() is modified to
allocate temporary storage space on the stack for use by the provider to
return the return address.
This is to support a new provider for amd64 which can instrument
arbitrary instructions, not just function entry and exit instructions as
FBT does.
In collaboration with: christos
Sponsored by: Google, Inc. (GSoC 2022)
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
dtrace invop handlers have access to the whole trapframe, just use that
to extract %rax/%eax for return probes instead of relying on an
additional parameter to the handler. No functional change intended.
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
With clang 15, the following -Werror warning is produced:
sys/cddl/dev/fbt/fbt.c:1273:11: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
fbt_unload()
^
void
This is because fbt_unload() is declared with a (void) argument list,
but defined with an empty argument list. Make the definition match the
declaration.
MFC after: 3 days
With clang 15, the following -Werror warnings is produced:
sys/cddl/dev/prototype.c:99:17: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
prototype_unload()
^
void
This is because prototype_unload() is declared with a (void) argument
list, but defined with an empty argument list. Make the definition match
the declaration.
MFC after: 3 days
With clang 15, the following -Werror warnings is produced:
In file included from sys/cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c:18440:
sys/cddl/dev/dtrace/dtrace_unload.c:26:14: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
dtrace_unload()
^
void
This is because dtrace_unload() is declared with a (void) argument list,
but defined with an empty argument list. Make the definition match the
declaration.
MFC after: 3 days
With clang 15, the following -Werror warnings is produced:
sys/cddl/dev/profile/profile.c:640:15: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
profile_unload()
^
void
This is because profile_unload() is declared with a (void) argument
list, but defined with an empty argument list. Make the definition match
the declaration.
MFC after: 3 days
With clang 15, the following -Werror warnings is produced:
sys/cddl/dev/dtmalloc/dtmalloc.c:177:16: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
dtmalloc_unload()
^
void
This is because dtmalloc_unload() is declared with a (void) argument
list, but defined with an empty argument list. Make the definition match
the declaration.
MFC after: 3 days
The general aim in this and subsequent patches is to minimize the
amount of code that directly references CTF types such as ctf_type_t,
ctf_array_t, etc. To that end, introduce some routines similar to the
existing fbt_get_ctt_size() (which exists to deal with differences
between v1 and v2) and change ctf_lookup_by_id() to return a void
pointer.
Support for v2 containers is preserved.
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34361
The Branch Target Identification (BTI) Armv8-A extension adds new
instructions that can be placed where we may indirrectly branch to,
e.g. at the start of a function called via a function pointer. We can't
emulate these in DTrace as the kernel will have raised a different
exception before the DTrace handler has run.
Skip over the BTI instruction if it's used as the first instruction in
a function.
Sponsored by: The FreeBSD Foundation
We had a hardcoded limit of 1/128-th of physical memory that was further
subdivided between all CPUs as principal buffers are allocated on the
per-CPU basis. Actually, the buffers could use up 1/64-th of the
memmory because with the default switch policy there are two buffers per
CPU.
This commit allows to change that limit.
Note that the discussed limit is per dtrace command invocation.
The idea is to limit the size of a single malloc(9) call, not the total
memory size used by DTrace buffers.
Reviewed by: markj
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D33648
As with arm and riscv fix return fbt probes on arm64. arg0 should be
the offset within the function of the return instruction and arg1
should be the return value.
Reviewed by: kp, markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33440
When writing to memory on arm64 we may be trying to be accessing a
read-only page. In this case try to access via the DMAP region to
get a writable location.
While here simplify writing data in DDB and stop trashing the size as
it is passed into the cache handling functions.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32053
Move the common kernel function signatures from machine/reg.h to a new
sys/reg.h. This is in preperation for adding PT_GETREGSET to ptrace(2).
Reviewed by: imp, markj
Sponsored by: DARPA, AFRL (original work)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D19830
This was ported from illumos but not completely done. Currently we do
not perform type deduplication between KLDs and the kernel, i.e., kernel
modules have a complete type graph. So, remove it for now since it's
not functional and complicates the task of modifying various CTF type
definitions, and we are hitting some limits in the current format which
necessitate an update.
No functional change intended.
MFC after: 2 weeks
In both cases, too few frames were trimmed, leading to exception handling
or DTrace internals being exposed in stack traces exposed by D's stack()
primitive.
MFC after: 3 days
Reviewed by: emaste, andrew
To trace leaf asm functions we can insert a single nop instruction as
the first instruction in a function and trigger off this.
Reviewed by: gnn
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D28132
high-resolution nanosecond timestamp used for the DTrace 'timestamp'
built-in variable. The new implementation uses the EL0 cycle
counter and frequency registers in ARMv8-A. This replaces a
previous implementation that relied on an instrumentation-safe
implementation of getnanotime(), which provided only timer
resolution.
MFC after: 3 days
Reviewed by: andrew, bsdimp (older version)
Useful comments appreciated: jrtc27, emaste
The existing implementation relies on each trap handler saving a normal
stack frame record, which is a waste of time and space when we're
already saving a trapframe to the stack. It's also wrong as it currently
saves LR not ELR.
Instead of patching it up, rewrite it based on the RISC-V implementation
with inspiration from the amd64 implementation for how to handle
vectored traps to provide an improved implementation. This includes
compressing the information down to one line like other architectures
rather than the highly-verbose old form that repeats itself by printing
LR and FP in one frame only to print them as PC and SP in the next. It
also includes printing out actually useful information about the traps
that occurred, though FAR is not saved in the trapframe so we cannot
print it (in general it can be clobbered between when the trap happened
and now), only ESR.
The AAPCS also allows the stack frame record to be located anywhere in
the frame, not just the top, so the caller's SP is not at a fixed offset
from the callee's FP like on almost all other architectures in
existence. This means there is no way to derive the caller's SP in the
unwinder, and so we have to drop that bit of (unused) state everywhere.
Reviewed by: jhb, markj
Differential Revision: https://reviews.freebsd.org/D28026
A more complete fix for this function is being worked on in D28054. Fix
the uninitialized variable error so that builds can at least proceed.
Reported by: several
