A rarely occurring event (e.g. an event that occurs less than 1000
times during execution of a program) may require a lower minimum
threshold than 1000. Replace the hardcoded 1000 with a sysctl that
the administrator can use to permit smaller sampling count values.
Reviewed by: mhorne, mav
Sponsored by: University of Cambridge, Google, Inc.
Differential Revision: https://reviews.freebsd.org/D35400
This allows to profile already running high-priority threads, that
otherwise by blocking thread migration to respective CPUs blocked PMC
management, i.e. profiling could start only when workload completed.
While there, return the thread to its original CPU after iterating
the list. Otherwise all threads using PMC end up on the last CPU.
MFC after: 1 month
After we wipe PMC configuration, including its local enable bit(s),
we don't really care about its global enable bit. Global enable bits
now may only be cleared by interrupt handler in case of error (sample
buffer overflow). Being set is actually a reset default for them.
This saves one WRMSR per process-scope PMC per context switch, that
is clearly visible in profiles.
MFC after: 1 month
The PMC subsystem is not designed for non-uniform CPU capabilities
(P/E-cores are different), but at least several working architectural
events like cpu_clk_unhalted.thread_p should be better than nothing.
MFC after: 1 month
Intel json's use event=0 to specify fixed counter number via umask.
Alternatively fixed counters have equivalent programmable event/umask.
MFC after: 1 month
Since version 2 Intel CPUs can freeze PMCs when intering PMI to reduce
PMI effects on collected statistics. Since version 4 hardware supports
"streamlined" mechanism, not requiring IA_GLOBAL_CTRL MSR access.
MFC after: 1 month
Support the "usr" and "os" qualifiers on arm64 events to restrict
event counting to either usermode or the kernel, respectively. If
neither qualifier is given, events are counted in both.
Reviewed by: emaste
Sponsored by: University of Cambridge, Google, Inc.
Differential Revision: https://reviews.freebsd.org/D34527
If a counter more than overflows just as we read it on switch out then,
if using sampling mode, we will negate this small value to give a huge
reload count, and if we later switch back in that context we will
validate that value against pm_reloadcount and panic an INVARIANTS
kernel with:
panic: [pmc,1470] pmcval outside of expected range cpu=2 ri=16 pmcval=fffff292 pm_reloadcount=10000
or similar. Presumably in a non-INVARIANTS kernel we will instead just
use the provided value as the reload count, which would lead to the
overflow not happing for a very long time (e.g. 78 minutes for a 48-bit
counter incrementing at an averate rate of 1GHz).
Instead, clamp the reload count to 0 (which corresponds precisely to the
value we would compute if it had just overflowed and no more), which
will result in hwpmc using the full original reload count again. This is
the approach used by core for Intel (for both fixed and programmable
counters).
As part of this, armv7 and arm64 are made conceptually simpler; rather
than skipping modifying the overflow count for sampling mode counters so
it's always kept as ~0, those special cases are removed so it's always
applicable and the concatentation of it and the hardware counter can
always be viewed as a 64-bit counter, which also makes them look more
like other architectures.
Whilst here, fix an instance of UB (shifting a 1 into the sign bit) for
amd in its sign-extension code.
Reviewed by: andrew, mhorne, kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D33654
Performance counters and overflow interrupts are assumed to be disabled
by default, but this is not guaranteed. Ensure we disable both during
per-cpu initialization, before enabling the PMU. Otherwise, some systems
(such as the Ampere eMAG) would experience an interrupt storm upon
loading the hwpmc module.
Reviewed by: br
MFC after: 5 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32854
Looking for "tsc-tsc" in the pmu tables will fail every time. Instead,
make this an alias for the static TSC event defined in pmc_events.h.
This fixes 'pmcstat -s cycles' on Intel and AMD.
Reviewed by: emaste
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32197
These checks were inconsistently applied across the various hwpmc
classes. The condition is already checked by the generic code in
hwpmc_mod.c, so remove them.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31388
It is always a good idea. In one case, attempting to allocate N+1 PMCs
from a class with N hardware counters would incorrectly attempt to
allocate from the next class in the list. Without this validation, this
can lead to all kinds of strange behaviour.
Since powerpc_allocate_pmc() is used by both the mpc7xxx and ppc970
classes, add a new global to track which is active (it will never be
both).
Reviewed by: luporl, ray
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31387
This variable is set based on the exact CPU model detected. If this
value is set too small, it could lead to a NULL-dereference from an
improperly initialized pmc_rowindex_to_classdep array.
Though it has been fixed, this was previously the case for Broadwell.
Add two asserts to catch this in DEBUG kernels, as it represents a
configuration error that may be hard to uncover otherwise.
PR: 253687
Reported by: Zhenlei Huang <zlei.huang@gmail.com>
Sponsored by: The FreeBSD Foundation
It was written for Nehalem and Westmere, with minor but incomplete
updates for Sandy Bridge in 78d763a29b. The uncore architecture
changed significantly with this generation, bringing new layouts and
locations for some MSRs.
Misprogramming these MSRs in ucp_start_pmc() may panic the system, and
this is trivially reproducible via pmcstat(8) on at least Broadwell and
Haswell. Disable the class on these CPUs until it can be updated more
completely and leave a TODO comment detailing some of the work required.
Note that the nclasses value for Broadwell was already incorrect and
doesn't need changing.
The result is that any uncore events listed by pmcstat -L will no longer
be allocatable, but this is already the case for newer generations of
Intel CPUs.
PR: 253687
Reported by: Zhenlei Huang <zlei.huang@gmail.com>
Reviewed by: kib
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31389
After b48a2770d4, static POWER8 definitions became unnecessary,
as all of them (and much more) are already present in libpmc's
PMU events.
Submitted by: Leonardo Bianconi <leonardo.bianconi@eldorado.org.br> (initial version)
Reviewed by: kbowling, mhorne
Sponsored by: Instituto de Pesquisas Eldorado (eldorado.org.br)
Differential Revision: https://reviews.freebsd.org/D31334
As the Processor Version Register (PVR) is a 32-bit PowerPC
register, change mfpvr() return type to match it and avoid
type casts on its callers.
Suggested by: jhibbits
Reviewed by: jhibbits, imp
Sponsored by: Instituto de Pesquisas Eldorado (eldorado.org.br)
Differential Revision: https://reviews.freebsd.org/D31332
Due to a mis-merge, the changes committed to libpmc never called
pmu_parse_event(), or set pm->pm_ev. However, this field shouldn't be
used to carry the actual pmc event code anyway, as it is expected to
contain the index into the pmu event array (otherwise, it breaks event
name lookup in pmclog_get_event()). Add a new MD field,
pm_md.pm_md_config, to pass the raw event code to arm64_allocate_pmc().
Additionally, the change made to pmc_md_op_pmcallocate was incorrect, as
this is a union, not a struct. Restore the proper padding size.
Reviewed by: luporl, ray, andrew
Fixes: 28dd6730a5 ("libpmc: enable pmu_utils on arm64")
Fixes: 8cc3815f02 ("hwpmc_arm64: accept raw event codes...")
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31221
As in commit 831850d8b0, this routine can trigger false positives, so
exclude it from instrumentation.
Reported by: pho
Sponsored by: The FreeBSD Foundation
Make it possible to specify event codes without an offset of
PMC_EV_ARMV8_FIRST, by setting a machine-dependent flag. This is
required to make use of event definitions from pmu-events.
Reviewed by: ray (slightly earlier version)
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D30602
This will be used to detect supported pmu events. The expected format is
the MIDR register with the revision and variant fields masked. See also:
lib/libpmc/pmu-events/arch/arm64/mapfile.csv.
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D30601
The lower pmc event bits were masked off to find the PMC event ID.
The doesn't work when there are more events. Switch it to use the
offser relative to the first event while also checking the ID is
in the expected range.
Reviewed by: gnn, ray
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D29600
Virtual PMCs could be running on multiple CPUs so this needs to be
a per-CPU value.
Submitted by: rwatson (earlier version)
Reviewed by: gnn
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D27973
When testing hwpmc on arm64 we found the counter could overflow while
reading the event count. Handle this case in the armv7 code by also
checking if the overflow bit is set and incrementing the overflow
cound as needed.
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D27969
