hwpmc(4): document debugging options

The debug options for hwpmc are not documented in detail anywhere, and
setting it up was error-prone the first time I had to figure it out (and
each time I've had to remember it). Add some explanation of the required
options and describe the kern.hwpmc.debugflags sysctl format.

Reviewed by:	emaste
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D40545

share/man/man4/hwpmc.4

@@ -1,10 +1,12 @@
 .\" Copyright (c) 2003-2008 Joseph Koshy
-.\" Copyright (c) 2007 The FreeBSD Foundation
-.\" All rights reserved.
+.\" Copyright (c) 2007,2023 The FreeBSD Foundation
 .\"
 .\" Portions of this software were developed by A. Joseph Koshy under
 .\" sponsorship from the FreeBSD Foundation and Google, Inc.
 .\"
+.\" Portions of this documentation were written by Mitchell Horne
+.\" under sponsorship from the FreeBSD Foundation.
+.\"
 .\" Redistribution and use in source and binary forms, with or without
 .\" modification, are permitted provided that the following conditions
 .\" are met:
@@ -56,6 +58,17 @@ Alternatively, to compile this driver into the kernel:
 .Bd -ragged -offset indent
 .Cd "device hwpmc"
 .Ed
+.Pp
+To enable debugging features
+.Po see
+.Sx DEBUGGING
+.Pc :
+.Bd -ragged -offset indent
+.Cd "options KTR"
+.Cd "options KTR_COMPILE=(KTR_SUBSYS)"
+.Cd "options KTR_MASK=(KTR_SUBSYS)"
+.Cd "options HWPMC_DEBUG"
+.Ed
 .Sh DESCRIPTION
 The
 .Nm
@@ -520,6 +533,92 @@ The value for tunable
 .Va kern.hwpmc.nsamples
 was negative or greater than 65535.
 .El
+.Sh DEBUGGING
+The
+.Nm
+module can be configured to record trace entries using the
+.Xr ktr 4
+interface.
+This is useful for debugging the driver's functionality, primarily during
+development.
+This debugging functionality is not enabled by default, and requires
+recompiling the kernel and
+.Nm
+module after adding the following to the kernel config:
+.Bd -literal -offset indent
+.Cd options KTR
+.Cd options KTR_COMPILE=(KTR_SUBSYS)
+.Cd options KTR_MASK=(KTR_SUBSYS)
+.Cd options HWPMC_DEBUG
+.Ed
+.Pp
+This alone is not enough to enable tracing; one must also configure the
+.Va kern.hwpmc.debugflags
+.Xr sysctl 8
+variable, which provides fine-grained control over which types of events are
+logged to the trace buffer.
+.Pp
+.Nm
+trace events are grouped by 'major' and 'minor' flag types.
+The major flag groups are as follows:
+.Pp
+.Bl -tag -compact -offset indent
+.It CPU
+CPU events
+.It CSW
+Context switch events
+.It LOG
+Logging events
+.It MDP
+Machine-dependent/class-dependent events
+.It MOD
+Miscellaneous events
+.It OWN
+PMC owner events
+.It PMC
+PMC management events
+.It PRC
+Process events
+.It SAM
+Sampling events
+.El
+.Pp
+The minor flags for each major flag group can vary.
+For the full list of flag groups, consult
+.In sys/pmc.h .
+.Pp
+The
+.Va kern.hwpmc.debugflags
+variable is a string with a custom format.
+The string should contain a space-separated list of event specifiers.
+Each event specifier consists of the major flag name, followed by an equal sign
+(=), followed by a comma-separated list of minor event types.
+To track all events for a major group, an asterisk (*) can be given instead.
+.Pp
+For example, to trace all allocation and release events, set
+.Va debugflags
+as follows:
+.Bd -literal -offset indent
+kern.hwpmc.debugflags="pmc=all,rel mdp=all,rel"
+.Ed
+.Pp
+To trace all process events, as well as context switches:
+.Bd -literal -offset indent
+kern.hwpmc.debugflags="prc=* csw=*"
+.Ed
+.Pp
+To disable all trace events, set the variable to an empty string.
+.Bd -literal -offset indent
+kern.hwpmc.debugflags=""
+.Ed
+.Pp
+Trace events are recorded by
+.Xr ktr 4 ,
+and can be inspected at run-time using the
+.Xr ktrdump 8
+utility, or at the
+.Xr ddb 4
+prompt after a panic with the 'show ktr' command.
 .Sh ERRORS
 A command issued to the
 .Nm
@@ -751,7 +850,10 @@ operation is not being monitored by
 .Xr kenv 1 ,
 .Xr pmc 3 ,
 .Xr pmclog 3 ,
+.Xr ddb 4 ,
+.Xr ktr 4 ,
 .Xr kldload 8 ,
+.Xr ktrdump 8 ,
 .Xr pmccontrol 8 ,
 .Xr pmcstat 8 ,
 .Xr sysctl 8 ,

sys/sys/pmc.h

@@ -1079,7 +1079,11 @@ extern struct pmc_driverstats pmc_stats;
 
 #define	__pmcdbg_used		/* unused variable annotation */
 
-/* debug flags, major flag groups */
+/*
+ * Debug flags, major flag groups.
+ *
+ * Please keep the DEBUGGING section of the hwpmc(4) man page in sync.
+ */
 struct pmc_debugflags {
 	int	pdb_CPU;
 	int	pdb_CSW;