Add support for user-supplied callbacks into phys pager operations,
providing custom getpages(), haspage(), and populate() methods
implementations. Pager stores user data ptr/val in the object to
provide context.
Add phys_pager_allocate() helper that takes user ops table as one of
the arguments.
Current code for these methods is moved to the 'default' ops table,
assigned automatically when vm_pager_alloc() is used.
Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D24652
When emulating a single thread system for testing reasons, mp_maxid can
be 0. This trips up our math for calculating the order.
Account for this to fix xive attachment when emulating a single-thread
core on qemu powernv (a configuration that doesn't exist in the real world.)
Sponsored by: Tag1 Consulting, Inc.
Future changes would require additional initialization of OBJT_PHYS
objects, and vm_object_allocate() is not suitable for it.
Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D24652
When doing a build for a modern CPUTYPE, llvm will throw errors if obsolete
instructions are used, even if they will never run due to runtime checks.
Hiding the dssall instruction from the assembler fixes kernel build when
overriding CPUTYPE, without having any effect on the generated binary.
This has been in my local tree for over a year and is well tested across
a variety of machines.
Sponsored by: Tag1 Consulting, Inc.
When enabling an interrupt, assert that we do in fact have a root PIC.
This would have saved me some debugging effort.
Sponsored by: Tag1 Consulting, Inc.
The cryptodev_process() method should either return 0 if it has
completed a request, or ERESTART to defer the request until later. If
a request encounters an error, the error should be reported via
crp_etype before completing the request via crypto_done().
Fix a few more drivers noticed by asomers@ similar to the fix in
r365389. This is an old bug, but went unnoticed since crypto requests
did not start failing as a normal part of operation until digest
verification was introduced which can fail requests with EBADMSG.
PR: 247986
Reported by: asomers
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D26361
There are multiple USB/SATA bridges on the market that unconditionally
cut some LBAs off connected media. This could be a problem
for pre-partitioned drives so GEOM complains and does not create
devices in /dev for slices/partitions preventing access to existing data.
We have kern.geom.part.check_integrity that allows us to correct
partitioning if changed from default 1 to 0 but it works for MBR only.
If backup copy of GPT is unavailable due to decreases number of LBAs,
kernel still does not give access to partitions and prints to dmesg:
GEOM: md0: corrupt or invalid GPT detected.
GEOM: md0: GPT rejected -- may not be recoverable.
This change makes it work for GPT too, so it created partitions in /dev
and prints to dmesg this instead:
GEOM: md0: the secondary GPT table is corrupt or invalid.
GEOM: md0: using the primary only -- recovery suggested.
Then "gpart recover" re-created backup copy of GPT
and allows further manipulations with partitions.
This change is no-op for default configuration having
kern.geom.part.check_integrity=1
Reported by: Alex Korchmar
MFC after: 3 days.
Doing it this way eliminates the assumption about homogeneous support
for the feature, since HWCAP values are only set if support is present
on all CPUs.
Reviewed by: tuexen, markj
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D26032
Expose some of the new HWCAP features added in r65304. This includes the
addition of elf_hwcap2 into the sysvec, and a separate function to parse
for those features.
This only exposes features which require no further configuration, e.g.
indicating the presence of certain instructions. Larger features (SVE)
will not be advertised until we actually support them. The exact list of
features/extensions this patch exposes is:
- ARMv8.0-DGH
- ARMv8.0-SB
- ARMv8.2-BF16
- ARMv8.2-DCCVADP
- ARMv8.2-I8MM
- ARMv8.4-LRCPC
- ARMv8.5-CondM
- ARMv8.5-FRINT
- ARMv8.5-RNG
- PSTATE.SSBS
While here, annotate elf_hwcap and elf_hwcap2 as __read_frequently, and
move the declarations to the machine/md_var.h header.
Submitted by: mikael@ (D22314 portion)
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D26031
Differential Revision: https://reviews.freebsd.org/D22314
FreeBSD exports CPU features as bits in the AT_HWCAP and AT_HWCAP2
vectors via elf_aux_info(3). This interface is similar to getauxval(3)
on Linux, and for simplicity to consumers we try to maintain an
identical set of feature flags on arm64.
The first batch of AT_HWCAP flags were added in r350166, corresponding
to definitions that already existed in Linux. Unfortunately, one flag
was missed, and a portion of the values are shifted one bit to the right
as a result.
Add the missing definition for HWCAP_ASIMDHP, and adjust the affected
values to match their Linux counterparts.
Although this is an ABI-breaking change, there is no plan to provide
compat code for old binaries. An audit of our ports tree and other
software via Debian code search indicates that there are not yet any
consumers of this interface for FreeBSD/arm64.
Bump __FreeBSD_version to be on the safe side, in case compat code needs
to be added in the future.
Reviewed by: emaste, manu
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D26329
There's a race where dying vnets move their interfaces back to their original
vnet, and if_epair cleanup (where deleting one interface also deletes the other
end of the epair). This is commonly triggered by the pf tests, but also by
cleanup of vnet jails.
As we've not yet been able to fix the root cause of the issue work around the
panic by not dereferencing a NULL softc in epair_qflush() and by not
re-attaching DYING interfaces.
This isn't a full fix, but makes a very common panic far less likely.
PR: 244703, 238870
Reviewed by: lutz_donnerhacke.de
MFC after: 4 days
Differential Revision: https://reviews.freebsd.org/D26324
This option was marked as broken because our riscv64-xtoolchain-gcc
package lacked support. Since we are moving away from xtoolchain gcc in
favor of freebsd-gcc9, there should be no issue in enabling this option
by default.
Notably, this enables -Wformat errors.
Reviewed by: kp, jhb
Differential Revision: https://reviews.freebsd.org/D26320
RISC-V is currently built with -Wno-format, which is how these went
undetected. Address them now before re-enabling those warnings.
Differential Revision: https://reviews.freebsd.org/D26319
A PL061 is a simple 8 pin GPIO controller. This GPIO device is used to
signal an internal request for shutdown on some virtual machines including
Arm-based Amazon EC2 instances.
Submitted by: Ali Saidi <alisaidi_amazon.com> (previouss version)
Reviewed by: Ali Saidi, manu
Differential Revision: https://reviews.freebsd.org/D24065
This is a fix to r334065.
Without this change I once got stuck I/O with endless partition switching:
(sdda0:aw_mmc_sim2:0:0:0): sddastart
(sdda0:aw_mmc_sim2:0:0:0): Partition 0 -> -525703168
(sdda0:aw_mmc_sim2:0:0:0): xpt_action: func 0x91d XPT_MMC_IO
(sdda0:aw_mmc_sim2:0:0:0): xpt_done: func= 0x91d XPT_MMC_IO status 0x1
(sdda0:aw_mmc_sim2:0:0:0): sddadone
(sdda0:aw_mmc_sim2:0:0:0): Card status: 00000000
(sdda0:aw_mmc_sim2:0:0:0): Current state: 4
(sdda0:aw_mmc_sim2:0:0:0): Compteting partition switch to 0
Note that -525703168 (an int) is 0xe0aa6800 in binary representation.
The partition indexes are actually stored as uint8_t, so that value
was converted / truncated to zero.
MFC after: 1 week
Implement the remaining pieces needed to allow userland timestamp reading.
Rewritten based on an intial essay into the problem by Justin Hibbits.
(Copyright changed to my own on his request.)
Tested on ppc64 (POWER9 Talos II), powerpcspe (e500v2 RB800), and
powerpc (g4 PowerBook).
Reviewed by: jhibbits (in irc)
Sponsored by: Tag1 Consulting, Inc.
Differential Revision: https://reviews.freebsd.org/D26347
Work around llvm 11 miscompile in 32 bit powerpc that appears to cause ifuncs
to branch to the wrong address by forcing -O2. This worked in previous
versions because -O was mapped to -O2 previously (but is now -O1.)
While here, remove the old temporary workaround from r224882 that does the
opposite thing for powerpc non-DEBUG kernels, bringing it in line with other
platforms that compile at -O2.
This fixes kernel boot on powerpc and powerpcspe after the llvm11 transition.
Sponsored by: Tag1 Consulting, Inc.
orig_object->type can change from OBJT_DEFAULT to OBJT_SWAP while
vm_object_split() is sleeping. In this case some pages in new_object
may be left unbusied, but vm_object_split() attempts to unbusy all of
them.
Track the beginning of the busied range. Add an assertion to verify
that pages are not re-added to the source object while sleeping.
Reported by: Olympios Petrakis <olympios.petrakis@netapp.com>
Reviewed by: alc, kib
Tested by: pho
MFC after: 1 week
Sponsored by: NetApp, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D26223
Similar to the userspace rtld check.
Reviewed by: dim, emaste (previous versions)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D26339
In order to enable VDSO timekeeping, it is necessary that there be exactly
one primary FreeBSD sysvec for each of the host and (optionally) compat32.
So, switch ELFv1 to being a secondary sysvec of ELFv2, so it does not get
double-allocated in the shared page.
Since secondary sysvecs use the same sigcode allocation as the primary,
define both to use the main sigcode64, and adjust the sv_sigcode_base on
ELFv2 after initialization to point to the correct offset.
This has the desirable side effect of avoiding having a separate copy of
the signal trampoline in the shared page. Our sigcode64 was already written
to take advantage of trampoline sharing, it was just not being allocated
that way until now.
Submitted by: jhibbits
Sponsored by: Tag1 Consulting, Inc.
Since x86 is the only 32-bit arch that has a 32-bit time_t, adjust the
private bintime32 struct in vdso to only use a 32 bit sec on amd64.
This matches the existing behavior in the compat code.
Noticed while implementing vdso timekeeping on powerpc.
This should also theoretically fix vdso timekeeping for arm binaries on
aarch64.
See tools attached to https://reviews.freebsd.org/D26347 for testing.
Reviewed by: kib (in irc)
Sponsored by: Tag1 Consulting, Inc.
In r178354 with the introduction of multi-bss ("vap") support factoring
out started and with r193340 ieee80211_media_change() no longer returned
ENETRESET but only 0 or error.
As ieee80211(9) tells the ieee80211_media_change() function should not
be called directly but is registered with ieee80211_vap_attach() instead.
Some drivers have not been fully converted. After fixing the return
checking some of these functions were simply wrappers between
ieee80211_vap_attach() and ieee80211_media_change(), so remove the extra
function, where possible as well.
PR: 248955
Submitted by: Tong Zhang (ztong0001 gmail.com) (original)
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Use the same link-level gateway when adding or deleting interface routes.
This helps nexthop checking in the upcoming multipath changes.
Differential Revision: https://reviews.freebsd.org/D26317
At least on Orange Pi PC Plus even the host mode does not work without
enabling the phy and setting it to the host mode.
The driver will now parse dr_mode property and will try to configure
itself and the phy accordingly.
OTG mode is not supported yet, so it is treated as the device / peripheral
mode.
The phy is enabled -- powered on -- only for the host mode.
The device mode requires support from a phy driver, e.g., aw_usbphy on
Allwinner platform.
aw_usbphy does not support the device mode, so it cannnot work yet.
MFC after: 6 weeks
Otherwise, I get this panic:
panic: awusbdrd_reg: Invalid register 0x342
It looks that musb code both writes and reads at least MUSB2_REG_TXDBDIS.
MFC after: 5 weeks
X-MFC after: r365399
Allwinner USB DRD is based on the Mentor USB OTG controller, with a
different register layout and a few missing registers.
The code is by Andrew Turner (andrew).
Reviewed by: hselasky, manu
Obtained from: andrew
MFC after: 5 weeks
Differential Revision: https://reviews.freebsd.org/D5881
Make that distinction more explicit and regular in the code.
The difference in behavior is documented in the respective datasheets.
Previously, the code handled the distinction by writing the control
register multiple times where at least one write was zero and another
was one.
This can be considered a follow-up to r363021.
Reviewed by: manu
MFC after: 4 weeks
Differential Revision: https://reviews.freebsd.org/D26308
wakeup() does not have any effect on pause(), so if a transfer was
not finished by the time of the first check, then the thread
would sleep full 30 ms.
To do: protect the transfer and interrupt code with the mutex,
switch from tsleep from msleep
Reviewed by: manu
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D26309
abs() takes a (signed) int as input.
Instead, it was used with unsigned 64-bit integers.
So, add and use a new helper function to calculate a difference between
two uint64_t-s.
Reviewed by: manu
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D26307
For the TLS case where there is a "user@domain" name specified in the
X.509 v3 certificate presented by the client in the otherName component
of subjectAltName, a gid list is allocated via mem_alloc().
This needs to be free'd. Otherwise xp_gidp == NULL and free() handles that.
(The size argument to mem_free() is not used by FreeBSD, so it can be 0.)
This leak would not have occurred for any other case than NFS over TLS
with the "user@domain" in the client's certificate.
If multiple threads race calling vfs_hash_insert() while creating vnodes
with the same identity, all of the vnodes which lose the race must be
destroyed before any other thread can see them. Previously this was
accomplished by the vput() in vfs_hash_insert() resulting in the vnode's
VOP_INACTIVE() method calling vgone() before the vnode lock was unlocked,
but at some point changes to the the vnode refcount/inactive logic have caused
that to no longer work, leading to crashes, so instead vfs_hash_insert()
must call vgone() itself before calling vput() on vnodes which lose the race.
Reviewed by: mjg, kib
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D26291
bbr_log_type_hrdwtso() is a file local static unused function.
Remove it to avoid warnings on kernel compiles.
Reviewed by: gallatin
Differential Revision: https://reviews.freebsd.org/D26331
m_segments() was added with r363464 but never used. Remove it to
avoid warnings when compiling kernels.
Reported by: rmacklem (also says jhb)
Reviewed by: gallatin, jhb
Differential Revision: https://reviews.freebsd.org/D26330
The kernel adjusts the stack by TF_SIZE and the RISC-V ABI requires
that it remain 16-byte aligned.
Reported by: CHERI, jrtc27
Reviewed by: mhorne
Obtained from: CheriBSD
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D26328
Investigating a hang I found having some more error information
available would be helpful, so be more verbose and also tell cam/xpt
status in case of error/panic.
Reviewed by: hselasky
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D26287
While debugging a hang I noticed that in case of error in
umass_cam_attach_sim() we miss a cam_sim_free() call.
Added that to not leak resources.
Reviewed by: hselasky
MFC after: 3 days
Differential Revision: D26287
cam_sim_free(), cam_sim_release(), and cam_sim_hold() all assign
a mtx variable during declaration and then if NULL or the mtx is
held may re-asign the variable and/or acquire/release a lock.
Harmonize the code, avoiding double assignments and make it look
the same for all three function (with cam_sim_free() not needing
an extra case).
No functional changes intended.
Reviewed by: imp; no-objections by: mav
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D26286
This package is intended to be used with ice(4) version 0.26.16. That
update will happen in a forthcoming commit.
MFC after: 3 days
Sponsored by: Intel Corporation
When using ifnet ktls, and when ktls_reset_send_tag()
fails to allocate a replacement tag, it leaves
the tls session's snd_tag pointer NULL. ktls_cleanup()
tries to release the send tag, and will trip over
this NULL pointer and panic unless NULL is checked for.
Reviewed by: jhb
Sponsored by: Netflix
This makes it possible to run an unmodified Linux syzkaller executor
against the Linuxulator, and have it gather code coverage information.
Sponsored by: The FreeBSD Foundation
After nexthop introduction, loopback routes for the interface addresses
were created without embedding actual interface index in the gateway.
The latter is needed to pass the IPv6 scope during transmission via loopback..
Fix the regression by actually using passed gateway data with interface index.
Differential Revision: https://reviews.freebsd.org/D26306
I have such a custom kernel configuration and its build failed with:
linking kernel.full
ld: error: undefined symbol: mac_vnode_assert_locked
>>> referenced by mac_framework.h:556 (/usr/devel/git/apu2c4/sys/security/mac/mac_framework.h:556)
>>> tmpfs_vnops.o:(mac_vnode_check_stat)
>>> referenced by mac_framework.h:556 (/usr/devel/git/apu2c4/sys/security/mac/mac_framework.h:556)
>>> vfs_default.o:(mac_vnode_check_stat)
>>> referenced by mac_framework.h:556 (/usr/devel/git/apu2c4/sys/security/mac/mac_framework.h:556)
>>> ufs_vnops.o:(mac_vnode_check_stat)
This is in sync with what is defined for Linux 5.8. Note that all bits
in HWCAP are exhausted, and HWCAP2 has been added.
This also revealed an error in some of the existing definitions. We are
missing HWCAP_ASIMDHP, and as a result a portion of the HWCAP values are
shifted right by one bit. This will be fixed in an upcoming change, but
the values being added now are compatible with what Linux defines.
Reviewed by: emaste, markj, manu
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D26030
We don't need these pagetables after the early boot. Remove the chance we
write to memory we didn't expect to and remove architectural undefined
behaviour.
Reviewed by: alc (earlier version), mmel
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D22606
Fix the build after r365264, I forgot to exclude arm in one more place.
Reported by: rpokala
Approved by: manu (implicit, build fix)
MFC after: 3 days
X-MFC-With: 365264
Pointy-hat to: zeising
mgb had a longstanding typo in function-like macro MGB_NEXT_RING_IDX's
argument, but the macro had no consumers. r365061 introduced a use of
the macro, after which mgb failed to build.
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
While rare, encountering an unimplemented system call early in init is
catastrophic and difficult to debug. Even after a SIGSYS handler is
registered, such configurations are problematic. As such, always report
such events for pid 1 (following kern.lognosys if non-zero).
Reviewed by: kevans, imp
Obtained from: CheriBSD (plus suggestions from kevans)
MFC after: 1 week
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D26288
We were needlessly acquiring the object lock to call
vm_page_grab_pages() even when all of the requested pages were looked up
locklessly. Fix that, stop testing for count == 0 in
vm_page_grab_pages(), and add assertions to help catch this kind of
mistake.
Reported by: cem
Reviewed by: alc, cem, dougm, jeff
Differential Revision: https://reviews.freebsd.org/D26304
hints data. Control register 2 holds the settings a user might want to
configure, such as the timeout value for idle busses and whether to enable
the mass-writes feature.
Also add hint support for disconnecting idle busses (which was already
supported using FDT data).
Update the manpage with the new features, and also split the hints section
into separate lists of required and optional hints.
- Change the type of hw.pagesizes to OPAQUE, since it returns an array.
- Modify the handler to only truncate the returned length if the caller
supplied an output buffer. This allows use of the trick of passing a
NULL output buffer to fetch the output size, while preserving
compatibility if MAXPAGESIZES is increased.
- Add a "S,pagesize" formatter to sysctl(8).
Reviewed by: alc, kib
MFC after: 2 weeks
Sponsored by: Juniper Networks, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D26239
Currently we use a single bit to indicate whether the virtual page is
part of a superpage. To support a forthcoming implementation of
non-transparent 1GB superpages, it is useful to provide more detailed
information about large page sizes.
The change converts MINCORE_SUPER into a mask for MINCORE_PSIND(psind)
values, indicating a mapping of size psind, where psind is an index into
the pagesizes array returned by getpagesizes(3), which in turn comes
from the hw.pagesizes sysctl. MINCORE_PSIND(1) is equal to the old
value of MINCORE_SUPER.
For now, two bits are used to record the page size, permitting values
of MAXPAGESIZES up to 4.
Reviewed by: alc, kib
Sponsored by: Juniper Networks, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D26238
This allows privileged userspace processes to find information about the
physical page backing a given mapping. It is useful in applications
such as DPDK which perform some of their own memory management.
Reviewed by: kib, jhb (previous version)
MFC after: 2 weeks
Sponsored by: Juniper Networks, Inc.
Sponsored by: Klara Inc.
Differential Revision: https://reviews.freebsd.org/D26237
Further improve the drm2 deprecation message, only displaying information
about the port for relevant architectures, and skipping the message
completely from arm, which uses some parts of drm2 still.
This is mostly intended to be merged to 12, since the base bits of drm2 on
FreeBSD 13 are only really used on arm.
Reviewed by: manu, mmel
Approved by: manu
MFC after: 3 days
X-MFC-with: r364737
Differential Revision: https://reviews.freebsd.org/D26275
If the call to _pmap_allocpte() is not sleepable, it is possible that
allocation of PML4 or PDP page is successful but either PDP or PD page
is not. Restructured code in _pmap_allocpte() leaves zero-referenced
page in the paging structure.
Handle it by checking refcount of the page one level above failed
alloc and free that page if its reference count is zero.
Reported and tested by: pho
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D26293
We must enter NET_EPOCH before calling ether_output_frame(). Several of the
functions it calls (pfil_run_hooks, if_transmit) expect to be running in the
NET_EPOCH.
While here remove an unneeded EPOCH entry (which wasn't wide enough to cover
BRIDGE_INPUT).
PR: 248958
Reviewed by: glebius, bz (previous version), melifaro (previous version)
Tested by: manu
Differential Revision: https://reviews.freebsd.org/D26226
Make sure that building dev/sdhci/sdhci_fsl_fdt.c has all the right
dependencies until a proper fix can be made.
Sponsored by: Mellanox Technologies // NVIDIA Networking
The CS_DRAIN flag cannot be set at the same time like the async-drain function
pointer is set. These are orthogonal features. Assert this at the beginning
of the function.
Before:
if (flags & CS_DRAIN) {
/* FALLTHROUGH */
} else if (xxx) {
return yyy;
}
if (drain) {
zzz = drain;
}
After:
if (flags & CS_DRAIN) {
/* FALLTHROUGH */
} else if (xxx) {
return yyy;
} else {
if (drain) {
zzz = drain;
}
}
Reviewed by: markj@
Tested by: callout_test
Differential Revision: https://reviews.freebsd.org/D26285
MFC after: 1 week
Sponsored by: Mellanox Technologies // NVIDIA Networking
This whitespace was intentionally added to help differentiate the different
register groups within this file.
While here add missing whitespace from earlier in the file,
Sponsored by: Innovate UK
Add support for new devices which are capable of 2.5 and 5G speeds, as well as
Energy Efficient Ethernet (EEE):
- introduce new device ids
- add ability to select 2.5 and 5G speeds on devices which support it
- add sysctls to enable EEE and read related statistics
Submitted by: Krzysztof Galazka <krzysztof.galazka@intel.com>
Reviewed by: #IntelNetworking
MFC after: 3 days
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D25549
