Add a virtual timer implementation based on SBI Time extension.
This is needed for Eswin EIC7700 SoC which does not include the newer SSTC
extension.
Timer interrupt pending bit (STIP) could not be cleared in the guest system,
so rework interrupts handling: add new "interrupts_pending" field. Use it
for timer interrupt only for now, but later we can extend to store all
pending interrupts (Timer, IPI and External).
With this I'm able to boot FreeBSD (SMP) guest on HiFive Premier P550,
which is the first real hardware with RISC-V 'H'-spec included.
Differential Revision: https://reviews.freebsd.org/D48133
Add reset controller driver for Eswin EIC7700.
This one has two reset cells in FDT, which is unusual, so provide a
custom hwreset_map method.
Tested on SiFive Premier P550.
Reviewed by: mhorne, jrtc27
Differential Revision: https://reviews.freebsd.org/D47853
Eswin EIC7700 has non-coherent DMAs but predate the standard RISC-V Zicbom
extension, so we need to use the SiFive CCache controller for non-standard
cache management operations.
Tested on SiFive Premier P550.
Reviewed by: mhorne, jrtc27
Differential Revision: https://reviews.freebsd.org/D47831
Check if RISC-V SSTC is available and advertise to the guest.
This is needed for Eswin EIC7700 that does not include SSTC.
As we don't have a mechanism for reporting extension presence
from the kernel to userspace, then use vm_cap_type for now.
Reviewed by: mhorne, markj
Differential Revision: https://reviews.freebsd.org/D48058
This is identical to AON clocks. The only difference is
BUS_PASS_ORDER_LAST which was needed for some reason. This has clocks
needed by PCIe controller driver.
Reviewed by: mhorne
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D47920
Switch to INTR_ROOT_COUNT as this name better describes its purpose.
Remove the default INTR_ROOT_IRQ from the core. Define it (redundantly)
in each architecture's header, but now placed alongside its sibling
values (if defined by the platform, e.g. arm64 INTR_ROOT_FIQ).
Reviewed by: mhorne
Pull Request: https://github.com/freebsd/freebsd-src/pull/1280
In order to match reality, allow using these functions with pointers on
const objects, and bring us closer to C11.
Remove the '+' modifier in the atomic_load_acq_64_i586()'s inline asm
statement's constraint for '*p' (the value to load). CMPXCHG8B always
writes back some value, even when the value exchange does not happen in
which case what was read is written back. atomic_load_acq_64_i586()
further takes care of the operation atomically writing back the same
value that was read in any case. All in all, this makes the inline
asm's write back undetectable by any other code, whether executing on
other CPUs or code on the same CPU before and after the call to
atomic_load_acq_64_i586(), except for the fact that CMPXCHG8B will
trigger a #GP(0) if the memory address is part of a read-only mapping.
This unfortunate property is however out of scope of the C abstract
machine, and in particular independent of whether the 'uint64_t' pointed
to is declared 'const' or not.
Approved by: markj (mentor)
MFC after: 5 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D46887
The GPIO controls the multiplexing of the D1 pins to its peripherals,
so this adds the definitions needed by the aw_gpio driver to support
the D1.
Also, this modifies the aw_gpio driver to support the differences of
the D1 controller:
- pins can have up to 15 functions
- each port is mapped with an alignment of 0x30
- CFG registers have 4 bits per pin
- DRV registers have 4 bits per pin
- the offset of PULL registers is 0x24
Signed-off-by: Julien Cassette <julien.cassette@gmail.com>
Reviewed by: mhorne
Differential Revision: https://reviews.freebsd.org/D35593
Enable the driver and dependencies (aw_sid, nvmem). Add compat strings,
etc.
Found on the Allwinner D1.
Reviewed by: manu
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D47923
Right now, pmap_enter() does not issue an sfence.vma after overwriting
an invalid PTE, so the kernel can trigger a page fault when accessing a
freshly created mapping. In this case, pmap_fault() can handle the
exception, but we may panic before that. Move the check; this is
consistent with arm64 and serves to ensure that we don't call vm_fault()
etc. from a context where that's not permitted.
Also fix a related bug: don't enable interrupts if they were disabled in
the context where the exception occurred.
Reviewed by: br
Tested by: br
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D47688
We can now include sys/intr.h to get these constants rather than using
genassym. Opt for this simpler method.
PR: 283041
Sponsored by: Netflix
Reviewed by: mmel, andrew
Differential Revision: https://reviews.freebsd.org/D47847
Sometimes we need defines from this file in assembler code. Today we do
the heavyweight approach of using genassym for that. However, they are
just #defines, so in the future we want to include sys/intr.h to pick up
the needed constants in exception.S.
PR: 283041
Sponsored by: Netflix
Reviewed by: mmel, andrew
Differential Revision: https://reviews.freebsd.org/D47846
The T-HEAD custom PTE bits are defined in such a way that the
default/normal memory type is non-zero value. This _unthoughtful_ choice
means that, unlike the Svpbmt and non-Svpbmt cases, this field cannot be
left bare in our bootstrap PTEs, or the hardware will fail to proceed
far enough in boot (cache strangeness). On the other hand, we cannot
unconditionally apply the PTE_THEAD_MA_NONE attributes, as this is not
compatible with spec-compliant RISC-V hardware, and will result in a
fatal exception.
Therefore, in order to handle this errata, we are forced to perform a
check of the CPU type at the first moment possible. Do so, and fix up
the PTEs with the correct memory attribute bits in the T-HEAD case.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D47458
Switch the boot argument registers to the unused s3 and s4. This ensures
the values will not be clobbered by SBI or function calls; they are
consumed late in the assembly routine.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D47457
T-HEAD CPUs provide a spec-violating implementation of page-based memory
types, using PTE bits [63:59]. Add basic support for this "errata",
referred to in some places as an "extension".
Note that this change is not enough on its own, but a workaround is
needed for the bootstrap (locore) page tables as well.
Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45472
This is the first major quirk we need to support in order to run on
current T-HEAD/XuanTie CPUs, e.g. the C906 or C910, found in several
existing RISC-V SBCs. With these custom dcache routines installed,
busdma can reliably communicate with devices which are not coherent
w.r.t. the CPU's data caches.
This patch introduces the first quirk/errata handling functions to
identcpu.c, and thus is forced to make some decisions about how this
code is structured. It will be amended with the changes that follow in
the series, yet I feel the final result is (unavoidably) somewhat
clumsy. I expect the CPU identification code will continue to evolve as
more CPUs and their quirks are eventually supported.
Discussed with: jrtc27
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D47455
Cache management operations were, for a long time, unspecified by the
RISC-V ISA, and thus these functions have been no-ops. To cope, hardware
with non-coherent I/O has implemented custom cache flush mechanisms,
either in the form of custom instructions or special device registers.
Additionally, the RISC-V CMO extension is ratified and these official
instructions will start to show up in hardware eventually. Therefore, a
method is needed to select the dcache management routines at runtime.
Add a simple set of function hooks, as well as a routine to install them
and specify the minimum dcache line size. The first consumer will be the
non-standard cache management instructions for T-HEAD CPUs.
The unused I-cache variables and macros are removed.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D47454
For current architectures, these are just aliases for the existing
operation on the relevant scalar integer.
Reviewed by: imp, kib
Obtained from: CheriBSD
Sponsored by: AFRL, DARPA
Differential Revision: https://reviews.freebsd.org/D47631
These use amoor and amoand rather than a loop.
Also define atomic_testandset_acq_(64|long) using amoor.aq.
Reviewed by: mhorne, kib
Sponsored by: AFRL, DARPA
Differential Revision: https://reviews.freebsd.org/D47627
Add the relevant compatible string and build glue. Update the man page
with relevant compat strings.
Reviewed by: manu
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D47516
Add the SOC_ALLWINNER_D1 config option, following other platforms.
Co-authored-by: mhorne
Reviewed by: manu (previous version)
Sponsored by: The FreeBSD Foundation (in part)
Differential Revision: https://reviews.freebsd.org/D47515
For the Allwinner D1 (Nehza) SBC.
This RTC driver is also a clock provider, which registers two fixed
clocks. In all the devices we currently support, the names of the clocks
are present in the "clock-output-names" property of the device tree.
This is not the case for the D1 DTS, as this property does not appear in
upstream. Therefore the clock definitions are statically assigned a
name, which is overridden when specified.
Reviewed by: manu
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D47514
vmmops_modinit() needs to create a device file, and this must happen
after SI_SUB_DEVFS. On non-EARLY_AP_STARTUP platforms (i.e., !x86) this
happens already by accident, but we shouldn't rely on it.
On riscv, remove the current SI_SUB_SMP ordering since that was copied
from arm64 and isn't needed. In particular, riscv's vmmops_modinit()
does not call smp_rendezvous().
Reported by: Oleksandr Kryvulia <shuriku@shurik.kiev.ua>
Fixes: a97f683fe3 ("vmm: Add a device file interface for creating and destroying VMs")
- Make machine/vmm_dev.h self-contained.
- Check for errors from vmmdev_init().
- Make VM_MAX_NAMELEN visible to userspace.
Reported by: Jenkins
Fixes: a97f683fe3 ("vmm: Add a device file interface for creating and destroying VMs")
Add kernel code for 'H' — Hypervisor Extension[1] to support
virtualization on RISC-V ISA.
This comes with a separate userspace patch allowing us to boot
unmodified freebsd/riscv guest. Other operating systems are untested.
This also comes with a U-Boot port that is configured to run in bhyve
guest environment — in RISC-V virtual supervisor mode.
The vmm SBI code emulates RISC-V machine-mode for the guest, handling
SBI calls partly in vmm kernel and partly in bhyve userspace.
Developed in Spike simulator during short period of time, the support
is considered experimental. The first real hardware with hypervisor
spec included should have just reached the market, so this was tested
in Spike and QEMU only. Note that this depends on Sstc extension
presence in the hardware (both Spike and QEMU have it).
Note that booting multiple guests at the same time is not tested and
may require additional work. Some TODOs are indicated within the
code, and some listed in the project's home page[2].
Many thanks to Jessica Clarke, Mitchell Horne and Mark Johnston
for help with parts, test and review.
1. https://riscv.org/technical/specifications/
2. https://wiki.freebsd.org/riscv/bhyve
Sponsored by: UK Research and Innovation
Differential Revision: https://reviews.freebsd.org/D45553
Some device trees report interrupt info in two cells: the first value is
the IRQ number, the second value indicates the trigger type, such as
IRQ_TYPE_LEVEL_HIGH. The device tree for the Allwinner D1(s) is one such
case. Thus, extend the PLIC driver to accept this extra information when
available.
Apparently, some PLIC implementations using edge-triggered interrupts
will require some special handling. This is not required for the D1, and
therefore not implemented in this change. However, to prevent
misbehaviour a check is added to reject this case and a message will be
printed.
Similarly, emit messages for the error paths, e.g. ncells == 3. Drivers
will fail to attach all the same, but the message will aid in diagnosing
the issue more quickly.
Reviewed by: br
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D47135
Otherwise, we are going to take another interrupt-induced exception
immediately upon execution of the sret instruction -- overall wasting
cycles.
Reviewed by: br
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D47134
Be clearer in the intention. 'h' implies a hart ID, but these take
logical CPU IDs instead. 'n' is acceptable, but '_irq' is clearer.
Reviewed by: br
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D47133
This reverts commit 536c8d948e. The
change seemed fine on the surface, but converting to an enum has raised
some concerns due to the asm <-> C interface. Back it out and let
someone else deal with it later if they'd like to.
Further context about the concerns can be found in D47279.
sys/intr.h originally started life as an extract of arm's intr.h, and
this include was dropped in its place. Changes in flight want to add
some MD definitions that we'll use in the more MI parts of INTRNG.
Let's formally reverse the dependency now since this is way more
common in general. All of the includes switched in this change that I
spot-checked were in-fact wanting declarations historically included in
sys/intr.h anyways.
Reviewed by: andrew, imp, jrtc27, mhorne, mmel, olce
Differential Revision: https://reviews.freebsd.org/D47002
uint32_t is handy for directly interfacing with assembly-language. For
the C portion, enum is much handier. In particular there is no need to
count the number of roots by hand. This also works better for being
able to build kernels with varying numbers of roots.
Switch to INTR_ROOT_COUNT as this better matches the purpose of the
value. Switch to root_type, rather than rootnum for similar reasons.
Remove the default from the core. Better to require the architectures
to declare the type since they will routinely deviate and a default
chosen now will likely be suboptimal.
Leave intr_irq_handler() taking a register type as that better matches
for interfacing with assembly-language.
Different types of interrupts may require using different exception
vectors so this commit adds support multiple interrupt roots to handle
these cases. Archs may opt-in to multiple interrupt roots by defining
INTR_ROOT_NUM as the number of roots in their intr.h. Based off
https://reviews.freebsd.org/D40161.
Signed-off-by: Ayrton Munoz <a.munoz3327@gmail.com>
Co-authored-by: Kyle Evans <kevans@FreeBSD.org>
Co-authored-by: Andrew Turner <andrew@FreeBSD.org>
Reviewed-by: imp,mmel,mhorne
Pull-Request: https://github.com/freebsd/freebsd-src/pull/1363
Fix csr_swap() macro so that we don't overwrite the argument (which is not
even possible when the argument is an immediate value)
Reviewed by: jrtc27
Differential Revision: https://reviews.freebsd.org/D46526
Those variables are not going to be changed at runtime. Make them const
to avoid potential overwriting. This will also help spotting accidental
global variables shadowing, since the variable's name such as `version`
is short and commonly used.
This change was inspired by reviewing khng's work D44760.
No functional change intended.
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D45227
This was ratified earlier this year as an alias for Zba_Zbb_Zbs. Whilst
we don't currently export multi-letter extensions, we can still export
this alias in AT_HWCAP.
Reviewed by: mhorne
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D46277
The Svpbmt extension provides specification of "Page-Based Memory
Types", or memory attributes (e.g. cacheability constraints).
Extend the pmap code to apply memory attributes when creating/updating
PTEs. This is done in a way which has no effect on CPUs lacking Svpbmt
support, and is non-hostile to alternate encodings of memory attributes
-- a future change will enable this for T-HEAD CPUs, which implement
this PTE feature in an different (incompatible) way.
Reviewed by: jhb
Tested by: br
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45471
For use in pmap_change_attr_locked(), where we might need to demote L1
pages in the DMAP.
Reviewed by: markj
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45628
This patch modifies pmap_growkernel in all pmaps to use VM_ALLOC_NOFREE
when allocating new pagetable pages. This should help reduce longterm
fragmentation as these pages are never released after
they are allocated.
Differential Revision: https://reviews.freebsd.org/D45998
Reviewed by: alc, markj, kib, mhorne
Tested by: alc
- Remove the IS_SWAPPED thread inhibitor state.
- Remove all uses of TD_IS_SWAPPED() in the kernel.
- Remove the TDF_CANSWAP flag.
- Remove the P_SWAPPINGOUT and P_SWAPPINGIN flags.
Tested by: pho
Reviewed by: alc, imp, kib
Differential Revision: https://reviews.freebsd.org/D46115
As of 9e6544dd6e contigfree(9) is no longer
needed and should not be used anymore. We leave a wrapper for 3rd party
code in at least 15.x but remove (almost) all other cases from the tree.
This leaves one use of contigfree(9) untouched; that was the original
trigger for 9e6544dd6e and is handled in D45813 (to be committed
seperately later).
Sponsored by: The FreeBSD Foundation
Reviewed by: markj, kib
Tested by: pho (10h stress test run)
Differential Revision: https://reviews.freebsd.org/D46099
