The current read from network is working from up to down - we have some
protocol needing the data from the network, so we build the buffer space
for that protocol, add the extra space for headers and pass this buffer
down to be filled by nif get call in hope, we have guessed the incoming
packet size right. Amazingly enough this approach mostly does work, but
not always...
So, this update does work from down to up - we allocate buffer (based
on MTU or frame size info), fill it up, and pass on for upper layers.
The obvious problem is that when we should free the buffer - if at all.
In the current implementation the upper layer will free the packet on error
or when the packet is no longer needed.
While working on the issue, the additional issue did pop up - the bios
implementation does not have generic get/put interface but is using pxe
udpsend/udpreceive instead. So the udp calls are gone and undi interface
is implemented instead. Which in turn means slight other changes as we
do not need to have duplicated pxe implementation and can just use dev_net.
To align packet content, the actual read from nic is using shifted buffer by
ETHER_ALIGN (2).
Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D10232
The current zfs reader is only checking first label from each device, however,
we do have 4 labels on device and we should check all 4 to be protected
against disk failures and incomplete label updates.
The difficulty is about the fact that 2 label copies are in front of the
pool data, and 2 are at the end, which means, we have to know the size of
the pool data area.
Since we have now the mechanism from common/disk.c to use the partition
information, it does help us in this task; however, there are still some
corner cases.
Namely, if the pool is created without partition, directly on the disk,
and firmware will give us the wrong size for the disk, we only can check
the first two label copies.
Reviewed by: allanjude
Differential Revision: https://reviews.freebsd.org/D10203
This patch adds a general mechanism for providing encryption keys to the
kernel from the boot loader. This is intended to enable GELI support at
boot time, providing a better mechanism for passing keys to the kernel
than environment variables. It is designed to be extensible to other
applications, and can easily handle multiple encrypted volumes with
different keys.
This mechanism is currently used by the pending GELI EFI work.
Additionally, this mechanism can potentially be used to interface with
GRUB, opening up options for coreboot+GRUB configurations with completely
encrypted disks.
Another benefit over the existing system is that it does not require
re-deriving the user key from the password at each boot stage.
Most of this patch was written by Eric McCorkle. It was extended by
Allan Jude with a number of minor enhancements and extending the keybuf
feature into boot2.
GELI user keys are now derived once, in boot2, then passed to the loader,
which reuses the key, then passes it to the kernel, where the GELI module
destroys the keybuf after decrypting the volumes.
Submitted by: Eric McCorkle <eric@metricspace.net> (Original Version)
Reviewed by: oshogbo (earlier version), cem (earlier version)
MFC after: 3 weeks
Relnotes: yes
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D9575
Make sure sensitive memory is properly cleared when finished with it
Reviewed by: Eric McCorkle <eric@metricspace.net>
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D9798
It is simpler and cleaner to have strip produce the stripped output
directly than copying the input to the output first.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
The Makefile rule to create vers.c for loader version info was
previously duplicated in each of the various loader Makefiles.
Instead, share a common rule in Makefile.inc.
Reviewed by: bapt
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8841
Instead of repeating "%s, Revision %s" "(%s %s)" in each loader, just
create the full version string in vers.c
Reviewed by: bapt
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8823
FICL definitions not in ficl/ficl32 files broke this generally. This
makes that stuff conditional on BOOT_FORTH. Also, move definitions
related to the architecture (FICL_CPUARCH and friends) into
Makefile.ficl that all parts of the tree that include files with ficl
need to include (but only if MK_FORTH == yes). In addition, had to fix
library ordering issue with LIBSTAND to keep it last. Without boot
forth, there's no references to memset to bring in memset.o from
libstand.a to satisfy libgeliboot.a's use of it. Listing libstand last
solves this issue (and it's the proper place for libstand to boot).
Add a new 'netproto' variable which can be set for now to
NET_TFTP or NET_NFS (default to NET_NONE)
From the dhcp options if one sets the root-path option to:
"ip:path", the loader will consider it is booting over NFS
(meaning same behaviour as the default current behaviour)
if the dhcp option "tftp server address" is set (option 150)
the loader will consider it is booting over tftpfs, it will then
consider the root-path options with 2 possible case
1. "path" then the IP of the tftp server will be the one passed by
the option 150, and the files will be retrieved under "path" on the tftp
server
2. "ip:path" then the IP of the tftp server will be the one passed in
the option "overwritting the IP from the option 150.
We could not "abuse" the rootpath option in the form or tftp://ip:path because
this is already used for other purpose by iPXE preventing any chainload from
iPXE to the FreeBSD loader.
Given at each open(), the loader loops over all available filesystems and keep
the "best" error, we needed to prevent tftpfs to fallback on nfs and vice versa.
the tftpfs and nfs implementation in libstand now return EINVAL early if
'netproto' for that purpose.
Reviewed by: tsoome
Relnotes: yes
Sponsored by: Gandi.net
Differential Revision: https://reviews.freebsd.org/D7628
The block cache implementation in loader has proven to be almost useless, and in worst case even slowing down the disk reads due to insufficient cache size and extra memory copy.
Also the current cache implementation does not cache reads from CDs, or work with zfs built on top of multiple disks.
Instead of an LRU, this code uses a simple hash (O(1) read from cache), and instead of a single global cache, a separate cache per block device.
The cache also implements limited read-ahead to increase performance.
To simplify read ahead management, the read ahead will not wrap over bcache end, so in worst case, single block physical read will be performed to fill the last block in bcache.
Booting from a virtual CD over IPMI:
0ms latency, before: 27 second, after: 7 seconds
60ms latency, before: over 12 minutes, after: under 5 minutes.
Submitted by: Toomas Soome <tsoome@me.com>
Reviewed by: delphij (previous version), emaste (previous version)
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D4713
These are no longer needed after the recent 'beforebuild: depend' changes
and hooking DIRDEPS_BUILD into a subset of FAST_DEPEND which supports
skipping 'make depend'.
Sponsored by: EMC / Isilon Storage Division
When called to early, new_currdev->d_type was not yet set
zfs_fmtdev() would then return null
While here, guard call to init_zfs_bootenv with if d_type == DEVT_ZFS
Reported by: tsoome at me.com
MFC after: 3 days
Sponsored by: ScaleEngine Inc.
Add a few other safeguards to ensure things do not break when the
boot device cannot be determined
Reported by: flo
MFC after: 3 days
Sponsored by: ScaleEngine Inc.
If the system was booted with ZFS, a new menu item (#7) appears
It contains an autogenerated list of ZFS Boot Environments
This allows the user to switch to an alternate root file system
Use Cases:
- Revert a failed upgrade
- Concurrently run different versions of FreeBSD with common home directory
- Easier integration with the sysadmin/beadm utility
Requested by: many
Reviewed by: dteske
MFC after: 10 days
Relnotes: yes
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D3167
pxeboot in tftp loader mode (when built with LOADER_TFTP_SUPPORT) now
prefix all the path to open with the path obtained via the option 'root-path'
directive.
This allows to be able to use the traditional content /boot out of box. Meaning
it now works pretty much like all other loaders. It simplifies hosting hosting
multiple version of FreeBSD on a tftp server.
As a consequence, pxeboot does not look anymore for a pxeboot.4th (which was
never provided)
Note: that pxeboot in tftp loader mode is not built by default.
Reviewed by: rpokala
Relnotes: yes
Sponsored by: Gandi.net
Differential Revision: https://reviews.freebsd.org/D4590
This is not properly respecting WITHOUT or ARCH dependencies in target/.
Doing so requires a massive effort to rework targets/ to do so. A
better approach will be to either include the SUBDIR Makefiles directly
and map to DIRDEPS or just dynamically lookup the SUBDIR. These lose
the benefit of having a userland/lib, userland/libexec, etc, though and
results in a massive package. The current implementation of targets/ is
very unmaintainable.
Currently rescue/rescue and sys/modules are still not connected.
Sponsored by: EMC / Isilon Storage Division
There have been .local version of each for user modifications for some time
This allows users to receive future updates to these files
PR: 183765
Submitted by: Bertram Scharpf, Nikolai Lifanov (patch)
Reviewed by: dteske, loos, eadler
Approved by: bapt (mentor)
MFC after: 1 month
Relnotes: yes
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D3176
Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.
Still need to add real targets under targets/ to build packages.
Differential Revision: D2796
Reviewed by: brooks imp
Implement a subset of the multiboot specification in order to boot Xen
and a FreeBSD Dom0 from the FreeBSD bootloader. This multiboot
implementation is tailored to boot Xen and FreeBSD Dom0, and it will
most surely fail to boot any other multiboot compilant kernel.
In order to detect and boot the Xen microkernel, two new file formats
are added to the bootloader, multiboot and multiboot_obj. Multiboot
support must be tested before regular ELF support, since Xen is a
multiboot kernel that also uses ELF. After a multiboot kernel is
detected, all the other loaded kernels/modules are parsed by the
multiboot_obj format.
The layout of the loaded objects in memory is the following; first the
Xen kernel is loaded as a 32bit ELF into memory (Xen will switch to
long mode by itself), after that the FreeBSD kernel is loaded as a RAW
file (Xen will parse and load it using it's internal ELF loader), and
finally the metadata and the modules are loaded using the native
FreeBSD way. After everything is loaded we jump into Xen's entry point
using a small trampoline. The order of the multiboot modules passed to
Xen is the following, the first module is the RAW FreeBSD kernel, and
the second module is the metadata and the FreeBSD modules.
Since Xen will relocate the memory position of the second
multiboot module (the one that contains the metadata and native
FreeBSD modules), we need to stash the original modulep address inside
of the metadata itself in order to recalculate its position once
booted. This also means the metadata must come before the loaded
modules, so after loading the FreeBSD kernel a portion of memory is
reserved in order to place the metadata before booting.
In order to tell the loader to boot Xen and then the FreeBSD kernel the
following has to be added to the /boot/loader.conf file:
xen_cmdline="dom0_mem=1024M dom0_max_vcpus=2 dom0pvh=1 console=com1,vga"
xen_kernel="/boot/xen"
The first argument contains the command line that will be passed to the Xen
kernel, while the second argument is the path to the Xen kernel itself. This
can also be done manually from the loader command line, by for example
typing the following set of commands:
OK unload
OK load /boot/xen dom0_mem=1024M dom0_max_vcpus=2 dom0pvh=1 console=com1,vga
OK load kernel
OK load zfs
OK load if_tap
OK load ...
OK boot
Sponsored by: Citrix Systems R&D
Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D517
For the Forth bits:
Submitted by: Julien Grall <julien.grall AT citrix.com>
gzip, and split). "Real" filesystems should always be listed first so
that the "bare" filename is tried before alternate filenames. For PXE
booting in particular this can remove a lot of spurious pathname lookups.
While here, move splitfs to the bottom after the bzip and gzip filesystems
as it is the least often used.
Tested by: Prokash Sinha <psinha@panasas.com>
MFC after: 1 week
bootloader. Implement the following routines:
pcibios-device-count count the number of instances of a devid
pcibios-read-config read pci config space
pcibios-write-config write pci config space
pcibios-find-devclass find the nth device with a given devclass
pcibios-find-device find the nth device with a given devid
pcibios-locator convert bus device function ti pcibios locator
These commands are thin wrappers over their PCI BIOS 2.1 counterparts. More
informaiton, such as it is, can be found in the standard.
Export a nunmber of pcibios.X variables into the environment to report
what the PCI IDENTIFY command returned.
Also implmenet a new command line primitive (pci-device-count), but don't
include it by default just yet, since it depends on the recently added
words and any errors here can render a system unbootable.
This is intended to allow the boot loader to do special things based
on the hardware it finds. This could be have special settings that are
optimized for the specific cards, or even loading special drivers. It
goes without saying that writing to pci config space should not be
done without a just cause and a sound mind.
Sponsored by: Netflix
1. 50+% of NO_PIE use is fixed by adding -fPIC to INTERNALLIB and other
build-only utility libraries.
2. Another 40% is fixed by generating _pic.a variants of various libraries.
3. Some of the NO_PIE use is a bit absurd as it is disabling PIE (and ASLR)
where it never would work anyhow, such as csu or loader. This suggests
there may be better ways of adding support to the tree. Many of these
cases can be fixed such that -fPIE will work but there is really no
reason to have it in those cases.
4. Some of the uses are working around hacks done to some Makefiles that are
really building libraries but have been using bsd.prog.mk because the code
is cleaner. Had they been using bsd.lib.mk then NO_PIE would not have
been needed.
We likely do want to enable PIE by default (opt-out) for non-tree consumers
(such as ports). For in-tree though we probably want to only enable PIE
(opt-in) for common attack targets such as remote service daemons and setuid
utilities. This is also a great performance compromise since ASLR is expected
to reduce performance. As such it does not make sense to enable it in all
utilities such as ls(1) that have little benefit to having it enabled.
Reported by: kib
particular, allow loaders to define the name of the RC script the
interpreter needs to use. Use this new-found control to have the
PXE loader (when compiled with TFTP support and not NFS support)
read from ${bootfile}.4th, where ${bootfile} is the name of the
file fetched by the PXE firmware.
The normal startup process involves reading the following files:
1. /boot/boot.4th
2. /boot/loader.rc or alternatively /boot/boot.conf
When these come from a FreeBSD-defined file system, this is all
good. But when we boot over the network, subdirectories and fixed
file names are often painful to administrators and there's really
no way for them to change the behaviour of the loader.
Obtained from: Juniper Networks, Inc.
This is currently an opt-in build flag. Once ASLR support is ready and stable
it should changed to opt-out and be enabled by default along with ASLR.
Each application Makefile uses opt-out to ensure that ASLR will be enabled by
default in new directories when the system is compiled with PIE/ASLR. [2]
Mark known build failures as NO_PIE for now.
The only known runtime failure was rtld.
[1] http://www.bsdcan.org/2014/schedule/events/452.en.html
Submitted by: Shawn Webb <lattera@gmail.com>
Discussed between: des@ and Shawn Webb [2]
The 32-bit bootloaders on amd64 now use the 32-bit version in ficl32,
as is done with libstand32. The native 64-bit ficl will be used by the
upcoming UEFI loader.
Sponsored by: The FreeBSD Foundation
A 32-bit libstand is needed on 64-bit platforms for use by various
bootloaders. Previously only the 32-bit version was built, installed as
/usr/lib/libstand.a.
A new 64-bit libstand consumer will arrive in the near future, so move
the bootloader-specific 32-bit version to sys/boot/libstand32/.
Explicitly link against this version in the 32-bit loaders.
Sponsored by: The FreeBSD Foundation
conditionally include (but ignore failures) /boot/loader.rc.local and
/boot/menu.rc.local -- to make customizing the menu easier.
Reviewed by: alfred
Discussed on: -hackers
kernel selection menu to the beastie menu. List of kernels is taken from
`kernels' in loader.conf(5) as a space (or comma) separated list of names
to display (up to 9). If not set, default value is "kernel kernel.old".
Does not validate that kernels exist because the next enhancement will be
to allow selection of the root device.
Discussed on: -current
MFC after: 3 days
