2023-12-20 Daniel Kiper Release 2.12 2023-12-20 Glenn Washburn efi: Add support for reproducible builds Having randomly generated bytes in the binary output breaks reproducible builds. Since build timestamps are usually the source of irreproducibility there is a standard which defines an environment variable SOURCE_DATE_EPOCH to be used when set for build timestamps. According to the standard [1], the value of SOURCE_DATE_EPOCH is a base-10 integer of the number of seconds since the UNIX epoch. Currently, this is a 10 digit number that fits into 32-bits, but will not shortly after the year 2100. So to be future-proof only use the least significant 32-bits. On 64-bit architectures, where the canary is also 64-bits, there is an extra 32-bits that can be filled to provide more entropy. The first byte is NUL to filter out string buffer overflow attacks and the remaining 24-bits are set to static random bytes. [1] https://reproducible-builds.org/specs/source-date-epoch Reviewed-by: Daniel Kiper 2023-12-20 Glenn Washburn efi: Generate stack protector canary at build time if urandom is available Generating the canary at build time allows the canary to be different for every build which could limit the effectiveness of certain exploits. Fallback to the statically generated random bytes if /dev/urandom is not readable, e.g. Windows. On 32-bit architectures, which use a 32-bit canary, reduce the canary to 4 bytes with one byte being NUL to filter out string buffer overflow attacks. Reviewed-by: Daniel Kiper 2023-12-20 Glenn Washburn efi: Initialize canary to non-zero value The canary, __stack_chk_guard, is in the BSS and so will get initialized to zero if it is not explicitly initialized. If the UEFI firmware does not support the RNG protocol, then the canary will not be randomized and will be zero. This seems like a possibly easier value to write by an attacker. Initialize canary to static random bytes, so that it is still random when there is no RNG protocol. Set at least one byte to NUL to protect against string buffer overflow attacks [1]. Code that writes NUL terminated strings will terminate when a NUL is encountered in the input byte stream. So the attacker will not be able to forge the canary by including it in the input stream without terminating the string operation and thus limiting the stack corruption. [1] https://www.sans.org/blog/stack-canaries-gingerly-sidestepping-the-cage/ Reviewed-by: Daniel Kiper 2023-12-14 Alec Brown gfxmenu/gui_image: Fix double free of bitmap In grub-core/gfxmenu/gui_image.c, Coverity detected a double free in the function load_image(). The function checks if self->bitmap and self->raw_bitmap aren't NULL and then frees them. In the case self->bitmap and self->raw_bitmap are the same, only self->raw_bitmap is freed which would also free the memory used by self->bitmap. However, in this case self->bitmap isn't being set to NULL which could lead to a double free later in the code. After self->raw_bitmap is freed, it gets set to the variable bitmap. If this variable is NULL, the code could have a path that would free self->bitmap a second time in the function rescale_image(). Fixes: CID 292472 Reviewed-by: Daniel Kiper 2023-12-13 Qiumiao Zhang commands/acpi: Fix calculation of ACPI tables addresses when processing RSDT and XSDT According to the ACPI specification the XSDT Entry field contains an array of 64-bit physical addresses which points to other DESCRIPTION_HEADERs. However, the entry_ptr iterator is defined as a 32-bit pointer. It means each 64-bit entry in the XSDT table is treated as two separate 32-bit entries then. Fix the issue by using correct addresses sizes when processing RSDT and XSDT tables. Reviewed-by: Daniel Kiper 2023-12-13 Vladimir Serbinenko libnvpair: Support prefixed nvlist symbol names as found on NetBSD NetBSD uses slightly different function names for the same functions. Reviewed-by: Daniel Kiper 2023-12-13 Vladimir Serbinenko bootstrap: Don't check gettext version NetBSD gettext is older than the check but we don't actually need 0.18.3, older one works fine. This is needed to make bootstrap work on NetBSD. Reviewed-by: Daniel Kiper 2023-12-13 Vladimir Serbinenko kern/mm: Use %x and cast for displaying sizeof() There is some variance in how compiler treats sizeof() especially on 32-bit platforms where it can be naturally either int or long. Explicit cast solves the issue. Reviewed-by: Daniel Kiper 2023-12-13 Vladimir Serbinenko configure: Add RPATH for freetype on NetBSD Without this build-time mkfont fails dynamic linking. This is not ideal but improves the situation until a better solution is available. Reviewed-by: Daniel Kiper 2023-12-13 Vladimir Serbinenko configure: Add *BSD font paths *BSD puts fonts in other places. Add them to the list. Reviewed-by: Daniel Kiper 2023-12-13 Vladimir Serbinenko autogen: Accept python3.10 as a python alternative NetBSD doesn't provide python or python3. Reviewed-by: Daniel Kiper 2023-12-12 Vladimir Serbinenko build: Rename HAVE_LIBZFS to USE_LIBZFS The HAVE_LIBZFS is defined by libzfs test and hence conflicts with manual definition. On NetBSD it ends up detecting zfs but not detecting nvpair and creates confusion. Split them. Reviewed-by: Daniel Kiper 2023-12-12 Vladimir Serbinenko gnulib: Tolerate always_inline attribute being ignored It's not critical, -Werror on it is inappropriate. We don't want to modify gnulib too much. This warning is pretty much irrelevant. Reviewed-by: Daniel Kiper 2023-12-12 Vladimir Serbinenko util/editenv: Don't use %m formatter It's not available on NetBSD outside of syslog. Using strerror() is more reliable as we retrieve errno immediately rather than down the stack. Reviewed-by: Daniel Kiper 2023-12-12 Vladimir Serbinenko osdep/bsd/hostdisk: Fix NetBSD compilation Wrong function and variable name cause a stupid compilation error on NetBSD and OpenBSD. Only NetBSD and OpenBSD use this file. No other platform is affected. Additionally, define RAW_FLOPPY_MAJOR constant if it is missing. Reviewed-by: Daniel Kiper 2023-12-12 Vladimir Serbinenko osdep/generic/blocklist: Fix compilation After recent change in blocklist types we have a type mismatch. Fixing it requires a wrapper or large changes. I feel like wrapper makes more sense. Without this patch we end up with a compilation problem and without wrapping callback data is not passed properly anymore. Reviewed-by: Daniel Kiper 2023-12-12 Vladimir Serbinenko disk/diskfilter: Remove unused variable Variable e is set but never used. We can just remove it now. Reviewed-by: Daniel Kiper 2023-12-12 Vladimir Serbinenko build: Tolerate unused-but-set in generated lexer/bison files We don't really control the small aspects of generated files and NetBSD version has an unused variable that is then detected by gcc as warning that is then promoted to error. Reviewed-by: Daniel Kiper 2023-12-12 Vladimir Serbinenko loader/i386/bsdXX: Fix loading after unaligned module Current code implicitly assumes that aligning chunk_size + *kern_end is the same as aligning on curload which is not the case because chunk_size starts at zero even if *kern_end is unaligned and ALIGN_PAGE moved curload to an aligned position but not *kern_end + chunk_size. This fixes booting of FreeBSD with zfs module. Reviewed-by: Daniel Kiper 2023-12-12 Mate Kukri grub-core/Makefile.am: Make path to extra_deps.lst relative to $(top_srcdir)/grub-core The commit 154dcb1ae (build: Allow explicit module dependencies) broke out of tree builds by introducing the extra_deps.lst file into the source tree but referencing it just by name in grub-core/Makefile.am. Fix it by adding $(top_srcdir)/grub-core to the path. Fixes: 154dcb1ae (build: Allow explicit module dependencies) Reviewed-by: Daniel Kiper 2023-12-12 Mate Kukri util/grub-install: Move platdir path canonicalization after files were copied to grubdir The commit 3f9eace2d (util/grub-install: Delay copying files to {grubdir,platdir} after install_device was validated) delaying copying of files caused a regression when installing without an existing directory structure. This patch ensures that the platform directory actually exists by the time the code tries to canonicalize its filename. Fixes: 3f9eace2d (util/grub-install: Delay copying files to {grubdir,platdir} after install_device was validated) Reviewed-by: Daniel Kiper 2023-12-12 Michael Chang util/grub-mkstandalone: Ensure deterministic tar file creation by sorting contents The add_tar_files() function currently iterates through a directory's content using readdir(), which doesn't guarantee a specific order. This lack of deterministic behavior impacts reproducibility in the build process. This commit resolves the issue by introducing sorting functionality. The list retrieved by readdir() is now sorted alphabetically before incorporation into the tar archive, ensuring consistent and predictable file ordering within the archive. On the occasion fix tfp memory leak. Reviewed-by: Daniel Kiper 2023-12-12 Michael Chang util/grub-mkstandalone: Ensure stable timestamps for generated images This change mirrors a previous fix [1] but is specific to images generated by grub-mkstandalone. The former fix, commit 85a7be241 (util/mkimage: Use stable timestamp when generating binaries.), focused on utilizing a stable timestamp during binary generation in the util/mkimage context. This commit extends that approach to the images produced by grub-mkstandalone, ensuring consistency and stability in timestamps across all generated binaries. [1] 85a7be241 util/mkimage: Use stable timestamp when generating binaries. Reviewed-by: Daniel Kiper 2023-12-05 Mate Kukri net/http: Fix gcc-13 errors relating to type signedness Replace definition of HTTP_PORT with a pre-processor macro that converts the constant to the correct grub_uint16_t type. Change "port" local variable definition in http_establish() to have the same type. Reviewed-by: Daniel Kiper templates: Reinstate unused version comparison functions with warning Revert the commit a79c567f6 (templates: Remove unused version comparison functions) and add a warning to the functions that they are deprecated. Removing the functions directly caused a lot of upgrade issues with custom user scripts that called the functions. In Debian and Ubuntu, grub-mkconfig is invoked as a post-installation script and would fail, causing upgrades to fail halfway through and putting the package manager into an inconsistent state. FWIW, we get one bug per 2 weeks basically, for an interim Ubuntu release which generally does not receive much usage, that is a high number. The proposal is to pick this for 2.12 and directly after the release remove it again. Then users will have time to fix their scripts without systems breaking immediately. This reverts commit a79c567f6 (templates: Remove unused version comparison functions). Cc: Mathieu Desnoyers Cc: Daniel Kiper Reviewed-by: Daniel Kiper 2023-12-05 Mate Kukri util/grub-install: Delay copying files to {grubdir,platdir} after install_device was validated Previously grub-install copied modules to grubdir before doing any validation on the install_device. When grub-install was called with an invalid install_device, modules were already copied to /boot before it found out and was forced to rely on atexit() rollback. This patch delays copying the modules after at least some install_device validation was done, and thus reduces reliance on successful rollback. Reviewed-by: Daniel Kiper 2023-12-05 Julian Andres Klode efi: Set shim_lock_enabled even if validation is disabled If validation has been disabled via MokSbState, secure boot on the firmware is still enabled, and the kernel fails to boot. This is a bit hacky, because shim_lock is not *fully* enabled, but it triggers the right code paths. Ultimately, all this will be resolved by shim gaining it's own image loading and starting protocol, so this is more a temporary workaround. Fixes: 6425c12cd (efi: Fallback to legacy mode if shim is loaded on x86 archs) Cc: Peter Jones Cc: Michael Chang Reviewed-by: Daniel Kiper 2023-12-05 Oliver Steffen docs: Improve bli module documentation Improve the documentation of the bli module and explain in more detail what it does. Make clear that GPT formatted drives are expected and other partition formats are ignored. Also reorder and reword this section a bit. Reviewed-by: Daniel Kiper 2023-12-05 Oliver Steffen bli: Add explicit dependency on the part_gpt module The bli module has a "hidden" dependency on the part_gpt module, which is not picked up automatically by the build system. One purpose of the bli module is to communicate the GPT UUID of the partition GRUB was launched from to Linux user-space (systemd-gpt-auto-generator). Without the part_gpt module, bli is not able to obtain the UUID. Since bli does its work in the module initialization function, the order in which the modules are loaded is also important: part_gpt needs to be loaded before the bli module. To solve this, track this dependency explicitly. Note that the Boot Loader Interface specification, which bli aims to implement, requires GPT formatted drives. The bli module ignores all other partition formats. Reviewed-by: Daniel Kiper 2023-12-05 Oliver Steffen build: Allow explicit module dependencies The build system deduces inter-module dependencies from the symbols required and exported by the modules. This works well, except for some rare cases where the dependency is indirect or hidden. A module might not make use of any function of some other module, but still expect its functionality to be available to GRUB. To solve this, introduce a new file, currently empty, called extra_deps.lst to track these cases manually. This file gets processed in the same way as the automatically generated syminfo.lst, making it possible to inject data into the dependency resolver. Since *.lst files are set to be ignored by git, add an exception for extra_deps.lst. Additionally, introduce a new keyword for the syminfo.lst syntax: "depends" allows specifying a module dependency directly: depends ... Reviewed-by: Daniel Kiper 2023-12-05 Stefan Berger kern/ieee1275/init/ppc64: Display upper_mem_limit when debugging Display upper_mem_limit and its rounded-down value in MiB. Reviewed-by: Daniel Kiper 2023-12-05 Stefan Berger kern/ieee1275/init/ppc64: Fix a comment Reviewed-by: Daniel Kiper 2023-12-05 Stefan Berger kern/ieee1275/ieee1275: Display successful memory claims when debugging Display successful memory claims with exact address and rounded-down MiB location and rounded-up size in MiB. Reviewed-by: Daniel Kiper Cc: Eric Snowberg Cc: Hari Bathini Cc: Pavithra Prakash Cc: Michael Ellerman Cc: Carolyn Scherrer Cc: Mahesh Salgaonkar Cc: Sourabh Jain 2023-12-05 Stefan Berger loader/powerpc/ieee1275: Use new allocation function for kernel and initrd On PowerVM and KVM on Power use the new memory allocation function that honors restrictions on which memory GRUB can actually use. In the request structure indicate the request for a single memory block along with address alignment restrictions. Request direct usage of the memory block by setting init_region to false (prevent it from being added to GRUB's heap). Initialize the found addr to -1, so that -1 will be returned to the loader in case no memory could be allocated. Report an out-of-memory error in case the initrd could not be loaded. Reviewed-by: Daniel Kiper Cc: Hari Bathini Cc: Pavithra Prakash Cc: Michael Ellerman Cc: Carolyn Scherrer Cc: Mahesh Salgaonkar Cc: Sourabh Jain 2023-12-05 Stefan Berger kern/ieee1275/cmain/ppc64: Introduce flags to identify KVM and PowerVM Introduce flags to identify PowerVM and KVM on Power and set them where each type of host has been detected. Reviewed-by: Daniel Kiper Cc: Hari Bathini Cc: Pavithra Prakash Cc: Michael Ellerman Cc: Carolyn Scherrer Cc: Mahesh Salgaonkar Cc: Sourabh Jain 2023-12-05 Stefan Berger kern/ieee1275/init/ppc64: Rename regions_claim() to grub_regions_claim() Rename regions_claim() to grub_regions_claim() to make it available for memory allocation. The ieee1275 loader will use this function on PowerVM and KVM on Power and thus avoid usage of memory that it is not allowed to use. Reviewed-by: Daniel Kiper Cc: Hari Bathini Cc: Pavithra Prakash Cc: Michael Ellerman Cc: Carolyn Scherrer Cc: Mahesh Salgaonkar Cc: Sourabh Jain 2023-12-05 Stefan Berger kern/ieee1275/init/ppc64: Add support for alignment requirements Add support for memory alignment requirements and adjust a candidate address to it before checking whether the block is large enough. This must be done in this order since the alignment adjustment can make a block smaller than what was requested. None of the current callers has memory alignment requirements but the ieee1275 loader for kernel and initrd will use it to convey them. Reviewed-by: Daniel Kiper Cc: Hari Bathini Cc: Pavithra Prakash Cc: Michael Ellerman Cc: Carolyn Scherrer Cc: Mahesh Salgaonkar Cc: Sourabh Jain 2023-12-05 Stefan Berger kern/ieee1275/init/ppc64: Return allocated address using context Return the allocated address of the memory block in the request structure if a memory allocation was actually done. Leave the address untouched otherwise. This enables a caller who wants to use the allocated memory directly, rather than adding the memory to the heap, to see where memory was allocated. None of the current callers need this but the converted ieee1275 loader will make use of it. Reviewed-by: Daniel Kiper Cc: Hari Bathini Cc: Pavithra Prakash Cc: Michael Ellerman Cc: Carolyn Scherrer Cc: Mahesh Salgaonkar Cc: Sourabh Jain 2023-12-05 Stefan Berger kern/ieee1275/init/ppc64: Decide by request whether to initialize region Let the regions_claim() request structure's init_region determine whether to call grub_mm_init_region() on it. This allows for adding memory to GRUB's memory heap if init_region is set to true, or direct usage of the memory otherwise. Set all current callers' init_region to true since they want to add memory regions to GRUB's heap. Reviewed-by: Daniel Kiper Cc: Hari Bathini Cc: Pavithra Prakash Cc: Michael Ellerman Cc: Carolyn Scherrer Cc: Mahesh Salgaonkar Cc: Sourabh Jain 2023-12-05 Stefan Berger kern/ieee1275/init/ppc64: Introduce a request for regions_claim() The regions_claim() function limits the allocation of memory regions by excluding certain memory areas from being used by GRUB. This for example includes a gap between 640MB and 768MB as well as an upper limit beyond which no memory may be used when an fadump is present. However, the ieee1275 loader for kernel and initrd currently does not use regions_claim() for memory allocation on PowerVM and KVM on Power and therefore may allocate memory in those areas that it should not use. To make the regions_claim() function more flexible and ultimately usable for the ieee1275 loader, introduce a request structure to pass various parameters to the regions_claim() function that describe the properties of requested memory chunks. In a first step, move the total and flags variables into this structure. Reviewed-by: Daniel Kiper Cc: Hari Bathini Cc: Pavithra Prakash Cc: Michael Ellerman Cc: Carolyn Scherrer Cc: Mahesh Salgaonkar Cc: Sourabh Jain 2023-11-22 Anthony Iliopoulos fs/xfs: Add large extent counters incompat feature support XFS introduced 64-bit extent counters for inodes via a series of upstream commits and the feature was marked as stable in v6.5 via commit 61d7e8274cd8 (xfs: drop EXPERIMENTAL tag for large extent counts). Further, xfsprogs release v6.5.0 switched this feature on by default in mkfs.xfs via commit e5b18d7d1d96 (mkfs: enable large extent counts by default). Filesystems formatted with large extent count support, nrext64=1, are thus currently not recognizable by GRUB, since this is an incompat feature. Add the required support so that those filesystems and inodes with large extent counters can be read by GRUB. Reviewed-by: Andrey Albershteyn Reviewed-by: Daniel Kiper Tested-by: Marta Lewandowska Tested-by: Sebastian Andrzej Siewior 2023-11-08 Vladimir Serbinenko gpt: Add compile time asserts for guid and gpt_partentry sizes With new alignment specification it's easy to screw up. Fortunately if it happens the size will be bigger than intended. Compile time assert will catch this. Reviewed-by: Daniel Kiper 2023-11-08 Vladimir Serbinenko types: Split aligned and packed guids On ia64 alignment requirements are strict. When we pass a pointer to UUID it needs to be at least 4-byte aligned or EFI will crash. On the other hand in device path there is no padding for UUID, so we need 2 types in one formor another. Make 4-byte aligned and unaligned types The code is structured in a way to accept unaligned inputs in most cases and supply 4-byte aligned outputs. Efiemu case is a bit ugly because there inputs and outputs are reversed and so we need careful casts to account for this inversion. Reviewed-by: Daniel Kiper 2023-11-06 Vladimir Serbinenko gpt_partition: Mark grub_gpt_partentry as having natural alignment gpt_partition contains grub_guid. We need to decide whether the whole structure is unaligned and then we need to use packed_guid. But we never have unaligned part entries as we read them in an aligned buffer from disk. Hence just make it all aligned. 2023-11-06 Vladimir Serbinenko efi: Deduplicate configuration table search function We do table search in many places doing exactly the same algorithm. The only minor variance in users is which table is used if several entries are present. As specification mandates uniqueness and even if it ever isn't, first entry is good enough, unify this code and always use the first entry. Reviewed-by: Daniel Kiper 2023-11-06 Vladimir Serbinenko lsefi: Add missing static qualifier known_protocols isn't used anywhere else and even misses grub_ prefix, so let's make it local (static). Reviewed-by: Daniel Kiper 2023-11-06 Vladimir Serbinenko types: Fix typo Just a small grammar mistake. Reviewed-by: Daniel Kiper 2023-10-30 Qiumiao Zhang util/grub-mount: Check file path sanity The function argp_parser() in util/grub-mount.c lacks a check on the sanity of the file path when parsing parameters. This results in a segmentation fault if a partition is mounted to a non-existent path. Reviewed-by: Daniel Kiper 2023-10-30 Richard Marko configure: Make the DJVU_FONT_SOURCE configurable with --with-dejavufont=FILE Font might be located in different location, the default font might not be available on all systems or other font might be preferred. Reviewed-by: Daniel Kiper 2023-10-30 Mads Kiilerich configure: Make the Unifont FONT_SOURCE configurable with --with-unifont=FILE Font might be located in different location, the default font might not be available on all systems or other font might be preferred. Reviewed-by: Daniel Kiper 2023-10-30 Jon DeVree fs/xfs: Fix XFS directory extent parsing The XFS directory entry parsing code has never been completely correct for extent based directories. The parser correctly handles the case where the directory is contained in a single extent, but then mistakenly assumes the data blocks for the multiple extent case are each identical to the single extent case. The difference in the format of the data blocks between the two cases is tiny enough that its gone unnoticed for a very long time. A recent change introduced some additional bounds checking into the XFS parser. Like GRUB's existing parser, it is correct for the single extent case but incorrect for the multiple extent case. When parsing a directory with multiple extents, this new bounds checking is sometimes (but not always) tripped and triggers an "invalid XFS directory entry" error. This probably would have continued to go unnoticed but the /boot/grub/ directory is large enough that it often has multiple extents. The difference between the two cases is that when there are multiple extents, the data blocks do not contain a trailer nor do they contain any leaf information. That information is stored in a separate set of extents dedicated to just the leaf information. These extents come after the directory entry extents and are not included in the inode size. So the existing parser already ignores the leaf extents. The only reason to read the trailer/leaf information at all is so that the parser can avoid misinterpreting that data as directory entries. So this updates the parser as follows: For the single extent case the parser doesn't change much: 1. Read the size of the leaf information from the trailer 2. Set the end pointer for the parser to the start of the leaf information. (The previous bounds checking set the end pointer to the start of the trailer, so this is actually a small improvement.) 3. Set the entries variable to the expected number of directory entries. For the multiple extent case: 1. Set the end pointer to the end of the block. 2. Do not set up the entries variable. Figuring out how many entries are in each individual block is complex and does not seem worth it when it appears to be safe to just iterate over the entire block. The bounds check itself was also dependent upon the faulty XFS parser because it accidentally used "filename + length - 1". Presumably this was able to pass the fuzzer because in the old parser there was always 8 bytes of slack space between the tail pointer and the actual end of the block. Since this is no longer the case the bounds check needs to be updated to "filename + length + 1" in order to prevent a regression in the handling of corrupt fliesystems. Notes: * When there is only one extent there will only ever be one block. If more than one block is required then XFS will always switch to holding leaf information in a separate extent. * B-tree based directories seems to be parsed properly by the same code that handles multiple extents. This is unlikely to ever occur within /boot though because its only used when there are an extremely large number of directory entries. Fixes: ef7850c75 (fs/xfs: Fix issues found while fuzzing the XFS filesystem) Fixes: b2499b29c (Adds support for the XFS filesystem.) Fixes: https://savannah.gnu.org/bugs/?64376 Reviewed-by: Daniel Kiper Tested-by: Sebastian Andrzej Siewior Tested-by: Marta Lewandowska 2023-10-30 Lidong Chen fs/xfs: Incorrect short form directory data boundary check After parsing of the current entry, the entry pointer is advanced to the next entry at the end of the "for" loop. In case where the last entry is at the end of the data boundary, the advanced entry pointer can point off the data boundary. The subsequent boundary check for the advanced entry pointer can cause a failure. The fix is to include the boundary check into the "for" loop condition. Reviewed-by: Daniel Kiper Tested-by: Sebastian Andrzej Siewior Tested-by: Marta Lewandowska 2023-10-12 Vladimir 'phcoder' Serbinenko Revert "zfsinfo: Correct a check for error allocating memory" Original commit is wrong because grub_file_get_device_name() may return NULL if we use implicit $root. Additionally, the grub_errno is guaranteed to be GRUB_ERR_NONE at the beginning of a command. So, everything should work as expected and Coverity report, CID 73668, WRT to this code should be treated as false positive. This reverts commit 7aab03418 (zfsinfo: Correct a check for error allocating memory). Fixes: 7aab03418 (zfsinfo: Correct a check for error allocating memory) Reviewed-by: Daniel Kiper 2023-10-12 ValdikSS disk/i386/pc/biosdisk: Read up to 63 sectors in LBA mode Current code imposes limitations on the amount of sectors read in a single call according to CHS layout of the disk even in LBA read mode. There's no need to obey CHS layout restrictions for LBA reads on LBA disks. It only slows down booting process. See: https://lore.kernel.org/grub-devel/d42a11fa-2a59-b5e7-08b1-d2c60444bb99@valdikss.org.ru/ Reviewed-by: Daniel Kiper 2023-10-12 ValdikSS kern/i386/pc/init: Flush cache only on VIA C3 and earlier The code flushes the cache on VIA processors unconditionally which is excessive. Check for cpuid family and execute wbinvd only on C3 and earlier. Fixes: https://savannah.gnu.org/bugs/?45149 Fixes: 25492a0f0 (Add wbinvd around bios call.) Reviewed-by: Daniel Kiper 2023-10-12 Fabian Vogt fs/btrfs: Zero file data not backed by extents Implicit holes in file data need to be zeroed explicitly, instead of just leaving the data in the buffer uninitialized. This led to kernels randomly failing to boot in "fun" ways when loaded from btrfs with the no_holes feature enabled, because large blocks of zeros in the kernel file contained random data instead. Reviewed-by: Daniel Kiper Reviewed-by: Qu Wenruo 2023-10-12 Stefan Berger kern/ieee1275/init: Restrict high memory in presence of fadump on ppc64 When a kernel dump is present then restrict the high memory regions to avoid allocating memory where the kernel dump resides. Use the ibm,kernel-dump node under /rtas to determine whether a kernel dump exists and up to which limit GRUB can use available memory. Set the upper_mem_limit to the size of the kernel dump section of type REAL_MODE_REGION and therefore only allow GRUB's memory usage for high addresses from RMO_ADDR_MAX to upper_mem_limit. This means that GRUB can use high memory in the range of RMO_ADDR_MAX (768MB) to upper_mem_limit and the kernel-dump memory regions above upper_mem_limit remain untouched. This change has no effect on memory allocations below linux_rmo_save (typically at 640MB). Also, fall back to allocating below rmo_linux_save in case the chunk of memory there would be larger than the chunk of memory above RMO_ADDR_MAX. This can for example occur if a free memory area is found starting at 300MB extending up to 1GB but a kernel dump is located at 768MB and therefore does not allow the allocation of the high memory area but requiring to use the chunk starting at 300MB to avoid an unnecessary out-of-memory condition. Reviewed-by: Hari Bathini Cc: Pavithra Prakash Cc: Michael Ellerman Cc: Carolyn Scherrer Cc: Mahesh Salgaonkar Cc: Sourabh Jain Reviewed-by: Daniel Kiper 2023-10-12 Glenn Washburn tests/util/grub-shell: Enable RNG device to better test stack smashing In certain firmwares, e.g. OVMF, the RNG protocol is not enabled unless there is an RNG device. When not enabled, GRUB fails to initialize the stack guard with random bytes. For testing, this is not a big issue, but there have been bugs found in the initialization. So turn this on for EFI platforms to catch any regressions. Reviewed-by: Daniel Kiper 2023-10-12 Glenn Washburn kern/efi/init: Disable stack smashing protection on grub_efi_init() GCC is electing to instrument grub_efi_init() to give it stack smashing protection when configuring with --enable-stack-protector on the x86_64-efi target. In the function prologue, the canary at the top of the stack frame is set to the value of the stack guard. And in the epilogue, the canary is checked to verify if it is equal to the guard and if not to call the stack check fail function. The issue is that grub_efi_init() sets up the guard by initializing it with random bytes, if the firmware supports the RNG protocol. So in its prologue the canary will be set with the value of the uninitialized guard, likely NUL bytes. Then the guard is initialized, and finally the epilogue checks the canary against the guard, which will almost certainly be different. This causes the code path for a smashed stack to be taken, causing the machine to print out a message that stack smashing was detected, wait 5 seconds, and then reboot. Disable grub_efi_init() instrumentation so there is no stack smashing false positive generated. Reviewed-by: Daniel Kiper 2023-10-12 Glenn Washburn disk/cryptodisk: Add support for LUKS2 in (proc)/luks_script The sector size in bytes is added to each line and it is allowed to be 6 decimal digits long, which covers the most common cases of 512 and 4096 byte sectors with space for two additional digits as future-proofing. The size allocation is updated to reflect this additional field. Also make clearer the size allocation calculation. Reviewed-by: Daniel Kiper 2023-10-12 Glenn Washburn disk/cryptodisk: Optimize luks_script_get() Use the return value of grub_snprintf() to move the string pointer forward, instead of incrementing the string pointer iteratively until a NULL byte is reached. Move the space out of the format string argument, a small optimization, but also makes the spacing clearer. Also, use the new PRIxGRUB_OFFSET instead of PRIuGRUB_UINT64_T to accurately reflect the format string for this type. Reviewed-by: Daniel Kiper 2023-10-12 Glenn Washburn term/serial: Ensure proper NULL termination after grub_strncpy() A large enough argument to the --port option could cause a string buffer to be not NULL terminated because grub_strncpy() does not guarantee NULL termination if copied string is longer than max characters to copy. Fixes: 712309eaae04 (term/serial: Use grub_strncpy() instead of grub_snprintf() when only copying string) Reviewed-by: Daniel Kiper 2023-10-12 Heinrich Schuchardt commands/efi/lsefisystab: Print the UEFI specification revision in human readable form E.g. 2.10 instead of 00020064 and 2.3.1 instead of 0002001f. See UEFI 2.10 specification, chapter 4.2.1 EFI_TABLE_HEADER. Reviewed-by: Daniel Kiper 2023-10-03 Maxim Suhanov fs/ntfs: Make code more readable Move some calls used to access NTFS attribute header fields into functions with human-readable names. Suggested-by: Daniel Kiper Reviewed-by: Daniel Kiper 2023-10-03 Maxim Suhanov fs/ntfs: Fix an OOB read when parsing a volume label This fix introduces checks to ensure that an NTFS volume label is always read from the corresponding file record segment. The current NTFS code allows the volume label string to be read from an arbitrary, attacker-chosen memory location. However, the bytes read are always treated as UTF-16LE. So, the final string displayed is mostly unreadable and it can't be easily converted back to raw bytes. The lack of this check is a minor issue, likely not causing a significant data leak. Reported-by: Maxim Suhanov Reviewed-by: Daniel Kiper 2023-10-03 Maxim Suhanov fs/ntfs: Fix an OOB read when parsing bitmaps for index attributes This fix introduces checks to ensure that bitmaps for directory indices are never read beyond their actual sizes. The lack of this check is a minor issue, likely not exploitable in any way. Reported-by: Maxim Suhanov Reviewed-by: Daniel Kiper 2023-10-03 Maxim Suhanov fs/ntfs: Fix an OOB read when parsing directory entries from resident and non-resident index attributes This fix introduces checks to ensure that index entries are never read beyond the corresponding directory index. The lack of this check is a minor issue, likely not exploitable in any way. Reported-by: Maxim Suhanov Reviewed-by: Daniel Kiper 2023-10-03 Maxim Suhanov fs/ntfs: Fix an OOB read when reading data from the resident $DATA attribute When reading a file containing resident data, i.e., the file data is stored in the $DATA attribute within the NTFS file record, not in external clusters, there are no checks that this resident data actually fits the corresponding file record segment. When parsing a specially-crafted file system image, the current NTFS code will read the file data from an arbitrary, attacker-chosen memory offset and of arbitrary, attacker-chosen length. This allows an attacker to display arbitrary chunks of memory, which could contain sensitive information like password hashes or even plain-text, obfuscated passwords from BS EFI variables. This fix implements a check to ensure that resident data is read from the corresponding file record segment only. Fixes: CVE-2023-4693 Reported-by: Maxim Suhanov Reviewed-by: Daniel Kiper 2023-10-03 Maxim Suhanov fs/ntfs: Fix an OOB write when parsing the $ATTRIBUTE_LIST attribute for the $MFT file When parsing an extremely fragmented $MFT file, i.e., the file described using the $ATTRIBUTE_LIST attribute, current NTFS code will reuse a buffer containing bytes read from the underlying drive to store sector numbers, which are consumed later to read data from these sectors into another buffer. These sectors numbers, two 32-bit integers, are always stored at predefined offsets, 0x10 and 0x14, relative to first byte of the selected entry within the $ATTRIBUTE_LIST attribute. Usually, this won't cause any problem. However, when parsing a specially-crafted file system image, this may cause the NTFS code to write these integers beyond the buffer boundary, likely causing the GRUB memory allocator to misbehave or fail. These integers contain values which are controlled by on-disk structures of the NTFS file system. Such modification and resulting misbehavior may touch a memory range not assigned to the GRUB and owned by firmware or another EFI application/driver. This fix introduces checks to ensure that these sector numbers are never written beyond the boundary. Fixes: CVE-2023-4692 Reported-by: Maxim Suhanov Reviewed-by: Daniel Kiper 2023-10-03 Michael Chang kern/acpi: Skip NULL entries in RSDT and XSDT During attempts to configure a serial console, a Page Fault Exception and system reset were encountered, specifically on release 2.12~rc1. This issue was not present in prior versions and seemed to affect only a specific machine, potentially pointing to hardware or firmware flaw. After investigation, it was discovered that the invalid page access occurred during the discovery of serial MMIO ports as specified by ACPI's SPCR table [1]. The recent change uncovered an issue in GRUB's ACPI driver. In certain cases, the XSDT/RSDT root table might contain a NULL entry as a terminator, depending on how the tables are assembled. GRUB cannot blindly trust the address in the root table to be valid and should perform a sanity check for NULL entries. This patch introduces this simple check. This fix is also inspired by a related Linux kernel fix [2]. [1] 7b192ec4c term/ns8250: Use ACPI SPCR table when available to configure serial [2] 0f929fbf0 ACPICA: Tables: Add new mechanism to skip NULL entries in RSDT and XSDT. Reviewed-by: Daniel Kiper 2023-10-03 Glenn Washburn util/grub-install-common: Print usable grub-mkimage command When grub-install is run with the verbose option, it will print a log message indicating the grub-mkimage command and arguments used. GRUB no longer calls the grub-mkimage binary internally, however the command logged is a command that if run should effectively be what grub-install used. However, as this has changed some of the newer options have been incorrectly added so that the printed command fails when run separately. This change makes the displayed command run as intended. Reviewed-by: Daniel Kiper 2023-10-03 Glenn Washburn util/grub-install-common: Minor improvements to printing of grub-mkimage command This is a preparatory patch to make the following patch less cluttered. The only visible change made here is to not print extra spaces when either or both --note or --disable-shim-lock are not given and to not print an extra space at the end of the command. The latter is done by constructing the trailing argument string with spaces in front of each argument rather than trailing. The allocation of the argument string is made precise, which has the benefit of saving a few bytes, but more importantly self-documenting what the needed allocated bytes are. Also, unneeded braces are removed from an if block. Reviewed-by: Daniel Kiper 2023-10-03 Vladimir 'phcoder' Serbinenko lib/i386/relocator64: Fix 64-bit FreeBSD boot on BIOS The commit 80948f532d (lib/i386/relocator64: Build fixes for i386) has broken 64-bit FreeBSD boot on BIOS. This patch fixes the issue. Fixes: 80948f532d (lib/i386/relocator64: Build fixes for i386) Reviewed-by: Daniel Kiper 2023-09-22 Anthony PERARD templates/linux_xen: Fix XSM entries generation It turns out that setting $xen_version in linux_entry_xsm() override $xen_version in the loop over $reverse_sorted_xen_list. This means that only one entry per Xen version is going to enable XSM, but all further entries are going to have "(XSM enabled)" in their titles without enabling XSM. When a "xenpolicy-$xen_version" file was found for the current $xen_version, it would overwrite $xen_version to add "(XSM enabled)" to the menu entry title. Once updated, the next call to linux_entry_xsm() would also have this modified $xen_version and would look for the file "xenpolicy-*(XSM enabled)" and fail. Reviewed-by: Daniel Kiper 2023-09-22 Xiaotian Wu loongarch: Eliminate cmodel compilation warnings In the configure phase, the "-mcmodel=large" CFLAGS passed the test, but because it has not been implemented in gcc, the following warning will appear when compiling: gcc: warning: 'large' is not supported, now cmodel is set to 'normal' Reviewed-by: Daniel Kiper 2023-09-22 Glenn Washburn configure: Enable -fno-omit-frame-pointer for backtrace module The backtrace module is written assuming that the frame pointer is in %ebp. By default, -Os optimization level is used, which enables the gcc option -fomit-frame-pointer. This breaks the backtrace functionality. Enabling this may cause an unnoticeable performance cost and virtually no size increase. The backtrace command on x86_64 and probably i386 is broken due to the above rationale. I've not verified, but presumably the backtrace that used to be printed for an unhandled CPU exception is also broken. Do any distros handle this? Considering that, to my knowledge, no one has complained about this in the over 13 years that -Os has been used, has this code actually been useful? Is it worth disabling -fomit-frame-pointer? Though, I don't see much downside right now in disabling it. Alternatively, we could disable/remove the backtrace code. I think it would be nice to keep it and have it working. Nowadays, presumably QEMU makes the GDB stub rarely used as I imagine most are developing in a virtual machines. Also, the GDB stub does not work in UEFI. So, if anyone is using it on real hardware, they are doing so on pretty old machines. The lack of a GDB stub does not seem to be a pain point because no one has got it working on UEFI. This patch gets the backtrace command working on x86_64-efi in QEMU for me. However, it hangs when run on my laptop. Not sure what's going on there. Reviewed-by: Daniel Kiper 2023-09-22 Ard Biesheuvel loader/efi/linux: Implement x86 mixed mode using legacy boot Recent mixed-mode Linux kernels, i.e., v4.0 or newer, can access EFI runtime services at OS runtime even when the OS was not entered via the EFI stub. This is because, instead of reverting back to the firmware's segment selectors, GDTs and IDTs, the 64-bit kernel simply calls 32-bit runtime services using compatibility mode, i.e., the same mode used for 32-bit user space, without taking down all interrupt handling, exception handling, etc. This means that GRUB's legacy x86 boot mode is sufficient to make use of this: 32-bit i686 builds of GRUB can already boot 64-bit kernels in EFI enlightened mode, but without going via the EFI stub, and provide all the metadata that the OS needs to map the EFI runtime regions and call EFI runtime services successfully. It does mean that GRUB should not attempt to invoke the firmware's LoadImage()/StartImage() methods on kernel builds that it knows cannot be started natively. So, add a check for this in the native EFI boot path and fall back to legacy x86 mode in such cases. Note that in the general case, booting non-native images of the same native word size, e.g., x64 EFI apps on arm64 firmware, might be supported by means of emulation. So, let's only disallow images that use a non-native word size. This will also permit booting i686 kernels on x86_64 builds, although without access to runtime services, as this is not supported by Linux. This change on top of 2.12-rc1 is sufficient to boot ordinary Linux mixed mode builds and get full access to the EFI runtime services. Cc: Daniel Kiper Cc: Steve McIntyre Cc: Julian Andres Klode Acked-by: Dimitri John Ledkov Reviewed-by: Daniel Kiper 2023-09-22 Ard Biesheuvel loader/i386/linux: Prefer entry in long mode when booting via EFI The x86_64 Linux kernel can be booted in 32-bit mode, in which case the startup code creates a set of preliminary page tables that map the first 4 GiB of physical memory 1:1 and enables paging. This is a prerequisite for 64-bit execution and can therefore only be implemented in 32-bit code. The x86_64 Linux kernel can also be booted in 64-bit mode directly: this implies that paging is already enabled and it is the responsibility of the bootloader to ensure that the active page tables cover the entire loaded image, including its BSS space, the size of which is described in the image's setup header. Given that the EFI spec mandates execution in long mode for x86_64 and stipulates that all system memory is mapped 1:1, the Linux/x86 requirements for 64-bit entry can be met trivially when booting on x86_64 via EFI. So, enter via the 64-bit entry point in this case. This involves inspecting the xloadflags field in the setup header to check whether the 64-bit entry point is supported. This field was introduced in Linux version v3.8 (early 2013). This change ensures that all EFI firmware tables and other assets passed by the firmware or bootloader in memory remain mapped and accessible throughout the early startup code. Avoiding the drop out of long mode will also be needed to support upcoming CPU designs that no longer implement 32-bit mode at all (as recently announced by Intel [0]). [0] https://www.intel.com/content/www/us/en/developer/articles/technical/envisioning-future-simplified-architecture.html Cc: Daniel Kiper Cc: Julian Andres Klode Reviewed-by: Daniel Kiper 2023-09-18 Vladimir Serbinenko ZFS: Check bonustype in addition to dnode type Some dnodes are shared with properties zap. This is used e.g. for quotas. Then dnode type is 0xc4 and GRUB stumbles on this. Check bonus type and if it's ok then ignore dnode type mismatch Reviewed-by: Daniel Kiper 2023-09-18 Vladimir Serbinenko ZFS: Don't iterate over null objsets Reading them is harmless but useless as they are empty by definition Reviewed-by: Daniel Kiper 2023-09-18 Vladimir Serbinenko ZFS: Fix invalid memcmp We ended up comparing over unset values as we had dnode_phys on one side and dnode on another Reviewed-by: Daniel Kiper 2023-09-18 Vladimir Serbinenko ZFS: support inode type embed into its ID This is a speedup used in some ZFS version. This trips GRUB and makes it unable to access directories. Just skip it for now and revisit if we ever need this speedup. Reviewed-by: Daniel Kiper 2023-08-31 Heinrich Schuchardt video/efi_gop: Require shadow if PixelBltOnly If the EFI graphics pixel format is PixelBltOnly, we cannot write directly to the frame buffer. We need the shadow frame buffer which we copy via the BitBlt operation to the hardware. If the pixel format is PixelBltOnly and allocation of the shadow frame buffer fails, we must raise an error to signal that the EFI GOP protocol is not usable. Reviewed-by: Daniel Kiper 2023-08-31 Glenn Washburn docs: Add menu to prevent older makeinfo versions from failing It has been reported that makeinfo version 4.13a complains and returns error when menus for chapter structuring commands are not present. It is also known that newer makeinfos, such as version 6.7, will create default menus when needed. Since the menu will be created regardless, explicitly create it to support older makeinfo versions. This also enables building to be successful when an older makeinfo is installed because in that case info files are attempted to be generated with the "all" target. Reported-by: Olaf Hering Reviewed-by: Daniel Kiper Tested-by: Olaf Hering 2023-08-31 Glenn Washburn docs: Use @ref instead of @xref The @xref command is meant to be used at the beginning of a sentence because its expansion creates a "See " prefix on all output formats, and on older makeinfo versions is strict about enforcing a "." or "," after the command. The @ref command has no such restriction and is just the link, which allows more control over output. This also fixes an issue where there was a repeated "see" in the output. Reported-by: Olaf Hering Reviewed-by: Daniel Kiper Tested-by: Olaf Hering 2023-08-31 Glenn Washburn tests/util/grub-shell-luks-tester: Allow setting timeout Allow using the envvar GRUB_SHELL_LUKS_TIMEOUT to change the default timeout. If not specified, use value of GRUB_SHELL_DEFAULT_TIMEOUT. And if that is not specified, fallback to original 600s timeout. Reviewed-by: Daniel Kiper 2023-08-31 Glenn Washburn disk/cryptodisk: Fix missing change when updating to use grub_uuidcasecmp() This was causing the cryptomount command to return failure even though the crypto device was successfully added. Of course, this meant that any script using the return code would behave unexpectedly. Fixes: 3cf2e848bc03 (disk/cryptodisk: Allows UUIDs to be compared in a dash-insensitive manner) Suggested-by: Olaf Hering Reviewed-by: Patrich Steinhardt Reviewed-by: Daniel Kiper 2023-08-31 Glenn Washburn kern/misc: Make grub_vsnprintf() C99/POSIX conformant To comply with C99 and POSIX standards, snprintf() should return the number of bytes that would be written to the string (excluding the terminating NUL byte) if the buffer size was big enough. Before this change, the return value was the minimum of the standard return and the length of the buffer. Rarely is the return value of grub_snprintf() or grub_vsnprintf() used with current code, and the few places where it is used do not need to be changed. Reviewed-by: Daniel Kiper 2023-08-31 Glenn Washburn tests: Add serial_test This test is meant to test output via various serial devices. Currently, only the PCI serial device is tested. Reviewed-by: Daniel Kiper 2023-08-31 Glenn Washburn tests/util/grub-shell: Allow explicitly using other serial ports for output While here, move "-qemu=*" case to be next to the "--qemu-opts=*" case. This causes no change in logic, but is more logically located. Reviewed-by: Daniel Kiper 2023-08-31 Glenn Washburn tests/util/grub-shell-luks-tester: Do not remove generated files when test fails to allow debugging Reviewed-by: Daniel Kiper tests/util/grub-shell: Convert spaces to TABs Reviewed-by: Daniel Kiper 2023-08-14 Glenn Washburn commands/ls: Print "????????????" if unable to get file size In long list mode, if the file can not be opened, the file is not printed. Instead, print the file but print the size as "????????????". Reviewed-by: Daniel Kiper 2023-08-14 Glenn Washburn commands/ls: Send correct dirname to print functions For each non-directory path argument to the ls command, the full path was being sent to the print functions, instead of the dirname. The long output print function expected dirname to be the directory containing the file and so could not open the file to get the file size because the generated path was incorrect. This caused the output to be a blank line. Reviewed-by: Daniel Kiper 2023-08-14 Glenn Washburn fs/archelp: If path given to grub_archelp_dir() is not a directory return error Specifically, return GRUB_ERR_BAD_FILE_TYPE because this is what is expected by the ls command when it is given a path to a non-directory. This fixes a bug where calling ls with a list of non-directory paths outputs a blank line for each such argument. Reviewed-by: Daniel Kiper 2023-08-14 Glenn Washburn commands/videoinfo: Prevent crash when run while video driver already active The videoinfo command will initialize all non-active video adapters. Video drivers tend to zero out the global framebuffer object on initialization. This is not a problem when there is no active video adapter. However, when there is, then outputting to the video adapter will cause a crash because methods in the framebuffer object are reinitialized. For example, this command sequence will cause a crash. terminal_output --append gfxterm; videoinfo When running in a QEMU headless with GRUB built for the x86_64-efi target, the first command initializes the Bochs video adapter, which, among other things, sets the set_page() member function. Then when videoinfo is run, all non-Bochs video adapters will be initialized, each one wiping the framebuffer and thus setting set_page to NULL. Soon after the videoinfo command finishes there will be a call to grub_refresh(), which will ultimately call the framebuffer's set_page which will be NULL and cause a crash when called. Reviewed-by: Daniel Kiper 2023-08-14 Glenn Washburn docs: Improve initrd documentation A list of improvements: * Remove reference to "initial ramdisk" and replace with "initrd". This then covers the case of ramdisk and ramfs, which is the usual method with kernels 2.6 and newer. * Add sentence with URL to initrd documentation Linux kernel. * Add a section documenting how to have the initrd command generate a new-style initrd via a specially crafted argument and include an example. * Update initrd16 to refer to the initrd section and make note that initrd16 is only on the pc platform. Reviewed-by: Oskari Pirhonen Reviewed-by: Paul Menzel Reviewed-by: Daniel Kiper 2023-08-14 Glenn Washburn term/ns8250-spcr: Continue processing SPCR table even if revision is < 2 According to commit 0231d00082 (ACPI: SPCR: Make SPCR available to x86) to the Linux kernel, "On x86, many systems have a valid SPCR table but the table version is not 2 so the table version check must be a warning." Reviewed-by: Benjamin Herrenschmidt Reviewed-by: Daniel Kiper 2023-08-14 Glenn Washburn docs: A note to cat that hexdump should be used for binary data The cat command should not be used to print binary data because it can show bytes not in the binary data and not show bytes that are in the data, which can lead to confusion. This happens because cat does some processing of the data stream, namely trying to decode substrings as UTF-8. Reviewed-by: Oskari Pirhonen Reviewed-by: Daniel Kiper 2023-08-14 Glenn Washburn docs: Document hexdump command Reviewed-by: Oskari Pirhonen Reviewed-by: Daniel Kiper docs: Group usage of user-space utilities into single chapter Reviewed-by: Oskari Pirhonen Reviewed-by: Daniel Kiper 2023-08-14 Qiumiao Zhang util/grub-mount: Fix memory leak in fuse_getattr() Reviewed-by: Daniel Kiper 2023-08-14 Michał Grzelak configure: Fix SDL2 typo by referencing value During configuration of SDL2, variable enable_grub_emu_sdl2 is checked whether to throw an error message. However, error could not happen because two unequal strings were compared. Fix this by referencing value of enable_grub_emu_sdl2, not name. Fixes: 17d6ac1a7 (emu: Add SDL2 support) Reviewed-by: Julian Andres Klode Reviewed-by: Daniel Kiper Reviewed-by: Paul Menzel 2023-08-14 Glenn Washburn docs: Add missing assumption Also reword a prior sentence to be more clear. Fixes: 5a3d2b4742df (docs: Add debugging chapter to development documentation) Reviewed-by: Oskari Pirhonen Reviewed-by: Daniel Kiper 2023-08-14 Oskari Pirhonen util/grub.d/25_bli.in: Fix shebang on unmerged-usr On an unmerged-usr system, grub-mkconfig errors out with the following error due to /usr/bin/sh not existing: /usr/sbin/grub-mkconfig: /etc/grub.d/25_bli: /usr/bin/sh: bad interpreter: No such file or directory Use a /bin/sh shebang to fix the error as well as match the other existing files. Fixes: 158a6583e (util/grub.d/25_bli.in: Activate bli module on EFI) Reviewed-by: Glenn Washburn Reviewed-by: Daniel Kiper Reviewed-by: Oliver Steffen 2023-08-14 Glenn Washburn tests/util/grub-shell-luks-tester: Allow GRUB_SHELL_LUKS_DEFAULT_DEBUG and GRUB_TEST_DEFAULT_DEBUG to specify the debug level to grub-shell Reviewed-by: Daniel Kiper 2023-08-14 Glenn Washburn tests/util/grub-shell: Allow setting the value of debug regardless of its previous state This allows an invocation of grub-shell to set the value of debug regardless of the global default environment variable GRUB_SHELL_DEFAULT_DEBUG. Reviewed-by: Daniel Kiper 2023-08-14 Glenn Washburn tests/util/grub-shell: Allow setting default timeout via GRUB_SHELL_DEFAULT_TIMEOUT envvar Reviewed-by: Daniel Kiper 2023-08-14 Glenn Washburn tests/util/grub-shell: Add --verbose to grub-mkrescue when $debug is greater than 2 Since this is fairly verbose output, do not enable first level of debug is turned on. Reviewed-by: Daniel Kiper 2023-07-10 Daniel Kiper Release 2.12~rc1 2023-07-03 Daniel Kiper efi: Fallback to legacy mode if shim is loaded on x86 archs The LoadImage() provided by the shim does not consult MOK when loading an image. So, simply signature verification fails when it should not. This means we cannot use Linux EFI stub to start the kernel when the shim is loaded. We have to fallback to legacy mode on x86 architectures. This is not possible on other architectures due to lack of legacy mode. This is workaround which should disappear when the shim provides LoadImage() which looks up MOK during signature verification. On the occasion align constants in include/grub/efi/sb.h. Reviewed-by: Ard Biesheuvel 2023-07-03 Daniel Kiper efi: Drop __grub_efi_api attribute from shim_lock->verify() function ... because (surprisingly) it does not use specific EFI calling convention... Fixes: 6a080b9cd (efi: Add calling convention annotation to all prototypes) Reviewed-by: Ard Biesheuvel 2023-07-03 Samuel Thibault templates: Start pci-arbiter before acpi on Hurd acpi actually needs to access PCI, while pci-arbiter will not be making use of ACPI, so we need to start acpi first. Reviewed-by: Daniel Kiper 2023-07-03 Michał Grzelak configure.ac: Fix typo by adding missing $ During configuration of SDL, variable enable_grub_emu_sdl is checked whether to throw an error message. However, error could not happen because two unequal strings were compared. Fix this by referencing value of enable_grub_emu_sdl, not name. Fixes: 17d6ac1a7 (emu: Add SDL2 support) Reviewed-by: Daniel Kiper 2023-07-03 Glenn Washburn docs: Minor corrections When referring to initrd16 the link for initrd16 should be used, not a link for initrd. Also, correct the spelling of additionally and add a comma after it to correct its grammatical usage. Reviewed-by: Daniel Kiper 2023-07-03 Glenn Washburn kern/misc: Add space after comma in function argument list Reviewed-by: Daniel Kiper commands/regexp: Fix typo Reviewed-by: Daniel Kiper 2023-07-03 Glenn Washburn term/serial: Use grub_strncpy() instead of grub_snprintf() when only copying string Using grub_strncpy() instead of grub_snprintf() is less overhead and indicates clearly that the dest should be the same string as the source. Also fix indentation. Reviewed-by: Daniel Kiper 2023-07-03 Glenn Washburn loader/linux: Print debug message for each generated newc path generated Reviewed-by: Daniel Kiper 2023-07-03 Glenn Washburn include/grub/types.h: Add PRI*GRUB_OFFSET and PRI*GRUB_DISK_ADDR These are currently always the same as PRI*GRUB_UINT64_T, but they may not be in the future. Reviewed-by: Daniel Kiper 2023-07-03 Glenn Washburn kern/misc: Support octal printf format code Also add parenthesis to nested ternary operator to improve clarity. Reviewed-by: Daniel Kiper 2023-07-03 Glenn Washburn gitignore: Ignore python bytecode files Python bytecode files, which end in .pyc, may be generated by the build system as needed and should not go into the git repository. Reviewed-by: Daniel Kiper 2023-07-03 Glenn Washburn loader/linux: Only emit newc directory once When creating at runtime a newc initrd via arguments to initrd with "newc:" prefixes, only emit a directory path record once. The original code intended to do that by bailing out of emitting the record when the record to be created matches an existing record. However, this does not happen because grub_memcmp() is improperly checked. Generating duplicate newc directory records does not cause any problems because the Linux unpacker will skip it once it sees the directory already exists. This fix saves a little processing and makes the generated newc cpio archive a little smaller. Fixes: 92750e4c60 (Add ability to generate newc additions on runtime.) Reviewed-by: Daniel Kiper 2023-07-03 Glenn Washburn loader/efi/linux: Fix formatting and remove unneeded parenthesis Reviewed-by: Daniel Kiper 2023-07-03 Glenn Washburn loader/efi/linux: Print EFI status as hex number instead of uint EFI status codes are of different classes depending on the first byte and all error status codes defined in appendix D of the main spec start from 1 and have the high bit set. When printing as a uint, the decimal is a very large number that needs have the high bit cleared get the spec error code. This can be easily visually done by a human if the number is printed as hex. Reviewed-by: Daniel Kiper 2023-07-03 Oskari Pirhonen docs: Minor edits to debugging chapter Small set of wording and grammatical edits which did not make it in time for the original review of the chapter. Reviewed-by: Glenn Washburn Reviewed-by: Daniel Kiper 2023-06-23 Daniel Kiper lib/relocator: Fix OOB write when initializing lo->freebytes[] Fixes: CID 96636 Reviewed-by: Vladimir Serbinenko 2023-06-23 Daniel Kiper lib/relocator: Enforce GRUB_RELOCATOR_FIRMWARE_REQUESTS_QUANT divisibility by 8 Most of leftover code blindly assumes GRUB_RELOCATOR_FIRMWARE_REQUESTS_QUANT divisibility by 8. So, enforce this at compile time. Reviewed-by: Vladimir Serbinenko 2023-06-23 Julian Andres Klode emu: Add SDL2 support So all we did with the surface in SDL1 was split into window, surface, renderer and texture. Instead of drawing into the surface and then flipping, you build your pixels, then update a texture and then copy the texture to the renderer. Here we use an empty RGB surface to hold our pixels, which enables us to keep most of the code the same. The SDL1 code has been adjusted to refer to "surface" instead of "window" when trying to access the properties of the surface. This approaches the configuration by adding a new --enable-grub-emu-sdl2 argument. If set to yes, or auto detected, it disables SDL1 support automatically. This duplicates the sdl module block in Makefile.core.def which may be something to be aware of, but we also don't want to build separate module. Fixes: https://bugs.debian.org/1038035 Reviewed-by: Daniel Kiper 2023-06-23 Julian Andres Klode emu: SDL style fixes These should be quite obvious and will make the SDL2 patch easier to read then doing it inline there. Reviewed-by: Daniel Kiper 2023-06-23 Michał Grzelak tpm: Enable boot despite unknown firmware failure Currently booting the system is prevented when call to EFI firmware hash_log_extend_event() returns unknown error. Solve this by following convention used in commit a4356538d (commands/tpm: Don't propagate measurement failures to the verifiers layer). Let the system to be bootable by default when unknown TPM error is encountered. Check environment variable tpm_fail_fatal to fallback to previous behaviour. Reviewed-by: Daniel Kiper 2023-06-23 Daniel Kiper bootstrap: Fix patching warnings Currently bootstrap complains in the following way when patching gnulib files: patching file argp-help.c Hunk #1 succeeded at 52 (offset 1 line). Hunk #2 succeeded at 1548 (offset 115 lines). patching file mbswidth.c patching file mbswidth.h Hunk #1 succeeded at 40 (offset -5 lines). Let's fix it by amending line numbers in the patch. Reviewed-by: Alec Brown 2023-06-23 Daniel Kiper efi: Add missing __grub_efi_api attributes The commit bb4aa6e06 (efi: Drop all uses of efi_call_XX() wrappers) did not add some __grub_efi_api attributes to the EFI calls. Lack of them led to hangs on x86_64-efi target. So, let's add missing __grub_efi_api attributes. Fixes: bb4aa6e06 (efi: Drop all uses of efi_call_XX() wrappers) Reported-by: Christian Hesse Reported-by: Robin Candau Tested-by: Robin Candau Tested-by: Christian Hesse Reviewed-by: Peter Jones 2023-06-23 Julian Andres Klode disk: Generalize MD_MAX_DISKS to GRUB_MDRAID_MAX_DISKS Move the constant from grub-core/osdep/linux/getroot.c to include/grub/disk.h and then reuse it in place of the hardcoded 1024 limit in diskfilter. Fixes: 2a5e3c1f2 (disk/diskfilter: Don't make a RAID array with more than 1024 disks) Cc: Daniel Axtens Cc: Kees Cook Reviewed-by: Kees Cook Reviewed-by: Daniel Kiper 2023-06-23 Xiaotian Wu loongarch: Disable relaxation relocations A working GRUB cannot be built with upcoming binutils and GCC, because linker relaxation was added [1] causing new unsupported relocations to appear in modules. So we pass -mno-relax to GCC if it is supported, to disable relaxation and make GRUB forward-compatible with new toolchains. While similar code already exists for sparc64 in configure.ac, sparc64 sets LDFLAGS while LoongArch requires CFLAGS to be set. If we only set LDFLAGS on LoongArch, GCC will still generate relaxation relocations in the .o files, so the sparc64 code cannot be reused. [1] https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=56576f4a722b7398d35802ecf7d4185c27d6d69b Reviewed-by: Daniel Kiper 2023-06-13 Xiaotian Wu loongarch: Add ELF relocation types documentation and comments See https://github.com/loongson/la-abi-specs/blob/release/laelf.adoc#relocations Reviewed-by: Daniel Kiper 2023-06-13 Xiaotian Wu loongarch: Rename function names According to the relocation documentation, the following function names are renamed to show their exact meaning: - from grub_loongarch64_xxx64_hi12() to grub_loongarch64_abs64_hi12(), - from grub_loongarch64_xxx64_hi12() to grub_loongarch64_abs64_lo20(). Reviewed-by: Daniel Kiper 2023-06-13 Xiaotian Wu util/grub-mkimagexx: Optimize code using pc variable We already have the pc variable, no need to calculate it again. Reviewed-by: Daniel Kiper 2023-06-13 Xiaotian Wu kern/{arm64,loongarch64}/dl_helper: Use the correct format specifier for formatted output Use PRIxGRUB_INT64_T format specifier for grub_int64_t type and drop redundant casts. Reviewed-by: Daniel Kiper 2023-06-13 Qiumiao Zhang kern/acpi: Use xsdt_addr if present According to the ACPI specification, in ACPI 2.0 or later, an ACPI-compatible OS must use the XSDT if present. So, we should use xsdt_addr instead of rsdt_addr if xsdt_addr is valid. Reviewed-by: Daniel Kiper 2023-06-13 Qiumiao Zhang commands/acpi: Use xsdt_addr if present According to the ACPI specification, in ACPI 2.0 or later, an ACPI-compatible OS must use the XSDT if present. So, we should use xsdt_addr instead of rsdt_addr if xsdt_addr is valid. Reviewed-by: Daniel Kiper 2023-06-13 Lidong Chen fs/udf: Fix out of bounds access Implemented a boundary check before advancing the allocation descriptors pointer. Reviewed-by: Darren Kenny Reviewed-by: Daniel Kiper 2023-06-13 Glenn Washburn docs: Add debugging chapter to development documentation Debugging GRUB can be tricky and require arcane knowledge. This will help those unfamiliar with the process to get started debugging GRUB with less effort. Reviewed-by: Daniel Kiper 2023-06-13 Darren Kenny fs/xfs: Fix issues found while fuzzing the XFS filesystem While performing fuzz testing with XFS filesystem images with ASAN enabled, several issues were found where the memory accesses are made beyond the data that is allocated into the struct grub_xfs_data structure's data field. The existing structure didn't store the size of the memory allocated into the buffer in the data field and had no way to check it. To resolve these issues, the data size is stored to enable checks into the data buffer. With these checks in place, the fuzzing corpus no longer cause any crashes. Reviewed-by: Daniel Kiper 2023-06-13 Alexander Kanavin util/import_unicode.py: Ensure output is deterministic Ensure the generated unidata.c file is deterministic by sorting the keys of the dict. Reviewed-by: Daniel Kiper 2023-06-13 Alexander Kanavin grub-core/genmoddep.awk: Ensure output is deterministic The output in moddep.lst generated from syminfo.lst using genmoddep.awk is not deterministic since the order of the dependencies on each line can vary depending on how awk sorts the values in the array. Be deterministic in the output by sorting the dependencies on each line. Reviewed-by: Daniel Kiper 2023-06-13 Alexander Kanavin gentpl.py: Ensure output is deterministic The output of the SOURCES lines in grub-core/Makefile.core.am, generated from grub-core/Makefile.core.def with gentpl.py is not deterministic due to missing sorting of the list used to generate it. Add such a sort. Reviewed-by: Daniel Kiper 2023-06-01 Glenn Washburn gdb: Add gdbinfo command for printing the load address of the EFI application EFI firmware determines where to load the GRUB EFI at runtime, and so the addresses of debug symbols are not known ahead of time. There is a command defined in the gdb_grub script which will load the debug symbols at the appropriate addresses, if given the application load address for GRUB. So add a command named "gdbinfo" to allow the user to print this GDB command string with the application load address on-demand. For the outputted GDB command to have any effect when entered into a GDB session, GDB should have been started with the script as an argument to the -x option or sourced into an active GDB session before running the outputted command. Documentation for the gdbinfo command is also added. Co-developed-by: Peter Jones Reviewed-by: Daniel Kiper 2023-06-01 Glenn Washburn loader/efi/chainloader: Do not require a $root visible to EFI firmware when chainloading The EFI chainloader checks that a device path can be created for the $root device before allowing chainloading to a given file. This is probably to ensure that the given file can be accessed and loaded by the firmware. However, since GRUB is loading the image itself, the firmware need not be able to access the file location of the image. So remove this check. Also, this fixes an issue where chainloading an image file on a location that is accessible by the firmware, e.g. (hd0,1)/efi/boot.efi, would fail when root is a location inaccessible by the firmware, e.g. memdisk. Use GRUB_EFI_BYTES_TO_PAGES() instead of doing the calculation explicitly. Add comment noting the section where the load options for the chainloaded EFI application is constructed. Reviewed-by: Ard Biesheuvel Reviewed-by: Daniel Kiper 2023-06-01 Glenn Washburn docs: Document extra arguments to chainloader on EFI Extra arguments given to chainloader on EFI platforms will be sent to the chainloaded application. Also, minor edit in the chainloading section to note that chainloading can be a jump via the firmware and not necessarily in real mode (which does not exist on some architectures). Reviewed-by: Ard Biesheuvel Reviewed-by: Daniel Kiper 2023-06-01 Oliver Steffen util/grub.d/25_bli.in: Activate bli module on EFI Add a new configuration drop-in file that loads the bli module and runs the command if booting on the EFI platform. Reviewed-by: Daniel Kiper 2023-06-01 Oliver Steffen bli: Add a module for the Boot Loader Interface Add a new module named bli. It implements a small but quite useful part of the Boot Loader Interface [0]. This interface uses EFI variables for communication between the boot loader and the operating system. When loaded, this module sets two EFI variables under the vendor GUID 4a67b082-0a4c-41cf-b6c7-440b29bb8c4f: - LoaderInfo: contains GRUB + . This allows the running operating system to identify the boot loader used during boot. - LoaderDevicePartUUID: contains the partition UUID of the EFI System Partition (ESP). This is used by systemd-gpt-auto-generator [1] to find the root partitions (and others too), via partition type IDs [2]. This module is available on EFI platforms only. The bli module relies on the part_gpt module which has to be loaded beforehand to make the GPT partitions discoverable. Update the documentation, add a new chapter "Modules" and describe the bli module there. [0] https://systemd.io/BOOT_LOADER_INTERFACE/ [1] https://www.freedesktop.org/software/systemd/man/systemd-gpt-auto-generator.html [2] https://uapi-group.org/specifications/specs/discoverable_partitions_specification/ Reviewed-by: Daniel Kiper 2023-06-01 Oliver Steffen kern: Check for NULL when closing devices and disks Add checks for NULL pointers to grub_device_close() and grub_disk_close() to make these functions more robust. Reviewed-by: Daniel Kiper 2023-06-01 Oliver Steffen docs: Reword section headings Reword some section headings, remove "The List of" from titles. While grammatically correct, this phrase can be omitted to increase readability, especially in the table of contents. Reviewed-by: Daniel Kiper 2023-06-01 Oliver Steffen efi: Add grub_efi_set_variable_to_string() Add a function that sets an EFI variable to a string value. The string is converted from UTF-8 to UTF-16. Reviewed-by: Daniel Kiper 2023-06-01 Oliver Steffen kern/misc, kern/efi: Extract UTF-8 to UTF-16 code Create a new function for UTF-8 to UTF-16 conversion called grub_utf8_to_utf16_alloc() in the grub-code/kern/misc.c and replace charset conversion code used in some places in the EFI code. It is modeled after the grub_utf8_to_ucs4_alloc() like functions in include/grub/charset.h. It can't live in include/grub/charset.h, because it needs to be reachable from the kern/efi code. Add a check for integer overflow and remove redundant NUL-termination. Reviewed-by: Daniel Kiper 2023-06-01 Oliver Steffen include/grub/types.h: Add GRUB_SSIZE_MAX In the same way as GRUB_SIZE_MAX, add GRUB_SSIZE_MAX. Reviewed-by: Daniel Kiper 2023-06-01 Oliver Steffen guid: Make use of GUID printf format specifier Use the new printf format specifier %pG. Fixes the text representation of GUIDs in the output of the lsefisystab command (missing 4th dash). Reviewed-by: Daniel Kiper 2023-06-01 Oliver Steffen kern/misc: Add a format specifier GUIDs Extend the printf format specifier for pointers (%p) to accept a suffix specifier G to print GUIDs: %pG can be used to print grub_guid structs. This does not interfere with the -Wformat checking of gcc. Note that the data type is not checked though (%p accepts void *). Reviewed-by: Daniel Kiper 2023-06-01 Oliver Steffen guid: Unify GUID types There are 3 implementations of a GUID in GRUB. Replace them with a common one, placed in types.h. It uses the "packed" flavor of the GUID structs, the alignment attribute is dropped, since it is not required. Reviewed-by: Daniel Kiper 2023-06-01 Oliver Steffen efi: Add grub_efi_set_variable_with_attributes() Add a function to the EFI module that allows setting EFI variables with specific attributes. This is useful for marking variables as volatile, for example. Reviewed-by: Daniel Kiper 2023-05-25 Alec Brown kern/efi/mm: Fix use-after-free in finish boot services In grub-core/kern/efi/mm.c, grub_efi_finish_boot_services() has an instance where the memory for the variable finish_mmap_buf is freed, but on the next iteration of a while loop, grub_efi_get_memory_map() uses finish_mmap_buf. To prevent this, we can set finish_mmap_buf to NULL after the free. Reviewed-by: Daniel Kiper 2023-05-25 Ard Biesheuvel efi: Handle NULL return value when getting loaded image protocol The EFI spec mandates that the handle produced by the LoadImage boot service has a LoadedImage protocol instance installed on it, but for robustness, we should still deal with a NULL return value from the helper routine that obtains this protocol pointer. If this happens, don't try to start the image but unload it and return an error. Reviewed-by: Daniel Kiper 2023-05-25 Ard Biesheuvel efi: Use generic EFI loader for x86_64 and i386 Switch the x86 based EFI platform builds to the generic EFI loader, which exposes the initrd via the LoadFile2 protocol instead of the x86-specific setup header. This will launch the Linux kernel via its EFI stub, which performs its own initialization in the EFI boot services context before calling ExitBootServices() and performing the bare metal Linux boot. Given that only Linux kernel versions v5.8 and later support this initrd loading method, the existing x86 loader is retained as a fallback, which will also be used for Linux kernels built without the EFI stub. In this case, GRUB calls ExitBootServices() before entering the Linux kernel, and all EFI related information is provided to the kernel via struct boot_params in the setup header, as before. Note that this means that booting EFI stub kernels older than v5.8 is not supported even when not using an initrd at all. Also, the EFI handover protocol, which has no basis in the UEFI specification, is not implemented. Reviewed-by: Daniel Kiper 2023-05-25 Ard Biesheuvel efi: Remove x86_64 call wrappers The call wrappers are no longer needed now that GCC can generate function calls using MS calling convention, so let's get rid of them. Reviewed-by: Daniel Kiper 2023-05-25 Ard Biesheuvel efi: Drop all uses of efi_call_XX() wrappers Now that GCC can generate function calls using the correct calling convention for us, we can stop using the efi_call_XX() wrappers, and just dereference the function pointers directly. This avoids the untyped variadic wrapper routines, which means better type checking for the method calls. Reviewed-by: Daniel Kiper 2023-05-25 Ard Biesheuvel efi: Add calling convention annotation to all prototypes UEFI mandates MS calling convention on x86_64, which was not supported on GCC when UEFI support was first introduced into GRUB. However, now we can use the ms_abi function type attribute to annotate functions and function pointers as adhering to the MS calling convention, and the compiler will generate the correct instruction sequence for us. So let's add the appropriate annotation to all the function prototypes. This will allow us to drop the special call wrappers in a subsequent patch. Reviewed-by: Daniel Kiper 2023-05-25 Ard Biesheuvel efi: Make EFI PXE protocol methods non-callable The grub_efi_pxe_t struct definition has placeholders for the various protocol method pointers, given that they are never called in the code, and the prototypes have been omitted, and therefore do not comply with the UEFI spec. So let's convert them into void* pointers, so they cannot be called inadvertently. Reviewed-by: Daniel Kiper 2023-05-25 Alec Brown loader/multiboot_elfxx: Check program header offset doesn't exceed constraints In grub-core/loader/multiboot_elfxx.c, we need to make sure that the program header offset is less than the file size along with the MULTIBOOT_SEARCH constant. We can do so by setting the variable phlimit to the minimum value of the two limits and check it each time we change program header index to insure that the program header offset isn't outside of the limits. Fixes: CID 314029 Fixes: CID 314038 Reviewed-by: Daniel Kiper 2023-05-25 Alec Brown loader/multiboot_elfxx: Check section header region before allocating memory In grub-core/loader/multiboot_elfxx.c, space is being allocated for the section header region, but isn't verifying if the region is within the file's size. Before calling grub_calloc(), we can add a conditional to check if the section header region is smaller than the file size. Fixes: CID 314029 Fixes: CID 314038 Reviewed-by: Daniel Kiper 2023-05-25 Alec Brown loader/multiboot_elfxx: Check program memory isn't larger than allocated memory size In grub-core/loader/multiboot_elfxx.c, the code is filling an area of memory with grub_memset() but doesn't check if there is space in the allocated memory before doing so. To make sure we aren't zeroing memory past the allocated memory region, we need to check that the offset into the allocated memory region plus the memory size of the program is smaller than the allocated memory size. Fixes: CID 314029 Fixes: CID 314038 Reviewed-by: Daniel Kiper 2023-05-25 WANG Xuerui kern/loongarch64/dl_helper: Avoid undefined behavior when popping from an empty reloc stack The return value of grub_loongarch64_stack_pop() is unsigned, so -1 should not be used in the first place. Replacing with 0 is enough to avoid the UB in this edge case. Technically though, proper error handling is needed throughout the management of the reloc stack, so no unexpected behavior will happen even in case of malformed object code input (right now, pushes become no-ops when the stack is full, and garbage results if the stack does not contain enough operands for an op). The refactor would touch some more places so would be best done in a separate series. Fixes: CID 407777 Fixes: CID 407778 Reviewed-by: Daniel Kiper 2023-05-25 Peter Zijlstra (Intel) pci: Rename GRUB_PCI_CLASS_* Glenn suggested to rename the existing PCI_CLASS defines to have explicit class and subclass names. Suggested-by: Glenn Washburn Reviewed-by: Daniel Kiper 2023-05-25 Peter Zijlstra (Intel) term/serial: Add support for PCI serial devices Loosely based on early_pci_serial_init() from Linux, allow GRUB to make use of PCI serial devices. Specifically, my Alderlake NUC exposes the Intel AMT SoL UART as a PCI enumerated device but doesn't include it in the EFI tables. Tested and confirmed working on a "Lenovo P360 Tiny" with Intel AMT enabled. This specific machine has (from lspci -vv): 00:16.3 Serial controller: Intel Corporation Device 7aeb (rev 11) (prog-if 02 [16550]) DeviceName: Onboard - Other Subsystem: Lenovo Device 330e Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- SERR- Reviewed-by: Daniel Kiper 2023-05-17 Glenn Washburn tests/util/grub-fs-tester: Avoid failing some file system tests due to file system filling up On some systems /usr/share/dict/american-english can be larger than the available space on the filesystem being tested (e.g. vfat12a). This causes a failure of the filesystem test and is not a real test failure. Instead, use dd to copy at most 1 MiB of data to the filesystem, which is enough for our purposes and will not fill any of the tested filesystems. Reviewed-by: Daniel Kiper 2023-05-17 Glenn Washburn docs: Command-line and menu entry commands are now separated The menu entry commands now have their own section. Change the wording in the section that they were in to reflect this. Reviewed-by: Daniel Kiper 2023-05-17 Roger Pau Monné lib/relocator: Always enforce the requested alignment in malloc_in_range() On failure to allocate from grub_relocator_firmware_alloc_region() in malloc_in_range() the function would stop enforcing the alignment, and the following was returned: lib/relocator.c:431: trying to allocate in 0x200000-0xffbf9fff aligned 0x200000 size 0x406000 lib/relocator.c:1197: allocated: 0x74de2000+0x406000 lib/relocator.c:1407: allocated 0x74de2000/0x74de2000 Fix this by making sure that target always contains a suitably aligned address. After the change the return from the function is: lib/relocator.c:431: trying to allocate in 0x200000-0xffb87fff aligned 0x200000 size 0x478000 lib/relocator.c:1204: allocated: 0x74c00000+0x478000 lib/relocator.c:1414: allocated 0x74c00000/0x74c00000 Fixes: 3a5768645c05 (First version of allocation from firmware) Reviewed-by: Daniel Kiper 2023-05-17 Benjamin Herrenschmidt term/ns8250: Fix incorrect usage of access_size The access_size is part of a union, so doesn't technically exist for a PIO port (i.e., not MMIO), but we set it anyways. This doesn't cause a bug today because the other leg of the union doesn't have anything overlapping with it now, but it's bad, I will punish myself for writing it that way :-) In the meantime, fix this and actually name the struct inside the union for clarity of intent and to avoid such issue in the future. Reviewed-by: Daniel Kiper 2023-05-17 Ákos Nagy util/grub-install-common: Fix the key of the --core-compress option Commit f23bc6510 (Transform -C option to grub-mkstandalone to --core-compress available in all grub-install flavours.) declared a new long option for specifying the compression method to use for the core image. However, the option key has not been replaced in the parser function, it still expects the old one formerly used by grub-mkstandalone. Because of this the option is not recognized by any of the utils for which it is listed as supported. Reviewed-by: Daniel Kiper 2023-05-17 Lidong Chen fs/hfsplus: Set grub_errno to prevent NULL pointer access When an invalid node size is detected in grub_hfsplus_mount(), data pointer is freed. Thus, file->data is not set. The code should also set the grub_errno when that happens to indicate an error and to avoid accessing the uninitialized file->data in grub_file_close(). Reviewed-by: Daniel Kiper 2023-05-17 Lidong Chen fs/hfsplus: Prevent out of bound access in catalog file A corrupted hfsplus can have a catalog key that is out of range. This can lead to out of bound access when advancing the pointer to access catalog file info. The valid range of a catalog key is specified in HFS Plus Technical Note TN1150 [1]. [1] https://developer.apple.com/library/archive/technotes/tn/tn1150.html Reviewed-by: Daniel Kiper 2023-05-17 Lidong Chen fs/hfsplus: Validate btree node size The invalid btree node size can cause crashes when parsing the btree. The fix is to ensure the btree node size is within the valid range defined in the HFS Plus technical note, TN1150 [1]. [1] https://developer.apple.com/library/archive/technotes/tn/tn1150.html Reviewed-by: Daniel Kiper 2023-05-17 Glenn Washburn INSTALL: Use exfat-utils package instead of exfatprogs The exfat-utils package is an older package complementing exfat-fuse, and was the only exfat tools for a long time. The exfat filesystem testing code was written with these tools in mind. A newer project exfatprogs appears to be of better quality and functionality and was written to complement the somewhat new exfat kernel module. Ideally we should be using the newer exfatprogs. However, the command line interface for mkfs.exfat is different between the two. So we can't use the exfatprogs tools until the test scripts have been updated to account for this. Recommend installing exfat-utils instead of exfatprogs for now. Reviewed-by: Daniel Kiper 2023-05-17 Glenn Washburn INSTALL: Document that building grub-mkfont requires xfonts-unifont Reviewed-by: Daniel Kiper 2023-05-17 Renaud Métrich net/dns: Fix lookup error when no IPv6 is returned When trying to resolve DNS names into IP addresses, the DNS code fails from time to time with the following error: -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- error: ../../grub-core/net/dns.c:688:no DNS record found. -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- This happens when both IPv4 and IPv6 queries are performed against the DNS server (e.g. 8.8.8.8) but there is no IP returned for IPv6 query, as shown below: -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- grub> net_del_dns 192.168.122.1 grub> net_add_dns 8.8.8.8 grub> net_nslookup ipv4.test-ipv6.com error: ../../grub-core/net/dns.c:688:no DNS record found. grub> net_nslookup ipv4.test-ipv6.com 216.218.228.115 -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- The root cause is the code exiting prematurely when the data->addresses buffer has been allocated in recv_hook(), even if there was no address returned last time recv_hook() executed. Reviewed-by: Daniel Kiper 2023-05-17 Renaud Métrich net/dns: Add debugging messages in recv_hook() function Reviewed-by: Daniel Kiper net/dns: Simplify error handling of recv_hook() function Reviewed-by: Daniel Kiper 2023-05-17 Renaud Métrich net/dns: Fix removal of DNS server When deleting the DNS server, we get the following error message: -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- grub> net_del_dns 192.168.122.1 error: ../../grub-core/net/dns.c:646:no DNS reply received. -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- This happens because the implementation is broken, it does a "add" internally instead of a "delete". Reviewed-by: Daniel Kiper 2023-05-17 Xiaotian Wu tests: Add LoongArch to various test cases I ran the test suite on a 3A5000 desktop, a LoongArch architecture machine, using Archlinux for LoongArch distro, see https://github.com/loongarchlinux. Some software versions are: * linux 6.3.0-rc4 * gcc 13.0.1 20230312 * binutils 2.40 * qemu 7.2.0 The test results of running "make check" with qemu 7.2 are as follows: ================================= GRUB 2.11: ./test-suite.log ================================= # TOTAL: 85 # PASS: 73 # SKIP: 8 # XFAIL: 0 # FAIL: 2 # XPASS: 0 # ERROR: 2 .. contents:: :depth: 2 ERROR: f2fs_test ================ mount: /tmp/grub-fs-tester.20230418175640563815408.f2fs.UDs/f2fs_rw: unknown filesystem type 'f2fs'. dmesg(1) may have more information after failed mount system call. MOUNT FAILED. ERROR f2fs_test (exit status: 99) FAIL: hfs_test ============== recode: Request `utf8..macroman' is erroneous mkfs.hfs: name required with -v option FAIL hfs_test (exit status: 1) ERROR: zfs_test =============== zpool not installed; cannot test zfs. ERROR zfs_test (exit status: 99) SKIP: pata_test =============== SKIP pata_test (exit status: 77) SKIP: ahci_test =============== SKIP ahci_test (exit status: 77) SKIP: uhci_test =============== SKIP uhci_test (exit status: 77) SKIP: ohci_test =============== SKIP ohci_test (exit status: 77) SKIP: ehci_test =============== SKIP ehci_test (exit status: 77) SKIP: fddboot_test ================== SKIP fddboot_test (exit status: 77) SKIP: netboot_test ================== SKIP netboot_test (exit status: 77) SKIP: pseries_test ================== SKIP pseries_test (exit status: 77) FAIL: grub_func_test ==================== WARNING: Image format was not specified for '/tmp/grub-shell.HeTAD8Ty3U/grub.iso' and probing guessed raw. Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. Specify the 'raw' format explicitly to remove the restrictions. Functional test failure: shift_test: ... gfxterm_menu_640x480xi16:3 failed: 0xce34981e vs 0xd9f04953 tests/video_checksum.c:checksum:615: assert failed: 0 Checksum gfxterm_menu_640x480xi16:2 failed: 0xa8fb749d vs 0xbf3fa5d0 tests/video_checksum.c:checksum:615: assert failed: 0 Checksum gfxterm_menu_640x480xi16:1 failed: 0xce34981e vs 0xd9f04953 gfxterm_menu: FAIL ... videotest_checksum: videotest_checksum: PASS exfctest: exfctest: PASS TEST FAILURE FAIL grub_func_test (exit status: 1) We got 2 errors: * f2fs_test The kernel uses 16k pages, causing failures when loading the f2fs kernel module, see https://github.com/torvalds/linux/blob/master/fs/f2fs/super.c#L4670 This error can be ignored. * zfs_test zfs does not support the LoongArch architecture and is not compatible with the 6.3 kernel. This error can be ignored. We got 2 failures: * hfs_test I use recode 3.7.14-1 on Archlinux, running `recode -l` gives no output `MacRoman`, so we get this error. On Linux systems that support LoongArch, there is currently no need to use HFS, so this failure can be ignored. * grub_func_test I don't know the reason for this failure. I guess it may be related to qemu's edk2. In the previous review, I was told that the failure here is the expected behavior. So, we can ignore this failure. Reviewed-by: Daniel Kiper 2023-05-17 Xiaotian Wu tests: Fix timezone inconsistency in squashfs_test The image timestamp was not returned in UTC, but the following logic expected and used UTC. This patch fixes the test failure like described below: unsquashfs -s /tmp/grub-fs-tester.20230407111703613257436.squash4_gzip.9R4/squash4_gzip_512_4096_1_0.img grep '^Creation' awk '{print $6 " " $7 " " $8 " " $9 " " $10; }' FSTIME='Fri Apr 7 11:17:05 2023' date -d 'Fri Apr 7 11:17:05 2023' -u '+%Y-%m-%d %H:%M:%S' FSTIME='2023-04-07 11:17:05' date -d '2023-04-07 11:17:05 UTC -1 second' -u '+%Y-%m-%d %H:%M:%S' FSTIMEM1='2023-04-07 11:17:04' date -d '2023-04-07 11:17:05 UTC -2 second' -u '+%Y-%m-%d %H:%M:%S' FSTIMEM2='2023-04-07 11:17:03' date -d '2023-04-07 11:17:05 UTC -3 second' -u '+%Y-%m-%d %H:%M:%S' FSTIMEM3='2023-04-07 11:17:02' grep -F 'Last modification time 2023-04-07 11:17:05' echo 'Device loop0: Filesystem type squash4 - Last modification time 2023-04-07 03:17:05 Friday - Sector size 512B - Total size 10680KiB' echo 'Device loop0: Filesystem type squash4 - Last modification time 2023-04-07 03:17:05 Friday - Sector size 512B - Total size 10680KiB' grep -F 'Last modification time 2023-04-07 11:17:04' echo 'Device loop0: Filesystem type squash4 - Last modification time 2023-04-07 03:17:05 Friday - Sector size 512B - Total size 10680KiB' grep -F 'Last modification time 2023-04-07 11:17:03' echo 'Device loop0: Filesystem type squash4 - Last modification time 2023-04-07 03:17:05 Friday - Sector size 512B - Total size 10680KiB' grep -F 'Last modification time 2023-04-07 11:17:02' echo FSTIME FAIL Reviewed-by: Glenn Washburn Reviewed-by: Daniel Kiper 2023-05-17 Xiaotian Wu loongarch: Add to build system This patch adds LoongArch to the GRUB build system and various tools, so GRUB can be built on LoongArch as a UEFI application. Reviewed-by: Daniel Kiper 2023-05-17 Xiaotian Wu loongarch: Add auxiliary files Add support for manipulating architectural cache and timers, and EFI memory maps. Reviewed-by: Daniel Kiper 2023-05-17 Xiaotian Wu loongarch: Add support for ELF psABI v2.00 relocations A new set of relocation types was added in the LoongArch ELF psABI v2.00 spec [1], [2] to replace the stack-based scheme in v1.00. Toolchain support is available from binutils 2.40 and gcc 13 onwards. This patch adds support for the new relocation types, that are simpler to handle (in particular, stack operations are gone). Support for the v1.00 relocs are kept for now, for compatibility with older toolchains. [1] https://github.com/loongson/LoongArch-Documentation/pull/57 [2] https://loongson.github.io/LoongArch-Documentation/LoongArch-ELF-ABI-EN.html#_appendix_revision_history Reviewed-by: Daniel Kiper 2023-05-17 Xiaotian Wu loongarch: Add support for ELF psABI v1.00 relocations This patch adds support of the stack-based LoongArch relocations throughout GRUB, including tools, dynamic linkage, and support for conversion of ELF relocations into PE ones. A stack machine is required to handle these per the spec [1] (see the R_LARCH_SOP types), of which a simple implementation is included. These relocations are produced by binutils 2.38 and 2.39, while the newer v2.00 relocs require more recent toolchain (binutils 2.40+ & gcc 13+, or LLVM 16+). GCC 13 has not been officially released as of early 2023, so support for v1.00 relocs are expected to stay relevant for a while. [1] https://loongson.github.io/LoongArch-Documentation/LoongArch-ELF-ABI-EN.html#_relocations Reviewed-by: Daniel Kiper 2023-05-17 Xiaotian Wu loongarch: Add early startup code On entry, we need to save the system table pointer as well as our image handle. Add an early startup file that saves them and then brings us into our main function. Reviewed-by: Daniel Kiper 2023-05-17 Xiaotian Wu loongarch: Add setjmp implementation This patch adds a setjmp implementation for LoongArch. Reviewed-by: Daniel Kiper 2023-05-17 Xiaotian Wu elf: Add LoongArch definitions Add ELF e_machine ID [1] and relocations types [2] for LoongArch to the current in-repo definitions. [1] https://loongson.github.io/LoongArch-Documentation/LoongArch-ELF-ABI-EN.html#_e_machine_identifies_the_machine [2] https://loongson.github.io/LoongArch-Documentation/LoongArch-ELF-ABI-EN.html#_relocations Reviewed-by: Daniel Kiper 2023-05-17 Xiaotian Wu pe: Add LoongArch definitions Add PE machine types [1] and relocation types [2] for LoongArch to the current in-repo definitions. [1] https://learn.microsoft.com/en-us/windows/win32/debug/pe-format#machine-types [2] https://learn.microsoft.com/en-us/windows/win32/debug/pe-format#base-relocation-types Reviewed-by: Daniel Kiper 2023-05-16 Chris Coulson font: Try opening fonts from the bundled memdisk GRUB since 93a786a00 (kern/efi/sb: Enforce verification of font files) has enforced verification of font files in secure boot mode. In order to continue to be able to load some default fonts, vendors may bundle them with their signed EFI image by adding them to the built-in memdisk. This change makes the font loader try loading fonts from the memdisk before the prefix path when attempting to load a font file by specifying its filename, which avoids having to make changes to GRUB configurations in order to accommodate memdisk bundled fonts. It expects the directory structure to be the same as fonts stored in the prefix path, i.e. /fonts/.pf2. Reviewed-by: Steve McIntyre <93sam@debian.org> Tested-by: Steve McIntyre <93sam@debian.org> Reviewed-by: Robbie Harwood Reviewed-by: Daniel Kiper 2023-05-16 Robbie Harwood Aaron Miller Peter Jones net: Read bracketed IPv6 addrs and port numbers Allow specifying port numbers for http and tftp paths and allow IPv6 addresses to be recognized with brackets around them, which is required to specify a port number. Reviewed-by: Daniel Kiper 2023-05-16 Robbie Harwood Revert "net/http: Allow use of non-standard TCP/IP ports" The notation introduced in ac8a37dda (net/http: Allow use of non-standard TCP/IP ports) contradicts that used in downstream distributions including Fedora, RHEL, Debian, Ubuntu, and others. Revert it and apply the downstream notation which was originally proposed to the GRUB in 2016. This reverts commit ac8a37dda (net/http: Allow use of non-standard TCP/IP ports). Reviewed-by: Daniel Kiper 2023-05-16 Riku Viitanen term/at_keyboard: Add timeout to fix hang on HP EliteBooks This fixes the GRUB on Coreboot on HP EliteBooks by implementing a 200 ms timeout. The GRUB used to hang. Fixes: https://ticket.coreboot.org/issues/141 Reviewed-by: Daniel Kiper 2023-04-13 Glenn Washburn tests/util/grub-fs-tester: Add missing redirect to /dev/null In filesystem timestamp test, a check is done to verify that the timestamp for a file as reported in Linux by the filesystem is within a few seconds of the timestamp as reported by GRUB. This is done by grepping the output of GRUB's ls command for the timestamp as reported by the filesystem in Linux and for each of 3 seconds past that timestamp. All of these checks except one redirect the output of grep to /dev/null. Fix this exception to behave as the other checks. Reviewed-by: Daniel Kiper 2023-04-13 Mukesh Kumar Chaurasiya disk: Replace transform_sector() function with grub_disk_to_native_sector() The transform_sector() function is not very clear in what it's doing and confusing. The GRUB already has a function which is doing the same thing in a very self explanatory way, i.e., grub_disk_to_native_sector(). So, it's much better to use self explanatory one than transform_sector(). Reviewed-by: Daniel Kiper 2023-04-13 Thomas Schmitt tests: Add test for iso9660 delayed CE hop The ISO filesystem image iso9660_early_ce.iso exposes the unusual situation that the Rock Ridge name entry of its only file is located after a CE entry which points to the next continuation area. The correct behavior is to read the Rock Ridge name and to only then load the next continuation area. If GRUB performs this correctly, then the name "RockRidgeName:x" will be read and reported by grub-fstest. If GRUB wrongly performs the CE hop immediately when encountering the CE entry, then the dull ISO 9660 name "rockridg" will not be overridden and be put out by grub-fstest. Tested-by: Lidong Chen Reviewed-by: Daniel Kiper 2023-04-13 Thomas Schmitt fs/iso9660: Delay CE hop until end of current SUSP area The SUSP specs demand that the reading of the next SUSP area which is depicted by a CE entry shall be delayed until reading of the current SUSP area is completed. Up to now GRUB immediately ends reading of the current area and loads the new one. So, buffer the parameters of a found CE entry and perform checks and reading of new data only after the reader loop has ended. Tested-by: Lidong Chen Reviewed-by: Daniel Kiper 2023-03-29 Avnish Chouhan kern/ieee1275/init: Extended support in Vec5 This patch enables multiple options in Vec5 which are required and solves the boot issues seen on some machines which are looking for these specific options. 1. LPAR: Client program supports logical partitioning and associated hcall()s. 2. SPLPAR: Client program supports the Shared Processor LPAR Option. 3. DYN_RCON_MEM: Client program supports the “ibm,dynamic-reconfiguration-memory” property and it may be presented in the device tree. 4. LARGE_PAGES: Client supports pages larger than 4 KB. 5. DONATE_DCPU_CLS: Client supports donating dedicated processor cycles. 6. PCI_EXP: Client supports PCI Express implementations utilizing Message Signaled Interrupts (MSIs). 7. CMOC: Enables the Cooperative Memory Over-commitment Option. 8. EXT_CMO: Enables the Extended Cooperative Memory Over-commit Option. 9. ASSOC_REF: Enables “ibm,associativity” and “ibm,associativity-reference-points” properties. 10. AFFINITY: Enables Platform Resource Reassignment Notification. 11. NUMA: Supports NUMA Distance Lookup Table Option. 12. HOTPLUG_INTRPT: Supports Hotplug Interrupts. 13. HPT_RESIZE: Enable Hash Page Table Resize Option. 14. MAX_CPU: Defines maximum number of CPUs supported. 15. PFO_HWRNG: Supports Random Number Generator. 16. PFO_HW_COMP: Supports Compression Engine. 17. PFO_ENCRYPT: Supports Encryption Engine. 18. SUB_PROCESSORS: Supports Sub-Processors. 19. DY_MEM_V2: Client program supports the “ibm,dynamic-memory-v2” property in the “ibm,dynamic-reconfiguration-memory” node and it may be presented in the device tree. 20. DRC_INFO: Client program supports the “ibm,drc-info” property definition and it may be presented in the device tree. Reviewed-by: Daniel Kiper 2023-03-29 Avnish Chouhan kern/ieee1275/init: Convert plain numbers to constants in Vec5 This patch converts the plain numbers used in Vec5 properties to constants. 1. LPAR: Client program supports logical partitioning and associated hcall()s. 2. SPLPAR: Client program supports the Shared Processor LPAR Option. 3. CMO: Enables the Cooperative Memory Over-commitment Option. 4. MAX_CPU: Defines maximum number of CPUs supported. Reviewed-by: Daniel Kiper 2023-03-29 Robbie Harwood loader/emu/linux: Work around systemctl kexec returning Per systemctl(1), it "is asynchronous; it will return after the reboot operation is enqueued, without waiting for it to complete". This differs from kexec(8), which calls reboot(2) and therefore does not return. When not using fallback, this confusingly results in: error trying to perform 'systemctl kexec': 0 Aborted. Press any key to exit. on screen for a bit, followed by successful kexec. To reduce the likelihood of hitting this case, add a delay on successful return. Ultimately, the systemd interface is racy: we can't avoid it entirely unless we never fallback on success. Reviewed-by: Daniel Kiper 2023-03-29 Michael Chang tpm: Disable the tpm verifier if the TPM device is not present When the tpm module is loaded, the verifier reads entire file into memory, measures it and uses verified content as a backing buffer for file accesses. However, this process may result in high memory utilization for file operations, sometimes causing a system to run out of memory which may finally lead to boot failure. To address this issue, among others, the commit 887f98f0d (mm: Allow dynamically requesting additional memory regions) have optimized memory management by dynamically allocating heap space to maximize memory usage and reduce threat of memory exhaustion. But in some cases problems may still arise, e.g., when large ISO images are mounted using loopback or when dealing with embedded systems with limited memory resources. Unfortunately current implementation of the tpm module doesn't allow elimination of the back buffer once it is loaded. Even if the TPM device is not present or it has been explicitly disabled. This may unnecessary allocate a lot memory. To solve this issue, a patch has been developed to detect the TPM status at module load and skip verifier registration if the device is missing or deactivated. This prevents allocation of memory for the back buffer, avoiding wasting memory when no real measure boot functionality is performed. Disabling the TPM device in the system can reduce memory usage in the GRUB. It is useful in scenarios where high memory utilization is a concern and measurements of loaded artifacts are not necessary. Reviewed-by: Daniel Kiper 2023-03-29 Glenn Washburn INSTALL: Document programs and packages needed for using gdb_grub script Now that the gdb_grub script uses the Python API in GDB, a GDB with Python support must be used. Note that this means a GDB with version greater than 7.0 must be used. This should not be an issue since that was released over a decade ago. Also, the minimum version of Python must be 3.5, which was released around 8 years ago. Reviewed-by: Daniel Kiper 2023-03-29 Atish Patra RISC-V: Use common linux loader RISC-V doesn't have to do anything very different from other architectures to loader EFI stub linux kernel. As a result, just use the common linux loader instead of defining a RISC-V specific linux loader. Reviewed-by: Daniel Kiper 2023-03-29 Atish Patra efi: Remove arch specific image headers for RISC-V, ARM64 and ARM The arch specific image header details are not very useful as most of the GRUB just looks at the PE/COFF spec parameters (PE32 magic and header offset). Remove the arch specific images headers and define a generic arch headers that provide enough PE/COFF fields for the GRUB to parse kernel images correctly. Reviewed-by: Daniel Kiper 2023-03-29 Atish Patra loader/efi: Move ARM64 linux loader to common code ARM64 linux loader code is written in such a way that it can be reused across different architectures without much change. Move it to common code so that RISC-V doesn't have to define a separate loader. Reviewed-by: Daniel Kiper 2023-03-14 Alec Brown util/grub-module-verifierXX: Add module_size parameter to functions for sanity checking In grub-module-verifierXX.c, the function grub_module_verifyXX() performs an initial check that the ELF section headers are within the module's size, but doesn't check if the sections being accessed have contents that are within the module's size. In particular, we need to check that sh_offset and sh_size are less than the module's size. However, for some section header types we don't need to make these checks. For the type SHT_NULL, the section header is marked as inactive and the rest of the members within the section header have undefined values, so we don't need to check for sh_offset or sh_size. In the case of the type SHT_NOBITS, sh_offset has a conceptual offset which may be beyond the module size. Also, this type's sh_size may have a non-zero size, but a section of this type will take up no space in the module. This can all be checked in the function get_shdr(), but in order to do so, the parameter module_size must be added to functions so that the value of the module size can be used in get_shdr() from grub_module_verifyXX(). Also, had to rework some for loops to ensure the index passed to get_shdr() is within bounds. Reviewed-by: Daniel Kiper 2023-03-14 Glenn Washburn gdb: Add extra early initialization symbols for i386-pc Add symbols for boot.image, disk.image, and lzma_decompress.image if the target is i386-pc. This is only done for i386-pc because that is the only target that uses the images. By loading the symbols for these images, these images can be more easily debugged by allowing the setting of break- points in that code and to see easily get the value of data symbols. Reviewed-by: Daniel Kiper 2023-03-14 Glenn Washburn gdb: Modify gdb prompt when running gdb_grub script This will let users know that the GDB session is using the GRUB gdb scripts. Reviewed-by: Daniel Kiper 2023-03-14 Glenn Washburn gdb: Allow running user-defined commands at GRUB start A new command, run_on_start, for things to do before GRUB starts executing. Currently, this is setting up the loading of module symbols as they are loaded and allowing user-defined script to be run if a command named "onstart" exists. On some platforms, notably x86, software breakpoints set in GDB before the GRUB image is loaded will be cleared when the image is loaded. This is because the breakpoints work by overwriting the memory of the break- point location with a special instruction which when hit will cause the debugger to stop execution. Just before execution is resumed by the debugger, the original instruction bytes are put back. When a breakpoint is set before the GRUB image is loaded, the special debugger instruction will be written to memory and when the GRUB image is loaded by the firmware, which has no knowledge of the debugger, the debugger instruction is overwritten. To the GDB user, GDB will show the breakpoint as set, but it will never be hit. Furthermore, GDB now becomes confused, such that even deleting and re-setting the breakpoint after the GRUB image is loaded will not allow for a working breakpoint. To work around this, in run_on_start, first a watchpoint is set on _start, which will be triggered when the firmware starts loading the GRUB image. When the _start watchpoint is hit, the current breakpoints are saved to a file and then deleted by GDB before they can be overwritten by the firmware and confuse GDB. Then a temporary software breakpoint is set on _start, which will get triggered when the firmware hands off to GRUB to execute. In that breakpoint load the previously saved and deleted breakpoints now that there is no worry of them getting overwritten by the firmware. This is needed for runtime_load_module to work when it is run before the GRUB image is loaded. Note that watchpoints are generally types of hardware breakpoints on x86, so its deleted as soon as it gets triggered so that a minimal set of hardware breakpoints are used, allowing more for the user. Reviewed-by: Daniel Kiper 2023-03-14 Glenn Washburn gdb: Add functions to make loading from dynamically positioned targets easier Many targets, such as EFI, load GRUB at addresses that are determined at runtime. So the load addresses in kernel.exec will almost certainly be wrong. Given the address of the start of the text segment, these functions will tell GDB to load the symbols at the proper locations. It is left up to the user to determine how to get the text address of the loaded GRUB image. Reviewed-by: Daniel Kiper 2023-03-14 Glenn Washburn gdb: Replace module symbol loading implementation with Python one Remove gmodule.pl and rewrite as a python in gdb_helper.py. This removes Perl dependency for the GRUB GDB script, but adds Python as a dependency. This is more desirable because Python is tightly integrated with GDB and can do things not even available to GDB native scripting language. GDB must be built with Python, however this is not a major limitation because every major distro non-end-of-life versions build GDB with Python support. And GDB has had support for Python since around 7.1-ish, which is about a decade. This re-implementation has an added feature. If there is a user defined command named "onload_", then that command will be executed after the symbols for the specified module are loaded. When debugging a module it can be desirable to set break points on code in the module. This is difficult in GRUB because, at GDB start, the module is not loaded and on EFI platforms its not known ahead of time where the module will be loaded. So allow users to create an "onload_" command which will be run when the module with name "modname" is loaded. Another addition is a new convenience function is defined $is_user_command(), which returns true if its string argument is the name of a user-defined command. A secondary benefit of these changes is that the script does not write temporary files and has better error handling capabilities. Reviewed-by: Daniel Kiper 2023-03-14 Glenn Washburn gdb: Only connect to remote target once when first sourced The gdb_grub script was originally meant to be run once when GDB first starts up via the -x argument. So it runs commands unconditionally assuming that the script has not been run before. Its nice to be able to source the script again when developing the script to modify/add commands. So only run the commands not defined in user-defined commands, if a variable $runonce has already been set and when those commands have been run to set $runonce. Reviewed-by: Daniel Kiper 2023-03-14 Glenn Washburn gdb: Conditionally run GDB script logic for dynamically or statically positioned GRUB There are broadly two classes of targets to consider when loading symbols for GRUB, targets that determine where to load GRUB at runtime (dynamically positioned) and those that do not (statically positioned). For statically positioned targets, symbol loading is determined at link time, so nothing more needs to be known to load the symbols. For dynamically positioned targets, such as EFI targets, at runtime symbols should be offset by an amount that depends on where the runtime chose to load GRUB. It is important to not load symbols statically for dynamic targets because then when subsequently loading the symbols correctly one must take care to remove the existing static symbols, otherwise there will be two sets of symbols and GDB seems to prefer the ones loaded first (i.e. the static ones). Use autoconf variables to generate a gdb_grub for a particular target, which conditionally run startup code depending on if the target uses static or dynamic loading. Reviewed-by: Daniel Kiper 2023-03-14 Glenn Washburn gdb: Move runtime module loading into runtime_load_module By moving this code into a function, it can be run re-utilized while gdb is running, not just when loading the script. This will also be useful in some following changes which will make a separate script path for targets which statically vs dynamically position GRUB code. Reviewed-by: Daniel Kiper 2023-03-07 Michael Chang osdep/devmapper/getroot: Fix build error on 32-bit host The gcc build has failed for 32-bit host (e.g. i386-emu and arm-emu) due to mismatch between format specifier and data type. ../grub-core/osdep/devmapper/getroot.c: In function 'grub_util_pull_devmapper': ../grub-core/osdep/devmapper/getroot.c:265:75: error: format '%lu' expects argument of type 'long unsigned int', but argument 2 has type 'int' [-Werror=format=] ../grub-core/osdep/devmapper/getroot.c:276:80: error: format '%lu' expects argument of type 'long unsigned int', but argument 2 has type 'int' [-Werror=format=] This patch fixes the problem by casting the type of calculated offset to grub_size_t and use platform PRIuGRUB_SIZE as format specifier. Reviewed-by: Daniel Kiper 2023-03-07 Stefan Berger commands/ieee1275/ibmvtpm: Add support for trusted boot using a vTPM 2.0 Add support for trusted boot using a vTPM 2.0 on the IBM IEEE1275 PowerPC platform. With this patch grub now measures text and binary data into the TPM's PCRs 8 and 9 in the same way as the x86_64 platform does. This patch requires Daniel Axtens's patches for claiming more memory. Note: The tpm_init() function cannot be called from GRUB_MOD_INIT() since it does not find the device nodes upon module initialization and therefore the call to tpm_init() must be deferred to grub_tpm_measure(). For vTPM support to work on PowerVM, system driver levels 1010.30 or 1020.00 are required. Note: Previous versions of firmware levels with the 2hash-ext-log API call have a bug that, once this API call is invoked, has the effect of disabling the vTPM driver under Linux causing an error message to be displayed in the Linux kernel log. Those users will have to update their machines to the firmware levels mentioned above. Cc: Eric Snowberg Reviewed-by: Daniel Kiper Tested-by: Nageswara R Sastry Reviewed-by: Robbie Harwood 2023-03-07 Daniel Axtens commands/memtools: Add memtool module with memory allocation stress-test When working on memory, it's nice to be able to test your work. Add a memtest module. When compiled with --enable-mm-debug, it exposes 3 commands: * lsmem - print all allocations and free space in all regions * lsfreemem - print free space in all regions * stress_big_allocs - stress test large allocations: - how much memory can we allocate in one chunk? - how many 1MB chunks can we allocate? - check that gap-filling works with a 1MB aligned 900kB alloc + a 100kB alloc. Reviewed-by: Daniel Kiper Tested-by: Nageswara R Sastry Reviewed-by: Robbie Harwood 2023-03-07 Diego Domingos ieee1275: Implement vec5 for cas negotiation As a legacy support, if the vector 5 is not implemented, Power Hypervisor will consider the max CPUs as 64 instead 256 currently supported during client-architecture-support negotiation. This patch implements the vector 5 and set the MAX CPUs to 256 while setting the others values to 0 (default). Acked-by: Daniel Axtens Tested-by: Nageswara R Sastry Reviewed-by: Robbie Harwood Reviewed-by: Daniel Kiper 2023-03-07 Daniel Axtens ieee1275: Support runtime memory claiming On powerpc-ieee1275, we are running out of memory trying to verify anything. This is because: - we have to load an entire file into memory to verify it. This is difficult to change with appended signatures. - We only have 32MB of heap. - Distro kernels are now often around 30MB. So we want to be able to claim more memory from OpenFirmware for our heap at runtime. There are some complications: - The grub mm code isn't the only thing that will make claims on memory from OpenFirmware: * PFW/SLOF will have claimed some for their own use. * The ieee1275 loader will try to find other bits of memory that we haven't claimed to place the kernel and initrd when we go to boot. * Once we load Linux, it will also try to claim memory. It claims memory without any reference to /memory/available, it just starts at min(top of RMO, 768MB) and works down. So we need to avoid this area. See arch/powerpc/kernel/prom_init.c as of v5.11. - The smallest amount of memory a ppc64 KVM guest can have is 256MB. It doesn't work with distro kernels but can work with custom kernels. We should maintain support for that. (ppc32 can boot with even less, and we shouldn't break that either.) - Even if a VM has more memory, the memory OpenFirmware makes available as Real Memory Area can be restricted. Even with our CAS work, an LPAR on a PowerVM box is likely to have only 512MB available to OpenFirmware even if it has many gigabytes of memory allocated. What should we do? We don't know in advance how big the kernel and initrd are going to be, which makes figuring out how much memory we can take a bit tricky. To figure out how much memory we should leave unused, I looked at: - an Ubuntu 20.04.1 ppc64le pseries KVM guest: vmlinux: ~30MB initrd: ~50MB - a RHEL8.2 ppc64le pseries KVM guest: vmlinux: ~30MB initrd: ~30MB So to give us a little wriggle room, I think we want to leave at least 128MB for the loader to put vmlinux and initrd in memory and leave Linux with space to satisfy its early allocations. Allow other space to be allocated at runtime. Tested-by: Stefan Berger Tested-by: Nageswara R Sastry Reviewed-by: Robbie Harwood Reviewed-by: Daniel Kiper 2023-03-07 Daniel Axtens ieee1275: Drop len -= 1 quirk in heap_init This was apparently "required by some firmware": commit dc9468500919 (2007-02-12 Hollis Blanchard ). It's not clear what firmware that was, and what platform from 14 years ago which exhibited the bug then is still both in use and buggy now. It doesn't cause issues on qemu (mac99 or pseries) or under PFW for Power8. I don't have access to old Mac hardware, but if anyone feels especially strongly we can put it under some feature flag. I really want to disable it under pseries because it will mess with region merging. Reviewed-by: Daniel Kiper Tested-by: Nageswara R Sastry Reviewed-by: Robbie Harwood 2023-03-07 Daniel Axtens ieee1275: Request memory with ibm, client-architecture-support On PowerVM, the first time we boot a Linux partition, we may only get 256MB of real memory area, even if the partition has more memory. This isn't enough to reliably verify a kernel. Fortunately, the Power Architecture Platform Reference (PAPR) defines a method we can call to ask for more memory: the broad and powerful ibm,client-architecture-support (CAS) method. CAS can do an enormous amount of things on a PAPR platform: as well as asking for memory, you can set the supported processor level, the interrupt controller, hash vs radix mmu, and so on. If: - we are running under what we think is PowerVM (compatible property of / begins with "IBM"), and - the full amount of RMA is less than 512MB (as determined by the reg property of /memory) then call CAS as follows: (refer to the Linux on Power Architecture Reference, LoPAR, which is public, at B.5.2.3): - Use the "any" PVR value and supply 2 option vectors. - Set option vector 1 (PowerPC Server Processor Architecture Level) to "ignore". - Set option vector 2 with default or Linux-like options, including a min-rma-size of 512MB. - Set option vector 3 to request Floating Point, VMX and Decimal Floating point, but don't abort the boot if we can't get them. - Set option vector 4 to request a minimum VP percentage to 1%, which is what Linux requests, and is below the default of 10%. Without this, some systems with very large or very small configurations fail to boot. This will cause a CAS reboot and the partition will restart with 512MB of RMA. Importantly, grub will notice the 512MB and not call CAS again. Notes about the choices of parameters: - A partition can be configured with only 256MB of memory, which would mean this request couldn't be satisfied, but PFW refuses to load with only 256MB of memory, so it's a bit moot. SLOF will run fine with 256MB, but we will never call CAS under qemu/SLOF because /compatible won't begin with "IBM".) - unspecified CAS vectors take on default values. Some of these values might restrict the ability of certain hardware configurations to boot. This is why we need to specify the VP percentage in vector 4, which is in turn why we need to specify vector 3. Finally, we should have enough memory to verify a kernel, and we will reach Linux. One of the first things Linux does while still running under OpenFirmware is to call CAS with a much fuller set of options (including asking for 512MB of memory). Linux includes a much more restrictive set of PVR values and processor support levels, and this CAS invocation will likely induce another reboot. On this reboot grub will again notice the higher RMA, and not call CAS. We will get to Linux again, Linux will call CAS again, but because the values are now set for Linux this will not induce another CAS reboot and we will finally boot all the way to userspace. On all subsequent boots, everything will be configured with 512MB of RMA, so there will be no further CAS reboots from grub. (phyp is super sticky with the RMA size - it persists even on cold boots. So if you've ever booted Linux in a partition, you'll probably never have grub call CAS. It'll only ever fire the first time a partition loads grub, or if you deliberately lower the amount of memory your partition has below 512MB.) Reviewed-by: Daniel Kiper Tested-by: Nageswara R Sastry Reviewed-by: Robbie Harwood 2023-02-28 Khem Raj RISC-V: Handle R_RISCV_CALL_PLT reloc GNU assembler starting 2.40 release always generates R_RISCV_CALL_PLT reloc for call in assembler [1], similarly LLVM does not make distinction between R_RISCV_CALL_PLT and R_RISCV_CALL [2]. Fixes "grub-mkimage: error: relocation 0x13 is not implemented yet.". [1] https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=70f35d72ef04cd23771875c1661c9975044a749c [2] https://reviews.llvm.org/D132530 Reviewed-by: Daniel Kiper 2023-02-28 John Paul Adrian Glaubitz osdep/hurd/getroot: Remove unused variables in grub_util_find_hurd_root_device() Found during a test build on Debian/hurd-i386 with --disable-werror enabled: In file included from grub-core/osdep/getroot.c:12: grub-core/osdep/hurd/getroot.c: In function ‘grub_util_find_hurd_root_device’: grub-core/osdep/hurd/getroot.c:126:13: error: unused variable ‘next’ [-Werror=unused-variable] 126 | char *next; | ^~~~ grub-core/osdep/hurd/getroot.c:125:14: error: unused variable ‘size’ [-Werror=unused-variable] 125 | size_t size; | ^~~~ Fixes: e981b0a24 (osdep/hurd/getroot: Use "part:" qualifier) Reviewed-by: Samuel Thibault Reviewed-by: Daniel Kiper 2023-02-28 Glenn Washburn gdb: If no modules have been loaded, do not try to load module symbols This prevents load_all_modules from failing when called before any modules have been loaded. Failures in GDB user-defined functions cause any function which called them to also fail. Reviewed-by: Daniel Kiper 2023-02-28 Glenn Washburn gdb: Prevent wrapping when writing to .segments.tmp GDB logging is redirected to write .segments.tmp, which means that GDB will wrap lines longer than what it thinks is the screen width (typically 80 characters). When wrapping does occur it causes gmodule.pl to misbehave. So disable line wrapping by using GDB's "with" command so that its guaranteed to return the width to the previous value upon command completion. Also disable command tracing when dumping the module sections because that output will go to .segments.tmp and thus cause gmodule.pl to misbehave. Reviewed-by: Daniel Kiper 2023-02-28 Glenn Washburn gdb: Fix redirection issue in dump_module_sections An error in any GDB command causes it to immediately abort with an error, this includes any command that calls that command. This leads to an issue in dump_module_sections where an error causes the command to exit without turning off file redirection. The user then ends up with a GDB command line where commands output nothing to the console. Instead do the work of dump_module_sections in the command dump_module_sections_helper and run the command using GDB's pipe command which does the redirection and undoes the redirection when it finishes regardless of any errors in the command. Also, remove .segments.tmp file prior to loading modules in case one was left from a previous run. Reviewed-by: Daniel Kiper 2023-02-28 Glenn Washburn efi: Allow expression as func argument to efi_call_* macros on all platforms On EFI platforms where EFI calls do not require a wrapper (notably i386-efi and arm64-efi), the func argument needs to be wrapped in parenthesis to allow valid syntax when func is an expression which evaluates to a function pointer. On EFI platforms that do need a wrapper, this was never an issue because func is passed to the C function wrapper as an argument and thus does not need parenthesis to be evaluated. Reviewed-by: Daniel Kiper 2023-02-28 Jeremy Szu loader/i386/linux: Correct wrong initrd address for debug The "addr" is used to request the memory with specific ranges but the real loadable address come from the relocator. Thus, print the final retrieved addresses, virtual and physical, for initrd. On the occasion migrate to PRIxGRUB_ADDR and PRIxGRUB_SIZE format specifiers. Reviewed-by: Daniel Kiper 2023-02-28 Glenn Washburn INSTALL: Document that the functional test requires the package xfonts-unifont Reviewed-by: Thomas Schmitt Reviewed-by: Daniel Kiper 2023-02-28 Glenn Washburn tests: Return hard error for functional test when unicode.pf2 does not exist The functional test requires unicode.pf2 to run successfully, so explicitly have the test return ERROR when its not found. Tested-by: Thomas Schmitt Reviewed-by: Daniel Kiper 2023-02-28 Glenn Washburn tests: grub_cmd_cryptomount should hard error when pre-requisites are not met Tests should be SKIP'd only when they do not apply to a particular target. Hard errors are for when the test should run but can not be setup properly. Reviewed-by: Daniel Kiper 2023-02-28 Glenn Washburn tests: Add pathological iso9660 filesystem tests These are not added to grub-fs-tester because they are not generated and none of the filesystem tests are run on these ISOs. The test is to run the command "ls /" on the ISO, and a failure is determined if the command times out, has non-zero return value or has any output. Tested-by: Thomas Schmitt Reviewed-by: Daniel Kiper 2023-02-14 Mukesh Kumar Chaurasiya osdep/linux/hostdisk: Modify sector by sysfs as disk sector The disk sector size provided by sysfs file system considers the sector size of 512 irrespective of disk sector size, thus causing the read by the GRUB to an incorrect offset from what was originally intended. Considering the 512 sector size of sysfs data the actual sector needs to be modified corresponding to disk sector size. Reviewed-by: Daniel Kiper 2023-02-14 Glenn Washburn tests/util/grub-fs-tester: Use shell variable instead of autoconf By using a shell variable that is set once by the expansion of an autoconf variable, the resulting script is more readable. Reviewed-by: Daniel Kiper 2023-02-14 Glenn Washburn tests/util/grub-fs-tester: Remove unused variable Reviewed-by: Daniel Kiper 2023-02-14 Alec Brown net/bootp: Fix unchecked return value In the function send_dhcp_packet(), added an error check for the return value of grub_netbuff_push(). Fixes: CID 404614 Reviewed-by: Daniel Kiper 2023-02-02 Zhang Boyang mm: Avoid complex heap growth math in hot path We do a lot of math about heap growth in hot path of grub_memalign(). However, the result is only used if out of memory is encountered, which is seldom. This patch moves these calculations away from hot path. These calculations are now only done if out of memory is encountered. This change can also help compiler to optimize integer overflow checks away. Reviewed-by: Daniel Kiper 2023-02-02 Zhang Boyang mm: Preallocate some space when adding new regions When grub_memalign() encounters out-of-memory, it will try grub_mm_add_region_fn() to request more memory from system firmware. However, it doesn't preallocate memory space for future allocation requests. In extreme cases, it requires one call to grub_mm_add_region_fn() for each memory allocation request. This can be very slow. This patch introduces GRUB_MM_HEAP_GROW_EXTRA, the minimal heap growth granularity. The new region size is now set to the bigger one of its original value and GRUB_MM_HEAP_GROW_EXTRA. Thus, it will result in some memory space preallocated if current allocations request is small. The value of GRUB_MM_HEAP_GROW_EXTRA is set to 1MB. If this value is smaller, the cost of small memory allocations will be higher. If this value is larger, more memory will be wasted and it might cause out-of-memory on machines with small amount of RAM. Reviewed-by: Daniel Kiper 2023-02-02 Zhang Boyang mm: Adjust new region size to take management overhead into account When grub_memalign() encounters out-of-memory, it will try grub_mm_add_region_fn() to request more memory from system firmware. However, the size passed to it doesn't take region management overhead into account. Adding a memory area of "size" bytes may result in a heap region of less than "size" bytes really available. Thus, the new region may not be adequate for current allocation request, confusing out-of-memory handling code. This patch introduces GRUB_MM_MGMT_OVERHEAD to address the region management overhead (e.g. metadata, padding). The value of this new constant must be large enough to make sure grub_memalign(align, size) always succeeds after a successful call to grub_mm_init_region(addr, size + align + GRUB_MM_MGMT_OVERHEAD), for any given addr and size (assuming no integer overflow). The size passed to grub_mm_add_region_fn() is now correctly adjusted, thus if grub_mm_add_region_fn() succeeded, current allocation request can always succeed. Reviewed-by: Daniel Kiper 2023-02-02 Glenn Washburn tests/util/grub-shell: Add $GRUB_QEMU_OPTS to run.sh to easily see unofficial QEMU arguments When re-running a failed test, even the non-standard grub-shell QEMU arguments should be preserved in the run.sh to more precisely replay the failed test run. Reviewed-by: Daniel Kiper 2023-02-02 Glenn Washburn tests/util/grub-shell: Create run.sh in working directory for easily running test again Now it becomes trivial to re-run a test from the output in its working directory. This also makes it easy to send a reproducible failing test to the mailing list. This has allowed a refactor so that the duplicated code to call QEMU has be condensed (e.g. the use of timeout and file descriptor redirection). The run.sh script will pass any arguments given to QEMU. This allows QEMU to be easily started in a state ready for GDB to be attached. Reviewed-by: Daniel Kiper 2023-02-02 Glenn Washburn tests: Allow turning on shell tracing from environment variables This allows turning on shell tracing for grub-shell and grub-fs-tester when its not practical or not possible to use command line arguments (e.g. from "make check"). Turn on tracing when the envvar is an integer greater than 1, since these can generate a lot of output. Since this change uses the environment variables to set the default value for debug in grub-shell, this allows enabling grub-shell's debug mode which will preserve various generated output files that are helpful for debugging tests. Reviewed-by: Daniel Kiper 2023-02-02 Glenn Washburn misc: Move *printf function declarations to same location Reviewed-by: Daniel Kiper 2023-02-02 Thomas Schmitt fs/iso9660: Prevent skipping CE or ST at start of continuation area If processing of a SUSP CE entry leads to a continuation area which begins by entry CE or ST, then these entries were skipped without interpretation. In case of CE this would lead to premature end of processing the SUSP entries of the file. In case of ST this could cause following non-SUSP bytes to be interpreted as SUSP entries. Tested-by: Lidong Chen Reviewed-by: Thomas Schmitt Reviewed-by: Daniel Kiper 2023-02-02 Lidong Chen fs/iso9660: Incorrect check for entry boundary An SL entry consists of the entry info and the component area. The entry info should take up 5 bytes instead of sizeof(*entry). The area after the first 5 bytes is the component area. It is incorrect to use the sizeof(*entry) to check the entry boundary. Reviewed-by: Thomas Schmitt Reviewed-by: Daniel Kiper 2023-02-02 Lidong Chen fs/iso9660: Avoid reading past the entry boundary Added a check for the SP entry data boundary before reading it. Reviewed-by: Thomas Schmitt Reviewed-by: Daniel Kiper 2023-02-02 Lidong Chen fs/iso9660: Prevent read past the end of system use area In the code, the for loop advanced the entry pointer to the next entry before checking if the next entry is within the system use area boundary. Another issue in the code was that there is no check for the size of system use area. For a corrupted system, the size of system use area can be less than the size of minimum SUSP entry size (4 bytes). These can cause buffer overrun. The fixes added the checks to ensure the read is valid and within the boundary. Reviewed-by: Thomas Schmitt Reviewed-by: Daniel Kiper 2023-02-02 Lidong Chen fs/iso9660: Add check to prevent infinite loop There is no check for the end of block when reading directory extents. It resulted in read_node() always read from the same offset in the while loop, thus caused infinite loop. The fix added a check for the end of the block and ensure the read is within directory boundary. Reviewed-by: Thomas Schmitt Reviewed-by: Daniel Kiper 2023-02-01 Pierre-Louis Bonicoli grub-fs-tester: Add LUKS1 and LUKS2 support The logical sector size used by LUKS1 is 512 bytes and LUKS2 uses 512 to 4069 bytes. The default password used is "pass", but can be overridden by setting the PASS environment variable. The device mapper name is set to the name of the temp directory so that its easy to correlate device mapper name with a particular test run. Also since this name is unique per test run, multiple simultaneous test runs are allowed. Note that cryptsetup is passing the --disable-locks parameter to allow cryptsetup run successfully when /run/lock/cryptsetup is not accessible. Since the device mapper name is unique per test run, there is no need to worry about locking the device to serialize access. Tested-by: Glenn Washburn Reviewed-by: Patrick Steinhardt Reviewed-by: Daniel Kiper 2023-02-01 Josselin Poiret osdep/devmapper/getroot: Set up cheated LUKS2 cryptodisk mount from DM parameters This lets a LUKS2 cryptodisk have its cipher and hash filled out, otherwise they wouldn't be initialized if cheat mounted. Tested-by: Glenn Washburn Reviewed-by: Patrick Steinhardt Reviewed-by: Daniel Kiper 2023-02-01 Josselin Poiret osdep/devmapper/getroot: Have devmapper recognize LUKS2 Changes UUID comparisons so that LUKS1 and LUKS2 are both recognized as being LUKS cryptodisks. Tested-by: Glenn Washburn Reviewed-by: Patrick Steinhardt Reviewed-by: Daniel Kiper 2023-02-01 Fabian Vogt disk/cryptodisk: When cheatmounting, use the sector info of the cheat device When using grub-probe with cryptodisk, the mapped block device from the host is used directly instead of decrypting the source device in GRUB code. In that case, the sector size and count of the host device needs to be used. This is especially important when using LUKS2, which does not assign total_sectors and log_sector_size when scanning, but only later when the segments in the JSON area are evaluated. With an unset log_sector_size, grub_device_open() complains. This fixes grub-probe failing with "error: sector sizes of 1 bytes aren't supported yet.". Reviewed-by: Patrick Steinhardt Tested-by: Glenn Washburn Reviewed-by: Glenn Washburn Reviewed-by: Patrick Steinhardt Reviewed-by: Daniel Kiper 2023-01-19 Daniel Axtens fs/f2fs: Fix off-by-one error in nat journal entries check Oops. You're allowed to have up to n = NAT_JOURNAL_ENTRIES entries _inclusive_, because the loop below uses i < n, not i <= n. D'oh. Fixes: 4bd9877f6216 (fs/f2fs: Do not read past the end of nat journal entries) Reported-by: программист нект Tested-by: программист нект Reviewed-by: Daniel Kiper 2023-01-19 Nicholas Vinson gentpl.py: Remove .interp section from .img files When building .img files, a .interp section from the .image files will sometimes be copied into the .img file. This additional section pushes the .img file beyond the 512-byte limit and causes grub-install to fail to run for i386-pc platforms. Reviewed-by: Daniel Kiper 2023-01-19 Glenn Washburn tests: Add cryptomount functional test The grub_cmd_cryptomount make check test performs some functional testing of cryptomount and by extension the underlying cryptodisk infrastructure. A utility test script named grub-shell-luks-tester is created to handle the complexities of the testing, making it simpler to add new test cases in grub_cmd_cryptomount. Reviewed-by: Daniel Kiper 2023-01-19 Glenn Washburn tests/util/grub-shell: Add halt_cmd variable to testcase namespace This allows test case scripts to use the appropriate halt command for the built architecture to end execution early. Otherwise, test case scripts have no way to know the appropriate mechanism for halting the test case early. Reviewed-by: Daniel Kiper 2023-01-19 Glenn Washburn tests/util/grub-shell: Trim line should always be matched from the beginning of the line When turning on shell tracing the trim line will be output before we actually want to start the trim. However, in this case the trim line never starts from the beginning of the line. So start trimming from the correct line by matching from the beginning of the line. Reviewed-by: Daniel Kiper 2023-01-19 Glenn Washburn tests/util/grub-shell: Allow specifying non-default trim line contents This will be useful for tests that have unwanted output from setup. This is not documented because its only intended to be internal at the moment. Also, --no-trim is allowed to explicitly turn off trim. Reviewed-by: Daniel Kiper 2023-01-19 Glenn Washburn tests/util/grub-shell: Only cleanup working directory file if QEMU does not fail or timeout This keeps the generated files to aid in diagnosing the source of the failure. Reviewed-by: Daniel Kiper 2023-01-19 Glenn Washburn tests/util/grub-shell: Set exit status to QEMU exit status This allows us to test if unexpected output in test scripts is because of a bug in GRUB, because there was an error in QEMU, or QEMU was killed due to a timeout. Reviewed-by: Daniel Kiper 2023-01-19 Glenn Washburn io/gzio: Remove confusing, out-dated comment The "transparent" parameter to grub_gzio_open() was removed in 2010, fc2ef1172c (* grub-core/io/gzio.c (grub_gzio_open): Removed "transparent" parameter.) Reviewed-by: Daniel Kiper 2023-01-19 Glenn Washburn efi: Fix spacing Reviewed-by: Daniel Kiper misc: Fix spacing Reviewed-by: Daniel Kiper misc: Spelling fixes Reviewed-by: Daniel Kiper gdb: Unregister gdbstub_break command when unloading module Reviewed-by: Daniel Kiper 2023-01-19 Glenn Washburn tests: Fix help test to reflect updated help output Commit f5759a878 (normal/help: Add paging instructions to normal and help prompts) changed the output of the help command, which broke the help test. This change allows the test to pass. On the occasion do s/outpu/output/. Reviewed-by: Daniel Kiper 2023-01-19 Benjamin Herrenschmidt term/serial: Improve detection of duplicate serial ports We currently rely on some pretty fragile comparison by name to identify whether a serial port being configured is identical Reviewed-by: Daniel Kiper 2023-01-19 Benjamin Herrenschmidt term/serial: Avoid double lookup of serial ports The various functions to add a port used to return port->name, and the callers would immediately iterate all registered ports to "find" the one just created by comparing that return value with ... port->name. This is a waste of cycles and code. Instead, have those functions return "port" directly. Reviewed-by: Daniel Kiper 2023-01-19 Benjamin Herrenschmidt term/serial: Replace usage of memcmp() with strncmp() We are comparing strings after all. Reviewed-by: Daniel Kiper 2023-01-19 Benjamin Herrenschmidt term/serial: Add ability to specify MMIO ports via "serial" command This adds the ability to explicitly add an MMIO based serial port via the "serial" command. The syntax is: serial --port=mmio,{.b,.w,.l,.q} Reviewed-by: Daniel Kiper 2023-01-19 Benjamin Herrenschmidt term/ns8250: Support more MMIO access sizes It is common for PCI based UARTs to use larger than one byte access sizes. This adds support for this and uses the information present in SPCR accordingly. Reviewed-by: Daniel Kiper 2023-01-19 Benjamin Herrenschmidt term/ns8250: Use ACPI SPCR table when available to configure serial "serial auto" is now equivalent to just "serial" and will use the SPCR to discover the port if present, otherwise defaults to "com0" as before. This allows to support MMIO ports specified by ACPI which is needed on AWS EC2 "metal" instances, and will enable GRUB to pickup the port configuration specified by ACPI in other cases. Reviewed-by: Daniel Kiper 2023-01-19 Benjamin Herrenschmidt term/ns8250: Add configuration parameter when adding ports This will allow ports to be added with a pre-set configuration. Reviewed-by: Daniel Kiper 2023-01-19 Benjamin Herrenschmidt term/ns8250: Move base clock definition to a header And while at it, unify it as clock frequency in Hz, to match the value in grub_serial_config struct and do the division by 16 in one common place. This will simplify adding SPCR support. Reviewed-by: Daniel Kiper 2023-01-19 Benjamin Herrenschmidt term/ns8250: Add base support for MMIO UARTs This adds the ability for the driver to access UARTs via MMIO instead of PIO selectively at runtime, and exposes a new function to add an MMIO port. In an ideal world, MMIO accessors would be generic and have architecture specific memory barriers. However, existing drivers don't have them and most of those "bare metal" drivers tend to be for x86 which doesn't need them. If necessary, those can be added later. Reviewed-by: Daniel Kiper 2023-01-18 Benjamin Herrenschmidt acpi: Add SPCR and generic address definitions This adds the definition of the two ACPI tables according to the spec. Reviewed-by: Daniel Kiper 2023-01-18 Benjamin Herrenschmidt kern/acpi: Export a generic grub_acpi_find_table() And convert grub_acpi_find_fadt() to use it. Reviewed-by: Daniel Kiper 2023-01-10 Maxim Fomin kern/fs: Fix possible integer overflow in i386-pc mode with large partitions The i386-pc mode supports MBR partition scheme where maximum partition size is 2 TiB. In case of large partitions left shift expression with unsigned long int "length" object may cause integer overflow making calculated partition size less than true value. This issue is fixed by increasing the size of "length" integer type. Reviewed-by: Daniel Kiper 2023-01-10 Glenn Washburn commands/cmp: Only return success when both files have the same contents This allows the cmp command to be used in GRUB scripts to conditionally run commands based on whether two files are the same. The command is now quiet by default and the -v switch can be given to enable verbose mode, the previous behavior. Update documentation accordingly. Suggested-by: Li Gen Reviewed-by: Daniel Kiper 2023-01-10 Glenn Washburn docs: Remove text about cryptodisk UUIDs no being able to use dashes This was fixed here: 3cf2e848bc (disk/cryptodisk: Allows UUIDs to be compared in a dash-insensitive manner). Reviewed-by: Daniel Kiper 2023-01-10 Glenn Washburn tests/util/grub-shell: Add GRUB output logfile with grub-shell --debug This allows seeing full QEMU output of grub-shell, which can be invaluable when debugging failing tests. Reviewed-by: Daniel Kiper 2023-01-10 Marek Marczykowski-Górecki templates/linux_xen: Fix detecting XSM policy The xenpolicy variable was left set from previous function call. This resulted in all-but-first menu entries including XSM policy, even if it did not exist. Fix this by initializing the xenpolicy variable. Reviewed-by: Daniel Kiper 2023-01-10 Zhang Boyang font: Reject fonts with negative max_char_width or max_char_height If max_char_width or max_char_height are negative wrong values can be propagated by grub_font_get_max_char_width() or grub_font_get_max_char_height(). Prevent this from happening. Reviewed-by: Daniel Kiper 2023-01-10 Zhang Boyang font: Assign null_font to unknown_glyph Like glyphs in ascii_font_glyph[], assign null_font to unknown_glyph->font in order to prevent grub_font_get_*() from dereferencing NULL pointer. Reviewed-by: Daniel Kiper 2023-01-10 Zhang Boyang font: Check return value of grub_malloc() in ascii_glyph_lookup() There is a problem in ascii_glyph_lookup(). It doesn't check the return value of grub_malloc(). If memory can't be allocated, then NULL pointer will be written to. This patch fixes the problem by fallbacking to unknown_glyph when grub_malloc() returns NULL. Reviewed-by: Daniel Kiper 2023-01-10 Maxim Fomin disk/plainmount: Support plain encryption mode This patch adds support for plain encryption mode, plain dm-crypt, via new module/command named "plainmount". Reviewed-by: Daniel Kiper Reviewed-by: Glenn Washburn 2023-01-10 Pete Batard util/grub-mkrescue: Search by file UUID rather than partition UUID for EFI boot The final piece needed to add UEFI file system transposition support is to ensure the boot media can be located regardless of how the boot partition was instantiated. Especially, we do not want to be reliant on brittle partition UUIDs, as these only work if a boot media is duplicated at the block level and not at the file system level. To accomplish this for EFI boot, we now create a UUID file in a .disk/ directory, that can then be searched for. Note: The switch from make_image_fwdisk_abs() to make_image_abs() is needed in order to use the search functionality. Reviewed-by: Daniel Kiper 2023-01-10 Pete Batard util/grub-mkrescue: Preserve a copy of the EFI bootloaders on the ISO 9660 file system To enable file system transposition support for UEFI, we also must ensure that there exists a copy of the EFI bootloaders, that are currently embedded in the efi.img for xorriso, at their expected UEFI location on the ISO 9660 file system. This is accomplished by removing the use of a temporary directory to create the efi/ content, to instead place it at the root of the ISO 9660 content. Reviewed-by: Daniel Kiper 2023-01-10 Pete Batard util/grub-mkrescue: Add support for FAT and NTFS on EFI boot In order to add file system transposition support for UEFI, i.e. the ability to copy the content of an grub-mkrescue ISO 9660 image onto user-formatted media, and have that boot on UEFI systems, the first thing we need to do is add support for the file systems that are natively handled by UEFI. This mandatorily includes FAT, but we also include NTFS as the latter is also commonly supported on modern x64 platforms. Reviewed-by: Daniel Kiper 2022-12-07 t.feng util/bash-completion: Disable SC2120 shellcheck warning SC2120 (warning): function references arguments, but none are ever passed. In grub-completion.bash.in line 63: __grub_get_options_from_help () { ^-- SC2120 (warning) local prog if [ $# -ge 1 ]; then prog="$1" The arg of __grub_get_options_from_help() is optional. So, the current code meets the exception and does not need to be modified. Ignoring the warning then. More: https://github.com/koalaman/shellcheck/wiki/SC2120 Reviewed-by: Daniel Kiper 2022-12-07 t.feng util/bash-completion: Fix SC2155 shellcheck warning SC2155 (warning): Declare and assign separately to avoid masking return values. The exit status of the command is overridden by the exit status of the creation of the local variable. In grub-completion.bash.in line 115: local config_file=$(__grub_dir)/grub.cfg ^---------^ SC2155 (warning) In grub-completion.bash.in line 126: local grub_dir=$(__grub_dir) ^------^ SC2155 (warning) More: https://github.com/koalaman/shellcheck/wiki/SC2155 Reviewed-by: Daniel Kiper 2022-12-07 t.feng util/bash-completion: Fix SC2207 shellcheck warning SC2207 (warning): Prefer mapfile or read -a to split command output (or quote to avoid splitting). In grub-completion.bash.in line 56: COMPREPLY=($(compgen -P "${2-}" -W "${1-}" -S "${4-}" -- "$cur")) ^-- SC2207 (warning) In grub-completion.bash.in line 119: COMPREPLY=( $(compgen \ ^-- SC2207 (warning) In grub-completion.bash.in line 128: COMPREPLY=( $( compgen -f -X '!*/*.mod' -- "${grub_dir}/$cur" | { ^-- SC2207 (warning) COMPREPLY=($(command)) are doing unquoted command expansion in an array. This will invoke the shell's sloppy word splitting and glob expansion. If we want to split the output into lines or words, use read -r and loops will be better. This prevents the shell from doing unwanted splitting and glob expansion, and therefore avoiding problems with output containing spaces or special characters. More: https://github.com/koalaman/shellcheck/wiki/SC2207 Reviewed-by: Daniel Kiper 2022-12-07 t.feng util/bash-completion: Fix SC2070 shellcheck error SC2070 (error): -n doesn't work with unquoted arguments. Quote or use [[ ]]. In grub-completion.bash.in line 130: [ -n $tmp ] && { ^--^ SC2070 (error) More: https://github.com/koalaman/shellcheck/wiki/SC2070 Reviewed-by: Daniel Kiper 2022-12-07 Steve McIntyre kern/file: Fix error handling in grub_file_open() grub_file_open() calls grub_file_get_device_name(), but doesn't check the return. Instead, it checks if grub_errno is set. However, nothing initialises grub_errno here when grub_file_open() starts. This means that trying to open one file that doesn't exist and then trying to open another file that does will (incorrectly) also fail to open that second file. Let's fix that. Reviewed-by: Daniel Kiper 2022-12-07 Jeremy Szu loader/i386/linux: Fix initrd maximum address overflow The current i386 initrd is limited under 1 GiB memory and it works with most compressed initrds (also initrd_addr_max case reported by kernel). addr = (addr_max - aligned_size) & ~0xFFF; Above line is used to calculate the reasonable address to store the initrd. However, if initrd size is greater than 1 GiB or initrd_addr_max, then it will get overflow, especially on x86_64 arch. Therefore, add a check point to prevent it overflows as well as having a debug log for complex story of initrd addresses. Reviewed-by: Daniel Kiper 2022-12-07 Dimitri John Ledkov templates: Enable fwsetup on EFI platforms only Only perform call to fwsetup if one is on EFI platform. On all other platforms fwsetup command does not exists, and thus returns 0 and a useless uefi-firmware menu entry gets generated. Reviewed-by: Daniel Kiper 2022-12-07 t.feng fs/xfs: Fix memory leaks in XFS module Reviewed-by: Daniel Kiper 2022-12-07 t.feng fs/squash4: Fix memory leaks in grub_squash_iterate_dir() Fixes: 20dd511c8 (Handle "." and ".." on squashfs) Reviewed-by: Daniel Kiper 2022-12-07 t.feng fs/iso9660: Fix memory leaks in grub_iso9660_susp_iterate() Fixes: 99373ce47 (* grub-core/fs/iso9660.c: Remove nested functions) Reviewed-by: Thomas Schmitt Reviewed-by: Daniel Kiper 2022-12-07 t.feng fs/hfsplus: Fix memory leak in grub_hfsplus_btree_search() Fixes: 58ea11d5b (fs/hfsplus: Don't fetch a key beyond the end of the node) Reviewed-by: Daniel Kiper 2022-12-07 t.feng fs/bfs: Fix memory leak in read_bfs_file() The l1_entries and l2_entries were not freed at the end of file read. Fixes: 5825b3794 (BFS implementation based on the specification) Reviewed-by: Daniel Kiper 2022-12-07 t.feng fs/ntfs: Fix memory leaks in grub_ntfs_read_symlink() Fixes: 5773fb641 (Support NTFS reparse points) Reviewed-by: Daniel Kiper 2022-12-07 t.feng fs/minix: Fix memory leaks in grub_minix_lookup_symlink() Fixes: a07e6ad01 (* grub-core/fs/minix.c: Remove variable length arrays) Reviewed-by: Daniel Kiper 2022-12-07 t.feng fs/btrfs: Fix memory leak in find_path() Fixes: 82591fa6e (Make / in btrfs refer to real root) Reviewed-by: Daniel Kiper 2022-12-07 t.feng fs/affs: Fix memory leaks in grub_affs_create_node() The hashtable is not freed if GRUB_AFFS_FILETYPE_HARDLINK and grub_disk_read() failed. If grub_affs_create_node() returns non-zero the hashtable should be freed too. By the way, the hashtable argument is unused in grub_affs_create_node(). So, we can remove the argument and free it in grub_affs_iterate_dir(). It allocates the memory and it should be responsible for releasing it. This is why commit ebf32bc4e9 (fs/affs: Fix resource leaks) missed this memory leak. Fixes: ebf32bc4e9 (fs/affs: Fix resource leaks) Reviewed-by: Daniel Kiper 2022-12-07 Ryan Cohen normal/cmdline: Fix two related integer underflows An unchecked decrement operation in cl_print() would cause a few integers to underflow. Where an output terminal's state is stored in cl_term, the values cl_term->ystart and cl_term->pos.y both underflow. This can be replicated with the following steps: 1. Get to the GRUB command line 2. Hold down the "d" key (or any key that enters a visible character) until it fills the entire row 3. Press "HOME" and then press "CTRL-k". This will clear every character entered in step 2 4. Continuously press "CTRL-y" until the terminal scrolls the original prompt ("grub> ") passed the terminal's top row. Now, no prompt should be visible. This step causes cl_term->ystart to underflow 5. Press "HOME" and then "d" (or any visible character). This can have different visual effects for different systems, but it will always cause cl_term->pos.y to underflow On BIOS systems, these underflows cause the output terminal to completely stop displaying anything. Characters can still be entered and commands can be run, but nothing will display on the terminal. From here, you can only get the display working by running a command to switch the current output terminal to a different type: terminal_output On UEFI systems, these replication steps do not break the output terminal. Until you press "ENTER", the cursor stops responding to input, but you can press "ENTER" after step 5 and the command line will work properly again. This patch is mostly important for BIOS systems where the output terminal is rendered unusable after the underflows occur. This patch adds two checks, one for each variable. It ensures that cl_term->ystart does not decrement passed 0. It also ensures that cl_term->pos.y does not get set passed the terminal's bottom row. When the previously listed replication steps are followed with this patch, the terminal's cursor will be set to the top row and the command line is still usable, even on BIOS systems. Reviewed-by: Daniel Kiper 2022-12-07 Ryan Cohen term/i386/pc/vga_text: Prevent out-of-bounds writes to VGA text buffer Coordinates passed to screen_write_char() did not have any checks to ensure they are not out-of-bounds. This adds an if statement to prevent out-of-bounds writes to the VGA text buffer. Reviewed-by: Daniel Kiper 2022-12-07 Gary Lin loader/linux: Ensure the newc pathname is NULL-terminated Per "man 5 cpio", the namesize in the cpio header includes the trailing NUL byte of the pathname and the pathname is followed by NUL bytes, but the current implementation ignores the trailing NUL byte when making the newc header. Although make_header() tries to pad the pathname string, the padding won't happen when strlen(name) + sizeof(struct newc_head) is a multiple of 4, and the non-NULL-terminated pathname may lead to unexpected results. Assume that a file is created with 'echo -n aaaa > /boot/test12' and loaded by grub2: linux /boot/vmlinuz initrd newc:test12:/boot/test12 /boot/initrd The initrd command eventually invoked grub_initrd_load() and sent 't''e''s''t''1''2' to make_header() to generate the header: 00000070 30 37 30 37 30 31 33 30 31 43 41 30 44 45 30 30 |070701301CA0DE00| 00000080 30 30 38 31 41 34 30 30 30 30 30 33 45 38 30 30 |0081A4000003E800| 00000090 30 30 30 30 36 34 30 30 30 30 30 30 30 31 36 33 |0000640000000163| 000000a0 37 36 45 34 35 32 30 30 30 30 30 30 30 34 30 30 |76E4520000000400| 000000b0 30 30 30 30 30 38 30 30 30 30 30 30 31 33 30 30 |0000080000001300| 000000c0 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 |0000000000000000| 000000d0 30 30 30 30 30 36 30 30 30 30 30 30 30 30 74 65 |00000600000000te| ^namesize 000000e0 73 74 31 32 61 61 61 61 30 37 30 37 30 31 30 30 |st12aaaa07070100| ^^ end of the pathname Since strlen("test12") + sizeof(struct newc_head) is 116 = 29 * 4, make_header() didn't pad the pathname, and the file content followed "test12" immediately. This violates the cpio format and may trigger such error during linux boot: Initramfs unpacking failed: ZSTD-compressed data is trunc To avoid the potential problems, this commit counts the trailing NUL byte in when calling make_header() and adjusts the initrd size accordingly. Now the header becomes 00000070 30 37 30 37 30 31 33 30 31 43 41 30 44 45 30 30 |070701301CA0DE00| 00000080 30 30 38 31 41 34 30 30 30 30 30 33 45 38 30 30 |0081A4000003E800| 00000090 30 30 30 30 36 34 30 30 30 30 30 30 30 31 36 33 |0000640000000163| 000000a0 37 36 45 34 35 32 30 30 30 30 30 30 30 34 30 30 |76E4520000000400| 000000b0 30 30 30 30 30 38 30 30 30 30 30 30 31 33 30 30 |0000080000001300| 000000c0 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 |0000000000000000| 000000d0 30 30 30 30 30 37 30 30 30 30 30 30 30 30 74 65 |00000700000000te| ^namesize 000000e0 73 74 31 32 00 00 00 00 61 61 61 61 30 37 30 37 |st12....aaaa0707| ^^ end of the pathname Besides the trailing NUL byte, make_header() pads 3 more NUL bytes, and the user can safely read the pathname without a further check. To conform to the cpio format, the headers for "TRAILER!!!" are also adjusted to include the trailing NUL byte, not ignore it. Reviewed-by: Daniel Kiper 2022-12-07 Jagannathan Raman fs/udf: Validate length of AED in grub_udf_read_block() Validate the length of Allocation Extent Descriptor in grub_udf_read_block(), based on the details in UDF spec. v2.01 section 2.3.11. Fixes: CID 314037 Reviewed-by: Daniel Kiper 2022-12-07 Ismael Luceno util/grub-install: Ensure a functional /dev/nvram This enables an early failure; for i386-ieee1275 and powerpc-ieee1275 on Linux, without /dev/nvram the system may be left in an unbootable state. Reviewed-by: Daniel Kiper 2022-12-07 Ismael Luceno templates: Set defaults using var substitution Reviewed-by: Daniel Kiper 2022-12-07 Glenn Washburn tests: Put all generated files into working dir and use better file names When running tests there are many invocations of grub-shell, and because the output files are all random names in the same tmp directory, it becomes more work to figure out which files went with which grub-shell invocations. So all generated files from one invocation of grub-shell are put into a randomly named directory, so as not to collide with other grub-shell invocations. And now that the generated files can be put in a location where they will not get stepped on, and they can be named sensible names. Reviewed-by: Daniel Kiper 2022-11-14 Zhang Boyang normal/charset: Fix an integer overflow in grub_unicode_aglomerate_comb() The out->ncomb is a bit-field of 8 bits. So, the max possible value is 255. However, code in grub_unicode_aglomerate_comb() doesn't check for an overflow when incrementing out->ncomb. If out->ncomb is already 255, after incrementing it will get 0 instead of 256, and cause illegal memory access in subsequent processing. This patch introduces GRUB_UNICODE_NCOMB_MAX to represent the max acceptable value of ncomb. The code now checks for this limit and ignores additional combining characters when limit is reached. Reported-by: Daniel Axtens Reviewed-by: Daniel Kiper 2022-11-14 Zhang Boyang font: Assign null_font to glyphs in ascii_font_glyph[] The calculations in blit_comb() need information from glyph's font, e.g. grub_font_get_xheight(main_glyph->font). However, main_glyph->font is NULL if main_glyph comes from ascii_font_glyph[]. Therefore grub_font_get_*() crashes because of NULL pointer. There is already a solution, the null_font. So, assign it to those glyphs in ascii_font_glyph[]. Reported-by: Daniel Axtens Reviewed-by: Daniel Kiper 2022-11-14 Zhang Boyang font: Harden grub_font_blit_glyph() and grub_font_blit_glyph_mirror() As a mitigation and hardening measure add sanity checks to grub_font_blit_glyph() and grub_font_blit_glyph_mirror(). This patch makes these two functions do nothing if target blitting area isn't fully contained in target bitmap. Therefore, if complex calculations in caller overflows and malicious coordinates are given, we are still safe because any coordinates which result in out-of-bound-write are rejected. However, this patch only checks for invalid coordinates, and doesn't provide any protection against invalid source glyph or destination glyph, e.g. mismatch between glyph size and buffer size. This hardening measure is designed to mitigate possible overflows in blit_comb(). If overflow occurs, it may return invalid bounding box during dry run and call grub_font_blit_glyph() with malicious coordinates during actual blitting. However, we are still safe because the scratch glyph itself is valid, although its size makes no sense, and any invalid coordinates are rejected. It would be better to call grub_fatal() if illegal parameter is detected. However, doing this may end up in a dangerous recursion because grub_fatal() would print messages to the screen and we are in the progress of drawing characters on the screen. Reported-by: Daniel Axtens Reviewed-by: Daniel Kiper 2022-11-14 Zhang Boyang font: Fix an integer underflow in blit_comb() The expression (ctx.bounds.height - combining_glyphs[i]->height) / 2 may evaluate to a very big invalid value even if both ctx.bounds.height and combining_glyphs[i]->height are small integers. For example, if ctx.bounds.height is 10 and combining_glyphs[i]->height is 12, this expression evaluates to 2147483647 (expected -1). This is because coordinates are allowed to be negative but ctx.bounds.height is an unsigned int. So, the subtraction operates on unsigned ints and underflows to a very big value. The division makes things even worse. The quotient is still an invalid value even if converted back to int. This patch fixes the problem by casting ctx.bounds.height to int. As a result the subtraction will operate on int and grub_uint16_t which will be promoted to an int. So, the underflow will no longer happen. Other uses of ctx.bounds.height (and ctx.bounds.width) are also casted to int, to ensure coordinates are always calculated on signed integers. Fixes: CVE-2022-3775 Reported-by: Daniel Axtens Reviewed-by: Daniel Kiper 2022-11-14 Zhang Boyang fbutil: Fix integer overflow Expressions like u64 = u32 * u32 are unsafe because their products are truncated to u32 even if left hand side is u64. This patch fixes all problems like that one in fbutil. To get right result not only left hand side have to be u64 but it's also necessary to cast at least one of the operands of all leaf operators of right hand side to u64, e.g. u64 = u32 * u32 + u32 * u32 should be u64 = (u64)u32 * u32 + (u64)u32 * u32. For 1-bit bitmaps grub_uint64_t have to be used. It's safe because any combination of values in (grub_uint64_t)u32 * u32 + u32 expression will not overflow grub_uint64_t. Other expressions like ptr + u32 * u32 + u32 * u32 are also vulnerable. They should be ptr + (grub_addr_t)u32 * u32 + (grub_addr_t)u32 * u32. This patch also adds a comment to grub_video_fb_get_video_ptr() which says it's arguments must be valid and no sanity check is performed (like its siblings in grub-core/video/fb/fbutil.c). Reviewed-by: Daniel Kiper 2022-11-14 Zhang Boyang kern/efi/sb: Enforce verification of font files As a mitigation and hardening measure enforce verification of font files. Then only trusted font files can be load. This will reduce the attack surface at cost of losing the ability of end-users to customize fonts if e.g. UEFI Secure Boot is enabled. Vendors can always customize fonts because they have ability to pack fonts into their GRUB bundles. This goal is achieved by: * Removing GRUB_FILE_TYPE_FONT from shim lock verifier's skip-verification list. * Adding GRUB_FILE_TYPE_FONT to lockdown verifier's defer-auth list, so font files must be verified by a verifier before they can be loaded. Suggested-by: Daniel Kiper Reviewed-by: Daniel Kiper 2022-11-14 Zhang Boyang font: Fix integer underflow in binary search of char index If search target is less than all entries in font->index then "hi" variable is set to -1, which translates to SIZE_MAX and leads to errors. This patch fixes the problem by replacing the entire binary search code with the libstdc++'s std::lower_bound() implementation. Reviewed-by: Daniel Kiper 2022-11-14 Zhang Boyang font: Fix integer overflow in BMP index The BMP index (font->bmp_idx) is designed as a reverse lookup table of char entries (font->char_index), in order to speed up lookups for BMP chars (i.e. code < 0x10000). The values in BMP index are the subscripts of the corresponding char entries, stored in grub_uint16_t, while 0xffff means not found. This patch fixes the problem of large subscript truncated to grub_uint16_t, leading BMP index to return wrong char entry or report false miss. The code now checks for bounds and uses BMP index as a hint, and fallbacks to binary-search if necessary. On the occasion add a comment about BMP index is initialized to 0xffff. Reviewed-by: Daniel Kiper 2022-11-14 Zhang Boyang font: Fix integer overflow in ensure_comb_space() In fact it can't overflow at all because glyph_id->ncomb is only 8-bit wide. But let's keep safe if somebody changes the width of glyph_id->ncomb in the future. This patch also fixes the inconsistency between render_max_comb_glyphs and render_combining_glyphs when grub_malloc() returns NULL. Reviewed-by: Daniel Kiper 2022-11-14 Zhang Boyang font: Remove grub_font_dup_glyph() Remove grub_font_dup_glyph() since nobody is using it since 2013, and I'm too lazy to fix the integer overflow problem in it. Reviewed-by: Daniel Kiper 2022-11-14 Zhang Boyang font: Fix several integer overflows in grub_font_construct_glyph() This patch fixes several integer overflows in grub_font_construct_glyph(). Glyphs of invalid size, zero or leading to an overflow, are rejected. The inconsistency between "glyph" and "max_glyph_size" when grub_malloc() returns NULL is fixed too. Fixes: CVE-2022-2601 Reported-by: Zhang Boyang Reviewed-by: Daniel Kiper 2022-11-14 Zhang Boyang font: Fix size overflow in grub_font_get_glyph_internal() The length of memory allocation and file read may overflow. This patch fixes the problem by using safemath macros. There is a lot of code repetition like "(x * y + 7) / 8". It is unsafe if overflow happens. This patch introduces grub_video_bitmap_calc_1bpp_bufsz(). It is safe replacement for such code. It has safemath-like prototype. This patch also introduces grub_cast(value, pointer), it casts value to typeof(*pointer) then store the value to *pointer. It returns true when overflow occurs or false if there is no overflow. The semantics of arguments and return value are designed to be consistent with other safemath macros. Reviewed-by: Daniel Kiper 2022-11-14 Zhang Boyang font: Reject glyphs exceeds font->max_glyph_width or font->max_glyph_height Check glyph's width and height against limits specified in font's metadata. Reject the glyph (and font) if such limits are exceeded. Reviewed-by: Daniel Kiper 2022-11-14 t.feng loader/multiboot_elfxx: Fix memory leak The commit eb33e61b3 (multiboot: fix memory leak) did not fix all issues. Fix all of them right now. Fixes: eb33e61b3 (multiboot: fix memory leak) Reviewed-by: Daniel Kiper 2022-11-14 Damian Szuberski docs: Correct GRUB_DISABLE_LINUX_PARTUUID documentation Reviewed-by: Daniel Kiper 2022-11-14 Arsen Arsenović osdep/unix/getroot: Pass -P to zpool status zpool status by default prints basenames of VDEVs, which means that GRUB would have to go around guessing to see whether a VDEV exists. Instead, it'd be more robust to simply tell zpool to give us full paths to VDEVs via -P. Reviewed-by: Daniel Kiper 2022-11-14 Robbie Harwood normal/help: Add paging instructions to normal and help prompts This is not an ideal solution, as interactive users must always run a command in order to get the behavior they want, but it avoids problematic interactions between prompting and sourcing files. Reviewed-by: Daniel Kiper 2022-11-14 Robbie Harwood commands/tpm: Don't propagate measurement failures to the verifiers layer Currently if an EFI firmware fails to do a TPM measurement for a file, the error will be propagated to the verifiers framework which will prevent it to be opened. This mean that buggy firmwares will lead to the system not booting because files won't be allowed to be loaded. But a failure to do a TPM measurement isn't expected to be a fatal error that causes the system to be unbootable. To avoid this, don't return errors from .write and .verify_string callbacks and just print a debug message in the case of a TPM measurement failure. Add an environment variable, tpm_fail_fatal, to restore the previous behavior. Also-authored-by: Javier Martinez Canillas Reviewed-by: Daniel Kiper 2022-11-14 Robbie Harwood kern/env: Add function for retrieving variables as booleans Reviewed-by: Daniel Kiper 2022-11-14 Robbie Harwood types: Make bool generally available Add an include on stdbool.h, making the bool type generally available within the GRUB without needing to add a file-specific include every time it would be used. Reviewed-by: Daniel Kiper 2022-11-14 Raymund Will loader: Add support for grub-emu to kexec Linux menu entries The GRUB emulator is used as a debugging utility but it could also be used as a user-space bootloader if there is support to boot an operating system. The Linux kernel is already able to (re)boot another kernel via the kexec boot mechanism. So the grub-emu tool could rely on this feature and have linux and initrd commands that are used to pass a kernel, initramfs image and command line parameters to kexec for booting a selected menu entry. By default the systemctl kexec option is used so systemd can shutdown all of the running services before doing a reboot using kexec. But if this is not present, it can fall back to executing the kexec user-space tool directly. The ability to force a kexec-reboot when systemctl kexec fails must only be used in controlled environments to avoid possible filesystem corruption and data loss. Reviewed-by: Daniel Kiper 2022-11-14 Denton Liu templates: Introduce GRUB_TOP_LEVEL_* vars A user may wish to use an image that is not sorted as the "latest" version as the top-level entry. For example, in Arch Linux, if a user has the LTS and regular kernels installed, "/boot/vmlinuz-linux-lts" gets sorted as the "latest" compared to "/boot/vmlinuz-linux", meaning the LTS kernel becomes the top-level entry. However, a user may wish to use the regular kernel as the top-level default with the LTS only existing as a backup. This need can be seen in Arch Linux's AUR with two user-submitted packages[0][1] providing an update hook which patches /etc/grub.d/10_linux to move the desired kernel to the top-level. This patch serves to solve this in a more generic way. Introduce the GRUB_TOP_LEVEL, GRUB_TOP_LEVEL_XEN and GRUB_TOP_LEVEL_OS_PROBER variables to allow users to specify the top-level entry. Create grub_move_to_front() as a helper function which moves entries to the front of a list. This function does the heavy lifting of moving the menu entry to the front in each script. In 10_netbsd, since there isn't an explicit list variable, extract the items that are being iterated through into a list so that we can optionally apply grub_move_to_front() to the list before the loop. [0]: https://aur.archlinux.org/packages/grub-linux-default-hook [1]: https://aur.archlinux.org/packages/grub-linux-rt-default-hook Reviewed-by: Oskari Pirhonen Reviewed-by: Daniel Kiper 2022-10-27 Alec Brown video/readers: Add artificial limit to image dimensions In grub-core/video/readers/jpeg.c, the height and width of a JPEG image don't have an upper limit for how big the JPEG image can be. In Coverity, this is getting flagged as an untrusted loop bound. This issue can also seen in PNG and TGA format images as well but Coverity isn't flagging it. To prevent this, the constant IMAGE_HW_MAX_PX is being added to include/grub/bitmap.h, which has a value of 16384, to act as an artificial limit and restrict the height and width of images. This value was picked as it is double the current max resolution size, which is 8K. Fixes: CID 292450 Reviewed-by: Darren Kenny Reviewed-by: Daniel Kiper 2022-10-27 Daniel Axtens disk/diskfilter: Don't make a RAID array with more than 1024 disks This is "belt and braces" with commit 12e20a6a695f (disk/diskfilter: Check calloc() result for NULL): we end up trying to use too much memory in situations like corrupted Linux software RAID setups purporting to use a huge number of disks. Simply refuse to permit such configurations. 1024 is a bit arbitrary, yes, and I feel a bit like I'm tempting fate here, but I think 1024 disks in an array (that GRUB has to read to boot!) should be enough for anyone. Reviewed-by: Daniel Kiper 2022-10-27 Ard Biesheuvel arm64/efi/linux: Ignore FDT unless we need to modify it Now that we implemented support for the LoadFile2 protocol for initrd loading, there is no longer a need to pass the initrd parameters via the device tree. This means that when the LoadFile2 protocol is being used, there is no reason to update the device tree in the first place, and so we can ignore it entirely. The only remaining reason to deal with the devicetree is if we are using the "devicetree" command to load one from disk, so tweak the logic in grub_fdt_install() to take that into account. Reviewed-by: Leif Lindholm Reviewed-by: Daniel Kiper 2022-10-27 Ard Biesheuvel arm64/efi/linux: Implement LoadFile2 initrd loading protocol for Linux Recent Linux kernels will invoke the LoadFile2 protocol installed on a well-known vendor media path to load the initrd if it is exposed by the firmware. Using this method is preferred for two reasons: - the Linux kernel is in charge of allocating the memory, and so it can implement any placement policy it wants (given that these tend to change between kernel versions), - it is no longer necessary to modify the device tree provided by the firmware. So let's install this protocol when handling the "initrd" command if such a recent kernel was detected (based on the PE/COFF image version), and defer loading the initrd contents until the point where the kernel invokes the LoadFile2 protocol. Reviewed-by: Heinrich Schuchardt Tested-by: Ilias Apalodimas Reviewed-by: Ilias Apalodimas Reviewed-by: Daniel Kiper 2022-10-27 Ard Biesheuvel efi/efinet: Don't close connections at fini_hw() time When GRUB runs on top of EFI firmware, it only has access to block and network device abstractions exposed by the firmware, and it is up to the firmware to quiesce the underlying hardware when exiting boot services and handing over to the OS. This is especially important for network devices, to prevent incoming packets from being DMA'd straight into memory after the OS has taken over but before it has managed to reconfigure the network hardware. GRUB handles this by means of the grub_net_fini_hw() preboot hook, which is executed before calling into the booted image. This means that all network devices disappear or become inoperable before the EFI stub executes on EFI targeted builds. This is problematic as it prevents the EFI stub from calling back into GRUB provided protocols such as LoadFile2 for the initrd, which we will provide in a subsequent patch. So add a flag that indicates to the network core that EFI network devices should not be closed when grub_net_fini_hw() is called. Reviewed-by: Heinrich Schuchardt Reviewed-by: Daniel Kiper 2022-10-27 Ard Biesheuvel loader/arm64/linux: Account for COFF headers appearing at unexpected offsets The way we load the Linux and PE/COFF image headers depends on a fixed placement of the COFF header at offset 0x40 into the file. This is a reasonable default, given that this is where Linux emits it today. However, in order to comply with the PE/COFF spec, which allows this header to appear anywhere in the file, let's ensure that we read the header from where it actually appears in the file if it is not located at offset 0x40. Reviewed-by: Daniel Kiper 2022-10-27 Ard Biesheuvel arm/linux: Unify ARM/arm64 vs Xen PE/COFF header handling Xen has its own version of the image header, to account for the additional PE/COFF header fields. Since we are adding references to those in the shared EFI loader code, update the common definitions and drop the Xen specific one which no longer has a purpose. Since in both cases, the call to grub_arch_efi_linux_check_image() is preceded by a load of the image header, let's move the load into that function, and rename it to grub_arch_efi_linux_load_image_header(). Reviewed-by: Daniel Kiper 2022-10-27 Ard Biesheuvel efi: Move MS-DOS stub out of generic PE header definition The PE/COFF spec permits the COFF signature and file header to appear anywhere in the file, and the actual offset is recorded in 4 byte little endian field at offset 0x3c of the image. When GRUB is emitted as a PE/COFF binary, we reuse the 128 byte MS-DOS stub (even for non-x86 architectures), putting the COFF signature and file header at offset 0x80. However, other PE/COFF images may use different values, and non-x86 Linux kernels use an offset of 0x40 instead. So let's get rid of the grub_pe32_header struct from pe32.h, given that it does not represent anything defined by the PE/COFF spec. Instead, introduce a minimal struct grub_msdos_image_header type based on the PE/COFF spec's description of the image header, and use the offset recorded at file position 0x3c to discover the actual location of the PE signature and the COFF image header. The remaining fields are moved into a struct grub_pe_image_header, which we will use later to access COFF header fields of arbitrary images (and which may therefore appear at different offsets) Reviewed-by: Daniel Kiper 2022-10-27 Jagannathan Raman kern/buffer: Handle NULL input pointer in grub_buffer_free() The grub_buffer_free() should handle NULL input pointer, similar to grub_free(). If the pointer is not referencing any memory location, grub_buffer_free() need not perform any function. Fixes: CID 396931 Reviewed-by: Ross Philipson Reviewed-by: Daniel Kiper 2022-10-27 Jagannathan Raman fs/zfs/zfs: Update dangling dn_new pointer in dnode_get_path() The dnode_get_path() traverses dnode structures to locate the dnode leaf of a given path. When the leaf is a symlink to another path, it restarts the traversal either from root or from a different path. In such cases, dn_new must be re-initialized Passes "make check". Fixes: CID 86750 Reviewed-by: Ross Philipson Reviewed-by: Daniel Kiper 2022-10-27 Darren Kenny build: Update to reflect minimum clang version 8.0 After doing some validation with clang from versions 3.8 and up, the builds prior to version 8.0.0 fail due to the use of safemath functions at link time. Reviewed-by: Daniel Kiper 2022-10-27 Darren Kenny configure: Fix building with clang Building the current code with clang and the latest gnulib fails due to the use of a variable-length-array (vla) warning, which turns in to an error due to the presence of the -Werror during the build. The gnulib team stated that their code should not be built with -Werror. At present, the only way to do this is for the complete code-base, by using the --disable-werror option to configure. Rather than doing this, and failing to gain any benefit that it provides, instead, if building with clang, this patch makes it possible to specifically not error on vlas, while retaining the -Werror functionality otherwise. Reviewed-by: Daniel Kiper 2022-10-27 Darren Kenny gnulib: Provide abort() implementation for gnulib The recent gnulib updates require an implementation of abort(), but the current macro provided by changeset: cd37d3d3916c gnulib: Drop no-abort.patch to config.h.in does not work with the clang compiler since it doesn't provide a __builtin_trap() implementation, so this element of the changeset needs to be reverted, and replaced. After some discussion with Vladimir 'phcoder' Serbinenko and Daniel Kiper it was suggested to bring back in the change from the changeset: db7337a3d353 * grub-core/gnulib/regcomp.c (regerror): ... Which implements abort() as an inline call to grub_abort(), but since that was made static by changeset: a8f15bceeafe * grub-core/kern/misc.c (grub_abort): Make static it is also necessary to revert the specific part that makes it a static function too. Another implementation of abort() was found in grub-core/kern/compiler-rt.c which needs to also be removed to be consistent. Reviewed-by: Daniel Kiper 2022-10-27 Alec Brown disk/cryptodisk: Fix unintentional integer overflow In the function grub_cryptodisk_endecrypt(), a for loop is incrementing the variable i by (1U << log_sector_size). The variable i is of type grub_size_t which is a 64-bit unsigned integer on x86_64 architecture. On the other hand, 1U is a 32-bit unsigned integer. By performing a left shift on a 32-bit value and assigning it to a 64-bit variable, the 64-bit variable may have incorrect values in the high 32-bits if the shift has an overflow. To avoid this, we replace 1U with (grub_size_t)1. Fixes: CID 307788 Reviewed-by: Darren Kenny Reviewed-by: Patrick Steinhardt Reviewed-by: Daniel Kiper 2022-10-27 Zhang Boyang mm: Try invalidate disk caches last when out of memory Every heap grow will cause all disk caches invalidated which decreases performance severely. This patch moves disk cache invalidation code to the last of memory squeezing measures. So, disk caches are released only when there are no other ways to get free memory. Reviewed-by: Daniel Kiper Reviewed-by: Patrick Steinhardt 2022-10-27 Qiumiao Zhang util/grub-mkfont: Use valid conversion specifiers in printf() and fprintf() For printf()/fprintf() functions, unsigned integers should use %u as the valid conversion specifier instead of %d. Reviewed-by: Daniel Kiper 2022-10-27 Chris Coulson efi: Compile kernel.img with -fshort-wchar on all EFI targets The stack check logs a console message on failure, and the EFI API expects a NULL terminated UCS-2 string. In order to define a UCS-2 string literal, kernel.img on amd64 and i386 EFI targets is built with -fshort-wchar. Also compile kernel.img on other EFI targets with -fshort-wchar. Fixes: 37ddd94 (kern/efi/init: Log a console error during a stack check failure) Reported-by: Glenn Washburn Reviewed-by: Daniel Kiper 2022-10-11 Benjamin Herrenschmidt normal/menu: Add Ctrl-L to refresh the menu This is useful on cloud instances with remote serial ports as it can be difficult to connect "fast enough" to get the initial menu display Reviewed-by: Daniel Kiper 2022-10-11 Michael Chang util/grub-install: Set point of no return for powerpc-ieee1275 install The point of no return is used to define a point where no change should be reverted in a wake of fatal error that consequently aborts the process. The powerpc-ieee1275 install apparently missed this point of no return definition that newly installed modules could be inadvertently reverted after successful image embedding so that boot failure is incurred due to inconsistent state. Reviewed-by: Daniel Kiper 2022-10-11 Daniel Axtens disk/diskfilter: Check calloc() result for NULL With wildly corrupt inputs, we can end up trying to calloc a very large amount of memory, which will fail and give us a NULL pointer. We need to check that to avoid a crash. (And, even if we blocked such inputs, it is good practice to check the results of allocations anyway.) Reviewed-by: Daniel Kiper 2022-10-11 Glenn Washburn disk/cryptodisk: Allows UUIDs to be compared in a dash-insensitive manner A user can now specify UUID strings with dashes, instead of having to remove dashes. This is backwards-compatibility preserving and also fixes a source of user confusion over the inconsistency with how UUIDs are specified between file system UUIDs and cryptomount UUIDs. Since cryptsetup, the reference implementation for LUKS, displays and generates UUIDs with dashes there has been additional confusion when using the UUID strings from cryptsetup as exact input into GRUB does not find the expected cryptodisk. A new function grub_uuidcasecmp() is added that is general enough to be used other places where UUIDs are being compared. Reviewed-by: Patrick Steinhardt Reviewed-by: Daniel Kiper 2022-10-11 Glenn Washburn kern/corecmd: Quote variable values when displayed by the set command Variable values may contain spaces at the end or newlines. However, when displayed without quotes this is not obvious and can lead to confusion as to the actual contents of variables. Also for some variables grub_env_get() returns a NULL pointer instead of a pointer to an empty string and previously would be printed as "var=(null)". Now such variables will be displayed as "var=''". Reviewed-by: Daniel Kiper 2022-10-11 Samuel Thibault templates: Add support for acpi on Hurd This adds acpi as bootstrap module whenever it is available. This opens the path for proper IRQ routing for fully-userland disk drivers. Reviewed-by: Daniel Kiper 2022-10-11 Peter Jones util/grub-module-verifierXX: Enable running standalone checkers Allow treating util/grub-module-verifierXX.c as a file you can build directly so syntax checkers like vim's "syntastic" plugin, which uses "gcc -x c -fsyntax-only" to build it, will work. One still has to do whatever setup is required to make it pick the right include dirs, which -I options we use, etc., but this makes it so you can do the checking on the file you're editing, rather than on a different file. Reviewed-by: Daniel Kiper 2022-10-04 Tuan Phan kern/compiler-rt: Fix __clzsi2() logic Fix the incorrect return value of __clzsi2() function. Fixes: e795b90 (RISC-V: Add libgcc helpers for clz) Reviewed-by: Daniel Kiper 2022-10-04 Daniel Axtens efi: Increase default memory allocation to 32 MiB We have multiple reports of things being slower with a 1 MiB initial static allocation, and a report (more difficult to nail down) of a boot failure as a result of the smaller initial allocation. Make the initial memory allocation 32 MiB. Reviewed-by: Daniel Kiper 2022-10-04 Christian Hesse templates: Filter C.UTF-8 locale for translation In addition to C locale there is also C.UTF-8 locale now. Filter that as well, by using ${grub_lang}, which contains a stripped value. This fixes the following message and resulting boot failure: error: file `/boot/grub/locale/C.gmo' not found. Reviewed-by: Daniel Kiper 2022-10-04 Steve McIntyre tests: Explicitly unset SOURCE_DATE_EPOCH before running fs tests In some filesystem utils like mksquashfs, they will silently change behaviour and cause timestamps to unexpectedly change. Build environments like Debian's set SOURCE_DATE_EPOCH in the environment, so remove it. Reproducible builds are good and useful for shipped artifacts, but this causes build-time tests to fail. Reviewed-by: Daniel Kiper 2022-10-04 Heinrich Schuchardt commands/efi/lsefisystab: Short text for EFI_CONFORMANCE_PROFILES_TABLE The EFI_CONFORMANCE_PROFILES_TABLE_GUID is used for a table of GUIDs for conformance profiles (cf. UEFI specification 2.10, 4.6.5 EFI_CONFORMANCE_PROFILE_TABLE). The lsefisystab command is used to display installed EFI configuration tables. Currently it only shows the GUID but not a short text for the table. Provide a short text for the EFI_CONFORMANCE_PROFILES_TABLE_GUID. Reviewed-by: Daniel Kiper 2022-10-04 Theodore Ts'o fs/ext2: Ignore the large_dir incompat feature Recently, ext4 added the large_dir feature, which adds support for a 3 level htree directory support. The GRUB supports existing file systems with htree directories by ignoring their existence, and since the index nodes for the hash tree look like deleted directory entries (by design), the GRUB can simply do a brute force O(n) linear search of directories. The same is true for 3 level deep htrees indicated by large_dir feature flag. Hence, it is safe for the GRUB to ignore the large_dir incompat feature. Fixes: https://savannah.gnu.org/bugs/?61606 Reviewed-by: Daniel Kiper 2022-10-04 Glenn Washburn disk/loopback: Support transparent decompression of backing file A new option is added to the loopback command, -D or --decompress, which when specified transparently decompresses the backing file. This allows compressed images to be used as if they were uncompressed. Add documentation to support this change. Suggested-by: Li Gen Reviewed-by: Daniel Kiper 2022-10-04 Glenn Washburn configure: Add -DGRUB_HAS_PCI when compiling C/C++ files on targets that support PCI The list of targets that support PCI is in gentpl.py. However, there is no support for generating makefile script from a .def file that will apply globally to the makefile, but on a per target basis. So instead, use gentpl.py in configure to get the list of targets and check if the current build target is one of them. If it is, set the automake conditional COND_HAVE_PCI. Then in conf/Makefile.common add -DGRUB_HAS_PCI for the platform if COND_HAVE_PCI is true. Reviewed-by: Daniel Kiper 2022-10-04 Li Gen commands/read: Fix overflow in grub_getline() Store returned value from grub_getkey() in int instead of char to prevent throwing away the extended bits. This was a problem because, for instance, the left arrow key press would return (GRUB_TERM_EXTENDED | 0x4b), which would have the GRUB_TERM_EXTENDED thrown away leaving 0x4b or 'K'. These extended keys should either work as intended or do nothing. This change has them do nothing, instead of inserting a key not pressed by the user. Reviewed-by: Daniel Kiper 2022-10-04 Li Gen efi: Correct function prototype for register_key_notify() method of grub_efi_simple_text_input_ex_interface The register_key_notify() method should have an output parameter which is a pointer to the unique handle assigned to the registered notification. Reviewed-by: Daniel Kiper 2022-10-04 Masahiro Matsuya net/drivers/ieee1275/ofnet: Fix incorrect netmask The netmask configured in firmware is not respected on ppc64 (big endian). When 255.255.252.0 is set as netmask in firmware, the following is the value of bootpath string in grub_ieee1275_parse_bootpath(): /vdevice/l-lan@30000002:speed=auto,duplex=auto,192.168.88.10,,192.168.89.113,192.168.88.1,5,5,255.255.252.0,512 The netmask in this bootpath is not a problem, since it's a value specified in firmware. But the value of subnet_mask.ipv4 was set with 0xfffffc00, and __builtin_ctz(~grub_le_to_cpu32(subnet_mask.ipv4)) returned 16 (not 22). As a result, 16 was used for netmask wrongly: 1111 1111 1111 1111 1111 1100 0000 0000 # subnet_mask.ipv4(=0xfffffc00) 0000 0000 1111 1100 1111 1111 1111 1111 # grub_le_to_cpu32(subnet_mask.ipv4) 1111 1111 0000 0011 0000 0000 0000 0000 # ~grub_le_to_cpu32(subnet_mask.ipv4) and the count of zero with __builtin_ctz() can be 16. This patch changes it as below: 1111 1111 1111 1111 1111 1100 0000 0000 # subnet_mask.ipv4(=0xfffffc00) 0000 0000 1111 1100 1111 1111 1111 1111 # grub_le_to_cpu32(subnet_mask.ipv4) 1111 1111 1111 1111 1111 1100 0000 0000 # grub_be_to_cpu32(subnet_mask.ipv4) 0000 0000 0000 0000 0000 0011 1111 1111 # ~grub_be_to_cpu32(subnet_mask.ipv4) The count of zero with __builtin_clz() can be 22 (clz counts the number of one bits preceding the most significant zero bit). Reviewed-by: Daniel Kiper 2022-10-04 Ross Philipson loader/i386/bsd: Initialize BSD relocator state variables Numerous register fields in the relocator state are simply not used depending on the relocator. This causes Coverity to flag these fields but there is no real bug here. Simply initializing the variable to {0} solves the issue. Fixed in the else case too for consistency. Fixes: CID 396932 Reviewed-by: Daniel Kiper 2022-08-20 Andrea G. Monaco docs: Add a link to environment variables This is trivial, but it might save some time to beginners. Reviewed-by: Glenn Washburn Reviewed-by: Daniel Kiper 2022-08-20 Robbie Harwood docs: Fix mismatched brackets in halt command Reviewed-by: Daniel Kiper docs: Document fwsetup command Reviewed-by: Daniel Kiper 2022-08-20 Robbie Harwood efi: Don't display a uefi-firmware entry if it's not supported Add a new --is-supported option to commands/efi/efifwsetup and conditionalize display on it. Reviewed-by: Daniel Kiper 2022-08-20 Javier Martinez Canillas commands/efi/efifwsetup: Print an error if boot to firmware setup is not supported The "fwsetup" command is only registered if the firmware supports booting to the firmware setup UI. But it could be possible that the GRUB config already contains a "fwsetup" entry, because it was generated in a machine that has support for this feature. To prevent users getting an error like: error: ../../grub-core/script/function.c:109:can't find command `fwsetup'. if it is not supported by the firmware, let's just always register the command but print a more accurate message if the firmware doesn't support this option. Reviewed-by: Daniel Kiper 2022-08-20 Javier Martinez Canillas templates: Check for EFI at runtime instead of config generation time The 30_uefi-firmware template checks if an OsIndicationsSupported UEFI var exists and EFI_OS_INDICATIONS_BOOT_TO_FW_UI bit is set, to decide whether a "fwsetup" menu entry would be added or not to the GRUB menu. But this has the problem that it will only work if the configuration file was created on an UEFI machine that supports booting to a firmware UI. This for example doesn't support creating GRUB config files when executing on systems that support both UEFI and legacy BIOS booting. Since creating the config file from legacy BIOS wouldn't allow to access the firmware UI. To prevent this, make the template to unconditionally create the grub.cfg snippet but check at runtime if was booted through UEFI to decide if this entry should be added. That way it won't be added when booting with BIOS. There's no need to check if EFI_OS_INDICATIONS_BOOT_TO_FW_UI bit is set, since that's already done by the "fwsetup" command when is executed. Reviewed-by: Daniel Kiper 2022-08-20 Robbie Harwood efi: Make all grub_efi_guid_t variables static This is believed to result in smaller code. Reviewed-by: Daniel Kiper 2022-08-20 Robbie Harwood commands/efi/efifwsetup: Add missing grub_free()s Each call of grub_efi_get_variable() needs a grub_free(). Reviewed-by: Daniel Kiper 2022-08-19 Jagannathan Raman fs/zfs/zfs: Pass pointer to dnode_end_t instead of value to fill_fs_info() Coverity reports that dnode_end_t argument of fill_fs_info() is too large to pass-by-value. Therefore, replace the argument with a pointer. Fixes: CID 73631 Reviewed-by: Darren Kenny Reviewed-by: Daniel Kiper 2022-08-19 Patrick Steinhardt disk/luks2: Fix decoding of digests and salts with escaped chars It was reported in the #grub IRC channel on Libera that decryption of LUKS2 partitions fails with errors about invalid digests and/or salts. In all of these cases, what failed was decoding the Base64 representation of these, where the encoded data contained invalid characters. As it turns out, the root cause is that json-c, which is used by cryptsetup to read and write the JSON header, will escape some characters by prepending a backslash when writing JSON strings by default. Most importantly, json-c also escapes the forward slash, which is part of the Base64 alphabet. Because GRUB doesn't know to unescape such characters, decoding this string will rightfully fail. Interestingly, this issue has until now only been reported by users of Ubuntu 18.04. And a bit of digging in fact reveals that cryptsetup has changed the logic in a054206d (Suppress useless slash escaping in json lib, 2018-04-20), which has been released with cryptsetup v2.0.3. Ubuntu 18.04 is still shipping with cryptsetup v2.0.2 though, which explains why this is not a more frequent issue. Fix the issue by using our new grub_json_unescape() helper function that handles unescaping for us. Reported-by: Afdal Reviewed-by: Daniel Kiper 2022-08-19 Patrick Steinhardt lib/json/json: Add function to unescape JSON-encoded strings JSON strings require certain characters to be encoded, either by using a single reverse solidus character "\" for a set of popular characters, or by using a Unicode representation of "\uXXXXX". The jsmn library doesn't handle unescaping for us, so we must implement this functionality for ourselves. Add a new function grub_json_unescape() that takes a potentially escaped JSON string as input and returns a new unescaped string. Reviewed-by: Daniel Kiper 2022-08-19 Nikita Ermakov loader: Drop argv[] argument in grub_initrd_load() In the case of an error grub_initrd_load() uses argv[] to print the filename that caused the error. It is also possible to obtain the filename from the file handles and there is no need to duplicate that information in argv[], so let's drop it. Reviewed-by: Daniel Kiper 2022-08-19 Alec Brown loader: Update error conditionals to use enums In grub-core/loader/i386/bsdXX.c and grub-core/loader/multiboot_elfxx.c, error conditionals are simplified to statements such as "if (err)". Even though the assumption that non-zero values give errors is correct, it would be clearer and more consistent to compare these conditionals to GRUB_ERR_NONE. Reviewed-by: Darren Kenny Reviewed-by: Daniel Kiper 2022-08-19 Alec Brown util/grub-module-verifierXX: Changed get_shnum() return type In util/grub-module-verifierXX.c, the function get_shnum() returns the variable shnum, which is of the type Elf_Word. In the function, shnum can be obtained by the e_shnum member of an Elf_Ehdr or the sh_size member of an Elf_Shdr. The sh_size member can either be grub_uint32_t or grub_uint64_t, depending on the architecture, but Elf_Word is only grub_uint32_t. To account for when sh_size is grub_uint64_t, we can set shnum to have type Elf_Shnum and have get_shnum() return an Elf_Shnum. Reviewed-by: Darren Kenny Reviewed-by: Daniel Kiper 2022-08-19 Alec Brown elf: Validate number of elf program header table entries In bsdXX.c and multiboot_elfxx.c, e_phnum is used to obtain the number of program header table entries, but it wasn't being checked if the value was there. According to the elf(5) manual page, "If the number of entries in the program header table is larger than or equal to PN_XNUM (0xffff), this member holds PN_XNUM (0xffff) and the real number of entries in the program header table is held in the sh_info member of the initial entry in section header table. Otherwise, the sh_info member of the initial entry contains the value zero." Since this check wasn't being made, grub_elfXX_get_phnum() is being added to elfXX.c to make this check and use e_phnum if it doesn't have PN_XNUM as a value, else use sh_info. We also need to make sure e_phnum isn't greater than PN_XNUM and sh_info isn't less than PN_XNUM. Note that even though elf.c and elfXX.c are located in grub-core/kern, they are compiled as modules and don't need the EXPORT_FUNC() macro to define the functions in elf.h. Also, changed casts of phnum to match variables being set as well as dropped casts when unnecessary. Reviewed-by: Darren Kenny Reviewed-by: Daniel Kiper 2022-08-19 Alec Brown elf: Validate elf section header table index for section name string table In multiboot_elfxx.c, e_shstrndx is used to obtain the section header table index of the section name string table, but it wasn't being checked if the value was there. According to the elf(5) manual page, "If the index of section name string table section is larger than or equal to SHN_LORESERVE (0xff00), this member holds SHN_XINDEX (0xffff) and the real index of the section name string table section is held in the sh_link member of the initial entry in section header table. Otherwise, the sh_link member of the initial entry in section header table contains the value zero." Since this check wasn't being made, grub_elfXX_get_shstrndx() is being added to elfXX.c to make this check and use e_shstrndx if it doesn't have SHN_XINDEX as a value, else use sh_link. We also need to make sure e_shstrndx isn't greater than or equal to SHN_LORESERVE and sh_link isn't less than SHN_LORESERVE. Note that even though elf.c and elfXX.c are located in grub-core/kern, they are compiled as modules and don't need the EXPORT_FUNC() macro to define the functions in elf.h. Reviewed-by: Darren Kenny Reviewed-by: Daniel Kiper 2022-08-19 Alec Brown elf: Validate number of elf section header table entries In bsdXX.c and multiboot_elfxx.c, e_shnum is used to obtain the number of section header table entries, but it wasn't being checked if the value was there. According to the elf(5) manual page, "If the number of entries in the section header table is larger than or equal to SHN_LORESERVE (0xff00), e_shnum holds the value zero and the real number of entries in the section header table is held in the sh_size member of the initial entry in section header table. Otherwise, the sh_size member of the initial entry in the section header table holds the value zero." Since this check wasn't being made, grub_elfXX_get_shnum() is being added to elfXX.c to make this check and use whichever member doesn't have a value of zero. If both are zero, then we must return an error. We also need to make sure that e_shnum doesn't have a value greater than or equal to SHN_LORESERVE and sh_size isn't less than SHN_LORESERVE. In order to get this function to work, the type ElfXX_Shnum is being added where Elf32_Shnum defines Elf32_Word and Elf64_Shnum defines Elf64_Xword. This new type is needed because if shnum obtains a value from sh_size, sh_size could be of type El32_Word for Elf32_Shdr structures or Elf64_Xword for Elf64_Shdr structures. Note that even though elf.c and elfXX.c are located in grub-core/kern, they are compiled as modules and don't need the EXPORT_FUNC() macro to define the functions in elf.h. For a few smaller changes, changed casts of shnum to match variables being set as well as dropped casts when unnecessary and fixed spacing errors in bsdXX.c. Also, shnum is an unsigned integer and is compared to int i in multiboot_elfxx.c, it should be unsigned to match shnum. Reviewed-by: Darren Kenny Reviewed-by: Daniel Kiper 2022-08-19 Mauricio Faria de Oliveira templates/linux_xen: Properly order the multiple initrd files The linux_xen template orders the "early" initrd file(s) _first_ (i.e., before the "real" initrd files) and that seems reasonable, as microcode updates usually come first. However, this usually breaks Linux boot with initrd under Xen because Xen assumes the real initrd is the first multiboot[2] module after the kernel, passing its address over to Linux in Xen's start_info struct. So, if a microcode-only initrd (i.e., without init/userspace) is found by grub-mkconfig, it ends up considered as a normal initrd by the Linux kernel, which cannot do anything with it (as it has no other files) and panic()s unable to mount root if it depends on a initrd to do that (e.g., root=UUID=...). ... Well, since Xen doesn't actually use the provided microcode by default / unless the 'ucode=' option is enabled, this isn't used in the general case (and breaks). Additionally, if an user enables the 'ucode=' option, that either specifies which module is to be used for microcode, or scans all modules (regardless of being first) for that. Thus, for Xen: - it is *not required* to have microcode first, - but it is *required* to have real initrd first So, fix it by ordering the real initrd before early initrd(s). After: # touch /boot/xen /boot/microcode.cpio # grub-mkconfig 2>/dev/null | grep -P '^\t(multiboot|module)' multiboot /boot/xen ... module /boot/vmlinuz-5.4.0-122-generic ... module --nounzip /boot/initrd.img-5.4.0-122-generic module --nounzip /boot/microcode.cpio ... Corner case specific to Xen implementation details: It is actually _possible_ to have a microcode initrd first, but that requires a non-default option (so can't rely on it), and it turns out to be inconsistent with its counterpart (really shouldn't rely on it, as it may get confusing; below). 'ucode=1' does manually specify the first module is microcode _AND_ clears its bit in the module bitmap. The next module is now the 'new first', and gets passed to Linux as initrd. Good. 'ucode=scan' checks all modules for microcode, but does _NOT_ clear a bit if it finds one (reasonable, as it can find that prepended in a "real" initrd anyway, which needs to be used). The first module still gets passed to Linux as initrd. Bad. Fixes: e86f6aafb8de (grub-mkconfig/20_linux_xen: Support multiple early initrd images) Acked-by: Juergen Gross Reviewed-by: Daniel Kiper 2022-08-19 Mauricio Faria de Oliveira templates/linux_xen: Properly load multiple initrd files The linux_xen template can put multiple initrd files in the same multiboot[2] module[2] command, which is against specs. This causes ONLY the _first_ initrd file to be loaded; other files just have filenames in a "cmdline" string of the first initrd file and are NOT loaded. Fix this by inserting a module[2] command per initrd file. Before: # touch /boot/xen /boot/microcode.cpio # grub-mkconfig 2>/dev/null | grep -P '^\t(multiboot|module)' multiboot /boot/xen ... module /boot/vmlinuz-5.4.0-122-generic ... module --nounzip /boot/microcode.cpio /boot/initrd.img-5.4.0-122-generic After: # touch /boot/xen /boot/microcode.cpio # grub-mkconfig 2>/dev/null | grep -P '^\t(multiboot|module)' multiboot /boot/xen ... module /boot/vmlinuz-5.4.0-122-generic ... module --nounzip /boot/microcode.cpio module --nounzip /boot/initrd.img-5.4.0-122-generic Cause: The code was copied from the linux template, which is *apparently* equivalent.. but its initrd command grub_cmd_initrd() *supports* multiple files (see grub_initrd_init()), while module/module2 in grub_cmd_module() *does not* (see grub_multiboot[2]_add_module()). See commit e86f6aafb8de (grub-mkconfig/20_linux_xen: Support multiple early initrd images): 'This is basically a copy of a698240d "grub-mkconfig/10_linux: Support multiple early initrd images" ...' Specs: Both multiboot and multiboot2 specifications mention support for 'multiple boot modules' (struct/tag used for kernel/initrd files): "Boot loaders don’t have to support multiple boot modules, but they are strongly encouraged to" [1,2] However, there is a 1:1 relationship between boot modules and files, more or less clearly; note the usage of singular/plural "module(s)". (Multiboot2, clearly: "One tag appears per module".) Multiboot [1]: "the ‘mods’ fields indicate ... what boot modules were loaded ..., and where they can be found. ‘mods_count’ contains the number of modules loaded" "The first two fields contain the start and end addresses of the boot module itself." Multiboot2 [2]: "This tag indicates ... what boot module was loaded ..., and where it can be found." "The ‘mod_start’ and ‘mod_end’ contain the start and end physical addresses of the boot module itself." "One tag appears per module. This tag type may appear multiple times." And both clearly mention the 'string' field of a boot module, which is to be used by the operating system, not boot loader: "The ‘string’ field provides an arbitrary string to be associated with that particular boot module ... its exact use is specific to the operating system." Links: [1] https://www.gnu.org/software/grub/manual/multiboot/multiboot.html 3.3 Boot information format [2] https://www.gnu.org/software/grub/manual/multiboot2/multiboot.html 3.6.6 Modules Fixes: e86f6aafb8de (grub-mkconfig/20_linux_xen: Support multiple early initrd images) Acked-by: Juergen Gross Reviewed-by: Daniel Kiper 2022-08-19 Glenn Washburn misc: Add cast in grub_strncasecmp() to drop sign when calling grub_tolower() Note this cast was fixed in grub_strcasecmp() in commit ce41ab7aab (* grub-core/kern/misc.c (grub_strcmp): Use unsigned comparison as per common usage and preffered in several parts of code.), but this commit omitted fixing it in grub_strncasecmp(). Reviewed-by: Daniel Kiper 2022-08-19 Glenn Washburn tests/util/grub-shell: Only show grub-mkrescue output if it returns an error The previous behavior ignored an error and the output from grub-mkrescue. This made it difficult to discover that grub-mkrescue was the reason that tests which rely on grub-shell were failing. Even after discovering grub-mkrescue was the culprit, there was no output to indicate why it was failing. It turns out that grub-mkrescue is a thin wrapper around xorriso. So if you do not have xorriso installed it will fail with an error message about not being able to find xorriso. This change will allow grub-mkrescue output to be written to stderr, only if grub-mkrescue fails. If grub-mkrescue succeeds, there will be no output from grub-mkrescue so as not to interfere with the functioning of tests. This change should have no effect on the running of tests or other uses of grub-shell as it only modifies the error path. Also, if grub-mkrescue fails, the script exits early. Since grub-shell needs the ISO image created by grub-mkresue to boot the QEMU instance, a failure here should be considered fatal. Reviewed-by: Daniel Kiper 2022-08-19 Ard Biesheuvel loader/arm64/linux: Remove magic number header field check The "ARM\x64" magic number in the file header identifies an image as one that implements the bare metal boot protocol, allowing the loader to simply move the file to a suitably aligned address in memory, with sufficient headroom for the trailing .bss segment (the required memory size is described in the header as well). Note of this matters for GRUB, as it only supports EFI boot. EFI does not care about this magic number, and nor should GRUB: this prevents us from booting other PE linux images, such as the generic EFI zboot decompressor, which is a pure PE/COFF image, and does not implement the bare metal boot protocol. So drop the magic number check. Reviewed-by: Daniel Kiper 2022-08-19 Darren Kenny util/grub-install-common: Confirm directory creation in grub_install_mkdir_p() Because grub_util_mkdir() is implemented to not return a value on any platform, grub_instal_mkdir_p() can test for success by confirming that the directory requested exists after attempting to create it, otherwise it should fail with an error and exit. While fixing this, a flaw in the logic was shown, where the first match of the path separator, which almost always was the first character in the path (e.g. /boot/grub2) would result in creating a directory with an empty name (i.e. ""). To avoid that, it should skip the handling of the path separator where p is pointing to the first character. Reviewed-by: Daniel Kiper 2022-08-19 Darren Kenny util: Ignore return value for grub_util_mkdir() on all platforms Coverity signaled 2 issues where the return value of grub_util_mkdir() was not being tested. The Windows variant of this code defines the function as having no return value (void), but the UNIX variants all are mapped using a macro to the libc mkdir() function, which returns an int value. To be consistent, the mapping should cast to void to for these too. Fixes: CID 73583 Fixes: CID 73617 Reviewed-by: Daniel Kiper 2022-08-19 Glenn Washburn disk/cryptodisk: Support encrypted volumes using detached headers on a partition Update the read hook to take into account encrypted volumes on a partition. GRUB disk read hooks supply an absolute sector number at which the read is started from. If the encrypted volume is in a partition, the sector number given to the read hook will be offset by the number of the sector at the start of the partition. The read hook then needs to subtract the partition start from the supplied sector to get the correct start sector for the read into the detached header file. Reported-by: brutser Tested-by: brutser Reviewed-by: Daniel Kiper 2022-08-10 Glenn Washburn tests/util/grub-shell: Use shell variable instead of autoconf By using shell variable that are set once by the expansion of an autoconf variable, the resulting shell script is more easily moved and modified from the build/install directory it was generated for. The resulting script is more readable as well. Reviewed-by: Daniel Kiper 2022-08-10 Stefan Agner Makefile: Make grub_fstest.pp depend on config-util.h If you build with "make -j25", sometimes you see: /build/output_generic_x86_64/host/bin/x86_64-buildroot-linux-gnu-gcc -E -DHAVE_CONFIG_H -I. -I.. -Wall -W -DGRUB_UTIL=1 -D_FILE_OFFSET_BITS=64 -I./include -DGRUB_FILE=\"util/grub-fstest.c\" -I. -I.. -I. -I.. -I../include -I./include -I../grub-core/lib/libgcrypt-grub/src/ -I./grub-core/lib/gnulib -I../grub-core/lib/gnulib -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -fno-stack-protector -D_FILE_OFFSET_BITS=64 \ -D'GRUB_MOD_INIT(x)=@MARKER@x@' ../util/grub-fstest.c ../grub-core/kern/emu/hostfs.c ../grub-core/disk/host.c ../grub-core/osdep/init.c > grub_fstest.pp || (rm -f grub_fstest.pp; exit 1) config.status: creating config-util.h ../grub-core/kern/emu/hostfs.c:20:10: fatal error: config-util.h: No such file or directory 20 | #include | ^~~~~~~~~~~~~~~ compilation terminated. Reviewed-by: Daniel Kiper 2022-08-10 Qiumiao Zhang util/grub-mkfont: Fix resource leaks Reviewed-by: Daniel Kiper 2022-08-10 Peter Jones kern/i386/tsc_pmtimer: Make pmtimer tsc calibration not take 51 seconds to fail On my laptop running at 2.4GHz, if I run a VM where tsc calibration using pmtimer will fail presuming a broken pmtimer, it takes ~51 seconds to do so (as measured with the stopwatch on my phone), with a tsc delta of 0x1cd1c85300, or around 125 billion cycles. If instead of trying to wait for 5-200ms to show up on the pmtimer, we try to wait for 5-200us, it decides it's broken in ~0x2626aa0 TSCs, aka ~2.4 million cycles, or more or less instantly. Additionally, this reading the pmtimer was returning 0xffffffff anyway, and that's obviously an invalid return. I've added a check for that and 0 so we don't bother waiting for the test if what we're seeing is dead pins with no response at all. If "debug" includes "pmtimer", you will see one of the following three outcomes. If pmtimer gives all 0 or all 1 bits, you will see: pmtimer: 0xffffff bad_reads: 1 pmtimer: 0xffffff bad_reads: 2 pmtimer: 0xffffff bad_reads: 3 pmtimer: 0xffffff bad_reads: 4 pmtimer: 0xffffff bad_reads: 5 pmtimer: 0xffffff bad_reads: 6 pmtimer: 0xffffff bad_reads: 7 pmtimer: 0xffffff bad_reads: 8 pmtimer: 0xffffff bad_reads: 9 pmtimer: 0xffffff bad_reads: 10 timer is broken; giving up. This outcome was tested using qemu+kvm with UEFI (OVMF) firmware and these options: -machine pc-q35-2.10 -cpu Broadwell-noTSX If pmtimer gives any other bit patterns but is not actually marching forward fast enough to use for clock calibration, you will see: pmtimer delta is 0x0 (1904 iterations) tsc delta is implausible: 0x2626aa0 This outcome was tested using GRUB patched to not ignore bad reads using qemu+kvm with UEFI (OVMF) firmware, and these options: -machine pc-q35-2.10 -cpu Broadwell-noTSX If pmtimer actually works, you'll see something like: pmtimer delta is 0xdff tsc delta is 0x278756 This outcome was tested using qemu+kvm with UEFI (OVMF) firmware, and these options: -machine pc-i440fx-2.4 -cpu Broadwell-noTSX I've also tested this outcome on a real Intel Xeon E3-1275v3 on an Intel Server Board S1200V3RPS using the SDV.RP.B8 "Release" build here: https://www.intel.com/content/www/us/en/download/674448/firmware-update-for-the-intel-server-board-s1200rp-uefi-development-kit-release-vb8.html Reviewed-by: Daniel Kiper 2022-08-10 Glenn Washburn disk/luks2: Continue trying all keyslots even if there are some failures luks2_get_keyslot() can fail for a variety of reasons that do not necessarily mean the next keyslot should not be tried (e.g. a new kdf type). So always try the next slot. This will make GRUB more resilient to non-spec json data that 3rd party systems may add. We do not care if some of the keyslots are unusable, only if there is at least one that is. Reviewed-by: Daniel Kiper 2022-08-10 Glenn Washburn efi: Add efitextmode command for getting/setting the text mode resolution This command is meant to behave similarly to the "mode" command of the EFI Shell application. In addition to allowing mode selection by giving the number of columns and rows as arguments, the command allows specifying the mode number to select the mode. Also supported are the arguments "min" and "max", which set the mode to the minimum and maximum mode respectively as calculated by the columns * rows of that mode. Reviewed-by: Daniel Kiper 2022-07-27 Robbie Harwood fs/fat: Don't error when mtime is 0 In the wild, we occasionally see valid ESPs where some file modification times are 0. For instance: ├── [Dec 31 1979] EFI │ ├── [Dec 31 1979] BOOT │ │ ├── [Dec 31 1979] BOOTX64.EFI │ │ └── [Dec 31 1979] fbx64.efi │ └── [Jun 27 02:41] fedora │ ├── [Dec 31 1979] BOOTX64.CSV │ ├── [Dec 31 1979] fonts │ ├── [Mar 14 03:35] fw │ │ ├── [Mar 14 03:35] fwupd-359c1169-abd6-4a0d-8bce-e4d4713335c1.cap │ │ ├── [Mar 14 03:34] fwupd-9d255c4b-2d88-4861-860d-7ee52ade9463.cap │ │ └── [Mar 14 03:34] fwupd-b36438d8-9128-49d2-b280-487be02d948b.cap │ ├── [Dec 31 1979] fwupdx64.efi │ ├── [May 10 10:47] grub.cfg │ ├── [Jun 3 12:38] grub.cfg.new.new │ ├── [May 10 10:41] grub.cfg.old │ ├── [Jun 27 02:41] grubenv │ ├── [Dec 31 1979] grubx64.efi │ ├── [Dec 31 1979] mmx64.efi │ ├── [Dec 31 1979] shim.efi │ ├── [Dec 31 1979] shimx64.efi │ └── [Dec 31 1979] shimx64-fedora.efi └── [Dec 31 1979] FSCK0000.REC 5 directories, 17 files This causes grub-probe failure, which in turn causes grub-mkconfig failure. They are valid filesystems that appear intact, and the Linux FAT stack is able to mount and manipulate them without complaint. The check for mtime of 0 has been present since 20def1a3c3952982395cd7c3ea7e78638527962b (fat: support file modification times). Reviewed-by: Daniel Kiper 2022-07-27 Robbie Harwood kern/fs: The grub_fs_probe() should dprint errors from filesystems When filesystem detection fails, all that's currently debug-logged is a series of messages like: grub-core/kern/fs.c:56:fs: Detecting ntfs... grub-core/kern/fs.c:76:fs: ntfs detection failed. repeated for each filesystem. Any messages provided to grub_error() by the filesystem are lost, and one has to break out gdb to figure out what went wrong. With this change, one instead sees: grub-core/kern/fs.c:56:fs: Detecting fat... grub-core/osdep/hostdisk.c:357:hostdisk: reusing open device `/path/to/device' grub-core/kern/fs.c:77:fs: error: invalid modification timestamp for /. grub-core/kern/fs.c:79:fs: fat detection failed. in the debug prints. Reviewed-by: Daniel Kiper 2022-07-27 Robbie Harwood util/grub-probe: Document the behavior of multiple -v Reviewed-by: Daniel Kiper 2022-07-27 Ross Philipson lib/relocator: Initialize local relocator subchunk struct to all zeros The way the code is written the tofree variable would never be passed to the free_subchunk() function uninitialized. Coverity cannot determine this and flags the situation as "Using uninitialized value...". The fix is just to initialize the local struct. Fixes: CID 314016 Reviewed-by: Darren Kenny Tested-by: Alec Brown Reviewed-by: Daniel Kiper 2022-07-27 Lu Ken efi/tpm: Add EFI_CC_MEASUREMENT_PROTOCOL support The EFI_CC_MEASUREMENT_PROTOCOL abstracts the measurement for virtual firmware in confidential computing environment. It is similar to the EFI_TCG2_PROTOCOL. It was proposed by Intel and ARM and approved by UEFI organization. It is defined in Intel GHCI specification: https://cdrdv2.intel.com/v1/dl/getContent/726790 . The EDKII header file is available at https://github.com/tianocore/edk2/blob/master/MdePkg/Include/Protocol/CcMeasurement.h . Reviewed-by: Daniel Kiper 2022-07-27 Lu Ken commands/efi/tpm: Use grub_strcpy() instead of grub_memcpy() The event description is a string, so using grub_strcpy() is cleaner than using grub_memcpy(). Reviewed-by: Daniel Kiper 2022-07-27 Lu Ken commands/efi/tpm: Refine the status of log event 1. Use macro GRUB_ERR_NONE instead of hard code 0. 2. Keep lowercase of the first char for the status string of log event. Reviewed-by: Daniel Kiper 2022-07-12 Nicholas Vinson configure: Warn if stack protector is not allowed Introduce ERROR_PLATFORM_NOT_SUPPORT_SSP environment variable to treat the "--enable-stack-protector is only supported on EFI platforms" message as a warning instead of an error. If ERROR_PLATFORM_NOT_SUPPORT_SSP is set to "no" (case-insensitive), then the message will be printed as a warning. Otherwise, it prints as an error. The default behavior is to print the message as an error. For any wrapper build script that has some variation of: for p in SELECTED_GRUB_PLATFORMS; do \ configure --enable-stack-protector \ --with-platform${P} ... || die; \ done make The GRUB will fail to build if SELECTED_GRUB_PLATFORMS contains a platform that does not support SSP. Such wrapper scripts need to work-around this issue by modifying the above for-loop, so it conditionally passes --enable-stack-protector to configure for the proper GRUB platform(s). However, if the above example is modified to have to conditionally pass in --enable-stack-protector, its behavior is effectively the same as the proposed change. Additionally, The list of SSP supported platforms is now in 2 places. One in the configure script and one in the build wrapper script. If the second list is not properly maintained it could mistakenly disable SSP for a platform that later gained support for it. Reviewed-by: Daniel Kiper 2022-07-12 Darren Kenny util/grub-mkfont: Fix tainted loop boundary issues with substitutions With gsub substitutions the offsets should be validated against the number of glyphs in a font face and the memory allocated for the gsub substitution data. Both the number of glyphs and the last address in the allocated data are passed in to process_cursive(), where the number of glyphs validates the end of the range. Enabling memory allocation validation uses two macros, one to simply check the address against the allocated space, and the other to check that the number of items of a given size doesn't extend outside of the allocated space. Fixes: CID 73770 Fixes: CID 314040 Reviewed-by: Daniel Kiper 2022-07-12 Glenn Washburn efi: Add missing header from include/grub/efi/console_control.h Reviewed-by: Daniel Kiper 2022-07-04 Glenn Washburn disk: Replace code that calculates the log of sector size with grub_log2ull() Reviewed-by: Daniel Kiper 2022-07-04 Mathieu Desnoyers templates: Remove unused version comparison functions There are no users left of version_find_latest(), version_test_gt(), and version_test_numeric(). Remove those unused helper functions. Using those helper functions is what caused the quadratic sorting performance issues in the first place, so removing them is a net win. Reviewed-by: Robbie Harwood Reviewed-by: Daniel Kiper 2022-07-04 Mathieu Desnoyers templates/kfreebsd: Fix quadratic algorithm for sorting menu items The current implementation of the 10_kfreebsd script implements its menu items sorting in bash with a quadratic algorithm, calling "sed", "sort", "head", and "grep" to compare versions between individual lines, which is annoyingly slow for kernel developers who can easily end up with 50-100 kernels in their boot partition. This fix is ported from the 10_linux script, which has a similar quadratic code pattern. Cc: debian-bsd@lists.debian.org Reviewed-by: Daniel Kiper 2022-07-04 Mathieu Desnoyers templates/hurd: Fix quadratic algorithm for sorting menu items The current implementation of the 10_hurd script implements its menu items sorting in bash with a quadratic algorithm, calling "sed", "sort", "head", and "grep" to compare versions between individual lines, which is annoyingly slow for kernel developers who can easily end up with 50-100 kernels in their boot partition. This fix is ported from the 10_linux script, which has a similar quadratic code pattern. Cc: Samuel Thibault Tested-by: Samuel Thibault Reviewed-by: Daniel Kiper 2022-07-04 Mathieu Desnoyers templates/linux_xen: Fix quadratic algorithm for sorting menu items The current implementation of the 20_linux_xen script implements its menu items sorting in bash with a quadratic algorithm, calling "sed", "sort", "head", and "grep" to compare versions between individual lines, which is annoyingly slow for kernel developers who can easily end up with 50-100 kernels in their boot partition. This fix is ported from the 10_linux script, which has a similar quadratic code pattern. Cc: xen-devel@lists.xenproject.org Tested-by: Jason Andryuk Reviewed-by: Daniel Kiper 2022-07-04 Mathieu Desnoyers templates/linux: Fix quadratic algorithm for sorting menu items The current implementation of the 10_linux script implements its menu items sorting in bash with a quadratic algorithm, calling "sed", "sort", "head", and "grep" to compare versions between individual lines, which is annoyingly slow for kernel developers who can easily end up with 50-100 kernels in /boot. As an example, on a Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz, running: /usr/sbin/grub-mkconfig > /dev/null With 44 kernels in /boot, this command takes 10-15 seconds to complete. After this fix, the same command runs in 5 seconds. With 116 kernels in /boot, this command takes 40 seconds to complete. After this fix, the same command runs in 8 seconds. For reference, the quadratic algorithm here is: while [ "x$list" != "x" ] ; do <--- outer loop linux=`version_find_latest $list` version_find_latest() for i in "$@" ; do <--- inner loop version_test_gt() fork+exec sed version_test_numeric() version_sort fork+exec sort fork+exec head -n 1 fork+exec grep list=`echo $list | tr ' ' '\n' | fgrep -vx "$linux" | tr '\n' ' '` tr fgrep tr So all commands executed under version_test_gt() are executed O(n^2) times where n is the number of kernel images in /boot. Here is the improved algorithm proposed: - Prepare a list with all the relevant information for ordering by a single sort(1) execution. This is done by renaming ".old" suffixes by " 1" and by suffixing all other files with " 2", thus making sure the ".old" entries will follow the non-old entries in reverse-sorted-order. - Call version_reverse_sort on the list (sort -r -V): A single execution of sort(1). For instance, GNU coreutils' sort will reverse-sort the list in O(n*log(n)) with a merge sort. - Replace the " 1" suffixes by ".old", and remove the " 2" suffixes. - Iterate on the reverse-sorted list to output each menu entry item. Therefore, the algorithm proposed has O(n*log(n)) complexity with GNU coreutils' sort compared to the prior O(n^2) complexity. Moreover, the constant time required for each list entry is much less because sorting is done within a single execution of sort(1) rather than requiring O(n^2) executions of sed(1), sort(1), head(1), and grep(1) in sub-shells. Reviewed-by: Robbie Harwood Reviewed-by: Daniel Kiper 2022-07-04 Glenn Washburn docs: Add documentation on detached header option to cryptomount Reviewed-by: Patrick Steinhardt Reviewed-by: Daniel Kiper 2022-07-04 Glenn Washburn cryptodisk: Add support for using detached header files Using the disk read hook mechanism, setup a read hook on the source disk which will read from the given header file during the scan and recovery cryptodisk backend functions. Disk read hooks are executed after the data has been read from the disk. This is okay, because the read hook is given the read buffer before its sent back to the caller. In this case, the hook can then overwrite the data read from the disk device with data from the header file sent in as the read hook data. This is transparent to the read caller. Since the callers of this function have just opened the source disk, there are no current read hooks, so there's no need to save/restore them nor consider if they should be called or not. This hook assumes that the header is at the start of the volume, which is not the case for some formats (e.g. GELI). So GELI will return an error if a detached header is specified. It also can only be used with formats where the detached header file can be written to the first blocks of the volume and the volume could still be unlocked. So the header file can not be formatted differently from the on-disk header. If these assumpts are not met, detached header file processing must be specially handled in the cryptodisk backend module. The hook will be called potentially many times by a backend. This is fine because of the assumptions mentioned and the read hook reads from absolute offsets and is stateless. Also add a --header (short -H) option to cryptomount which takes a file argument. Reviewed-by: Patrick Steinhardt Reviewed-by: Daniel Kiper 2022-07-04 Glenn Washburn disk: Allow read hook callback to take read buffer to potentially modify it It will be desirable in the future to allow having the read hook modify the data passed back from a read function call on a disk or file. This adds that infrastructure and has no impact on code flow for existing uses of the read hook. Also changed is that now when the read hook callback is called it can also indicate what error code should be sent back to the read caller. Reviewed-by: Patrick Steinhardt Reviewed-by: Daniel Kiper 2022-07-04 Glenn Washburn docs: Document undocumented variables Document the variables net__clientid, net__clientuuid, lockdown, and shim_lock in the list of special environment variables. Reviewed-by: Daniel Kiper 2022-07-04 Patrick Steinhardt kern/efi/mm: Implement runtime addition of pages Adjust the interface of grub_efi_mm_add_regions() to take a set of GRUB_MM_ADD_REGION_* flags, which most notably is currently only the GRUB_MM_ADD_REGION_CONSECUTIVE flag. This allows us to set the function up as callback for the memory subsystem and have it call out to us in case there's not enough pages available in the current heap. Reviewed-by: Daniel Kiper Tested-by: Patrick Steinhardt 2022-07-04 Patrick Steinhardt kern/efi/mm: Pass up errors from add_memory_regions() The function add_memory_regions() is currently only called on system initialization to allocate a fixed amount of pages. As such, it didn't need to return any errors: in case it failed, we cannot proceed anyway. This will change with the upcoming support for requesting more memory from the firmware at runtime, where it doesn't make sense anymore to fail hard. Refactor the function to return an error to prepare for this. Note that this does not change the behaviour when initializing the memory system because grub_efi_mm_init() knows to call grub_fatal() in case grub_efi_mm_add_regions() returns an error. Reviewed-by: Daniel Kiper Tested-by: Patrick Steinhardt 2022-07-04 Patrick Steinhardt kern/efi/mm: Extract function to add memory regions In preparation of support for runtime-allocating additional memory region, this patch extracts the function to retrieve the EFI memory map and add a subset of it to GRUB's own memory regions. Reviewed-by: Daniel Kiper Tested-by: Patrick Steinhardt 2022-07-04 Patrick Steinhardt kern/efi/mm: Always request a fixed number of pages on init When initializing the EFI memory subsystem, we will by default request a quarter of the available memory, bounded by a minimum/maximum value. Given that we're about to extend the EFI memory system to dynamically request additional pages from the firmware as required, this scaling of requested memory based on available memory will not make a lot of sense anymore. Remove this logic as a preparatory patch such that we'll instead defer to the runtime memory allocator. Note that ideally, we'd want to change this after dynamic requesting of pages has been implemented for the EFI platform. But because we'll need to split up initialization of the memory subsystem and the request of pages from the firmware, we'd have to duplicate quite some logic at first only to remove it afterwards again. This seems quite pointless, so we instead have patches slightly out of order. Reviewed-by: Daniel Kiper Tested-by: Patrick Steinhardt 2022-07-04 Patrick Steinhardt mm: Allow dynamically requesting additional memory regions Currently, all platforms will set up their heap on initialization of the platform code. While this works mostly fine, it poses some limitations on memory management on us. Most notably, allocating big chunks of memory in the gigabyte range would require us to pre-request this many bytes from the firmware and add it to the heap from the beginning on some platforms like EFI. As this isn't needed for most configurations, it is inefficient and may even negatively impact some usecases when, e.g., chainloading. Nonetheless, allocating big chunks of memory is required sometimes, where one example is the upcoming support for the Argon2 key derival function in LUKS2. In order to avoid pre-allocating big chunks of memory, this commit implements a runtime mechanism to add more pages to the system. When a given allocation cannot be currently satisfied, we'll call a given callback set up by the platform's own memory management subsystem, asking it to add a memory area with at least "n" bytes. If this succeeds, we retry searching for a valid memory region, which should now succeed. If this fails, we try asking for "n" bytes, possibly spread across multiple regions, in hopes that region merging means that we end up with enough memory for things to work out. Tested-by: Stefan Berger Reviewed-by: Daniel Kiper Tested-by: Patrick Steinhardt 2022-07-04 Patrick Steinhardt mm: Drop unused unloading of modules on OOM In grub_memalign(), there's a commented section which would allow for unloading of unneeded modules in case where there is not enough free memory available to satisfy a request. Given that this code is never compiled in, let's remove it together with grub_dl_unload_unneeded(). Reviewed-by: Daniel Kiper Tested-by: Patrick Steinhardt 2022-07-04 Daniel Axtens mm: Debug support for region operations This is handy for debugging. Enable with "set debug=regions". Reviewed-by: Daniel Kiper Tested-by: Patrick Steinhardt 2022-07-04 Daniel Axtens mm: When adding a region, merge with region after as well as before On x86_64-efi (at least) regions seem to be added from top down. The mm code will merge a new region with an existing region that comes immediately before the new region. This allows larger allocations to be satisfied that would otherwise be the case. On powerpc-ieee1275, however, regions are added from bottom up. So if we add 3x 32MB regions, we can still only satisfy a 32MB allocation, rather than the 96MB allocation we might otherwise be able to satisfy. * Define 'post_size' as being bytes lost to the end of an allocation due to being given weird sizes from firmware that are not multiples of GRUB_MM_ALIGN. * Allow merging of regions immediately _after_ existing regions, not just before. As with the other approach, we create an allocated block to represent the new space and the pass it to grub_free() to get the metadata right. Tested-by: Stefan Berger Reviewed-by: Daniel Kiper Tested-by: Patrick Steinhardt 2022-06-29 Daniel Axtens mm: Assert that we preserve header vs region alignment grub_mm_region_init() does: h = (grub_mm_header_t) (r + 1); where h is a grub_mm_header_t and r is a grub_mm_region_t. Cells are supposed to be GRUB_MM_ALIGN aligned, but while grub_mm_dump ensures this vs the region header, grub_mm_region_init() does not. It's better to be explicit than implicit here: rather than changing grub_mm_region_init() to ALIGN_UP(), require that the struct is explicitly a multiple of the header size. Reviewed-by: Daniel Kiper Tested-by: Patrick Steinhardt 2022-06-28 Daniel Axtens tests: Only pass SeaBIOS fw_opt for x86 non-EFI platforms This breaks the tests on pseries - just restrict it to x86 platforms that don't specify an EFI. Reviewed-by: Daniel Kiper 2022-06-07 Darren Kenny fs/btrfs: Fix more fuzz issues related to chunks The corpus was generating issues in grub_btrfs_read_logical() when attempting to iterate over stripe entries in the superblock's bootmapping. In most cases the reason for the failure was that the number of stripes in chunk->nstripes exceeded the possible space statically allocated in superblock bootmapping space. Each stripe entry in the bootmapping block consists of a grub_btrfs_key followed by a grub_btrfs_chunk_stripe. Another issue that came up was that while calculating the chunk size, in an earlier piece of code in that function, depending on the data provided in the btrfs file system, it would end up calculating a size that was too small to contain even 1 grub_btrfs_chunk_item, which is obviously invalid too. Reviewed-by: Daniel Kiper 2022-06-07 Darren Kenny fs/btrfs: Fix more ASAN and SEGV issues found with fuzzing The fuzzer is generating btrfs file systems that have chunks with invalid combinations of stripes and substripes for the given RAID configurations. After examining the Linux kernel fs/btrfs/tree-checker.c code, it appears that sub-stripes should only be applied to RAID10, and in that case there should only ever be 2 of them. Similarly, RAID single should only have 1 stripe, and RAID1/1C3/1C4 should have 2. 3 or 4 stripes respectively, which is what redundancy corresponds. Some of the chunks ended up with a size of 0, which grub_malloc() still returned memory for and in turn generated ASAN errors later when accessed. While it would be possible to specifically limit the number of stripes, a more correct test was on the combination of the chunk item, and the number of stripes by the size of the chunk stripe structure in comparison to the size of the chunk itself. Reviewed-by: Daniel Kiper 2022-06-07 Darren Kenny fs/btrfs: Fix several fuzz issues with invalid dir item sizing According to the btrfs code in Linux, the structure of a directory item leaf should be of the form: |struct btrfs_dir_item|name|data| in GRUB the name len and data len are in the grub_btrfs_dir_item structure's n and m fields respectively. The combined size of the structure, name and data should be less than the allocated memory, a difference to the Linux kernel's struct btrfs_dir_item is that the grub_btrfs_dir_item has an extra field for where the name is stored, so we adjust for that too. Reviewed-by: Daniel Kiper 2022-06-07 Sudhakar Kuppusamy fs/f2fs: Do not copy file names that are too long A corrupt f2fs file system might specify a name length which is greater than the maximum name length supported by the GRUB f2fs driver. We will allocate enough memory to store the overly long name, but there are only F2FS_NAME_LEN bytes in the source, so we would read past the end of the source. While checking directory entries, do not copy a file name with an invalid length. Reviewed-by: Daniel Kiper 2022-06-07 Sudhakar Kuppusamy fs/f2fs: Do not read past the end of nat bitmap A corrupt f2fs filesystem could have a block offset or a bitmap offset that would cause us to read beyond the bounds of the nat bitmap. Introduce the nat_bitmap_size member in grub_f2fs_data which holds the size of nat bitmap. Set the size when loading the nat bitmap in nat_bitmap_ptr(), and catch when an invalid offset would create a pointer past the end of the allocated space. Check against the bitmap size in grub_f2fs_test_bit() test bit to avoid reading past the end of the nat bitmap. Reviewed-by: Daniel Kiper 2022-06-07 Sudhakar Kuppusamy fs/f2fs: Do not read past the end of nat journal entries A corrupt f2fs file system could specify a nat journal entry count that is beyond the maximum NAT_JOURNAL_ENTRIES. Check if the specified nat journal entry count before accessing the array, and throw an error if it is too large. Reviewed-by: Daniel Kiper 2022-06-07 Daniel Axtens net/http: Error out on headers with LF without CR In a similar vein to the previous patch, parse_line() would write a NUL byte past the end of the buffer if there was an HTTP header with a LF rather than a CRLF. RFC-2616 says: Many HTTP/1.1 header field values consist of words separated by LWS or special characters. These special characters MUST be in a quoted string to be used within a parameter value (as defined in section 3.6). We don't support quoted sections or continuation lines, etc. If we see an LF that's not part of a CRLF, bail out. Fixes: CVE-2022-28734 Reviewed-by: Daniel Kiper 2022-06-07 Daniel Axtens net/http: Fix OOB write for split http headers GRUB has special code for handling an http header that is split across two packets. The code tracks the end of line by looking for a "\n" byte. The code for split headers has always advanced the pointer just past the end of the line, whereas the code that handles unsplit headers does not advance the pointer. This extra advance causes the length to be one greater, which breaks an assumption in parse_line(), leading to it writing a NUL byte one byte past the end of the buffer where we reconstruct the line from the two packets. It's conceivable that an attacker controlled set of packets could cause this to zero out the first byte of the "next" pointer of the grub_mm_region structure following the current_line buffer. Do not advance the pointer in the split header case. Fixes: CVE-2022-28734 Reviewed-by: Daniel Kiper 2022-06-07 Daniel Axtens net/http: Do not tear down socket if it's already been torn down It's possible for data->sock to get torn down in tcp error handling. If we unconditionally tear it down again we will end up doing writes to an offset of the NULL pointer when we go to tear it down again. Detect if it has been torn down and don't do it again. Reviewed-by: Daniel Kiper 2022-06-07 Daniel Axtens net/tftp: Avoid a trivial UAF Under tftp errors, we print a tftp error message from the tftp header. However, the tftph pointer is a pointer inside nb, the netbuff. Previously, we were freeing the nb and then dereferencing it. Don't do that, use it and then free it later. This isn't really _bad_ per se, especially as we're single-threaded, but it trips up fuzzers. Reviewed-by: Daniel Kiper 2022-06-07 Daniel Axtens net/tftp: Prevent a UAF and double-free from a failed seek A malicious tftp server can cause UAFs and a double free. An attempt to read from a network file is handled by grub_net_fs_read(). If the read is at an offset other than the current offset, grub_net_seek_real() is invoked. In grub_net_seek_real(), if a backwards seek cannot be satisfied from the currently received packets, and the underlying transport does not provide a seek method, then grub_net_seek_real() will close and reopen the network protocol layer. For tftp, the ->close() call goes to tftp_close() and frees the tftp_data_t file->data. The file->data pointer is not nulled out after the free. If the ->open() call fails, the file->data will not be reallocated and will continue point to a freed memory block. This could happen from a server refusing to send the requisite ack to the new tftp request, for example. The seek and the read will then fail, but the grub_file continues to exist: the failed seek does not necessarily cause the entire file to be thrown away (e.g. where the file is checked to see if it is gzipped/lzio/xz/etc., a read failure is interpreted as a decompressor passing on the file, not as an invalidation of the entire grub_file_t structure). This means subsequent attempts to read or seek the file will use the old file->data after free. Eventually, the file will be close()d again and file->data will be freed again. Mark a net_fs file that doesn't reopen as broken. Do not permit read() or close() on a broken file (seek is not exposed directly to the file API - it is only called as part of read, so this blocks seeks as well). As an additional defence, null out the ->data pointer if tftp_open() fails. That would have lead to a simple null pointer dereference rather than a mess of UAFs. This may affect other protocols, I haven't checked. Reviewed-by: Daniel Kiper 2022-06-07 Daniel Axtens net/dns: Don't read past the end of the string we're checking against I don't really understand what's going on here but fuzzing found a bug where we read past the end of check_with. That's a C string, so use grub_strlen() to make sure we don't overread it. Reviewed-by: Daniel Kiper 2022-06-07 Daniel Axtens net/dns: Fix double-free addresses on corrupt DNS response grub_net_dns_lookup() takes as inputs a pointer to an array of addresses ("addresses") for the given name, and pointer to a number of addresses ("naddresses"). grub_net_dns_lookup() is responsible for allocating "addresses", and the caller is responsible for freeing it if "naddresses" > 0. The DNS recv_hook will sometimes set and free the addresses array, for example if the packet is too short: if (ptr + 10 >= nb->tail) { if (!*data->naddresses) grub_free (*data->addresses); grub_netbuff_free (nb); return GRUB_ERR_NONE; } Later on the nslookup command code unconditionally frees the "addresses" array. Normally this is fine: the array is either populated with valid data or is NULL. But in these sorts of error cases it is neither NULL nor valid and we get a double-free. Only free "addresses" if "naddresses" > 0. It looks like the other use of grub_net_dns_lookup() is not affected. Reviewed-by: Daniel Kiper 2022-06-07 Daniel Axtens net/netbuff: Block overly large netbuff allocs A netbuff shouldn't be too huge. It's bounded by MTU and TCP segment reassembly. If we are asked to create one that is unreasonably big, refuse. This is a hardening measure: if we hit this code, there's a bug somewhere else that we should catch and fix. This commit: - stops the bug propagating any further. - provides a spot to instrument in e.g. fuzzing to try to catch these bugs. I have put instrumentation (e.g. __builtin_trap() to force a crash) here and have not been able to find any more crashes. Reviewed-by: Daniel Kiper 2022-06-07 Daniel Axtens net/ip: Do IP fragment maths safely We can receive packets with invalid IP fragmentation information. This can lead to rsm->total_len underflowing and becoming very large. Then, in grub_netbuff_alloc(), we add to this very large number, which can cause it to overflow and wrap back around to a small positive number. The allocation then succeeds, but the resulting buffer is too small and subsequent operations can write past the end of the buffer. Catch the underflow here. Fixes: CVE-2022-28733 Reviewed-by: Daniel Kiper 2022-06-07 Daniel Axtens normal/charset: Fix array out-of-bounds formatting unicode for display In some cases attempting to display arbitrary binary strings leads to ASAN splats reading the widthspec array out of bounds. Check the index. If it would be out of bounds, return a width of 1. I don't know if that's strictly correct, but we're not really expecting great display of arbitrary binary data, and it's certainly not worse than an OOB read. Reviewed-by: Daniel Kiper 2022-06-07 Daniel Axtens video/readers/jpeg: Block int underflow -> wild pointer write Certain 1 px wide images caused a wild pointer write in grub_jpeg_ycrcb_to_rgb(). This was caused because in grub_jpeg_decode_data(), we have the following loop: for (; data->r1 < nr1 && (!data->dri || rst); data->r1++, data->bitmap_ptr += (vb * data->image_width - hb * nc1) * 3) We did not check if vb * width >= hb * nc1. On a 64-bit platform, if that turns out to be negative, it will underflow, be interpreted as unsigned 64-bit, then be added to the 64-bit pointer, so we see data->bitmap_ptr jump, e.g.: 0x6180_0000_0480 to 0x6181_0000_0498 ^ ~--- carry has occurred and this pointer is now far away from any object. On a 32-bit platform, it will decrement the pointer, creating a pointer that won't crash but will overwrite random data. Catch the underflow and error out. Fixes: CVE-2021-3697 Reviewed-by: Daniel Kiper 2022-06-07 Daniel Axtens video/readers/jpeg: Refuse to handle multiple start of streams An invalid file could contain multiple start of stream blocks, which would cause us to reallocate and leak our bitmap. Refuse to handle multiple start of streams. Additionally, fix a grub_error() call formatting. Reviewed-by: Daniel Kiper 2022-06-07 Daniel Axtens video/readers/jpeg: Do not reallocate a given huff table Fix a memory leak where an invalid file could cause us to reallocate memory for a huffman table we had already allocated memory for. Reviewed-by: Daniel Kiper 2022-06-07 Daniel Axtens video/readers/jpeg: Abort sooner if a read operation fails Fuzzing revealed some inputs that were taking a long time, potentially forever, because they did not bail quickly upon encountering an I/O error. Try to catch I/O errors sooner and bail out. Reviewed-by: Daniel Kiper 2022-06-07 Daniel Axtens video/readers/png: Sanity check some huffman codes ASAN picked up two OOB global reads: we weren't checking if some code values fit within the cplens or cpdext arrays. Check and throw an error if not. Reviewed-by: Daniel Kiper 2022-06-07 Daniel Axtens video/readers/png: Avoid heap OOB R/W inserting huff table items In fuzzing we observed crashes where a code would attempt to be inserted into a huffman table before the start, leading to a set of heap OOB reads and writes as table entries with negative indices were shifted around and the new code written in. Catch the case where we would underflow the array and bail. Fixes: CVE-2021-3696 Reviewed-by: Daniel Kiper 2022-06-07 Daniel Axtens video/readers/png: Drop greyscale support to fix heap out-of-bounds write A 16-bit greyscale PNG without alpha is processed in the following loop: for (i = 0; i < (data->image_width * data->image_height); i++, d1 += 4, d2 += 2) { d1[R3] = d2[1]; d1[G3] = d2[1]; d1[B3] = d2[1]; } The increment of d1 is wrong. d1 is incremented by 4 bytes per iteration, but there are only 3 bytes allocated for storage. This means that image data will overwrite somewhat-attacker-controlled parts of memory - 3 bytes out of every 4 following the end of the image. This has existed since greyscale support was added in 2013 in commit 3ccf16dff98f (grub-core/video/readers/png.c: Support grayscale). Saving starfield.png as a 16-bit greyscale image without alpha in the gimp and attempting to load it causes grub-emu to crash - I don't think this code has ever worked. Delete all PNG greyscale support. Fixes: CVE-2021-3695 Reviewed-by: Daniel Kiper 2022-06-07 Daniel Axtens video/readers/png: Refuse to handle multiple image headers This causes the bitmap to be leaked. Do not permit multiple image headers. Reviewed-by: Daniel Kiper 2022-06-07 Daniel Axtens video/readers/png: Abort sooner if a read operation fails Fuzzing revealed some inputs that were taking a long time, potentially forever, because they did not bail quickly upon encountering an I/O error. Try to catch I/O errors sooner and bail out. Reviewed-by: Daniel Kiper 2022-06-07 Daniel Axtens kern/file: Do not leak device_name on error in grub_file_open() If we have an error in grub_file_open() before we free device_name, we will leak it. Free device_name in the error path and null out the pointer in the good path once we free it there. Reviewed-by: Daniel Kiper 2022-06-07 Julian Andres Klode kern/efi/sb: Reject non-kernel files in the shim_lock verifier We must not allow other verifiers to pass things like the GRUB modules. Instead of maintaining a blocklist, maintain an allowlist of things that we do not care about. This allowlist really should be made reusable, and shared by the lockdown verifier, but this is the minimal patch addressing security concerns where the TPM verifier was able to mark modules as verified (or the OpenPGP verifier for that matter), when it should not do so on shim-powered secure boot systems. Fixes: CVE-2022-28735 Reviewed-by: Daniel Kiper 2022-06-07 Chris Coulson loader/efi/chainloader: Use grub_loader_set_ex() This ports the EFI chainloader to use grub_loader_set_ex() in order to fix a use-after-free bug that occurs when grub_cmd_chainloader() is executed more than once before a boot attempt is performed. Fixes: CVE-2022-28736 Reviewed-by: Daniel Kiper 2022-06-07 Chris Coulson commands/boot: Add API to pass context to loader Loaders rely on global variables for saving context which is consumed in the boot hook and freed in the unload hook. In the case where a loader command is executed twice, calling grub_loader_set() a second time executes the unload hook, but in some cases this runs when the loader's global context has already been updated, resulting in the updated context being freed and potential use-after-free bugs when the boot hook is subsequently called. This adds a new API, grub_loader_set_ex(), which allows a loader to specify context that is passed to its boot and unload hooks. This is an alternative to requiring that loaders call grub_loader_unset() before mutating their global context. Reviewed-by: Daniel Kiper 2022-06-07 Chris Coulson loader/efi/chainloader: Simplify the loader state The chainloader command retains the source buffer and device path passed to LoadImage(), requiring the unload hook passed to grub_loader_set() to free them. It isn't required to retain this state though - they aren't required by StartImage() or anything else in the boot hook, so clean them up before grub_cmd_chainloader() finishes. Reviewed-by: Daniel Kiper 2022-06-07 Jagannathan Raman fs/zfs/zfs: zfs_mount() - avoid pointer downcasting Coverity reports that while loopis in the following functions uses tainted data as boundary: zfs_mount() -> check_mos_features() -> dnode_get() -> zfs_log2() zfs_mount() -> grub_memmove() The defect type is "Untrusted loop bound" caused as a result of "tainted_data_downcast". Coverity does not like the pointer downcast here and we need to address it. We believe Coverity flags pointer downcast for the following two reasons: 1. External data: The pointer downcast could indicate that the source is external data, which we need to further sanitize - such as verifying its limits. In this case, the data is read from an external source, which is a disk. But, zio_read(), which reads the data from the disk, sanitizes it using a checksum. checksum is the best facility that ZFS offers to verify external data, and we don't believe a better way exists. Therefore, no further action is possible for this. 2. Corruption due to alignment: downcasting a pointer from a strict type to less strict type could result in data corruption. For example, the following cast would corrupt because uint32_t is 4-byte aligned, and won't be able to point to 0x1003 which is not 4-byte aligned. uint8_t *ptr = 0x1003; uint32_t *word = ptr; (incorrect, alignment issues) This patch converts the "osp" pointer in zfs_mount() from a "void" type to "objset_phys_t" type to address this issue. We are not sure if there are any other reasons why Coverity flags the downcast. However, the fix for alignment issue masks/suppresses any other issues from showing up. Fixes: CID 314023 Reviewed-by: Darren Kenny Reviewed-by: Daniel Kiper 2022-06-07 Jagannathan Raman fs/zfs/zfs: make_mdn() - avoid pointer downcasting Coverity reports that the while loop in the following function uses tainted data as boundary: fill_fs_info() -> dnode_get() -> zfs_log2() The tainted originated from: fill_fs_info() -> make_mdn() The defect type is "Untrusted loop bound" caused as a result of "tainted_data_downcast". Coverity does not like the pointer downcast here and we need to address it. We believe Coverity flags pointer downcast for the following two reasons: 1. External data: The pointer downcast could indicate that the source is external data, which we need to further sanitize - such as verifying its limits. In this case, the data is read from an external source, which is a disk. But, zio_read(), which reads the data from the disk, sanitizes it using a checksum. checksum is the best facility that ZFS offers to verify external data, and we don't believe a better way exists. Therefore, no further action is possible for this. 2. Corruption due to alignment: downcasting a pointer from a strict type to less strict type could result in data corruption. For example, the following cast would corrupt because uint32_t is 4-byte aligned, and won't be able to point to 0x1003 which is not 4-byte aligned. uint8_t *ptr = 0x1003; uint32_t *word = ptr; (incorrect, alignment issues) This patch converts the "osp" pointer in make_mdn() from a "void" type to "objset_phys_t" type to address the issue. We are not sure if there are any other reasons why Coverity flags the downcast. However, the fix for alignment issue masks/suppresses any other issues from showing up. Fixes: CID 314020 Reviewed-by: Darren Kenny Reviewed-by: Daniel Kiper 2022-06-07 Alec Brown util/grub-module-verifierXX: Add e_shoff check in get_shdr() In util/grub-module-verifierXX.c, the function get_shdr() is used to obtain the section header at a given index but isn't checking that there is an offset for the section header table. To validate that there is, we can check that e_shoff isn't 0. Reviewed-by: Daniel Kiper Reviewed-by: Darren Kenny 2022-06-07 Alec Brown grub-core/loader/i386/bsdXX: Avoid downcasting (char *) to (Elf_Shdr *) In bsdXX.c, a couple of untrusted loop bound and untrusted allocation size bugs were flagged by Coverity in the functions grub_openbsd_find_ramdisk() and grub_freebsd_load_elfmodule(). These bugs were flagged by coverity because the variable shdr was downcasting from a char pointer to an Elf_Shdr pointer whenever it was used to set the base value in for loops. To avoid this, we need to set shdr as an Elf_Shdr pointer where it is initialized. In the function read_headers(), the function is reading elf section header data from a file and passing it to the variable shdr as data for a char pointer. If we switch the type of shdr to an Elf_Shdr pointer in read_headers() as well as other functions, then we won't need to downcast to an Elf_Shdr pointer. By doing this, the issue becomes masked from Coverity's view. In the following patches, we check limits to ensure the data isn't tainted. Also, switched use of (char *) to (grub_uint8_t *) to give a better indication of pointer arithmetic and not suggest use of a C string. Fixes: CID 314018 Fixes: CID 314030 Fixes: CID 314031 Fixes: CID 314039 Reviewed-by: Daniel Kiper Reviewed-by: Darren Kenny 2022-06-07 Stefan Agner disk/efi/efidisk: Pass buffers with higher alignment Some devices report IoAlign values but seem to require buffers with higher alignment. The UEFI specification is saying: "IoAlign values of 0 and 1 mean that the buffer can be placed anywhere in memory. Otherwise, IoAlign must be a power of 2, and the requirement is that the start address of a buffer must be evenly divisible by IoAlign with no remainder." Some devices report IoAlign of 2, however seem to require 4 bytes aligned buffers. It seems that this got misinterpreted by some vendors assuming IoAlign is 2^IoAlign. There is also such a hint in an example in earlier versions of the Driver Writer's Guide: ScsiPassThruMode.IoAlign = 2; // Data must be alligned on 4-byte boundary Some devices report no alignment requirements at all but seem to read corrupted data or report read errors when passing unaligned buffers. Work around by using an alignment of at least BlockSize (typically 512 bytes) in any case. If IoAlign (interpreted as per UEFI specification) requests a higher alignment than BlockSize, follow IoAlign still. Note: The problem has only noticed with compressed squashfs. It seems that ext4 (and presumably other file system drivers) pass buffers with a higher alignment already. Acked-by: Heinrich Schuchardt Reviewed-by: Daniel Kiper 2022-06-07 Samuel Thibault osdep/hurd/getroot: Use "part:" qualifier When using userland drivers such as rumpdisk, we'd rather make ext2fs use parted-based libstore partitioning support. That can be used for kernelland drivers as well, so we can just make GRUB always use the "part:" qualifier to switch ext2fs to it. grub_util_find_hurd_root_device() then has to understand this syntax and translate it into the /dev/ entry name. Reviewed-by: Daniel Kiper 2022-06-07 Glenn Washburn docs: Add documentation on keyfile option to cryptomount Reviewed-by: Daniel Kiper disk/cryptodisk: Use enum constants as indexes into cryptomount option array Reviewed-by: Daniel Kiper 2022-06-07 John Lane disk/cryptodisk: Add options to cryptomount to support keyfiles Add the options --key-file, --keyfile-offset, and --keyfile-size to cryptomount and code to put read the requested key file data and pass via the cargs struct. Note, key file data is for all intents and purposes equivalent to a password given to cryptomount. So there is no need to enable support for key files in the various crypto backends (e.g. LUKS1) because the key data is passed just as if it were a password. Reviewed-by: Daniel Kiper 2022-06-07 Denis 'GNUtoo' Carikli disk/geli: Unify grub_cryptodisk_dev function names Reviewed-by: Patrick Steinhardt Reviewed-by: Daniel Kiper disk/luks: Unify grub_cryptodisk_dev function names Reviewed-by: Patrick Steinhardt Reviewed-by: Daniel Kiper 2022-06-07 Glenn Washburn util/probe: Remove unused header includes Reviewed-by: Daniel Kiper commands/macbless: Remove whitespace between N_ macro and open parenthesis Reviewed-by: Daniel Kiper 2022-06-07 Glenn Washburn tests: Add /sbin and /usr/sbin to path in partmap test The partmap test requires no elevated privileges. However, it uses parted which can be used as a normal user, but is usually located in /sbin or /usr/bin (eg. on Debian systems). Whereas the normal user does not usually have /sbin or /usr/sbin added to their path, thus parted will not be found causing the test to abort. Add /sbin and /usr/sbin to the path for the partmap test so that the test can run successfully as an unprivileged user. Reviewed-by: Daniel Kiper 2022-06-07 Glenn Washburn tests: Show host determined fs UUID when hfs UUID test fails On failure, the hfs test should show both the host and GRUB determined fs UUID. Prior to this change, both outputs where generated by GRUB, which is less helpful in determining the cause of failure. Reviewed-by: Daniel Kiper 2022-05-24 Glenn Washburn docs: Add section for general undocumented commands The section is an itemized list of commands that are not listed else where in the command sections. Reviewed-by: Daniel Kiper 2022-05-24 Glenn Washburn docs: Add under documented loader commands to beginning of loader section Reviewed-by: Daniel Kiper 2022-05-24 Glenn Washburn docs: Create command section for loader commands Move loader commands documented in the general commands list into the loader command section. Reviewed-by: Daniel Kiper 2022-05-24 Glenn Washburn docs: Markup loader commands with @command tag Also, add period to terminate sentence. Reviewed-by: Daniel Kiper 2022-05-24 Glenn Washburn docs: Make note of i386-pc specific usage of halt command The --no-apm option is only available on the i396-pc target. Reviewed-by: Daniel Kiper 2022-05-24 Glenn Washburn docs: Make note that sendkey is only available on i386-pc Reviewed-by: Daniel Kiper docs: Fix spelling typo and remove unnecessary spaces Reviewed-by: Daniel Kiper 2022-05-24 Glenn Washburn net/net: Fix incorrect condition for calling grub_net_tcp_retransmit() The commit 848724273e4 (net/net: Avoid unnecessary calls to grub_net_tcp_retransmit()) needs to have its condition inverted to avoid unnecessary calls to grub_net_tcp_retransmit(). As it is, it creates many unnecessary calls and does not call grub_net_tcp_retransmit() when needed. The call to grub_net_tcp_retransmit() should only be made when grub_net_cards does _not_ equal NULL, meaning that there are potentially network cards that need TCP retransmission. Fixes: 848724273e4 (net/net: Avoid unnecessary calls to grub_net_tcp_retransmit()) Reviewed-by: Daniel Kiper 2022-05-24 Oskari Pirhonen templates: Improve initramfs detection Add detection for initramfs of the form *.img.old. For example, Gentoo's sys-kernel/genkernel installs it as initramfs-*.img and moves any existing one to initramfs-*.img.old. Apply the same scheme to initrd-*.img and initrd-*.gz files for consistency. Reviewed-by: Daniel Kiper 2022-05-24 Samuel Thibault osdep/hurd: Support device entries with @/dev/disk: qualifier Those are used with non-bootstrap disk drivers, for which libstore has to open /dev/disk before calling device_open on it instead of on the device master port. Normally in that case all /dev/ entries also have the @/dev/disk: qualifier, so we can just drop it. Reviewed-by: Daniel Kiper 2022-05-24 Darren Kenny grub-mkimage: Creating aarch64 images from x86 host is broken A recent fix that made appears to have broken the ability to create an aarch64 boot image on a x86-based host. This was due to an overzealous testing of the architecture when building grub-mkimage and removing the code that build an ARM image when not built on ARM. On the occasion remove redundant break. Fixes: 8541f319 (grub-mkimage: Only check aarch64 relocations when built for aarch64) Tested-by: Selva Ganesan Reviewed-by: Daniel Kiper 2022-05-24 Icenowy Zheng grub-install: Allow to install to non-EFI ESP when --force Although the EFI specification enforces support for FAT ESP, it's free for EFI implementations to implement support for ESPs with other formats (e.g. ext4, ntfs, etc), and at least U-Boot EFI will support ext4 ESP if U-Boot is built with ext4 support. In some situations a GRUB installation on such a non-FAT ESP could be useful (e.g. a NTFS-based USB disk that can dual boot a Windows installation media and a Linux LiveCD). As this is advanced and implementation-dependent behavior, let grub-install allow this kind of installation, but only when --force is specified. Reviewed-by: Daniel Kiper 2022-04-26 Qiumiao Zhang net: Fix NULL pointer dereference when parsing ICMP6_ROUTER_ADVERTISE messages During UEFI PXE boot in IPv6 network, if the DHCP server adopts stateful automatic configuration, then the client receives a ICMP6_ROUTER_ADVERTISE multicast message from the server. This may be received without the interface having a configured network address, so orig_inf will be NULL, which can lead to a NULL dereference when creating the default route. Actually, in this case, the client obtains the default route through DHCPv6 instead of RA messages. So if orig_inf == NULL and route_inf == NULL, we should not set the default route. Fixes: https://savannah.gnu.org/bugs/?62072 Reviewed-by: Daniel Kiper 2022-04-26 Glenn Washburn tests: Ensure that loopback devices and zfs devices are cleaned up ZFS file systems are not unmounted using umount, but instead by exporting them. So export the ZFS file system that has the same label as the one that was created during the test, if such one exists. This is required to delete the loopback device that uses the ZFS image file. Otherwise the added code to delete all loopback devices setup during the test run will never be able to finish because the loopback device can not be deleted while in use. Reviewed-by: Daniel Kiper 2022-04-26 Glenn Washburn tests: Ensure that mountpoints are unmounted before exiting When all tests complete successfully, filesystems mounted by grub-fs-tester will be unmounted before exiting. However, on certain test failures the tester will exit with a failure code and not unmount previously mounted filesystems. Now keep track of mounts and umounts and run an exit handler on exit or process interruption that will umount all mounts that haven't already been unmounted. Reviewed-by: Daniel Kiper 2022-04-20 Glenn Washburn docs: Use correct list format Using "*" to prefix list items leads to undesirable display output for at least the generation of the html documentation. Use the @itemize and @item directives to get itemized list output. Also fix some wording and punctuation issues. Reviewed-by: Daniel Kiper 2022-04-20 Glenn Washburn docs: Clarify meaning of "list" and "cond" for "if" and "while" commands respectively It is not clear from the documentation what a "list" is in the context of the "if" command. Note that its a list of simple commands separated by a ";" and that only the exit status of the last command matters. The same is true for the "cond" parameter to the "while" command. Reviewed-by: Daniel Kiper 2022-04-20 Glenn Washburn docs: Add note that drivemap is only available on i386-pc Reviewed-by: Daniel Kiper 2022-04-20 Glenn Washburn tests: Give grub-fs-tester temp directory a better name Instead of "tmp" the name is prefixed by the name of the scripts (e.g. grub-fs-tester). A timestamp is added in the name to allow for easily seeing a chronological sorting of runs and the name of the filesystem being tested. The random component is set to the minimal possible, 3 characters, because the timestamp should provide enough uniqueness. Reviewed-by: Daniel Kiper 2022-04-20 Glenn Washburn tests: Disable blkid cache usage Using the blkid cache can cause issues when running many file system tests in parallel. We do not need it, as its only there to improve performance, and using the cache does not provide significant performance improvements. Reviewed-by: Daniel Kiper 2022-04-20 Glenn Washburn configure: Fix default -O2 being added when CFLAGS not set Autoconf will set a default CFLAGS of "-g -O2" if CFLAGS is not set. CFLAGS was defaulted to "" early in configure to prevent this. A recent commit ad9ccf660 (configure: Fix various new autotools warnings) added AC_USE_SYSTEM_EXTENSIONS, which pulls in the autoconf CFLAGS check, before we default CFLAGS and thus setting the autoconf default for CFLAGS. Move the default setting of CFLAGS to before AC_USE_SYSTEM_EXTENSIONS so that autoconf will see CFLAGS as set and not give it a default. CFLAGS is also moved above AC_CONFIG_AUX_DIR, because CFLAGS should be defaulted to "" as soon as possible to catch any autoconf macros that try to use some other default. Regardless, this currently has no effect as that macro does not consider the CFLAGS variable. Reviewed-by: Robbie Harwood Reviewed-by: Daniel Kiper 2022-04-20 Darren Kenny video/readers/jpeg: Fix possible invalid loop boundary condition The value of next_marker is adjusted based on the word sized value read from data->file. The updated next_marker value should reference a location in the file just beyond the huffman table, and as such should not have a value larger than the size of the file. Fixes: CID 73657 Reviewed-by: Daniel Kiper 2022-04-20 Michael Chang lib/reed_solomon: Fix array subscript 0 is outside array bounds The grub_absolute_pointer() is a compound expression that can only work within a function. We are out of luck here when the pointer variables require global definition due to ATTRIBUTE_TEXT that have to use fully initialized global definition because of the way linkers work. static gf_single_t * const gf_powx ATTRIBUTE_TEXT = (void *) 0x100000; For the reason given above, use GCC diagnostic pragmas to suppress the array-bounds warning. Reviewed-by: Daniel Kiper 2022-04-20 Michael Chang build: Fix -Werror=array-bounds array subscript 0 is outside array bounds The GRUB is failing to build with GCC-12 in many places like this: In function 'init_cbfsdisk', inlined from 'grub_mod_init' at ../../grub-core/fs/cbfs.c:391:3: ../../grub-core/fs/cbfs.c:345:7: error: array subscript 0 is outside array bounds of 'grub_uint32_t[0]' {aka 'unsigned int[]'} [-Werror=array-bounds] 345 | ptr = *(grub_uint32_t *) 0xfffffffc; | ~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This is caused by GCC regression in 11/12 [1]. In a nut shell, the warning is about detected invalid accesses at non-zero offsets to NULL pointers. Since hardwired constant address is treated as NULL plus an offset in the same underlying code, the warning is therefore triggered. Instead of inserting #pragma all over the places where literal pointers are accessed to avoid diagnosing array-bounds, we can try to borrow the idea from Linux kernel that the absolute_pointer() macro [2][3] is used to disconnect a pointer using literal address from it's original object, hence GCC won't be able to make assumptions on the boundary while doing pointer arithmetic. With that we can greatly reduce the code we have to cover up by making initial literal pointer assignment to use the new wrapper but not having to track everywhere literal pointers are accessed. This also makes code looks cleaner. Please note the grub_absolute_pointer() macro requires to be invoked in a function as long as it is compound expression. Some global variables with literal pointers has been changed to local ones in order to use grub_absolute_pointer() to initialize it. The shuffling is basically done in a selective and careful way that the variable's scope doesn't matter being local or global, for example, the global variable must not get modified at run time throughout. For the record, here's the list of global variables got shuffled in this patch: grub-core/commands/i386/pc/drivemap.c:int13slot grub-core/term/i386/pc/console.c:bios_data_area grub-core/term/ns8250.c:serial_hw_io_addr [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99578 [2] https://elixir.bootlin.com/linux/v5.16.14/source/include/linux/compiler.h#L180 [3] https://elixir.bootlin.com/linux/v5.16.14/source/include/linux/compiler-gcc.h#L31 Reviewed-by: Daniel Kiper 2022-04-20 Michael Chang util/mkimage: Fix dangling pointer may be used error The warning is real as long as dangling pointer to tmp_ may be used if o32 and o64 are both NULL. However that is not going to happen and can be ignored safely because the PE_OHDR is being used in a context that either o32 or o64 must have been properly initialized. Sadly compiler seems not to always optimize that unused tmp_ away so explicit suppression remain needed here. ../util/mkimage.c: In function 'grub_install_generate_image': ../util/mkimage.c:1422:41: error: dangling pointer to 'tmp_' may be used [-Werror=dangling-pointer=] 1422 | PE_OHDR (o32, o64, header_size) = grub_host_to_target32 (header_size); ../util/mkimage.c:857:28: note: 'tmp_' declared here 857 | __typeof__((o64)->field) tmp_; \ | ^~~~ Reviewed-by: Daniel Kiper 2022-04-20 Chad Kimes net/drivers/efi/efinet: Configure VLAN from UEFI device used for PXE This patch handles automatic configuration of VLAN when booting from PXE on UEFI hardware. Reviewed-by: Daniel Kiper 2022-04-20 Chad Kimes kern/efi/efi: Print VLAN info in EFI device path Reviewed-by: Daniel Kiper 2022-04-20 Chad Kimes net/net: Add net_set_vlan command Previously there was no way to set the 802.1Q VLAN identifier, despite support for vlantag in the net module. The only location vlantag was being populated was from PXE boot and only for Open Firmware hardware. This commit allows users to manually configure VLAN information for any interface. Example usage: grub> net_ls_addr efinet1 00:11:22:33:44:55 192.0.2.100 grub> net_set_vlan efinet1 100 grub> net_ls_addr efinet1 00:11:22:33:44:55 192.0.2.100 vlan100 grub> net_set_vlan efinet1 0 efinet1 00:11:22:33:44:55 192.0.2.100 Reviewed-by: Daniel Kiper 2022-04-20 Chad Kimes net/net: Add vlan information to net_ls_addr output Example output: grub> net_ls_addr efinet1 00:11:22:33:44:55 192.0.2.100 vlan100 Reviewed-by: Daniel Kiper 2022-04-04 Chris Coulson kern/efi/init: Log a console error during a stack check failure The initial implementation of the stack protector just busy looped in __stack_chk_fail in order to reduce the amount of code being executed after the stack has been compromised because of a lack of firmware memory protections. With future firmware implementations incorporating memory protections such as W^X, call in to boot services when an error occurs in order to log a message to the console before automatically rebooting the machine. Reviewed-by: Daniel Kiper 2022-04-04 Alec Brown loader/i386/xnu: Fix uninitialized scalar variable In the function grub_xnu_boot(), struct grub_relocator32_state state is called but isn't being initialized. This results in the members grub_uint32_t ebx, grub_uint32_t ecx, grub_uint32_t edx, grub_uint32_t edi, and grub_uint32_t esi being filled with junk data from the stack since none of them are being set to any values. We can prevent this by setting state to {0}. Fixes: CID 375035 Reviewed-by: Darren Kenny Reviewed-by: Daniel Kiper 2022-04-04 Alec Brown loader/i386/xnu: Fix uninitialized scalar variable In the function grub_xnu_boot_resume(), struct grub_relocator32_state state is called but isn't being initialized. This results in the members grub_uint32_t ebx, grub_uint32_t ecx, grub_uint32_t edx, grub_uint32_t esi, and grub_uint32_t edi being filled with junk data from the stack since none of them are being set to any values. We can prevent this by setting state to {0}. Fixes: CID 375031 Reviewed-by: Darren Kenny Reviewed-by: Daniel Kiper 2022-04-04 Alec Brown loader/i386/pc/linux: Fix uninitialized scalar variable In the function grub_linux16_boot(), struct grub_relocator16_state state is called but isn't being initialized. This results in the members grub_uint32_t ebx, grub_uint32_t edx, grub_uint32_t esi, and grub_uint32_t ebp being filled with junk data from the stack since none of them are being set to any values. We can prevent this by setting state to {0}. Fixes: CID 375028 Reviewed-by: Darren Kenny Reviewed-by: Daniel Kiper 2022-04-04 Alec Brown loader/i386/bsd: Fix uninitialized scalar variable In the function grub_netbsd_setup_video(), struct grub_netbsd_btinfo_framebuf params is called but isn't being initialized. The member grub_uint8_t reserved[16] isn't set to any values and is instead filled with junk data from the stack. We can prevent this by setting params to {0}. Fixes: CID 375026 Reviewed-by: Darren Kenny Reviewed-by: Daniel Kiper 2022-04-04 Alec Brown net/net: Fix uninitialized scalar variable In the function grub_net_ipv6_get_link_local(), grub_net_network_level_address_t addr is called but isn't being initialized. This results in the member grub_dns_option_t option being filled with junk data from the stack. We can prevent this by setting the option member in addr to 0. Fixes: CID 375033 Reviewed-by: Darren Kenny Reviewed-by: Daniel Kiper 2022-04-04 Alec Brown net/bootp: Fix uninitialized scalar variable In the function grub_net_configure_by_dhcp_ack(), grub_net_network_level_address_t addr is called but isn't being initialized. This results in the member grub_dns_option_t option being filled with junk data from the stack. To prevent this, we can set the option member in addr to 0. Fixes: CID 375036 Reviewed-by: Darren Kenny Reviewed-by: Daniel Kiper 2022-04-04 Alec Brown net/arp: Fix uninitialized scalar variable In the function grub_net_arp_receive(), grub_net_network_level_address_t sender_addr and target_addr are being called but aren't being initialized. In both of these structs, each member is being set to a value except for grub_dns_option_t option. This results in this member being filled with junk data from the stack. To prevent this, we can set the option member in both structs to 0. Fixes: CID 375030 Reviewed-by: Darren Kenny Reviewed-by: Daniel Kiper 2022-04-04 Glenn Washburn net/tcp: Only call grub_get_time_ms() when there are sockets to potentially retransmit for If the machine has network cards found, but there are no tcp open sockets (because the user doesn't use the network to boot), then grub_net_tcp_retransmit() should be a noop. Thus GRUB doesn't need to call grub_get_time_ms(), which does a call into firmware on powerpc-ieee1275, and probably other targets. So only call grub_get_time_ms() if there are tcp sockets. Aside from improving performance, its also useful to stay out of the firmware as much as possible when debugging via QEMU because its a pain to get back in to GRUB execution. grub_net_tcp_retransmit() can get called very frequently via grub_net_poll_cards_idle() when GRUB is waiting for a keypress (grub_getkey_noblock() calls grub_net_poll_cards_idle()). This can be annoying when debugging an issue in GRUB on PowerPC in QEMU with GDB when GRUB is waiting for a keypress because interrupting via GDB nearly always lands in the OpenBIOS firmware's milliseconds call. Reviewed-by: Daniel Kiper 2022-04-04 Glenn Washburn net/net: Avoid unnecessary calls to grub_net_tcp_retransmit() In grub_net_poll_cards_idle_real(), only call grub_net_tcp_retransmit() if there are network cards found. If there are no network card found, there can be no tcp sockets to transmit on. So no need to go through that logic. Reviewed-by: Daniel Kiper 2022-04-04 Glenn Washburn net/net: Unset grub_net_poll_cards_idle when net module has been unloaded This looks like it was a copy/paste error. If the net module is unloaded, grub_net_poll_cards_idle should be NULL so that GRUB does not try to call a function which now doesn't exist. Reviewed-by: Daniel Kiper 2022-04-04 Glenn Washburn INSTALL: Add information on using --build when cross-compiling The autoconf 2.65 manual [1] strongly recommends specifying the --build option when the --host is used. Add this to the example and add a note that this is recommended. [1] https://www.gnu.org/software/autoconf/manual/autoconf-2.65/html_node/Hosts-and-Cross_002dCompilation.html Reviewed-by: Daniel Kiper 2022-04-04 Glenn Washburn configure: Whitespace changes to improve readability Reviewed-by: Daniel Kiper 2022-04-04 Glenn Washburn configure: Remove unused CFLAGS definitions These CFLAGS definitions are reset below them before they have a change to affect anything. The exception is the *-emu case, which is put in the next if block, which is the only place its used before getting reset. Reviewed-by: Daniel Kiper 2022-04-04 Glenn Washburn configure: Remove dead code It appears as though the intent of this code is to define abort() and main() symbols for some configure tests. However, it never gets used because the if is only entered when not building for *-emu, but the next if block only runs when building for *-emu. And the if block after that unconditionally resets CFLAGS. So this code can have no effect. Additionally, s/aclocal.m4/acinclude.m4/ and move grub_ASM_USCORE to put with other marcos defined in acinclude.m4. Reviewed-by: Daniel Kiper 2022-04-04 Glenn Washburn configure: Sort AM_CONDITIONALs alphabetically Reviewed-by: Daniel Kiper 2022-04-04 Glenn Washburn configure: Allow HOST_CC to override CC According to the INSTALL, "The HOST_* variables override not prefixed variables". This change makes it so, instead of previous behavior, which was to ignore the HOST_CC environment variable. Reviewed-by: Daniel Kiper 2022-04-04 Glenn Washburn gdb: Add malloc and free symbols to kernel.exec to improve gdb functionality Add linker flags when linking kernel.exec to have malloc and free point to grub_malloc() and grub_free() respectively. Some gdb functionality depends on gdb locating the symbols "malloc" and "free", such as dynamically creating strings for arguments to injected function calls. A trivial example would the gdb command 'p strlen("astring")'. Make sure not to do this on emu platforms, or an infinite loop occurs because emu has a special grub_malloc() that calls malloc(). Reviewed-by: Daniel Kiper 2022-04-04 Renaud Métrich commands/search: Add new --efidisk-only option for EFI systems When using "search" on EFI systems, we sometimes want to exclude devices that are not EFI disks, e.g. md, lvm. This is typically used when wanting to chainload when having a software raid (md) for EFI partition: with no option, "search --file /EFI/redhat/shimx64.efi" sets root envvar to "md/boot_efi" which cannot be used for chainloading since there is no effective EFI device behind. Reviewed-by: Daniel Kiper 2022-04-04 Renaud Métrich commands/search: Refactor --no-floppy option to have something generic Reviewed-by: Daniel Kiper 2022-04-04 Hans de Goede kern/main: Suppress the "Welcome to GRUB!" message in EFI builds GRUB EFI builds are now often used in combination with flicker-free boot, but this breaks with upstream GRUB because the "Welcome to GRUB!" message will kick the EFI fb into text mode and show the msg, breaking the flicker-free experience. EFI systems are so fast, that when the menu or the countdown are enabled the message will be immediately overwritten, so in these cases not printing the message does not matter. And in case when the timeout_style is set to TIMEOUT_STYLE_HIDDEN, the user has asked GRUB to be quiet (for example to allow flickfree boot) and thus the message should not be printed. Reviewed-by: Robbie Harwood Reviewed-by: Daniel Kiper 2022-04-04 Hans de Goede normal/menu: Don't show "Booting `%s'" msg when auto-booting with TIMEOUT_STYLE_HIDDEN When the user has asked the menu code to be hidden/quiet and the current entry is being autobooted because the timeout has expired don't show the "Booting `%s'" msg. This is necessary to let flicker-free boots really be flicker free, otherwise the "Booting `%s'" msg will kick the EFI fb into text mode and show the msg, breaking the flicker-free experience. Reviewed-by: Robbie Harwood Reviewed-by: Daniel Kiper 2022-03-21 Hans de Goede term/efi/console: Do not set cursor until the first text output To allow flickerfree boot the EFI console code does not call grub_efi_set_text_mode(1) until some text is actually output. Depending on if the output text is because of an error loading, e.g. the .cfg file, or because of showing the menu the cursor needs to be on or off when the first text is shown. So far the cursor was hardcoded to being on, but this is causing drawing artifacts + slow drawing of the menu as reported here: https://bugzilla.redhat.com/show_bug.cgi?id=1946969 Handle the cursorstate in the same way as the colorstate to fix this, when no text has been output yet, just cache the cursorstate and then use the last set value when the first text is output. Fixes: 2d7c3abd871f (efi/console: Do not set text-mode until we actually need it) Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1946969 Reviewed-by: Daniel Kiper 2022-03-21 Hans de Goede term/efi/console: Do not set colorstate until the first text output GRUB_MOD_INIT(normal) does an unconditional: grub_env_set ("color_normal", "light-gray/black"); which triggers a grub_term_setcolorstate() call. The original version of the "efi/console: Do not set text-mode until we actually need it" patch, https://lists.gnu.org/archive/html/grub-devel/2018-03/msg00125.html, protected against this by caching the requested state in grub_console_setcolorstate() and then only applying it when the first text output actually happens. During refactoring to move the grub_console_setcolorstate() up higher in the grub-core/term/efi/console.c file the code to cache the color-state + bail early was accidentally dropped. Restore the cache the color-state + bail early behavior from the original. Fixes: 2d7c3abd871f (efi/console: Do not set text-mode until we actually need it) Cc: Javier Martinez Canillas Reviewed-by: Daniel Kiper 2022-03-21 Darren Kenny kern/rescue_parser: Ensure that parser allocated memory is not leaked While it would appear unlikely that the memory allocated in *argv in grub_parser_split_cmdline() would be leaked, we should try ensure that it doesn't leak by calling grub_free() before we return from grub_rescue_parse_line(). To avoid a possible double-free, grub_parser_split_cmdline() is being changed to assign *argv = NULL when we've called grub_free() in the fail section. Fixes: CID 96680 Reviewed-by: Daniel Kiper 2022-03-21 Darren Kenny grub-mkimage: Only check aarch64 relocations when built for aarch64 Coverity flagged the switch checks for R_AARCH64_* as being logically dead code, since it could never happen on x86 due to the masking of the values earlier in the code. A check for building on __arm__ (which gcc and clang define) and for MKIMAGE_ELF64 (which GRUB defines) has been added to avoid this dead code being built in. Fixes: CID 158599 Reviewed-by: Daniel Kiper 2022-03-21 Daniel Kiper lib/posix_wrap/errno.h: Add __set_errno() macro $ ./configure --target=x86_64-w64-mingw32 --with-platform=efi --host=x86_64-w64-mingw32 $ make [...] cat syminfo.lst | sort | gawk -f ./genmoddep.awk > moddep.lst || (rm -f moddep.lst; exit 1) __imp__errno in regexp is not defined This happens because grub-core/lib/gnulib/malloc/dynarray_resize.c and grub-core/lib/gnulib/malloc/dynarray_emplace_enlarge.c (both are used by regexp module) from the latest Gnulib call __set_errno() which originally sets errno variable (Windows builds add __imp__ prefix). Of course it is not defined and grub_errno should be used instead. Reviewed-by: Daniel Kiper 2022-03-21 Robbie Harwood configure: Fix various new autotools warnings Reviewed-by: Daniel Kiper 2022-03-21 Robbie Harwood gnulib: Handle warnings introduced by updated gnulib - Fix type of size variable in luks2_verify_key() - Avoid redefinition of SIZE_MAX and ATTRIBUTE_ERROR - Work around gnulib's int types on older compilers Reviewed-by: Daniel Kiper 2022-03-21 Robbie Harwood gnulib: Update gnulib version and drop most gnulib patches In addition to the changes carried in our gnulib patches, several Coverity and code hygiene fixes that were previously downstream are also included in this 3-year gnulib increment. Unfortunately, fix-width.patch is retained. Bump minimum autoconf version from 2.63 to 2.64 and automake from 1.11 to 1.14, as required by gnulib. Sync bootstrap script itself with gnulib. Update regexp module for new dynarray dependency. Reviewed-by: Daniel Kiper 2022-03-21 Robbie Harwood gnulib: Drop no-abort.patch Originally added in commit db7337a3d (grub-core/lib/posix_wrap/stdlib.h (abort): Removed), this patched out all relevant invocations of abort() in gnulib. While it was not documented why at the time, testing suggests that there's no abort() implementation available for gnulib to use. gnulib's position is that the use of abort() is correct here, since it happens when input violates a "shall" from POSIX. Additionally, the code in question is probably not reachable. Since abort() is more friendly to user-space, they prefer to make no change, so we can just carry a define instead (suggested by Paul Eggert). Reviewed-by: Daniel Kiper 2022-03-21 Robbie Harwood gnulib: Drop fix-base64.patch Originally added in commit 9fbdec2f (bootstrap: Add gnulib's base64 module) and subsequently modified in commit 552c9fd08 (gnulib: Fix build of base64 when compiling with memory debugging), fix-base64.patch handled two problems we have using gnulib, which are exercised by the base64 module but not directly caused by it. First, GRUB defines its own bool type, while gnulib expects the equivalent of stdbool.h to be present. Rather than patching gnulib, instead use gnulib's stdbool module to provide a bool type if needed (suggested by Simon Josefsson). Second, our config.h doesn't always inherit config-util.h, which is where gnulib-related options like _GL_ATTRIBUTE_CONST end up. fix-base64.h worked around this by defining the attribute away, but this workaround is better placed in config.h itself, not a gnulib patch. Reviewed-by: Daniel Kiper 2022-03-21 Robbie Harwood config: Where present, ensure config-util.h precedes config.h gnulib defines go in config-util.h, and we need to know whether to provide duplicates in config.h or not. Reviewed-by: Daniel Kiper 2022-03-21 Robbie Harwood config.h.in: Use visual indentation Reviewed-by: Daniel Kiper 2022-03-14 Robbie Harwood INSTALL: Drop mention of libusb The commit 9d25b0da9 (Remove emu libusb support.) dropped use of libusb, but did not remove mention of it from INSTALL file. Reviewed-by: Daniel Kiper 2022-03-14 Daniel Kiper INSTALL: Add more cross-compiling Debian packages The mingw-w64-tools is especially important because with out it some Windows builds may fail due to lack of proper pkg-config. Reviewed-by: Robbie Harwood 2022-03-14 Daniel Kiper configure: Drop ${grub_coredir} unneeded references These are probably stray references left after earlier removals. Reviewed-by: Robbie Harwood 2022-03-14 Daniel Kiper conf/i386-cygwin-img-ld: Do not discard .data and .edata sections $ ./configure --target=i686-w64-mingw32 --with-platform=efi --host=i686-w64-mingw32 [...] checking if __bss_start is defined by the compiler... no checking if edata is defined by the compiler... no checking if _edata is defined by the compiler... no configure: error: none of __bss_start, edata or _edata is defined This happens on machines with quite recent ld due to an error: `edata' referenced in section `.text' of /tmp/cc72w9E4.o: defined in discarded section `.data' of conftest.exe collect2: error: ld returned 1 exit status So, we have to tell linker to not discard .data and .edata sections. The trick comes from ld documentation: 3.6.7 Output Section Discarding The linker will not normally create output sections with no contents. This is for convenience when referring to input sections that may or may not be present in any of the input files. For example: .foo : { *(.foo) } will only create a ‘.foo’ section in the output file if there is a ‘.foo’ section in at least one input file, and if the input sections are not all empty. Other link script directives that allocate space in an output section will also create the output section. So too will assignments to dot even if the assignment does not create space, except for ‘. = 0’, ‘. = . + 0’, ‘. = sym’, ‘. = . + sym’ and ‘. = ALIGN (. != 0, expr, 1)’ when ‘sym’ is an absolute symbol of value 0 defined in the script. This allows you to force output of an empty section with ‘. = .’. This change does not impact generated binaries because the conf/i386-cygwin-img-ld.sc linker script is used only when you run configure. Reviewed-by: Robbie Harwood 2022-03-14 Daniel Kiper commands/i386/pc/sendkey: Fix "writing 1 byte into a region of size 0" build error Latest GCC may complain in that way: commands/i386/pc/sendkey.c: In function ‘grub_sendkey_postboot’: commands/i386/pc/sendkey.c:223:21: error: writing 1 byte into a region of size 0 [-Werror=stringop-overflow=] 223 | *((char *) 0x41a) = 0x1e; | ~~~~~~~~~~~~~~~~~~^~~~~~ The volatile keyword addition helps and additionally assures us the compiler will not optimize out fixed assignments. Reviewed-by: Robbie Harwood 2022-03-14 Daniel Kiper loader/i386/bsd: Initialize ptr variable in grub_bsd_add_meta() Latest GCC may complain in that way: In file included from ../include/grub/disk.h:31, from ../include/grub/file.h:26, from ../include/grub/loader.h:23, from loader/i386/bsd.c:19: loader/i386/bsd.c: In function ‘grub_cmd_openbsd’: ../include/grub/misc.h:71:10: error: ‘ptr’ may be used uninitialized in this function [-Werror=maybe-uninitialized] 71 | return grub_memmove (dest, src, n); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ loader/i386/bsd.c:266:9: note: ‘ptr’ was declared here 266 | void *ptr; | ^~~ So, let's fix it by assigning NULL to ptr in grub_bsd_add_meta(). Reviewed-by: Robbie Harwood 2022-03-14 Daniel Kiper osdep/windows/platform: Disable gcc9 -Waddress-of-packed-member $ ./configure --target=x86_64-w64-mingw32 --with-platform=efi --host=x86_64-w64-mingw32 $ make [...] In file included from grub-core/osdep/platform.c:4: grub-core/osdep/windows/platform.c: In function ‘grub_install_register_efi’: grub-core/osdep/windows/platform.c:382:41: error: taking address of packed member of ‘struct grub_efi_file_path_device_path’ may result in an unaligned pointer value [-Werror=address-of-packed-member] 382 | path16_len = grub_utf8_to_utf16 (filep->path_name, | ~~~~~^~~~~~~~~~~ Disable the -Wadress-of-packaed-member diagnostic for grub_utf8_to_utf16() call which contains filep->path_name reference. It seems safe because the structure is defined according to the UEFI spec and we hope authors did not make any mistake... :-) This fix is similar to the fix in the commit 8e8723a6b (f2fs: Disable gcc9 -Waddress-of-packed-member). Reviewed-by: Robbie Harwood 2022-03-14 Glenn Washburn po: Un-transliterate the %zu format code Commit 45bffae13 (util/resolve: Bail with error if moddep.lst file line is too long) uses the %zu format specifier which has not been used in any translated strings yet. So the sed scripts used for transliterating certain languages need to be updated otherwise creation of the message indexes will fail on an unknown format code. This is essentially the same issue fixed for the %m format code in commit 2e246b6f (po: Fix replacement of %m in sed programs). Also reorder transliteration lines to be more lexicographically ordered. Reviewed-by: Daniel Kiper 2022-03-14 Daniel Axtens net: Check against nb->tail in grub_netbuff_pull() GRUB netbuff structure members track 2 different things: the extent of memory allocated for the packet, and the extent of memory currently being worked on. This works out in the structure as follows: nb->head: beginning of the allocation nb->data: beginning of the working data nb->tail: end of the working data nb->end: end of the allocation The head and end pointers are set in grub_netbuff_alloc() and do not change. The data and tail pointers are initialised to point at start of the allocation (that is, head == data == tail initially), and are then manipulated by grub_netbuff_*() functions. Key functions are as follows: - grub_netbuff_put(): "put" more data into the packet - advance nb->tail - grub_netbuff_unput(): trim the tail of the packet - retract nb->tail - grub_netbuff_pull(): "consume" some packet data - advance nb->data - grub_netbuff_reserve(): reserve space for future headers - advance nb->data and nb->tail - grub_netbuff_push(): "un-consume" data to allow headers to be written - retract nb->data Each of those functions does some form of error checking. For example, grub_netbuff_put() does not allow nb->tail to exceed nb->end, and grub_netbuff_push() does not allow nb->data to be before nb->head. However, grub_netbuff_pull()'s error checking is a bit weird. It advances nb->data and checks that it does not exceed nb->end. That allows you to get into the situation where nb->data > nb->tail, which should not be. Make grub_netbuff_pull() check against both nb->tail and nb->end. In theory just checking against ->tail should be sufficient but the extra check should be cheap and seems like good defensive practice. Reviewed-by: Daniel Kiper 2022-03-14 Fabian Vogt grub-mount: Add support for libfuse3 The libfuse 3.0.0 got released in 2016, with some API changes compared to 2.x. This commit introduces support for 3.x while keeping it compatible with 2.6 as a fallback still. To detect fuse3, switch configure over to use pkg-config, which is simpler yet more reliable than looking for library and header manually. Also set FUSE_USE_VERSION that way, as it depends on the used libfuse version. Now that the CFLAGS are read from pkg-config, use just , which works with 2.x as well as 3.x and is recommended by libfuse upstream. One behavior change of libfuse3 is that FUSE_ATOMIC_O_TRUNC is set by default, which means that open with O_TRUNC is passed as-is instead of calling the truncate operation. With libfuse2, truncate failed with -ENOSYS and that was returned to the application. To make O_TRUNC fail with libfuse3, return -EROFS explicitly if writing was requested. Reviewed-by: Daniel Kiper 2022-03-14 Elyes Haouas include: Remove trailing whitespaces Reviewed-by: Daniel Kiper util: Remove trailing whitespaces Reviewed-by: Daniel Kiper video: Remove trailing whitespaces Reviewed-by: Daniel Kiper tests: Remove trailing whitespaces Reviewed-by: Daniel Kiper term: Remove trailing whitespaces Reviewed-by: Daniel Kiper script: Remove trailing whitespaces Reviewed-by: Daniel Kiper partmap: Remove trailing whitespaces Reviewed-by: Daniel Kiper osdep: Remove trailing whitespaces Reviewed-by: Daniel Kiper normal: Remove trailing whitespaces Reviewed-by: Daniel Kiper net: Remove trailing whitespaces Reviewed-by: Daniel Kiper loader: Remove trailing whitespaces Reviewed-by: Daniel Kiper lib: Remove trailing whitespaces Reviewed-by: Daniel Kiper kern: Remove trailing whitespaces Reviewed-by: Daniel Kiper io: Remove trailing whitespaces Reviewed-by: Daniel Kiper gfxmenu: Remove trailing whitespaces Reviewed-by: Daniel Kiper gfxmenu: Remove trailing whitespaces Reviewed-by: Daniel Kiper fs: Remove trailing whitespaces Reviewed-by: Daniel Kiper font: Remove trailing whitespaces Reviewed-by: Daniel Kiper disk: Remove trailing whitespaces Reviewed-by: Daniel Kiper commands: Remove trailing whitespaces Reviewed-by: Daniel Kiper bus: Remove trailing whitespaces Reviewed-by: Daniel Kiper 2022-03-07 Chad Kimes net/ethernet: Fix VLAN networking on little-endian systems VLAN configuration seems to have never worked on little-endian systems. This is likely because VLANTAG_IDENTIFIER is not byte-swapped before copying into the net buffer, nor is inf->vlantag. We can resolve this by using grub_cpu_to_be16{_compile_time}() and its inverse when copying VLAN info to/from the net buffer. Reviewed-by: Daniel Kiper 2022-03-07 Heinrich Schuchardt commands/efi/lsefisystab: Short text EFI_IMAGE_SECURITY_DATABASE_GUID The EFI_IMAGE_SECURITY_DATABASE_GUID is used for the image execution information table (cf. UEFI specification 2.9, 32.5.3.1 Using The Image Execution Information Table). The lsefisystab command is used to display installed EFI configuration tables. Currently it only shows the GUID but not a short text for the table. Provide a short text for the EFI_IMAGE_SECURITY_DATABASE_GUID. Reviewed-by: Daniel Kiper 2022-03-07 Glenn Washburn tests: Fix whitespace formatting Reviewed-by: Daniel Kiper 2022-03-07 Peter Jones ChangeLog: Retire ChangeLog-2015 ChangeLog-2015 has been untouched for over 7 years now, and any information in it is purely for historical purposes. At the same time, grepping for code winds up matching this file quite a bit, almost never accomplishing anything other than cluttering up your grep results. We don't need this in the main repo, and "git show" will find it if you're looking at the old history of commits on some file. This patch deletes it and the Makefile.am rule to distribute it. Reviewed-by: Daniel Axtens Reviewed-by: Robbie Harwood Reviewed-by: Javier Martinez Canillas Reviewed-by: Daniel Kiper 2022-03-07 Peter Levine templates: Properly handle multiple initrd paths in 30_os-prober os-prober now effectively handles multiple paths passed to initrd, but grub-mkconfig still truncates off any subsequent space-delimited paths. Support proper parsing of space-delimited initrd paths passed from os-prober for distributions, like Manjaro, that require it. Fixes: https://savannah.gnu.org/bugs/?47681 Reviewed-by: Daniel Kiper 2022-03-07 Samuel Thibault templates: Add support for pci-arbiter and rumpdisk on Hurd This adds pci-arbiter and rumpdisk as bootstrap modules whenever they are available. This opens the path for fully-userland disk support. Reviewed-by: Daniel Kiper 2022-03-07 Glenn Washburn mm: Temporarily disable grub_mm_debug while calling grub_vprintf() in grub_printf() To prevent infinite recursion when grub_mm_debug is on, disable it when calling grub_vprintf(). One such call loop is: grub_vprintf() -> parse_printf_args() -> parse_printf_arg_fmt() -> grub_debug_calloc() -> grub_printf() -> grub_vprintf(). Reviewed-by: Daniel Kiper 2022-03-07 Glenn Washburn mm: Export grub_mm_dump() and grub_mm_dump_free() These functions may be useful within modules as well. Export them so that modules can use them. Reviewed-by: Daniel Kiper 2022-03-07 Glenn Washburn configure: Properly handle MM_DEBUG Define MM_DEBUG in config.h when --enable-mm-debug is passed to configure. It was being defined in config-util.h which only gets used when building GRUB utilities for the host side. The enabling of debugging for memory management in include/grub/mm.h explicitly does not happen when compiling for the GRUB utilities. So this debugging code effectively could never be enabled. Note, that MM_DEBUG is defined in an #if directive because the enabling of debugging checks if MM_DEBUG is defined, not what its value is. So even if MM_DEBUG were defined to nothing, the debugging code would still be enabled. Reviewed-by: Daniel Kiper 2022-03-07 Fangrui Song configure: Replace -Wl,-r,-d with -Wl,-r and add -fno-common In GNU ld and ld.lld, -d is used with -r to allocate space to COMMON symbols. This behavior is presumably to work around legacy projects which inspect relocatable output by themselves and do not handle COMMON symbols. The GRUB does not do this. See https://github.com/llvm/llvm-project/issues/53660 -d is quite useless and ld.lld 15.0.0 will make -d no-op. COMMON symbols have special symbol resolution semantics which can cause surprise (see https://maskray.me/blog/2022-02-06-all-about-common-symbols). GCC<10 and Clang<11 defaulted to -fcommon. Just use -fno-common to avoid COMMON symbols. Reviewed-by: Daniel Kiper 2022-03-07 Glenn Washburn tests: Add check-native and check-nonnative make targets This allows for testing only tests that run directly on the build machine or only tests that run in a virtualized environment. When testing multiple targets on the same build machine the native tests only need to be run once for all targets. Whereas, the nonnative tests must be run for each target because the test is potentially compiled differently for each target. Reviewed-by: Daniel Kiper 2022-03-07 Renaud Métrich commands/search: Fix bug stopping iteration when --no-floppy is used When using --no-floppy and a floppy was encountered, iterate_device() was returning 1, causing the iteration to stop instead of continuing. Reviewed-by: Daniel Kiper 2022-03-07 Glenn Washburn Revert "iee1275/datetime: Fix off-by-1 error." This is causing the test grub_cmd_date() to fail because the returned date is one day more than it should be. This reverts commit 607d66116 (iee1275/datetime: Fix off-by-1 error.). Tested-by: Daniel Axtens Reviewed-by: Daniel Kiper 2022-02-08 Glenn Washburn tests: Remove $((BASE#NUM)) bashism in grub-fs-tester This bashism allows converting NUM in base BASE to decimal. Its not needed because the only place its used is to convert from hexidecimal and this can also be done with the more portable $((0xHEXNUM)) syntax. Reviewed-by: Daniel Kiper 2022-02-08 Glenn Washburn tests: Skip pata_test on i386-efi In comparison to other i386 targets, on i386-efi the Q35 QEMU machine type is used to do the testing to be able to make use of the EFI firmware in QEMU. On the Q35 machine type there is no way to use ATA to communicate with an IDE, only AHCI. Reviewed-by: Daniel Kiper 2022-02-08 Glenn Washburn tests: Do not remove image file on error in pata_test The image file can be useful in debugging an issue when the test fails. Reviewed-by: Daniel Kiper 2022-02-08 Alec Brown util/grub-module-verifierXX: Validate elf section header table index for section name string table In grub-module-verifierXX.c, the function find_section() uses the value from grub_target_to_host16 (e->e_shstrndx) to obtain the section header table index of the section name string table, but it wasn't being checked if the value was there. According to the elf(5) manual page, "If the index of section name string table section is larger than or equal to SHN_LORESERVE (0xff00), this member holds SHN_XINDEX (0xffff) and the real index of the section name string table section is held in the sh_link member of the initial entry in section header table. Otherwise, the sh_link member of the initial entry in section header table contains the value zero." Since this check wasn't being made, the function get_shstrndx() is being added to make this check and use e_shstrndx if it doesn't have SHN_XINDEX as a value, else use sh_link. We also need to make sure e_shstrndx isn't greater than or equal to SHN_LORESERVE and sh_link isn't less than SHN_LORESERVE. Note that it may look as though the argument *arch isn't being used, it's actually required in order to use the macros grub_target_to_host*(x) which are unwinded to grub_target_to_host*_real(arch, (x)) based on defines earlier in the file. Reviewed-by: Darren Kenny Reviewed-by: Daniel Kiper 2022-02-08 Alec Brown util/grub-module-verifierXX: Validate number of elf section header table entries In grub-module-verifierXX.c, grub_target_to_host16 (e->e_shnum) is used to obtain the number of section header table entries, but it wasn't being checked if the value was there. According to the elf(5) manual page, "If the number of entries in the section header table is larger than or equal to SHN_LORESERVE (0xff00), e_shnum holds the value zero and the real number of entries in the section header table is held in the sh_size member of the intial entry in section header table. Otherwise, the sh_size member of the initial entry in the section header table holds the value zero." Since this check wasn't being made, the function get_shnum() is being added to make this check and use whichever member doesn't have a value of zero. If both are zero, then we must return an error. We also need to make sure that e_shnum doesn't have a value greater than or equal to SHN_LORESERVE and sh_size isn't less than SHN_LORESERVE. Note that it may look as though the argument *arch isn't being used, it's actually required in order to use the macros grub_target_to_host*(x) which are unwinded to grub_target_to_host*_real(arch, (x)) based on defines earlier in the file. Fixes: CID 314021 Fixes: CID 314027 Fixes: CID 314033 Reviewed-by: Darren Kenny Reviewed-by: Daniel Kiper 2022-02-08 Alec Brown util/grub-module-verifierXX: Add function to calculate section headers Added the function get_shdr() which returns the section header at a given index parameter passed into this function. This helps traverse the section header table and reduces repeated calls to lengthy equations used to obtain section headers. Note that it may look as though the argument *arch isn't being used, it's actually required in order to use the macros grub_target_to_host*(x) which are unwinded to grub_target_to_host*_real(arch, (x)) based on defines earlier in the file. Reviewed-by: Darren Kenny Reviewed-by: Daniel Kiper 2022-02-08 Alec Brown fs/affs: Fix resource leaks In commit 178ac5107389 (affs: Fix memory leaks), fixes were made to grub_affs_iterate_dir() to prevent memory leaks from occurring after it returns without freeing node. However, there were still some instances where node was causing a memory leak when the function returns after calling grub_affs_create_node(). In this function, new memory is allocated to node but doesn't get freed until the hook() function is called near the end. Before hook() is called, node should be freed in grub_affs_create_node() before returning out of it. Fixes: 178ac5107389 (affs: Fix memory leaks) Fixes: CID 73759 Reviewed-by: Darren Kenny Reviewed-by: Daniel Kiper 2022-02-08 Heinrich Schuchardt RISC-V: Adjust -march flags for binutils 2.38 As of version 2.38 binutils defaults to ISA specification version 2019-12-13. This version of the specification has has separated the the csr read/write (csrr*/csrw*) instructions and the fence.i from the I extension and put them into separate Zicsr and Zifencei extensions. This implies that we have to adjust the -march flag passed to the compiler accordingly. Reviewed-by: Daniel Kiper 2022-02-08 Heinrich Schuchardt efi: Correct struct grub_efi_boot_services The UEFI specification defines that the EFI_BOOT_SERVICES.Exit(() service may return EFI_SUCCESS or EFI_INVALID_PARAMETER. So it cannot be __attribute__((noreturn)). Reviewed-by: Daniel Kiper 2022-02-08 Glenn Washburn conf/Makefile.common: Order alphabetically variables Reviewed-by: Daniel Kiper 2022-02-08 Stephen Balousek net/http: Allow use of non-standard TCP/IP ports Allow the use of HTTP servers listening on ports other 80. This is done with an extension to the http notation: (http[,server[,port]]) - or - (http[,server[:port]]) Reviewed-by: Daniel Kiper 2022-02-08 Glenn Washburn Makefile: Only look for @MARKER@ at the start of a line when generating libgrub_a_init.lst Under certain conditions libgrub.pp gets generated with a such that it contains a bunch of CPP defines, at least one of which contains "@MARKER@". This line should not be used when generating libgrub_a_init.lst, otherwise we get compiler errors like: libgrub_a_init.c:22:18: error: stray ‘#’ in program 22 | extern void grub_#define_init (void); | ^ libgrub_a_init.c:22:19: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘define_init’ 22 | extern void grub_#define_init (void); | ^~~~~~~~~~~ libgrub_a_init.c:23:18: error: stray ‘#’ in program 23 | extern void grub_#define_fini (void); | ^ libgrub_a_init.c:23:19: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘define_fini’ 23 | extern void grub_#define_fini (void); | ^~~~~~~~~~~ ... When generating libgrub_a_init.lst only lines starting with "@MARKER@" are desired. Reviewed-by: Daniel Kiper 2022-02-08 Glenn Washburn gentpl.py: Fix issue where sometimes marker files have CPP defines When generating video.lst, modules whose marker file contains the string VIDEO_LIST_MARKER are selected. But when the marker file contains the CPP defines, one of the defines is VIDEO_LIST_MARKER and is present in all marker files, so they are all selected. By removing the defines, the correct modules are selected. Reviewed-by: Daniel Kiper 2022-02-08 Glenn Washburn util/resolve: Bail with error if moddep.lst file line is too long The code reads each line into a buffer of size 1024 and does not check if the line is longer. So a line longer than 1024 will be read as a valid line followed by an invalid line. Then an error confusing to the user is sent with the test "invalid line format". But the line format is perfectly fine, the problem is in GRUB's parser. Check if we've hit a line longer than the size of the buffer, and if so send a more correct and reasonable error. Reviewed-by: Daniel Kiper 2022-02-08 Glenn Washburn util/resolve: Do not read past the end of the array in read_dep_list() If the last non-NULL byte of "buf" is not a white-space character (such as when a read line is longer than the size of "buf"), then "p" will eventually point to the byte after the last byte in "buf". After which "p" will be dereferenced in the while conditional leading to an out of bounds read. Make sure that "p" is inside "buf" before dereferencing it. Reviewed-by: Daniel Kiper 2022-02-07 Glenn Washburn kern/misc: Allow selective disabling of debug facility names Sometimes you only know which debug logging facility names you want to turn off, not necessarily all the ones you want enabled. This patch allows the debug string to contain facility names in the $debug variable which are prefixed with a "-" to disable debug log messages for that conditional. Say you want all debug logging on except for btrfs and scripting, then do: "set debug=all,-btrfs,-scripting" Note, that only the last occurrence of the facility name with or without a leading "-" is considered. So simply appending ",-facilityname" to the $debug variable will disable that conditional. To illustrate, the command "set debug=all,-btrfs,-scripting,btrfs" will enable btrfs. Also, add documentation explaining this new behavior. Reviewed-by: Daniel Kiper 2022-02-07 Glenn Washburn cryptodisk: Fix Coverity use after free bug The Coverity output is: *** CID 366905: Memory - illegal accesses (USE_AFTER_FREE) /grub-core/disk/cryptodisk.c: 1064 in grub_cryptodisk_scan_device_real() 1058 cleanup: 1059 if (askpass) 1060 { 1061 cargs->key_len = 0; 1062 grub_free (cargs->key_data); 1063 } >>> CID 366905: Memory - illegal accesses (USE_AFTER_FREE) >>> Using freed pointer "dev". 1064 return dev; 1065 } 1066 1067 #ifdef GRUB_UTIL 1068 #include 1069 grub_err_t Here the "dev" variable can point to a freed cryptodisk device if the function grub_cryptodisk_insert() fails. This can happen only on a OOM condition, but when this happens grub_cryptodisk_insert() calls grub_free on the passed device. Since grub_cryptodisk_scan_device_real() assumes that grub_cryptodisk_insert() is always successful, it will return the device, though the device was freed. Change grub_cryptodisk_insert() to not free the passed device on failure. Then on grub_cryptodisk_insert() failure, free the device pointer. This is done by going to the label "error", which will call cryptodisk_close() to free the device and set the device pointer to NULL, so that a pointer to freed memory is not returned. Fixes: CID 366905 Reviewed-by: Daniel Kiper 2021-12-23 Daniel Axtens mm: Document grub_mm_init_region() The grub_mm_init_region() does some things that seem magical, especially around region merging. Make it a bit clearer. Reviewed-by: Daniel Kiper 2021-12-23 Daniel Axtens mm: Document grub_free() The grub_free() possesses a surprising number of quirks, and also uses single-letter variable names confusingly to iterate through the free list. Document what's going on. Use prev and cur to iterate over the free list. Reviewed-by: Daniel Kiper 2021-12-23 Daniel Axtens mm: grub_real_malloc(): Make small allocs comment match code Small allocations move the region's *first pointer. The comment says that this happens for allocations under 64K. The code says it's for allocations under 32K. Commit 45bf8b3a7549 changed the code intentionally: make the comment match. Fixes: 45bf8b3a7549 (* grub-core/kern/mm.c (grub_real_malloc): Decrease cut-off of moving the) Reviewed-by: Daniel Kiper 2021-12-23 Daniel Axtens mm: Clarify grub_real_malloc() When iterating through the singly linked list of free blocks, grub_real_malloc() uses p and q for the current and previous blocks respectively. This isn't super clear, so swap to using prev and cur. This makes another quirk more obvious. The comment at the top of grub_real_malloc() might lead you to believe that the function will allocate from *first if there is space in that block. It actually doesn't do that, and it can't do that with the current data structures. If we used up all of *first, we would need to change the ->next of the previous block to point to *first->next, but we can't do that because it's a singly linked list and we don't have access to *first's previous block. What grub_real_malloc() actually does is set *first to the initial previous block, and *first->next is the block we try to allocate from. That allows us to keep all the data structures consistent. Document that. Reviewed-by: Daniel Kiper 2021-12-23 Daniel Axtens mm: Document GRUB internal memory management structures I spent more than a trivial quantity of time figuring out pre_size and whether a memory region's size contains the header cell or not. Document the meanings of all the properties. Hopefully now no-one else has to figure it out! Reviewed-by: Daniel Kiper 2021-12-23 Michael Chang fs/btrfs: Use full btrfs bootloader area Up to now GRUB can only embed to the first 64 KiB before primary superblock of btrfs, effectively limiting the GRUB core size. That could consequently pose restrictions to feature enablement like advanced zstd compression. This patch attempts to utilize full unused area reserved by btrfs for the bootloader outlined in the document [1]: The first 1MiB on each device is unused with the exception of primary superblock that is on the offset 64KiB and spans 4KiB. Apart from that, adjacent sectors to superblock and first block group are not used for embedding in case of overflow and logged access to adjacent sectors could be useful for tracing it up. This patch has been tested to provide out of the box support for btrfs zstd compression with which GRUB has been installed to the partition. [1] https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs(5)#BOOTLOADER_SUPPORT Reviewed-by: Daniel Kiper 2021-12-23 Glenn Washburn tests: Refactor building xorriso command for iso9660 tests The iso9660 tests test creating isos with different combinations of Joliet, Rock Ridge, and ISO 9660 conformance level. Refactor xorriso argument generation for more readability and extensibility. Reviewed-by: Thomas Schmitt Reviewed-by: Daniel Kiper 2021-12-23 Glenn Washburn cryptodisk: Improve handling of partition name in cryptomount password prompt Call grub_partition_get_name() unconditionally to initialize the part variable. Then part will only be NULL when grub_partition_get_name() errors. Note that when source->partition is NULL, then grub_partition_get_name() returns an allocated empty string. So no comma or partition will be printed, as desired. Reviewed-by: Daniel Kiper 2021-12-23 Glenn Washburn cryptodisk: Move global variables into grub_cryptomount_args struct Note that cargs.search_uuid does not need to be initialized in various parts of the cryptomount argument parsing, just once when cargs is declared with a struct initializer. The previous code used a global variable which would retain the value across cryptomount invocations. Reviewed-by: Daniel Kiper 2021-12-23 Glenn Washburn cryptodisk: Refactor password input out of crypto dev modules into cryptodisk The crypto device modules should only be setting up the crypto devices and not getting user input. This has the added benefit of simplifying the code such that three essentially duplicate pieces of code are merged into one. Add documentation of passphrase option for cryptomount as it is now usable. Reviewed-by: Daniel Kiper 2021-12-23 Glenn Washburn cryptodisk: Add infrastructure to pass data from cryptomount to cryptodisk modules Previously, the cryptomount arguments were passed by global variable and function call argument, neither of which are ideal. This change passes data via a grub_cryptomount_args struct, which can be added to over time as opposed to continually adding arguments to the cryptodisk scan and recover_key. As an example, passing a password as a cryptomount argument is implemented. However, the backends are not implemented, so testing this will return a not implemented error. Also, add comments to cryptomount argument parsing to make it more obvious which argument states are being handled. Reviewed-by: Daniel Kiper 2021-12-23 Glenn Washburn cryptodisk: Improve cryptomount -u error message When a cryptmount is specified with a UUID, but no cryptodisk backends find a disk with that UUID, return a more detailed message giving telling the user that they might not have a needed cryptobackend module loaded. Reviewed-by: Daniel Kiper 2021-12-23 Glenn Washburn cryptodisk: Improve error messaging in cryptomount invocations Update such that "cryptomount -u UUID" will not print two error messages when an invalid passphrase is given and the most relevant error message will be displayed. Reviewed-by: Daniel Kiper 2021-12-23 Glenn Washburn cryptodisk: Return failure in cryptomount when no cryptodisk modules are loaded This displays an error notifying the user that they'll want to load a backend module to make cryptomount useful. Reviewed-by: Daniel Kiper 2021-12-23 Glenn Washburn cryptodisk: Refactor to discard have_it global The global "have_it" was never used by the crypto-backends, but was used to determine if a crypto-backend successfully mounted a cryptodisk with a given UUID. This is not needed however, because grub_device_iterate() will return 1 if and only if grub_cryptodisk_scan_device() returns 1. And grub_cryptodisk_scan_device() will now only return 1 if a search_uuid has been specified and a cryptodisk was successfully setup by a crypto-backend or a cryptodisk of the requested UUID is already open. To implement this grub_cryptodisk_scan_device_real() is modified to return a cryptodisk or NULL on failure and having the appropriate grub_errno set to indicated failure. Note that grub_cryptodisk_scan_device_real() will fail now with a new errno GRUB_ERR_BAD_MODULE when none of the cryptodisk backend modules succeed in identifying the source disk. With this change grub_device_iterate() will return 1 when a crypto device is successfully decrypted or when the source device has already been successfully opened. Prior to this change, trying to mount an already successfully opened device would trigger an error with the message "no such cryptodisk found", which is at best misleading. The mount should silently succeed in this case, which is what happens with this patch. Reviewed-by: Daniel Kiper 2021-12-23 Glenn Washburn luks2: Add debug message to align with luks and geli modules Reviewed-by: Daniel Kiper configure: Fix misspelled variable BUILD_LDFAGS -> BUILD_LDFLAGS Reviewed-by: Daniel Kiper 2021-12-23 Michael Chang grub-mkconfig: Restore umask for the grub.cfg The commit ab2e53c8a (grub-mkconfig: Honor a symlink when generating configuration by grub-mkconfig) has inadvertently discarded umask for creating grub.cfg in the process of running grub-mkconfig. The resulting wrong permission (0644) would allow unprivileged users to read GRUB configuration file content. This presents a low confidentiality risk as grub.cfg may contain non-secured plain-text passwords. This patch restores the missing umask and sets the creation file mode to 0600 preventing unprivileged access. Fixes: CVE-2021-3981 Reviewed-by: Daniel Kiper 2021-12-23 Heinrich Schuchardt efi: Create the grub_efi_close_protocol() library function Create a library function for CloseProtocol() and use it for the SNP driver. Reviewed-by: Daniel Kiper 2021-12-23 Heinrich Schuchardt efinet: Correct closing of SNP protocol In the context of the implementation of the EFI_LOAD_FILE2_PROTOCOL for the initial ramdisk it was observed that opening the SNP protocol failed. https://lists.gnu.org/archive/html/grub-devel/2021-10/msg00020.html This is due to an incorrect call to CloseProtocol(). The first parameter of CloseProtocol() is the handle, not the interface. We call OpenProtocol() with ControllerHandle == NULL. Hence we must also call CloseProtcol() with ControllerHandel == NULL. Each call of OpenProtocol() for the same network card handle is expected to return the same interface pointer. If we want to close the protocol which we opened non-exclusively when searching for a card, we have to do this before opening the protocol exclusively. As there is no guarantee that we successfully open the protocol add checks in the transmit and receive functions. Reported-by: Andreas Schwab Reviewed-by: Daniel Kiper 2021-12-23 Colin Watson minilzo: Update to minilzo-2.10 minilzo fails to build on a number of Debian release architectures (armel, mips64el, mipsel, ppc64el) with errors such as: ../../grub-core/lib/minilzo/minilzo.c: In function 'lzo_memops_get_le16': ../../grub-core/lib/minilzo/minilzo.c:3479:11: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing] 3479 | * (lzo_memops_TU2p) (lzo_memops_TU0p) (dd) = * (const lzo_memops_TU2p) (const lzo_memops_TU0p) (ss); \ | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ../../grub-core/lib/minilzo/minilzo.c:3530:5: note: in expansion of macro 'LZO_MEMOPS_COPY2' 3530 | LZO_MEMOPS_COPY2(&v, ss); | ^~~~~~~~~~~~~~~~ The latest upstream version is 2.10, so updating to it seems like a good idea on general principles, and it fixes builds on all the above architectures. The update procedure documented in the GRUB Developers Manual worked; I just updated the version numbers to make it clear that it's been executed recently. Reviewed-by: Daniel Kiper 2021-12-23 Glenn Washburn docs: Add documentation on packages for building documentation Reviewed-by: Daniel Kiper 2021-12-23 Glenn Washburn docs: Fix broken links in development docs Use the Git Book as a reference for documentation on Git as no other link was provided. Other links were broken because they used @url instead of @uref and needed a comma separator between link and link text. Reviewed-by: Daniel Kiper 2021-12-23 Glenn Washburn docs: Update development docs to include information on running test suite Add a section with minimal description on setting up and running the test suite with a link to the INSTALL documentation which is a little more detailed in terms of package requirements. Reviewed-by: Daniel Kiper 2021-12-23 Glenn Washburn docs: Add sentence on where Debian packages can be searched for online Reviewed-by: Daniel Kiper 2021-12-23 Qu Wenruo fs/btrfs: Make extent item iteration to handle gaps The GRUB btrfs implementation can't handle two very basic btrfs file layouts: 1. Mixed inline/regualr extents # mkfs.btrfs -f test.img # mount test.img /mnt/btrfs # xfs_io -f -c "pwrite 0 1k" -c "sync" -c "falloc 0 4k" \ -c "pwrite 4k 4k" /mnt/btrfs/file # umount /mnt/btrfs # ./grub-fstest ./grub-fstest --debug=btrfs ~/test.img hex "/file" Such mixed inline/regular extents case is not recommended layout, but all existing tools and kernel can handle it without problem. 2. NO_HOLES feature # mkfs.btrfs -f test.img -O no_holes # mount test.img /mnt/btrfs # xfs_io -f -c "pwrite 0 4k" -c "pwrite 8k 4k" /mnt/btrfs/file # umount /mnt/btrfs # ./grub-fstest ./grub-fstest --debug=btrfs ~/test.img hex "/file" NO_HOLES feature is going to be the default mkfs feature in the incoming v5.15 release, and kernel has support for it since v4.0. The way GRUB btrfs code iterates through file extents relies on no gap between extents. If any gap is hit, then GRUB btrfs will error out, without any proper reason to help debug the bug. This is a bad assumption, since a long long time ago btrfs has a new feature called NO_HOLES to allow btrfs to skip the padding hole extent to reduce metadata usage. The NO_HOLES feature is already stable since kernel v4.0 and is going to be the default mkfs feature in the incoming v5.15 btrfs-progs release. When there is a extent gap, instead of error out, just try next item. This is still not ideal, as kernel/progs/U-boot all do the iteration item by item, not relying on the file offset continuity. But it will be way more time consuming to correct the whole behavior than starting from scratch to build a proper designed btrfs module for GRUB. Reviewed-by: Daniel Kiper 2021-11-22 Alec Brown disk/ldm: Fix resource leak Commit 23e39f50ca7a (disk/ldm: Make sure comp data is freed before exiting from make_vg()) fixed several spots in make_vg() where comp data was leaking memory when an error was being handled but missed one. To avoid leaking memory, comp should be freed when an error is being handled after comp has been successfully allocated memory in the for loop. Fixes: 23e39f50ca7a (disk/ldm: Make sure comp data is freed before exiting from make_vg()) Fixes: CID 73804 Reviewed-by: Daniel Kiper 2021-11-22 Alec Brown commands/probe: Fix resource leaks Commit 1fc860bb76bb (commands/probe: Fix a resource leak when probing disks), missed other cases where grub_device_close() should be called before a return statement is called. Also found that grub_disk_close() wasn't being called when an error is being returned. To avoid conflict with grub_errno, grub_error_push() should be called before either grub_device_close() or grub_disk_close() is called and grub_error_pop() should be called before grub_errno is returned. Fixes: 1fc860bb76bb (commands/probe: Fix a resource leak when probing disks) Fixes: CID 292443 Reviewed-by: Daniel Kiper 2021-11-22 Michael Chang templates: Filter out POSIX locale for translation The POSIX locale is default or native operating system's locale identical to the C locale, so no translation to human speaking languages are provided. For this reason we should filter out LANG=POSIX as well as LANG=C upon generating grub.cfg to avoid looking up for it's gettext's message catalogs that will consequently result in an unpleasant message: error: file `/boot/grub/locale/POSIX.gmo' not found Reviewed-by: Daniel Kiper 2021-11-02 Darren Kenny io/gzio: Fix possible use of uninitialized variable in huft_build() In huft_build() it is possible to reach the for loop where "r" is being assigned to "q[j]" without "r.v" ever being initialized. Fixes: CID 314024 Reviewed-by: Daniel Kiper 2021-11-02 Darren Kenny fs/zfs/zfs: Fix possible insecure use of chunk size in zap_leaf_array_get() In zap_leaf_array_get() the chunk size passed in is considered tainted by Coverity, and is being used before it is tested for validity. To fix this the assignment of "la" is moved until after the test of the value of "chunk". Fixes: CID 314014 Reviewed-by: Daniel Kiper 2021-11-02 Darren Kenny util/grub-mkfont: Fix memory leak in write_font_pf2() In the function write_font_pf2() memory is allocated for font_name to construct a new name, but it is not released before returning from the function, leaking the allocated memory. Fixes: CID 314015 Reviewed-by: Daniel Kiper 2021-11-02 Darren Kenny util/grub-fstest: Fix resource leaks in cmd_cmp() In the function cmd_cmp() within the while loop, srcnew and destnew are being allocated but are never freed either before leaving scope or in the recursive calls being made to cmd_cmp(). Fixes: CID 314032 Fixes: CID 314045 Reviewed-by: Daniel Kiper 2021-11-02 Darren Kenny util/grub-mkrescue: Fix memory leak in write_part() In the function write_part(), the value of inname is not used beyond the grub_util_fopen() call, so it should be freed to avoid leakage. Fixes: CID 314028 Reviewed-by: Daniel Kiper 2021-11-02 Darren Kenny util/grub-install-common: Fix memory leak in copy_all() The copy_all() function skips a section of code using continue, but fails to free the memory in srcf first, leaking it. Fixes: CID 314026 Reviewed-by: Daniel Kiper 2021-11-02 Robbie Harwood kern/dl: Print module name on license check failure Prior to this change, the GRUB would only indicate that the check had been failed, but not by what module. This made it difficult to track down either the problem module, or debug the false positive further. Before performing the license check, resolve the module name so that it can be printed if the license check fails. Reviewed-by: Daniel Kiper 2021-10-25 Glenn Washburn kern/misc: Add debug log condition to log output Adding the conditional to debug log messages allows the GRUB user to construct the $debug variable without needing to consult the source to find the conditional (especially useful for situations where the source is not readily available). Reviewed-by: Daniel Kiper 2021-10-25 Glenn Washburn tests: In partmap_test, use ${parted} variable when checking for binary Reviewed-by: Daniel Kiper 2021-10-25 Glenn Washburn tests: Test aborts due to missing requirements should be marked as error instead of skipped Many tests abort due to not being root or missing tools, for instance mkfs commands for file system tests. The tests are exited with code 77, which means they were skipped. A skipped test is a test that should not be run, e.g. a test specific to ARM64 should not be run on an x86 build. These aborts are actually a hard error, code 99. That means that the test could not be completed, but not because what was supposed to be tested failed, e.g. in these cases where a missing tool prevents the running of a test. Reviewed-by: Daniel Kiper 2021-10-25 Glenn Washburn tests: Boot PowerPC using PMU instead of CUDA for power management A recent refactoring of CUDA command code has exposed a bug in OpenBIOS [1] which was causing system powerdown and system reset to fail, thus causing the QEMU instance to hang. This in turn caused the grub-shell command to timeout causing it to return an error code when the test actually completed successfully. Since it could be a while before the patch fixing this issue in OpenBIOS filters down to the average distro, switch to PMU to allow powerdowns and reboots to work as expected. [1] https://gitlab.com/qemu-project/qemu/-/issues/624 Reviewed-by: Daniel Kiper 2021-10-14 Kees Cook osdep/linux: Fix md array device enumeration GET_ARRAY_INFO's info.nr_disks does not map to GET_DISK_INFO's disk.number, which is an internal kernel index. If an array has had drives added, removed, etc., there may be gaps in GET_DISK_INFO's results. But since the consumer of devicelist cannot tolerate gaps (it expects to walk a NULL-terminated list of device name strings), the devicelist index (j) must be tracked separately from the disk.number index (i). As part of this, since GRUB wants to only examine active (i.e. present and non-failed) disks, the count of remaining disks (remaining) must be tracked separately from the devicelist index (j). Additionally, drop a line with empty spaces only. Fixes: 49de079bbe1c (... (grub_util_raid_getmembers): Handle "removed" disks) Fixes: 2b00217369ac (... Added support for RAID and LVM) Fixes: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1912043 Fixes: https://savannah.gnu.org/bugs/index.php?59887 Reviewed-by: Petr Vorel Reviewed-by: Daniel Kiper 2021-10-14 Glenn Washburn docs: Add fuller accounting of "make check" prerequisites Many of the prerequisites for exercising the full "make check" test suite have not been documented. This adds them along with a note that some tests require elevated privileges to run. Add an incomplete list of cross compiling toolchain packages for Debian and trusted sources for other distros. Add statement at the start of the document to clarify that package names are from Debian 11. Reviewed-by: Daniel Kiper 2021-10-14 Glenn Washburn tests: Do not delete filesystem images on error The filesystem images created for the filesystem test can be useful when debugging why a filesystem test failed. So, keep them around and let the user clean them up. Reviewed-by: Daniel Kiper 2021-10-14 Glenn Washburn tests: Output list of devices when partmap fails Reviewed-by: Daniel Kiper 2021-10-14 Glenn Washburn tests: Skip HFS test only when mac_roman module is not loaded and not loadable Allow the HFS tests to not be skipped if the mac_roman modules is loaded in the kernel, but not accessible to modprobe. Reviewed-by: Daniel Kiper 2021-10-14 Glenn Washburn tests: Change FAT volume label to be with in the valid character range The ";", semi-colon, character is not a valid character for a FAT filesystem label. This test used to succeed because prior to v4.2 of dosfstools mkfs.vfat did not enforce the character restrictions for volume labels. So, change the volume label string to be valid but contain symbol characters to test odd volume labels. Reviewed-by: Daniel Kiper 2021-10-14 Glenn Washburn tests: Only test MINIX3 volumes of 1 KiB block size Apparently there used to be a -B option for mkfs.minix to create a volume with a specified block size. This version is hard to come by and does not appear to be available in Debian distributions. So, remove support for testing a variety of blocks sizes for MINIX3. This allows the MINIX tests to run because they were being skipped due to not finding a mkfs.minix with the -B option. Reviewed-by: Daniel Kiper 2021-10-14 Glenn Washburn tests: mkfs.btrfs now supports only 4 KiB sector sizes and above Reviewed-by: Daniel Kiper tests: Disable ReiserFS tests for old format because newer kernels do not support them Reviewed-by: Daniel Kiper tests: mkreiserfs only supports 4096 block size Reviewed-by: Daniel Kiper tests: Rename variable filtime -> filetime as its meant to be Reviewed-by: Daniel Kiper 2021-10-14 Glenn Washburn tests: Use @BUILD_SHEBANG@ autoconf var instead of literal shell This bring this test in line with the rest of the test scripts. Reviewed-by: Daniel Kiper 2021-10-14 Glenn Washburn tests: Exit with skipped exit code when test not performed These tests were not performed and therefore did not pass, nor fail. This fixes misleading test exit code where, for instance, the pseries_test will pass on i386-pc, which is not a pseries architecture. Reviewed-by: Daniel Kiper 2021-10-14 Glenn Washburn tests: A failure of mktemp should cause the test script to exit with code 99 A test exiting with code 99 means that there was an error in the test itself and not a failure in the thing being tested (also known as a hard error). Reviewed-by: Daniel Kiper 2021-10-14 Glenn Washburn tests: Make setup errors in grub-fs-tester hard errors When a test program fails because it failed to setup the test properly, this does not indicate a failure in what is attempting to be tested because the test is never run. So exit with a hard error exit status to note this difference. This will allow easier detection of tests that are not actually being run and those that are really failing. Reviewed-by: Daniel Kiper 2021-10-14 Glenn Washburn tests: Do not occlude grub-shell return code The script grub-shell does the bulk of the testing. If it returns an error code, that means that the test failed and the test should immediately exit with that error code. When grub-shell is used as a non-terminating command in a pipeline, e.g. when data needs to be extracted from its output, its error code will be occluded by the last command in the pipeline. Refactor tests so that the shell will error with the exit code of grub-shell by breaking up pipelines such that grub-shell is always the last command in the pipeline that it is used in. Reviewed-by: Daniel Kiper 2021-10-14 Glenn Washburn tests: Do not occlude subshell error codes when used as input to the test command When using the output of a subshell as input, its error code is ignored in the context of "set -e". Many test scripts use grub-shell in a subshell with output used as an argument to the test command to test for expected output. Refactor these tests so that the subshell output goes to a shell variable, so that if the subshell errors the script will immediately exit with an error code. Reviewed-by: Daniel Kiper 2021-10-14 Glenn Washburn tests: Add set -e to missing tests This helps to ensure that error codes do not get ignored. Reviewed-by: Daniel Kiper 2021-10-14 Glenn Washburn tests: When checking squashfs fstime, use superblock last modified time Currently, the filesystem timestamp check in grub-fs-tester uses the squashfs image file's last modified timestamp and checks to see if that time stamp is within 3 seconds of the superblock timestamp as determined by grub. The image file's timestamp could be more than 3 seconds off if mksquashfs takes more than 3 seconds to generate the image, as is the case on a virtual machine. Instead use squashfs tools to get the filesystem timestamp directly. Reviewed-by: Daniel Kiper 2021-10-14 Glenn Washburn tests: Fix partmap_test for arm*-efi, disk numbering has changed Perhaps using a newer UEFI firmware is the reason for the created test disk showing up as hd2 instead of hd3. Reviewed-by: Daniel Kiper 2021-10-04 Nikolai Kostrigin docs/grub-dev: Fix typos Reviewed-by: Daniel Kiper 2021-10-04 Michael Chang build: Fix build error with binutils 2.36 The following procedure to build xen/pvgrub is broken. git clone https://git.savannah.gnu.org/git/grub.git cd grub ./bootstrap mkdir build-xen cd build-xen ../configure --with-platform=xen make It fails with the message: /usr/lib64/gcc/x86_64-suse-linux/10/../../../../x86_64-suse-linux/bin/ld: section .note.gnu.property VMA [0000000000400158,0000000000400187] overlaps section .bss VMA [000000000000f000,000000000041e1af] The most significant factor is that new assembler (GNU as) generates the .note.gnu.property section as default. This note section overlaps with .bss because it doesn't reposition with -Wl,-Ttext,0 with which the base address of .text section is set, rather the address of .note.gnu.property is calculated for some reason from 0x400000 where the ELF executable defaults to start. Using -Ttext-segment doesn't help either, though it is said to set the address of the first byte of the text segment according to "man ld". What it actually does is to override the default 0x400000, aka the image base address, to something else. The entire process can be observed in the default linker script used by gcc [1]. Therefore we can't expect it to achieve the same thing as -Ttext given that the first segment where .text resides is offset by SIZEOF_HEADERS plus some sections may be preceding it within the first segment. The end result is .text always has to start with non-zero address with -Wl,-Ttext-segment,0 if using default linker script. It is also worth mentioning that binutils upstream apparently doesn't seem to consider this as a bug [2] and proposed to use -Wl,-Ttext-segment,0 which is not fruitful as what has been tested by Gentoo [3]. As long as GRUB didn't use ISA information encoded in .note.gnu.property, we can safely drop it via -Wa,-mx86-used-note=no assembler option to fix the linker error above. This is considered a better approach than using custom linker script to drop the .note.gnu.property section because object file manipulation can also be hampered one way or the other in that linker script may not be helpful. See also this commit removing the section in the process of objcopy. 6643507ce build: Fix GRUB i386-pc build with Ubuntu gcc [1] In /usr/lib64/ldscripts/elf_x86_64.x or use 'gcc -Wl,--verbose ...' PROVIDE (__executable_start = SEGMENT_START("text-segment", 0x400000)); . = SEGMENT_START("text-segment", 0x400000) + SIZEOF_HEADERS; [2] https://sourceware.org/bugzilla/show_bug.cgi?id=27377 [3] https://bugs.gentoo.org/787221 Reviewed-by: Daniel Kiper 2021-10-04 Michael Chang disk/diskfilter: Use nodes in logical volume's segment as member device Currently the grub_diskfilter_memberlist() function returns all physical volumes added to a volume group to which a logical volume (LV) belongs. However, this is suboptimal as it doesn't fit the intended behavior of returning underlying devices that make up the LV. To give a clear picture, the result should be identical to running commands below to display the logical volumes with underlying physical volumes in use. localhost:~ # lvs -o lv_name,vg_name,devices /dev/system/root LV VG Devices root system /dev/vda2(512) localhost:~ # lvdisplay --maps /dev/system/root --- Logical volume --- ... --- Segments --- Logical extents 0 to 4604: Type linear Physical volume /dev/vda2 Physical extents 512 to 5116 As shown above, we can know system-root LV uses only /dev/vda2 to allocate it's extents, or we can say that /dev/vda2 is the member device comprising the system-root LV. It is important to be precise on the member devices, because that helps to avoid pulling in excessive dependency. Let's use an example to demonstrate why it is needed. localhost:~ # findmnt / TARGET SOURCE FSTYPE OPTIONS / /dev/mapper/system-root ext4 rw,relatime localhost:~ # pvs PV VG Fmt Attr PSize PFree /dev/mapper/data system lvm2 a-- 1020.00m 0 /dev/vda2 system lvm2 a-- 19.99g 0 localhost:~ # cryptsetup status /dev/mapper/data /dev/mapper/data is active and is in use. type: LUKS1 cipher: aes-xts-plain64 keysize: 512 bits key location: dm-crypt device: /dev/vdb sector size: 512 offset: 4096 sectors size: 2093056 sectors mode: read/write localhost:~ # vgs VG #PV #LV #SN Attr VSize VFree system 2 3 0 wz--n- 20.98g 0 localhost:~ # lvs -o lv_name,vg_name,devices LV VG Devices data system /dev/mapper/data(0) root system /dev/vda2(512) swap system /dev/vda2(0) We can learn from above that /dev/mapper/data is an encrypted volume and also gets assigned to volume group "system" as one of it's physical volumes. And also it is not used by root device, /dev/mapper/system-root, for allocating extents, so it shouldn't be taking part in the process of setting up GRUB to access root device. However, running grub-install reports error as volume group "system" contains encrypted volume. error: attempt to install to encrypted disk without cryptodisk enabled. Set `GRUB_ENABLE_CRYPTODISK=y' in file `/etc/default/grub'. Certainly we can enable GRUB_ENABLE_CRYPTODISK=y and move on, but that is not always acceptable since the server may need to be booted unattended. Additionally, typing passphrase for every system startup can be a big hassle of which most users would like to avoid. This patch solves the problem by returning exact physical volume, /dev/vda2, rightly used by system-root from the example above, thus grub-install will not error out because the excessive encrypted device to boot the root device is not configured. Tested-by: Olav Reinert Reviewed-by: Daniel Kiper 2021-10-04 Krzysztof Nowicki fs/ext2: Fix handling of missing sparse extent leafs When a file on ext4 is stored as sparse the data belonging to zero-filled blocks is not written to storage and the extent map is missing entries for these blocks. Such case can happen both for depth 0 extents (leafs) as well as higher-level tables. Consider a scenario of a file which has a zero-filled beginning (e.g. ISO image). In such case real data starts at block 8. If such a file is stored using 2-level extent structure the extent list in the inode will be depth 1 and will have an entry to a depth 0 (leaf) extent header for blocks 8-n. Unfortunately existing GRUB2 ext2 driver is only able to handle missing entries in leaf extent tables, for which the grub_ext2_read_block() function returns 0. In case the whole leaf extent list is missing for a block the function fails with "invalid extent" error. The fix for this problem relies on the grub_ext4_find_leaf() helper function to distinguish two error cases: missing extent and error walking through the extent tree. The existing error message is raised only for the latter case, while for the missing leaf extent zero is returned from grub_ext2_read_block() indicating a sparse block. Reviewed-by: Daniel Kiper 2021-10-04 Daniel Axtens powerpc: Drop Open Hack'Ware - remove GRUB_IEEE1275_FLAG_NO_ANSI Open Hack'Ware was the only user. Reviewed-by: Daniel Kiper 2021-10-04 Daniel Axtens powerpc: Drop Open Hack'Ware - remove GRUB_IEEE1275_FLAG_CANNOT_INTERPRET Open Hack'Ware was the only user. Reviewed-by: Daniel Kiper 2021-10-04 Daniel Axtens powerpc: Drop Open Hack'Ware - remove GRUB_IEEE1275_FLAG_CANNOT_SET_COLORS Open Hack'Ware was the only user. Reviewed-by: Daniel Kiper 2021-10-04 Daniel Axtens powerpc: Drop Open Hack'Ware - remove GRUB_IEEE1275_FLAG_FORCE_CLAIM Open Hack'Ware was the only user. It added a lot of complexity. Reviewed-by: Daniel Kiper 2021-10-04 Daniel Axtens powerpc: Drop Open Hack'Ware Open Hack'Ware was an alternative firmware of powerpc under QEMU. The last commit to any Open Hack'Ware repo I can find is from 2014 [1]. Open Hack'Ware was used for the QEMU "prep" machine type, which was deprecated in QEMU in commit 54c86f5a4844 (hw/ppc: deprecate the machine type 'prep', replaced by '40p') in QEMU v3.1, and had reportedly been broken for years before without anyone noticing. Support was removed in February 2020 by commit b2ce76a0730e (hw/ppc/prep: Remove the deprecated "prep" machine and the OpenHackware BIOS). Open Hack'Ware's limitations require some messy code in GRUB. This complexity is not worth carrying any more. Remove detection of Open Hack'Ware. We will clean up the feature flags in following commits. [1]: https://github.com/qemu/openhackware and https://repo.or.cz/w/openhackware.git are QEMU submodules. They have only small changes on top of OHW v0.4.1, which was imported into QEMU SCM in 2010. I can't find anything resembling an official repo any more. Reviewed-by: Daniel Kiper 2021-09-20 Glenn Washburn docs/grub: Improve search documentation, by adding short options and section on hints Reviewed-by: Daniel Kiper 2021-09-20 Glenn Washburn fs/udf: Fix regression which is preventing symlink access This code was broken by commit 3f05d693 (malloc: Use overflow checking primitives where we do complex allocations), which added overflow checking in many areas. The problem here is that the changes update the local variable sz, which was already in use and which was not updated before the change. So the code using sz was getting a different value of than it would have previously for the same UDF image. This causes the logic getting the destination of the symlink to not realize that its gotten the full destination, but keeps trying to read past the end of the destination. The bytes after the end are generally NULL padding bytes, but that's not a valid component type (ECMA-167 14.16.1.1). So grub_udf_read_symlink() branches to error logic, returning NULL, instead of the symlink destination path. The result of this bug is that the UDF filesystem tests were failing in the symlink test with the grub-fstest error message: grub-fstest: error: cannot open `(loop0)/sym': invalid symlink. This change stores the result of doubling sz in another local variable s, so as not to modify sz. Also remove unnecessary grub_add(), which increased the output by 1, presumably to account for a NULL byte. This isn't needed because an output buffer of size twice sz is already guaranteed to be more than enough to contain the path components converted to UTF-8. The value of sz contains at least 4 bytes for the path component header (ECMA-167 14.16.1), which means that 2 * 4 bytes are allocated but will not be used for UTF-8 characters, so the NULL byte is accounted for. Reviewed-by: Daniel Kiper 2021-09-20 Chris Vogel templates: Add GRUB_CMDLINE_LINUX_RECOVERY When generating grub.cfg using grub-mkconfig and the scripts 10_linux and 20_linux_xen there is no way to add kernel command line parameters _only_ to the recovery entries generated. This is needed to e.g. start a debug shell in installations using systemd using the kernel command line parameter "systemd.debug-shell" or to recover in a system with encrypted root in situations where the decryption of the root filesystem per crypttab in the intiramfs image is broken and the recovery entry should contain information how to decrypt the rootfs (cryptopts=). This patch does not change the default behaviour of the GRUB if GRUB_CMDLINE_LINUX_RECOVERY is not set. If GRUB_CMDLINE_LINUX_RECOVERY is set and the generated recovery entry should include the kernel parameter "single" the parameter must be explicitly included in GRUB_CMDLINE_LINUX_RECOVERY. As far as I know all credits for the idea and the initial implementation go to Kyle Ranking of Purism. Reviewed-by: Daniel Kiper 2021-09-20 Michael Chang emu: Fix executable stack marking The gcc by default assumes executable stack is required if the source object file doesn't have .note.GNU-stack section in place. If any of the source objects doesn't incorporate the GNU-stack note, the resulting program will have executable stack flag set in PT_GNU_STACK program header to instruct program loader or kernel to set up the executable stack when program loads to memory. Usually the .note.GNU-stack section will be generated by gcc automatically if it finds that executable stack is not required. However it doesn't take care of generating .note.GNU-stack section for those object files built from assembler sources. This leads to unnecessary risk of security of exploiting the executable stack because those assembler sources don't actually require stack to be executable to work. The grub-emu and grub-emu-lite are found to flag stack as executable revealed by execstack tool. $ mkdir -p build-emu && cd build-emu $ ../configure --with-platform=emu && make $ execstack -q grub-core/grub-emu grub-core/grub-emu-lite X grub-core/grub-emu X grub-core/grub-emu-lite This patch will add the missing GNU-stack note to the assembler source used by both utilities, therefore the result doesn't count on gcc default behavior and the executable stack is disabled. $ execstack -q grub-core/grub-emu grub-core/grub-emu-lite - grub-core/grub-emu - grub-core/grub-emu-lite Reviewed-by: Daniel Kiper 2021-09-13 Thomas Schmitt tests: Keep grub-fs-tester ziso9660 from failing for wrong reasons The test for the ability to decompress zisofs encoded files is supposed to fail due to the lack of this ability in GRUB. But it fails early with xorriso : FAILURE : -volid: Text too long (1650 > 32) because "ziso9660" is not in the list of filesystems which accept at most 32 bytes in their FSLABEL. If this is fixed, the test returns false success because the xorriso run does not produce any zisofs compressed files. The problem is in the sequence of native xorriso commands used. The command -set_filter_r applies only to the files which are already inserted into the emerging ISO filesystem. In the current sequence no files have been inserted yet by command -add when the last of two -set_filter_r commands is executed. After this is corrected, xorriso refuses to work because the global settings of command -zisofs can be made only before command -set_filter_r has attached zisofs filters to the data files in the emerging ISO. Further: A bug in xorriso causes a false warning about FSLABEL being too long for Joliet. Shortcomings of Joliet cause warnings about symbolic links. Such warnings might distract from the actual reason why the test is expected to fail. So, add "ziso9660" to the 32-byte FSLABEL list. Fix the xorriso run to produce compressed files which for now cause righteous failure of the test. Do this by removing a surplus group of -set_filter_r and -zisofs commands, by moving the other such group behind -add, and by swapping -set_filter_r and -zisofs. Remove the -as mkisofs options which produce a Joliet filesystem tree. Reviewed-by: Daniel Kiper 2021-09-13 Glenn Washburn commands/read: Add silent mode to read command to suppress input echo This conforms to the behavior of the -s option of the Bash read command. docs/grub: Document the -s option for the read command. Reviewed-by: Daniel Kiper 2021-09-13 Glenn Washburn kern/fs: Allow number of blocks in block list to be optional, defaulting length to device length This is primarily useful to do something like "loopback newdev (dev)8+" to create a device that skips the first 4 KiB, which may contain a container header, e.g. a non-standard RAID1 header, that GRUB does not recognize. This would allow that container data to be potentially accessed up to the end of container, which may be necessary for some layouts that store data at the end. There is currently not a good way to programmatically get the number of sectors on a disk to set the appropriate length of the blocklist. Reviewed-by: Daniel Kiper 2021-09-06 Petr Vorel autogen.sh: Detect python It helps to avoid an error on distros which has only python3 binary: ./autogen.sh: line 20: python: command not found Use python3 as the default as python2 is EOL since Jan 2020. However, check also for python which is on most distros, if not all, python2 because code still works on python2. Although it should not be needed keep the possibility to define PYTHON variable. For detection use "command -v" which is POSIX and supported on all common shells (bash, zsh, dash, busybox sh, mksh) instead requiring "which" as an extra dependency (usable on containers). Update the INSTALL file too. Reviewed-by: Daniel Kiper 2021-09-06 Petr Vorel bootstrap: Require GNU patch The bootstrap.conf uses patch, let's require it. Better than multiple messages: ./bootstrap.conf: line 84: patch: command not found Mention it also in the INSTALL file. Reviewed-by: Daniel Kiper 2021-09-06 Thomas Schmitt tests: Let xorriso fixely assume UTF-8 as local character set The iso9660_test fails if the effective locale is not UTF-8. This happens because xorriso needs to convert file names and FSLABEL to UCS-2 when preparing a Joliet tree. The grub-fs-tester obviously intends to use UTF-8 as character set, but xorriso assumes by default the result of nl_langinfo(3) with item CODESET. So, override the result of nl_langinfo(CODESET) by options of xorriso -as mkisofs. Reviewed-by: Daniel Kiper 2021-09-06 Fangrui Song via Grub-devel configure: Check for -falign-jumps=1 beside -falign-loops=1 The Clang does not support -falign-jumps and only recently gained support for -falign-loops. The -falign-jumps=1 should be tested beside -fliang-loops=1 to avoid passing unrecognized options to the Clang: clang-14: error: optimization flag '-falign-jumps=1' is not supported [-Werror,-Wignored-optimization-argument] The -falign-functions=1 is supported by GCC 5.1.0/Clang 3.8.0. So, just add the option unconditionally. Acked-by: Paul Menzel Reviewed-by: Daniel Kiper 2021-09-06 Fangrui Song via Grub-devel configure: Remove obsoleted -malign-{jumps, loops, functions} The GCC warns "cc1: warning: ‘-malign-loops’ is obsolete, use ‘-falign-loops’". The Clang silently ignores -malign-{jumps,loops,functions}. The preferred -falign-* forms have been supported since GCC 3.2. So, just remove -malign-{jumps,loops,functions}. Acked-by: Paul Menzel Reviewed-by: Daniel Kiper 2021-09-06 Erwan Velu fs/xfs: Fix unreadable filesystem with v4 superblock The commit 8b1e5d193 (fs/xfs: Add bigtime incompat feature support) introduced the bigtime support by adding some features in v3 inodes. This change extended grub_xfs_inode struct by 76 bytes but also changed the computation of XFS_V2_INODE_SIZE and XFS_V3_INODE_SIZE. Prior this commit, XFS_V2_INODE_SIZE was 100 bytes. After the commit it's 84 bytes XFS_V2_INODE_SIZE becomes 16 bytes too small. As a result, the data structures aren't properly aligned and the GRUB generates "attempt to read or write outside of partition" errors when trying to read the XFS filesystem: GNU GRUB version 2.11 .... grub> set debug=efi,gpt,xfs grub> insmod part_gpt grub> ls (hd0,gpt1)/ partmap/gpt.c:93: Read a valid GPT header partmap/gpt.c:115: GPT entry 0: start=4096, length=1953125 fs/xfs.c:931: Reading sb fs/xfs.c:270: Validating superblock fs/xfs.c:295: XFS v4 superblock detected fs/xfs.c:962: Reading root ino 128 fs/xfs.c:515: Reading inode (128) - 64, 0 fs/xfs.c:515: Reading inode (739521961424144223) - 344365866970255880, 3840 error: attempt to read or write outside of partition. This commit change the XFS_V2_INODE_SIZE computation by subtracting 76 bytes instead of 92 bytes from the actual size of grub_xfs_inode struct. This 76 bytes value comes from added members: 20 grub_uint8_t unused5 1 grub_uint64_t flags2 48 grub_uint8_t unused6 This patch explicitly splits the v2 and v3 parts of the structure. The unused4 is still ending of the v2 structures and the v3 starts at unused5. Thanks to this we will avoid future corruptions of v2 or v3 inodes. The XFS_V2_INODE_SIZE is returning to its expected size and the filesystem is back to a readable state: GNU GRUB version 2.11 .... grub> set debug=efi,gpt,xfs grub> insmod part_gpt grub> ls (hd0,gpt1)/ partmap/gpt.c:93: Read a valid GPT header partmap/gpt.c:115: GPT entry 0: start=4096, length=1953125 fs/xfs.c:931: Reading sb fs/xfs.c:270: Validating superblock fs/xfs.c:295: XFS v4 superblock detected fs/xfs.c:962: Reading root ino 128 fs/xfs.c:515: Reading inode (128) - 64, 0 fs/xfs.c:515: Reading inode (128) - 64, 0 fs/xfs.c:931: Reading sb fs/xfs.c:270: Validating superblock fs/xfs.c:295: XFS v4 superblock detected fs/xfs.c:962: Reading root ino 128 fs/xfs.c:515: Reading inode (128) - 64, 0 fs/xfs.c:515: Reading inode (128) - 64, 0 fs/xfs.c:515: Reading inode (128) - 64, 0 fs/xfs.c:515: Reading inode (131) - 64, 768 efi/ fs/xfs.c:515: Reading inode (3145856) - 1464904, 0 grub2/ fs/xfs.c:515: Reading inode (132) - 64, 1024 grub/ fs/xfs.c:515: Reading inode (139) - 64, 2816 grub> Fixes: 8b1e5d193 (fs/xfs: Add bigtime incompat feature support) Tested-by: Carlos Maiolino Reviewed-by: Daniel Kiper 2021-09-06 Heinrich Schuchardt libgcrypt: Avoid -Wempty-body in rijndael do_setkey() Avoid a warning lib/libgcrypt-grub/cipher/rijndael.c:229:9: warning: suggest braces around empty body in an ‘if’ statement [-Wempty-body] 229 | ; | ^ Reviewed-by: Daniel Kiper 2021-09-06 Heinrich Schuchardt libgcrypt: Avoid -Wsign-compare in rijndael do_setkey() Avoid a warning lib/libgcrypt-grub/cipher/rijndael.c:352:21: warning: comparison of integer expressions of different signedness: ‘int’ and ‘unsigned int’ [-Wsign-compare] 352 | for (i = 0; i < keylen; i++) | Reviewed-by: Daniel Kiper 2021-09-06 Wouter van Kesteren commands/setpci: Honor write mask argument In the case that one passes a write mask with ":" the write_mask is obtained from grub_strtoul() and then promptly overwritten by 0xffffffff three lines later. This appears to have been so since the initial version of setpci in 2009. I'm surprised no one else has hit this issue in the past 12 years... Reviewed-by: Daniel Kiper 2021-07-22 Jeff Mahoney osdep/linux/hostdisk: Use stat() instead of udevadm for partition lookup The sysfs_partition_path() calls udevadm to resolve the sysfs path for a block device. That can be accomplished by stating the device node and using the major/minor to follow the symlinks in /sys/dev/block/. This cuts the execution time of grub-mkconfig to somewhere near 55% on system without LVM (which uses libdevmapper instead sysfs_partition_path()). Remove udevadm call as it does not help us more than calling stat() directly. Reviewed-by: Daniel Kiper 2021-07-22 Petr Vorel osdep: Introduce include/grub/osdep/major.h and use it ... to factor out fix for glibc 2.25 introduced in 7a5b301e3 (build: Use AC_HEADER_MAJOR to find device macros). Note: Once glibc 2.25 is old enough and this fix is not needed also AC_HEADER_MAJOR in configure.ac should be removed. Reviewed-by: Daniel Kiper 2021-07-22 Daniel Axtens ieee1275: Drop HEAP_MAX_ADDR and HEAP_MIN_SIZE constants The HEAP_MAX_ADDR is confusing. Currently it is set to 32MB, except on ieee1275 on x86, where it is 64MB. There is a comment which purports to explain it: /* If possible, we will avoid claiming heap above this address, because it seems to cause relocation problems with OSes that link at 4 MiB */ This doesn't make a lot of sense when the constants are well above 4MB already. It was not always this way. Prior to commit 7b5d0fe4440c (Increase heap limit) in 2010, HEAP_MAX_SIZE and HEAP_MAX_ADDR were indeed 4MB. However, when the constants were increased the comment was left unchanged. It's been over a decade. It doesn't seem like we have problems with claims over 4MB on powerpc or x86 ieee1275. The SPARC does things completely differently and never used the constant. Drop the constant and the check. The only use of HEAP_MIN_SIZE was to potentially override the HEAP_MAX_ADDR check. It is now unused. Remove it too. Tested-by: Stefan Berger Reviewed-by: Daniel Kiper 2021-07-22 Marius Bakke tests/ahci: Change "ide-drive" deprecated QEMU device name to "ide-hd" The "ide-drive" device was removed in QEMU 6.0. The "ide-hd" has been available for more than 10 years now in QEMU. Thus there shouldn't be any need for backwards compatible names. Reviewed-by: Daniel Kiper 2021-07-22 Javier Martinez Canillas fs/ext2: Ignore checksum seed incompat feature This incompat feature is used to denote that the filesystem stored its metadata checksum seed in the superblock. This is used to allow tune2fs changing the UUID on a mounted metdata_csum filesystem without having to rewrite all the disk metadata. However, the GRUB doesn't use the metadata checksum at all. So, it can just ignore this feature if it is enabled. This is consistent with the GRUB filesystem code in general which just does a best effort to access the filesystem's data. The checksum seed incompat feature has to be removed from the ignore list if the support for metadata checksum verification is added to the GRUB ext2 driver later. Suggested-by: Eric Sandeen Suggested-by: Lukas Czerner Reviewed-by: Lukas Czerner Reviewed-by: Daniel Kiper 2021-06-08 Glenn Washburn zfs: Use grub_uint64_t instead of 1ULL in BF64_*CODE() macros The underlying type of 1ULL does not change across architectures but grub_uint64_t does. This allows using the BF64_*CODE() macros as arguments to format string functions that use the PRI* format string macros that also vary with architecture. Change the grub_error() call, where this was previously an issue and temporarily fixed by casting and using a format string literal code, to now use PRI* macros and remove casting. Reviewed-by: Daniel Kiper 2021-06-08 Daniel Kiper Bump version to 2.11 Skip versions between 2.07 and 2.10 to avoid leading zeros in minor version number. This way version parsing in scripts should be easier. Release 2.06 2021-06-08 Daniel Kiper SECURITY: Add SECURITY file The SECURITY file describes the GRUB project security policy. It is based on https://github.com/wireapp/wire/blob/master/SECURITY.md 2021-06-08 Daniel Kiper MAINTAINERS: Add MAINTAINERS file The MAINTAINERS file provides basic information about the GRUB project and its maintainers. 2021-06-01 Dimitri John Ledkov grub-install: Add backup and restore Refactor clean_grub_dir() to create a backup of all the files, instead of just irrevocably removing them as the first action. If available, register atexit() handler to restore the backup if errors occur before point of no return, or remove the backup if everything was successful. If atexit() is not available, the backup remains on disk for manual recovery. Some platforms defined a point of no return, i.e. after modules & core images were updated. Failures from any commands after that stage are ignored, and backup is cleaned up. For example, on EFI platforms update is not reverted when efibootmgr fails. Extra care is taken to ensure atexit() handler is only invoked by the parent process and not any children forks. Some older GRUB codebases can invoke parent atexit() hooks from forks, which can mess up the backup. This allows safer upgrades of MBR & modules, such that modules/images/fonts/translations are consistent with MBR in case of errors. For example accidental grub-install /dev/non-existent-disk currently clobbers and upgrades modules in /boot/grub, despite not actually updating any MBR. This patch only handles backup and restore of files copied to /boot/grub. This patch does not perform backup (or restoration) of MBR itself or blocklists. Thus when installing i386-pc platform, corruption may still occur with MBR and blocklists which will not be attempted to be automatically recovered. Also add modinfo.sh and *.efi to the cleanup/backup/restore code path, to ensure it is also cleaned, backed up and restored. Reviewed-by: Daniel Kiper 2021-06-01 Dimitri John Ledkov osdep/unix/exec: Avoid atexit() handlers when child execvp() fails The functions grub_util_exec_pipe() and grub_util_exec_pipe_stderr() currently call execvp(). If the call fails for any reason, the child currently calls exit(127). This in turn executes the parents atexit() handlers from the forked child, and then the same handlers are called again from parent. This is usually not desired, and can lead to deadlocks, and undesired behavior. So, change the exit() calls to _exit() calls to avoid calling atexit() handlers from child. Fixes: e75cf4a58 (unix exec: avoid atexit handlers when child exits) Reviewed-by: Daniel Kiper 2021-06-01 Jan (janneke) Nieuwenhuizen lib/i386/relocator64: Build fixes for i386 This fixes cross-compiling to x86 (e.g., the Hurd) from x86-linux of grub-core/lib/i386/relocator64.S This file has six sections that only build with a 64-bit assembler, yet only the first two sections had support for a 32-bit assembler. This patch completes this for the remaining sections. To reproduce, update the GRUB source description in your local Guix archive and run ./pre-inst-env guix build --system=i686-linux --target=i586-pc-gnu grub or install an x86 cross-build environment on x86-linux (32-bit!) and configure to cross build and make, e.g., do something like ./configure \ CC_FOR_BUILD=gcc \ --build=i686-unknown-linux-gnu \ --host=i586-pc-gnu make Additionally, remove a line with redundant spaces. Reviewed-by: Daniel Kiper 2021-06-01 Javier Martinez Canillas fs/xfs: Add needsrepair incompat feature support The XFS now has an incompat feature flag to indicate that a filesystem needs to be repaired. The Linux kernel refuses to mount the filesystem that has it set and only the xfs_repair tool is able to clear that flag. The GRUB doesn't have the concept of mounting filesystems and just attempts to read the files. But it does some sanity checking before attempting to read from the filesystem. Among the things which are tested, is if the super block only has set of incompatible features flags that are supported by GRUB. If it contains any flags that are not listed as supported, reading the XFS filesystem fails. Since the GRUB doesn't attempt to detect if the filesystem is inconsistent nor replays the journal, the filesystem access is a best effort. For this reason, ignore if the filesystem needs to be repaired and just print a debug message. That way, if reading or booting fails later, the user is able to figure out that the failures can be related to broken XFS filesystem. Suggested-by: Eric Sandeen Reviewed-by: Daniel Kiper 2021-06-01 Carlos Maiolino fs/xfs: Add bigtime incompat feature support The XFS filesystem supports a bigtime feature to overcome y2038 problem. This patch makes the GRUB able to support the XFS filesystems with this feature enabled. The XFS counter for the bigtime enabled timestamps starts at 0, which translates to GRUB_INT32_MIN (Dec 31 20:45:52 UTC 1901) in the legacy timestamps. The conversion to Unix timestamps is made before passing the value to other GRUB functions. For this to work properly, GRUB requires an access to flags2 field in the XFS ondisk inode. So, the grub_xfs_inode structure has been updated to cover full ondisk inode. Reviewed-by: Daniel Kiper 2021-06-01 Carlos Maiolino fs: Use 64-bit type for filesystem timestamp Some filesystems nowadays use 64-bit types for timestamps. So, update grub_dirhook_info struct to use an grub_int64_t type to store mtime. This also updates the grub_unixtime2datetime() function to receive a 64-bit timestamp argument and do 64-bit-safe divisions. All the remaining conversion from 32-bit to 64-bit should be safe, as 32-bit to 64-bit attributions will be implicitly casted. The most critical part in the 32-bit to 64-bit conversion is in the function grub_unixtime2datetime() where it needs to deal with the 64-bit type. So, for that, the grub_divmod64() helper has been used. These changes enables the GRUB to support dates beyond y2038. Reviewed-by: Daniel Kiper 2021-05-28 Javier Martinez Canillas types: Define PRI{x,d}GRUB_INT{32,64}_T format specifiers There are already PRI*_T constants defined for unsigned integers but not for signed integers. Add format specifiers for the latter. Suggested-by: Daniel Kiper Reviewed-by: Daniel Kiper 2021-05-28 Tianjia Zhang kern/efi/sb: Remove duplicate efi_shim_lock_guid variable The efi_shim_lock_guid local variable and shim_lock_guid global variable have the same GUID value. Only the latter is retained. Reviewed-by: Daniel Kiper 2021-05-10 Javier Martinez Canillas util/mkimage: Fix wrong PE32+ section sizes for some arches The commit f60ba9e5945 (util/mkimage: Refactor section setup to use a helper) added a helper function to setup PE sections. But it also changed how the raw data offsets were calculated since all the section sizes are aligned. However, for some platforms, i.e ia64-efi and arm64-efi, the kernel image size is not aligned using the section alignment. This leads to the situation in which the mods section offset in its PE section header does not match its real placement in the PE file. So, finally the GRUB is not able to locate and load built-in modules. The problem surfaces on ia64-efi and arm64-efi because both platforms require additional relocation data which is added behind .bss section. So, we have to add some padding behind this extra data to make the beginning of mods section properly aligned in the PE file. Fix it by aligning the kernel_size to the section alignment. That makes the sizes and offsets in the PE section headers to match relevant sections in the PE32+ binary file. Reported-by: John Paul Adrian Glaubitz Tested-by: John Paul Adrian Glaubitz Reviewed-by: Daniel Kiper 2021-05-10 Daniel Kiper term/terminfo: Fix the terminfo command help and documentation Additionally, fix the terminfo spelling mistake in the GRUB development documentation. Reviewed-by: Javier Martinez Canillas 2021-05-10 Daniel Kiper i18n: Align N_() formatting with the rest of GRUB code Reviewed-by: Javier Martinez Canillas 2021-05-10 Daniel Kiper i18n: Format large integers before the translation message - take 2 This is an additional fix which has been missing from the commit 837fe48de (i18n: Format large integers before the translation message). Reviewed-by: Javier Martinez Canillas 2021-04-13 Miguel Ángel Arruga Vivas i18n: Format large integers before the translation message The GNU gettext only supports the ISO C99 macros for integral types. If there is a need to use unsupported formatting macros, e.g. PRIuGRUB_UINT64_T, according to [1] the number to a string conversion should be separated from the code printing message requiring the internationalization. So, the function grub_snprintf() is used to print the numeric values to an intermediate buffer and the internationalized message contains a string format directive. [1] https://www.gnu.org/software/gettext/manual/html_node/Preparing-Strings.html#No-string-concatenation Reviewed-by: Daniel Kiper 2021-04-12 Daniel Axtens video/fb/fbfill: Use unsigned integers for width/height Since commit 7ce3259f67ac (video/fb/fbfill: Fix potential integer overflow), clang builds of grub-emu have failed with messages like: /usr/bin/ld: libgrubmods.a(libgrubmods_a-fbfill.o): in function `grub_video_fbfill_direct24': fbfill.c:(.text+0x28e): undefined reference to `__muloti4' This appears to be due to a weird quirk in how clang compiles grub_mul(dst->mode_info->bytes_per_pixel, width, &rowskip) which is grub_mul(unsigned int, int, &grub_size_t). It looks like clang somewhere promotes everything to 128-bit maths before ultimately reducing down to 64 bit for grub_size_t. I think this is because width is signed, and indeed converting width to an unsigned int makes the problem go away. This conversion also makes more sense generally: - the caller of all the fbfill_directN functions is grub_video_fb_fill_dispatch() and it takes width and height as unsigned ints already, - it doesn't make sense to fill a negative width or height. Convert the width and height arguments and associated loop counters to unsigned ints. Fixes: 7ce3259f67ac (video/fb/fbfill: Fix potential integer overflow) Reviewed-by: Daniel Kiper 2021-04-12 Glenn Washburn docs: Conform badmem and cutmem description indentations with other commands Reviewed-by: Daniel Kiper docs: Add note to cryptomount that UUIDs should be specified without dashes Reviewed-by: Daniel Kiper 2021-04-12 Aru Sahni templates: Fix user-facing typo with an incorrect use of "it's" Since the possessive form of "it" is being used, the apostrophe must be omitted. Reviewed-by: Daniel Kiper 2021-04-12 Colin Watson buffer: Sync up out-of-range error message The messages associated with other similar GRUB_ERR_OUT_OF_RANGE errors were lacking the trailing full stop. Syncing up the strings saves a small amount of precious core image space on i386-pc. DOWN: obj/i386-pc/grub-core/kernel.img (31740 > 31708) - change: -32 DOWN: i386-pc core image (biosdisk ext2 part_msdos) (27453 > 27452) - change: -1 DOWN: i386-pc core image (biosdisk ext2 part_msdos diskfilter mdraid09) (32367 > 32359) - change: -8 Reviewed-by: Daniel Kiper 2021-04-12 Glenn Washburn usb/usbhub: Use GRUB_USB_MAX_CONF macro instead of literal in hub for maximum configs Reviewed-by: Daniel Kiper 2021-04-12 Daniel Drake fs/minix: Avoid mistakenly probing ext2 filesystems The ext2 (and ext3, ext4) filesystems write the number of free inodes to location 0x410. On a MINIX filesystem, that same location is used for the MINIX superblock magic number. If the number of free inodes on an ext2 filesystem is equal to any of the four MINIX superblock magic values plus any multiple of 65536, GRUB's MINIX filesystem code will probe it as a MINIX filesystem. In the case of an OS using ext2 as the root filesystem, since there will ordinarily be some amount of file creation and deletion on every bootup, it effectively means that this situation has a 1:16384 chance of being hit on every reboot. This will cause GRUB's filesystem probing code to mistakenly identify an ext2 filesystem as MINIX. This can be seen by e.g. "search --label" incorrectly indicating that no such ext2 partition with matching label exists, whereas in fact it does. After spotting the rough cause of the issue I was facing here, I borrowed much of the diagnosis/explanation from meierfra who found and investigated the same issue in util-linux in 2010: https://bugs.launchpad.net/ubuntu/+source/util-linux/+bug/518582 This was fixed in util-linux by having the MINIX code check for the ext2 magic. Do the same here. Reviewed-by: Derek Foreman Reviewed-by: Daniel Kiper 2021-03-12 Daniel Kiper Release 2.06~rc1 2021-03-11 Ard Biesheuvel arm/linux: Fix ARM Linux header layout The hdr_offset member of the ARM Linux image header appears at offset 0x3c, matching the PE/COFF spec's placement of the COFF header offset in the MS-DOS header. We're currently off by four, so fix that. Reviewed-by: Daniel Kiper 2021-03-10 Glenn Washburn style: Format string macro should have a space between quotes Reviewed-by: Daniel Kiper 2021-03-10 Glenn Washburn grub/err: Do compile-time format string checking on grub_error() This should help prevent format string errors and thus improve the quality of error reporting. Reviewed-by: Daniel Kiper 2021-03-10 Glenn Washburn fs/zfs/zfs: Use format code "%llu" for 64-bit uint bp->blk_prop in grub_error() This is a temporary, less-intrusive change to get the build to success with compiler format string checking turned on. There is a better fix which addresses this issue, but it needs more testing. Use this change so that format string checking on grub_error() can be turned on until the better change is fully tested. Reviewed-by: Daniel Kiper 2021-03-10 Glenn Washburn fs/hfsplus: Use format code PRIuGRUB_UINT64_T for 64-bit typed fileblock in grub_error() Reviewed-by: Daniel Kiper 2021-03-10 Glenn Washburn dl/elf: Use format code PRIxGRUB_UINT64_T for 64-bit arg in grub_error() The macro ELF_R_TYPE does not change the underlying type. Here its argument is a 64-bit Elf64_Xword. Make sure the format code matches. For the RISC-V architecture, rel->r_info could be either Elf32_Xword or Elf64_Xword depending on if 32 or 64-bit RISC-V is being built. So cast to 64-bit value regardless. Reviewed-by: Daniel Kiper 2021-03-10 Glenn Washburn disk/ata: Use format code PRIxGRUB_UINT64_T for 64-bit uint argument in grub_error() Reviewed-by: Daniel Kiper 2021-03-10 Glenn Washburn loader/i386/pc/linux: Use PRI* macros to get correct format string code across architectures Also remove casting of format string args so that the architecture dependent type is preserved. Reviewed-by: Daniel Kiper 2021-03-10 Glenn Washburn kern/efi/mm: Format string error in grub_error() The second format string argument, GRUB_EFI_MAX_USABLE_ADDRESS, is a macro to a number literal. However, depending on what the target architecture, the type can be 32 or 64 bits. Cast to a 64-bit integer. Also, change the format string literals "%llx" to use PRIxGRUB_UINT64_T. Reviewed-by: Daniel Kiper 2021-03-10 Glenn Washburn commands/pgp: Format code for grub_error() is incorrect The format code is for a 32-bit int, but the argument, keyid, is declared as a 64 bit int. The comment above says keyid is 32-bit. I'm not sure if the comment or declaration is wrong, so force the display of a 64-bit int for now. Reviewed-by: Daniel Kiper 2021-03-10 Glenn Washburn grub_error: Use format code PRIuGRUB_SIZE for variables of type grub_size_t Reviewed-by: Daniel Kiper 2021-03-10 Glenn Washburn disk/dmraid_nvidia: Format string error in grub_error() The grub_error() has a format string expecting two arguments, but only one provided. According to the comments in the struct grub_nv_super definition, the version field looks like a version number where major.minor is encoded as each a byte in the two-byte short. Reviewed-by: Daniel Kiper 2021-03-10 Glenn Washburn video/bochs: grub_error() format string add missing format code Reviewed-by: Daniel Kiper 2021-03-10 Glenn Washburn parttool/msdospart: grub_error() missing format string argument Its obvious from the error message that the variable named "type" was accidentally omitted. Reviewed-by: Daniel Kiper 2021-03-10 Glenn Washburn misc: Format string for grub_error() should be a literal Reviewed-by: Daniel Kiper 2021-03-10 Philip Müller templates: Properly disable the os-prober by default This patch does the following: - really disables os-prober by default in the util/grub-mkconfig.in by setting GRUB_DISABLE_OS_PROBER to true, - fixes the logic in the util/grub.d/30_os-prober.in, - updates the grub_warn() lines. Reason for the code shuffling in the util/grub-mkconfig.in: The default was GRUB_DISABLE_OS_PROBER=false if you don't set GRUB_DISABLE_OS_PROBER at all. To prevent os-prober from starting we have to set it by default to true and shuffle GRUB_DISABLE_OS_PROBER to code section, which is executed by the script. However we still give an option to the user to overwrite it with false, if he wants to execute os-prober after all. Fixes: e3464147 (templates: Disable the os-prober by default) Reported-by: Didier Spaier Reported-by: Lennart Sorensen Reported-by: John Paul Adrian Glaubitz Reviewed-by: Daniel Kiper 2021-03-10 Michael Chang kern/efi/sb: Add chainloaded image as shim's verifiable object While attempting to dual boot Microsoft Windows with UEFI chainloader, it failed with below error when UEFI Secure Boot was enabled: error ../../grub-core/kern/verifiers.c:119:verification requested but nobody cares: /EFI/Microsoft/Boot/bootmgfw.efi. It is a regression, as previously it worked without any problem. It turns out chainloading PE image has been locked down by commit 578c95298 (kern: Add lockdown support). However, we should consider it as verifiable object by shim to allow booting in UEFI Secure Boot mode. The chainloaded PE image could also have trusted signature created by vendor with their pubkey cert in db. For that matters it's usage should not be locked down under UEFI Secure Boot, and instead shim should be allowed to validate a PE binary signature before running it. Fixes: 578c95298 (kern: Add lockdown support) Reviewed-by: Daniel Kiper 2021-03-10 Glenn Washburn disk/pata: Suppress error message "no device connected" This error message comes from the grub_print_error() in grub_pata_device_initialize(), which does not pass on the error, and is raised in check_device(). The function check_device() needs to return this as an error because check_device() is also used in grub_pata_open(), which does pass on this error to indicate that the device can not be used. This is actually not an error when displayed by grub_pata_device_initialize() because it just indicates that there are no pata devices seen. This may be confusing to end users who do not have pata devices yet are loading the pata module (perhaps implicitly via nativedisk). This also causes unnecessary output which may need to be accounted for in functional testing. Instead print to the debug log when check_device() raises this "error" and pop the error from the error stack. If there is another error on the stack then print the error stack as those should be real errors. Acked-by: Paul Menzel Reviewed-by: Daniel Kiper 2021-03-10 Yi Zhao fs/ext2: Fix a file not found error when a symlink filesize is equal to 60 We encountered a file not found error when the symlink filesize is equal to 60: $ ls -l initrd lrwxrwxrwx 1 root root 60 Jan 6 16:37 initrd -> secure-core-image-initramfs-5.10.2-yoctodev-standard.cpio.gz When booting, we got the following error in the GRUB: error: file `/initrd' not found The root cause is that the size of diro->inode.symlink is equal to 60 and a symlink name has to be terminated with NUL there. So, if the symlink filesize is exactly 60 then it is also stored in a separate block rather than in the inode itself. Reviewed-by: Daniel Kiper 2021-03-02 Tianjia Zhang loader/i386/linux: Do not use grub_le_to_cpu32() for relocatable variable The relocatable variable is defined as grub_uint8_t. Relevant member in setup_header structure is also defined as one byte in Linux boot protocol. By semantic definition it is a bool type. It is not appropriate to treat it as a four bytes. This patch fixes the issue. Reviewed-by: Daniel Kiper 2021-03-02 Tianjia Zhang loader/i386/linux: Remove redundant code from in grub_cmd_linux() The preferred_address has been assigned to GRUB_LINUX_BZIMAGE_ADDR during initialization in grub_cmd_linux(). The assignment here is redundant and should be removed. Reviewed-by: Daniel Kiper 2021-03-02 Heinrich Schuchardt efi: The device-tree must be in EfiACPIReclaimMemory According to the Embedded Base Boot Requirements (EBBR) specification the device-tree passed to Linux as a configuration table must reside in EfiACPIReclaimMemory. Reviewed-by: Daniel Kiper 2021-03-02 Heinrich Schuchardt commands/efi/lsefisystab: Add short text for EFI_RT_PROPERTIES_TABLE_GUID UEFI specification 2.8 errata B introduced the EFI_RT_PROPERTIES_TABLE describing the services available at runtime. The lsefisystab command is used to display installed EFI configuration tables. Currently it only shows the GUID but not a short text for the new table. Provide a short text for the EFI_RT_PROPERTIES_TABLE_GUID. Reviewed-by: Daniel Kiper 2021-03-02 Petr Vorel docs/luks2: Mention key derivation function support To give users hint why Argon2, the default in cryptsetup for LUKS2, does not work. Acked-by: Paul Menzel Reviewed-by: Patrick Steinhardt Reviewed-by: Daniel Kiper 2021-03-02 Derek Foreman commands/file: Fix array/enum desync The commit f1957dc8a (RISC-V: Add to build system) added two entries to the options array, but only 1 entry to the enum. This resulted in everything after the insertion point being off by one. This broke at least the "file --is-hibernated-hiberfil" command. Bring the two back in sync by splitting the IS_RISCV_EFI enum entry into two, as is done for other architectures. Reviewed-by: Daniel Kiper 2021-03-02 Marco A Benatto kern/mm: Fix grub_debug_calloc() compilation error Fix compilation error due to missing parameter to grub_printf() when MM_DEBUG is defined. Fixes: 64e26162e (calloc: Make sure we always have an overflow-checking calloc() available) Reviewed-by: Daniel Kiper 2021-03-02 Alex Burmashev templates: Disable the os-prober by default The os-prober is enabled by default what may lead to potentially dangerous use cases and borderline opening attack vectors. This patch disables the os-prober, adds warning messages and updates GRUB_DISABLE_OS_PROBER configuration option documentation. This way we make it clear that the os-prober usage is not recommended. Simplistic nature of this change allows downstream vendors, who really want os-prober to be enabled out of the box in their relevant products, easily revert to it's old behavior. Reported-by: NyankoSec (, https://twitter.com/NyankoSec), working with SSD Secure Disclosure Reviewed-by: Daniel Kiper 2021-03-02 Thomas Frauendorfer | Miray Software gfxmenu/gui: Check printf() format in the gui_progress_bar and gui_label The gui_progress_bar and gui_label components can display the timeout value. The format string can be set through a theme file. This patch adds a validation step to the format string. If a user loads a theme file into the GRUB without this patch then a GUI label with the following settings + label { ... id = "__timeout__" text = "%s" } will interpret the current timeout value as string pointer and print the memory at that position on the screen. It is not desired behavior. Reviewed-by: Daniel Kiper 2021-03-02 Thomas Frauendorfer | Miray Software kern/misc: Add function to check printf() format against expected format The grub_printf_fmt_check() function parses the arguments of an untrusted printf() format and an expected printf() format and then compares the arguments counts and arguments types. The arguments count in the untrusted format string must be less or equal to the arguments count in the expected format string and both arguments types must match. To do this the parse_printf_arg_fmt() helper function is extended in the following way: 1. Add a return value to report errors to the grub_printf_fmt_check(). 2. Add the fmt_check argument to enable stricter format verification: - the function expects that arguments definitions are always terminated by a supported conversion specifier. - positional parameters, "$", are not allowed, as they cannot be validated correctly with the current implementation. For example "%s%1$d" would assign the first args entry twice while leaving the second one unchanged. - Return an error if preallocated space in args is too small and allocation fails for the needed size. The grub_printf_fmt_check() should verify all arguments. So, if validation is not possible for any reason it should return an error. This also adds a case entry to handle "%%", which is the escape sequence to print "%" character. 3. Add the max_args argument to check for the maximum allowed arguments count in a printf() string. This should be set to the arguments count of the expected format. Then the parse_printf_arg_fmt() function will return an error if the arguments count is exceeded. The two additional arguments allow us to use parse_printf_arg_fmt() in printf() and grub_printf_fmt_check() calls. When parse_printf_arg_fmt() is used by grub_printf_fmt_check() the function parse user provided untrusted format string too. So, in that case it is better to be too strict than too lenient. Reviewed-by: Daniel Kiper 2021-03-02 Thomas Frauendorfer | Miray Software kern/misc: Add STRING type for internal printf() format handling Set printf() argument type for "%s" to new type STRING. This is in preparation for a follow up patch to compare a printf() format string against an expected printf() format string. For "%s" the corresponding printf() argument is dereferenced as pointer while all other argument types are defined as integer value. However, when validating a printf() format it is necessary to differentiate "%s" from "%p" and other integers. So, let's do that. Reviewed-by: Daniel Kiper 2021-03-02 Thomas Frauendorfer | Miray Software kern/misc: Split parse_printf_args() into format parsing and va_list handling This patch is preparing for a follow up patch which will use the format parsing part to compare the arguments in a printf() format from an external source against a printf() format with expected arguments. Reviewed-by: Daniel Kiper 2021-03-02 Dimitri John Ledkov shim_lock: Only skip loading shim_lock verifier with explicit consent Commit 32ddc42c (efi: Only register shim_lock verifier if shim_lock protocol is found and SB enabled) reintroduced CVE-2020-15705 which previously only existed in the out-of-tree linuxefi patches and was fixed as part of the BootHole patch series. Under Secure Boot enforce loading shim_lock verifier. Allow skipping shim_lock verifier if SecureBoot/MokSBState EFI variables indicate skipping validations, or if GRUB image is built with --disable-shim-lock. Fixes: 132ddc42c (efi: Only register shim_lock verifier if shim_lock protocol is found and SB enabled) Fixes: CVE-2020-15705 Fixes: CVE-2021-3418 Reported-by: Dimitri John Ledkov Reviewed-by: Daniel Kiper 2021-03-02 Dimitri John Ledkov grub-install-common: Add --sbat option Reviewed-by: Daniel Kiper 2021-03-02 Peter Jones util/mkimage: Add an option to import SBAT metadata into a .sbat section Add a --sbat option to the grub-mkimage tool which allows us to import an SBAT metadata formatted as a CSV file into a .sbat section of the EFI binary. Reviewed-by: Daniel Kiper 2021-03-02 Peter Jones util/mkimage: Refactor section setup to use a helper Add a init_pe_section() helper function to setup PE sections. This makes the code simpler and easier to read. Reviewed-by: Daniel Kiper 2021-03-02 Peter Jones util/mkimage: Improve data_size value calculation According to "Microsoft Portable Executable and Common Object File Format Specification", the Optional Header SizeOfInitializedData field contains: Size of the initialized data section, or the sum of all such sections if there are multiple data sections. Make this explicit by adding the GRUB kernel data size to the sum of all the modules sizes. The ALIGN_UP() is not required by the PE spec but do it to avoid alignment issues. Reviewed-by: Daniel Kiper 2021-03-02 Peter Jones util/mkimage: Reorder PE optional header fields set-up This makes the PE32 and PE32+ header fields set-up easier to follow by setting them closer to the initialization of their related sections. Reviewed-by: Daniel Kiper 2021-03-02 Peter Jones util/mkimage: Unify more of the PE32 and PE32+ header set-up There's quite a bit of code duplication in the code that sets the optional header for PE32 and PE32+. The two are very similar with the exception of a few fields that have type grub_uint64_t instead of grub_uint32_t. Factor out the common code and add a PE_OHDR() macro that simplifies the set-up and make the code more readable. Reviewed-by: Daniel Kiper 2021-03-02 Peter Jones util/mkimage: Always use grub_host_to_target32() to initialize PE stack and heap stuff This change does not impact final result of initialization itself. However, it eases PE code unification in subsequent patches. Reviewed-by: Daniel Kiper 2021-03-02 Peter Jones util/mkimage: Use grub_host_to_target32() instead of grub_cpu_to_le32() The latter doesn't take into account the target image endianness. There is a grub_cpu_to_le32_compile_time() but no compile time variant for function grub_host_to_target32(). So, let's keep using the other one for this case. Reviewed-by: Daniel Kiper 2021-03-02 Javier Martinez Canillas util/mkimage: Remove unused code to add BSS section The code is compiled out so there is no reason to keep it. Additionally, don't set bss_size field since we do not add a BSS section. Reviewed-by: Daniel Kiper 2021-03-02 Chris Coulson kern/efi: Add initial stack protector implementation It works only on UEFI platforms but can be quite easily extended to others architectures and platforms if needed. Reviewed-by: Marco A Benatto Reviewed-by: Javier Martinez Canillas 2021-03-02 Chris Coulson kern/parser: Fix a stack buffer overflow grub_parser_split_cmdline() expands variable names present in the supplied command line in to their corresponding variable contents and uses a 1 kiB stack buffer for temporary storage without sufficient bounds checking. If the function is called with a command line that references a variable with a sufficiently large payload, it is possible to overflow the stack buffer via tab completion, corrupt the stack frame and potentially control execution. Fixes: CVE-2020-27749 Reported-by: Chris Coulson Reviewed-by: Daniel Kiper 2021-03-02 Chris Coulson kern/buffer: Add variable sized heap buffer Add a new variable sized heap buffer type (grub_buffer_t) with simple operations for appending data, accessing the data and maintaining a read cursor. Reviewed-by: Daniel Kiper 2021-03-02 Chris Coulson kern/parser: Refactor grub_parser_split_cmdline() cleanup Introduce a common function epilogue used for cleaning up on all return paths, which will simplify additional error handling to be introduced in a subsequent commit. Reviewed-by: Daniel Kiper 2021-03-02 Chris Coulson kern/parser: Introduce terminate_arg() helper process_char() and grub_parser_split_cmdline() use similar code for terminating the most recent argument. Add a helper function for this. Reviewed-by: Daniel Kiper 2021-03-02 Chris Coulson kern/parser: Introduce process_char() helper grub_parser_split_cmdline() iterates over each command line character. In order to add error checking and to simplify the subsequent error handling, split the character processing in to a separate function. Reviewed-by: Daniel Kiper 2021-03-02 Chris Coulson kern/parser: Fix a memory leak The getline() function supplied to grub_parser_split_cmdline() returns a newly allocated buffer and can be called multiple times, but the returned buffer is never freed. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens fs/btrfs: Squash some uninitialized reads We need to check errors before calling into a function that uses the result. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens fs/btrfs: Validate the number of stripes/parities in RAID5/6 This prevents a divide by zero if nstripes == nparities, and also prevents propagation of invalid values if nstripes ends up less than nparities. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens disk/lvm: Do not allow a LV to be it's own segment's node's LV This prevents infinite recursion in the diskfilter verification code. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens disk/lvm: Sanitize rlocn->offset to prevent wild read rlocn->offset is read directly from disk and added to the metadatabuf pointer to create a pointer to a block of metadata. It's a 64-bit quantity so as long as you don't overflow you can set subsequent pointers to point anywhere in memory. Require that rlocn->offset fits within the metadata buffer size. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens disk/lvm: Do not overread metadata We could reach the end of valid metadata and not realize, leading to some buffer overreads. Check if we have reached the end and bail. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens disk/lvm: Do not crash if an expected string is not found Clean up a bunch of cases where we could have strstr() fail and lead to us dereferencing NULL. We'll still leak memory in some cases (loops don't clean up allocations from earlier iterations if a later iteration fails) but at least we're not crashing. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens disk/lvm: Bail on missing PV list There's an if block for the presence of "physical_volumes {", but if that block is absent, then p remains NULL and a NULL-deref will result when looking for logical volumes. It doesn't seem like LVM makes sense without physical volumes, so error out rather than crashing. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens disk/lvm: Don't blast past the end of the circular metadata buffer This catches at least some OOB reads, and it's possible I suppose that if 2 * mda_size is less than GRUB_LVM_MDA_HEADER_SIZE it might catch some OOB writes too (although that hasn't showed up as a crash in fuzzing yet). It's a bit ugly and I'd appreciate better suggestions. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens disk/lvm: Don't go beyond the end of the data we read from disk We unconditionally trusted offset_xl from the LVM label header, even if it told us that the PV header/disk locations were way off past the end of the data we read from disk. Require that the offset be sane, fixing an OOB read and crash. Fixes: CID 314367, CID 314371 Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens io/gzio: Zero gzio->tl/td in init_dynamic_block() if huft_build() fails If huft_build() fails, gzio->tl or gzio->td could contain pointers that are no longer valid. Zero them out. This prevents a double free when grub_gzio_close() comes through and attempts to free them again. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens io/gzio: Catch missing values in huft_build() and bail In huft_build(), "v" is a table of values in order of bit length. The code later (when setting up table entries in "r") assumes that all elements of this array corresponding to a code are initialized and less than N_MAX. However, it doesn't enforce this. With sufficiently manipulated inputs (e.g. from fuzzing), there can be elements of "v" that are not filled. Therefore a lookup into "e" or "d" will use an uninitialized value. This can lead to an invalid/OOB read on those values, often leading to a crash. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens io/gzio: Add init_dynamic_block() clean up if unpacking codes fails init_dynamic_block() didn't clean up gzio->tl and td in some error paths. This left td pointing to part of tl. Then in grub_gzio_close(), when tl was freed the storage for td would also be freed. The code then attempts to free td explicitly, performing a UAF and then a double free. Explicitly clean up tl and td in the error paths. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens io/gzio: Bail if gzio->tl/td is NULL This is an ugly fix that doesn't address why gzio->tl comes to be NULL. However, it seems to be sufficient to patch up a bunch of NULL derefs. It would be good to revisit this in future and see if we can have a cleaner solution that addresses some of the causes of the unexpected NULL pointers. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens fs/nilfs2: Properly bail on errors in grub_nilfs2_btree_node_lookup() We just introduced an error return in grub_nilfs2_btree_node_lookup(). Make sure the callers catch it. At the same time, make sure that grub_nilfs2_btree_node_lookup() always inits the index pointer passed to it. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens fs/nilfs2: Don't search children if provided number is too large NILFS2 reads the number of children a node has from the node. Unfortunately, that's not trustworthy. Check if it's beyond what the filesystem permits and reject it if so. This blocks some OOB reads. I'm not sure how controllable the read is and what could be done with invalidly read data later on. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens fs/nilfs2: Reject too-large keys NILFS2 has up to 7 keys, per the data structure. Do not permit array indices in excess of that. This catches some OOB reads. I don't know how controllable the invalidly read data is or if that could be used later in the program. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens fs/jfs: Catch infinite recursion It's possible with a fuzzed filesystem for JFS to keep getblk()-ing the same data over and over again, leading to stack exhaustion. Check if we'd be calling the function with exactly the same data as was passed in, and if so abort. I'm not sure what the performance impact of this is and am open to better ideas. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens fs/jfs: Limit the extents that getblk() can consider getblk() implicitly trusts that treehead->count is an accurate count of the number of extents. However, that value is read from disk and is not trustworthy, leading to OOB reads and crashes. I am not sure to what extent the data read from OOB can influence subsequent program execution. Require callers to pass in the maximum number of extents for which they have storage. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens fs/jfs: Do not move to leaf level if name length is negative Fuzzing JFS revealed crashes where a negative number would be passed to le_to_cpu16_copy(). There it would be cast to a large positive number and the copy would read and write off the end of the respective buffers. Catch this at the top as well as the bottom of the loop. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens fs/sfs: Fix over-read of root object name There's a read of the name of the root object that assumes that the name is nul-terminated within the root block. This isn't guaranteed - it seems SFS would require you to read multiple blocks to get a full name in general, but maybe that doesn't apply to the root object. Either way, figure out how much space is left in the root block and don't over-read it. This fixes some OOB reads. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens fs/hfs: Disable under lockdown HFS has issues such as infinite mutual recursion that are simply too complex to fix for such a legacy format. So simply do not permit it to be loaded under lockdown. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens fs/hfsplus: Don't use uninitialized data on corrupt filesystems Valgrind identified the following use of uninitialized data: ==2782220== Conditional jump or move depends on uninitialised value(s) ==2782220== at 0x42B364: grub_hfsplus_btree_search (hfsplus.c:566) ==2782220== by 0x42B21D: grub_hfsplus_read_block (hfsplus.c:185) ==2782220== by 0x42A693: grub_fshelp_read_file (fshelp.c:386) ==2782220== by 0x42C598: grub_hfsplus_read_file (hfsplus.c:219) ==2782220== by 0x42C598: grub_hfsplus_mount (hfsplus.c:330) ==2782220== by 0x42B8C5: grub_hfsplus_dir (hfsplus.c:958) ==2782220== by 0x4C1AE6: grub_fs_probe (fs.c:73) ==2782220== by 0x407C94: grub_ls_list_files (ls.c:186) ==2782220== by 0x407C94: grub_cmd_ls (ls.c:284) ==2782220== by 0x4D7130: grub_extcmd_dispatcher (extcmd.c:55) ==2782220== by 0x4045A6: execute_command (grub-fstest.c:59) ==2782220== by 0x4045A6: fstest (grub-fstest.c:433) ==2782220== by 0x4045A6: main (grub-fstest.c:772) ==2782220== Uninitialised value was created by a heap allocation ==2782220== at 0x483C7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==2782220== by 0x4C0305: grub_malloc (mm.c:42) ==2782220== by 0x42C21D: grub_hfsplus_mount (hfsplus.c:239) ==2782220== by 0x42B8C5: grub_hfsplus_dir (hfsplus.c:958) ==2782220== by 0x4C1AE6: grub_fs_probe (fs.c:73) ==2782220== by 0x407C94: grub_ls_list_files (ls.c:186) ==2782220== by 0x407C94: grub_cmd_ls (ls.c:284) ==2782220== by 0x4D7130: grub_extcmd_dispatcher (extcmd.c:55) ==2782220== by 0x4045A6: execute_command (grub-fstest.c:59) ==2782220== by 0x4045A6: fstest (grub-fstest.c:433) ==2782220== by 0x4045A6: main (grub-fstest.c:772) This happens when the process of reading the catalog file goes sufficiently wrong that there's an attempt to read the extent overflow file, which has not yet been loaded. Keep track of when the extent overflow file is fully loaded and refuse to use it before then. The load valgrind doesn't like is btree->nodesize, and that's then used to allocate a data structure. It looks like there are subsequently a lot of reads based on that pointer so OOB reads are likely, and indeed crashes (albeit difficult-to-replicate ones) have been observed in fuzzing. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens fs/hfsplus: Don't fetch a key beyond the end of the node Otherwise you get a wild pointer, leading to a bunch of invalid reads. Check it falls inside the given node. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens fs/fshelp: Catch impermissibly large block sizes in read helper A fuzzed HFS+ filesystem had log2blocksize = 22. This gave log2blocksize + GRUB_DISK_SECTOR_BITS = 31. 1 << 31 = 0x80000000, which is -1 as an int. This caused some wacky behavior later on in the function, leading to out-of-bounds writes on the destination buffer. Catch log2blocksize + GRUB_DISK_SECTOR_BITS >= 31. We could be stricter, but this is the minimum that will prevent integer size weirdness. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens term/gfxterm: Don't set up a font with glyphs that are too big Catch the case where we have a font so big that it causes the number of rows or columns to be 0. Currently we continue and allocate a virtual_screen.text_buffer of size 0. We then try to use that for glpyhs and things go badly. On the emu platform, malloc() may give us a valid pointer, in which case we'll access heap memory which we shouldn't. Alternatively, it may give us NULL, in which case we'll crash. For other platforms, if I understand grub_memalign() correctly, we will receive a valid but small allocation that we will very likely later overrun. Prevent the creation of a virtual screen that isn't at least 40 cols by 12 rows. This is arbitrary, but it seems that if your width or height is half a standard 80x24 terminal, you're probably going to struggle to read anything anyway. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens video/readers/jpeg: Don't decode data before start of stream When a start of stream marker is encountered, we call grub_jpeg_decode_sos() which allocates space for a bitmap. When a restart marker is encountered, we call grub_jpeg_decode_data() which then fills in that bitmap. If we get a restart marker before the start of stream marker, we will attempt to write to a bitmap_ptr that hasn't been allocated. Catch this and bail out. This fixes an attempt to write to NULL. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens video/readers/jpeg: Catch OOB reads/writes in grub_jpeg_decode_du() The key line is: du[jpeg_zigzag_order[pos]] = val * (int) data->quan_table[qt][pos]; jpeg_zigzag_order is grub_uint8_t[64]. I don't understand JPEG decoders quite well enough to explain what's going on here. However, I observe sometimes pos=64, which leads to an OOB read of the jpeg_zigzag_order global then an OOB write to du. That leads to various unpleasant memory corruption conditions. Catch where pos >= ARRAY_SIZE(jpeg_zigzag_order) and bail. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens video/readers/jpeg: Catch files with unsupported quantization or Huffman tables Our decoder only supports 2 quantization tables. If a file asks for a quantization table with index > 1, reject it. Similarly, our decoder only supports 4 Huffman tables. If a file asks for a Huffman table with index > 3, reject it. This fixes some out of bounds reads. It's not clear what degree of control over subsequent execution could be gained by someone who can carefully set up the contents of memory before loading an invalid JPEG file. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens kern/misc: Always set *end in grub_strtoull() Currently, if there is an error in grub_strtoull(), *end is not set. This differs from the usual behavior of strtoull(), and also means that some callers may use an uninitialized value for *end. Set *end unconditionally. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens commands/menuentry: Fix quoting in setparams_prefix() Commit 9acdcbf32542 (use single quotes in menuentry setparams command) says that expressing a quoted single quote will require 3 characters. It actually requires (and always did require!) 4 characters: str: a'b => a'\''b len: 3 => 6 (2 for the letters + 4 for the quote) This leads to not allocating enough memory and thus out of bounds writes that have been observed to cause heap corruption. Allocate 4 bytes for each single quote. Commit 22e7dbb2bb81 (Fix quoting in legacy parser.) does the same quoting, but it adds 3 as extra overhead on top of the single byte that the quote already needs. So it's correct. Fixes: 9acdcbf32542 (use single quotes in menuentry setparams command) Fixes: CVE-2021-20233 Reported-by: Daniel Axtens Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens script/execute: Don't crash on a "for" loop with no items The following crashes the parser: for x in; do 0 done This is because grub_script_arglist_to_argv() doesn't consider the possibility that arglist is NULL. Catch that explicitly. This avoids a NULL pointer dereference. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens lib/arg: Block repeated short options that require an argument Fuzzing found the following crash: search -hhhhhhhhhhhhhf We didn't allocate enough option space for 13 hints because the allocation code counts the number of discrete arguments (i.e. argc). However, the shortopt parsing code will happily keep processing a combination of short options without checking if those short options require an argument. This means you can easily end writing past the allocated option space. This fixes a OOB write which can cause heap corruption. Fixes: CVE-2021-20225 Reported-by: Daniel Axtens Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens script/execute: Avoid crash when using "$#" outside a function scope "$#" represents the number of arguments to a function. It is only defined in a function scope, where "scope" is non-NULL. Currently, if we attempt to evaluate "$#" outside a function scope, "scope" will be NULL and we will crash with a NULL pointer dereference. Do not attempt to count arguments for "$#" if "scope" is NULL. This will result in "$#" being interpreted as an empty string if evaluated outside a function scope. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens commands/ls: Require device_name is not NULL before printing This can be triggered with: ls -l (0 0*) and causes a NULL deref in grub_normal_print_device_info(). I'm not sure if there's any implication with the IEEE 1275 platform. Reviewed-by: Daniel Kiper 2021-03-02 Daniel Axtens script/execute: Fix NULL dereference in grub_script_execute_cmdline() Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny util/glue-efi: Fix incorrect use of a possibly negative value It is possible for the ftell() function to return a negative value, although it is fairly unlikely here, we should be checking for a negative value before we assign it to an unsigned value. Fixes: CID 73744 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny util/grub-editenv: Fix incorrect casting of a signed value The return value of ftell() may be negative (-1) on error. While it is probably unlikely to occur, we should not blindly cast to an unsigned value without first testing that it is not negative. Fixes: CID 73856 Reviewed-by: Daniel Kiper 2021-03-02 Daniel Kiper util/grub-install: Fix NULL pointer dereferences Two grub_device_open() calls does not have associated NULL checks for returned values. Fix that and appease the Coverity. Fixes: CID 314583 Reviewed-by: Javier Martinez Canillas 2021-03-02 Paulo Flabiano Smorigo loader/xnu: Check if pointer is NULL before using it Fixes: CID 73654 Reviewed-by: Daniel Kiper 2021-03-02 Marco A Benatto loader/xnu: Free driverkey data when an error is detected in grub_xnu_writetree_toheap() ... to avoid memory leaks. Fixes: CID 96640 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny loader/xnu: Fix memory leak The code here is finished with the memory stored in name, but it only frees it if there curvalue is valid, while it could actually free it regardless. The fix is a simple relocation of the grub_free() to before the test of curvalue. Fixes: CID 96646 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny loader/bsd: Check for NULL arg up-front The code in the next block suggests that it is possible for .set to be true but .arg may still be NULL. This code assumes that it is never NULL, yet later is testing if it is NULL - that is inconsistent. So we should check first if .arg is not NULL, and remove this check that is being flagged by Coverity since it is no longer required. Fixes: CID 292471 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny gfxmenu/gui_list: Remove code that coverity is flagging as dead The test of value for NULL before calling grub_strdup() is not required, since the if condition prior to this has already tested for value being NULL and cannot reach this code if it is. Fixes: CID 73659 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny video/readers/jpeg: Test for an invalid next marker reference from a jpeg file While it may never happen, and potentially could be caught at the end of the function, it is worth checking up front for a bad reference to the next marker just in case of a maliciously crafted file being provided. Fixes: CID 73694 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny video/fb/video_fb: Fix possible integer overflow It is minimal possibility that the values being used here will overflow. So, change the code to use the safemath function grub_mul() to ensure that doesn't happen. Fixes: CID 73761 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny video/fb/video_fb: Fix multiple integer overflows The calculation of the unsigned 64-bit value is being generated by multiplying 2, signed or unsigned, 32-bit integers which may overflow before promotion to unsigned 64-bit. Fix all of them. Fixes: CID 73703, CID 73767, CID 73833 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny video/fb/fbfill: Fix potential integer overflow The multiplication of 2 unsigned 32-bit integers may overflow before promotion to unsigned 64-bit. We should ensure that the multiplication is done with overflow detection. Additionally, use grub_sub() for subtraction. Fixes: CID 73640, CID 73697, CID 73702, CID 73823 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny video/efi_gop: Remove unnecessary return value of grub_video_gop_fill_mode_info() The return value of grub_video_gop_fill_mode_info() is never able to be anything other than GRUB_ERR_NONE. So, rather than continue to return a value and checking it each time, it is more correct to redefine the function to not return anything and remove checks of its return value altogether. Fixes: CID 96701 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny commands/probe: Fix a resource leak when probing disks Every other return statement in this code is calling grub_device_close() to clean up dev before returning. This one should do that too. Fixes: CID 292443 Reviewed-by: Daniel Kiper 2021-03-02 Chris Coulson commands/hashsum: Fix a memory leak check_list() uses grub_file_getline(), which allocates a buffer. If the hash list file contains invalid lines, the function leaks this buffer when it returns an error. Fixes: CID 176635 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny normal/completion: Fix leaking of memory when processing a completion It is possible for the code to reach the end of the function without freeing the memory allocated to argv and argc still to be 0. We should always call grub_free(argv). The grub_free() will handle a NULL argument correctly if it reaches that code without the memory being allocated. Fixes: CID 96672 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny syslinux: Fix memory leak while parsing In syslinux_parse_real() the 2 points where return is being called didn't release the memory stored in buf which is no longer required. Fixes: CID 176634 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny libgcrypt/mpi: Fix possible NULL dereference The code in gcry_mpi_scan() assumes that buffer is not NULL, but there is no explicit check for that, so we add one. Fixes: CID 73757 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny libgcrypt/mpi: Fix possible unintended sign extension The array of unsigned char gets promoted to a signed 32-bit int before it is finally promoted to a size_t. There is the possibility that this may result in the signed-bit being set for the intermediate signed 32-bit int. We should ensure that the promotion is to the correct type before we bitwise-OR the values. Fixes: CID 96697 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny affs: Fix memory leaks The node structure reference is being allocated but not freed if it reaches the end of the function. If any of the hooks had returned a non-zero value, then node would have been copied in to the context reference, but otherwise node is not stored and should be freed. Similarly, the call to grub_affs_create_node() replaces the allocated memory in node with a newly allocated structure, leaking the existing memory pointed by node. Finally, when dir->parent is set, then we again replace node with newly allocated memory, which seems unnecessary when we copy in the values from dir->parent immediately after. Fixes: CID 73759 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny zfsinfo: Correct a check for error allocating memory While arguably the check for grub_errno is correct, we should really be checking the return value from the function since it is always possible that grub_errno was set elsewhere, making this code behave incorrectly. Fixes: CID 73668 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny zfs: Fix possible integer overflows In all cases the problem is that the value being acted upon by a left-shift is a 32-bit number which is then being used in the context of a 64-bit number. To avoid overflow we ensure that the number being shifted is 64-bit before the shift is done. Fixes: CID 73684, CID 73695, CID 73764 Reviewed-by: Daniel Kiper 2021-03-02 Paulo Flabiano Smorigo zfs: Fix resource leaks while constructing path There are several exit points in dnode_get_path() that are causing possible memory leaks. In the while(1) the correct exit mechanism should not be to do a direct return, but to instead break out of the loop, setting err first if it is not already set. The reason behind this is that the dnode_path is a linked list, and while doing through this loop, it is being allocated and built up - the only way to correctly unravel it is to traverse it, which is what is being done at the end of the function outside of the loop. Several of the existing exit points correctly did a break, but not all so this change makes that more consistent and should resolve the leaking of memory as found by Coverity. Fixes: CID 73741 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny zfs: Fix possible negative shift operation While it is possible for the return value from zfs_log2() to be zero (0), it is quite unlikely, given that the previous assignment to blksz is shifted up by SPA_MINBLOCKSHIFT (9) before 9 is subtracted at the assignment to epbs. But, while unlikely during a normal operation, it may be that a carefully crafted ZFS filesystem could result in a zero (0) value to the dn_datalbkszsec field, which means that the shift left does nothing and assigns zero (0) to blksz, resulting in a negative epbs value. Fixes: CID 73608 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny hfsplus: Check that the volume name length is valid HFS+ documentation suggests that the maximum filename and volume name is 255 Unicode characters in length. So, when converting from big-endian to little-endian, we should ensure that the name of the volume has a length that is between 0 and 255, inclusive. Fixes: CID 73641 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny disk/cryptodisk: Fix potential integer overflow The encrypt and decrypt functions expect a grub_size_t. So, we need to ensure that the constant bit shift is using grub_size_t rather than unsigned int when it is performing the shift. Fixes: CID 307788 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny disk/ldm: Fix memory leak on uninserted lv references The problem here is that the memory allocated to the variable lv is not yet inserted into the list that is being processed at the label fail2. As we can already see at line 342, which correctly frees lv before going to fail2, we should also be doing that at these earlier jumps to fail2. Fixes: CID 73824 Reviewed-by: Daniel Kiper 2021-03-02 Paulo Flabiano Smorigo disk/ldm: If failed then free vg variable too Fixes: CID 73809 Reviewed-by: Daniel Kiper 2021-03-02 Marco A Benatto disk/ldm: Make sure comp data is freed before exiting from make_vg() Several error handling paths in make_vg() do not free comp data before jumping to fail2 label and returning from the function. This will leak memory. So, let's fix all issues of that kind. Fixes: CID 73804 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny kern/partition: Check for NULL before dereferencing input string There is the possibility that the value of str comes from an external source and continuing to use it before ever checking its validity is wrong. So, needs fixing. Additionally, drop unneeded part initialization. Fixes: CID 292444 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny zstd: Initialize seq_t structure fully While many compilers will initialize this to zero, not all will, so it is better to be sure that fields not being explicitly set are at known values, and there is code that checks this fields value elsewhere in the code. Fixes: CID 292440 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny io/lzopio: Resolve unnecessary self-assignment errors These 2 assignments are unnecessary since they are just assigning to themselves. Fixes: CID 73643 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny gnulib/regcomp: Fix uninitialized re_token This issue has been fixed in the latest version of gnulib, so to maintain consistency, I've backported that change rather than doing something different. Fixes: CID 73828 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny gnulib/regexec: Fix possible null-dereference It appears to be possible that the mctx->state_log field may be NULL, and the name of this function, clean_state_log_if_needed(), suggests that it should be checking that it is valid to be cleaned before assuming that it does. Fixes: CID 86720 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny gnulib/argp-help: Fix dereference of a possibly NULL state All other instances of call to __argp_failure() where there is a dgettext() call is first checking whether state is NULL before attempting to dereference it to get the root_argp->argp_domain. Fixes: CID 292436 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny gnulib/regcomp: Fix uninitialized token structure The code is assuming that the value of br_token.constraint was initialized to zero when it wasn't. While some compilers will ensure that, not all do, so it is better to fix this explicitly than leave it to chance. Fixes: CID 73749 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny gnulib/regexec: Resolve unused variable This is a really minor issue where a variable is being assigned to but not checked before it is overwritten again. The reason for this issue is that we are not building with DEBUG set and this in turn means that the assert() that reads the value of the variable match_last is being processed out. The solution, move the assignment to match_last in to an ifdef DEBUG too. Fixes: CID 292459 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny kern/efi/mm: Fix possible NULL pointer dereference The model of grub_efi_get_memory_map() is that if memory_map is NULL, then the purpose is to discover how much memory should be allocated to it for the subsequent call. The problem here is that with grub_efi_is_finished set to 1, there is no check at all that the function is being called with a non-NULL memory_map. While this MAY be true, we shouldn't assume it. The solution to this is to behave as expected, and if memory_map is NULL, then don't try to use it and allow memory_map_size to be filled in, and return 0 as is done later in the code if the buffer is too small (or NULL). Additionally, drop unneeded ret = 1. Fixes: CID 96632 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny kern/efi: Fix memory leak on failure Free the memory allocated to name before returning on failure. Fixes: CID 296222 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny kern/parser: Fix resource leak if argc == 0 After processing the command-line yet arriving at the point where we are setting argv, we are allocating memory, even if argc == 0, which makes no sense since we never put anything into the allocated argv. The solution is to simply return that we've successfully processed the arguments but that argc == 0, and also ensure that argv is NULL when we're not allocating anything in it. There are only 2 callers of this function, and both are handling a zero value in argc assuming nothing is allocated in argv. Fixes: CID 96680 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny net/tftp: Fix dangling memory pointer The static code analysis tool, Parfait, reported that the valid of file->data was left referencing memory that was freed by the call to grub_free(data) where data was initialized from file->data. To ensure that there is no unintentional access to this memory referenced by file->data we should set the pointer to NULL. Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny net/net: Fix possible dereference to of a NULL pointer It is always possible that grub_zalloc() could fail, so we should check for a NULL return. Otherwise we run the risk of dereferencing a NULL pointer. Fixes: CID 296221 Reviewed-by: Daniel Kiper 2021-03-02 Darren Kenny mmap: Fix memory leak when iterating over mapped memory When returning from grub_mmap_iterate() the memory allocated to present is not being released causing it to leak. Fixes: CID 96655 Reviewed-by: Daniel Kiper 2021-03-02 Javier Martinez Canillas usb: Avoid possible out-of-bound accesses caused by malicious devices The maximum number of configurations and interfaces are fixed but there is no out-of-bound checking to prevent a malicious USB device to report large values for these and cause accesses outside the arrays' memory. Fixes: CVE-2020-25647 Reported-by: Joseph Tartaro Reported-by: Ilja Van Sprundel Reviewed-by: Daniel Kiper 2021-03-02 Javier Martinez Canillas dl: Only allow unloading modules that are not dependencies When a module is attempted to be removed its reference counter is always decremented. This means that repeated rmmod invocations will cause the module to be unloaded even if another module depends on it. This may lead to a use-after-free scenario allowing an attacker to execute arbitrary code and by-pass the UEFI Secure Boot protection. While being there, add the extern keyword to some function declarations in that header file. Fixes: CVE-2020-25632 Reported-by: Chris Coulson Reviewed-by: Daniel Kiper 2021-03-02 Javier Martinez Canillas docs: Document the cutmem command The command is not present in the docs/grub.texi user documentation. Reported-by: Daniel Kiper Reviewed-by: Javier Martinez Canillas 2021-03-02 Javier Martinez Canillas loader/xnu: Don't allow loading extension and packages when locked down The shim_lock verifier validates the XNU kernels but no its extensions and packages. Prevent these to be loaded when the GRUB is locked down. Reviewed-by: Daniel Kiper 2021-03-02 Javier Martinez Canillas gdb: Restrict GDB access when locked down The gdbstub* commands allow to start and control a GDB stub running on local host that can be used to connect from a remote debugger. Restrict this functionality when the GRUB is locked down. Reviewed-by: Daniel Kiper 2021-03-02 Javier Martinez Canillas commands/hdparm: Restrict hdparm command when locked down The command can be used to get/set ATA disk parameters. Some of these can be dangerous since change the disk behavior. Restrict it when locked down. Reviewed-by: Daniel Kiper 2021-03-02 Javier Martinez Canillas commands/setpci: Restrict setpci command when locked down This command can set PCI devices register values, which makes it dangerous in a locked down configuration. Restrict it so can't be used on this setup. Reviewed-by: Daniel Kiper 2021-03-02 Javier Martinez Canillas commands: Restrict commands that can load BIOS or DT blobs when locked down There are some more commands that should be restricted when the GRUB is locked down. Following is the list of commands and reasons to restrict: * fakebios: creates BIOS-like structures for backward compatibility with existing OSes. This should not be allowed when locked down. * loadbios: reads a BIOS dump from storage and loads it. This action should not be allowed when locked down. * devicetree: loads a Device Tree blob and passes it to the OS. It replaces any Device Tree provided by the firmware. This also should not be allowed when locked down. Reviewed-by: Daniel Kiper 2021-03-02 Javier Martinez Canillas mmap: Don't register cutmem and badram commands when lockdown is enforced The cutmem and badram commands can be used to remove EFI memory regions and potentially disable the UEFI Secure Boot. Prevent the commands to be registered if the GRUB is locked down. Fixes: CVE-2020-27779 Reported-by: Teddy Reed Reviewed-by: Daniel Kiper 2021-03-02 Javier Martinez Canillas acpi: Don't register the acpi command when locked down The command is not allowed when lockdown is enforced. Otherwise an attacker can instruct the GRUB to load an SSDT table to overwrite the kernel lockdown configuration and later load and execute unsigned code. Fixes: CVE-2020-14372 Reported-by: Máté Kukri Reviewed-by: Daniel Kiper 2021-03-02 Javier Martinez Canillas efi: Use grub_is_lockdown() instead of hardcoding a disabled modules list Now the GRUB can check if it has been locked down and this can be used to prevent executing commands that can be utilized to circumvent the UEFI Secure Boot mechanisms. So, instead of hardcoding a list of modules that have to be disabled, prevent the usage of commands that can be dangerous. This not only allows the commands to be disabled on other platforms, but also properly separate the concerns. Since the shim_lock verifier logic should be only about preventing to run untrusted binaries and not about defining these kind of policies. Reviewed-by: Daniel Kiper 2021-03-02 Javier Martinez Canillas efi: Lockdown the GRUB when the UEFI Secure Boot is enabled If the UEFI Secure Boot is enabled then the GRUB must be locked down to prevent executing code that can potentially be used to subvert its verification mechanisms. Reviewed-by: Daniel Kiper 2021-03-02 Javier Martinez Canillas kern/lockdown: Set a variable if the GRUB is locked down It may be useful for scripts to determine whether the GRUB is locked down or not. Add the lockdown variable which is set to "y" when the GRUB is locked down. Suggested-by: Dimitri John Ledkov Reviewed-by: Daniel Kiper 2021-03-02 Javier Martinez Canillas kern: Add lockdown support When the GRUB starts on a secure boot platform, some commands can be used to subvert the protections provided by the verification mechanism and could lead to booting untrusted system. To prevent that situation, allow GRUB to be locked down. That way the code may check if GRUB has been locked down and further restrict the commands that are registered or what subset of their functionality could be used. The lockdown support adds the following components: * The grub_lockdown() function which can be used to lockdown GRUB if, e.g., UEFI Secure Boot is enabled. * The grub_is_lockdown() function which can be used to check if the GRUB was locked down. * A verifier that flags OS kernels, the GRUB modules, Device Trees and ACPI tables as GRUB_VERIFY_FLAGS_DEFER_AUTH to defer verification to other verifiers. These files are only successfully verified if another registered verifier returns success. Otherwise, the whole verification process fails. For example, PE/COFF binaries verification can be done by the shim_lock verifier which validates the signatures using the shim_lock protocol. However, the verification is not deferred directly to the shim_lock verifier. The shim_lock verifier is hooked into the verification process instead. * A set of grub_{command,extcmd}_lockdown functions that can be used by code registering command handlers, to only register unsafe commands if the GRUB has not been locked down. Reviewed-by: Daniel Kiper 2021-03-02 Marco A Benatto efi: Move the shim_lock verifier to the GRUB core Move the shim_lock verifier from its own module into the core image. The Secure Boot lockdown mechanism has the intent to prevent the load of any unsigned code or binary when Secure Boot is enabled. The reason is that GRUB must be able to prevent executing untrusted code if UEFI Secure Boot is enabled, without depending on external modules. Reviewed-by: Daniel Kiper 2021-03-02 Marco A Benatto verifiers: Move verifiers API to kernel image Move verifiers API from a module to the kernel image, so it can be used there as well. There are no functional changes in this patch. Reviewed-by: Daniel Kiper 2020-12-18 Glenn Washburn docs: Add documentation of disk size limitations Document the artificially imposed 1 EiB disk size limit and size limitations with LUKS volumes. Fix a few punctuation issues. Reviewed-by: Daniel Kiper 2020-12-18 Glenn Washburn luks2: Use grub_log2ull() to calculate log_sector_size and improve readability Reviewed-by: Daniel Kiper misc: Add grub_log2ull() macro for calculating log base 2 of 64-bit integers Reviewed-by: Daniel Kiper 2020-12-18 Glenn Washburn mips: Enable __clzdi2() This patch is similar to commit 9dab2f51e (sparc: Enable __clzsi2() and __clzdi2()) but for MIPS target and __clzdi2() only, __clzsi2() was already enabled. Suggested-by: Daniel Kiper Reviewed-by: Daniel Kiper 2020-12-18 Glenn Washburn luks2: Better error handling when setting up the cryptodisk Do some sanity checking on data coming from the LUKS2 header. If segment.size is "dynamic", verify that the offset is not past the end of disk. Otherwise, check for errors from grub_strtoull() when converting segment size from string. If a GRUB_ERR_BAD_NUMBER error was returned, then the string was not a valid parsable number, so skip the key. If GRUB_ERR_OUT_OF_RANGE was returned, then there was an overflow in converting to a 64-bit unsigned integer. So this could be a very large disk (perhaps large RAID array). In this case skip the key too. Additionally, enforce some other limits and fail if needed. Reviewed-by: Daniel Kiper 2020-12-18 Glenn Washburn luks2: Do not handle disks of size GRUB_DISK_SIZE_UNKNOWN for now Check to make sure that source disk has a known size. If not, print a message and return error. There are 4 cases where GRUB_DISK_SIZE_UNKNOWN is set (biosdisk, obdisk, ofdisk, and uboot), and in all those cases processing continues. So this is probably a bit conservative. However, 3 of the cases seem pathological, and the other, biosdisk, happens when booting from a CD-ROM. Since I doubt booting from a LUKS2 volume on a CD-ROM is a big use case, we'll error until someone complains. Reviewed-by: Daniel Kiper 2020-12-18 Glenn Washburn luks2: Convert to crypt sectors from GRUB native sectors The function grub_disk_native_sectors(source) returns the number of sectors of source in GRUB native (512-byte) sectors, not source sized sectors. So the conversion needs to use GRUB_DISK_SECTOR_BITS, the GRUB native sector size. Reviewed-by: Daniel Kiper 2020-12-12 Glenn Washburn luks2: Error check segment.sector_size Reviewed-by: Daniel Kiper 2020-12-12 Glenn Washburn cryptodisk: Properly handle non-512 byte sized sectors By default, dm-crypt internally uses an IV that corresponds to 512-byte sectors, even when a larger sector size is specified. What this means is that when using a larger sector size, the IV is incremented every sector. However, the amount the IV is incremented is the number of 512 byte blocks in a sector (i.e. 8 for 4K sectors). Confusingly the IV does not correspond to the number of, for example, 4K sectors. So each 512 byte cipher block in a sector will be encrypted with the same IV and the IV will be incremented afterwards by the number of 512 byte cipher blocks in the sector. There are some encryption utilities which do it the intuitive way and have the IV equal to the sector number regardless of sector size (ie. the fifth sector would have an IV of 4 for each cipher block). And this is supported by dm-crypt with the iv_large_sectors option and also cryptsetup as of 2.3.3 with the --iv-large-sectors, though not with LUKS headers (only with --type plain). However, support for this has not been included as grub does not support plain devices right now. One gotcha here is that the encrypted split keys are encrypted with a hard- coded 512-byte sector size. So even if your data is encrypted with 4K sector sizes, the split key encrypted area must be decrypted with a block size of 512 (ie the IV increments every 512 bytes). This made these changes less aesthetically pleasing than desired. Reviewed-by: Daniel Kiper 2020-12-12 Glenn Washburn luks2: grub_cryptodisk_t->total_sectors is the max number of device native sectors We need to convert the sectors from the size of the underlying device to the cryptodisk sector size; segment.size is in bytes which need to be converted to cryptodisk sectors as well. Also, removed an empty statement. Reviewed-by: Daniel Kiper 2020-12-12 Glenn Washburn cryptodisk: Add macros GRUB_TYPE_U_MAX/MIN(type) to replace literals Add GRUB_TYPE_U_MAX/MIN(type) macros to get the max/min values for an unsigned number with size of type. Reviewed-by: Daniel Kiper 2020-12-12 Glenn Washburn cryptodisk: Add macro GRUB_TYPE_BITS() to replace some literals The new macro GRUB_TYPE_BITS(type) returns the number of bits allocated for type. Reviewed-by: Daniel Kiper 2020-12-12 Glenn Washburn luks2: Add string "index" to user strings using a json index This allows error messages to be more easily distinguishable between indexes and slot keys. The former include the string "index" in the error/debug string, and the later are surrounded in quotes. Reviewed-by: Daniel Kiper 2020-12-12 Glenn Washburn luks2: Rename json index variables to names that they are obviously json indexes Reviewed-by: Daniel Kiper 2020-12-12 Glenn Washburn luks2: Use more intuitive object name instead of json index in user messages Use the object name in the json array rather than the 0 based index in the json array for keyslots, segments, and digests. This is less confusing for the end user. For example, say you have a LUKS2 device with a key in slot 1 and slot 4. When using the password for slot 4 to unlock the device, the messages using the index of the keyslot will mention keyslot 1 (its a zero-based index). Furthermore, with this change the keyslot number will align with the number used to reference the keyslot when using the --key-slot argument to cryptsetup. Reviewed-by: Daniel Kiper 2020-12-12 Glenn Washburn luks2: Add idx member to struct grub_luks2_keyslot/segment/digest This allows code using these structs to know the named key associated with these json data structures. In the future we can use these to provide better error messages to the user. Get rid of idx local variable in luks2_get_keyslot() which was overloaded to be used for both keyslot and segment slot keys. Reviewed-by: Daniel Kiper 2020-12-12 Glenn Washburn luks2: Make sure all fields of output argument in luks2_parse_digest() are written to We should assume that the output argument "out" is uninitialized and could have random data. So, make sure to initialize the segments and keyslots bit fields because potentially not all bits of those fields are written to. Otherwise, the digest could say it belongs to keyslots and segments that it does not. Reviewed-by: Patrick Steinhardt Reviewed-by: Daniel Kiper 2020-12-12 Glenn Washburn luks2: Remove unused argument in grub_error() call Reviewed-by: Patrick Steinhardt Reviewed-by: Daniel Kiper luks2: Convert 8 spaces to tabs Reviewed-by: Patrick Steinhardt Reviewed-by: Daniel Kiper 2020-12-12 Glenn Washburn misc: Add parentheses around ALIGN_UP() and ALIGN_DOWN() arguments This ensures that expected order of operations is preserved when arguments are expressions. Reviewed-by: Patrick Steinhardt Reviewed-by: Daniel Kiper 2020-12-12 Glenn Washburn disk: Rename grub_disk_get_size() to grub_disk_native_sectors() The function grub_disk_get_size() is confusingly named because it actually returns a sector count where the sectors are sized in the GRUB native sector size. Rename to something more appropriate. Suggested-by: Daniel Kiper Reviewed-by: Patrick Steinhardt Reviewed-by: Daniel Kiper 2020-12-12 Glenn Washburn loopback: Do not automaticaly replace existing loopback dev, error instead If there is a loopback device with the same name as the one to be created, instead of closing the old one and replacing it with the new one, return an error instead. If the loopback device was created, its probably being used by something and just replacing it may cause GRUB to crash unexpectedly. This fixes obvious problems like "loopback d (d)/somefile". Its not too onerous to force the user to delete the loopback first with the "-d" switch. Reviewed-by: Daniel Kiper 2020-12-12 Glenn Washburn disk: Move hardcoded max disk size literal to a GRUB_DISK_MAX_SECTORS in disk.h There is a hardcoded maximum disk size that can be read or written from, currently set at 1 EiB in grub_disk_adjust_range(). Move the literal into a macro in disk.h, so our assumptions are more visible. This hard coded limit does not prevent using larger disks, just GRUB won't read/write past the limit. The comment accompanying this restriction didn't quite make sense to me, so its been modified too. Reviewed-by: Daniel Kiper 2020-12-12 Glenn Washburn fs: Fix block lists not being able to address to end of disk sometimes When checking if a block list goes past the end of the disk, make sure the total size of the disk is in GRUB native sector sizes, otherwise there will be blocks at the end of the disk inaccessible by block lists. Reviewed-by: Daniel Kiper 2020-12-12 Vladimir Serbinenko mbr: Document new limitations on MBR gap support Reviewed-by: Daniel Kiper 2020-12-12 Vladimir Serbinenko mbr: Warn if MBR gap is small and user uses advanced modules We don't want to support small MBR gap in pair with anything but the simplest config of biosdisk + part_msdos + simple filesystem. In this path "simple filesystems" are all current filesystems except ZFS and Btrfs. Reviewed-by: Daniel Kiper 2020-12-12 Tianjia Zhang efi/tpm: Extract duplicate code into independent functions Part of the code logic for processing the return value of efi log_extend_event is repetitive and complicated. Extract the repetitive code into an independent function. Reviewed-by: Daniel Kiper 2020-12-12 Tianjia Zhang efi/tpm: Add debug information for device protocol and eventlog Add a number of debug logs to the tpm module. The condition tag for opening debugging is "tpm". On TPM machines, this will bring great convenience to diagnosis and debugging. Reviewed-by: Daniel Kiper 2020-12-12 Daniel Kiper loader/linux: Report the UEFI Secure Boot status to the Linux kernel Now that the GRUB has a grub_efi_get_secureboot() function to check the UEFI Secure Boot status, use it to report that to the Linux kernel. Reviewed-by: Daniel Kiper 2020-12-12 Javier Martinez Canillas efi: Only register shim_lock verifier if shim_lock protocol is found and SB enabled The shim_lock module registers a verifier to call shim's verify, but the handler is registered even when the shim_lock protocol was not installed. This doesn't cause a NULL pointer dereference in shim_lock_write() because the shim_lock_init() function just returns GRUB_ERR_NONE if sl isn't set. But in that case there's no point to even register the shim_lock verifier since won't do anything. Additionally, it is only useful when Secure Boot is enabled. Finally, don't assume that the shim_lock protocol will always be present when the shim_lock_write() function is called, and check for it on every call to this function. Reported-by: Michael Chang Reported-by: Peter Jones Reviewed-by: Daniel Kiper 2020-12-11 Daniel Kiper efi: Add secure boot detection Introduce grub_efi_get_secureboot() function which returns whether UEFI Secure Boot is enabled or not on UEFI systems. Reviewed-by: Daniel Kiper 2020-12-11 Daniel Kiper efi: Add a function to read EFI variables with attributes It will be used to properly detect and report UEFI Secure Boot status to the x86 Linux kernel. The functionality will be added by subsequent patches. Reviewed-by: Daniel Kiper 2020-12-11 Daniel Kiper efi: Return grub_efi_status_t from grub_efi_get_variable() This is needed to properly detect and report UEFI Secure Boot status to the x86 Linux kernel. The functionality will be added by subsequent patches. Reviewed-by: Daniel Kiper 2020-12-11 Daniel Kiper efi: Make shim_lock GUID and protocol type public The GUID will be used to properly detect and report UEFI Secure Boot status to the x86 Linux kernel. The functionality will be added by subsequent patches. The shim_lock protocol type is made public for completeness. Additionally, fix formatting of four preceding GUIDs. Reviewed-by: Daniel Kiper 2020-12-11 Javier Martinez Canillas arm/term: Fix linking error due multiple ps2_state definitions When building with --target=arm-linux-gnu --with-platform=coreboot a linking error occurs caused by multiple definitions of the ps2_state variable. Mark them as static since they aren't used outside their compilation unit. Reviewed-by: Daniel Kiper 2020-12-11 Javier Martinez Canillas include/grub/i386/linux.h: Include missing header This header uses types defined in but does not include it, which leads to compile errors like the following: In file included from ../include/grub/cpu/linux.h:19, from kern/efi/sb.c:21: ../include/grub/i386/linux.h:80:3: error: unknown type name ‘grub_uint64_t’ 80 | grub_uint64_t addr; Reviewed-by: Daniel Kiper 2020-12-11 Javier Martinez Canillas i386: Don't include in coreboot and ieee1275 startup.S Nothing defined in the header file is used in the assembly code but it may lead to build errors if some headers are included through this and contains definitions that are not recognized by the assembler, e.g.: ../include/grub/types.h: Assembler messages: ../include/grub/types.h:76: Error: no such instruction: `typedef signed char grub_int8_t' ../include/grub/types.h:77: Error: no such instruction: `typedef short grub_int16_t' ../include/grub/types.h:78: Error: no such instruction: `typedef int grub_int32_t' Reviewed-by: Daniel Kiper 2020-11-20 Glenn Washburn luks2: Rename index variable "j" to "i" in luks2_get_keyslot() Looping variable "j" was named such because the variable name "i" was taken. Since "i" has been renamed in the previous patch, we can rename "j" to "i". Reviewed-by: Patrick Steinhardt Reviewed-by: Daniel Kiper 2020-11-20 Glenn Washburn luks2: Rename variable "i" to "keyslot_idx" in luks2_get_keyslot() Variables named "i" are usually looping variables. So, rename it to "keyslot_idx" to ease luks2_get_keyslot() reading. Reviewed-by: Patrick Steinhardt Reviewed-by: Daniel Kiper 2020-11-20 Glenn Washburn luks2: Use correct index variable when looping in luks2_get_keyslot() The loop variable "j" should be used to index the digests and segments json array, instead of the variable "i", which is the keyslot index. Reviewed-by: Patrick Steinhardt Reviewed-by: Daniel Kiper 2020-11-20 Glenn Washburn luks2: Rename source disk variable named "disk" to "source" as in luks.c This makes it more obvious to the reader that the disk referred to is the source disk, as opposed to say the disk holding the cryptodisk. Reviewed-by: Patrick Steinhardt Reviewed-by: Daniel Kiper 2020-11-20 Glenn Washburn cryptodisk: Rename "offset" in grub_cryptodisk_t to "offset_sectors" This makes it clear that the offset represents sectors, not bytes, in order to improve readability. Reviewed-by: Patrick Steinhardt Reviewed-by: Daniel Kiper 2020-11-20 Glenn Washburn cryptodisk: Rename "total_length" field in grub_cryptodisk_t to "total_sectors" This creates an alignment with grub_disk_t naming of the same field and is more intuitive as to how it should be used. Reviewed-by: Patrick Steinhardt Reviewed-by: Daniel Kiper 2020-11-20 Glenn Washburn types: Define GRUB_CHAR_BIT based on compiler macro instead of using literal Reviewed-by: Daniel Kiper 2020-11-20 Javier Martinez Canillas include/grub/arm64/linux.h: Include missing header This header uses types defined in but does not include it, which leads to compile errors like the following: ../include/grub/cpu/linux.h:27:3: error: unknown type name ‘grub_uint32_t’ 27 | grub_uint32_t code0; /* Executable code */ | ^~~~~~~~~~~~~ Reviewed-by: Daniel Kiper 2020-11-20 Javier Martinez Canillas include/grub/arm/system.h: Include missing header The header uses the EXPORT_FUNC() macro defined in but doesn't include it, which leads to the following compile error on arm: ../include/grub/cpu/system.h:12:13: error: ‘EXPORT_FUNC’ declared as function returning a function 12 | extern void EXPORT_FUNC(grub_arm_disable_caches_mmu) (void); | ^~~~~~~~~~~ ../include/grub/cpu/system.h:12:1: warning: parameter names (without types) in function declaration 12 | extern void EXPORT_FUNC(grub_arm_disable_caches_mmu) (void); | ^~~~~~ make[3]: *** [Makefile:36581: kern/efi/kernel_exec-sb.o] Error 1 Reviewed-by: Daniel Kiper 2020-11-20 Daniel Axtens docs: grub-install --pubkey has been supported for some time grub-install --pubkey is supported, so we can now document it. Reviewed-by: Daniel Kiper 2020-11-20 Daniel Axtens docs: grub-install is no longer a shell script Since commit cd46aa6cefab in 2013, grub-install hasn't been a shell script. The para doesn't really add that much, especially since it's the user manual, so just drop it. Reviewed-by: Daniel Kiper 2020-10-30 Jacob Kroon Makefile: Remove unused GRUB_PKGLIBDIR definition Reviewed-by: Daniel Kiper 2020-10-30 Daniel Axtens lzma: Fix compilation error under clang 10 Compiling under clang 10 gives: grub-core/lib/LzmaEnc.c:1362:9: error: misleading indentation; statement is not part of the previous 'if' [-Werror,-Wmisleading-indentation] { ^ grub-core/lib/LzmaEnc.c:1358:7: note: previous statement is here if (repIndex == 0) ^ 1 error generated. It's not really that unclear in context: there's a commented-out if-statement. But tweak the alignment anyway so that clang is happy. Reviewed-by: Daniel Kiper 2020-10-30 Cao jin kern/i386/realmode: Update comment Commit b81d609e4c did not update it. Reviewed-by: Daniel Kiper 2020-10-30 Glenn Washburn cryptodisk: Fix cipher IV mode "plain64" always being set as "plain" When setting cipher IV mode, detection is done by prefix matching the cipher IV mode part of the cipher mode string. Since "plain" matches "plain64", we must check for "plain64" first. Otherwise, "plain64" will be detected as "plain". Reviewed-by: Patrick Steinhardt Reviewed-by: Daniel Kiper 2020-09-18 Glenn Washburn crypto: Remove GPG_ERROR_CFLAGS from gpg_err_code_t enum This was probably added by accident when originally creating the file. Reviewed-by: Daniel Kiper 2020-09-18 Glenn Washburn script: Do not allow a delimiter between function name and block start Currently the following is valid syntax but should be a syntax error: grub> function f; { echo HERE; } grub> f HERE This fix is not backward compatible, but current syntax is not documented either and has no functional value. So any scripts with this unintended syntax are technically syntactically incorrect and should not be relying on this behavior. Reviewed-by: Daniel Kiper 2020-09-18 Glenn Washburn docs: Support for loading and concatenating multiple initrds This has been available since January of 2012 but has not been documented. Reviewed-by: Daniel Kiper 2020-09-18 Glenn Washburn lexer: char const * should be const char * Reviewed-by: Daniel Kiper cryptodisk: Use cipher name instead of object in error message Reviewed-by: Daniel Kiper 2020-09-18 Glenn Washburn tests: F2FS test should use MOUNTDEVICE like other tests LODEVICES is not an array variable and should not be accessed as such. This allows the f2fs test to pass as it was failing because a device name had a space prepended to the path. Acked-by: Jaegeuk Kim Tested-by: Paul Menzel Reviewed-by: Daniel Kiper 2020-09-18 Florian La Roche grub-mkconfig: If $hints is not set reduce the output into grub.cfg to just 1 line Reviewed-by: Daniel Kiper 2020-09-18 Petr Vorel travis: Run bootstrap to fix build autogen.sh isn't enough: $ ./autogen.sh Gnulib not yet bootstrapped; run ./bootstrap instead. The command "./autogen.sh" exited with 1. Additionally, using bootstrap requires to install autopoint package. Reviewed-by: Daniel Kiper 2020-09-18 Patrick Steinhardt luks2: Strip dashes off of the UUID The UUID header for LUKS2 uses a format with dashes, same as for LUKS(1). But while we strip these dashes for the latter, we don't for the former. This isn't wrong per se, but it's definitely inconsistent for users as they need to use the dashed format for LUKS2 and the non-dashed format for LUKS when e.g. calling "cryptomount -u $UUID". Fix this inconsistency by stripping dashes off of the LUKS2 UUID. Reviewed-by: Daniel Kiper 2020-09-18 Tianjia Zhang efi/tpm: Remove unused functions and structures Although the tpm_execute() series of functions are defined they are not used anywhere. Several structures in the include/grub/efi/tpm.h header file are not used too. There is even nonexistent grub_tpm_init() declaration in this header. Delete all that unneeded stuff. If somebody needs the functionality implemented in the dropped code then he/she can re-add it later. Now it needlessly increases the GRUB code/image size. Reviewed-by: Daniel Kiper 2020-09-18 Tianjia Zhang shim_lock: Enable module for all EFI architectures Like the tpm the shim_lock module is only enabled for x86_64 target. However, there's nothing specific to x86_64 in the implementation and it can be enabled for all EFI architectures. Reviewed-by: Daniel Kiper 2020-09-18 Daniel Kiper efi/tpm: Fix typo in grub_efi_tpm2_protocol struct Rename get_active_pcr_blanks() to get_active_pcr_banks(). Reviewed-by: Javier Martinez Canillas 2020-09-18 Daniel Kiper i386/efi/init: Drop bogus include Reviewed-by: Javier Martinez Canillas 2020-09-18 Daniel Kiper docs: Fix devicetree command description Specifically fix the subsection and drop bogus reference to the GNU/Linux. Reported-by: Patrick Higgins Reviewed-by: Javier Martinez Canillas 2020-09-18 Martin Whitaker grub-install: Fix inverted test for NLS enabled when copying locales Commit 3d8439da8 (grub-install: Locale depends on nls) attempted to avoid copying locale files to the target directory when NLS was disabled. However the test is inverted, and it does the opposite. Reviewed-by: Javier Martinez Canillas 2020-09-11 Javier Martinez Canillas tftp: Roll-over block counter to prevent data packets timeouts Commit 781b3e5efc3 (tftp: Do not use priority queue) caused a regression when fetching files over TFTP whose size is bigger than 65535 * block size. grub> linux /images/pxeboot/vmlinuz grub> echo $? 0 grub> initrd /images/pxeboot/initrd.img error: timeout reading '/images/pxeboot/initrd.img'. grub> echo $? 28 It is caused by the block number counter being a 16-bit field, which leads to a maximum file size of ((1 << 16) - 1) * block size. Because GRUB sets the block size to 1024 octets (by using the TFTP Blocksize Option from RFC 2348 [0]), the maximum file size that can be transferred is 67107840 bytes. The TFTP PROTOCOL (REVISION 2) RFC 1350 [1] does not mention what a client should do when a file size is bigger than the maximum, but most TFTP hosts support the block number counter to be rolled over. That is, acking a data packet with a block number of 0 is taken as if the 65356th block was acked. It was working before because the block counter roll-over was happening due an overflow. But that got fixed by the mentioned commit, which led to the regression when attempting to fetch files larger than the maximum size. To allow TFTP file transfers of unlimited size again, re-introduce a block counter roll-over so the data packets are acked preventing the timeouts. [0]: https://tools.ietf.org/html/rfc2348 [1]: https://tools.ietf.org/html/rfc1350 Fixes: 781b3e5efc3 (tftp: Do not use priority queue) Suggested-by: Peter Jones Reviewed-by: Daniel Kiper 2020-09-11 Florian La Roche templates: Remove unnecessary trailing semicolon Reviewed-by: Daniel Kiper 2020-09-11 Glenn Washburn cryptodisk: Fix incorrect calculation of start sector Here dev is a grub_cryptodisk_t and dev->offset is offset in sectors of size native to the cryptodisk device. The sector is correctly transformed into native grub sector size, but then added to dev->offset which is not transformed. It would be nice if the type system would help us with this. Reviewed-by: Patrick Steinhardt Reviewed-by: Daniel Kiper 2020-09-11 Glenn Washburn cryptodisk: Unregister cryptomount command when removing module Reviewed-by: Patrick Steinhardt Reviewed-by: Daniel Kiper 2020-09-11 Patrick Steinhardt luks2: Improve error reporting when decrypting/verifying key While we already set up error messages in both luks2_verify_key() and luks2_decrypt_key(), we do not ever print them. This makes it really hard to discover why a given key actually failed to decrypt a disk. Improve this by including the error message in the user-visible output. Reviewed-by: Daniel Kiper 2020-09-11 Patrick Steinhardt luks: Fix out-of-bounds copy of UUID When configuring a LUKS disk, we copy over the UUID from the LUKS header into the new grub_cryptodisk_t structure via grub_memcpy(). As size we mistakenly use the size of the grub_cryptodisk_t UUID field, which is guaranteed to be strictly bigger than the LUKS UUID field we're copying. As a result, the copy always goes out-of-bounds and copies some garbage from other surrounding fields. During runtime, this isn't noticed due to the fact that we always NUL-terminate the UUID and thus never hit the trailing garbage. Fix the issue by using the size of the local stripped UUID field. Reviewed-by: Daniel Kiper 2020-09-11 Patrick Steinhardt json: Remove invalid typedef redefinition The C standard does not allow for typedef redefinitions, even if they map to the same underlying type. In order to avoid including the jsmn.h in json.h and thus exposing jsmn's internals, we have exactly such a forward-declaring typedef in json.h. If enforcing the GNU99 C standard, clang may generate a warning about this non-standard construct. Fix the issue by using a simple "struct jsmntok" forward declaration instead of using a typedef. Tested-by: Chuck Tuffli Reviewed-by: Daniel Kiper 2020-09-11 Cao jin i386/relocator_common: Drop empty #ifdef Reviewed-by: Daniel Kiper 2020-09-11 Ave Milia video/bochs: Fix typo Reviewed-by: Daniel Kiper 2020-07-29 Colin Watson linux: Fix integer overflows in initrd size handling These could be triggered by a crafted filesystem with very large files. Fixes: CVE-2020-15707 Reviewed-by: Jan Setje-Eilers Reviewed-by: Daniel Kiper 2020-07-29 Peter Jones loader/linux: Avoid overflow on initrd size calculation Reviewed-by: Daniel Kiper 2020-07-29 Alexey Makhalov efi: Fix use-after-free in halt/reboot path commit 92bfc33db984 ("efi: Free malloc regions on exit") introduced memory freeing in grub_efi_fini(), which is used not only by exit path but by halt/reboot one as well. As result of memory freeing, code and data regions used by modules, such as halt, reboot, acpi (used by halt) also got freed. After return to module code, CPU executes, filled by UEFI firmware (tested with edk2), 0xAFAFAFAF pattern as a code. Which leads to #UD exception later. grub> halt !!!! X64 Exception Type - 06(#UD - Invalid Opcode) CPU Apic ID - 00000000 !!!! RIP - 0000000003F4EC28, CS - 0000000000000038, RFLAGS - 0000000000200246 RAX - 0000000000000000, RCX - 00000000061DA188, RDX - 0A74C0854DC35D41 RBX - 0000000003E10E08, RSP - 0000000007F0F860, RBP - 0000000000000000 RSI - 00000000064DB768, RDI - 000000000832C5C3 R8 - 0000000000000002, R9 - 0000000000000000, R10 - 00000000061E2E52 R11 - 0000000000000020, R12 - 0000000003EE5C1F, R13 - 00000000061E0FF4 R14 - 0000000003E10D80, R15 - 00000000061E2F60 DS - 0000000000000030, ES - 0000000000000030, FS - 0000000000000030 GS - 0000000000000030, SS - 0000000000000030 CR0 - 0000000080010033, CR2 - 0000000000000000, CR3 - 0000000007C01000 CR4 - 0000000000000668, CR8 - 0000000000000000 DR0 - 0000000000000000, DR1 - 0000000000000000, DR2 - 0000000000000000 DR3 - 0000000000000000, DR6 - 00000000FFFF0FF0, DR7 - 0000000000000400 GDTR - 00000000079EEA98 0000000000000047, LDTR - 0000000000000000 IDTR - 0000000007598018 0000000000000FFF, TR - 0000000000000000 FXSAVE_STATE - 0000000007F0F4C0 Proposal here is to continue to free allocated memory for exit boot services path but keep it for halt/reboot path as it won't be much security concern here. Introduced GRUB_LOADER_FLAG_EFI_KEEP_ALLOCATED_MEMORY loader flag to be used by efi halt/reboot path. Reviewed-by: Darren Kenny Reviewed-by: Daniel Kiper 2020-07-29 Daniel Kiper efi/chainloader: Propagate errors from copy_file_path() Without any error propagated to the caller, make_file_path() would then try to advance the invalid device path node with GRUB_EFI_NEXT_DEVICE_PATH(), which would fail, returning a NULL pointer that would subsequently be dereferenced. Hence, propagate errors from copy_file_path(). Reviewed-by: Daniel Kiper 2020-07-29 Peter Jones efi: Fix some malformed device path arithmetic errors Several places we take the length of a device path and subtract 4 from it, without ever checking that it's >= 4. There are also cases where this kind of malformation will result in unpredictable iteration, including treating the length from one dp node as the type in the next node. These are all errors, no matter where the data comes from. This patch adds a checking macro, GRUB_EFI_DEVICE_PATH_VALID(), which can be used in several places, and makes GRUB_EFI_NEXT_DEVICE_PATH() return NULL and GRUB_EFI_END_ENTIRE_DEVICE_PATH() evaluate as true when the length is too small. Additionally, it makes several places in the code check for and return errors in these cases. Reviewed-by: Daniel Kiper 2020-07-29 Peter Jones emu: Make grub_free(NULL) safe The grub_free() implementation in grub-core/kern/mm.c safely handles NULL pointers, and code at many places depends on this. We don't know that the same is true on all host OSes, so we need to handle the same behavior in grub-emu's implementation. Reviewed-by: Darren Kenny Reviewed-by: Daniel Kiper 2020-07-29 Peter Jones lvm: Fix two more potential data-dependent alloc overflows It appears to be possible to make a (possibly invalid) lvm PV with a metadata size field that overflows our type when adding it to the address we've allocated. Even if it doesn't, it may be possible to do so with the math using the outcome of that as an operand. Check them both. Reviewed-by: Daniel Kiper 2020-07-29 Peter Jones hfsplus: Fix two more overflows Both node->size and node->namelen come from the supplied filesystem, which may be user-supplied. We can't trust them for the math unless we know they don't overflow. Making sure they go through grub_add() or grub_calloc() first will give us that. Reviewed-by: Darren Kenny Reviewed-by: Daniel Kiper 2020-07-29 Alexey Makhalov relocator: Fix grub_relocator_alloc_chunk_align() top memory allocation Current implementation of grub_relocator_alloc_chunk_align() does not allow allocation of the top byte. Assuming input args are: max_addr = 0xfffff000; size = 0x1000; And this is valid. But following overflow protection will unnecessarily move max_addr one byte down (to 0xffffefff): if (max_addr > ~size) max_addr = ~size; ~size + 1 will fix the situation. In addition, check size for non zero to do not zero max_addr. Reviewed-by: Daniel Kiper 2020-07-29 Chris Coulson script: Avoid a use-after-free when redefining a function during execution Defining a new function with the same name as a previously defined function causes the grub_script and associated resources for the previous function to be freed. If the previous function is currently executing when a function with the same name is defined, this results in use-after-frees when processing subsequent commands in the original function. Instead, reject a new function definition if it has the same name as a previously defined function, and that function is currently being executed. Although a behavioural change, this should be backwards compatible with existing configurations because they can't be dependent on the current behaviour without being broken. Fixes: CVE-2020-15706 Reviewed-by: Daniel Kiper 2020-07-29 Chris Coulson script: Remove unused fields from grub_script_function struct Reviewed-by: Daniel Kiper 2020-07-29 Alexey Makhalov relocator: Protect grub_relocator_alloc_chunk_align() max_addr against integer underflow This commit introduces integer underflow mitigation in max_addr calculation in grub_relocator_alloc_chunk_align() invocation. It consists of 2 fixes: 1. Introduced grub_relocator_alloc_chunk_align_safe() wrapper function to perform sanity check for min/max and size values, and to make safe invocation of grub_relocator_alloc_chunk_align() with validated max_addr value. Replace all invocations such as grub_relocator_alloc_chunk_align(..., min_addr, max_addr - size, size, ...) by grub_relocator_alloc_chunk_align_safe(..., min_addr, max_addr, size, ...). 2. Introduced UP_TO_TOP32(s) macro for the cases where max_addr is 32-bit top address (0xffffffff - size + 1) or similar. Reviewed-by: Daniel Kiper 2020-07-29 Alexey Makhalov relocator: Protect grub_relocator_alloc_chunk_addr() input args against integer underflow/overflow Use arithmetic macros from safemath.h to accomplish it. In this commit, I didn't want to be too paranoid to check every possible math equation for overflow/underflow. Only obvious places (with non zero chance of overflow/underflow) were refactored. Reviewed-by: Daniel Kiper 2020-07-29 Alexey Makhalov tftp: Do not use priority queue There is not need to reassemble the order of blocks. Per RFC 1350, server must wait for the ACK, before sending next block. Data packets can be served immediately without putting them to priority queue. Logic to handle incoming packet is this: - if packet block id equal to expected block id, then process the packet, - if packet block id is less than expected - this is retransmit of old packet, then ACK it and drop the packet, - if packet block id is more than expected - that shouldn't happen, just drop the packet. It makes the tftp receive path code simpler, smaller and faster. As a benefit, this change fixes CID# 73624 and CID# 96690, caused by following while loop: while (cmp_block (grub_be_to_cpu16 (tftph->u.data.block), data->block + 1) == 0) where tftph pointer is not moving from one iteration to another, causing to serve same packet again. Luckily, double serving didn't happen due to data->block++ during the first iteration. Fixes: CID 73624, CID 96690 Reviewed-by: Daniel Kiper 2020-07-29 Konrad Rzeszutek Wilk multiboot2: Fix memory leak if grub_create_loader_cmdline() fails Fixes: CID 292468 Reviewed-by: Daniel Kiper 2020-07-29 Konrad Rzeszutek Wilk udf: Fix memory leak Fixes: CID 73796 Reviewed-by: Daniel Kiper Reviewed-by: Jan Setje-Eilers 2020-07-29 Konrad Rzeszutek Wilk term: Fix overflow on user inputs This requires a very weird input from the serial interface but can cause an overflow in input_buf (keys) overwriting the next variable (npending) with the user choice: (pahole output) struct grub_terminfo_input_state { int input_buf[6]; /* 0 24 */ int npending; /* 24 4 */ <- CORRUPT ...snip... The magic string requires causing this is "ESC,O,],0,1,2,q" and we overflow npending with "q" (aka increase npending to 161). The simplest fix is to just to disallow overwrites input_buf, which exactly what this patch does. Fixes: CID 292449 Reviewed-by: Daniel Kiper 2020-07-29 Konrad Rzeszutek Wilk lzma: Make sure we don't dereference past array The two dimensional array p->posSlotEncoder[4][64] is being dereferenced using the GetLenToPosState() macro which checks if len is less than 5, and if so subtracts 2 from it. If len = 0, that is 0 - 2 = 4294967294. Obviously we don't want to dereference that far out so we check if the position found is greater or equal kNumLenToPosStates (4) and bail out. N.B.: Upstream LZMA 18.05 and later has this function completely rewritten without any history. Fixes: CID 51526 Reviewed-by: Daniel Kiper 2020-07-29 Chris Coulson json: Avoid a double-free when parsing fails. When grub_json_parse() succeeds, it returns the root object which contains a pointer to the provided JSON string. Callers are responsible for ensuring that this string outlives the root object and for freeing its memory when it's no longer needed. If grub_json_parse() fails to parse the provided JSON string, it frees the string before returning an error. This results in a double free in luks2_recover_key(), which also frees the same string after grub_json_parse() returns an error. This changes grub_json_parse() to never free the JSON string passed to it, and updates the documentation for it to make it clear that callers are responsible for ensuring that the string outlives the root JSON object. Fixes: CID 292465 Reviewed-by: Daniel Kiper 2020-07-29 Alexey Makhalov xnu: Fix double free in grub_xnu_devprop_add_property() grub_xnu_devprop_add_property() should not free utf8 and utf16 as it get allocated and freed in the caller. Minor improvement: do prop fields initialization after memory allocations. Fixes: CID 292442, CID 292457, CID 292460, CID 292466 Reviewed-by: Daniel Kiper 2020-07-29 Alexey Makhalov gfxmenu: Fix double free in load_image() self->bitmap should be zeroed after free. Otherwise, there is a chance to double free (USE_AFTER_FREE) it later in rescale_image(). Fixes: CID 292472 Reviewed-by: Daniel Kiper 2020-07-29 Daniel Kiper font: Do not load more than one NAME section The GRUB font file can have one NAME section only. Though if somebody crafts a broken font file with many NAME sections and loads it then the GRUB leaks memory. So, prevent against that by loading first NAME section and failing in controlled way on following one. Reported-by: Chris Coulson Reviewed-by: Jan Setje-Eilers 2020-07-29 Peter Jones iso9660: Don't leak memory on realloc() failures Reviewed-by: Daniel Kiper 2020-07-29 Peter Jones malloc: Use overflow checking primitives where we do complex allocations This attempts to fix the places where we do the following where arithmetic_expr may include unvalidated data: X = grub_malloc(arithmetic_expr); It accomplishes this by doing the arithmetic ahead of time using grub_add(), grub_sub(), grub_mul() and testing for overflow before proceeding. Among other issues, this fixes: - allocation of integer overflow in grub_video_bitmap_create() reported by Chris Coulson, - allocation of integer overflow in grub_png_decode_image_header() reported by Chris Coulson, - allocation of integer overflow in grub_squash_read_symlink() reported by Chris Coulson, - allocation of integer overflow in grub_ext2_read_symlink() reported by Chris Coulson, - allocation of integer overflow in read_section_as_string() reported by Chris Coulson. Fixes: CVE-2020-14309, CVE-2020-14310, CVE-2020-14311 Reviewed-by: Daniel Kiper 2020-07-29 Peter Jones calloc: Use calloc() at most places This modifies most of the places we do some form of: X = malloc(Y * Z); to use calloc(Y, Z) instead. Among other issues, this fixes: - allocation of integer overflow in grub_png_decode_image_header() reported by Chris Coulson, - allocation of integer overflow in luks_recover_key() reported by Chris Coulson, - allocation of integer overflow in grub_lvm_detect() reported by Chris Coulson. Fixes: CVE-2020-14308 Reviewed-by: Daniel Kiper 2020-07-29 Peter Jones calloc: Make sure we always have an overflow-checking calloc() available This tries to make sure that everywhere in this source tree, we always have an appropriate version of calloc() (i.e. grub_calloc(), xcalloc(), etc.) available, and that they all safely check for overflow and return NULL when it would occur. Reviewed-by: Daniel Kiper 2020-07-29 Peter Jones safemath: Add some arithmetic primitives that check for overflow This adds a new header, include/grub/safemath.h, that includes easy to use wrappers for __builtin_{add,sub,mul}_overflow() declared like: bool OP(a, b, res) where OP is grub_add, grub_sub or grub_mul. OP() returns true in the case where the operation would overflow and res is not modified. Otherwise, false is returned and the operation is executed. These arithmetic primitives require newer compiler versions. So, bump these requirements in the INSTALL file too. Reviewed-by: Daniel Kiper 2020-07-29 Peter Jones yylex: Make lexer fatal errors actually be fatal When presented with a command that can't be tokenized to anything smaller than YYLMAX characters, the parser calls YY_FATAL_ERROR(errmsg), expecting that will stop further processing, as such: #define YY_DO_BEFORE_ACTION \ yyg->yytext_ptr = yy_bp; \ yyleng = (int) (yy_cp - yy_bp); \ yyg->yy_hold_char = *yy_cp; \ *yy_cp = '\0'; \ if ( yyleng >= YYLMAX ) \ YY_FATAL_ERROR( "token too large, exceeds YYLMAX" ); \ yy_flex_strncpy( yytext, yyg->yytext_ptr, yyleng + 1 , yyscanner); \ yyg->yy_c_buf_p = yy_cp; The code flex generates expects that YY_FATAL_ERROR() will either return for it or do some form of longjmp(), or handle the error in some way at least, and so the strncpy() call isn't in an "else" clause, and thus if YY_FATAL_ERROR() is *not* actually fatal, it does the call with the questionable limit, and predictable results ensue. Unfortunately, our implementation of YY_FATAL_ERROR() is: #define YY_FATAL_ERROR(msg) \ do { \ grub_printf (_("fatal error: %s\n"), _(msg)); \ } while (0) The same pattern exists in yyless(), and similar problems exist in users of YY_INPUT(), several places in the main parsing loop, yy_get_next_buffer(), yy_load_buffer_state(), yyensure_buffer_stack, yy_scan_buffer(), etc. All of these callers expect YY_FATAL_ERROR() to actually be fatal, and the things they do if it returns after calling it are wildly unsafe. Fixes: CVE-2020-10713 Reviewed-by: Daniel Kiper 2020-05-25 Marc Zyngier arm: Fix 32-bit ARM handling of the CTR register When booting on an ARMv8 core that implements either CTR.IDC or CTR.DIC (indicating that some of the cache maintenance operations can be removed when dealing with I/D-cache coherency, GRUB dies with a "Unsupported cache type 0x........" message. This is pretty likely to happen when running in a virtual machine hosted on an arm64 machine (I've triggered it on a system built around a bunch of Cortex-A55 cores, which implements CTR.IDC). It turns out that the way GRUB deals with the CTR register is a bit harsh for anything from ARMv7 onwards. The layout of the register is backward compatible, meaning that nothing that gets added is allowed to break earlier behaviour. In this case, ignoring IDC is completely fine, and only results in unnecessary cache maintenance. We can thus avoid being paranoid, and align the 32bit behaviour with its 64bit equivalent. This patch has the added benefit that it gets rid of a (gnu-specific) case range too. Reviewed-by: Leif Lindholm Reviewed-by: Daniel Kiper 2020-05-25 Ian Jackson templates/20_linux_xen: Support Xen Security Modules (XSM/FLASK) XSM is enabled by adding "flask=enforcing" as a Xen command line argument, and providing the policy file as a grub module. We make entries for both with and without XSM. If XSM is not compiled into Xen, then there are no policy files, so no change to the boot options. Reviewed-by: Daniel Kiper 2020-05-25 Ian Jackson templates/20_linux_xen: Ignore xenpolicy and config files too file_is_not_sym() currently only checks for xen-syms. Extend it to disregard xenpolicy (XSM policy files) and files ending .config (which are built by the Xen upstream build system in some configurations and can therefore end up in /boot). Rename the function accordingly, to file_is_not_xen_garbage(). Reviewed-by: Daniel Kiper 2020-05-25 Javier Martinez Canillas net: Break out nested function Nested functions are not supported in C, but are permitted as an extension in the GNU C dialect. Commit cb2f15c5448 ("normal/main: Search for specific config files for netboot") added a nested function which caused the build to break when compiling with clang. Break that out into a static helper function to make the code portable again. Reported-by: Daniel Axtens Tested-by: Daniel Axtens Reviewed-by: Daniel Kiper 2020-05-25 Javier Martinez Canillas tpm: Enable module for all EFI platforms The module is only enabled for x86_64, but there's nothing specific to x86_64 in the implementation and can be enabled for all EFI platforms. Reviewed-by: Daniel Kiper 2020-05-25 Daniel Kiper INSTALL/configure: Update install doc and configure comment ..to reflect the GRUB build reality in them. Additionally, fix text formatting a bit. Reviewed-by: Leif Lindholm 2020-05-25 Daniel Kiper configure: Set gnu99 C language standard by default Commit d5a32255d (misc: Make grub_strtol() "end" pointers have safer const qualifiers) introduced "restrict" keyword into some functions definitions. This keyword was introduced in C99 standard. However, some compilers by default may use C89 or something different. This behavior leads to the breakage during builds when c89 or gnu89 is in force. So, let's set gnu99 C language standard for all compilers by default. This way a bit random build issue will be fixed and the GRUB source will be build consistently regardless of type and version of the compiler. It was decided to use gnu99 C language standard because it fixes the issue mentioned above and also provides some useful extensions which are used here and there in the GRUB source. Potentially we can use gnu11 too. However, this may reduce pool of older compilers which can be used to build the GRUB. So, let's live with gnu99 until we discover that we strongly require a feature from newer C standard. The user is still able to override C language standard using relevant *_CFLAGS variables. Reviewed-by: Leif Lindholm 2020-05-15 Tianjia Zhang tpm: Rename function grub_tpm_log_event() to grub_tpm_measure() grub_tpm_log_event() and grub_tpm_measure() are two functions that have the same effect. So, keep grub_tpm_log_event() and rename it to grub_tpm_measure(). This way we get also a more clear semantics. Reviewed-by: Daniel Kiper 2020-05-15 Daniel Kiper autogen: Replace -iname with -ipath in find command ..because -iname cannot be used to match paths. Reviewed-by: Javier Martinez Canillas Reviewed-by: Leif Lindholm Reviewed-by: Daniel Axtens 2020-05-15 Daniel Kiper INSTALL: Update configure example ..to make it more relevant. Reviewed-by: Leif Lindholm 2020-05-15 Daniel Kiper configure: Drop unneeded TARGET_CFLAGS expansion Reviewed-by: Javier Martinez Canillas Reviewed-by: Leif Lindholm 2020-05-15 Jacob Kroon docs/grub: Support for probing partition UUID on MSDOS disks Support was implemented in commit c7cb11b21 (probe: Support probing for msdos PARTUUID). Reviewed-by: Daniel Kiper 2020-05-15 Tianjia Zhang verifiers: Add verify string debug message Like grub_verifiers_open(), the grub_verify_string() should also display this debug message, which is very helpful for debugging. Reviewed-by: Daniel Kiper 2020-05-15 Javier Martinez Canillas envblk: Fix buffer overrun when attempting to shrink a variable value If an existing variable is set with a value whose length is smaller than the current value, a memory corruption can happen due copying padding '#' characters outside of the environment block buffer. This is caused by a wrong calculation of the previous free space position after moving backward the characters that followed the old variable value. That position is calculated to fill the remaining of the buffer with the padding '#' characters. But since isn't calculated correctly, it can lead to copies outside of the buffer. The issue can be reproduced by creating a variable with a large value and then try to set a new value that is much smaller: $ grub2-editenv --version grub2-editenv (GRUB) 2.04 $ grub2-editenv env create $ grub2-editenv env set a="$(for i in {1..500}; do var="b$var"; done; echo $var)" $ wc -c env 1024 grubenv $ grub2-editenv env set a="$(for i in {1..50}; do var="b$var"; done; echo $var)" malloc(): corrupted top size Aborted (core dumped) $ wc -c env 0 grubenv Reported-by: Renaud Métrich Reviewed-by: Daniel Kiper 2020-05-15 Hans Ulrich Niedermann docs: Remove docs for non-existing uppermem command Remove all documentation of and mentions of the uppermem command from the docs/grub.texi file. The uppermem command is not implemented in the GRUB source at all and appears to never have been implemented despite former plans to add an uppermem command. To reduce user confusion, this even removes the paragraph describing how GRUB's uppermem command was supposed to complement the Linux kernel's mem= parameter. Reviewed-by: Daniel Kiper 2020-05-15 Hans Ulrich Niedermann docs: Remove docs for non-existing pxe_unload command Remove the documentation of the pxe_unload command from the docs/grub.texi file. The pxe_unload command is not implemented in the grub source at this time at all. It appears to have been removed in commit 671a78acb (cleanup pxe and efi network release). Reviewed-by: Daniel Kiper 2020-05-15 Hans Ulrich Niedermann gitignore: Add a few forgotten file patterns Add a few patterns to .gitignore to cover files which are generated by building grub ("make", "make check", "make dist") but which have been forgotten to add to .gitignore in the past. Reviewed-by: Daniel Kiper 2020-05-15 Hans Ulrich Niedermann gitignore: Add leading slashes where appropriate Going through the list of gitignore patterns without a leading slash, this adds a leading slash where it appears to have been forgotten. Some gitignore patterns like ".deps/" or "Makefile" clearly should match everywhere, so those definitively need no leading slash. For some patterns like "ascii.bitmaps", it is unclear where in the source tree they should match. Those patterns are kept as they are, matching the patterns in the whole tree of subdirectories. Reviewed-by: Daniel Kiper 2020-05-15 Hans Ulrich Niedermann gitignore: Add trailing slashes for directories Add trailing slashes for all patterns matching directories. Note that we do *not* add trailing slashes for *symlinks* to directories. Reviewed-by: Daniel Kiper 2020-05-15 Hans Ulrich Niedermann gitignore: Sort both pattern groups alphabetically Alphabetically sort the two groups of gitignore patterns: * The group of patterns without slashes, matching anywhere in the directory subtree. * The group of patterns with slashes, matching relative to the .gitignore file's directory Reviewed-by: Daniel Kiper 2020-05-15 Hans Ulrich Niedermann gitignore: Group patterns with and without slash Group the .gitignore patterns into two groups: * Pattern not including a slash, i.e. matching files anywhere in the .gitignore file's directory and all of its subdirectories. * Patterns including a slash, i.e. matching only relative to the .gitignore file's directory. Reviewed-by: Daniel Kiper 2020-05-15 Hans Ulrich Niedermann gitignore: Consistent leading slash is easier to read As all gitignore patterns containing a left or middle slash match only relative to the .gitignore file's directory, we write them all in the same manner with a leading slash. This makes the file significantly easier to read. Reviewed-by: Daniel Kiper 2020-05-15 Daniel Kiper mips/cache: Add missing nop's in delay slots Lack of them causes random instructions to be executed before the jump really happens. Reviewed-by: Daniel Kiper 2020-04-21 Patrick Steinhardt luks2: Propagate error when reading area key fails When decrypting a given keyslot, all error cases except for one set up an error and return the error code. The only exception is when we try to read the area key: instead of setting up an error message, we directly print it via grub_dprintf(). Convert the outlier to use grub_error() to allow more uniform handling of errors. Reviewed-by: Daniel Kiper 2020-04-21 Patrick Steinhardt json: Get rid of casts for "jsmntok_t" With the upstream change having landed that adds a name to the previously anonymous "jsmntok" typedef, we can now add a forward declaration for that struct in our code. As a result, we no longer have to store the "tokens" member of "struct grub_json" as a void pointer but can instead use the forward declaration, allowing us to get rid of casts of that field. Reviewed-by: Daniel Kiper 2020-04-21 Patrick Steinhardt json: Update jsmn library to upstream commit 053d3cd Update our embedded version of the jsmn library to upstream commit 053d3cd (Merge pull request #175 from pks-t/pks/struct-type, 2020-04-02). Reviewed-by: Daniel Kiper 2020-04-21 Steve Langasek templates: Output a menu entry for firmware setup on UEFI FastBoot systems The fwsetup command allows to reboot into the EFI firmware setup menu, add a template to include a menu entry on EFI systems that makes use of that command to reboot into the EFI firmware settings. This is useful for users since the hotkey to enter into the EFI setup menu may not be the same on all systems so users can use the menu entry without needing to figure out what key needs to be pressed. Also, if fastboot is enabled in the BIOS then often it is not possible to enter the firmware setup menu. So the entry is again useful for this case. Reviewed-by: Daniel Kiper 2020-04-21 Hans de Goede kern/term: Accept ESC, F4 and holding SHIFT as user interrupt keys On some devices the ESC key is the hotkey to enter the BIOS/EFI setup screen, making it really hard to time pressing it right. Besides that ESC is also pretty hard to discover for a user who does not know it will unhide the menu. This commit makes F4, which was chosen because is not used as a hotkey to enter the BIOS setup by any vendor, also interrupt sleeps / stop the menu countdown. This solves the ESC gets into the BIOS setup and also somewhat solves the discoverability issue, but leaves the timing issue unresolved. This commit fixes the timing issue by also adding support for keeping SHIFT pressed during boot to stop the menu countdown. This matches what Ubuntu is doing, which should also help with discoverability. Reviewed-by: Daniel Kiper 2020-04-21 Hans de Goede efi/console: Do not set text-mode until we actually need it If we're running with a hidden menu we may never need text mode, so do not change the video-mode to text until we actually need it. This allows to boot a machine without unnecessary graphical transitions and provide a seamless boot experience to users. Reviewed-by: Daniel Kiper 2020-04-21 Hans de Goede efi/console: Implement getkeystatus() support Implement getkeystatus() support in the EFI console driver. This is needed because the logic to determine if a key was pressed to make the menu countdown stop will be changed by a later patch to also take into account the SHIFT key being held down. For this reason the EFI console driver has to support getkeystatus() to allow detecting that event. Note that if a non-modifier key gets pressed and repeated calls to getkeystatus() are made then it will return the modifier status at the time of the non-modifier key, until that key-press gets consumed by a getkey() call. This is a side-effect of how the EFI simple-text-input protocol works and cannot be avoided. Reviewed-by: Daniel Kiper 2020-04-21 Hans de Goede efi/console: Add grub_console_read_key_stroke() helper function This is a preparatory patch for adding getkeystatus() support to the EFI console driver. We can get modifier status through the simple_text_input read_key_stroke() method, but if a non-modifier key is (also) pressed the read_key_stroke() call will consume that key from the firmware's queue. The new grub_console_read_key_stroke() helper buffers upto 1 key-stroke. If it has a non-modifier key buffered, it will return that one, if its buffer is empty, it will fills its buffer by getting a new key-stroke. If called with consume=1 it will empty its buffer after copying the key-data to the callers buffer, this is how getkey() will use it. If called with consume=0 it will keep the last key-stroke buffered, this is how getkeystatus() will call it. This means that if a non-modifier key gets pressed, repeated getkeystatus() calls will return the modifiers of that key-press until it is consumed by a getkey() call. Reviewed-by: Daniel Kiper 2020-04-21 Hans de Goede kern/term: Make grub_getkeystatus() helper function available everywhere Move grub_getkeystatushelper() function from grub-core/commands/keystatus.c to grub-core/kern/term.c and export it so that it can be used outside of the keystatus command code too. There's no logic change in this patch. The function definition is moved so it can be called from grub-core/kern/term.c in a subsequent patch. It will be used to determine if a SHIFT key has was held down and use that also to interrupt the countdown, without the need to press a key at the right time. Reviewed-by: Daniel Kiper 2020-04-21 Javier Martinez Canillas efi/console: Move grub_console_set{colorstate,cursor} higher in the file This is just a preparatory patch to move the functions higher in the file, since these will be called by the grub_prepare_for_text_output() function that will be introduced in a later patch. The logic is unchanged by this patch. Functions definitions are just moved to avoid a forward declaration in a later patch, keeping the code clean. Reviewed-by: Daniel Kiper 2020-04-21 Paul Menzel docs/grub: Fix typo in *preferred* Reviewed-by: Daniel Kiper 2020-04-21 Daniel Axtens powerpc/mkimage: Fix CHRP note descsz Currently, an image generated with 'grub-mkimage -n' causes an error when read with 'readelf -a': Displaying notes found at file offset 0x000106f0 with length 0x0000002c: Owner Data size Description readelf: Warning: note with invalid namesz and/or descsz found at offset 0x0 readelf: Warning: type: 0x1275, namesize: 0x00000008, descsize: 0x0000002c, alignment: 4 This is because the descsz of the CHRP note is set to sizeof (struct grub_ieee1275_note) which is the size of the entire note, including name and elf header. The desczs should contain only the contents, not the name and header sizes. Set the descsz instead to 'sizeof (struct grub_ieee1275_note_desc)' Resultant readelf output: Displaying notes found at file offset 0x00010710 with length 0x0000002c: Owner Data size Description PowerPC 0x00000018 Unknown note type: (0x00001275) description data: ff ff ff ff 00 c0 00 00 ff ff ff ff ff ff ff ff ff ff ff ff 00 00 40 00 So far as I can tell this issue has existed for as long as the note generation code has existed, but I guess nothing really checks descsz. Reviewed-by: Daniel Kiper 2020-03-31 Flavio Suligoi efi: Add missed space in GRUB_EFI_GLOBAL_VARIABLE_GUID Reviewed-by: Daniel Kiper 2020-03-31 Michael Chang zfs: Fix gcc10 error -Werror=zero-length-bounds We bumped into the build error while testing gcc-10 pre-release. In file included from ../../include/grub/file.h:22, from ../../grub-core/fs/zfs/zfs.c:34: ../../grub-core/fs/zfs/zfs.c: In function 'zap_leaf_lookup': ../../grub-core/fs/zfs/zfs.c:2263:44: error: array subscript '' is outside the bounds of an interior zero-length array 'grub_uint16_t[0]' {aka 'short unsigned int[0]'} [-Werror=zero-length-bounds] 2263 | for (chunk = grub_zfs_to_cpu16 (l->l_hash[LEAF_HASH (blksft, h, l)], endian); ../../include/grub/types.h:241:48: note: in definition of macro 'grub_le_to_cpu16' 241 | # define grub_le_to_cpu16(x) ((grub_uint16_t) (x)) | ^ ../../grub-core/fs/zfs/zfs.c:2263:16: note: in expansion of macro 'grub_zfs_to_cpu16' 2263 | for (chunk = grub_zfs_to_cpu16 (l->l_hash[LEAF_HASH (blksft, h, l)], endian); | ^~~~~~~~~~~~~~~~~ In file included from ../../grub-core/fs/zfs/zfs.c:48: ../../include/grub/zfs/zap_leaf.h:72:16: note: while referencing 'l_hash' 72 | grub_uint16_t l_hash[0]; | ^~~~~~ Here I'd like to quote from the gcc document [1] which seems best to explain what is going on here. "Although the size of a zero-length array is zero, an array member of this kind may increase the size of the enclosing type as a result of tail padding. The offset of a zero-length array member from the beginning of the enclosing structure is the same as the offset of an array with one or more elements of the same type. The alignment of a zero-length array is the same as the alignment of its elements. Declaring zero-length arrays in other contexts, including as interior members of structure objects or as non-member objects, is discouraged. Accessing elements of zero-length arrays declared in such contexts is undefined and may be diagnosed." The l_hash[0] is apparnetly an interior member to the enclosed structure while l_entries[0] is the trailing member. And the offending code tries to access members in l_hash[0] array that triggers the diagnose. Given that the l_entries[0] is used to get proper alignment to access leaf chunks, we can accomplish the same thing through the ALIGN_UP macro thus eliminating l_entries[0] from the structure. In this way we can pacify the warning as l_hash[0] now becomes the last member to the enclosed structure. [1] https://gcc.gnu.org/onlinedocs/gcc/Zero-Length.html Reviewed-by: Daniel Kiper 2020-03-31 Michael Chang mdraid1x_linux: Fix gcc10 error -Werror=array-bounds We bumped into the build error while testing gcc-10 pre-release. ../../grub-core/disk/mdraid1x_linux.c: In function 'grub_mdraid_detect': ../../grub-core/disk/mdraid1x_linux.c:181:15: error: array subscript is outside array bounds of 'grub_uint16_t[0]' {aka 'short unsigned int[0]'} [-Werror=array-bounds] 181 | (char *) &sb.dev_roles[grub_le_to_cpu32 (sb.dev_number)] | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ../../grub-core/disk/mdraid1x_linux.c:98:17: note: while referencing 'dev_roles' 98 | grub_uint16_t dev_roles[0]; /* Role in array, or 0xffff for a spare, or 0xfffe for faulty. */ | ^~~~~~~~~ ../../grub-core/disk/mdraid1x_linux.c:127:33: note: defined here 'sb' 127 | struct grub_raid_super_1x sb; | ^~ cc1: all warnings being treated as errors Apparently gcc issues the warning when trying to access sb.dev_roles array's member, since it is a zero length array as the last element of struct grub_raid_super_1x that is allocated sparsely without extra chunks for the trailing bits, so the warning looks legitimate in this regard. As the whole thing here is doing offset computation, it is undue to use syntax that would imply array member access then take address from it later. Instead we could accomplish the same thing through basic array pointer arithmetic to pacify the warning. Reviewed-by: Daniel Kiper 2020-03-31 Simon Hardy build: Fix GRUB i386-pc build with Ubuntu gcc With recent versions of gcc on Ubuntu a very large lzma_decompress.img file is output. (e.g. 134479600 bytes instead of 2864.) This causes grub-mkimage to fail with: "error: Decompressor is too big." This seems to be caused by a section .note.gnu.property that is placed at an offset such that objcopy needs to pad the img file with zeros. This issue is present on: Ubuntu 19.10 with gcc (Ubuntu 8.3.0-26ubuntu1~19.10) 8.3.0 Ubuntu 19.10 with gcc (Ubuntu 9.2.1-9ubuntu2) 9.2.1 20191008 This issue is not present on: Ubuntu 19.10 with gcc (Ubuntu 7.5.0-3ubuntu1~19.10) 7.5.0 RHEL 8.0 with gcc 8.3.1 20190507 (Red Hat 8.3.1-4) The issue can be fixed by removing the section using objcopy as shown in this patch. Reviewed-by: Daniel Kiper 2020-03-31 Tianjia Zhang efi/tpm: Fix memory leak in grub_tpm1/2_log_event() The memory requested for the event is not released here, causing memory leaks. This patch fixes this problem. Reviewed-by: Javier Martinez Canillas Reviewed-by: Daniel Kiper 2020-03-31 Michael Chang docs: Document notes on LVM cache booting Add notes on LVM cache booting to the GRUB manual to help user understanding the outstanding issue and status. Reviewed-by: Daniel Kiper 2020-03-31 Michael Chang lvm: Add LVM cache logical volume handling The LVM cache logical volume is the logical volume consisting of the original and the cache pool logical volume. The original is usually on a larger and slower storage device while the cache pool is on a smaller and faster one. The performance of the original volume can be improved by storing the frequently used data on the cache pool to utilize the greater performance of faster device. The default cache mode "writethrough" ensures that any data written will be stored both in the cache and on the origin LV, therefore grub can be straight to read the original lv as no data loss is guarenteed. The second cache mode is "writeback", which delays writing from the cache pool back to the origin LV to have increased performance. The drawback is potential data loss if losing the associated cache device. During the boot time grub reads the LVM offline i.e. LVM volumes are not activated and mounted, hence it should be fine to read directly from original lv since all cached data should have been flushed back in the process of taking it offline. It is also not much helpful to the situation by adding fsync calls to the install code. The fsync did not force to write back dirty cache to the original device and rather it would update associated cache metadata to complete the write transaction with the cache device. IOW the writes to cached blocks still go only to the cache device. To write back dirty cache, as LVM cache did not support dirty cache flush per block range, there'no way to do it for file. On the other hand the "cleaner" policy is implemented and can be used to write back "all" dirty blocks in a cache, which effectively drain all dirty cache gradually to attain and last in the "clean" state, which can be useful for shrinking or decommissioning a cache. The result and effect is not what we are looking for here. In conclusion, as it seems no way to enforce file writes to the original device, grub may suffer from power failure as it cannot assemble the cache device and read the dirty data from it. However since the case is only applicable to writeback mode which is sensitive to data lost in nature, I'd still like to propose my (relatively simple) patch and treat reading dirty cache as improvement. Reviewed-by: Daniel Kiper 2020-03-10 Patrick Steinhardt gnulib: Fix build of base64 when compiling with memory debugging When building GRUB with memory management debugging enabled, then the build fails because of `grub_debug_malloc()` and `grub_debug_free()` being undefined in the luks2 module. The cause is that we patch "base64.h" to unconditionaly include "config-util.h", which shouldn't be included for modules at all. As a result, `MM_DEBUG` is defined when building the module, causing it to use the debug memory allocation functions. As these are not built into modules, we end up with a linker error. Fix the issue by removing the include altogether. The sole reason it was included was for the `_GL_ATTRIBUTE_CONST` macro, which we can simply define as empty in case it's not set. Reviewed-by: Daniel Kiper 2020-03-10 Patrick Steinhardt build: Fix option to explicitly disable memory debugging The memory management system supports a debug mode that can be enabled at build time by passing "--enable-mm-debug" to the configure script. Passing the option will cause us define MM_DEBUG as expected, but in fact the reverse option "--disable-mm-debug" will do the exact same thing and also set up the define. This currently causes the build of "lib/gnulib/base64.c" to fail as it tries to use `grub_debug_malloc()` and `grub_debug_free()` even though both symbols aren't defined. Seemingly, `AC_ARG_ENABLE()` will always execute the third argument if either the positive or negative option was passed. Let's thus fix the issue by moving the call to`AC_DEFINE()` into an explicit `if test $xenable_mm_debug` block, similar to how other defines work. Reviewed-by: Daniel Kiper Reviewed-by: Paul Menzel 2020-03-10 David Michael fat: Support file modification times This allows comparing file ages on EFI system partitions. Reviewed-by: Daniel Kiper 2020-03-10 David Michael exfat: Save the matching directory entry struct when searching This provides the node's attributes outside the iterator function so the file modification time can be accessed and reported. Reviewed-by: Daniel Kiper 2020-03-10 Mike Gilbert datetime: Enable the datetime module for the emu platform Fixes a build failure: grub-core/commands/date.c:49: undefined reference to `grub_get_weekday_name' grub-core/commands/ls.c:155: undefined reference to `grub_unixtime2datetime' Bug: https://bugs.gentoo.org/711512 Reviewed-by: Javier Martinez Canillas Tested-by: Javier Martinez Canillas Reviewed-by: Daniel Kiper 2020-03-10 John Paul Adrian Glaubitz build: Add soft-float handling for SuperH (sh4) While GRUB has no platform support for SuperH (sh4) yet, this change adds the target-specific handling of soft-floats such that the GRUB utilities can be built on this target. Reviewed-by: Daniel Kiper 2020-03-10 Peter Jones efi: Fix the type of grub_efi_status_t Currently, in some builds with some checkers, we see: 1. grub-core/disk/efi/efidisk.c:601: error[shiftTooManyBitsSigned]: Shifting signed 64-bit value by 63 bits is undefined behaviour This is because grub_efi_status_t is defined as grub_efi_intn_t, which is signed, and shifting into the sign bit is not defined behavior. UEFI fixed this in the spec in 2.3: 2.3 | Change the defined type of EFI_STATUS from INTN to UINTN | May 7, 2009 And the current EDK2 code has: MdePkg/Include/Base.h-// MdePkg/Include/Base.h-// Status codes common to all execution phases MdePkg/Include/Base.h-// MdePkg/Include/Base.h:typedef UINTN RETURN_STATUS; MdePkg/Include/Base.h- MdePkg/Include/Base.h-/** MdePkg/Include/Base.h- Produces a RETURN_STATUS code with the highest bit set. MdePkg/Include/Base.h- MdePkg/Include/Base.h- @param StatusCode The status code value to convert into a warning code. MdePkg/Include/Base.h- StatusCode must be in the range 0x00000000..0x7FFFFFFF. MdePkg/Include/Base.h- MdePkg/Include/Base.h- @return The value specified by StatusCode with the highest bit set. MdePkg/Include/Base.h- MdePkg/Include/Base.h-**/ MdePkg/Include/Base.h-#define ENCODE_ERROR(StatusCode) ((RETURN_STATUS)(MAX_BIT | (StatusCode))) MdePkg/Include/Base.h- MdePkg/Include/Base.h-/** MdePkg/Include/Base.h- Produces a RETURN_STATUS code with the highest bit clear. MdePkg/Include/Base.h- MdePkg/Include/Base.h- @param StatusCode The status code value to convert into a warning code. MdePkg/Include/Base.h- StatusCode must be in the range 0x00000000..0x7FFFFFFF. MdePkg/Include/Base.h- MdePkg/Include/Base.h- @return The value specified by StatusCode with the highest bit clear. MdePkg/Include/Base.h- MdePkg/Include/Base.h-**/ MdePkg/Include/Base.h-#define ENCODE_WARNING(StatusCode) ((RETURN_STATUS)(StatusCode)) MdePkg/Include/Base.h- MdePkg/Include/Base.h-/** MdePkg/Include/Base.h- Returns TRUE if a specified RETURN_STATUS code is an error code. MdePkg/Include/Base.h- MdePkg/Include/Base.h- This function returns TRUE if StatusCode has the high bit set. Otherwise, FALSE is returned. MdePkg/Include/Base.h- MdePkg/Include/Base.h- @param StatusCode The status code value to evaluate. MdePkg/Include/Base.h- MdePkg/Include/Base.h- @retval TRUE The high bit of StatusCode is set. MdePkg/Include/Base.h- @retval FALSE The high bit of StatusCode is clear. MdePkg/Include/Base.h- MdePkg/Include/Base.h-**/ MdePkg/Include/Base.h-#define RETURN_ERROR(StatusCode) (((INTN)(RETURN_STATUS)(StatusCode)) < 0) ... Uefi/UefiBaseType.h:typedef RETURN_STATUS EFI_STATUS; This patch makes grub's implementation match the Edk2 declaration with regards to the signedness of the type. Reviewed-by: Daniel Kiper 2020-03-10 Peter Jones efi/gop: Add debug output on GOP probing Add debug information to EFI GOP video driver probing function. Reviewed-by: Daniel Kiper 2020-03-10 Peter Jones efi/uga: Use video instead of fb as debug condition All other video drivers use "video" as the debug condition instead of "fb" so change this in the efi/uga driver to make it consistent with the others. Reviewed-by: Daniel Kiper 2020-03-10 Peter Jones efi: Print error messages to grub_efi_allocate_pages_real() No messages were printed in this function, add some to ease debugging. Also, the function returns a void * pointer so return NULL instead of 0 to make the code more readable. Reviewed-by: Daniel Kiper 2020-03-10 Andrei Borzenkov efi/uga: Use 64 bit for fb_base We get 64 bit from PCI BAR but then truncate by assigning to 32 bit. Make sure to check that pointer does not overflow on 32 bit platform. Closes: 50931 Reviewed-by: Daniel Kiper 2020-03-10 Alexander Graf efi/gop: Add support for BLT_ONLY adapters EFI GOP has support for multiple different bitness types of frame buffers and for a special "BLT only" type which is always defined to be RGBx. Because grub2 doesn't ever directly access the frame buffer but instead only renders graphics via the BLT interface anyway, we can easily support these adapters. The reason this has come up now is the emerging support for virtio-gpu in OVMF. That adapter does not have the notion of a memory mapped frame buffer and thus is BLT only. Reviewed-by: Daniel Kiper 2020-03-10 Peter Jones normal/completion: Fix possible NULL pointer dereference Coverity Scan reports that the grub_strrchr() function can return NULL if the character is not found. Check if that's the case for dirfile pointer. Reviewed-by: Daniel Kiper 2020-03-10 Peter Jones kern: Add grub_debug_enabled() Add a grub_debug_enabled() helper function instead of open coding it. Reviewed-by: Daniel Kiper 2020-03-10 Peter Jones Makefile: Make libgrub.pp depend on config-util.h If you build with "make -j48" a lot, sometimes you see: gcc -E -DHAVE_CONFIG_H -I. -I.. -Wall -W -DGRUB_UTIL=1 -D_FILE_OFFSET_BITS=64 -I./include -DGRUB_FILE=\"grub_script.tab.h\" -I. -I.. -I. -I.. -I../include -I./include -I../grub-core/lib/libgcrypt-grub/src/ -I../grub-core/lib/minilzo -I../grub-core/lib/xzembed -DMINILZO_HAVE_CONFIG_H -Wall -W -DGRUB_UTIL=1 -D_FILE_OFFSET_BITS=64 -I./include -DGRUB_FILE=\"grub_script.tab.h\" -I. -I.. -I. -I.. -I../include -I./include -I../grub-core/lib/libgcrypt-grub/src/ -I./grub-core/gnulib -I../grub-core/gnulib -I/builddir/build/BUILD/grub-2.02/grub-aarch64-efi-2.02 -D_FILE_OFFSET_BITS=64 \ -D'GRUB_MOD_INIT(x)=@MARKER@x@' grub_script.tab.h grub_script.yy.h ../grub-core/commands/blocklist.c ../grub-core/commands/macbless.c ../grub-core/commands/xnu_uuid.c ../grub-core/commands/testload.c ../grub-core/commands/ls.c ../grub-core/disk/dmraid_nvidia.c ../grub-core/disk/loopback.c ../grub-core/disk/lvm.c ../grub-core/disk/mdraid_linux.c ../grub-core/disk/mdraid_linux_be.c ../grub-core/disk/mdraid1x_linux.c ../grub-core/disk/raid5_recover.c ../grub-core/disk/raid6_recover.c ../grub-core/font/font.c ../grub-core/gfxmenu/font.c ../grub-core/normal/charset.c ../grub-core/video/fb/fbblit.c ../grub-core/video/fb/fbutil.c ../grub-core/video/fb/fbfill.c ../grub-core/video/fb/video_fb.c ../grub-core/video/video.c ../grub-core/video/capture.c ../grub-core/video/colors.c ../grub-core/unidata.c ../grub-core/io/bufio.c ../grub-core/fs/affs.c ../grub-core/fs/afs.c ../grub-core/fs/bfs.c ../grub-core/fs/btrfs.c ../grub-core/fs/cbfs.c ../grub-core/fs/cpio.c ../grub-core/fs/cpio_be.c ../grub-core/fs/odc.c ../grub-core/fs/newc.c ../grub-core/fs/ext2.c ../grub-core/fs/fat.c ../grub-core/fs/exfat.c ../grub-core/fs/fshelp.c ../grub-core/fs/hfs.c ../grub-core/fs/hfsplus.c ../grub-core/fs/hfspluscomp.c ../grub-core/fs/iso9660.c ../grub-core/fs/jfs.c ../grub-core/fs/minix.c ../grub-core/fs/minix2.c ../grub-core/fs/minix3.c ../grub-core/fs/minix_be.c ../grub-core/fs/minix2_be.c ../grub-core/fs/minix3_be.c ../grub-core/fs/nilfs2.c ../grub-core/fs/ntfs.c ../grub-core/fs/ntfscomp.c ../grub-core/fs/reiserfs.c ../grub-core/fs/romfs.c ../grub-core/fs/sfs.c ../grub-core/fs/squash4.c ../grub-core/fs/tar.c ../grub-core/fs/udf.c ../grub-core/fs/ufs2.c ../grub-core/fs/ufs.c ../grub-core/fs/ufs_be.c ../grub-core/fs/xfs.c ../grub-core/fs/zfs/zfscrypt.c ../grub-core/fs/zfs/zfs.c ../grub-core/fs/zfs/zfsinfo.c ../grub-core/fs/zfs/zfs_lzjb.c ../grub-core/fs/zfs/zfs_lz4.c ../grub-core/fs/zfs/zfs_sha256.c ../grub-core/fs/zfs/zfs_fletcher.c ../grub-core/lib/envblk.c ../grub-core/lib/hexdump.c ../grub-core/lib/LzFind.c ../grub-core/lib/LzmaEnc.c ../grub-core/lib/crc.c ../grub-core/lib/adler32.c ../grub-core/lib/crc64.c ../grub-core/normal/datetime.c ../grub-core/normal/misc.c ../grub-core/partmap/acorn.c ../grub-core/partmap/amiga.c ../grub-core/partmap/apple.c ../grub-core/partmap/sun.c ../grub-core/partmap/plan.c ../grub-core/partmap/dvh.c ../grub-core/partmap/sunpc.c ../grub-core/partmap/bsdlabel.c ../grub-core/partmap/dfly.c ../grub-core/script/function.c ../grub-core/script/lexer.c ../grub-core/script/main.c ../grub-core/script/script.c ../grub-core/script/argv.c ../grub-core/io/gzio.c ../grub-core/io/xzio.c ../grub-core/io/lzopio.c ../grub-core/kern/ia64/dl_helper.c ../grub-core/kern/arm/dl_helper.c ../grub-core/kern/arm64/dl_helper.c ../grub-core/lib/minilzo/minilzo.c ../grub-core/lib/xzembed/xz_dec_bcj.c ../grub-core/lib/xzembed/xz_dec_lzma2.c ../grub-core/lib/xzembed/xz_dec_stream.c ../util/misc.c ../grub-core/kern/command.c ../grub-core/kern/device.c ../grub-core/kern/disk.c ../grub-core/lib/disk.c ../util/getroot.c ../grub-core/osdep/unix/getroot.c ../grub-core/osdep/getroot.c ../grub-core/osdep/devmapper/getroot.c ../grub-core/osdep/relpath.c ../grub-core/kern/emu/hostdisk.c ../grub-core/osdep/devmapper/hostdisk.c ../grub-core/osdep/hostdisk.c ../grub-core/osdep/unix/hostdisk.c ../grub-core/osdep/exec.c ../grub-core/osdep/sleep.c ../grub-core/osdep/password.c ../grub-core/kern/emu/misc.c ../grub-core/kern/emu/mm.c ../grub-core/kern/env.c ../grub-core/kern/err.c ../grub-core/kern/file.c ../grub-core/kern/fs.c ../grub-core/kern/list.c ../grub-core/kern/misc.c ../grub-core/kern/partition.c ../grub-core/lib/crypto.c ../grub-core/disk/luks.c ../grub-core/disk/geli.c ../grub-core/disk/cryptodisk.c ../grub-core/disk/AFSplitter.c ../grub-core/lib/pbkdf2.c ../grub-core/commands/extcmd.c ../grub-core/lib/arg.c ../grub-core/disk/ldm.c ../grub-core/disk/diskfilter.c ../grub-core/partmap/gpt.c ../grub-core/partmap/msdos.c ../grub-core/fs/proc.c ../grub-core/fs/archelp.c > libgrub.pp || (rm -f libgrub.pp; exit 1) rm -f stamp-h1 touch ../config-util.h.in cd . && /bin/sh ./config.status config-util.h config.status: creating config-util.h In file included from ../include/grub/mm.h:25:0, from ../include/grub/disk.h:29, from ../include/grub/file.h:26, from ../grub-core/fs/btrfs.c:21: ./config.h:38:10: fatal error: ./config-util.h: No such file or directory #include ^~~~~~~~~~~~~~~ compilation terminated. make: *** [Makefile:13098: libgrub.pp] Error 1 This is because libgrub.pp is built with -DGRUB_UTIL=1, which means it'll try to include config-util.h, but a parallel make is actually building that file. I think. Reviewed-by: Daniel Kiper 2020-03-10 Peter Jones efi: Print more debug info in our module loader The function that searches the mods section base address does not have any debug information. Add some debugging outputs that could be useful. Reviewed-by: Daniel Kiper 2020-03-10 Peter Jones linux/getroot: Handle rssd storage device names The Micron PCIe SSDs Linux driver (mtip32xx) exposes block devices as /dev/rssd[a-z]+[0-9]*. Add support for these rssd device names. Reviewed-by: Daniel Kiper 2020-03-10 Julian Andres Klode smbios: Add a --linux argument to apply linux modalias-like filtering Linux creates modalias strings by filtering out non-ASCII, space, and colon characters. Provide an option that does the same filtering so people can create a modalias string in GRUB, and then match their modalias patterns against it. Reviewed-by: Daniel Kiper 2020-03-10 Mike Gilbert po: Fix replacement of %m in sed programs When running make dist, I hit this error: rm -f en@arabic.gmo && /usr/bin/gmsgfmt -c --statistics --verbose -o en@arabic.gmo en@arabic.po en@arabic.po:5312: 'msgstr' is not a valid C format string, unlike 'msgid'. Reason: The character that terminates the directive number 3 is not a valid conversion specifier. /usr/bin/gmsgfmt: found 1 fatal error This was caused by "%m" being replaced with foreign Unicode characters. For example: msgid "cannot rename the file %s to %s: %m" msgstr "ﺹﺎﻨﻧﻮﺗ ﺮﻌﻧﺎﻤﻋ ﺖﻬﻋ ﻒִﻴﻠﻋ %s ﺕﻭ %s: %ﻡ" Mimic the workaround used for "%s" by reversing the replacement of "%m" at the end of the sed programs. Reviewed-by: Daniel Kiper 2020-03-10 Colin Watson gettext: Restore patches to po/Makefile.in.in These were inadvertently lost during the conversion to Gnulib (gnulib: Upgrade Gnulib and switch to bootstrap tool; commit 35b909062). The files in po/gettext-patches/ can be imported using "git am" on top of the gettext tag corresponding to AM_GNU_GETTEXT_VERSION in configure.ac (currently 0.18.3). They handle translation of messages in shell files, make msgfmt output in little-endian format, and arrange to use @SHELL@ rather than /bin/sh. There were some changes solely for the purpose of distributing extra files; for ease of maintenance, I've added these to conf/Makefile.extra-dist instead. Fixes: https://savannah.gnu.org/bugs/?57298 Reviewed-by: Daniel Kiper 2020-02-28 Peter Jones misc: Make grub_strtol() "end" pointers have safer const qualifiers Currently the string functions grub_strtol(), grub_strtoul(), and grub_strtoull() don't declare the "end" pointer in such a way as to require the pointer itself or the character array to be immutable to the implementation, nor does the C standard do so in its similar functions, though it does require us not to change any of it. The typical declarations of these functions follow this pattern: long strtol(const char * restrict nptr, char ** restrict endptr, int base); Much of the reason for this is historic, and a discussion of that follows below, after the explanation of this change. (GRUB currently does not include the "restrict" qualifiers, and we name the arguments a bit differently.) The implementation is semantically required to treat the character array as immutable, but such accidental modifications aren't stopped by the compiler, and the semantics for both the callers and the implementation of these functions are sometimes also helped by adding that requirement. This patch changes these declarations to follow this pattern instead: long strtol(const char * restrict nptr, const char ** const restrict endptr, int base); This means that if any modification to these functions accidentally introduces either an errant modification to the underlying character array, or an accidental assignment to endptr rather than *endptr, the compiler should generate an error. (The two uses of "restrict" in this case basically mean strtol() isn't allowed to modify the character array by going through *endptr, and endptr isn't allowed to point inside the array.) It also means the typical use case changes to: char *s = ...; const char *end; long l; l = strtol(s, &end, 10); Or even: const char *p = str; while (p && *p) { long l = strtol(p, &p, 10); ... } This fixes 26 places where we discard our attempts at treating the data safely by doing: const char *p = str; long l; l = strtol(p, (char **)&ptr, 10); It also adds 5 places where we do: char *p = str; while (p && *p) { long l = strtol(p, (const char ** const)&p, 10); ... /* more calls that need p not to be pointer-to-const */ } While moderately distasteful, this is a better problem to have. With one minor exception, I have tested that all of this compiles without relevant warnings or errors, and that /much/ of it behaves correctly, with gcc 9 using 'gcc -W -Wall -Wextra'. The one exception is the changes in grub-core/osdep/aros/hostdisk.c , which I have no idea how to build. Because the C standard defined type-qualifiers in a way that can be confusing, in the past there's been a slow but fairly regular stream of churn within our patches, which add and remove the const qualifier in many of the users of these functions. This change should help avoid that in the future, and in order to help ensure this, I've added an explanation in misc.h so that when someone does get a compiler warning about a type error, they have the fix at hand. The reason we don't have "const" in these calls in the standard is purely anachronistic: C78 (de facto) did not have type qualifiers in the syntax, and the "const" type qualifier was added for C89 (I think; it may have been later). strtol() appears to date from 4.3BSD in 1986, which means it could not be added to those functions in the standard without breaking compatibility, which is usually avoided. The syntax chosen for type qualifiers is what has led to the churn regarding usage of const, and is especially confusing on string functions due to the lack of a string type. Quoting from C99, the syntax is: declarator: pointer[opt] direct-declarator direct-declarator: identifier ( declarator ) direct-declarator [ type-qualifier-list[opt] assignment-expression[opt] ] ... direct-declarator [ type-qualifier-list[opt] * ] ... pointer: * type-qualifier-list[opt] * type-qualifier-list[opt] pointer type-qualifier-list: type-qualifier type-qualifier-list type-qualifier ... type-qualifier: const restrict volatile So the examples go like: const char foo; // immutable object const char *foo; // mutable pointer to object char * const foo; // immutable pointer to mutable object const char * const foo; // immutable pointer to immutable object const char const * const foo; // XXX extra const keyword in the middle const char * const * const foo; // immutable pointer to immutable // pointer to immutable object const char ** const foo; // immutable pointer to mutable pointer // to immutable object Making const left-associative for * and right-associative for everything else may not have been the best choice ever, but here we are, and the inevitable result is people using trying to use const (as they should!), putting it at the wrong place, fighting with the compiler for a bit, and then either removing it or typecasting something in a bad way. I won't go into describing restrict, but its syntax has exactly the same issue as with const. Anyway, the last example above actually represents the *behavior* that's required of strtol()-like functions, so that's our choice for the "end" pointer. Reviewed-by: Daniel Kiper 2020-02-28 Mike Gilbert build: Disable PIE in TARGET_CCASFLAGS if needed PIE should be disabled in assembly sources as well, or else GRUB will fail to boot. Bug: https://bugs.gentoo.org/667852 Reviewed-by: Daniel Kiper Tested-by: John Paul Adrian Glaubitz 2020-02-28 Mike Gilbert build: Move TARGET_* assignments earlier On a 32-bit SPARC userland, configure fails to compile assembly and the build fails: checking for options to compile assembly... configure: error: could not compile assembly config.log shows: asm-tests/sparc64.S: Assembler messages: asm-tests/sparc64.S:5: Error: Architecture mismatch on "lduw [%o4+4],%o4". asm-tests/sparc64.S:5: (Requires v9|v9a|v9b|v9c|v9d|v9e|v9v|v9m|m8; requested architecture is sparclite.) asm-tests/sparc64.S:7: Error: Architecture mismatch on "stw %o5,[%o3]". asm-tests/sparc64.S:7: (Requires v9|v9a|v9b|v9c|v9d|v9e|v9v|v9m|m8; requested architecture is sparclite.) asm-tests/sparc64.S:8: Error: Architecture mismatch on "bne,pt %icc,1b ,pt %icc,1b". asm-tests/sparc64.S:8: (Requires v9|v9a|v9b|v9c|v9d|v9e|v9v|v9m|m8; requested architecture is sparclite.) Simply moving these blocks earlier in configure.ac is sufficient to ensure that the tests are executed with the appropriate flags (specifically -m64 in this case). Bug: https://bugs.gentoo.org/667850 Reviewed-by: Daniel Kiper Tested-by: John Paul Adrian Glaubitz 2020-02-28 Patrick Steinhardt luks2: Add missing newline to debug message The debug message printed when decryption with a keyslot fails is missing its trailing newline. Add it to avoid mangling it with subsequent output. Reviewed-by: Daniel Kiper 2020-02-18 Michael Chang verifiers: Fix calling uninitialized function pointer The necessary check for NULL before use of function ver->close is not taking place in the failure path. This patch simply adds the missing check and fixes the problem that GRUB hangs indefinitely after booting rogue image without valid signature if secure boot is turned on. Now it displays like this for booting rogue UEFI image: error: bad shim signature error: you need to load the kernel first Press any key to continue... and then you can go back to boot menu by pressing any key or after a few seconds expired. Reviewed-by: Javier Martinez Canillas Reviewed-by: Daniel Kiper 2020-02-18 Peter Jones grub-editenv: Make grub-editenv chase symlinks including those across devices The grub-editenv create command will wrongly overwrite /boot/grub2/grubenv with a regular file if grubenv is a symbolic link. But instead, it should create a new file in the path the symlink points to. This lets /boot/grub2/grubenv be a symlink to /boot/efi/EFI/fedora/grubenv even when they're different mount points, which allows grub2-editenv to be the same across platforms (i.e. UEFI vs BIOS). For example, in Fedora the GRUB EFI builds have prefix set to /EFI/fedora (on the EFI System Partition), but for BIOS machine it'll be /boot/grub2 (which may or may not be its own mountpoint). With this patch, on EFI machines we can make /boot/grub2/grubenv a symlink to /boot/efi/EFI/fedora/grubenv, and the same copy of grub-set-default will work on both kinds of systems. Windows doesn't implement a readlink primitive, so the current behaviour is maintained for this operating system. Reviewed-by: Adam Jackson Reviewed-by: Daniel Kiper 2020-02-18 Peter Jones grub-editenv: Add grub_util_readlink() Currently grub-editenv and related tools are not able to follow symbolic links when finding their config file. For example the grub-editenv create command will wrongly overwrite a symlink in /boot/grub2/grubenv with a new regular file, instead of creating a file in the path the symlink points to. A following patch will change that and add support in grub-editenv to follow symbolic links when finding the grub environment variables file. Add a grub_util_readlink() helper function that is just a wrapper around the platform specific function to read the value of a symbolic link. This helper function will be used by the following patch for grub-editenv. The helper function is not added for Windows, since this operating system doesn't have a primitive to read the contents of a symbolic link. Reviewed-by: Adam Jackson Reviewed-by: Daniel Kiper 2020-02-18 Robert Marshall docs: Update info with grub.cfg netboot selection order Add documentation to the GRUB manual that specifies the order netboot clients use to select a GRUB configuration file. Also explain that the feature is enabled by default but can be disabled by setting the "feature_net_search_cfg" environment variable to "n" in an embedded configuration file. Reviewed-by: Daniel Kiper 2020-02-18 Paulo Flabiano Smorigo normal/main: Search for specific config files for netboot This patch implements a search for a specific configuration when the config file is on a remoteserver. It uses the following order: 1) DHCP client UUID option. 2) MAC address (in lower case hexadecimal with dash separators); 3) IP (in upper case hexadecimal) or IPv6; 4) The original grub.cfg file. This procedure is similar to what is used by pxelinux and yaboot: http://www.syslinux.org/wiki/index.php/PXELINUX#config It is enabled by default but can be disabled by setting the environment variable "feature_net_search_cfg" to "n" in an embedded configuration. Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=873406 Reviewed-by: Daniel Kiper 2020-02-18 Paulo Flabiano Smorigo net/dhcp: Set net__client{id, uuid} variables from DHCP options This patch sets a net__clientid and net__clientuuid GRUB environment variables, using the DHCP client ID and UUID options if these are found. In the same way than net__