From f215e02bf85f68d3a6106c2a1f4f7f063f819064 Mon Sep 17 00:00:00 2001 From: Daniel Baumann Date: Thu, 11 Apr 2024 10:17:27 +0200 Subject: Adding upstream version 7.0.14-dfsg. Signed-off-by: Daniel Baumann --- doc/manual/en_US/user_Technical.xml | 962 ++++++++++++++++++++++++++++++++++++ 1 file changed, 962 insertions(+) create mode 100644 doc/manual/en_US/user_Technical.xml (limited to 'doc/manual/en_US/user_Technical.xml') diff --git a/doc/manual/en_US/user_Technical.xml b/doc/manual/en_US/user_Technical.xml new file mode 100644 index 00000000..7469c9e7 --- /dev/null +++ b/doc/manual/en_US/user_Technical.xml @@ -0,0 +1,962 @@ + + + +%all.entities; +]> + + + Technical Background + + + This chapter provides additional information for readers who are + familiar with computer architecture and technology and wish to find + out more about how &product-name; works under the + hood. The contents of this chapter are not required + reading in order to use &product-name; successfully. + + + + + Where &product-name; Stores its Files + + + In &product-name;, a virtual machine and its settings are + described in a virtual machine settings file in XML format. In + addition, most virtual machines have one or more virtual hard + disks. These are typically represented by disk images, such as + those in VDI format. The location of these files may vary, + depending on the host operating system. See + . + + + + Global configuration data for &product-name; is maintained in + another location on the host. See + . + + + + + The Machine Folder + + + By default, each virtual machine has a directory on your host + computer where all the files of that machine are stored: the XML + settings file, with a .vbox file extension, + and its disk images. This is called the machine + folder. + + + + By default, this machine folder is located in a common folder + called VirtualBox VMs, which &product-name; + creates in the current system user's home directory. The + location of this home directory depends on the conventions of + the host operating system, as follows: + + + + + + + On Windows, this is the location returned by the + SHGetFolderPath function of the Windows + system library Shell32.dll, asking for the user profile. A + typical location is + C:\Users\username. + + + + + + On Linux, macOS, and Oracle Solaris, this is generally + taken from the environment variable + $HOME, except for the user + root where it is taken from the account + database. This is a workaround for the frequent trouble + caused by users using &product-name; in combination with the + tool sudo, which by default does not + reset the environment variable $HOME. + + + + A typical location on Linux and Oracle Solaris is + /home/username + and on macOS is + /Users/username. + + + + + + + For simplicity, we abbreviate the location of the home directory + as $HOME. Using that convention, the common + folder for all virtual machines is $HOME/VirtualBox + VMs. + + + + As an example, when you create a virtual machine called "Example + VM", &product-name; creates the following: + + + + + + + A machine folder: $HOME/VirtualBox VMs/Example + VM/ + + + + + + In the machine folder, a settings file: Example + VM.vbox + + + + + + In the machine folder, a virtual disk image: + Example VM.vdi. + + + + + + + This is the default layout if you use the + Create New Virtual Machine + wizard described in . Once you + start working with the VM, additional files are added. Log files + are in a subfolder called Logs, and if you + have taken snapshots, they are in a + Snapshots subfolder. For each VM, you can + change the location of its snapshots folder in the VM settings. + + + + You can change the default machine folder by selecting + Preferences from the + File menu in the &product-name; + main window. Then, in the displayed window, click on the + General tab. Alternatively, use + the VBoxManage setproperty machinefolder + command. See . + + + + + + + Global Settings + + + In addition to the files for the virtual machines, + &product-name; maintains global configuration data in the + following directory: + + + + + + + Linux and Oracle Solaris: + $HOME/.config/VirtualBox. + + + + + + Windows: + $HOME/.VirtualBox. + + + + + + macOS: + $HOME/Library/VirtualBox. + + + + + + + &product-name; creates this configuration directory + automatically, if necessary. You can specify an alternate + configuration directory by either setting the + VBOX_USER_HOME environment variable, or on + Linux or Oracle Solaris by using the standard + XDG_CONFIG_HOME variable. Since the global + VirtualBox.xml settings file points to all + other configuration files, this enables switching between + several &product-name; configurations. + + + + In this configuration directory, &product-name; stores its + global settings file, an XML file called + VirtualBox.xml. This file includes global + configuration options and a list of registered virtual machines + with pointers to their XML settings files. + + + + + + + Summary of Configuration Data Locations + + + The following table gives a brief overview of the configuration + data locations on an &product-name; host. + + + + Configuration File Locations + + + + + Setting + + + Location + + + + + + + Default machines folder + + + $HOME/VirtualBox VMs + + + + + Default disk image location + + + In each machine's folder + + + + + Machine settings file extension + + + .vbox + + + + + Media registry + + + Each machine settings file + + + + + + Media registration is done automatically when a + storage medium is attached to a VM + + + + +
+ + + + + + &product-name; XML Files + + + &product-name; uses XML for both the machine settings files and + the global configuration file, + VirtualBox.xml. + + + + All &product-name; XML files are versioned. When a new settings + file is created, for example because a new virtual machine is + created, &product-name; automatically uses the settings format + of the current &product-name; version. These files may not be + readable if you downgrade to an earlier version of + &product-name;. However, when &product-name; encounters a + settings file from an earlier version, such as after upgrading + &product-name;, it attempts to preserve the settings format as + much as possible. It will only silently upgrade the settings + format if the current settings cannot be expressed in the old + format, for example because you enabled a feature that was not + present in an earlier version of &product-name;. + + + + In such cases, &product-name; backs up the old settings file in + the virtual machine's configuration directory. If you need to go + back to the earlier version of &product-name;, then you will + need to manually copy these backup files back. + + + + We intentionally do not document the specifications of the + &product-name; XML files, as we must reserve the right to modify + them in the future. We therefore strongly suggest that you do + not edit these files manually. &product-name; provides complete + access to its configuration data through its the + VBoxManage command line tool, see + and its API, see + . + + + + + + + + + &product-name; Executables and Components + + + &product-name; was designed to be modular and flexible. When the + &product-name; graphical user interface (GUI) is opened and a VM + is started, at least the following three processes are running: + + + + + + + VBoxSVC, the &product-name; service process + which always runs in the background. This process is started + automatically by the first &product-name; client process and + exits a short time after the last client exits. The first + &product-name; service can be &vbox-mgr;, + VBoxManage, + VBoxHeadless, the web service amongst + others. The service is responsible for bookkeeping, + maintaining the state of all VMs, and for providing + communication between &product-name; components. This + communication is implemented using COM/XPCOM. + + + + + When we refer to clients here, we mean + the local clients of a particular VBoxSVC + server process, not clients in a network. &product-name; + employs its own client/server design to allow its processes + to cooperate, but all these processes run under the same + user account on the host operating system, and this is + totally transparent to the user. + + + + + + + The GUI process, VirtualBoxVM, a client + application based on the cross-platform Qt library. When + started without the option, this + application acts as &vbox-mgr;, displaying the VMs and their + settings. It then communicates settings and state changes to + VBoxSVC and also reflects changes effected + through other means, such as the VBoxManage + command. + + + + + + If the VirtualBoxVM client application is + started with the argument, it loads + the VMM library which includes the actual hypervisor and then + runs a virtual machine and provides the input and output for + the guest. + + + + + + + Any &product-name; front-end, or client, will communicate with the + service process and can both control and reflect the current + state. For example, either the VM selector or the VM window or + VBoxManage can be used to pause the running VM, and other + components will always reflect the changed state. + + + + The &product-name; GUI application, called &vbox-mgr;, is only one + of several available front ends, or clients. The complete list + shipped with &product-name; is as follows: + + + + + + + VirtualBoxVM: The Qt front end implementing + &vbox-mgr; and running VMs. + + + + + + VBoxManage: A less user-friendly but more + powerful alternative. See . + + + + + + VBoxHeadless: A VM front end which does not + directly provide any video output and keyboard or mouse input, + but enables redirection through the VirtualBox Remote Desktop + Extension. See . + + + + + + vboxwebsrv: The &product-name; web service + process which enables control of an &product-name; host + remotely. This is described in detail in the &product-name; + Software Development Kit (SDK) reference. See + . + + + + + + The &product-name; Python shell: A Python alternative to + VBoxManage. This is also described in the + SDK reference. + + + + + + + Internally, &product-name; consists of many more or less separate + components. You may encounter these when analyzing &product-name; + internal error messages or log files. These include the following: + + + + + + + IPRT: A portable runtime library which abstracts file access, + threading, and string manipulation. Whenever &product-name; + accesses host operating features, it does so through this + library for cross-platform portability. + + + + + + VMM (Virtual Machine Monitor): The heart of the hypervisor. + + + + + + EM (Execution Manager): Controls execution of guest code. + + + + + + TRPM (Trap Manager): Intercepts and processes guest traps and + exceptions. + + + + + + HM (Hardware Acceleration Manager): Provides support for VT-x + and AMD-V. + + + + + + GIM (Guest Interface Manager): Provides support for various + paravirtualization interfaces to the guest. + + + + + + PDM (Pluggable Device Manager): An abstract interface between + the VMM and emulated devices which separates device + implementations from VMM internals and makes it easy to add + new emulated devices. Through PDM, third-party developers can + add new virtual devices to &product-name; without having to + change &product-name; itself. + + + + + + PGM (Page Manager): A component that controls guest paging. + + + + + + TM (Time Manager): Handles timers and all aspects of time + inside guests. + + + + + + CFGM (Configuration Manager): Provides a tree structure which + holds configuration settings for the VM and all emulated + devices. + + + + + + SSM (Saved State Manager): Saves and loads VM state. + + + + + + VUSB (Virtual USB): A USB layer which separates emulated USB + controllers from the controllers on the host and from USB + devices. This component also enables remote USB. + + + + + + DBGF (Debug Facility): A built-in VM debugger. + + + + + + &product-name; emulates a number of devices to provide the + hardware environment that various guests need. Most of these + are standard devices found in many PC compatible machines and + widely supported by guest operating systems. For network and + storage devices in particular, there are several options for + the emulated devices to access the underlying hardware. These + devices are managed by PDM. + + + + + + Guest Additions for various guest operating systems. This is + code that is installed from within a virtual machine. See + . + + + + + + The "Main" component is special. It ties all the above bits + together and is the only public API that &product-name; + provides. All the client processes listed above use only this + API and never access the hypervisor components directly. As a + result, third-party applications that use the &product-name; + Main API can rely on the fact that it is always well-tested + and that all capabilities of &product-name; are fully exposed. + It is this API that is described in the &product-name; SDK. + See . + + + + + + + + + + Hardware Virtualization + + + &product-name; enables software in the virtual machine to run + directly on the processor of the host, but an array of complex + techniques is employed to intercept operations that would + interfere with your host. Whenever the guest attempts to do + something that could be harmful to your computer and its data, + &product-name; steps in and takes action. In particular, for lots + of hardware that the guest believes to be accessing, + &product-name; simulates a certain virtual + environment according to how you have configured a virtual + machine. For example, when the guest attempts to access a hard + disk, &product-name; redirects these requests to whatever you have + configured to be the virtual machine's virtual hard disk. This is + normally an image file on your host. + + + + Unfortunately, the x86 platform was never designed to be + virtualized. Detecting situations in which &product-name; needs to + take control over the guest code that is executing, as described + above, is difficult. To achieve this, &product-name; uses + hardware virtualization. + + + + Intel and AMD processors have support for hardware virtualization. + This means that these processors can help &product-name; to + intercept potentially dangerous operations that a guest operating + system may be attempting and also makes it easier to present + virtual hardware to a virtual machine. + + + + These hardware features differ between Intel and AMD processors. + Intel named its technology VT-x, AMD calls theirs AMD-V. The Intel + and AMD support for virtualization is very different in detail, + but not very different in principle. + + + + + On many systems, the hardware virtualization features first need + to be enabled in the BIOS before &product-name; can use them. + + + + + Enabling hardware virtualization is required + in the following scenarios: + + + + + + + Certain rare guest operating systems like OS/2 make use of + very esoteric processor instructions. For virtual machines + that are configured to use such an operating system, hardware + virtualization is enabled automatically. + + + + + + &product-name;'s 64-bit guest and multiprocessing (SMP) + support both require hardware virtualization to be enabled. + This is not much of a limitation since the vast majority of + 64-bit and multicore CPUs ship with hardware virtualization. + The exceptions to this rule are some legacy Intel and AMD + CPUs. + + + + + + + + Do not run other hypervisors, either open source or commercial + virtualization products, together with &product-name;. While + several hypervisors can normally be + installed in parallel, do not attempt to + run several virtual machines from competing + hypervisors at the same time. &product-name; cannot track what + another hypervisor is currently attempting to do on the same + host, and especially if several products attempt to use hardware + virtualization features such as VT-x, this can crash the entire + host. + + + + + See for a technical discussion of + hardware virtualization. + + + + + + + Details About Hardware Virtualization + + + With Intel VT-x, there are two distinct modes of CPU operation: + VMX root mode and non-root mode. + + + + + + + In root mode, the CPU operates much like older generations of + processors without VT-x support. There are four privilege + levels, called rings, and the same instruction set is + supported, with the addition of several virtualization + specific instruction. Root mode is what a host operating + system without virtualization uses, and it is also used by a + hypervisor when virtualization is active. + + + + + + In non-root mode, CPU operation is significantly different. + There are still four privilege rings and the same instruction + set, but a new structure called VMCS (Virtual Machine Control + Structure) now controls the CPU operation and determines how + certain instructions behave. Non-root mode is where guest + systems run. + + + + + + + Switching from root mode to non-root mode is called "VM entry", + the switch back is "VM exit". The VMCS includes a guest and host + state area which is saved/restored at VM entry and exit. Most + importantly, the VMCS controls which guest operations will cause + VM exits. + + + + The VMCS provides fairly fine-grained control over what the guests + can and cannot do. For example, a hypervisor can allow a guest to + write certain bits in shadowed control registers, but not others. + This enables efficient virtualization in cases where guests can be + allowed to write control bits without disrupting the hypervisor, + while preventing them from altering control bits over which the + hypervisor needs to retain full control. The VMCS also provides + control over interrupt delivery and exceptions. + + + + Whenever an instruction or event causes a VM exit, the VMCS + contains information about the exit reason, often with + accompanying detail. For example, if a write to the CR0 register + causes an exit, the offending instruction is recorded, along with + the fact that a write access to a control register caused the + exit, and information about source and destination register. Thus + the hypervisor can efficiently handle the condition without + needing advanced techniques such as CSAM and PATM described above. + + + + VT-x inherently avoids several of the problems which software + virtualization faces. The guest has its own completely separate + address space not shared with the hypervisor, which eliminates + potential clashes. Additionally, guest OS kernel code runs at + privilege ring 0 in VMX non-root mode, obviating the problems by + running ring 0 code at less privileged levels. For example the + SYSENTER instruction can transition to ring 0 without causing + problems. Naturally, even at ring 0 in VMX non-root mode, any I/O + access by guest code still causes a VM exit, allowing for device + emulation. + + + + The biggest difference between VT-x and AMD-V is that AMD-V + provides a more complete virtualization environment. VT-x requires + the VMX non-root code to run with paging enabled, which precludes + hardware virtualization of real-mode code and non-paged + protected-mode software. This typically only includes firmware and + OS loaders, but nevertheless complicates VT-x hypervisor + implementation. AMD-V does not have this restriction. + + + + Of course hardware virtualization is not perfect. Compared to + software virtualization, the overhead of VM exits is relatively + high. This causes problems for devices whose emulation requires + high number of traps. One example is a VGA device in 16-color + mode, where not only every I/O port access but also every access + to the framebuffer memory must be trapped. + + + + + + + Paravirtualization Providers + + + &product-name; enables the exposure of a paravirtualization + interface, to facilitate accurate and efficient execution of + software within a virtual machine. These interfaces require the + guest operating system to recognize their presence and make use of + them in order to leverage the benefits of communicating with the + &product-name; hypervisor. + + + + Most modern, mainstream guest operating systems, including Windows + and Linux, ship with support for one or more paravirtualization + interfaces. Hence, there is typically no need to install + additional software in the guest to take advantage of this + feature. + + + + Exposing a paravirtualization provider to the guest operating + system does not rely on the choice of host platforms. For example, + the Hyper-V paravirtualization provider can + be used for VMs to run on any host platform supported by + &product-name; and not just Windows. + + + + &product-name; provides the following interfaces: + + + + + + + Minimal: Announces the + presence of a virtualized environment. Additionally, reports + the TSC and APIC frequency to the guest operating system. This + provider is mandatory for running any Mac OS X guests. + + + + + + KVM: Presents a Linux KVM + hypervisor interface which is recognized by Linux kernels + version 2.6.25 or later. &product-name;'s implementation + currently supports paravirtualized clocks and SMP spinlocks. + This provider is recommended for Linux guests. + + + + + + Hyper-V: Presents a Microsoft + Hyper-V hypervisor interface which is recognized by Windows 7 + and newer operating systems. &product-name;'s implementation + currently supports paravirtualized clocks, APIC frequency + reporting, guest debugging, guest crash reporting and relaxed + timer checks. This provider is recommended for Windows guests. + + + + + + + + + + Nested Paging and VPIDs + + + In addition to normal hardware virtualization, your processor may + also support the following additional sophisticated techniques: + + + + + + + Nested paging implements some memory management in hardware, + which can greatly accelerate hardware virtualization since + these tasks no longer need to be performed by the + virtualization software. + + + + With nested paging, the hardware provides another level of + indirection when translating linear to physical addresses. + Page tables function as before, but linear addresses are now + translated to "guest physical" addresses first and not + physical addresses directly. A new set of paging registers now + exists under the traditional paging mechanism and translates + from guest physical addresses to host physical addresses, + which are used to access memory. + + + + Nested paging eliminates the overhead caused by VM exits and + page table accesses. In essence, with nested page tables the + guest can handle paging without intervention from the + hypervisor. Nested paging thus significantly improves + virtualization performance. + + + + On AMD processors, nested paging has been available starting + with the Barcelona (K10) architecture. They now call it rapid + virtualization indexing (RVI). Intel added support for nested + paging, which they call extended page tables (EPT), with their + Core i7 (Nehalem) processors. + + + + If nested paging is enabled, the &product-name; hypervisor can + also use large pages to reduce TLB usage + and overhead. This can yield a performance improvement of up + to 5%. To enable this feature for a VM, you use the + VBoxManage modifyvm --large-pages command. + See . + + + + If you have an Intel CPU with EPT, please consult + for security concerns + regarding EPT. + + + + + + On Intel CPUs, a hardware feature called Virtual Processor + Identifiers (VPIDs) can greatly accelerate context switching + by reducing the need for expensive flushing of the processor's + Translation Lookaside Buffers (TLBs). + + + + To enable these features for a VM, you use the + VBoxManage modifyvm --vtx-vpid and + VBoxManage modifyvm --large-pages commands. + See . + + + + + + + + -- cgit v1.2.3