summaryrefslogtreecommitdiffstats
path: root/docs/plat/rpi3.rst
diff options
context:
space:
mode:
Diffstat (limited to 'docs/plat/rpi3.rst')
-rw-r--r--docs/plat/rpi3.rst466
1 files changed, 466 insertions, 0 deletions
diff --git a/docs/plat/rpi3.rst b/docs/plat/rpi3.rst
new file mode 100644
index 0000000..38c3dfa
--- /dev/null
+++ b/docs/plat/rpi3.rst
@@ -0,0 +1,466 @@
+Raspberry Pi 3
+==============
+
+The `Raspberry Pi 3`_ is an inexpensive single-board computer that contains four
+Arm Cortex-A53 cores.
+
+The following instructions explain how to use this port of the TF-A with the
+default distribution of `Raspbian`_ because that's the distribution officially
+supported by the Raspberry Pi Foundation. At the moment of writing this, the
+officially supported kernel is a AArch32 kernel. This doesn't mean that this
+port of TF-A can't boot a AArch64 kernel. The `Linux tree fork`_ maintained by
+the Foundation can be compiled for AArch64 by following the steps in
+`AArch64 kernel build instructions`_.
+
+**IMPORTANT NOTE**: This port isn't secure. All of the memory used is DRAM,
+which is available from both the Non-secure and Secure worlds. This port
+shouldn't be considered more than a prototype to play with and implement
+elements like PSCI to support the Linux kernel.
+
+Design
+------
+
+The SoC used by the Raspberry Pi 3 is the Broadcom BCM2837. It is a SoC with a
+VideoCore IV that acts as primary processor (and loads everything from the SD
+card) and is located between all Arm cores and the DRAM. Check the `Raspberry Pi
+3 documentation`_ for more information.
+
+This explains why it is possible to change the execution state (AArch64/AArch32)
+depending on a few files on the SD card. We only care about the cases in which
+the cores boot in AArch64 mode.
+
+The rules are simple:
+
+- If a file called ``kernel8.img`` is located on the ``boot`` partition of the
+ SD card, it will load it and execute in EL2 in AArch64. Basically, it executes
+ a `default AArch64 stub`_ at address **0x0** that jumps to the kernel.
+
+- If there is also a file called ``armstub8.bin``, it will load it at address
+ **0x0** (instead of the default stub) and execute it in EL3 in AArch64. All
+ the cores are powered on at the same time and start at address **0x0**.
+
+This means that we can use the default AArch32 kernel provided in the official
+`Raspbian`_ distribution by renaming it to ``kernel8.img``, while TF-A and
+anything else we need is in ``armstub8.bin``. This way we can forget about the
+default bootstrap code. When using a AArch64 kernel, it is only needed to make
+sure that the name on the SD card is ``kernel8.img``.
+
+Ideally, we want to load the kernel and have all cores available, which means
+that we need to make the secondary cores work in the way the kernel expects, as
+explained in `Secondary cores`_. In practice, a small bootstrap is needed
+between TF-A and the kernel.
+
+To get the most out of a AArch32 kernel, we want to boot it in Hypervisor mode
+in AArch32. This means that BL33 can't be in EL2 in AArch64 mode. The
+architecture specifies that AArch32 Hypervisor mode isn't present when AArch64
+is used for EL2. When using a AArch64 kernel, it should simply start in EL2.
+
+Placement of images
+~~~~~~~~~~~~~~~~~~~
+
+The file ``armstub8.bin`` contains BL1 and the FIP. It is needed to add padding
+between them so that the addresses they are loaded to match the ones specified
+when compiling TF-A. This is done automatically by the build system.
+
+The device tree block is loaded by the VideoCore loader from an appropriate
+file, but we can specify the address it is loaded to in ``config.txt``.
+
+The file ``kernel8.img`` contains a kernel image that is loaded to the address
+specified in ``config.txt``. The `Linux kernel tree`_ has information about how
+a AArch32 Linux kernel image is loaded in ``Documentation/arm/Booting``:
+
+::
+
+ The zImage may also be placed in system RAM and called there. The
+ kernel should be placed in the first 128MiB of RAM. It is recommended
+ that it is loaded above 32MiB in order to avoid the need to relocate
+ prior to decompression, which will make the boot process slightly
+ faster.
+
+There are no similar restrictions for AArch64 kernels, as specified in the file
+``Documentation/arm64/booting.txt``.
+
+This means that we need to avoid the first 128 MiB of RAM when placing the
+TF-A images (and specially the first 32 MiB, as they are directly used to
+place the uncompressed AArch32 kernel image. This way, both AArch32 and
+AArch64 kernels can be placed at the same address.
+
+In the end, the images look like the following diagram when placed in memory.
+All addresses are Physical Addresses from the point of view of the Arm cores.
+Again, note that this is all just part of the same DRAM that goes from
+**0x00000000** to **0x3F000000**, it just has different names to simulate a real
+secure platform!
+
+::
+
+ 0x00000000 +-----------------+
+ | ROM | BL1
+ 0x00020000 +-----------------+
+ | FIP |
+ 0x00200000 +-----------------+
+ | |
+ | ... |
+ | |
+ 0x01000000 +-----------------+
+ | DTB | (Loaded by the VideoCore)
+ +-----------------+
+ | |
+ | ... |
+ | |
+ 0x02000000 +-----------------+
+ | Kernel | (Loaded by the VideoCore)
+ +-----------------+
+ | |
+ | ... |
+ | |
+ 0x10000000 +-----------------+
+ | Secure SRAM | BL2, BL31
+ 0x10100000 +-----------------+
+ | Secure DRAM | BL32 (Secure payload)
+ 0x11000000 +-----------------+
+ | Non-secure DRAM | BL33
+ +-----------------+
+ | |
+ | ... |
+ | |
+ 0x3F000000 +-----------------+
+ | I/O |
+ 0x40000000 +-----------------+
+
+The area between **0x10000000** and **0x11000000** has to be manually protected
+so that the kernel doesn't use it. The current port tries to modify the live DTB
+to add a memreserve region that reserves the previously mentioned area.
+
+If this is not possible, the user may manually add ``memmap=16M$256M`` to the
+command line passed to the kernel in ``cmdline.txt``. See the `Setup SD card`_
+instructions to see how to do it. This system is strongly discouraged.
+
+The last 16 MiB of DRAM can only be accessed by the VideoCore, that has
+different mappings than the Arm cores in which the I/O addresses don't overlap
+the DRAM. The memory reserved to be used by the VideoCore is always placed at
+the end of the DRAM, so this space isn't wasted.
+
+Considering the 128 MiB allocated to the GPU and the 16 MiB allocated for
+TF-A, there are 880 MiB available for Linux.
+
+Boot sequence
+~~~~~~~~~~~~~
+
+The boot sequence of TF-A is the usual one except when booting an AArch32
+kernel. In that case, BL33 is booted in AArch32 Hypervisor mode so that it
+can jump to the kernel in the same mode and let it take over that privilege
+level. If BL33 was running in EL2 in AArch64 (as in the default bootflow of
+TF-A) it could only jump to the kernel in AArch32 in Supervisor mode.
+
+The `Linux kernel tree`_ has instructions on how to jump to the Linux kernel
+in ``Documentation/arm/Booting`` and ``Documentation/arm64/booting.txt``. The
+bootstrap should take care of this.
+
+This port support a direct boot of the Linux kernel from the firmware (as a BL33
+image). Alternatively, U-Boot or other bootloaders may be used.
+
+Secondary cores
+~~~~~~~~~~~~~~~
+
+This port of the Trusted Firmware-A supports ``PSCI_CPU_ON``,
+``PSCI_SYSTEM_RESET`` and ``PSCI_SYSTEM_OFF``. The last one doesn't really turn
+the system off, it simply reboots it and asks the VideoCore firmware to keep it
+in a low power mode permanently.
+
+The kernel used by `Raspbian`_ doesn't have support for PSCI, so it is needed to
+use mailboxes to trap the secondary cores until they are ready to jump to the
+kernel. This mailbox is located at a different address in the AArch32 default
+kernel than in the AArch64 kernel.
+
+Kernels with PSCI support can use the PSCI calls instead for a cleaner boot.
+
+Also, this port of TF-A has another Trusted Mailbox in Shared BL RAM. During
+cold boot, all secondary cores wait in a loop until they are given given an
+address to jump to in this Mailbox (``bl31_warm_entrypoint``).
+
+Once BL31 has finished and the primary core has jumped to the BL33 payload, it
+has to call ``PSCI_CPU_ON`` to release the secondary CPUs from the wait loop.
+The payload then makes them wait in another waitloop listening from messages
+from the kernel. When the primary CPU jumps into the kernel, it will send an
+address to the mailbox so that the secondary CPUs jump to it and are recognised
+by the kernel.
+
+Build Instructions
+------------------
+
+To boot a AArch64 kernel, only the AArch64 toolchain is required.
+
+To boot a AArch32 kernel, both AArch64 and AArch32 toolchains are required. The
+AArch32 toolchain is needed for the AArch32 bootstrap needed to load a 32-bit
+kernel.
+
+The build system concatenates BL1 and the FIP so that the addresses match the
+ones in the memory map. The resulting file is ``armstub8.bin``, located in the
+build folder (e.g. ``build/rpi3/debug/armstub8.bin``). To know how to use this
+file, follow the instructions in `Setup SD card`_.
+
+The following build options are supported:
+
+- ``RPI3_BL33_IN_AARCH32``: This port can load a AArch64 or AArch32 BL33 image.
+ By default this option is 0, which means that TF-A will jump to BL33 in EL2
+ in AArch64 mode. If set to 1, it will jump to BL33 in Hypervisor in AArch32
+ mode.
+
+- ``PRELOADED_BL33_BASE``: Used to specify the address of a BL33 binary that has
+ been preloaded by any other system than using the firmware. ``BL33`` isn't
+ needed in the build command line if this option is used. Specially useful
+ because the file ``kernel8.img`` can be loaded anywhere by modifying the file
+ ``config.txt``. It doesn't have to contain a kernel, it could have any
+ arbitrary payload.
+
+- ``RPI3_DIRECT_LINUX_BOOT``: Disabled by default. Set to 1 to enable the direct
+ boot of the Linux kernel from the firmware. Option ``RPI3_PRELOADED_DTB_BASE``
+ is mandatory when the direct Linux kernel boot is used. Options
+ ``PRELOADED_BL33_BASE`` will most likely be needed as well because it is
+ unlikely that the kernel image will fit in the space reserved for BL33 images.
+ This option can be combined with ``RPI3_BL33_IN_AARCH32`` in order to boot a
+ 32-bit kernel. The only thing this option does is to set the arguments in
+ registers x0-x3 or r0-r2 as expected by the kernel.
+
+- ``RPI3_PRELOADED_DTB_BASE``: Auxiliary build option needed when using
+ ``RPI3_DIRECT_LINUX_BOOT=1``. This option allows to specify the location of a
+ DTB in memory.
+
+- ``RPI3_RUNTIME_UART``: Indicates whether the UART should be used at runtime
+ or disabled. ``-1`` (default) disables the runtime UART. Any other value
+ enables the default UART (currently UART1) for runtime messages.
+
+- ``RPI3_USE_UEFI_MAP``: Set to 1 to build ATF with the altername memory
+ mapping required for an UEFI firmware payload. These changes are needed
+ to be able to run Windows on ARM64. This option, which is disabled by
+ default, results in the following memory mappings:
+
+::
+
+ 0x00000000 +-----------------+
+ | ROM | BL1
+ 0x00010000 +-----------------+
+ | DTB | (Loaded by the VideoCore)
+ 0x00020000 +-----------------+
+ | FIP |
+ 0x00030000 +-----------------+
+ | |
+ | UEFI PAYLOAD |
+ | |
+ 0x00200000 +-----------------+
+ | Secure SRAM | BL2, BL31
+ 0x00300000 +-----------------+
+ | Secure DRAM | BL32 (Secure payload)
+ 0x00400000 +-----------------+
+ | |
+ | |
+ | Non-secure DRAM | BL33
+ | |
+ | |
+ 0x01000000 +-----------------+
+ | |
+ | ... |
+ | |
+ 0x3F000000 +-----------------+
+ | I/O |
+
+- ``BL32``: This port can load and run OP-TEE. The OP-TEE image is optional.
+ Please use the code from `here <https://github.com/OP-TEE/optee_os>`__.
+ Build the Trusted Firmware with option ``BL32=tee-header_v2.bin
+ BL32_EXTRA1=tee-pager_v2.bin BL32_EXTRA2=tee-pageable_v2.bin``
+ to put the binaries into the FIP.
+
+ .. warning::
+ If OP-TEE is used it may be needed to add the following options to the
+ Linux command line so that the USB driver doesn't use FIQs:
+ ``dwc_otg.fiq_enable=0 dwc_otg.fiq_fsm_enable=0 dwc_otg.nak_holdoff=0``.
+ This will unfortunately reduce the performance of the USB driver. It is
+ needed when using Raspbian, for example.
+
+- ``TRUSTED_BOARD_BOOT``: This port supports TBB. Set this option to 1 to enable
+ it. In order to use TBB, you might want to set ``GENERATE_COT=1`` to let the
+ contents of the FIP automatically signed by the build process. The ROT key
+ will be generated and output to ``rot_key.pem`` in the build directory. It is
+ able to set ROT_KEY to your own key in PEM format. Also in order to build,
+ you need to clone mbed TLS from `here <https://github.com/ARMmbed/mbedtls>`__.
+ ``MBEDTLS_DIR`` must point at the mbed TLS source directory.
+
+- ``ENABLE_STACK_PROTECTOR``: Disabled by default. It uses the hardware RNG of
+ the board.
+
+The following is not currently supported:
+
+- AArch32 for TF-A itself.
+
+- ``EL3_PAYLOAD_BASE``: The reason is that you can already load anything to any
+ address by changing the file ``armstub8.bin``, so there's no point in using
+ TF-A in this case.
+
+- ``MULTI_CONSOLE_API=0``: The multi console API must be enabled. Note that the
+ crash console uses the internal 16550 driver functions directly in order to be
+ able to print error messages during early crashes before setting up the
+ multi console API.
+
+Building the firmware for kernels that don't support PSCI
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This is the case for the 32-bit image of Raspbian, for example. 64-bit kernels
+always support PSCI, but they may not know that the system understands PSCI due
+to an incorrect DTB file.
+
+First, clone and compile the 32-bit version of the `Raspberry Pi 3 TF-A
+bootstrap`_. Choose the one needed for the architecture of your kernel.
+
+Then compile TF-A. For a 32-bit kernel, use the following command line:
+
+.. code:: shell
+
+ CROSS_COMPILE=aarch64-linux-gnu- make PLAT=rpi3 \
+ RPI3_BL33_IN_AARCH32=1 \
+ BL33=../rpi3-arm-tf-bootstrap/aarch32/el2-bootstrap.bin
+
+For a 64-bit kernel, use this other command line:
+
+.. code:: shell
+
+ CROSS_COMPILE=aarch64-linux-gnu- make PLAT=rpi3 \
+ BL33=../rpi3-arm-tf-bootstrap/aarch64/el2-bootstrap.bin
+
+However, enabling PSCI support in a 64-bit kernel is really easy. In the
+repository `Raspberry Pi 3 TF-A bootstrap`_ there is a patch that can be applied
+to the Linux kernel tree maintained by the Raspberry Pi foundation. It modifes
+the DTS to tell the kernel to use PSCI. Once this patch is applied, follow the
+instructions in `AArch64 kernel build instructions`_ to get a working 64-bit
+kernel image and supporting files.
+
+Building the firmware for kernels that support PSCI
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For a 64-bit kernel:
+
+.. code:: shell
+
+ CROSS_COMPILE=aarch64-linux-gnu- make PLAT=rpi3 \
+ PRELOADED_BL33_BASE=0x02000000 \
+ RPI3_PRELOADED_DTB_BASE=0x01000000 \
+ RPI3_DIRECT_LINUX_BOOT=1
+
+For a 32-bit kernel:
+
+.. code:: shell
+
+ CROSS_COMPILE=aarch64-linux-gnu- make PLAT=rpi3 \
+ PRELOADED_BL33_BASE=0x02000000 \
+ RPI3_PRELOADED_DTB_BASE=0x01000000 \
+ RPI3_DIRECT_LINUX_BOOT=1 \
+ RPI3_BL33_IN_AARCH32=1
+
+AArch64 kernel build instructions
+---------------------------------
+
+The following instructions show how to install and run a AArch64 kernel by
+using a SD card with the default `Raspbian`_ install as base. Skip them if you
+want to use the default 32-bit kernel.
+
+Note that this system won't be fully 64-bit because all the tools in the
+filesystem are 32-bit binaries, but it's a quick way to get it working, and it
+allows the user to run 64-bit binaries in addition to 32-bit binaries.
+
+1. Clone the `Linux tree fork`_ maintained by the Raspberry Pi Foundation. To
+ speed things up, do a shallow clone of the desired branch.
+
+.. code:: shell
+
+ git clone --depth=1 -b rpi-4.18.y https://github.com/raspberrypi/linux
+ cd linux
+
+2. Configure and compile the kernel. Adapt the number after ``-j`` so that it is
+ 1.5 times the number of CPUs in your computer. This may take some time to
+ finish.
+
+.. code:: shell
+
+ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bcmrpi3_defconfig
+ make -j 6 ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu-
+
+3. Copy the kernel image and the device tree to the SD card. Replace the path
+ by the corresponding path in your computers to the ``boot`` partition of the
+ SD card.
+
+.. code:: shell
+
+ cp arch/arm64/boot/Image /path/to/boot/kernel8.img
+ cp arch/arm64/boot/dts/broadcom/bcm2710-rpi-3-b.dtb /path/to/boot/
+ cp arch/arm64/boot/dts/broadcom/bcm2710-rpi-3-b-plus.dtb /path/to/boot/
+
+4. Install the kernel modules. Replace the path by the corresponding path to the
+ filesystem partition of the SD card on your computer.
+
+.. code:: shell
+
+ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
+ INSTALL_MOD_PATH=/path/to/filesystem modules_install
+
+5. Follow the instructions in `Setup SD card`_ except for the step of renaming
+ the existing ``kernel7.img`` (we have already copied a AArch64 kernel).
+
+Setup SD card
+-------------
+
+The instructions assume that you have an SD card with a fresh install of
+`Raspbian`_ (or that, at least, the ``boot`` partition is untouched, or nearly
+untouched). They have been tested with the image available in 2018-03-13.
+
+1. Insert the SD card and open the ``boot`` partition.
+
+2. Rename ``kernel7.img`` to ``kernel8.img``. This tricks the VideoCore
+ bootloader into booting the Arm cores in AArch64 mode, like TF-A needs,
+ even though the kernel is not compiled for AArch64.
+
+3. Copy ``armstub8.bin`` here. When ``kernel8.img`` is available, The VideoCore
+ bootloader will look for a file called ``armstub8.bin`` and load it at
+ address **0x0** instead of a predefined one.
+
+4. To enable the serial port "Mini UART" in Linux, open ``cmdline.txt`` and add
+ ``console=serial0,115200 console=tty1``.
+
+5. Open ``config.txt`` and add the following lines at the end (``enable_uart=1``
+ is only needed to enable debugging through the Mini UART):
+
+::
+
+ enable_uart=1
+ kernel_address=0x02000000
+ device_tree_address=0x01000000
+
+If you connect a serial cable to the Mini UART and your computer, and connect
+to it (for example, with ``screen /dev/ttyUSB0 115200``) you should see some
+text. In the case of an AArch32 kernel, you should see something like this:
+
+::
+
+ NOTICE: Booting Trusted Firmware
+ NOTICE: BL1: v1.4(release):v1.4-329-g61e94684-dirty
+ NOTICE: BL1: Built : 00:09:25, Nov 6 2017
+ NOTICE: BL1: Booting BL2
+ NOTICE: BL2: v1.4(release):v1.4-329-g61e94684-dirty
+ NOTICE: BL2: Built : 00:09:25, Nov 6 2017
+ NOTICE: BL1: Booting BL31
+ NOTICE: BL31: v1.4(release):v1.4-329-g61e94684-dirty
+ NOTICE: BL31: Built : 00:09:25, Nov 6 2017
+ [ 0.266484] bcm2835-aux-uart 3f215040.serial: could not get clk: -517
+
+ Raspbian GNU/Linux 9 raspberrypi ttyS0
+ raspberrypi login:
+
+Just enter your credentials, everything should work as expected. Note that the
+HDMI output won't show any text during boot.
+
+.. _default Arm stub: https://github.com/raspberrypi/tools/blob/master/armstubs/armstub7.S
+.. _default AArch64 stub: https://github.com/raspberrypi/tools/blob/master/armstubs/armstub8.S
+.. _Linux kernel tree: https://github.com/torvalds/linux
+.. _Linux tree fork: https://github.com/raspberrypi/linux
+.. _Raspberry Pi 3: https://www.raspberrypi.org/products/raspberry-pi-3-model-b/
+.. _Raspberry Pi 3 TF-A bootstrap: https://github.com/AntonioND/rpi3-arm-tf-bootstrap
+.. _Raspberry Pi 3 documentation: https://www.raspberrypi.org/documentation/
+.. _Raspbian: https://www.raspberrypi.org/downloads/raspbian/