Josef “Jeff” Sipek

Raspberry Pi Bootloader

As I mentioned previously, I decided to do some hardware hacking and as a result I ended up with a Raspberry Pi B+. After playing with Raspbian for a bit, I started trying to read up on how the Pi boots. That’s what today’s post is about.

Standalone UART

While searching for information about how the Pi boots, I stumbled across a git repo with a standalone UART program. Conveniently, the repo includes ELF files as well as raw binary images. (This actually made me go “ewww” at first.)

Before even running it, I looked at the code and saw that it prints out 0x12345678 followed by the PC of one of the first instructions of the program. Pretty minimal, but quite enough.

Boot Process

Just about everyone on the internet (and their grandmothers!) knows that the Pi has a multi stage boot. First of all, it is the GPU that starts executing first. The ARM core just sits there waiting for a reset.

The Pi requires an SD card to boot — on it must be a FAT formatted partition with a couple of files that you can download from the Raspberry Pi Foundation.

Here’s the short version of what happens:

  1. Some sort of baked-in firmware loads bootcode.bin from the SD card and executes it.
  2. bootcode.bin does a bit of setup, and then loads and executes start.elf.
  3. start.elf does a whole lot of setup.

    1. Reads config.txt which is a text file with a bunch of options.
    2. Splits the RAM between the GPU and CPU (with the help of fixup.dat).
    3. Loads the kernel and the ramdisk.
    4. Loads the device tree file or sets up ATAGs.
    5. Resets the ARM core. The ARM core then begins executing the kernel.

This sounds pretty nice, right? For the most part, it is. But as they say, the devil’s in the details.

Booting ELF Files

It doesn’t take a lot of reading to figure out that start.elf can deal with kernel files in ELF format. This made me quite happy since I’ve written an ELF loader before for HVF and while it wasn’t difficult, I didn’t want to go through the exercise just to get something booting.

So, I copied uart02.elf to the SD card, and made a minimal config file:

kernel=uart02.elf

A power-cycle later, I saw the two hex numbers. (Ok, this is a bit of a distortion of reality. It took far too many reboots because I was playing with multiple things at the same time — including U-Boot which turned out to be a total waste of my time.)

The PC was not what I expected it to be. It was 0x8080 instead of 0x800c. After a lot of trial and error, I realized that it just so happened that the .text section is 0x74 bytes into the ELF file. Then I had a horrifying thought… start.elf understands ELF files enough to jump to the right instruction but it does nothing to make the contents of the ELF file properly laid out in memory. It just reads the file into memory starting at address 0x8000, gets the start address from the ELF header, converts it into a file offset and jumps to that location in memory. This is wrong.

Sure enough, booting uart02.bin printed the right number. So much for avoiding writing my own ELF loader.

Ramdisk

Once I had a reliable way to get code to execute and communicate via the serial port, I started playing with ramdisks. The code I was booting parsed the ATAGs and looked for ATAG_INITRD2 which describes the location of the ramdisk.

So, I extended my config to include the ramfsfile parameter to specify the filename of the ramdisk:

kernel=foo
ramfsfile=initrd

Reboot, aaaand…the code panicked because it couldn’t find ATAG_INITRD2. It was weird, the file was present, no misspellings, etc. After a while of rebooting and trying different things, I decide to use the initramfs option and just pick some arbitrary high address for the ramdisk to be loaded at. The config changed to:

kernel=foo
initramfs initrd 0x800000

Another reboot later, I was looking at a dump of the ATAG_INITRD2. Everything worked as expected! So, it turns out that the boot loader on the Pi is not capable of picking an address for the initial ramdisk by itself.

Command line

Finally, I just had to experiment with passing a command line string. I created a file called cmdline.txt on the SD card with some text. A reboot later, I saw that the dump of the ATAG_CMDLINE had some cruft prepended. The entire value of the ATAG looked like (with some spaces replaced by line breaks):

dma.dmachans=0x7f35 bcm2708_fb.fbwidth=656 bcm2708_fb.fbheight=416
bcm2708.boardrev=0x10 bcm2708.serial=0xbd225074
smsc95xx.macaddr=B8:27:EB:45:74:FA bcm2708_fb.fbswap=1
bcm2708.disk_led_gpio=47 bcm2708.disk_led_active_low=0
sdhci-bcm2708.emmc_clock_freq=250000000 vc_mem.mem_base=0x1ec00000
vc_mem.mem_size=0x20000000  foo

This isn’t exactly the worst thing, but it does mean that the option parsing code has to handle cruft prepended to what it would expect the command line to look. I wish that these settings were passed via a separate ATAG.

4 Comments »

  1. That the GPU boots the CPU continues to seem backwards and disturbing to me. Do you know if there's been progress on an open replacement for the GPU blob? Your investigations look interesting - it seems as though the bootloader is rather hasty. I guess it was good enough for their purposes?

    Comment by Steve — February 20, 2015 @ 21:02

  2. Heh, yeah... the GPU booting the CPU does seem backwards but only because our brains are conditioned to consider the GPU to be a PCI/whatever device. On the Pi, you are dealing with a SoC that has all the bits in one place. A GPU is a processor too, so who's to say that this processor shouldn't be the one to boot the whole thing? :)

    Supposedly, Broadcom released docs for the GPU just about a year ago. I haven't heard anything about the unencumbered bootcode.bin, but I haven't looked for it specifically. My hands are quite full without thinking about the GPU :)

    I totally expect the ELF issues to be caused by a misunderstanding of how ELF works. Perhaps even a work-in-progress that got abandoned (Linux kernel does not use the ELF support to boot)!

    Comment by JeffPC — February 21, 2015 @ 13:56

  3. Hello Jeff. I'm not as up on the Raspberry Pi (and probably technology in general) as you are. Thanks for publishing your blog and posting this info. I have a Raspberry Pi 3 (model 'B' I think) that I've added a 3.5" LCD touchscreen to (it's the Kumantech LCD - details in the link shared below). With the correct drivers install in Raspbian, I have a full-fledged handheld Debian based device that can be held in the palm of my hand and has a working LCD display that supported touch operation. It's really cool.

    However, I run BerryBoot and there doesn't seem to be a means to enable the LCD display in BerryBoot. I've tried to contact the Kumantech folks, as well as the BerryTerminal folks, without much luck.

    I know you're not a Raspberry Pi tech support site, so I'm not looking for you to solve my problems. I came across your blog while researching Raspberry Pi bootloaders. Wondering if you have some thoughts around the issue I face, and also wanting to understand if based on your findings, there is an alternative bootloader available for the Raspberry Pi where the bootloader outputs and accepts input on either a non-HDMI LCD display or serial interface? My goal is to have some sort of mechanism to select the Operating System that boots when the Pi is first powered up without having to connect to an HDMI display.

    Thanks so much for your time and your blog.

    Best Regards,
    Chris

    Comment by Chris Thompson — January 13, 2017 @ 20:04

  4. I'm not really familiar with this side of the Raspberry Pi ecosystem (I got my Pis to do OS development, so I never really used it for Linux, etc.).

    I vaguely remember the NOOBS image having some sort of GUI bootloader, but I only noticed it via HDMI - in other words, I don't know if it does more than just HDMI.

    As far as I know, the FreeBSD port to the Pi uses the FreeBSD bootloader which has a nice UI ever over serial. I have no idea how difficult it would be to get just the bootloader instead of installing FreeBSD, but it's one option you can explore.

    Comment by JeffPC — April 30, 2017 @ 13:55

Atom feed for comments on this post.

Leave a comment

Powered by blahgd