Bits of networks

Bits of networks

Networking, system, research

30 Sep 24

BananaPi BPI-F3: bootstraping upstream Debian

This post is part of the Upstream RISC-V serie:

Debian bootstrap method

Since July 2023, riscv64 is an official Debian architecture. It means that generating a RISC-V Debian rootfs is really straightforward.

However, without kernel support, it is not that straightforward to actually install Debian on RISC-V hardware.

Here, we will run the Bianbu Linux system from a SD card and use it to bootstrap Debian on the internal eMMC storage, using debootstrap. For the kernel, we will simply re-use the Bianbu kernel. This is clearly not ideal but it is the only way currently (all distributions with early support for this hardware are doing the same).

Prerequisite

To follow this tutorial, you will need:

  • a BananaPi BPI-F3, of course :) I tried two versions of the hardware: one with 4 GB RAM + 16 GB eMMC, and one with 16 GB RAM + 128 GB eMMC
  • a UART serial-to-USB converter, necessary to debug things and connect through the serial console
  • a good-quality USB-C power supply. Don't underestimate this requirement: I spent too much time debugging weird issues on various boards, and 90% of the time it was caused by the power supply. I am now using a 45 W laptop power supply that is capable of delivering 12 V and everything works fine.
  • a good-quality micro-SD card to be able to boot Bianbu Linux. Again, I spent hours failing to boot any image on this board, even though I was trying two different SD cards. It turns out that both were old and not booting correctly for some reason. A third recent SD card, a SanDisk Ultra Plus, worked fine.
  • optionally, a NVMe drive. Typically you can setup /home on the NVMe (this is quite standard Linux sysadmin and not explained here)

Debian installation

Step 1: boot Bianbu Linux

Get the most recent version of Bianbu from http://archive.spacemit.com/image/k1/version/bianbu/. In particular, version 2 images have a newer kernel. Use the minimal image for SD card, for instance bianbu-24.04-minimal-k1-v2.0rc1-release-20240909131808.img.zip

On your laptop, unzip the image and flash it to a SD card, for instance using a standard dd copy.

After inserting the SD card on the BPI-F3, it should boot without problem and you will see the login prompt on the serial console.

Step 2: prepare the internal eMMC with partitions and bootloader

Login with user root and password bianbu on the serial console. Then either continue through the serial console, or setup networking and SSH to login through SSH.

Let's partition the internal eMMC storage mmcblk2:

export LANG=C.UTF-8

cat <<EOF | sfdisk /dev/mmcblk2
label: gpt
first-lba: 256
1 : start=256, size=512, name=unused
2 : start=768, size=128, name=env
3 : start=2048, size=2048, name=opensbi
4 : start=4096, size=4096, name=uboot
5 : start=8192, size=400M, name=bootfs
6 : name=rootfs
EOF

For simplicity, this partition scheme is very similar to the vendor scheme, with a larger boot filesystem. The first partition is not actually useful on the eMMC, but we need it, otherwise u-boot gets really confused when it tries to compute which partition is the root partition to pass to the kernel.

Next, you need to obtain the magic bootinfo specific for the eMMC. Without it, booting will fail. It can be found in the other zip file (e.g. bianbu-24.04-minimal-k1-v2.0rc1-release-20240909131808.zip). This bootinfo magic and the FSBL (copied from the SD card) will need to be flashed to a special portion of the eMMC.

You can get it directly from the board:

wget http://archive.spacemit.com/image/k1/version/bianbu/v2.0rc1/bianbu-24.04-minimal-k1-v2.0rc1-release-20240909131808.zip
mkdir bianbu-24.04-minimal-k1-v2.0rc1-release-20240909131808
cd bianbu-24.04-minimal-k1-v2.0rc1-release-20240909131808
unzip ../bianbu-24.04-minimal-k1-v2.0rc1-release-20240909131808.zip
cd ..

Then flash bootinfo and FSBL (thanks stintel for the trick to make this part of the eMMC writable):

echo 0 > /sys/block/mmcblk2boot0/force_ro
dd if=bianbu-24.04-minimal-k1-v2.0rc1-release-20240909131808/factory/bootinfo_emmc.bin of=/dev/mmcblk2boot0 bs=512 count=1
dd if=/dev/mmcblk0p1 of=/dev/mmcblk2boot0 bs=512 seek=1

Then copy all other bootloader stages from the SD card:

# u-boot env
dd if=/dev/mmcblk0p2 of=/dev/mmcblk2p2 bs=64K status=progress
# opensbi
dd if=/dev/mmcblk0p3 of=/dev/mmcblk2p3 bs=64K status=progress
# u-boot
dd if=/dev/mmcblk0p4 of=/dev/mmcblk2p4 bs=64K status=progress

Step 3: debootstrap Debian

Prepare the filesystems for the future /boot and / and mount them:

# Create boot and root filesystems
mkfs.ext4 -m 0 /dev/mmcblk2p5
mkfs.ext4 -m 0 /dev/mmcblk2p6
# Mount everything
mkdir -p /mnt/target
mount /dev/mmcblk2p6 /mnt/target/
mkdir -p /mnt/target/boot
mount /dev/mmcblk2p5 /mnt/target/boot

Then run debootstrap. Here we include some base packages, feel free to modify the list.

# Debootstrap with some base packages
PACKAGES=openssh-server,systemd-timesyncd,bash-completion,vim,htop,wget,xz-utils,zstd,file,locales,dbus,openssl,ca-certificates
debootstrap --arch=riscv64 --include="$PACKAGES" unstable /mnt/target http://deb.debian.org/debian

Step 4: finishing touches before first boot

Copy the kernel, kernel modules, and necessary firmware:

cp -a /boot/* /mnt/target/boot/
cp -a /usr/lib/modules /mnt/target/usr/lib/
mkdir -p /mnt/target/usr/lib/firmware
cp -a /usr/lib/firmware/esos.elf /mnt/target/usr/lib/firmware/

Make sure you will be able to login to the system:

# Setup a root password for the console
echo 'root:CHANGEME' | chpasswd -R /mnt/target
# Setup a SSH key
mkdir /mnt/target/.ssh
echo "YOUR_SSH_KEY" > /mnt/target/.ssh/authorized_keys

Setup target fstab, using blkid to get the UUID of the new boot and root partitions, and then editing fstab manually:

blkid
cp /etc/fstab /mnt/target/etc/fstab
nano /mnt/target/etc/fstab

Your fstab file should look like this:

# <file system>     <dir>    <type>  <options>                          <dump> <pass>
UUID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX   /        ext4    defaults,noatime,errors=remount-ro 0      1
UUID=YYYYYYYY-YYYY-YYYY-YYYY-YYYYYYYYYYYY   /boot    ext4    defaults                           0      2

Finally, setup basic DHCP network, adapt to your need:

cat <<EOF >> /mnt/target/etc/network/interfaces   
 
auto end0
iface end0 inet dhcp
EOF

You are now done with the setup, you can umount and shutdown the board:

umount /mnt/target/boot/
umount /mnt/target
sync
poweroff

Step 5: first boot on Debian

Boot without the SD card: Debian should come up on the serial console!

Login on the serial console and/or SSH, and finish the setup according to your needs, e.g. set a locale:

localectl set-locale C.UTF-8

Step 6 (optional): fix initramfs generation

If you are playing with kernels, you will quickly discover that an initramfs built from Debian will hang during boot. The initramfs provided by Bianbu, /boot/initrd.img-6.6.36, works fine though.

This is because a firmware needs to be added to the initramfs (thanks aurel32 for the tip and hook script below).

First, backup the vendor initramfs:

cp -a /boot/initrd.img-6.6.36 /boot/backup-initrd.img-6.6.36

Install initramfs-tools if needed:

apt install initramfs-tools

Then add a hook /etc/initramfs-tools/hooks/bianbu-firmware that copies the required firmware into the initramfs:

#!/bin/sh

#
# List the soft prerequisites here.  This is a space separated list of
# names, of scripts that are in the same directory as this one, that
# must be run before this one can be.
#
PREREQ=""

prereqs()
{
        echo "$PREREQ"
}

case $1 in
# get pre-requisites
prereqs)
        prereqs
        exit 0
        ;;
esac

. /usr/share/initramfs-tools/hook-functions

# Ensure the esos.elf firmware is included
copy_file firmware "/usr/lib/firmware/esos.elf"

Don't forget to make the hook executable, and rebuild the initramfs:

chmod +x /etc/initramfs-tools/hooks/bianbu-firmware
update-initramfs -u

Reboot to check if everything works fine. Otherwise, you can also boot back on Bianbu using the SD card to fix your system.