Table of Contents

Raspberry Pi OS

RPi imager

RPi imager

RPi imager has been released which has removed the manual process of downloading and installing a new OS. You can use this method and then skip Download/Install below.

Within the imager select the LITE version of the OS and configure remote ssh access in the advanced options.

If imager is used on Windows and a prompt to FORMAT the drive is presented cancel that operation.

Download

Visit https://www.raspberrypi.com/software/operating-systems/ and locate the latest download image.

Fetch the lite zip image

wget "https://downloads.raspberrypi.org/raspios_lite_armhf/images/raspios_lite_armhf-2021-05-28/2021-05-07-raspios-buster-armhf-lite.zip"

sha256sum 2021-05-07-raspios-buster-armhf-lite.zip

c5dad159a2775c687e9281b1a0e586f7471690ae28f2f2282c90e7d59f64273c

Install

Place a microsd card in your desktop system and write the image.

In this instance, the microsd card is detected as sdb.

unzip 2021-05-07-raspios-buster-armhf-lite.zip

sudo dd if=2021-05-07-raspios-buster-armhf-lite.img of=/dev/sdb bs=4M

Reload partition table

echo 1 > /sys/block/sdb/device/rescan

Initialise boot partition

mount /dev/sdb1 /mnt

Setup SSH and WiFI

touch /mnt/ssh
cp /etc/wpa_supplicant/wpa_supplicant.conf /mnt

Edit config if required, Eg.

sdtv_mode=2
#dtparam=audio=on
gpu_mem=16
dtparam=spi=on
dtoverlay=spi-bcm2835
#dtoverlay=mcp2515-can0,oscillator=16000000,interrupt=25
dtparam=i2c_arm=on
dtparam=i2c_vc=on
#dtoverlay=w1-gpio,gpiopin=4
#dtoverlay=dht11,gpiopin=17

Unmount

umount /mnt

First boot on console

Boot Pi, it will take quite some time.

Login on console, unlock WiFI if needed with rfkill, change root password and configure networking.

Example networking options

vi /etc/resolv.conf
chattr +i /mnt/etc/resolv.conf
echo "ipv6" >> /mnt/etc/modules

Restart to login over LAN

reboot

Run level

If the system booted with systemd, ensure the runlevel is 3

Systemd runlevel targets
┌─────────┬───────────────────┐
│Runlevel │ Target            │
├─────────┼───────────────────┤
│0        │ poweroff.target   │
├─────────┼───────────────────┤
│1        │ rescue.target     │
├─────────┼───────────────────┤
│2, 3, 4  │ multi-user.target │
├─────────┼───────────────────┤
│5        │ graphical.target  │
├─────────┼───────────────────┤
│6        │ reboot.target     │
└─────────┴───────────────────┘
Switch runlevel

Change the active runlevel.

systemctl isolate multi-user.target
Default runlevel

Set the default runlevel for next boot.

systemctl set-default multi-user.target
Check default runlevel
readlink /etc/systemd/system/default.target
/lib/systemd/system/multi-user.target

Second boot over LAN to fix the OS

ssh pi@X.X.X.X
sudo -s

Update

apt-get update
apt-get full-upgrade

Disable undesired services

Some services refuse to disable on debian bullseye and and a number of other issues occur such as systemd not being able to reboot the first time after disabling the following.

All systems
update-rc.d avahi-daemon disable
update-rc.d bluetooth disable
update-rc.d cron disable
update-rc.d dbus disable
update-rc.d dhcpcd disable
update-rc.d dphys-swapfile disable
update-rc.d paxctld disable
update-rc.d raspi-config disable
update-rc.d rsync disable
update-rc.d sudo disable
update-rc.d triggerhappy disable
update-rc.d cups disable
update-rc.d cups-browsed disable
update-rc.d rng-tools-debian disable
Bullseye

Some extra steps are needed to disable the systemd INI startup files

systemctl disable dbus
systemctl disable dbus.socket
systemctl mask dbus.socket
systemctl disable dbus.service
systemctl mask dbus.service
systemctl disable systemd-timesyncd
systemctl disable wpa_supplicant.service
systemctl disable hciuart.service
systemctl disable bluealsa.service
systemctl disable bluetooth.service
systemctl disable cups.service
systemctl disable cups-browsed.service

For the latest versions of bullseye dhcpcd switched to using systemd. This should be disabled otherwise /etc/network/interfaces are ignored.

systemctl disable dhcpcd.service

Remove repositories

In older versions of Raspbian, remove the Microsoft repository.

sed -i 's/^deb/#deb/g' /etc/apt/sources.list.d/vscode.list
chattr +i /etc/apt/sources.list.d/vscode.list
mv /etc/apt/trusted.gpg.d/microsoft.gpg /etc/apt/trusted.gpg.d/no-microsoft.gpg
apt-get update

Set up swap

Because we disabled the useless swap generator, do it by hand.

swapoff /var/swap
dd if=/dev/zero of=/var/swap bs=4M count=512
mkswap /var/swap
echo "/var/swap none swap sw 0 0" >> /etc/fstab
swapon /var/swap

For 256MB pi, use count=128 and for pi zero, count=256.

Install runit

On older systems we can replace systemd with runit/sysv but sysv had now been removed entirely so this is not possible. Various security issues are likely to occur due to systemd in future.

Pre bullseye
apt-get install runit-sysv
apt-get autoremove
/bin/echo -e 'Package: *systemd*\nPin: release *\nPin-Priority: -1' > /etc/apt/preferences.d/systemd
reboot
Bullseye
apt-get install runit runit-run runit-systemd

Third boot

Set up runit as required and you are done.

WiFi problems

Hotplug

At least on one setup the WiFi fails to start during boot. It seems that the interface has no name as yet or is renamed to something else then renamed back to wlan0 but after wpa_supplicant runs which results in network failure.

Error looks like this in syslog:

wpa_supplicant[434]: Successfully initialized wpa_supplicant
wpa_supplicant[434]: Could not read interface wlan0 flags: No such device
wpa_supplicant[434]: nl80211: Driver does not support authentication/association or connect commands
wpa_supplicant[434]: nl80211: deinit ifname=wlan0 disabled_11b_rates=0
wpa_supplicant[434]: Could not read interface wlan0 flags: No such device
wpa_supplicant[434]: rfkill: Cannot get wiphy information
wpa_supplicant[434]: Could not read interface wlan0 flags: No such device
wpa_supplicant[434]: WEXT: Could not set interface 'wlan0' UP
wpa_supplicant[434]: wlan0: Failed to initialize driver interface

To solve this configure /etc/network/interfaces using “allow-hotplug wlan0”, but if this also fails, replace rc.local with the following:

#!/bin/sh
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#

check() {
        /sbin/ifconfig wlan0 1>/dev/null 2>/dev/null
}
check
while test $? -eq 1; do
        sleep 1
        check
done
ifup wlan0

# Print the IP address
_IP=$(hostname -I) || true
if [ "$_IP" ]; then
  printf "My IP address is %s\n" "$_IP"
fi

exit 0
Keep alive

Pi4 WiFi can both disconnect with wpa_supplicant still running or crash.

In the former case we can reconnect but in the latter a reboot is required.

#! /bin/bash

check() {
        LOSS=`ping -c 3 -i 1 192.168.0.1 | awk 'match($0, /[0-9]+%/) { print substr($0, RSTART, RLENGTH) }'`
}
while test 1; do
        sleep 30
        check
        if test "$LOSS" = "100%"; then
                ifdown wlan0 1>/dev/mull 2>/dev/null
                ifup   wlan0 1>/dev/null 2>/dev/null
                sleep 5
                check
                if test "$LOSS" = "100%"; then
                        reboot
                fi
        fi
done

Change the IP address to that of your access point or modem.

Upgrade

Update system first

apt update
apt full-upgrade
apt autoremove
apt autoclean
apt clean

Remove old pins

rm -f /etc/apt/preferences.d/*

Change sources.list

EG.

to bullseye from buster

sed -i 's/buster/bullseye/g' /etc/apt/sources.list

or edit for from bullseye to bookworm

deb http://deb.debian.org/debian bookworm main contrib non-free non-free-firmware
deb http://deb.debian.org/debian-security/ bookworm-security main contrib non-free non-free-firmware
deb http://deb.debian.org/debian bookworm-updates main contrib non-free non-free-firmware

If keys need importing (if import fails try again, and again)

apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 0E98404D386FA1D9
apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 6ED0E7B82643E131
apt-key adv --recv-keys --keyserver keyserver.ubuntu.com F8D2585B8783D481
apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 54404762BBB6E853
apt-key adv --recv-keys --keyserver keyserver.ubuntu.com BDE6D2B9216EC7A8

Update

apt update
apt install gcc-8-base
apt full-upgrade

Change RPi sources list

sed -i 's/buster/bullseye/g' /etc/apt/sources.list.d/raspi.list

or

sed -i 's/bullseye/bookworm/g' /etc/apt/sources.list.d/raspi.list

Update

apt update
apt full-upgrade

Update kernel

If the kernel gets stuck on an old version, Eg 5.10.103-v7l+ (1:1.20230509~buster-1)

Do this

wget https://archive.raspberrypi.org/debian/pool/main/r/raspberrypi-firmware/raspberrypi-bootloader_1.20230405-1_armhf.deb
wget https://archive.raspberrypi.org/debian/pool/main/r/raspberrypi-firmware/raspberrypi-kernel_1.20230405-1_armhf.deb
wget https://archive.raspberrypi.org/debian/pool/main/r/raspberrypi-firmware/raspberrypi-kernel-headers_1.20230405-1_armhf.deb
dpkg -i ./raspberrypi-bootloader_1.20230405-1_armhf.deb
dpkg -i ./raspberrypi-kernel_1.20230405-1_armhf.deb
dpkg -i ./raspberrypi-kernel-headers_1.20230405-1_armhf.deb

After reboot the the updated version is 6.1.21-v8+

MicroSD problems

MicroSD cards can be unreliable and often times installed binaries become corrupt.

Detect

Install debsums to detect package errors.

debsums -s
dpkg-query: unrecoverable fatal error, aborting:
 loading files list file for package 'libsensors5:armhf': cannot read /var/lib/dpkg/info/libsensors5:armhf.list (Input/output error)

In the above example, the system froze due to file system corruption and fsck during reboot was necessary.

Fix

When debsums reports a broken binary, simply reinstall it

apt install --reinstall libsensors5:armhf
Reading package lists... Done
Building dependency tree
Reading state information... Done
0 upgraded, 0 newly installed, 1 reinstalled, 0 to remove and 119 not upgraded.
Need to get 0 B/49.5 kB of archives.
After this operation, 0 B of additional disk space will be used.
(Reading database ... 101000 files and directories currently installed.)
Preparing to unpack .../libsensors5_1%3a3.5.0-3_armhf.deb ...
Unpacking libsensors5:armhf (1:3.5.0-3) over (1:3.5.0-3) ...
Setting up libsensors5:armhf (1:3.5.0-3) ...
Processing triggers for libc-bin (2.28-10+rpt2+rpi1) ...

Continue from the start until debums reports no errors in binary files. If there are lots of problem it may be sensible to reinstall the OS or use a different sdcard completely.

A script

#! /bin/bash
PACKAGES=`debsums -s 2>&1 | awk '/^debsums:\ changed\ file/ { print $6 }' | sort -u`
for PACKAGE in $PACKAGES; do
        sudo apt-get install --reinstall "$PACKAGE"
done
exit 0

Kernel

When building kernel modules on the Pi either the headers need to be installed or in some cases the full kernel build from source to enable missing features.

After installing a replacement kernel, the stock one must be held back to stop it upgrading.

apt-mark hold raspberrypi-bootloader
raspberrypi-bootloader set on hold.
apt-mark hold raspberrypi-kernel
raspberrypi-kernel set on hold.
Headers

To build a kernel module we need the current kernel header files

apt update
apt full-upgrade
apt install linux-headers

This can take a long time to install on a microsd card.

The current kernel config may be inspected

less /usr/src/linux-headers-`uname -r`/.config

If an option is not available that is required, then a kernel must be built from scratch.

Source

Local compile

Install tools for local build

apt install build-essential git bc bison flex libssl-dev make libncurses5-dev device-tree-compiler rsync

Fetch source

cd /usr/src
git clone --depth=1 https://github.com/raspberrypi/linux
cd linux

If you already checked the source, pull changes instead.

cd linux
git pull

Prepare to build

make mrproper

Configure the KERNEL for the target using bash.

Model KERNEL Arch Config
RPi / Zero / Zero W kernel 32-bit bcmrpi_defconfig
RPi2 / RPi3 / Zero 2W kernel7 32-bit bcm2709_defconfig
RPi4 / RPi400 kernel7l 32-bit bcm2711_defconfig
RPi3 / Zero 2 W / RPi4 / RPi400 kernel8 64-bit bcm2711_defconfig

RPi2 / RPi3_

KERNEL=kernel7
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bcm2709_defconfig

Configure the kernel

Here you should enable missing features you require in your custom kernel. Eg. USB CAN bus devices, kernel 1PPS.

make menuconfig

It is recommended to change the local version string when editing the kernel configuration.

General setup --->
 (-v7-KEWL) Local version - append to kernel release

Build the kernel (in this example on a RPi3 with 4 cores).

make -j4 dtbs
make -j4 zImage
make -j4 modules
cp arch/arm/boot/dts/*.dtb /boot
cp arch/arm/boot/dts/overlays/*.dtb* /boot/overlays
cp arch/arm/boot/dts/overlays/README /boot/overlays
cp arch/arm/boot/zImage /boot/$KERNEL.img
make modules_install

Building the kernel will took about 3 hours with /usr/src mounted on an external Maxtor Basics USB2 hard disk.

Reboot after installing and login.

Cross compile

On a debian or debian derived system the RPi kernel may be cross compiled as below. In this example Kali linux on WSL1 was tested.

Install tools

apt install build-essential git bc bison flex libssl-dev make libncurses5-dev device-tree-compiler rsync
apt install crossbuild-essential-armhf crossbuild-essential-arm64

Fetch source

git clone --depth=1 https://github.com/raspberrypi/linux
cd linux

If you already checked the source, pull changes instead.

cd linux
git pull

Prepare to build

make mrproper

Configure the KERNEL for the target using bash.

Model KERNEL Arch Config
RPi / Zero / Zero W kernel 32-bit bcmrpi_defconfig
RPi2 / RPi3 / Zero 2W kernel7 32-bit bcm2709_defconfig
RPi4 / RPi400 kernel7l 32-bit bcm2711_defconfig
RPi3 / Zero 2 W / RPi4 / RPi400 kernel8 64-bit bcm2711_defconfig

RPi2 / RPi3_

KERNEL=kernel7
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bcm2709_defconfig

Configure the kernel

Here you should enable missing features you require in your custom kernel. Eg. USB CAN bus devices, kernel 1PPS.

make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig

It is recommended to change the local version string when editing the kernel configuration.

General setup --->
 (-v7-KEWL) Local version - append to kernel release

Build the kernel (in this example on a Ryzen 5 4500U with 6 cores).

32-bit

make -j6 ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- dtbs
make -j6 ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- zImage
make -j6 ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- modules

64-bit

make -j6 ARCH=arm CROSS_COMPILE=aarch64-linux-gnueabihf- dtbs
make -j6 ARCH=arm CROSS_COMPILE=aarch64-linux-gnueabihf- zImage
make -j6 ARCH=arm CROSS_COMPILE=aarch64-linux-gnueabihf- modules

Compilation takes about half an hour or so.

Prepare files to install in /root/v7-KEWL

mkdir -p /root/v7-KEWL/boot/overlays
cp arch/arm/boot/dts/*.dtb /root/v7-KEWL/boot
cp arch/arm/boot/dts/overlays/*.dtb* /root/v7-KEWL/boot/overlays
cp arch/arm/boot/dts/overlays/README /root/v7-KEWL/boot/overlays
cp arch/arm/boot/zImage /root/v7-KEWL/boot/$KERNEL.img
make ARCH=arm INSTALL_MOD_PATH=/root/v7-KEWL modules_install

Archive kernel and modules and copy to target

Kernel

cd /root/v7-KEWL/boot
tar zcvf /root/boot.tgz *
scp /root/boot.tgz username@host:

Modules

cd /root/v7-KEWL/lib/modules
tar zcvf /root/modules.tgz *
scp /root/modules.tgz username@host:

Install on target

Kernel

cd /boot
tar zxvf ~/boot.tgz

Modules

cd /lib/modules
tar zxvf ~/modules.tgz

Reboot and login

uname

uname -a
Linux PiE 5.15.56-v7-KEWL+ #1 SMP Wed Jul 27 03:44:30 BST 2022 armv7l GNU/Linux
uname -a
Linux Pi1 5.15.83-v7-KEWL+ #1 SMP Fri Dec 16 14:15:43 GMT 2022 armv7l GNU/Linux

If the version of modprobe on the target does not support xz format modules (ELF errors seen in boot) then do the following and reboot.

find /lib/modules/ -name "*.xz" -exec xz -d {} \;
depmod -a

The source tree can be archived and copied to a target if neccessary

cd /usr/src
tar zcvf ~/linux.tgz --exclude=linux/.git linux
scp ~/linux.tgz username@host:

CPU speed

Install tools to manage cpu core speed

apt-get install cpufrequtils

Set speed

cpufreq-set -g userspace
cpufreq-set -f 900000

Miscellaneous

Install shell

apt install tcsh

Install locales

dpkg-reconfigure locales

Set timezone to Etc/UTC

dpkg-reconfigure tzdata

Set editor to VIM

apt install vim
update-alternatives --config editor

Resources

Linux kernel