RPi imager has been released which has removed the manual process of downloading and installing a new OS. You can use this method and then skip Download/Install below.

Within the imager select the LITE version of the OS and configure remote ssh access in the advanced options.

If imager is used on Windows and a prompt to FORMAT the drive is presented cancel that operation.

Visit https://www.raspberrypi.com/software/operating-systems/ and locate the latest download image.

Fetch the lite zip image

wget "https://downloads.raspberrypi.org/raspios_lite_armhf/images/raspios_lite_armhf-2021-05-28/2021-05-07-raspios-buster-armhf-lite.zip"

sha256sum 2021-05-07-raspios-buster-armhf-lite.zip

c5dad159a2775c687e9281b1a0e586f7471690ae28f2f2282c90e7d59f64273c

Place a microsd card in your desktop system and write the image.

In this instance, the microsd card is detected as sdb.

unzip 2021-05-07-raspios-buster-armhf-lite.zip

sudo dd if=2021-05-07-raspios-buster-armhf-lite.img of=/dev/sdb bs=4M

Reload partition table

echo 1 > /sys/block/sdb/device/rescan

Initialise boot partition

mount /dev/sdb1 /mnt

Setup SSH and WiFI

touch /mnt/ssh
cp /etc/wpa_supplicant/wpa_supplicant.conf /mnt

Edit config if required, Eg.

sdtv_mode=2
#dtparam=audio=on
gpu_mem=16
dtparam=spi=on
dtoverlay=spi-bcm2835
#dtoverlay=mcp2515-can0,oscillator=16000000,interrupt=25
dtparam=i2c_arm=on
dtparam=i2c_vc=on
#dtoverlay=w1-gpio,gpiopin=4
#dtoverlay=dht11,gpiopin=17

Unmount

umount /mnt

Boot Pi, it will take quite some time.

Login on console, unlock WiFI if needed with rfkill, change root password and configure networking.

Example networking options

vi /etc/resolv.conf
chattr +i /mnt/etc/resolv.conf
echo "ipv6" >> /mnt/etc/modules

Restart to login over LAN

reboot

If the system booted with systemd, ensure the runlevel is 3

┌─────────┬───────────────────┐
│Runlevel │ Target            │
├─────────┼───────────────────┤
│0        │ poweroff.target   │
├─────────┼───────────────────┤
│1        │ rescue.target     │
├─────────┼───────────────────┤
│2, 3, 4  │ multi-user.target │
├─────────┼───────────────────┤
│5        │ graphical.target  │
├─────────┼───────────────────┤
│6        │ reboot.target     │
└─────────┴───────────────────┘

Change the active runlevel.

systemctl isolate multi-user.target

Set the default runlevel for next boot.

systemctl set-default multi-user.target

readlink /etc/systemd/system/default.target
/lib/systemd/system/multi-user.target

ssh pi@X.X.X.X
sudo -s

apt-get update
apt-get full-upgrade

Some services refuse to disable on debian bullseye and and a number of other issues occur such as systemd not being able to reboot the first time after disabling the following.

update-rc.d avahi-daemon disable
update-rc.d bluetooth disable
update-rc.d cron disable
update-rc.d dbus disable
update-rc.d dhcpcd disable
update-rc.d dphys-swapfile disable
update-rc.d paxctld disable
update-rc.d raspi-config disable
update-rc.d rsync disable
update-rc.d sudo disable
update-rc.d triggerhappy disable
update-rc.d cups disable
update-rc.d cups-browsed disable
update-rc.d rng-tools-debian disable

Some extra steps are needed to disable the systemd INI startup files

systemctl disable dbus
systemctl disable dbus.socket
systemctl mask dbus.socket
systemctl disable dbus.service
systemctl mask dbus.service
systemctl disable systemd-timesyncd
systemctl disable wpa_supplicant.service
systemctl disable hciuart.service
systemctl disable bluealsa.service
systemctl disable bluetooth.service
systemctl disable cups.service
systemctl disable cups-browsed.service

For the latest versions of bullseye dhcpcd switched to using systemd. This should be disabled otherwise /etc/network/interfaces are ignored.

systemctl disable dhcpcd.service

In older versions of Raspbian, remove the Microsoft repository.

sed -i 's/^deb/#deb/g' /etc/apt/sources.list.d/vscode.list
chattr +i /etc/apt/sources.list.d/vscode.list
mv /etc/apt/trusted.gpg.d/microsoft.gpg /etc/apt/trusted.gpg.d/no-microsoft.gpg
apt-get update

Because we disabled the useless swap generator, do it by hand.

swapoff /var/swap
dd if=/dev/zero of=/var/swap bs=4M count=512
mkswap /var/swap
echo "/var/swap none swap sw 0 0" >> /etc/fstab
swapon /var/swap

For 256MB pi, use count=128 and for pi zero, count=256.

On older systems we can replace systemd with runit/sysv but sysv had now been removed entirely so this is not possible. Various security issues are likely to occur due to systemd in future.

apt-get install runit-sysv
apt-get autoremove
/bin/echo -e 'Package: *systemd*\nPin: release *\nPin-Priority: -1' > /etc/apt/preferences.d/systemd
reboot

apt-get install runit runit-run runit-systemd

Set up runit as required and you are done.

At least on one setup the WiFi fails to start during boot. It seems that the interface has no name as yet or is renamed to something else then renamed back to wlan0 but after wpa_supplicant runs which results in network failure.

Error looks like this in syslog:

wpa_supplicant[434]: Successfully initialized wpa_supplicant
wpa_supplicant[434]: Could not read interface wlan0 flags: No such device
wpa_supplicant[434]: nl80211: Driver does not support authentication/association or connect commands
wpa_supplicant[434]: nl80211: deinit ifname=wlan0 disabled_11b_rates=0
wpa_supplicant[434]: Could not read interface wlan0 flags: No such device
wpa_supplicant[434]: rfkill: Cannot get wiphy information
wpa_supplicant[434]: Could not read interface wlan0 flags: No such device
wpa_supplicant[434]: WEXT: Could not set interface 'wlan0' UP
wpa_supplicant[434]: wlan0: Failed to initialize driver interface

To solve this configure /etc/network/interfaces using “allow-hotplug wlan0”, but if this also fails, replace rc.local with the following:

#!/bin/sh
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#

check() {
        /sbin/ifconfig wlan0 1>/dev/null 2>/dev/null
}
check
while test $? -eq 1; do
        sleep 1
        check
done
ifup wlan0

# Print the IP address
_IP=$(hostname -I) || true
if [ "$_IP" ]; then
  printf "My IP address is %s\n" "$_IP"
fi

exit 0

Pi4 WiFi can both disconnect with wpa_supplicant still running or crash.

In the former case we can reconnect but in the latter a reboot is required.

#! /bin/bash

check() {
        LOSS=`ping -c 3 -i 1 192.168.0.1 | awk 'match($0, /[0-9]+%/) { print substr($0, RSTART, RLENGTH) }'`
}
while test 1; do
        sleep 30
        check
        if test "$LOSS" = "100%"; then
                ifdown wlan0 1>/dev/mull 2>/dev/null
                ifup   wlan0 1>/dev/null 2>/dev/null
                sleep 5
                check
                if test "$LOSS" = "100%"; then
                        reboot
                fi
        fi
done

Change the IP address to that of your access point or modem.

Update system first

apt update
apt full-upgrade
apt autoremove
apt autoclean
apt clean

Remove old pins

rm -f /etc/apt/preferences.d/*

Change sources.list

EG.

to bullseye from buster

sed -i 's/buster/bullseye/g' /etc/apt/sources.list

or edit for from bullseye to bookworm

deb http://deb.debian.org/debian bookworm main contrib non-free non-free-firmware
deb http://deb.debian.org/debian-security/ bookworm-security main contrib non-free non-free-firmware
deb http://deb.debian.org/debian bookworm-updates main contrib non-free non-free-firmware

If keys need importing (if import fails try again, and again)

apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 0E98404D386FA1D9
apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 6ED0E7B82643E131
apt-key adv --recv-keys --keyserver keyserver.ubuntu.com F8D2585B8783D481
apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 54404762BBB6E853
apt-key adv --recv-keys --keyserver keyserver.ubuntu.com BDE6D2B9216EC7A8

Update

apt update
apt install gcc-8-base
apt full-upgrade

Change RPi sources list

sed -i 's/buster/bullseye/g' /etc/apt/sources.list.d/raspi.list

or

sed -i 's/bullseye/bookworm/g' /etc/apt/sources.list.d/raspi.list

Update

apt update
apt full-upgrade

Update kernel

If the kernel gets stuck on an old version, Eg 5.10.103-v7l+ (1:1.20230509~buster-1)

Do this

wget https://archive.raspberrypi.org/debian/pool/main/r/raspberrypi-firmware/raspberrypi-bootloader_1.20230405-1_armhf.deb
wget https://archive.raspberrypi.org/debian/pool/main/r/raspberrypi-firmware/raspberrypi-kernel_1.20230405-1_armhf.deb
wget https://archive.raspberrypi.org/debian/pool/main/r/raspberrypi-firmware/raspberrypi-kernel-headers_1.20230405-1_armhf.deb
dpkg -i ./raspberrypi-bootloader_1.20230405-1_armhf.deb
dpkg -i ./raspberrypi-kernel_1.20230405-1_armhf.deb
dpkg -i ./raspberrypi-kernel-headers_1.20230405-1_armhf.deb

After reboot the the updated version is 6.1.21-v8+

MicroSD cards can be unreliable and often times installed binaries become corrupt.

Install debsums to detect package errors.

debsums -s
dpkg-query: unrecoverable fatal error, aborting:
 loading files list file for package 'libsensors5:armhf': cannot read /var/lib/dpkg/info/libsensors5:armhf.list (Input/output error)

In the above example, the system froze due to file system corruption and fsck during reboot was necessary.

When debsums reports a broken binary, simply reinstall it

apt install --reinstall libsensors5:armhf
Reading package lists... Done
Building dependency tree
Reading state information... Done
0 upgraded, 0 newly installed, 1 reinstalled, 0 to remove and 119 not upgraded.
Need to get 0 B/49.5 kB of archives.
After this operation, 0 B of additional disk space will be used.
(Reading database ... 101000 files and directories currently installed.)
Preparing to unpack .../libsensors5_1%3a3.5.0-3_armhf.deb ...
Unpacking libsensors5:armhf (1:3.5.0-3) over (1:3.5.0-3) ...
Setting up libsensors5:armhf (1:3.5.0-3) ...
Processing triggers for libc-bin (2.28-10+rpt2+rpi1) ...

Continue from the start until debums reports no errors in binary files. If there are lots of problem it may be sensible to reinstall the OS or use a different sdcard completely.

A script

#! /bin/bash
PACKAGES=`debsums -s 2>&1 | awk '/^debsums:\ changed\ file/ { print $6 }' | sort -u`
for PACKAGE in $PACKAGES; do
        sudo apt-get install --reinstall "$PACKAGE"
done
exit 0

When building kernel modules on the Pi either the headers need to be installed or in some cases the full kernel build from source to enable missing features.

After installing a replacement kernel, the stock one must be held back to stop it upgrading.

apt-mark hold raspberrypi-bootloader
raspberrypi-bootloader set on hold.
apt-mark hold raspberrypi-kernel
raspberrypi-kernel set on hold.

To build a kernel module we need the current kernel header files

apt update
apt full-upgrade
apt install linux-headers

This can take a long time to install on a microsd card.

The current kernel config may be inspected

less /usr/src/linux-headers-`uname -r`/.config

If an option is not available that is required, then a kernel must be built from scratch.

Local compile

Install tools for local build

apt install build-essential git bc bison flex libssl-dev make libncurses5-dev device-tree-compiler rsync

Fetch source

cd /usr/src
git clone --depth=1 https://github.com/raspberrypi/linux
cd linux

If you already checked the source, pull changes instead.

cd linux
git pull

Prepare to build

make mrproper

Configure the KERNEL for the target using bash.

RPi2 / RPi3_

KERNEL=kernel7
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bcm2709_defconfig

Configure the kernel

Here you should enable missing features you require in your custom kernel. Eg. USB CAN bus devices, kernel 1PPS.

make menuconfig

It is recommended to change the local version string when editing the kernel configuration.

General setup --->
 (-v7-KEWL) Local version - append to kernel release

Build the kernel (in this example on a RPi3 with 4 cores).

make -j4 dtbs
make -j4 zImage
make -j4 modules
cp arch/arm/boot/dts/*.dtb /boot
cp arch/arm/boot/dts/overlays/*.dtb* /boot/overlays
cp arch/arm/boot/dts/overlays/README /boot/overlays
cp arch/arm/boot/zImage /boot/$KERNEL.img
make modules_install

Building the kernel will took about 3 hours with /usr/src mounted on an external Maxtor Basics USB2 hard disk.

Reboot after installing and login.

Cross compile

On a debian or debian derived system the RPi kernel may be cross compiled as below. In this example Kali linux on WSL1 was tested.

Install tools

apt install build-essential git bc bison flex libssl-dev make libncurses5-dev device-tree-compiler rsync
apt install crossbuild-essential-armhf crossbuild-essential-arm64

Fetch source

git clone --depth=1 https://github.com/raspberrypi/linux
cd linux

If you already checked the source, pull changes instead.

cd linux
git pull

Prepare to build

make mrproper

Configure the KERNEL for the target using bash.

RPi2 / RPi3_

KERNEL=kernel7
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bcm2709_defconfig

Configure the kernel

Here you should enable missing features you require in your custom kernel. Eg. USB CAN bus devices, kernel 1PPS.

make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig

It is recommended to change the local version string when editing the kernel configuration.

General setup --->
 (-v7-KEWL) Local version - append to kernel release

Build the kernel (in this example on a Ryzen 5 4500U with 6 cores).

32-bit

make -j6 ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- dtbs
make -j6 ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- zImage
make -j6 ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- modules

64-bit

make -j6 ARCH=arm CROSS_COMPILE=aarch64-linux-gnueabihf- dtbs
make -j6 ARCH=arm CROSS_COMPILE=aarch64-linux-gnueabihf- zImage
make -j6 ARCH=arm CROSS_COMPILE=aarch64-linux-gnueabihf- modules

Compilation takes about half an hour or so.

Prepare files to install in /root/v7-KEWL

mkdir -p /root/v7-KEWL/boot/overlays
cp arch/arm/boot/dts/*.dtb /root/v7-KEWL/boot
cp arch/arm/boot/dts/overlays/*.dtb* /root/v7-KEWL/boot/overlays
cp arch/arm/boot/dts/overlays/README /root/v7-KEWL/boot/overlays
cp arch/arm/boot/zImage /root/v7-KEWL/boot/$KERNEL.img
make ARCH=arm INSTALL_MOD_PATH=/root/v7-KEWL modules_install

Archive kernel and modules and copy to target

Kernel

cd /root/v7-KEWL/boot
tar zcvf /root/boot.tgz *
scp /root/boot.tgz username@host:

Modules

cd /root/v7-KEWL/lib/modules
tar zcvf /root/modules.tgz *
scp /root/modules.tgz username@host:

Install on target

Kernel

cd /boot
tar zxvf ~/boot.tgz

Modules

cd /lib/modules
tar zxvf ~/modules.tgz

Reboot and login

uname

uname -a
Linux PiE 5.15.56-v7-KEWL+ #1 SMP Wed Jul 27 03:44:30 BST 2022 armv7l GNU/Linux

uname -a
Linux Pi1 5.15.83-v7-KEWL+ #1 SMP Fri Dec 16 14:15:43 GMT 2022 armv7l GNU/Linux

If the version of modprobe on the target does not support xz format modules (ELF errors seen in boot) then do the following and reboot.

find /lib/modules/ -name "*.xz" -exec xz -d {} \;
depmod -a

The source tree can be archived and copied to a target if neccessary

cd /usr/src
tar zcvf ~/linux.tgz --exclude=linux/.git linux
scp ~/linux.tgz username@host:

Install tools to manage cpu core speed

apt-get install cpufrequtils

Set speed

cpufreq-set -g userspace
cpufreq-set -f 900000

Install shell

apt install tcsh

Install locales

dpkg-reconfigure locales

Set timezone to Etc/UTC

dpkg-reconfigure tzdata

Set editor to VIM

apt install vim
update-alternatives --config editor

Linux kernel