When reverse-engineering Linux-based embedded firmwares, it’s often useful to be able to run the firmware userspace (or other binaries like /bin/sh) in an emulated environment locally.

It’s very common to do so using a combination of QEMU user-space emulation, and chroot. For example:

sudo apt install qemu-user
sudo chroot /path/to/embedded/rootfs /bin/sh

On the Ubuntu version I run, this works because the qemu-user package also installs a systemd service that adds binfmt interpreters that invokes the qemu binaries to emulate execution of non-native binaries. The qemu-user binaries are statically compiled, allowing them to be used even within a chroot.

However, using chroot is somewhat sub-optimal because the processes in this environment run as real root on the host. Vendor binaries may end up trashing the state of the host (block devices, network interface states, etc…) as part of how they operate. They may also make other assumptions like init being PID 1.

I’ve found that a much nicer way is to use unshare to create an unprivileged container instead of using chroot:

unshare --fork \
    --user \
    --mount \
    --pid \
    --ipc \
    --uts \
    --time \
    --net \
    --map-root-user \
    --mount-proc \
    --root=/path/to/embedded/rootfs \
    /bin/sh

On Ubuntu, some hardening options may need to be relaxed to allow unprivileged users to create new user namespaces:

sudo sysctl -w kernel.unprivileged_userns_clone=1
sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0

This gives an environment where:

  • The program run is given PID 1
  • The program run appears to be running as root (but in actuality, is limited in privileges as the user running unshare)
  • Host processes aren’t visible
  • Host network interfaces aren’t accessible (if networking is needed, it’s always possible to pipe through a veth interface)
  • Changes to the hostname/time don’t affect the host
  • Calling reboot or poweroff merely shuts down the container, rather than the host.

This is much safer than giving full host root access, and is much closer to emulating a real execution environment for the firmware.