Introduction

This repository contains artifact evaluation guide for ACSAC'22 submission SPACELORD: Private and Secure Smart Space Sharing.

The Spacelord artifacts can be broadly separated into four categories:

  1. Hub image (4.3)
    • This is a disk image uploaded to the board. It contains Spacelord bootloader, image downloader, and the signature generator.
  2. User token and OS image (4.2, 4.4, 4.5)
    • User token communicates with Spacelord bootloader to upload user image after verifying the identity of the hub.
    • We provide reference implementation of user token as a Linux userspace program.
  3. Peripheral image (4.6)
    • Reference implementation of Spacelord protocol on ESP32-based camera device.
  4. Image and storage server

All artifacts can be downloaded at this OneDrive link.

Note: "Sledgehammer" is the old codename for this project and may appear in some of the artifacts.

Hub Image

  • RockPro64: hub-images/rockpro64-downloader.img
  • iMX8MQ: hub-images/imx8mq-evk-downloader.img
  • QEMU (for artifact evaluation without hardware): qemu-testing/qemu-downloader.img and qemu-testing/run-qemu.sh.

For hardware based evaluation, the target board, an NVME SSD, and an eMMC card are required. Download the corresponding hub image and upload the image to the eMMC card with a disk flashing tool (e.g., Etcher, dd - example).

To minimize the hardware requirement, we disabled TPM requirement for RockPro64 hub image. TPM requirement do not affect the system performance because TPM is only used to store secret keys at the boot stage.

User Token

User token verifies the current status of Spacelord hub and hands over space configuration and user secrets (e.g., LUKS password) once the hub's identity and the integrity are verified.

User token can be downloaded at token which is a static Linux binary. Please add the execution permission (chmod +x ./token) after downloading it.

User OS Image

Spacelord compatible user OS images establish a connection with the user token after booting the kernel. After the identity and the integrity of the board is confirmed by the user token, the user token sends over the user secrets that are required to decrypt the user layer.

Sample user OS image for each board can be downloaded at the following locations. Decompress the downloaded image and keep the original .lz4 file.

  • RockPro64: os-images/rockPro64-user.img.lz4
  • iMX8MQ: os-images/imx8mq-evk-user.img.lz4
  • QEMU (for artifact evaluation without hardware): qemu-testing/qemu-user.img.lz4

Spacelord peripheral devices

A prototype implementation of a Spacelord peripheral device that targets the ESP-EYE board can be downloaded from the peripheral directory.

  • peripheral/secure-hub-peripheral.elf
  • peripheral/bootloader.bin
  • peripheral/partition-table.bin

Please install the Espressif ESP-IDF and tools following the instructions from these links.

Connect the ESP32 board to the test computer via USB cable. This should produce a serial port on the test machine. Find out the name of the serial port based on your operating system. For example, on Windows, the ESP32 board should appear in Device Manager under `Ports (COM & LPT)' as a COM port. On Linux, it should appear under /dev/ttyUSB*. With the three .bin files from the OneDrive peripheral directory in the same directory as the command line, run

python \frameworks\esp-idf-v5.0\components\esptool_py\esptool\esptool.py -p -b 460800 --before default_reset --after hard_reset --chip esp32 write_flash --flash_mode dio --flash_size 2MB --flash_freq 80m 0x1000 bootloader.bin 0x8000 partition-table.bin 0x10000 secure-hub-peripheral.bin

For example, on my Windows test machine, I used the following line:

python C:\Espressif\frameworks\esp-idf-v5.0\components\esptool_py\esptool\esptool.py -p COM8 -b 460800 --before default_reset --after hard_reset --chip esp32 write_flash --flash_mode dio --flash_size 2MB --flash_freq 80m 0x1000 bootloader.bin 0x8000 partition-table.bin 0x10000 secure-hub-peripheral.bin

This should produce screen output indicating that a new image is being flashed to the ESP32.

The hub communicates with the board through Wi-Fi, and the prototype uses hardcoded Wi-Fi SSID and password. You will need to set up a Wi-Fi router or other acceess point with SSID "myssid" and WPA2 password "mypassword" (without quotes). The ESP32 board should automatically connect after being powered on. Alternatively, we can provide a binary with alternative SSID/PW upon request.

Next, you need to look up the ESP32 board's IP address. This can be done throught the control menu of your Wi-Fi router (or other access point) or by plugging the ESP32 board into a computer and running a serial port console on the bord's serial port with baud rate 115200. After resetting the device, it should produce a considerable amount of debugging output through the serial port. The IP address should appear toward the end.

Minimal hub implementation

The above camera implementation is intended to communicate with OpenHAB hub software through HTTPS (REST). Both sides (the hub and the ESP32 board) must use TLS client authentication to verify the identity of each other. To reduce the hassle of setting up a patched OpenHAB, we provide a Linux CLI program written in Rust that performs the Spacelord TLS authentication.

  • Use peripheral/hub-x86_64 to perform peripheral tests on a laptop
  • Use peripheral/hub-aarch64 to perform peripheral tests with a Spacelord board

With the ESP32 board powered up, run this hub program as follows:

hub-x86_64 `IP address of ESP32 board'

Next, a command-line menu should appear that allows you to control the sensor (i.e., camera) and actuator (i.e., LED) of the ESP32 board.

Server Setup

Spacelord hub device communicates with an image server and a storage server. The image server can be any HTTP server (e.g., static-web-server, Python SimpleHTTPServer) that can serve static files. The serving location should be where .lz4 compressed user images are.

Spacelord's remote encrypted storage utilizes iSCSI and WireGuard. We provide a qcow virtual disk image for the storage server. Download storage-server/run-storage-server.sh and storage-server.qcow2.lz4 in the same directory and execute run-storage-server.sh to run the storage server.

Currently, run-storage-server.sh assumes that it can forward host port 51820 to the storage server for WireGuard communication. If host port 51820 is reserved for other purposes, both run-storage-server.sh (hostfwd) and config.toml (WG_LISTEN_PORT) should be revised accordingly.

Testing

Spacelord authorized boot testing

Make sure to download artifacts listed in the introduction page based on your hardware requirement.

Preconfiguration

  1. Run the image server
  2. Run the storage server
  3. Edit config.toml based on the location of the image server and the web server. server.IMAGE_SERVER_URL should point to the image server including the protocol and the port number (e.g., http://localhost:8080/). server.wireguard.ISCSI_TARGET_IP should point to the IP where the storage server is running without the port number (e.g., 127.0.0.1). WG_LISTEN_PORT_LOCAL and WG_LISTEN_PORT (as well as run-qemu.sh and run-storage-server.sh) might have to be changed if the host machine uses these port numbers for other purposes. All the other settings are hardcoded in the provided storage server image, so reviewers do not need to modify them (e.g., wireguard subnet IPs).
    • Inside QEMU, use 10.0.2.2 to connect to the host.
  4. Put config.toml at the same directory with token program.
  5. Flash the hub image to the board or prepare the QEMU image.

Spacelord boot testing

  1. Turn on the hub board
    • Connect the board to the ethernet and turn it on. The board will boot into the hub manager and print a message: Please connect to <your_ip>. Remember this IP address.
    • For QEMU-based testing, use run-qemu.sh script. The provided script forward internal ports outside, so use the host's IP number instead of the printed one in the following steps.
      • We used Ubuntu 18.04 server and qemu-system-aarch64 2.11.1 during testing. Matching this version is important because different version of QEMU provides different device tree files, and our authenticated booting verifies the device tree file.
  2. Run user token program
    • Run user token with the following command: ./token <hub_ip> <os_image>.
    • The local image name and the name on the image server should match. Specifically, the hub program looks up ${LOCAL_IMAGE_DIR}/${IMAGE_NAME}, and the hub downloads the image from ${IMAGE_SERVER_URL}/${IMAGE_NAME}.lz4.
    • The token program will connect to the hub manager (through port number 15758), sends the location of the image server, and chooses a nonce for this communication session.
    • If the communication is successful, the hub downloads the image and reboots, and the token program will proceed to the next phase.
  3. Wait for the authenticated boot
    • When the board reboots, the bootloader calculates the hash of the downloaded image and generates a certificate based on the measurement result and the nonce.
    • Concurrently, the token program calculates the expected hash of the user image. When the hashing is done, the token program waits in a loop and prints "Waiting for the authenticator..." message until the hub is ready.
      • For QEMU testing, you might see repeated messages of "Error: io error: Connection reset by peer (os error 104)" until the user image is fully loaded. This is expected behavior, and it happens because how QEMU forwards connection to the guest OS.
    • When the both parts finish their work, the board boots into the user OS image and runs the agent application that performs authenticated key exchange with the user token. The reference implementation uses TLS client authentication (port number 15759).
    • If the authentication succeeds, the user token sends user credentials that are necessary to mount the user layer.
  4. User layer mount
    • As the last step, the user layer is mounted with the credential provided from the user token.
    • When everything is done, the Linux-based user OS image will launch.

Peripheral testing

  1. Setup ESP-EYE device based on the instruction at peripheral setup guide.
  2. Run the sample hub binary
    • ./hub-x86_64 <peripheral_ip> or ./hub-aarch64 <peripheral_ip>
    • The hub binary prints CLI menu that can interact with the peripheral. Internally, both side use TLS client authentication to ensure their identity.

Performance testing

Download eval-scripts/phoronix_install_script.sh and eval-scripts/phoronix_run_tests.sh and upload them to the board after the board boots up. Run them in the provisioned board to reproduce the performance evaluation in section 6.1.