Infrastructure · SAPIENCE
Deploying a fleet in disconnected environments
At UAH, I built the tools we used to get software onto our SAPIENCE aircraft, including in the field without reliable internet.
- Software factory A push rebuilds only the artifacts that changed, for amd64 and arm64, with unit tests in the build.
- Test in Gazebo On request, those images come up in Gazebo with PX4 SITL.
- Stand up and sync Stand up K3s, then copy the image bundle and its dependencies while connected.
- Deploy with Helm One declarative Helm package gives each drone its pod.
- Serve behind the airgap K3s’s registry mirrors send image pulls to Spegel, which serves them across the local network.
- Observe Prometheus scrapes the services, and Grafana charts them.
Getting ready for the field
We needed to deploy software to both simulated aircraft and the Jetson computers onboard the drones. Once we were in the field, we couldn’t rely on an internet connection to fetch images or finish installing the software.
I automated the builds, simulation tests, and deployment workflow. GitLab CI built the images for amd64 and arm64, with unit tests in the build. We could also bring them up in Gazebo with PX4 SITL to test the SAR mission. I also worked on SBOM and SLSA provenance generation for this software factory, which didn’t make the final product.
How deployment worked
A CLI installed K3s on the ground station and drones and joined them over the mesh network. While still connected, the control plane copied the images we needed, including upstream dependencies.
A Helm chart gave each drone its pod. In the field, K3s’s registry mirrors sent each image pull to Spegel, and any node holding the image served it to the others. Prometheus scraped the services, and Grafana charted them.
We deployed three real aircraft this way, and the same approach scaled to 25+ drones in simulation. The configuration that flew had a reduced set of services.
Working with the team
The CI/CD and deployment tools gave our six-person student team a repeatable workflow for testing changes and getting them onto the fleet. This was part of UAH’s work on SAPIENCE. For more on the project itself, see sapienceproject.com.