The term “Attested Ephemeral CI Runners” describes on-demand build workers that prove their identity and integrity using hardware-backed attestation (TPM or Intel SGX) and disappear after a job finishes; this guide explains a practical GitOps-driven approach to provisioning these runners, managing ephemeral secrets, and producing verifiable build attestations so teams can ship reproducible, secure releases.
Why use attested ephemeral CI runners?
Traditional persistent runners and shared build pools increase attack surface and introduce flakiness from long-lived state. Attested ephemeral CI runners combine two powerful ideas:
- Hardware-backed attestation: TPM/SGX proves a runner booted an expected OS/agent image and has not been tampered with.
- Ephemeral lifecycle: Runners are created on-demand, receive short-lived secrets, run a single CI job, emit signed attestations, and are destroyed—minimizing persistence risk and eliminating long-term drift.
Architectural overview
A secure, auditable Attested Ephemeral CI Runner system typically includes these components:
- GitOps control plane (e.g., ArgoCD/Flux) to declaratively provision runner templates and CRs.
- Provisioner/orchestrator (Kubernetes, K3s, or cloud VM API) to spin up ephemeral nodes.
- Hardware attestation service interacting with TPM/SGX to verify host identity and measured boot values.
- Secrets broker (HashiCorp Vault, cloud KMS with dynamic credentials) issuing ephemeral credentials on successful attestation.
- CI orchestration (self-hosted GitHub/GitLab runners, Tekton, Jenkins agents) that execute the build and produce signed attestations (e.g., using Sigstore/cosign, in-toto attestations, SLSA provenance).
- Artifact signing and storage with verifiable provenance (signed container images, SBOMs, and attestations stored in an immutable store).
Provisioning flow: step-by-step
1. Declare runner templates in Git (GitOps)
Store declarative templates for runner types (OS image, init scripts, attestation policies) in a Git repo. GitOps will reconcile desired runner capacity and configuration, making changes auditable and versioned.
2. Spin up ephemeral host
When a job is scheduled, the orchestrator requests a new VM or pod with a hardware-backed attestation capability. Cloud APIs or Kubernetes operators launch the worker with a minimal, immutable image.
3. Perform hardware attestation
Before granting any secrets or job tokens, the runner performs attestation with a trusted attestation service. For TPM, retrieve measured boot PCRs and quote those to the attestation server; for SGX, validate enclave quotes. The attestation server evaluates measurements against a known-good policy and produces an attestation token.
4. Issue ephemeral secrets
On successful attestation, the secrets broker issues short-lived credentials (CI job token, artifact repo write token, registry push creds). These credentials are scoped narrowly and TTL-limited, and are minted dynamically—never embedded in images or Git.
5. Execute build and produce attestations
The runner performs the build and immediately produces cryptographic provenance: in-toto statements, SLSA provenance lines, and signed artifacts with tools like Sigstore/cosign. The runner attaches the attestation token from step 3 so verifiers can map host attestation to build provenance.
6. Destroy runner and revoke secrets
After job completion (or on failure), orchestrator tears down the host and revokes the ephemeral credentials. Logs and attestations remain in immutable storage for audits; no long-lived credentials remain on the host.
Managing ephemeral secrets securely
- Use a dedicated secrets broker (Vault/Cloud KMS) with a policy that only mints credentials after attestation tokens are verified.
- Limit credential scope to a single job and short TTL (e.g., minutes to a few hours).
- Log issuance and revocation events to an audit store that correlates with GitOps commits and job IDs.
Verifiable build attestations and reproducibility
To make builds reproducible and verifiable:
- Sign artifacts and container images with an ephemeral key whose public verification is stored with the attestation chain (Sigstore/cosign can keyless-sign using OIDC and transparency logs).
- Record build environment and inputs (compiler versions, dependency checksums, SBOM) in a provenance document (SLSA/in-toto) and sign it.
- Publish attestations and SBOMs alongside artifacts to an immutable store or provenance registry; consumers and auditors can re-run verification by checking signatures and attestation-to-host bindings.
GitOps patterns for automation
Use GitOps to standardize runner blueprints, attestation policies, and secrets factory configurations. Typical patterns:
- Store attestation policy YAML and image hashes in Git; reconcile via ArgoCD/Flux to ensure only approved images are used.
- Use pull-request workflows to modify runner templates, ensuring human review and audit trails.
- Emit status back to Git (e.g., annotations or a status repo) after each provisioning event for end-to-end traceability.
Operational considerations and failure modes
- Attestation failures: ensure fallback logging shows PCRs/quotes and compare with policy; avoid silent acceptance of unverified hosts.
- Network interruptions: cache minimal policy and retry attestation; revoke any issued secrets immediately on ambiguous states.
- Performance: ephemeral provisioning adds latency—use warm pools of pre-attested, short-lived hosts if you need low-latency starts, but maintain frequent reprovisioning to reduce drift.
- Cost: ephemeral cloud VMs can increase bill; balance job batching, spot instances, and warm pools to optimize cost vs. security.
Tooling recommendations
- Attestation: TPM tooling (tpm2-tools), Intel SGX SDK, or cloud attestation services (Azure Attestation, AWS Nitro Enclaves attestation).
- Secrets and dynamic credentials: HashiCorp Vault, AWS IAM Roles Anywhere, or short-lived cloud IAM tokens.
- CI orchestration: self-hosted GitHub/GitLab runners, Tekton TaskRuns, or ephemeral Jenkins agents managed via Kubernetes.
- Attestation/provenance: Sigstore (cosign, rekor), in-toto, SLSA, and SBOM generation (cyclonedx/spdx).
Conclusion
Attested Ephemeral CI Runners bring together hardware-backed identity, ephemeral secrets, GitOps automation, and verifiable attestations to deliver CI that is both secure and auditable. By following a declarative GitOps flow, validating hosts via TPM/SGX attestation, minting narrowly scoped credentials, and signing provenance with tools like Sigstore and in-toto, organizations can substantially reduce pipeline attack surface while improving reproducibility and trust in releases.
Ready to prototype attested ephemeral runners? Start by creating a GitOps repo for runner templates and wire a Vault attestation policy—then iterate by adding signed provenance to your builds.
