Skip to content

Ephemeral Environments for GitHub Pull Requests

This guide demonstrates how to use the Flux Operator ResourceSet API to automate the deployment of applications changes made in GitHub Pull Requests to ephemeral environments for testing and validation.

Development workflow

  • A developer opens a Pull Request with changes to the app code and Helm chart.
  • The CI builds and pushes the app container image to GitHub Container Registry. The image is tagged with the Git commit SHA.
  • Another developer reviews the changes and labels the Pull Request with the deploy/flux-preview label.
  • Flux Operator running in the preview cluster scans the repository and finds the new PR using the label filter.
  • Flux Operator installs a Helm release using the PR number and the commit SHA inputs to deploy the app and chart changes in the cluster.
  • The app is accessible at a preview URL composed of the PR number and the app name.
  • The developers iterate over changes, with each push to the PR branch triggering a Helm release upgrade in the cluster.
  • The developers are notified of the Helm release status in the Slack channel.
  • Once the PR is approved and merged, the Flux Operator uninstalls the Helm release from the cluster.

GitOps workflow

To enable the development workflow, we'll define a series of Flux Operator custom resources in the preview cluster. Note that the preview cluster must be provisioned with a Flux Instance and the Kubernetes manifests part of the GitOps workflow should be stored in the Git repository used by the Flux Instance.

Preview namespace

First we'll create a dedicated namespace called app-preview where all the app instances generated from GitHub Pull Requests will be deployed. We'll also create a service account for Flux that limits the permissions to the app-preview namespace.

apiVersion: v1
kind: Namespace
metadata:
  name: app-preview
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: flux
  namespace: app-preview
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: flux
  namespace: app-preview
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: admin
subjects:
  - kind: ServiceAccount
    name: flux
    namespace: app-preview

GitHub authentication

In the app-preview namespace, we'll create a Kubernetes Secret containing a GitHub PAT that grants read access to the app repository and PRs.

flux -n app-preview create secret git github-auth \
  --url=https://github.com/org/app \
  --username=flux \
  --password=${GITHUB_TOKEN}

Alternatively, we can use a GitHub App token for authentication:

flux create secret githubapp github-auth \
  --app-id="1" \
  --app-installation-id="2" \
  --app-private-key=./private-key-file.pem

Note that GitHub App support was added in Flux v2.5 and Flux Operator v0.15.

ResourceSet input provider

In the app-preview namespace, we'll create a ResourceSetInputProvider that tells Flux Operator to scan the repository for PRs labeled with deploy/flux-preview:

apiVersion: fluxcd.controlplane.io/v1
kind: ResourceSetInputProvider
metadata:
  name: app-pull-requests
  namespace: app-preview
  annotations:
    fluxcd.controlplane.io/reconcileEvery: "10m"
spec:
  type: GitHubPullRequest
  url: https://github.com/org/app
  secretRef:
    name: github-auth
  filter:
    labels:
      - "deploy/flux-preview"
  defaultValues:
    chart: "charts/app"

GitHub Webhook

Optionally, we can create a Flux Webhook Receiver that GitHub will call to notify the Flux Operator when a new PR is opened or updated:

apiVersion: notification.toolkit.fluxcd.io/v1
kind: Receiver
metadata:
  name: github-receiver
  namespace: app-preview
spec:
  type: github
  secretRef:
    name: receiver-token
  resources:
    - apiVersion: fluxcd.controlplane.io/v1
      kind: ResourceSetInputProvider
      name: app-pull-requests

ResourceSet template

Finally, to deploy the app from PRs we'll create a ResourceSet that takes its inputs from the ResourceSetInputProvider:

apiVersion: fluxcd.controlplane.io/v1
kind: ResourceSet
metadata:
  name: app
  namespace: app-preview
spec:
  serviceAccountName: flux
  inputsFrom:
    - apiVersion: fluxcd.controlplane.io/v1
      kind: ResourceSetInputProvider
      name: app-pull-requests
  resources:
    - apiVersion: source.toolkit.fluxcd.io/v1
      kind: GitRepository
      metadata:
        name: app-<< inputs.id >>
        namespace: app-preview
      spec:
        provider: generic # or 'github' if using GitHub App
        interval: 1h
        url: https://github.com/org/app
        ref:
          commit: << inputs.sha >>
        secretRef:
          name: github-auth
    - apiVersion: helm.toolkit.fluxcd.io/v2
      kind: HelmRelease
      metadata:
        name: app-<< inputs.id >>
        namespace: app-preview
        annotations:
          event.toolkit.fluxcd.io/preview-url: "https://app-<< inputs.id >>.example.com"
          event.toolkit.fluxcd.io/branch: << inputs.branch | quote >>
          event.toolkit.fluxcd.io/author: << inputs.author | quote >>
      spec:
        serviceAccountName: flux
        interval: 10m
        releaseName: app-<< inputs.id >>
        chart:
          spec:
            chart: << inputs.chart >>
            reconcileStrategy: Revision
            sourceRef:
              kind: GitRepository
              name: app-<< inputs.id >>
        values:
          image:
            tag: << inputs.sha >>
          ingress:
            hosts:
              - host: app-<< inputs.id >>.example.com

The above ResouceSet will generate a Flux GitRepository and a HelmRelease for each opened PR. The PR number passed as << inputs.id >> is used as the name suffix for the Flux objects, and is also used to compose the Ingress host name where the app can be accessed.

The latest commit SHA pushed to the PR HEAD is passed as << inputs.sha >>, the SHA is used to set the app image tag in the Helm release values.

The preview URL, branch name and author are set as annotations on the HelmRelease object to enrich the Flux notifications that the dev team receives.

To verify the ResourceSet templates are valid, we can use the Flux Operator CLI and build them locally:

flux-operator build resourceset -f app-resourceset.yaml \
  --inputs-from test-inputs.yaml

The test-inputs.yaml file should contain mock PR data e.g.:

   - author: test
     branch: feat/test
     id: "1"
     sha: bf5d6e01cf802734853f6f3417b237e3ad0ba35d
     title: 'testing'

Notifications

To receive notifications when a PR triggers a Helm release install, upgrade and uninstall (including any deploy errors), a Flux Alert can be created in the app-preview namespace:

---
apiVersion: notification.toolkit.fluxcd.io/v1beta3
kind: Provider
metadata:
  name: slack-bot
  namespace: app-preview
spec:
  type: slack
  channel: general
  address: https://slack.com/api/chat.postMessage
  secretRef:
    name: slack-bot-token
---
apiVersion: notification.toolkit.fluxcd.io/v1beta3
kind: Alert
metadata:
  name: slack
  namespace: app-preview
spec:
  providerRef:
    name: slack-bot
  eventSources:
    - kind: GitRepository
      name: '*'
    - kind: HelmRelease
      name: '*'
  eventMetadata:
    cluster: "preview-cluster-1"
    region: "us-east-1"

Further reading

To learn more about ResourceSets and the various configuration options, see the following docs: