Create dependencies between jobs in GitHub Actions

Question

I'm new to GitHub Actions, playing with various options to work out good approaches to CI/CD pipelines.

Initially I had all my CI steps under one job, doing the following:

• checkout code from repo
• lint
• scan source for vulnerabilities
• build
• test
• create image
• scan image for vulnerabilities
• push to AWS ECR

Some of those steps don't need to be done in sequence though; e.g. we could run linting and source code vulnerability scanning in parallel with the build; saving time (if we assume that those steps are going to pass).

i.e. essentially I'd like my pipeline to do something like this:

job1 = {
 - checkout code from repo #required per job, since each job runs on a different runner
 - lint
}
job2 = {
 - checkout code from repo
 - scan source for vulnerabilities
}
job3 = {
 - checkout code from repo
 - build
 - test
 - create image
 - scan image for vulnerabilities
 - await job1 & job2
 - push to AWS ECR
}

I have a couple of questions:

1. Is it possible to setup some await jobN rule within a job; i.e. to view the status of one job from another?
2. (only relevant if the answer to 1 is Yes): Is there any way to have the failure of one job immediately impact other jobs in the same workflow? i.e. If my linting job detects issues then I can immediately call this a fail, so would want the failure in job1 to immediately stop jobs 2 and 3 from consuming additional time, since they're no longer adding value.

Answer 1

Ideally, some of your jobs should be encapsulated in their own workflows, for example:

• Workflow for testing the source by whatever means.
• Workflow for (building and-) deploying.

and then, have these workflows depend on each other, or be triggered using different triggers.

Unfortunately, at least for the time being, workflow dependency is not an existing feature (reference).

Edit: Dependencies between workflows is now also possible, as discussed in this StackOverflow question.

Although I feel that including all of your mentioned jobs in a single workflow would create a long and hard to maintain file, I believe you can still achieve your goal by using some of the conditionals provided by the GitHub actions syntax.

Possible options:

• jobs.<job_id>.if
• jobs.<job_id>.needs

Using the latter, a sample syntax may look like this:

jobs:
  job1:
  job2:
    needs: job1
  job3:
    needs: [job1, job2]

And here is a workflow ready to be used for testing of the above approach. In this example, job 2 will run only after job 1 completes, and job 3 will not run, since it depends on a job that failed.

name: Experiment
on: [push]

jobs:
  job1:
    name: Job 1
    runs-on: ubuntu-latest

    steps:
    - name: Sleep and Run
      run: |
        echo "Sleeping for 10"
        sleep 10

  job2:
    name: Job 2
    needs: job1
    runs-on: ubuntu-latest

    steps:
    - name: Dependant is Running
      run: |
        echo "Completed job 2, but triggering failure"
        exit 1

  job3:
    name: Job 3
    needs: job2
    runs-on: ubuntu-latest

    steps:
    - name: Will never run
      run: |
        echo "If you can read this, the experiment failed"

Relevant docs:

• Workflow syntax for GitHub Actions
• Context and expression syntax for GitHub Actions

Answer 2

If the depended job run a lot faster than its dependents. You can just let them run in parallel.

For example: Deploy an expressjs server is usually a lot faster than publishing to Google Play Console.