Gitlab coverage badge always unknow

Question

The problem

I'm working on a Laravel package project and setting up a pipeline to test it (against php 7.3, 7.4 and 8.0) on merge request and publishing it to my package registry on main branch.

I'm trying to set up a badge for code coverage, but i can't make it work correctly since it is always dipslayng unknown percentage.

What i've done

The pipeline

stages:
  - dependancies
  - linting
  - test
  - publish

# Dependancies job
download dependancies:
  stage: dependancies
  # job stuff

# Linting jobs
phpcs:
  stage: linting
  # job stuff

# test jobs templates
.testing_template: &testing
  stage: test
  script:
    - php -dxdebug.mode=coverage vendor/bin/phpunit --coverage-text --colors=never
  only:
    - merge_requests

# Testing jobs
php 8:
  image: mileschou/xdebug:8.0
  <<: *testing

php 7.4:
  image: mileschou/xdebug:7.4
  <<: *testing

php 7.3:
  image: mileschou/xdebug:7.3
  <<: *testing


#  Publish to package registry
publish:
  only:
    - master
    - tags
  # job stuff

So my pipeline works as follow:

• on the feature branch
  • download dependancies
  • lint code
  • test against php 7.3, 7.4 and 8.0
• on the main branch and on tags
  • publish the package to the packages registry

So the part executed on the merge request is as follow

Moreover, i can see code coverage for each test job in the pipeline job section

(I'm not sure why the tests results in different code coverage percentages since the codebase is the same, but is obviously related to the php version and probably specific optimizations done in code parsing.)

The badge

Since this is a Laravel package project i'm testing it with PhpUnit, so i used the given regex for the test coverage parsing setting (taken from the Gitlab docs).

^\s*Lines:\s*\d+.\d+\%

I want a badge that show the code coverage status on the master branch, not on the feature branch, so i've created a coverage badge with the badge image url as follow (i added a reference to a specific test job, i.e. php 7.4)

https://gitlab.com/%{project_path}/badges/%{default_branch}/coverage.svg?job=php%207.4

but it keeps being displayed as

.

I've tought that Gitlab can't find the desired php 7.4 job since it is not executed on the master branch but on a feature branch, so i tried replacing the %{default_branch} placeholder with the %{commit_sha} to reference a specific commit, but this doesn't work neither since (i suppose) it reference the merge commit (that triggered just the publish job) and not the one that triggered the entire pipeline.

I'm stuck

I cannot figure out how to get the code coverage percentage for the master branch.

I don't want to execute testing again on the master branche since i've already tested the feature branch before merging it.

How can i do? Am I missing something?

Answer 1

I was struggling with this too for a python project and found that the test coverage results is now deprecated (https://docs.gitlab.com/ee/ci/pipelines/settings.html#add-test-coverage-results-to-a-merge-request-deprecated)

The answer for me was to use the coverage keyword in my .gitlab-ci.yml with the required regex to extract the percentage from the job log

e.g.

test:
  stage: test
  extends: .python-reports
  script:
    - pipenv run pytest --cov --cov-report term-missing --cov-report xml:./coverage_out_report.xml --junitxml=./test_out_report.xml
  coverage: '/TOTAL.*\s+(\d+%)$/'

And my badge defined as

Link https://gitlab.com/%{project_path}/-/commits/%{default_branch}

Badge image URL https://gitlab.com/%{project_path}/badges/%{default_branch}/coverage.svg

Now I see the coverage as 100% on main (or master) and a nice green badge

Answer 2

I had the same issue in a different stack (Java, Gradle, and Jacoco). I've already solved the issue, and I will share what I've done. I hope my solution gives you some idea.

First, In your project, go to Settings > CI/CD and expand the General pipelines section. Fill the Test coverage parsing textbox with an appropriate regular expression based on the tool you are using. You can find the sample list of these regular expressions on this page: Test coverage parsing samples

Gitlab expects that the test runner tool will print the test coverage percent based on the given regular expression somewhere on the CI job's log. If the test runner engine doesn't print it to the console, you can do it manually. For instance:

test:
  stage: test
  script:
    - gradle test jacocoTestReport --info
    # Print test coverage to console
    - cat build/reports/jacoco/test/html/index.html | grep -o 'Total[^%]*%'
  artifacts:
    paths:
      - build/reports/jacoco/test/*

After making sure that the test coverage exists in the log, the badge will work perfectly.

pipeline and coverage badge sample

Answer 3

I was facing this issue too, where the code coverage was showing up on MRs but displaying unknown on the badge. For my case, I needed to ensure that there is the code coverage job is running for that branch's pipeline

e.g. in gitlab.ci, make sure the run unit tests + code coverage job is triggered when merging to dev branch so that the badge for dev branch will display the %.