Google Git
Sign in
chromium / chromiumos / platform / touchpad-tests / refs/heads/factory-rex-15708.B
commita7fda17019fb0c5ed3ddbd787af288c440ddc21b[log] [tgz]
authorseobrien <seobrien@chromium.org>Fri Oct 20 15:49:26 2023
committerChromeos LUCI <chromeos-scoped@luci-project-accounts.iam.gserviceaccount.com>Mon Oct 23 21:37:56 2023
tree2a1d9b4fd4ef1223fc2a0dafcc898b5fb3955823
parent6a1a3f5aa8dfa163e0506363c99626e05d51b01e [diff]
Make score threshold for regression more lenient

The preupload check uses 10 significant figures to check that the score
hasn't changed. This is excessive and reports many false regressions.
This CL reduces the threshold to 5 significant figures.

BUG=b:268180311
TEST=run preupload check fro gestures repo.  tests no longer fail with a
delta of `-0.0000`

Change-Id: I41f44aa1f572055f4b779e730a206e28edca1def
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/touchpad-tests/+/4958758
Reviewed-by: Henry Barnor <hbarnor@chromium.org>
Commit-Queue: Sean O'Brien <seobrien@chromium.org>
Tested-by: Sean O'Brien <seobrien@chromium.org>
1 file changed
tree: 2a1d9b4fd4ef1223fc2a0dafcc898b5fb3955823
  1. framework/
  2. tests/
  3. .gitignore
  4. DIR_METADATA
  5. Makefile
  6. OWNERS
  7. README.md
  8. touchtests
  9. touchtests.template
  10. unblocked_terms.txt
README.md

Touch tests

Introduction

This repository contains automated tests for Chromium OS's Gestures library. Each test has a log of evdev events which are replayed, a properties file containing gesture properties to set while the Gestures library runs, and a Python function which verifies the output and returns a test score.

Setting up

Assuming that you've followed the developer guide, simply run the following inside your chroot:

(inside)
$ cd ~/trunk/src/platform/touchpad-tests
$ sudo make setup-in-place

Running tests

To run all tests, simply run touchtests. To run one or more specific tests, you can pass a test name or a glob:

(inside)
$ touchtests atlas-1.0/fat-thumb-fail
$ touchtests atlas-1.0/palm-while-typing*

Each test will return a status, with the following meanings:

  • success: the test succeeded, with the given score out of 1.
  • failure: the test failed.
  • error: an error occurred while running the test, so the behavior of the gestures library couldn't be evaluated.
  • incomplete: (baseline tests only) the evdev log for this platform hasn't been created.

Checking for regressions

The --out (or -o) switch creates a report file that future runs can be compared against with the --ref (or -r) switch:

(inside)
$ touchtests --out baseline.json
# (cause some regressions)
$ touchtests --ref baseline.json

The output table will contain a delta column that indicates any regressions or improvements, and an error message will be shown if regressions exist.