for
Click to the left of each report name for more information about it.
Test results summary
This report shows all the test runs performed recently curl's master branch.
More recent runs are on the left side of the page, while older ones are on the
right (you may need to scroll to see them all). All test runs made at a
specific commit are found in the same column. Successful test runs are
highlighted in green and show the number of passed tests, while unsuccessful
ones are red and show the number of failed tests. Test runs that were aborted
are coloured yellow and show the number of passed tests. Clicking on the number
will take you to the test log that will show you details of the run. Different
shades of colour are used for runs older than a few days. Obsolete test run
configurations that are no longer being used have their names with a grey
background. Test configurations that have experienced flaky tests in the time
frame shown say flaky while those that are currently always failing say
permafail. Hovering over any coloured square will show you more details
about that test or run.
Counts of recent
test failures
This report shows a list of tests that have recently failed. It is ordered with
the tests experiencing the most failures at the top and those with the fewest
failures at the bottom. The kind of failure is shown in the Result
column. FAIL is a normal failure, FAILIGNORE is a failure that
was suppressed (usually for tests that are known not to work in a
configuration), and UNKNOWN is for tests whose result could not be
determined, such as when a test run aborted or timed out before its status could
be determined.
Test run
statistics
This report shows many statistics about recent test runs.
Test job feature
matrix
This report shows which characteristics or features that each job tests. It is
presented as a matrix of job names on the Y axis and feature names on the X
axis. It is not possible to detect some features in some jobs, so these show as
question marks. This sometimes happens even if the feature data should
be available, but it's not in the test running step of the CI job, and
Test Clutch looks at each test step independently.
Test run
metadata summary
This report shows all the metadata fields set in recent tests. Not all fields
are set for every test run; some are only available in certain configurations.
Clicking on the field name will show you all the values set in the tests, except
for certain fields where that would show too many values that are not very
interesting (such as the time each test started). More information is available
about the meaning of some
metadata fields.
It is designed especially for projects with many tests running on (potentially) many different CI systems. The curl project is a good example. It has two independent test suites totalling thousands of test cases that are run on dozens of different build configurations. Every single code change can cause over 100,000 individual tests to be run on over 100 different servers. With that many tests being run, even a tiny false positive rate makes it hard to tell if a failed test is a genuine problem or just a flaky test. This site is a Test Clutch installation tailored for curl.
Test Clutch is intended to make testing code just a little bit easier. See the README file for a sense of where we're going with this. It's early days, so come back often to see how things are progressing! It's all Open Source, so feel free to jump in and help out if you share the vision of improving presentation of test results.
cURL home page
Test Clutch source code
Test Clutch for curl deployment code
This site is built on code available under the
AGPL3
and other licenses.