问题
I have a Jenkins job that runs Selenium tests on Browserstack via the Nightwatch.js framework. We have an entire suite of tests that nightwatch runs in separate processes, and we need a way to get the pass/fail value back to Jenkins after all tests have run.
I have been trying to use the Nightwatch hook teardown to run a piece of code at the end of each module with if(this.results.failed) { take action }, but I can't figure out what action I need to take in order to make the failure accessible to the Jenkins job.
My initial thought was to create an environment variable, and set it to false any time a test fails, but I think each process runs in a separate subshell, so the variables created/modified by the test module are not accessible.
My second idea was to create a counter in the globals module, but that value does not increment as expected when referenced by a global after method.
回答1:
Here is how I would recommend you do it:
- Create a synchronised writer in the afterEach function (https://github.com/nightwatchjs/nightwatch-docs/blob/master/guide/using-nightwatch/test-hooks.md) and write the test results to a flat file(csv, tsv, txt etc) on the system running jenkins. You may choose to write only failures if(this.results.failed) OR write all testIDs with pass/fail status to the file
- Once your testing build has completed, you will have all the details in the flat file (including details on failures).
- You may now trigger a downstream job (a different job on Jenkins using downstream jenkins plugin) which has a simple script(shell script) looking for failures in the test output file you wrote in step 1. Using this, Jenkins will have details if there were test failures
Note: If you have multiple independent builds running tests, please ensure you are using the jenkins build blocker plugin to start the downstream job after all independent builds are completed
回答2:
I was able to solve this issue using the Jenkins Log Parser Plugin to fail the test in the event that the console output contains the string "TEST FAILURE".
This was accomplished by creating a rule file that contains a single line:
error /TEST FAILURE/
来源：https://stackoverflow.com/questions/46596304/failing-a-jenkins-job-when-nightwatch-test-assertion-fails-in-browserstack