Improve the test runner: Allow "direct" tests (instead of only steps-based tests) #2711
Conversation
Codecov Report
@@ Coverage Diff @@
## master #2711 +/- ##
==========================================
- Coverage 86.23% 86.23% -0.01%
==========================================
Files 529 529
Lines 36401 36388 -13
==========================================
- Hits 31390 31378 -12
+ Misses 5011 5010 -1
|
psychon
force-pushed
the
improve_test_runner
branch
from
5334eda
to
c79b06f
Compare
|
|
||
| --- This function is called to indicate that a test does not use the run_steps() | ||
| -- facility, but instead runs something else directly. | ||
| function runner.run_direct() |
There was a problem hiding this comment.
s/run_direct/is_running/ maybe?
The current name indicated that it would do something, although with the example in test-gravity it is called only after the test - and it seems like the runner.run_direct() call is not necessary there after all?!
It could also just set runner.is_running instead maybe.
There was a problem hiding this comment.
Well, no. test-gravity.lua starts an external process and afterwards it calls run_direct(). However, at this time, the test is not yet done. Only when the external process finishes some time later is done() actually called. So, it's still asynchronous.
The idea why I call run_direct() at the end is because it "disarms" a timer in _runner.lua that would otherwise complain that the test was not started correctly and quit awesome. So, if starting the external process would fail, awesome would quit relatively quickly instead of waiting for the 180 seconds timeout.
Right now I don't see a nice way to make this more explicit... well, run_direct could get a function as its argument which it just runs, but that might also be weird...
Ideas?
There was a problem hiding this comment.
Thanks for the detailed explanation, as always.
It is fine by me.
psychon
force-pushed
the
improve_test_runner
branch
from
6b5f1c1
to
e5b4eca
Compare
The current _runner.lua expects a table containing steps to be passed in. However, not all tests look like this. This commits adds an API to the runner that allows tests to run however they like. They just have to call run_direct() initially and call done() when they are finished. Signed-off-by: Uli Schlachter <[email protected]>
This commit makes test-gravity.lua use the new infrastructure that was added in the previous commit: Instead of pretending to be a steps-based test, this is now a direct test. This gets rid of all the useless wait_a_bit steps that exist purely to satisfy the steps-based test runner. This commit makes test-gravity.lua about one third shorter. Also, the test might run a tiny bit faster, since there is no more timer that regularly checks if the test is done, but instead it finishes immediately when the external process finishes. Signed-off-by: Uli Schlachter <[email protected]>
psychon
force-pushed
the
improve_test_runner
branch
from
4bccb4d
to
a7f4777
Compare
|
|
||
| --- This function is called to indicate that a test does not use the run_steps() | ||
| -- facility, but instead runs something else directly. | ||
| function runner.run_direct() |
There was a problem hiding this comment.
Thanks for the detailed explanation, as always.
It is fine by me.
No description provided.