Randomness in test code, whether introduced intentionally to cover multiple scenarios or unintentionally through non-deterministic library
functions, undermines the principles of effective testing. In most cases, randomness leads to problems, resulting in code that is unreliable and
difficult to debug. Consequently, deterministic and reproducible tests are preferred, primarily for the following reasons:
- When a test fails, the ability to reproduce the conditions that led to the failure is crucial for effective debugging. Randomness can make it
difficult or even impossible to pinpoint the root cause, as subsequent runs may not exhibit the same failure.
- Being able to replay a scenario allows us to easily compare logs between different test runs.
- Determinism gives us confidence that a bug is fixed when it no longer appears in tests. If they behave randomly, a passing test after a fix
might be coincidental due to a specific random input, rather than a genuine resolution of the underlying problem.
- Flaky tests, which pass or fail intermittently without any code changes, are a significant problem for CI pipelines (continuous integration).
They erode confidence in the CI system, lead to unnecessary investigations and reruns, and ultimately slow down the development and release process.
A stable CI pipeline relies on deterministic test outcomes.
This rule raises an issue when new Random() or UUID.randomUUID() are called in test code.
