Repository navigation
more fine grained categorisation for tests #4370
Description
as a tester, one would want a more fine grained categorisation than "smoke", "component", "integration" to be able to quickly test a PR against features it touches on.
ISSUE TYPE
- Improvement Request
- Enhancement Request
- Feature Idea
- Other : testing aid
COMPONENT NAME
Marvin
CLOUDSTACK VERSION
CONFIGURATION
OS / ENVIRONMENT
SUMMARY
when a PR is tested it is usually tested against a set of smoke tests in travis or a private cloud with s.moke tests and than maybe avery feature specific (often manual) test. A better categorisation would help in testing more related code and would allow for bypassing any tests that have no baring on the subject of the change.
A blueorangutan smoke test run takes anywhere between 8 and 15 hours and fails only once after the run, albeit with multiple reports. It would be helpful to run a smaller set of basic tests. if this set passes than a more high level batch could be choosen for a next step.
STEPS TO REPRODUCE
go-test smoke
go-test accounts-and-roles
go-test instance-groups
go-test ...
go-test could be any test infra structure like blueorangutan. It is used to illustrate the categorisation, not the test-infrastructure.
EXPECTED RESULTS
ACTUAL RESULTS
Activity
nvazquez commentedon Aug 2, 2021
@DaanHoogland is this still on hold?
DaanHoogland commentedon Aug 2, 2021
yes, becoming more actual, but might not be ready for 4.16
yadvr commentedon Aug 9, 2021
Moving it to unplanned based on @DaanHoogland 's comment.
yadvr commentedon Aug 17, 2022
Closing this - this has been open for 2+yrs; any categorisation effort won't speed up the CI/CD, will create new PRs/issues to track work towards improving CI/CD timings; one idea I want to explore is if it's possible to even run tests in parallel (or certain groups that can run in parallel).
DaanHoogland commentedon Aug 17, 2022
this is valid as long as we use integration tests and really not yet implemented to satisfaction
1 remaining item
github-actions commentedon May 8, 2026
This issue is stale because it has been open for 120 days with no activity. It may be removed by administrators of this project at any time. Remove the stale label or comment to request for removal of it to prevent this.