Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[testing] Reorganize the test structure. #1243

Merged
merged 5 commits into from
Feb 16, 2024

Conversation

schweitzpgi
Copy link
Collaborator

@schweitzpgi schweitzpgi commented Feb 15, 2024

The various directories should be organized as follows:

  • test - these are small regression tests that can be run very quickly and test small combinations of functionality to prevent regressions. does not use gtest framework, but rather FileCheck and friends.

  • unittests - similar to test, but fixating on one particular "unit". tests the unit's boundaries, limits, etc. to make sure that a particular encapsulated blob of computation (the unit) works as advertised. uses gtest framework.

  • targettests - not like the 2 above. these are end-to-end tests that might take a long time to run but test a whole 9 yards of functionality. so if you wanted to run a game of quantum nethack, it would go here. note: eventually these tests should be executable out-of-tree so that installations of cuda quantum can be tested sans source code.

  • python/tests - these should be folded into the above testing infrastructure.

@bmhowe23
Copy link
Collaborator

What is the desired convention for when items would go into test/NVQPP instead of targettests?

@boschmitt
Copy link
Collaborator

It would be great to put your PR description on a README for future reference.

@schweitzpgi
Copy link
Collaborator Author

schweitzpgi commented Feb 16, 2024

What is the desired convention for when items would go into test/NVQPP instead of targettests?

Basically, don't put a test that is expected to execute itself anywhere in test/. The holdovers were not executing themselves, but printing out the compiler's version string, making sure nvq++ was functional as a linker, etc.

I was thinking of renaming the test/NVQPP directory, but backed off on that in this PR. For one thing, it's not clear what would be a "perfect name" exactly.

@schweitzpgi
Copy link
Collaborator Author

FWIW: With this change, the regression tests run quickly and do not freeze or crush a smaller/weaker/malnourished system.

@schweitzpgi
Copy link
Collaborator Author

It would be great to put your PR description on a README for future reference.

Added test/README.txt.

@schweitzpgi schweitzpgi enabled auto-merge (squash) February 16, 2024 18:00
Copy link

CUDA Quantum Docs Bot: A preview of the documentation can be found here.

github-actions bot pushed a commit that referenced this pull request Feb 16, 2024
The various directories should be organized as follows:

test - these are small regression tests that can be run very quickly and test
  small combinations of functionality to prevent regressions. does not use
  gtest framework, but rather FileCheck and friends.

unittests - similar to test, but fixating on one particular "unit". tests the
  unit's boundaries, limits, etc. to make sure that a particular encapsulated
  blob of computation (the unit) works as advertised. uses gtest framework.

targettests - not like the 2 above. these are end-to-end tests that might take
  a long time to run but test a whole 9 yards of functionality. so if you
  wanted to run a game of quantum nethack, it would go here.

  note: eventually these tests should be executable out-of-tree so that
  installations of cuda quantum can be tested sans source code.

python/tests - these should be folded into the above testing
  infrastructure.
Copy link

CUDA Quantum Docs Bot: A preview of the documentation can be found here.

github-actions bot pushed a commit that referenced this pull request Feb 16, 2024
Copy link
Collaborator

@bmhowe23 bmhowe23 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, Eric.

@schweitzpgi schweitzpgi merged commit 6059cbf into NVIDIA:main Feb 16, 2024
133 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Feb 16, 2024
@schweitzpgi schweitzpgi deleted the ch-rearrange.tests branch February 16, 2024 22:39
@bettinaheim bettinaheim added the maintenance Work items to update and improve the code base label Mar 8, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
maintenance Work items to update and improve the code base
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants