Apply pytest unit testing - #358
Conversation
|
@Fuad-HH do you know what's standard with other packages for the python tests? I.e., are they typically in |
jacobmerson
left a comment
There was a problem hiding this comment.
I know we briefly discussed test coverage. Is there a convenient way for doing test coverage on python? Going through this PR, I'm seeing a bunch of stuff missing.
Note: that would be a separate PR, and unless it's trivial, not prioritized over other things.
Also, not to be fixed in this PR, but I realized I forgot to fix the python API to the "field"/"function" naming scheme we introduced in #336 , see #359
| happens exactly once per test run. | ||
| """ | ||
| lib = omega_h.OmegaHLibrary() | ||
| yield lib |
There was a problem hiding this comment.
I'm not too familiar with pytest, so pardon the naive question. Is this some sort of python way of creating the library and keeping it in scope for the whole test suite?
There was a problem hiding this comment.
I think using a fixture with scope="session" is one way to share the same library instance across all tests. No sure if it is the most standard way. If I understand correctly, it doesn't matter whether we use yield or return in the current code. However, yield seems like a better practice because it leaves room for cleanup or finalization in the future. Technically, we could perform some finalization after the yield, which may or may not help with the existing library lifecycle issue, although I haven't figured out the solution yet.
| Session-scoped; the world is obtained from the session-scoped library | ||
| and shared by all tests that need mesh-building capabilities. | ||
| """ | ||
| return omega_h_lib.world() |
There was a problem hiding this comment.
Why does this session fixture return, but the lib one yields?
| @@ -0,0 +1,10 @@ | |||
| [pytest] | |||
There was a problem hiding this comment.
I'm not 100% sure, but should this be included in the pyproject.toml
There was a problem hiding this comment.
I think that's correct.
For most projects I have seen where python bindings are used, the python tests are collected in a specific directory rather than being in the |
|
I agree with Fuad about the location. It makes more sense to me to have them in |
Fuad-HH
left a comment
There was a problem hiding this comment.
I have added some small suggestions.
| python_files = "test_*.py" | ||
| python_classes = "Test*" | ||
| python_functions = "test_*" | ||
| addopts = "-v --tb=short --strict-markers -p no:cacheprovider" |
There was a problem hiding this comment.
Should we hardcode these options here? If -v , --tb, and -p no:cacheprovider are needed for CI/CD, they can be easily added there in the pytest run. --strict-markers looks good to me.
In CI/CD, the following can be used:
pytest -v --tb=short -p no:cacheproviderThere was a problem hiding this comment.
I think it depends on what default options we want for the pytest command. Not all of them are specifically meant for CI. I feel --strict-markers looks good in any case. -p no:cacheprovider is specific to CI, so we could remove it here. I don't have a preference on the other options. Do you find it easier to test without verbose and shorter traceback?
There was a problem hiding this comment.
I mean we can leave it upto the user and they are free to use whichever option they want when running pytest.
There was a problem hiding this comment.
Sounds good. Just removed that.
|
@Sichao25 can you also add this to the self-hosted runner? I think it would be good if the python tests run in GPU configuration since Fuad already found some bugs in the python API related to GPUs (and we will need to add support for PyTorch device tensors for our LEADS collaboration). |
|
@jacobmerson Sure. That may also require the update of omega_h dependency with |
|
Rebuilding all the AMPERE80 deps. |
|
@Sichao25 pcms build linking python is working. I suspect you need to add some things to the |
|
@jacobmerson Thanks, the issue is fixed by correcting the PYTHONPATH. But I'm curious why the path was |
|
@Sichao25 do you think it's worth fixing the |
|
We can use return for now. If we eventually need yield, we can add it later. My understanding is that if we eventually need to perform some cleanup for the library, adding it after yield is the recommended approach. |
Close #278 . I put the tests in a separate folder called
pyteststo locate tests easier. Let me know if it is preferred to keep both python and c++ tests in one folder.