tests: run every test against a temp copy - #8
Open
dingf3ng wants to merge 1 commit into
Open
Conversation
219dbad moved three tests onto temp_example_copy() and left "around a dozen others" pointing at the tracked examples/*.v. This finishes that migration. Why it matters: coqpyt writes every accepted change straight back to the file on disk (coqpyt/coq/base_file.py::_make_change), and nothing puts it back -- CoqInterface.load() alone pops the trailing "Admitted." and the file loses its proof terminator permanently. Reproduced against the tracked file: before: 4a0ceb92 tail: 'Proof.\n\nAdmitted.\n' after load: c0a19927 tail: 'Proof.\n' after close: c0a19927 <- close() does not restore git status: M examples/main_loop_invariant_2_established_Coq.v The tests that guarded this with reset/restore were not safe either. A finally: block only runs while the interpreter unwinds: SIGTERM -> finally block: NOT RUN SIGINT -> finally block: FINALLY RAN SIGKILL -> finally block: NOT RUN SIGTERM is what plain `timeout` sends, what `docker stop` sends, and what CI sends on a step timeout -- and these tests drive coq-lsp, so being killed is how they routinely end. `timeout -s TERM 6 python3 tests/test_coqpyt_simple.py` was enough to leave examples/example.v rewritten with an orphan .backup beside it. temp_example_copy() now also copies examples/_CoqProject next to the copy, so tests driving coqpyt's ProofFile directly still resolve the libframac imports; a CoqInterface with auto_setup_coqproject regenerates the same content. Two follow-ons in the same files: test_proof_tree_step_by_step wrote its PNG output into examples/, and test_coqpyt_svcomp_clean printed "file not available" and carried on; both now use the temp directory and a real pytest.skip respectively. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
dingf3ng
force-pushed
the
tests-temp-copies
branch
from
September 2, 2026 04:44
751a3cb to
a6694e1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
coqpyt writes every accepted change straight back to the file on disk (
coqpyt/coq/base_file.py::_make_change), and nothing puts it back. Any test pointed at a trackedexamples/*.vtherefore edits the repo.CoqInterface.load()alone is enough — it pops the trailingAdmitted.:219dbadmoved three tests ontotemp_example_copy()and left ~12 behind. This finishes it.The reset/restore pattern wasn't safe either
Some tests guarded themselves with
reset_coq_file_to_admitted(...)+ afinally:restore. Afinally:block only runs while the interpreter unwinds:finally:runs?SIGTERM is what plain
timeoutsends, whatdocker stopsends, and what CI sends on a step timeout. These tests drive coq-lsp, so being killed is how they routinely end.Reproducer — nothing exotic:
leaves
examples/example.vholdingAdmitted.in place of its proof, plus an orphanexample.v.backupthat.gitignorehides fromgit status.What changed
temp_example_copy()temp_example_copy()also copiesexamples/_CoqProjectbeside the copy, so tests driving coqpyt'sProofFiledirectly still resolve their libframac importstest_proof_tree_step_by_stepwrote PNGs intoexamples/;test_coqpyt_svcomp_cleanprinted "file not available" and carried on — now a realpytest.skip