Skip to content

tests: run every test against a temp copy - #8

Open
dingf3ng wants to merge 1 commit into
example-v-unprovenfrom
tests-temp-copies
Open

tests: run every test against a temp copy#8
dingf3ng wants to merge 1 commit into
example-v-unprovenfrom
tests-temp-copies

Conversation

@dingf3ng

@dingf3ng dingf3ng commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Why

coqpyt writes every accepted change straight back to the file on disk (coqpyt/coq/base_file.py::_make_change), and nothing puts it back. Any test pointed at a tracked examples/*.v therefore edits the repo.

CoqInterface.load() alone is enough — it pops the trailing Admitted.:

before:      4a0ceb92   tail: 'Proof.\n\nAdmitted.\n'
after load:  c0a19927   tail: 'Proof.\n'
after close: c0a19927   <- close() does NOT restore
git status:  M examples/main_loop_invariant_2_established_Coq.v

219dbad moved three tests onto temp_example_copy() and left ~12 behind. This finishes it.

The reset/restore pattern wasn't safe either

Some tests guarded themselves with reset_coq_file_to_admitted(...) + a finally: restore. A finally: block only runs while the interpreter unwinds:

signal finally: runs?
SIGTERM NO
SIGINT yes
SIGKILL NO

SIGTERM is what plain timeout sends, what docker stop sends, and what CI sends on a step timeout. These tests drive coq-lsp, so being killed is how they routinely end.

Reproducer — nothing exotic:

timeout -s TERM 6 python3 tests/test_coqpyt_simple.py

leaves examples/example.v holding Admitted. in place of its proof, plus an orphan example.v.backup that .gitignore hides from git status.

What changed

  • 13 remaining tests moved to temp_example_copy()
  • temp_example_copy() also copies examples/_CoqProject beside the copy, so tests driving coqpyt's ProofFile directly still resolve their libframac imports
  • two follow-ons in the same files: test_proof_tree_step_by_step wrote PNGs into examples/; test_coqpyt_svcomp_clean printed "file not available" and carried on — now a real pytest.skip

Base automatically changed from example-v-unproven to rocq-9-support September 1, 2026 16:16
219dbad moved three tests onto temp_example_copy() and left "around a
dozen others" pointing at the tracked examples/*.v. This finishes that
migration.

Why it matters: coqpyt writes every accepted change straight back to the
file on disk (coqpyt/coq/base_file.py::_make_change), and nothing puts it
back -- CoqInterface.load() alone pops the trailing "Admitted." and the
file loses its proof terminator permanently. Reproduced against the
tracked file:

    before:      4a0ceb92  tail: 'Proof.\n\nAdmitted.\n'
    after load:  c0a19927  tail: 'Proof.\n'
    after close: c0a19927   <- close() does not restore
    git status:  M examples/main_loop_invariant_2_established_Coq.v

The tests that guarded this with reset/restore were not safe either. A
finally: block only runs while the interpreter unwinds:

    SIGTERM  -> finally block: NOT RUN
    SIGINT   -> finally block: FINALLY RAN
    SIGKILL  -> finally block: NOT RUN

SIGTERM is what plain `timeout` sends, what `docker stop` sends, and what
CI sends on a step timeout -- and these tests drive coq-lsp, so being
killed is how they routinely end. `timeout -s TERM 6 python3
tests/test_coqpyt_simple.py` was enough to leave examples/example.v
rewritten with an orphan .backup beside it.

temp_example_copy() now also copies examples/_CoqProject next to the
copy, so tests driving coqpyt's ProofFile directly still resolve the
libframac imports; a CoqInterface with auto_setup_coqproject regenerates
the same content.

Two follow-ons in the same files: test_proof_tree_step_by_step wrote its
PNG output into examples/, and test_coqpyt_svcomp_clean printed "file not
available" and carried on; both now use the temp directory and a real
pytest.skip respectively.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@dingf3ng
dingf3ng changed the base branch from rocq-9-support to example-v-unproven September 2, 2026 04:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant