Skip to content

Fix rmg_env_command dropping PYTHONPATH needed for source-tree RMG-Py - #994

Open
alongd wants to merge 1 commit into
mainfrom
rmg-env-command-pythonpath
Open

Fix rmg_env_command dropping PYTHONPATH needed for source-tree RMG-Py#994
alongd wants to merge 1 commit into
mainfrom
rmg-env-command-pythonpath

Conversation

@alongd

@alongd alongd commented Aug 17, 2026

Copy link
Copy Markdown
Member

Failure mode

rmg_env_command()'s RMG_PYTHON branch (direct-interpreter invocation, used
on conda/mambaforge installs where micromamba's conda shim is broken)
unsets _ARC_ENV_ACTIVATION_VARS before invoking rmg_env's interpreter,
to scrub ARC's env leakage (BABEL_LIBDIR, LD_LIBRARY_PATH, CONDA_*,
etc. — otherwise bound to arc_env's tree and causing ABI-mismatch
crashes). PYTHONPATH is included in that unset list.

That's correct when RMG-Py/Arkane are pip-installed into rmg_env. But on
a host where RMG-Py/Arkane are used from a source checkout reachable
only via PYTHONPATH
, unsetting it and stopping there leaves the child
interpreter unable to import rmgpy/arkane at all. Every call through
this branch dies with:

/home/alon/anaconda3/envs/rmg_env/bin/python: No module named arkane

Real-run evidence:
/home/alon/runs/t3-pes-CH2O2/iteration_1/PDep_SA/network1_1/MSC/stderr.log
contains exactly that line. Consequence: T3's master-equation sensitivity
job can never run on such a host, so no P-dep network can ever qualify for
QM refinement — the entire PDep→QM feature is silently dead, surfaced only
as the bland "No PDep networks qualified for QM refinement".

Why unsetting PYTHONPATH is still right

Leaving ARC's own PYTHONPATH bound in the child is exactly the kind of
env leakage this branch exists to scrub (see the module docstring / the
_ARC_ENV_ACTIVATION_VARS comment) — a stale entry (e.g. an old checkout
on the caller's PYTHONPATH) could shadow rmg_env's own site-packages
or ARC's own modules could bleed into the child. The unset is correct; the
bug is stopping there instead of restoring the target env's own path.

The fix

After the unset, re-export PYTHONPATH from ARC's own RMG_PATH setting
(already resolved in arc/settings/settings.py, alongside RMG_PYTHON),
shell-quoted like the neighbouring lines, and only when RMG_PATH is
truthy (never emit an empty export PYTHONPATH=, which would be worse
than leaving it unset):

unset CONDA_PREFIX ... PYTHONPATH PYTHONHOME
export PATH=/home/alon/anaconda3/envs/rmg_env/bin:"$PATH"
export PYTHONPATH=/home/alon/Code/RMG-Py-t3pes   # <-- restored
/home/alon/anaconda3/envs/rmg_env/bin/python -c "import arkane; print(arkane.__file__)"
# -> arkane OK

Verified manually on the affected host before implementing.

The other two branches

  • MAMBA_EXE branch and the launcher-hunt (bash -l) branch never unset
    PYTHONPATH in the first place — they route through a launcher's run
    (which re-fires the target env's own activation hooks) or a login shell
    that sources the user's profile. Neither has this gap, so neither was
    changed.

Test plan

  • Baseline: python -m pytest arc/job/env_run_test.py — 36 passed (unmodified branch)
  • Post-fix: same command — 38 passed (36 + 2 new regression tests)
  • New tests assert PYTHONPATH is re-exported from RMG_PATH (after
    the unset) in the RMG_PYTHON branch, and that nothing is emitted
    when RMG_PATH is falsy. Settings are patched via patch.dict, so
    the tests don't depend on this machine's real paths.
  • ruff check arc/job/env_run.py arc/job/env_run_test.py — clean

@codecov

codecov Bot commented Aug 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 64.51%. Comparing base (7ae26d3) to head (7d86561).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #994      +/-   ##
==========================================
+ Coverage   64.46%   64.51%   +0.05%     
==========================================
  Files         119      119              
  Lines       39636    39641       +5     
  Branches    10276    10277       +1     
==========================================
+ Hits        25550    25575      +25     
+ Misses      11102    11082      -20     
  Partials     2984     2984              
Flag Coverage Δ
functionaltests 64.51% <ø> (+0.05%) ⬆️
unittests 64.51% <ø> (+0.05%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes rmg_env_command()’s direct-interpreter (RMG_PYTHON) execution path so it can still import rmgpy/arkane when those are provided via a source checkout on PYTHONPATH, while still scrubbing ARC’s leaked activation variables to avoid ABI/library mismatches.

Changes:

  • In the RMG_PYTHON branch, restore PYTHONPATH from RMG_PATH after unsetting ARC activation variables.
  • Add regression tests covering (a) PYTHONPATH re-export when RMG_PATH is truthy and (b) no PYTHONPATH export when RMG_PATH is falsy.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
arc/job/env_run.py Re-exports PYTHONPATH from RMG_PATH in the direct-interpreter branch after scrubbing leaked activation variables.
arc/job/env_run_test.py Adds regression tests asserting PYTHONPATH re-export behavior in the RMG_PYTHON branch.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread arc/job/env_run_test.py

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (1)

arc/job/env_run_test.py:410

  • The new test hard-codes a developer-specific absolute path (/home/alon/...) and duplicates it in both settings_overrides and the assertion. Using a generic placeholder path (as other tests in this file do, e.g. /home/alice/...) and storing it in a variable avoids leaking a real username and makes the test easier to maintain.
        settings_overrides = {'RMG_ENV_NAME': 'rmg_env', 'RMG_PYTHON': self.fake_rmg_python,
                               'RMG_PATH': '/home/alon/Code/RMG-Py-t3pes'}
        with patch.dict('arc.job.env_run.settings', settings_overrides):
            script = rmg_env_command("-c 'pass'")
        self.assertIn(f'export PYTHONPATH={shlex.quote("/home/alon/Code/RMG-Py-t3pes")}', script)

@alongd
alongd force-pushed the rmg-env-command-pythonpath branch from fc55722 to 1078239 Compare August 21, 2026 13:48
The RMG_PYTHON branch unsets _ARC_ENV_ACTIVATION_VARS (including
PYTHONPATH) before invoking rmg_env's interpreter directly, correctly
scrubbing ARC's env leakage. But on hosts where RMG-Py/Arkane are used
from a source checkout reachable only via PYTHONPATH (not pip-installed
into rmg_env), this left the child unable to import rmgpy/arkane at all,
e.g. "No module named arkane".

Re-export PYTHONPATH from ARC's own RMG_PATH setting after the unset, so
a source checkout stays importable while ARC's leaked activation vars
are still scrubbed. Only emit the export when RMG_PATH is truthy, to
avoid exporting an empty PYTHONPATH.

The MAMBA_EXE and launcher-hunt branches never unset PYTHONPATH in the
first place (they route through the launcher's `run`, or a login shell
that sources the user's profile), so they don't have this gap.
@alongd
alongd force-pushed the rmg-env-command-pythonpath branch from 1078239 to 7d86561 Compare August 22, 2026 04:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants