Skip to content

Fix AttributeError in pkgrepo when an FMRI is in two publisher stores - #134

Merged
Toasterson merged 1 commit into
OpenIndiana:oifrom
Toasterson:fix-matching-fmris-references
Aug 24, 2026
Merged

Fix AttributeError in pkgrepo when an FMRI is in two publisher stores#134
Toasterson merged 1 commit into
OpenIndiana:oifrom
Toasterson:fix-matching-fmris-references

Conversation

@Toasterson

@Toasterson Toasterson commented Aug 19, 2026

Copy link
Copy Markdown

Problem

pkgrepo remove aborts with an internal error on a repository where the same FMRI is reachable from more than one repository store:

# pkgrepo -s /zdata/oirepo/oi-sparc remove consolidation/userland/userland-incorporation
Traceback (most recent call last):
  File "/usr/bin/pkgrepo", line 247, in subcmd_remove
    matching, refs = repo.get_matching_fmris(pargs, pubs=pubs)
  File ".../pkg/server/repository.py", line 3598, in get_matching_fmris
    merge(mrefs, references)
  File ".../pkg/server/repository.py", line 3578, in merge
    dest[k].extend(v)
AttributeError: 'str' object has no attribute 'extend'

Cause

pkg.catalog.Catalog.get_matching_fmris() documents references as mapping each matching FMRI to the list of patterns that matched it:

{
 fmri1: [pat1, pat2, ...],
 ...
}

The dict comprehension that builds it yields (f, p), so each FMRI gets a bare string instead.

pkg.server.repository.Repository.get_matching_fmris() aggregates every repository store and merges the results with dest[k].extend(v), relying on the documented contract. The keys are PkgFmri objects, and PkgFmri.__eq__ includes the publisher, so the merge only collides when two stores hand back the identical FMRI. That narrow trigger is presumably why the same code is still sitting unfixed in Oracle's tree and in OmniOS. With one publisher, or with -p, there is no collision and the string never gets extended.

A repository reaches that state when one publisher's catalog holds entries belonging to another publisher. pkgrepo rebuild will produce it, because add_package() takes the publisher from the manifest's own pkg.fmri rather than from the directory the manifest sits in. Manifests that ended up under the wrong publisher therefore get catalogued under their real publisher inside the wrong store. From then on pkgrepo remove is unusable on that repository unless -p narrows it to a single store.

Fix

Build references the way the docstring says. That also repairs a second defect which failed silently. When several patterns matched one FMRI, the comprehension kept only the last one.

pull.py is the only consumer that read these values as strings, and it now flattens them. pkgrepo(1) discards the references returned by remove, and pkgsurf and sign never look at them.

Tests

t_catalog.py now checks that references maps an FMRI to every pattern that matched it rather than to a bare string. t_pkgrepo.py has a new test, test_09a_remove_duplicate_fmri, which reproduces the crash by adding a second repository store whose catalog still describes its packages as belonging to the first publisher, then checks that remove works.

I could not run the suite. It wants an illumos build host and I only had macOS available, so neither new test has actually executed. The four touched files compile. I also reproduced the old and new references construction standalone, where the old one raises the AttributeError above and the new one completes. Please run this on a build machine before merging.

🤖 Generated with Claude Code

pkg.catalog.Catalog.get_matching_fmris() documents that it returns
'references' as a dict mapping each matching FMRI to the list of
patterns that matched it:

    {
     fmri1: [pat1, pat2, ...],
     ...
    }

It actually mapped each FMRI to a single bare pattern string, because
the dict comprehension building it used (f, p) rather than accumulating
per FMRI.

pkg.server.repository.Repository.get_matching_fmris() aggregates the
results of every repository store and merges them with dest[k].extend(v),
relying on the documented contract.  As soon as the same FMRI is returned
by two stores, that merge fails:

  # pkgrepo -s /path/to/repo remove consolidation/userland/userland-incorporation
  Traceback (most recent call last):
    ...
    File ".../pkg/server/repository.py", line 3598, in get_matching_fmris
      merge(mrefs, references)
    File ".../pkg/server/repository.py", line 3578, in merge
      dest[k].extend(v)
  AttributeError: 'str' object has no attribute 'extend'

This is reachable on any repository where a publisher's catalog holds
entries belonging to another publisher -- for instance after a rebuild
has indexed manifests that ended up below the wrong publisher directory,
since rebuild takes the publisher from the manifest's own pkg.fmri and
not from the directory name.  Once in that state, pkgrepo remove cannot
be used on the repository at all unless -p is given to restrict the
operation to a single store.

Building references as documented also fixes a second, silent defect:
when one FMRI was matched by several patterns, all but the last were
discarded.

The only consumer that read these values as strings is pull.py, which is
updated to flatten them.  pkgrepo(1) discards the references returned by
remove, and pkgsurf/sign never use them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Toasterson
Toasterson merged commit 6795125 into OpenIndiana:oi Aug 24, 2026
@Toasterson
Toasterson deleted the fix-matching-fmris-references branch August 24, 2026 10:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant