Fix AttributeError in pkgrepo when an FMRI is in two publisher stores - #134
Merged
Toasterson merged 1 commit intoAug 24, 2026
Merged
Conversation
pkg.catalog.Catalog.get_matching_fmris() documents that it returns
'references' as a dict mapping each matching FMRI to the list of
patterns that matched it:
{
fmri1: [pat1, pat2, ...],
...
}
It actually mapped each FMRI to a single bare pattern string, because
the dict comprehension building it used (f, p) rather than accumulating
per FMRI.
pkg.server.repository.Repository.get_matching_fmris() aggregates the
results of every repository store and merges them with dest[k].extend(v),
relying on the documented contract. As soon as the same FMRI is returned
by two stores, that merge fails:
# pkgrepo -s /path/to/repo remove consolidation/userland/userland-incorporation
Traceback (most recent call last):
...
File ".../pkg/server/repository.py", line 3598, in get_matching_fmris
merge(mrefs, references)
File ".../pkg/server/repository.py", line 3578, in merge
dest[k].extend(v)
AttributeError: 'str' object has no attribute 'extend'
This is reachable on any repository where a publisher's catalog holds
entries belonging to another publisher -- for instance after a rebuild
has indexed manifests that ended up below the wrong publisher directory,
since rebuild takes the publisher from the manifest's own pkg.fmri and
not from the directory name. Once in that state, pkgrepo remove cannot
be used on the repository at all unless -p is given to restrict the
operation to a single store.
Building references as documented also fixes a second, silent defect:
when one FMRI was matched by several patterns, all but the last were
discarded.
The only consumer that read these values as strings is pull.py, which is
updated to flatten them. pkgrepo(1) discards the references returned by
remove, and pkgsurf/sign never use them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
pkgrepo removeaborts with an internal error on a repository where the same FMRI is reachable from more than one repository store:Cause
pkg.catalog.Catalog.get_matching_fmris()documentsreferencesas mapping each matching FMRI to the list of patterns that matched it:The dict comprehension that builds it yields
(f, p), so each FMRI gets a bare string instead.pkg.server.repository.Repository.get_matching_fmris()aggregates every repository store and merges the results withdest[k].extend(v), relying on the documented contract. The keys arePkgFmriobjects, andPkgFmri.__eq__includes the publisher, so the merge only collides when two stores hand back the identical FMRI. That narrow trigger is presumably why the same code is still sitting unfixed in Oracle's tree and in OmniOS. With one publisher, or with-p, there is no collision and the string never gets extended.A repository reaches that state when one publisher's catalog holds entries belonging to another publisher.
pkgrepo rebuildwill produce it, becauseadd_package()takes the publisher from the manifest's ownpkg.fmrirather than from the directory the manifest sits in. Manifests that ended up under the wrong publisher therefore get catalogued under their real publisher inside the wrong store. From then onpkgrepo removeis unusable on that repository unless-pnarrows it to a single store.Fix
Build
referencesthe way the docstring says. That also repairs a second defect which failed silently. When several patterns matched one FMRI, the comprehension kept only the last one.pull.pyis the only consumer that read these values as strings, and it now flattens them.pkgrepo(1)discards thereferencesreturned byremove, andpkgsurfandsignnever look at them.Tests
t_catalog.pynow checks thatreferencesmaps an FMRI to every pattern that matched it rather than to a bare string.t_pkgrepo.pyhas a new test,test_09a_remove_duplicate_fmri, which reproduces the crash by adding a second repository store whose catalog still describes its packages as belonging to the first publisher, then checks thatremoveworks.I could not run the suite. It wants an illumos build host and I only had macOS available, so neither new test has actually executed. The four touched files compile. I also reproduced the old and new
referencesconstruction standalone, where the old one raises theAttributeErrorabove and the new one completes. Please run this on a build machine before merging.🤖 Generated with Claude Code