Skip to content

feat: migrate to Fess 15.8 - #5

Open
marevol wants to merge 1 commit into
masterfrom
feat/fess-15.8
Open

feat: migrate to Fess 15.8#5
marevol wants to merge 1 commit into
masterfrom
feat/fess-15.8

Conversation

@marevol

@marevol marevol commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator

Moves this deployment to Fess 15.8.0 / OpenSearch 3.8.0, and brings along the pieces that do not follow those pins on their own.

Changes

Version pins.env: FESS_VERSION=15.8.0, OPENSEARCH_VERSION=3.8.0 (Fess 15.8.0 targets OpenSearch 3.8.0, per fess-parent-15.8.0). render-fess-config.sh re-fetches the fess_config.properties base from the fess-15.8.0 tag with no change needed: every key the codesearch overlay redefines still exists in the 15.8 base. The api.v2.*api.* renames in 15.8 do not touch any key the overlay owns.

Plugin downloadbin/setup.sh pins fess-ds-git 15.8.0 and downloads from https://maven.codelibs.org/release. Fess plugins are no longer published to Maven Central from the 15.8 line on, so the old URL 404s for 15.8.0 and setup.sh would abort. FESS_PLUGIN_REPO_URL overrides the repository for pinning an older plugin release (≤ 15.7.0, still on Central).

Index schemafess_indices/_codesearch is a hand-maintained fork, so it does not track the version. Refreshed with what upstream added in 15.8:

  • fess.json: index.knn, index.knn.derived_source.enabled, index.merge.policy
  • fess/doc.json: content_chunk_vector (nested knn_vector), content_chunk_status

The fork is now identical to codelibs/fess@fess-15.8.0 apart from the codesearch tuning it exists for (line_number_filter, operator_filter / dotnum_filter / code_stop_filter, the seven codesearch document fields, important_content unindexed, no content_minhash_bits copy).

Codesearch does not use semantic search — the Content Chunk Vector Indexer job and content_chunker.search.enabled are both off by default in 15.8 — but a mapping is only ever applied to a brand-new index, so carrying it now is what keeps enabling it later from requiring a full re-crawl. It does make the k-NN plugin a hard requirement; ghcr.io/codelibs/fess-opensearch bundles it.

bin/verify.sh fix — the search API probe read record_count at the top level of the response, but /api/v2/search has always wrapped its payload in a {"response": {...}} envelope. The probe therefore never found a count and silently SKIPped — the one check that catches the "HTTP 200 with 0 hits" failure mode this script exists for (#3 / Redmine 6708). Now reads through the envelope, with a fallback to the flat shape.

Docs — README version references, the plugin-repository change, the 15.8 index-schema note, and an explicit warning that an in-place upgrade keeps the old index mapping (Fess applies a mapping only to an index that has none yet).

Verification

Run against an isolated stack (ghcr.io/codelibs/fess:15.8.0 + ghcr.io/codelibs/fess-opensearch:3.8.0, separate compose project / container names / ports) built from exactly this branch's tracked files.

  • bin/setup.sh completes: fess-ds-git 15.8.0 downloaded from maven.codelibs.org, theme packaged from fess-themes main (codesearch 1.0.9), base config fetched from codelibs/fess@fess-15.8.0.
  • First boot creates the index with the codesearch schema. Live settings: index.knn=true, index.knn.derived_source.enabled=false, index.merge.policy.floor_segment=16mb, max_merge_at_once=30. Live mapping: all 7 codesearch fields plus content_chunk_status and content_chunk_vector (dimension=768, engine=lucene, space_type=cosinesimil, method=hnsw). No reindex needed.
  • bin/verify.sh: all checks pass, including the codesearch schema has every core field of codelibs/fess@fess-15.8.0 and the now-live API probe (the theme's facet request returns hits (18 for 'fess')). The one remaining WARN (duplicate dynamic_templates names lang_ca / lang_en / lang_hu) is pre-existing and unrelated.
  • Registered two repositories with bin/register_github.sh and crawled: 20 documents, domain/organization/repository/filetype all populated.
  • Search: /api/v2/search returns real facet payloads for organization, repository, filename, filetype; the theme's facet qualifiers work (repository: → 10, filetype:java → 5, organization: → 18). An unsupported facet field correctly answers HTTP 400 invalid_request.
  • SPA renders on 15.8 (headless): 18 result cards matching record_count, org/repo/path crumbs, filetype badges, the facet rail with counts, and line-number gutters — confirming the codesearch line_number_filter and appendLineNumber handler still line up. Home and /help render clean; /search/advance serves the SPA (/advance is not an allowlisted theme UI prefix, and the theme links to the canonical path).
  • Content Chunk Vector Indexer present and available=false, as expected.
  • Zero ERROR lines in fess.log across boot, crawl and search.

Pin Fess 15.8.0 / OpenSearch 3.8.0, and bring along the pieces that do
not follow those pins on their own.

- .env: FESS_VERSION=15.8.0, OPENSEARCH_VERSION=3.8.0. The
  fess_config.properties base is re-fetched from the fess-15.8.0 tag by
  render-fess-config.sh with no further change; every key the codesearch
  overlay redefines still exists in the 15.8 base.

- bin/setup.sh: fess-ds-git 15.8.0, downloaded from
  maven.codelibs.org/release. Fess plugins are no longer published to
  Maven Central from the 15.8 line on, so the previous URL 404s for
  15.8.0 and setup.sh would abort. FESS_PLUGIN_REPO_URL overrides the
  repository for pinning an older plugin release.

- fess_indices/_codesearch: refresh the hand-maintained fork with what
  upstream added in 15.8 - index.knn (plus knn.derived_source.enabled
  and the merge policy) in fess.json, and the content_chunk_vector /
  content_chunk_status document fields in fess/doc.json. The fork is now
  identical to codelibs/fess@fess-15.8.0 apart from the codesearch
  tuning it exists for. Codesearch does not use semantic search - the
  Content Chunk Vector Indexer job and content_chunker.search.enabled
  are both off by default - but a mapping is only ever applied to a
  brand-new index, so carrying it now is what keeps enabling it later
  from requiring a full re-crawl.

- bin/verify.sh: read record_count out of the {"response": {...}}
  envelope that /api/v2/search actually returns. The probe read it at
  the top level, so it never found a count and silently SKIPped instead
  of running - the one check that catches the "HTTP 200 with 0 hits"
  failure mode this script exists for.

- README, register_github.sh, compose.yaml: version references, the
  plugin repository change, and a note that an in-place upgrade keeps
  the old index mapping.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant