fix: Remove non-shipped dependencies from the SBOMs

[dervoeti]

Description

Our SBOMs report dependencies that are never shipped, like test frameworks, build tooling and integrations that are not compiled in. They show up as false positives in the vulnerability scans.

Vector: builds with an explicit, reduced feature set, but cargo cyclonedx resolved the default features and reported every integration (AWS, Azure, etc.).

Rust base images: now pass --no-build-deps, which drops the build-time dependencies.

Kafka: extended the list of skipped projects

OpenSearch (3.1.0 and 3.4.0): additionally skips the test framework, the test fixtures, the QA projects and the benchmarks. OpenSearch 3.6.0 was already fine since there an allow-list instead of a deny-list is used.
The Prometheus exporter 3.6.0.0 gets includeConfigs = ["runtimeClasspath"] on its cyclonedxDirectBom task, the cyclonedxBom extension only configures the aggregation there. 3.1.0.0 and 3.4.0.0 use plugin version 2.4.1, where the extension still applies to the project itself, so they are already correct.

OpenSearch Dashboards and the HBase web UI pass --required-only to cdxgen, which leaves out devDependencies and optional dependencies.

Airflow and Superset got two fixes:

• cyclonedx-bom was installed into the product environment, so it was part of its own SBOM, and uninstalling it afterwards left its dependencies behind in the image. It is now installed into a throw-away venv that is removed after the SBOM is written.
• The Airflow and Superset packages are installed from a locally built wheel, so cyclonedx-py records the file URL of that wheel instead of a purl. Without a purl they appear twice in the image SBOM (no deduplication possible). The purl is added in post-processing with jq.

Definition of Done Checklist

Note

Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant.

Please make sure all these things are done and tick the boxes

• Changes are OpenShift compatible
• All added packages (via microdnf or otherwise) have a comment on why they are added
• Things not downloaded from Red Hat repositories should be mirrored in the Stackable repository and downloaded from there
• All packages should have (if available) signatures/hashes verified
• Add an entry to the CHANGELOG.md file
• Integration tests ran successfully

TIP: Running integration tests with a new product image

The image can be built and uploaded to the kind cluster with the following commands:

boil build <IMAGE> --image-version <RELEASE_VERSION> --strip-architecture --load
kind load docker-image <MANIFEST_URI> --name=<name-of-your-test-cluster>

See the output of boil to retrieve the image manifest URI for <MANIFEST_URI>.

[StefanFl]

lgtm