CI: Deduplicate links in the Check Links report

[Esteban82]

The same broken link often gets listed once per file that references
it, so a single dead link can flood the weekly "Link Checker Report"
issue with repeated entries.

Added a post-processing step that keeps only the first occurrence of
each link in the report and drops the per-file subsections that end
up empty after deduplication.

Assisted-by: Claude Sonnet 5 (High effort)

[seisman]

It's easy to write a ~100-line script with Claude to process the output, but I feel it's too complex for us to maintain.

Looking at the latest report in #9139, maybe we should ignore timeout errors and 403 errors? I think we care about 404 errors the most.

[Esteban82]

I could simplify this to a short bash pipeline instead:

- name: Deduplicate links in the report
  if: steps.lychee.outputs.exit_code != 0
  run: |
    {
      awk '/^## /{exit} {print}' /tmp/lychee-out.md
      echo
      echo "## Broken/timed-out links"
      echo
      grep '^\* \[' /tmp/lychee-out.md | sed -E 's/ \(at [^)]*\)//' | sort -u
    } > /tmp/lychee-dedup.md
    mv /tmp/lychee-dedup.md /tmp/lychee-out.md

The trade-off is that the report becomes a flat, deduplicated list instead of being grouped per source file, so it wouldn't say which file(s) each broken link came from anymore.

Would that be better for you?

[seisman]

The trade-off is that the report becomes a flat, deduplicated list instead of being grouped per source file, so it wouldn't say which file(s) each broken link came from anymore.

The Python script has the same drawback because it reports only the first occurrence of a broken link.

I usually use grep/ack to find the files with broken links, so I feel it's not a big issue. In other words, I feel the bash script version is better than the Python version.