Skip to content

feat: expose ingest sources through source index - #3869

Draft
djwhitt wants to merge 9 commits into
mainfrom
feat/ingest-source-index
Draft

feat: expose ingest sources through source index#3869
djwhitt wants to merge 9 commits into
mainfrom
feat/ingest-source-index

Conversation

@djwhitt

@djwhitt djwhitt commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Summary

  • extend the existing GET /api/sources index to support ingest-compatible credentials
  • preserve the full management JSON response for private tokens and partner impersonation while returning only token and name for ingest credentials
  • exclude system sources from minimal JSON and CSV discovery while retaining them in private management JSON
  • enforce source-scope ownership filtering, including mixed owned and foreign scopes, and fail closed for invalid credential owners or partner credentials without impersonation
  • add deterministic RFC 4180 CSV through Accept: text/csv content negotiation and return all successful source lists with Cache-Control: no-store
  • keep source mutations and nested management actions private-scoped
  • document the negotiated response in OpenAPI and the public access-token guide

Why reuse the existing endpoint

This alternative keeps source discovery at the established GET /api/sources URL instead of adding another API resource. Private management clients retain the current full JSON representation, while ingest credentials receive a restricted representation selected from the credential type. CSV is always the minimal token,name representation.

The tradeoff is that one route now has an authentication-dependent JSON schema and requires its index authorization to be separated from the remaining private source-management routes. The OpenAPI response documents both representations, and focused tests cover the compatibility and authorization boundaries.

This is an alternative to #3866; only one approach is intended to merge.

Testing

  • ../bin/test test/logflare_web/controllers/api/source_discovery_controller_test.exs --seed 0
  • ../bin/test test/logflare_web/controllers/api/source_controller_test.exs
  • ../bin/test test/logflare_web/controllers/plugs/verify_api_access_test.exs
  • ../bin/test test/logflare_web/api_spec_test.exs test/logflare_web/open_api_test.exs
  • MIX_ENV=test ../bin/format --check-formatted
  • MIX_ENV=test ../bin/x mix lint.all
  • MIX_ENV=test ../bin/x mix compile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant