prisma_cloud: fix alert data stream dropping data on pagination and bad responses by kcreddy · Pull Request #20917 · elastic/integrations

kcreddy · 2026-08-26T13:49:52Z

Proposed commit message

prisma_cloud: address review feedback on alert error handling and auth

Follow-up to the alert data stream fixes, applying reviewer and bot
feedback.

  - Simplify the response guard: decode_json() returns a catchable error
    value, so drop the redundant try()/bytes() wrapper and check
    is_error(inner_body) directly. Move the status-code condition onto the
    do_request().as(resp, ...) line and fix indentation.

  - Re-authenticate on a token deadline instead of gating login on
    want_more. Log in when there is no access token or the stored
    token_expiry has passed. This removes a stale-token loop: when a token
    expired mid-pagination (want_more=true) the previous logic skipped
    login and reused the dead token indefinitely.

  - Set the token deadline to 9m. The /v2/alert stream uses the Prisma
    Cloud CSPM API (POST /login, x-redlock-auth), whose JWT is valid for
    ~10 minutes.

  - Force re-auth on a non-200 response: expire token_expiry in that
    branch so a token revoked or expired early (or a 401 mid-pagination)
    recovers on the next poll instead of waiting out the deadline.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.
I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

New script tests pass:


┌─────────────────────┬──────────────┐
│        Test         │    Result    │
├─────────────────────┼──────────────┤
│ env                 │ PASS         │
├─────────────────────┼──────────────┤
│ error_json_response │ PASS (1m40s) │
└─────────────────────┴──────────────┘

…ad responses The alert data stream's CEL program could send an invalid request and crash on non-JSON responses, causing successful API calls to yield no indexed documents. Two fixes: - During pagination, when want_more was true but the cursor's first_time_amount was null, the program emitted a literal "&timeAmount=null" with no &timeUnit, producing an invalid request. Fall back to the configured time_amount/time_unit, matching the non-pagination branch. - Guard the response decode: a synthetic 200 with an empty or non-JSON (e.g. HTML 414) body no longer fails the program. Wrap the decode in a StatusCode/try/is_error check and return {"events": []} instead of crashing, preserving state for the next poll. Add script tests for the alert data stream: an env smoke test and an error_json_response test that interleaves HTML error pages with valid alert responses and asserts a valid alert is still collected without a spurious document. Both pass against a live stack.

github-actions · 2026-08-26T13:51:54Z

✅ Elastic Docs Style Checker (Vale)

No issues found on modified lines!

The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

infra-vault-gh-plugin-prod · 2026-08-26T13:52:07Z

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

Copilot

Pull request overview

This PR updates the prisma_cloud integration to prevent the alert data stream’s CEL program from dropping data during pagination (invalid timeAmount=null URL) and from crashing on empty/non-JSON HTTP 200 bodies, and adds script tests to guard the behavior.

Changes:

Fix pagination URL construction by falling back to configured time_amount/time_unit when cursor.first_time_amount is null.
Guard response decoding with StatusCode == 200 + try(...decode_json...) + is_error(...) to avoid CEL evaluation failures on non-JSON/empty bodies.
Add script tests (env, error_json_response) and bump package version + changelog.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
packages/prisma_cloud/manifest.yml	Bumps package version to 4.3.1.
packages/prisma_cloud/data_stream/alert/agent/stream/input.yml.hbs	Fixes pagination URL fallback and adds guarded JSON decoding for `/v2/alert` responses.
packages/prisma_cloud/data_stream/alert/_dev/test/scripts/error_json_response.txt	Adds regression script test for alternating HTML/non-JSON and valid JSON responses.
packages/prisma_cloud/data_stream/alert/_dev/test/scripts/env.txt	Adds basic env “smoke” script test.
packages/prisma_cloud/changelog.yml	Documents the 4.3.1 bugfix.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

efd6 · 2026-08-26T20:54:58Z

+        :
+          {"events": []}


The alternative is to make the auth token more robust; the documentation says that tokens live for 30min. So we can keep a token deadline next to the stored token and check that instead of abusing want_more.

The data stream seems to use a different API: https://pan.dev/prisma-cloud/api/cspm/prisma-cloud-login-api-overview/ whose JWT is only valid for 10m. Updated accordingly.

elastic-vault-github-plugin-prod · 2026-08-26T14:29:26Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

efd6 · 2026-08-26T20:49:09Z

      }).do_request().as(resp,
-        bytes(resp.Body).decode_json().as(inner_body, {
+        (resp.StatusCode == 200) ?
+        try(bytes(resp.Body).decode_json()).as(inner_body,
+          is_error(inner_body.with({})) ?
+            {"events": []}
+          :


Move the status code condition to the line above and fix the indentation.

Addressed in 0859951

efd6 · 2026-08-26T20:51:05Z

+        try(bytes(resp.Body).decode_json()).as(inner_body,
+          is_error(inner_body.with({})) ?


try will never return an error; its evaluation will always be the result of the child evaluation or either an object or a string depending on the call signature.

Suggested change

try(bytes(resp.Body).decode_json()).as(inner_body,

is_error(inner_body.with({})) ?

resp.Body.decode_json().as(inner_body,

is_error(inner_body) ?

Done in 0859951

efd6 · 2026-08-26T20:54:58Z

+        :
+          {"events": []}


The alternative is to make the auth token more robust; the documentation says that tokens live for 30min. So we can keep a token deadline next to the stored token and check that instead of abusing want_more.

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

+        :
+          {"events": []}


Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

efd6 · 2026-08-27T09:14:05Z

-      ).do_request().as(resp, bytes(resp.Body).decode_json().as(body, {
+      ).do_request().as(resp, resp.Body.decode_json().as(body, {
          "access_token": body.token,
+          "token_expiry": string(now + duration("9m")),


Suggested change

"token_expiry": string(now + duration("9m")),

"token_expiry": string(now() + duration("9m")),

efd6 · 2026-08-27T09:14:34Z

+          {
          "events": inner_body.items.map(e, {
            "message": e.encode_json(),
          }),


I think the indentation is wrong here.

Fixed in de6a04b

efd6 · 2026-08-27T09:15:35Z

-          }))
+          })
+        :
+          {"events": [], "token_expiry": string(now - duration("1m"))}


Why are we expiring this like this?

Agreed. We can force the auth immediately here. Attempted now in de6a04b. Now on a non-200 we clear access_token and the login guard re-auths on an empty token, forcing a fresh login next poll if the token is rejected before the 9m deadline (early revocation / mid-pagination 401).

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

packages/prisma_cloud/data_stream/alert/agent/stream/input.yml.hbs:131

On non-200 responses, the code only clears access_token. Because the rest of the pagination state (want_more, page_token, total_rows) is left untouched, a persistent error (e.g., 400 due to an invalid/expired pageToken) can cause the stream to keep retrying the same failing paginated request indefinitely. Resetting pagination state here makes recovery deterministic.

          {"events": [], "access_token": ""}

+      ).do_request().as(resp, resp.Body.decode_json().as(body, {
          "access_token": body.token,
+          "token_expiry": string(now() + duration("9m")),
      }))


elastic-vault-github-plugin-prod · 2026-08-27T09:47:08Z

✅ All changelog entries have the correct PR link.

vera-review-bot · 2026-08-27T09:47:32Z

🟢 No issues across the latest commits de6a04b.

Review summary

Issues found across earlier commits 0859951 — 1 high

🟠 The 25m token_expiry deadline likely exceeds Prisma Cloud's ~10-minute JWT lifetime, so the token goes stale before re-auth fires — reduce the deadline below the documented token TTL (and/or re-auth on a 401). (link) (Resolved)

A new commit triggers another review — at most once every 15 minutes. I skip the PR while it's approved or has merge conflicts.

🤖 AI-Generated Review | Vera Review Bot - v0.2.7 | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

infra-vault-gh-plugin-prod · 2026-08-27T10:19:30Z

💚 Build Succeeded

Buildkite Build
Commit: de6a04b

History

💚 Build #48651 succeeded b4e3584
💚 Build #48581 succeeded 95cd9ac

cc @kcreddy

efd6

nits only

efd6 · 2026-08-27T20:56:11Z

 program: |
  (
-    state.with(has(state.want_more) && !(state.want_more) ?
+    state.with(!has(state.access_token) || state.access_token == "" || !has(state.token_expiry) || now() >= timestamp(state.token_expiry) ?


Suggested change

state.with(!has(state.access_token) || state.access_token == "" || !has(state.token_expiry) || now() >= timestamp(state.token_expiry) ?

state.with(state.?access_token.orValue("") != "" || state.?token_expiry.optMap(t, now() >= timestamp(t)).orValue(true) ?

efd6 · 2026-08-27T20:57:54Z

-      ).do_request().as(resp, bytes(resp.Body).decode_json().as(body, {
+      ).do_request().as(resp, resp.Body.decode_json().as(body, {
          "access_token": body.token,
+          "token_expiry": string(now() + duration("9m")),


I think maybe it would be worth parameterising this to make it obvious. It can be a constant string in the state decl.

mergify · 2026-08-27T20:59:02Z

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

Queue this pull request

kcreddy added 2 commits August 26, 2026 19:18

update pr number

95cd9ac

kcreddy self-assigned this Aug 26, 2026

kcreddy added bugfix Pull request that fixes a bug issue Integration:prisma_cloud Palo Alto Prisma Cloud Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] labels Aug 26, 2026

kcreddy marked this pull request as ready for review August 26, 2026 13:52

kcreddy requested a review from a team as a code owner August 26, 2026 13:52

Copilot AI lite review requested due to automatic review settings August 26, 2026 13:52

kcreddy requested a review from a team as a code owner August 26, 2026 13:52

Copilot started reviewing on behalf of kcreddy August 26, 2026 13:52 View session

Copilot AI reviewed Aug 26, 2026

View reviewed changes

efd6 reviewed Aug 26, 2026

View reviewed changes

Address comments

0859951

Copilot AI review requested due to automatic review settings August 27, 2026 07:57

Copilot started reviewing on behalf of kcreddy August 27, 2026 07:58 View session

Copilot AI reviewed Aug 27, 2026

View reviewed changes

Comment thread packages/prisma_cloud/data_stream/alert/agent/stream/input.yml.hbs Outdated

Comment on lines +130 to +131

:

{"events": []}

vera-review-bot Bot reviewed Aug 27, 2026

View reviewed changes

Comment thread packages/prisma_cloud/data_stream/alert/agent/stream/input.yml.hbs Outdated

Review bot comment

b4e3584

Copilot AI review requested due to automatic review settings August 27, 2026 08:20

Copilot started reviewing on behalf of kcreddy August 27, 2026 08:21 View session

Copilot AI reviewed Aug 27, 2026

View reviewed changes

Comment thread packages/prisma_cloud/data_stream/alert/agent/stream/input.yml.hbs Outdated

kcreddy requested a review from efd6 August 27, 2026 08:54

efd6 reviewed Aug 27, 2026

View reviewed changes

Address comments

de6a04b

Copilot AI review requested due to automatic review settings August 27, 2026 09:41

Copilot started reviewing on behalf of kcreddy August 27, 2026 09:41 View session

Copilot AI reviewed Aug 27, 2026

View reviewed changes

Comment thread packages/prisma_cloud/data_stream/alert/agent/stream/input.yml.hbs

Comment on lines +36 to 39

).do_request().as(resp, resp.Body.decode_json().as(body, {

"access_token": body.token,

"token_expiry": string(now() + duration("9m")),

}))

efd6 approved these changes Aug 27, 2026

View reviewed changes

		try(bytes(resp.Body).decode_json()).as(inner_body,
		is_error(inner_body.with({})) ?

	"token_expiry": string(now + duration("9m")),
	"token_expiry": string(now() + duration("9m")),

	state.with(!has(state.access_token) \|\| state.access_token == "" \|\| !has(state.token_expiry) \|\| now() >= timestamp(state.token_expiry) ?
	state.with(state.?access_token.orValue("") != "" \|\| state.?token_expiry.optMap(t, now() >= timestamp(t)).orValue(true) ?

Uh oh!

Conversation

kcreddy commented Aug 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed commit message

Checklist

How to test this PR locally

Uh oh!

github-actions Bot commented Aug 26, 2026

✅ Elastic Docs Style Checker (Vale)

Uh oh!

infra-vault-gh-plugin-prod Bot commented Aug 26, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

elastic-vault-github-plugin-prod Bot commented Aug 26, 2026

🚀 Benchmarks report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

efd6 Aug 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

elastic-vault-github-plugin-prod Bot commented Aug 27, 2026

Uh oh!

vera-review-bot Bot commented Aug 27, 2026

Review summary

Uh oh!

infra-vault-gh-plugin-prod Bot commented Aug 27, 2026

💚 Build Succeeded

History

Uh oh!

efd6 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mergify Bot commented Aug 27, 2026

Uh oh!

Reviewers

kcreddy commented Aug 26, 2026 •

edited

Loading

efd6 Aug 26, 2026 •

edited

Loading