test(integ-test): stabilize stream-order commands across shards

[mengweieric]

Summary

Several Streamstats, Reverse, Dedup, and Patterns tests relied on the incidental encounter order of a single-shard index. On multiple shards, the commands returned valid results for a different stream order and the tests asserted different row content.

This change uses deterministic makeresults streams where exact order is part of the test, and membership/cardinality assertions where representative selection is not defined. Real multi-shard index coverage remains through order-independent property checks. Tests that require nullable numeric streams or expose known engine gaps are intentionally unchanged.

No production behavior is modified.

Validation

• Verified on an external cluster forced to five primary shards
• Exercised direct and no-pushdown paths
• Affected suite failures reduced from 104 to 40; remaining failures are documented engine/contract gaps
• spotlessCheck, compileTestJava, and git diff --check pass

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🧪 PR contains tests

🔒 No security concerns identified

✅ No TODO sections

🔀 No multiple PR themes

⚡ No major issues detected

[github-actions]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
Possible issue	Add null check for server lookup The test assumes messagesByServer.get(row.getString(0)) always returns a non-null set, but if an unexpected server name appears in the data, this will cause a NullPointerException. Add a null check or assertion to fail gracefully with a clear error message. integ-test/src/test/java/org/opensearch/sql/calcite/remote/CalciteStreamstatsCommandIT.java [1531-1545] -Map<String, Set<String>> messagesByServer = - Map.of( - "server1", Set.of("Database connection failed", "High memory usage"), - "server2", Set.of("Service started", "Backup completed"), - "server3", Set.of("Disk space low")); -JSONArray rows = actual.getJSONArray("datarows"); -assertEquals(5, rows.length()); for (int i = 0; i < rows.length(); i++) { JSONArray row = rows.getJSONArray(i); - Set<String> validMessages = messagesByServer.get(row.getString(0)); + String server = row.getString(0); + Set<String> validMessages = messagesByServer.get(server); + assertNotNull("unexpected server: " + server, validMessages); assertTrue(validMessages.contains(row.getString(1))); assertTrue(validMessages.contains(row.getString(2))); assertTrue(validMessages.contains(row.getString(3))); } Suggestion importance[1-10]: 7 __ Why: The code assumes messagesByServer.get(row.getString(0)) returns a non-null set, which could cause a NullPointerException if an unexpected server appears. Adding a null check would make the test fail with a clearer error message, improving debuggability.	Medium
General	Extract duplicated helper method The dataRows helper method is duplicated across multiple test files. Consider extracting this common JSON-to-list conversion logic into a shared test utility class to reduce code duplication and improve maintainability. integ-test/src/test/java/org/opensearch/sql/calcite/remote/CalcitePPLDedupIT.java [582-594] -JSONArray arr = response.getJSONArray("datarows"); -for (int i = 0; i < arr.length(); i++) { - JSONArray r = arr.getJSONArray(i); - List<Object> row = new ArrayList<>(); - for (int j = 0; j < r.length(); j++) { - row.add(r.isNull(j) ? null : r.get(j)); - } - rows.add(row); -} +// Extract to shared utility class (e.g., TestUtils.dataRows(response)) +return TestUtils.dataRows(response); Suggestion importance[1-10]: 6 __ Why: The dataRows helper is duplicated in multiple test files (CalcitePPLDedupIT and CalcitePPLPatternsIT). Extracting it to a shared utility would reduce duplication and improve maintainability, though the impact is moderate since it's test code.	Low
	Remove redundant email validation The email validation is redundant since the token reconstruction already verifies the structure. If the tokens reconstruct the sample exactly and the pattern is @., the email format is implicitly validated. Remove the redundant regex check to simplify the assertion. integ-test/src/test/java/org/opensearch/sql/calcite/remote/CalcitePPLPatternsIT.java [189-192] for (int i = 0; i < samples.size(); i++) { assertEquals(samples.get(i), t1.get(i) + "@" + t2.get(i) + "." + t3.get(i)); - assertTrue("not an email: " + samples.get(i), EMAIL.matcher(samples.get(i)).matches()); } Suggestion importance[1-10]: 4 __ Why: The email regex validation is indeed redundant given the token reconstruction already verifies the exact structure. However, the redundancy provides an additional safety check and the performance impact is negligible in tests, so removing it offers only minor benefit.	Low

[penghuo]

The test set not human readable. 2 ideas

• change take(content, 1) to min/max.
• add containsInAnyOrder

[penghuo]

does sort help?