Skip to content

Expand import diagnostics with validation failure playbooks and artifact tooling - #2173

Open
rohitkumarbhagat wants to merge 27 commits into
datacommonsorg:masterfrom
rohitkumarbhagat:failure-playbooks
Open

Expand import diagnostics with validation failure playbooks and artifact tooling#2173
rohitkumarbhagat wants to merge 27 commits into
datacommonsorg:masterfrom
rohitkumarbhagat:failure-playbooks

Conversation

@rohitkumarbhagat

Copy link
Copy Markdown
Contributor

Summary

  • Add hypothesis-driven troubleshooting for network failures, incomplete source acquisition, validation failures, and deleted observations.
  • Document auto-refresh artifacts and the Import Differ, validation, and StatVar Processor contracts used during diagnosis.
  • Add local artifact-querying guidance and an approval-gated helper for creating short-lived BigQuery tables.
  • Require read-only BigQuery use after table creation and prohibit table updates or deletion.
  • Clarify diagnostic evidence boundaries, cloud configuration review, Python execution, and dependency handling.

Testing

  • git diff --check upstream/master...HEAD
  • bash -n agents/skills/dc-import-diagnostics/scripts/create_short_lived_bq_table.sh
  • agents/skills/dc-import-diagnostics/scripts/create_short_lived_bq_table.sh --help
  • Skill validation with quick_validate.py
  • .env/bin/python -m unittest agents.common.scripts.skill_contract_test

@rohitkumarbhagat
rohitkumarbhagat enabled auto-merge (squash) August 21, 2026 15:48

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the import diagnostics agent skills, documentation, and troubleshooting guides. Key changes include removing the optional sibling import checkout checks, adding comprehensive guides for debugging deleted observations, network failures, and source acquisition completeness, and introducing a script (create_short_lived_bq_table.sh) to load GCS files into short-lived BigQuery tables for querying. It also expands documentation for the Import Differ, Import Validation Framework, and StatVar Processor. A high-severity issue was identified in the new BigQuery script where the bq show command is called with an unsupported --dataset flag.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant