Skip to content

feat(soc-optimization-unified): add SOCFW Framework Health dashboard - #1174

Merged
scottbrumley merged 1 commit into
mainfrom
feat/socfw-health-dashboard
Sep 24, 2026
Merged

scottbrumley merged 1 commit into
mainfrom
feat/socfw-health-dashboard

Conversation

@scottbrumley

Copy link
Copy Markdown
Contributor

The framework-health half of the 15 Sep dashboard split. Lifecycle-outcome dashboards moved to the NIST IR packs in #1169-#1171; this is the other half and it ships in core.

Command errors first, because the data found a live problem nothing was surfacing.

What it shows

Measured on a reference tenant over 30 days:

command executions with has_error 3,830 of 5,771 (66%)
every one of them action_status = integration_unavailable
minutes credited to commands that never ran 7,200

Nine widgets

  • Command Error Rate (%), Failed Commands, Minutes Credited to Failed Commands — the headline three
  • Failing Commands: Universal → Vendor — the pair that matters, not the verb alone. soc-enrich-endpoint appears twice against two different vendor commands
  • Failure Reason and Command Outcomes by Status — why failures happen, and the shape of all outcomes including simulated, which is shadow mode working rather than a fault
  • Failing Commands - Detail — drilldown table: verb, product, vendor command, failed vs total, error rate, affected cases. Every failing pair currently reads 100%, so this is unavailability rather than flakiness
  • Command Failures by Day — a single bad day dominates the 30-day average; the rate needs reading beside the trend
  • Framework Write Failures — from playbook_tasks: socfw-post-to-dataset failed 1,274 times, so the execution dataset is missing rows and every metric built on it understates

Build notes

Structure copied from a shipped dashboard rather than authored from scratch. Two install rules learned the hard way, both via 101704:

  • a table widget takes no | view clause — every other type requires one
  • there is no line chart type in the shipped vocabulary (single, column, pie, funnel, table only)

default_dashboard_id left absent, matching the Value Driver dashboards. Widget keys conform to xql_<13-digit epoch>, row ids to row-<digits>.

Verification

Iterated through the dashboards REST API (/public_api/v1/dashboards/{get,insert,delete}) rather than repeated pack installs, so no orphaned dashboards were created while building. Every query run against a reference tenant before it went in the file. check_contribution including the upload step, green.

The framework-health half of the 15 Sep dashboard split. Lifecycle-outcome
dashboards moved to the NIST IR packs in #1169-#1171; this is the other half
and it ships in core.

Command errors first, because the data found a live problem: on a reference
tenant, 3,830 of 5,771 command executions over 30 days carry has_error, every
one of them action_status = integration_unavailable. Nothing surfaced that.

Seven widgets:
- Command Error Rate (%) and Failed Commands, side by side
- Minutes Credited to Failed Commands - the value-inflation number, to read
  against hours returned on the NIST IR dashboards
- Failures by Universal Command - which framework verb is failing
- Failure Reason - by action_status; does not assume integration_unavailable
- Failing Commands, Detail - the drilldown: verb, product, vendor command,
  reason and affected case count per failing command
- Command Failures by Day - a single bad day dominates a 30-day average, so
  the trend has to be readable beside the rate

Structure copied from an existing shipped dashboard rather than authored from
scratch: xql_<epoch> widget keys, row-<digits> row ids, default_dashboard_id
left at the platform constant 1, is_public/is_predefined/creator_mail on every
widget.

Every query run against a reference tenant before it went in the file.
@scottbrumley scottbrumley added the version:patch Bug fix or hotfix → x.x.N label Sep 24, 2026
@scottbrumley
scottbrumley merged commit 88acbf0 into main Sep 24, 2026
15 of 24 checks passed
@scottbrumley
scottbrumley deleted the feat/socfw-health-dashboard branch September 24, 2026 20:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

version:patch Bug fix or hotfix → x.x.N

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant