Route studio create --from URLs through Data Liberation

[chubes4]

Summary

• Routes studio create --from <url> through the bundled Data Liberation CLI.
• Uses Data Liberation's portable HTML site directory as the handoff to the Static Site Importer materialization path landed in Add studio create --from for static sources #4725.
• Syncs the bundled Data Liberation package to the exact draft head of Remove the WordPress destination data-liberation-agent#119 (6a81d1a3) after rebasing it onto current Data Liberation main.
• Removes the obsolete WordPress reconstruction surface from Studio's vendored Data Liberation snapshot.
• Keeps local files, directories, ZIPs, and canonical artifacts on the existing Add studio create --from for static sources #4725 path.

Flow

1. Studio invokes the bundled Data Liberation CLI with the source URL, a deterministic output base, and --resume.
2. Data Liberation captures the site and reports its portable website directory through the CLI's Site: output contract.
3. Studio validates that path is a directory inside the requested output base.
4. Studio stages the portable site through Static Site Importer, which owns WordPress materialization.
5. The capture is retained after failure for resume and removed after a successful import.

studio create \
  --path ./imported-site \
  --name "Imported Site" \
  --from https://example.com \
  --skip-browser

Completion Gate

This PR and Automattic/data-liberation-agent#119 remain draft while Static Site Importer and Blocks Engine mature. The portable-HTML handoff is integrated here for proving and iteration; it is not ready to replace the existing Data Liberation path until representative external sites can be imported mostly reliably as static, editable WordPress sites through this canonical flow.

Data Liberation owns complete portable HTML capture, Static Site Importer owns deterministic WordPress materialization, and Blocks Engine owns HTML-to-block conversion. Remaining failures should be fixed in those owning layers using generic primitives. Individual generated sites still require exact frontend parity, usable block-editor output, and zero fallback blocks before acceptance.

Validation

• Focused Studio URL and SSI tests: 81 passed.
• Data Liberation suite: 856 passed.
• Data Liberation liberation/path contract tests: 11 passed.
• Studio CLI TypeScript check passed.
• Focused Studio CLI ESLint passed.
• Studio CLI development build passed.
• The built Studio distribution contains and executes the bundled Data Liberation CLI (0.1.0).
• git diff --check passed with generated bundles marked as deterministic binary artifacts.

Dependency State

Automattic/data-liberation-agent#119 remains open, draft, and unmerged at 6a81d1a3, based directly on Data Liberation main at 505a04c1. This PR vendors that exact reviewed snapshot.

AI Assistance

OpenAI GPT-5.6 Sol via OpenCode was used to inspect the existing branches, reconstruct this PR on current trunk, wire and test the Data Liberation CLI handoff, rebase and audit the vendored #119 snapshot, and run verification. Chris Huber remains responsible for the submitted changes.

[chubes4]

CI follow-up pushed in 7d7f45d8c:

• Unit lane now runs npm run test -- --tagsFilter='!e2e', keeping tagged CLI integration suites in the dedicated serialized CLI E2E lane.
• Filtered Unit verification passed: 211 files, 2747 passed, 60 skipped.
• Focused start-stop.e2e.test.ts passed: 2/2.
• The full local CLI E2E lane remains machine-sensitive under concurrent Playground/PHP load; isolated lifecycle coverage passes and the existing dedicated lane remains serialized.

AI assistance: openai/gpt-5.6-sol via OpenCode diagnosed the CI lane overlap, drafted the one-line test-selection change, and ran verification. Chris Huber remains responsible for the change.

[chubes4]

1,145-route scale validation

The latest branch head (ca6191a59) is pushed and the end-to-end Richlynn run completed successfully through this Studio CLI path.

• Source: https://www.richlynngroup.com/
• Routes: 1,145/1,145 completed
• Batches: 124/124 completed
• Final state: zero failed or pending batches
• WordPress routes: 1,148 unique page URIs, no duplicate paths
• Cache: 5,364 network requests avoided
• Recovery: resumed across compiler interruption, encoded Unicode paths, transient HTML classification, and canonical route collisions

This proves the Studio orchestration and dependent SSI batching path can handle the target scale. It does not mark the generated site as fidelity-complete: Richlynn still has 13,055 core/html blocks, 1,235 reported fallbacks, 396 retained external assets, and no exact visual/editor parity proof. The generated site remains evidence under the zero-fallback gate.

Dependency PRs:

• Support resumable large URL site imports static-site-importer#798
• Harden bounded site artifact compilation blocks-engine#776

Validation on the pushed Studio head: npx vitest run apps/cli/commands/site/tests/create.test.ts apps/cli/lib/tests/run-wp-cli-command.test.ts (66/66 passed).

AI assistance: openai/gpt-5.6-sol via OpenCode was used for scale-run diagnosis, implementation support, verification, and drafting this evidence update. Chris Huber remains responsible for the submitted changes.

[borkweb]

The /liberate command already imports static sites into Studio via the blocks-engine 🕺

You can do so via: /liberate ~/path/to/static/site

That being said, switching out liberate's tooling to a different set of direct calls would make sense if the php-transformer has caught up to the JS approach! I also like the direct CLI command that you have for it.

[chubes4]

@borkweb I have not compared the outputs side by side, but I was hoping the approaches could coexist initially. This PR intentionally avoids disrupting existing behavior.

Longer term, I think Data Liberation could own the browser-dependent stages: client-side rendering, runtime behavior, and the visual feedback loop, and use the Studio CLI to materialize the site.

The shared contract would be a clean HTML-based website artifact, regardless of the original source.

I think that refactoring can be explored separately from this PR. Redirecting /liberate is not necessary for this primitive to exist.

[jordesign]

studio create --from <url> fails by default: bundled SSI release predates the request-bundle resolver this PR's own request-building code depends on

DEFAULT_STATIC_SITE_IMPORTER_PLUGIN_URL (apps/cli/commands/site/create.ts) points at the published v1.8.5 release zip. resolveStaticSiteImporterSource()/buildStaticSiteImporterRequest() in this same file build directory-sourced import requests as { type: 'files', ref: 'request-bundle:source' } — an opaque reference that only resolves if the installed SSI plugin registers a static_site_importer_resolve_source_reference filter for it via static_site_importer_cli_prepare_request_bundle() in includes/cli.php.

That resolver mechanism was added in Automattic/static-site-importer#1433 / #1448 / #1459 — all merged 2026-08-31. v1.8.5 predates it. I confirmed directly: the plugin actually installed from the default URL has no static_site_importer_cli_prepare_request_bundle function at all.

Reproduction

studio create --path ./imported-site --name "Imported Site" \
  --from https://www.arrowmartialarts.com/ --skip-browser

Liberation and WordPress-server startup succeed; the static-site import step fails immediately:

{"success":false,"error":{"code":"static_site_importer_source_reference_unresolved","message":"The opaque source reference was not resolved by a server-owned resolver."}}

This isn't specific to this site — it's guaranteed for every --from <url> import against the default plugin URL, for any user who doesn't manually pass --static-site-importer-path with a build newer than v1.8.5.

The code comment above DEFAULT_STATIC_SITE_IMPORTER_PLUGIN_URL already documents the intended workaround (npm run build:dev-package -- --blocks-engine-path <path>), but that path is currently blocked by Automattic/static-site-importer#1272 (open) — homeboy rejects the reconstructed snapshot for lacking a .git root. I worked around both by hand-building a plugin zip from SSI main (composer install --no-dev + zip) and passing it via --static-site-importer-path; that build has the resolver and imports proceed past this step.

Suggested direction

Either (a) cut a new SSI release that includes the resolver and bump DEFAULT_STATIC_SITE_IMPORTER_PLUGIN_URL to it, or (b) have Studio build/vendor SSI from a pinned source ref the way it already vendors data-liberation-agent, rather than depending on an external release's cadence for a feature this PR's own request-building code assumes exists. Not proposing a specific fix here since it's a distribution/architecture call — flagging with full repro so it's visible before this comes out of draft.

AI assistance

Claude Code (Sonnet 5) reproduced this while running the loop described in this PR's own workflow against a live external site, traced it to the resolver/version mismatch, and verified the fix path by hand-building a compatible local package. Jordan Gillman orchestrated the investigation and is responsible for this report.