FlatFilers · obvious-autobuild-staging · Aug 24, 2026
diff --git a/.obvious/codebase-map.md b/.obvious/codebase-map.md
@@ -0,0 +1,22 @@
+# Codebase Map — FlatFilers/Examples
+
+Three independent demo apps, no shared code. Depth cap 2.
+
+| Path | What it is |
+|---|---|
+| `S3-upload/` | Express demo: Flatfile CSV importer that uploads to Amazon S3 via presigned URLs |
+| `S3-upload/index.js` | Server; `GET /` presigns an S3 `putObject` URL and renders it with the license into the view. Config block (AWS creds, license) lives inline here — but see obvious.md: env vars are what actually work |
+| `S3-upload/views/index.ejs` | Importer page; loads Flatfile + PapaParse from CDNs |
+| `S3-upload/public/` | Static assets: `importer.js` (client launch logic), `index.css`, `robots-clean.csv` sample data |
+| `mongodb-schema-importer/` | Express demo: generates a Flatfile importer config from MongoDB documents and/or Mongoose schemas |
+| `mongodb-schema-importer/index.js` | Server; `GET /` merges DB-derived field keys with Mongoose schema config, `GET /populate-defaults` seeds 4 robot documents. Reads gitignored `.env.js` |
+| `mongodb-schema-importer/schemas.js` | Mongoose `Robot` schema — edit this to change the generated importer fields |
+| `mongodb-schema-importer/configFromSchema.js` | Mongoose schema → Flatfile field/validator config (required, enum→select, regex, numeric, boolean→checkbox) |
+| `mongodb-schema-importer/views/`, `public/` | EJS importer page + static client assets (`importer.js`, `index.css`) |
+| `vue-demo/` | Vue 2 SPA demo of the Flatfile importer |
+| `vue-demo/src/` | `main.js` bootstrap, `App.vue` shell, `components/Welcome.vue` (launch button, raw-output pane, reads gitignored `env.js`) |
+| `vue-demo/env.example.js` | Template for gitignored `env.js` (license + importer fields) |
+| `vue-demo/public/` | `index.html` shell, `robots-clean.csv` sample data |
+| `.obvious/` | Agent contract: this map, `obvious.md`, `config.yml`, `skills/local-dev/SKILL.md` |
+
+Gitignored local files you must create before running: `mongodb-schema-importer/.env.js`, `vue-demo/env.js`, `mongodb-schema-importer/data/` (mongod dbpath).
diff --git a/.obvious/config.yml b/.obvious/config.yml
@@ -0,0 +1,6 @@
+# Repo policy for Obvious workers.
+# Auto-detected from GitHub repo settings (FlatFilers/Examples) on 2026-08-24.
+defaultBranch: master
+# Repo allows merge commits, squash, and rebase; squash is the default choice.
+mergeMethod: squash
+deleteBranchOnMerge: false
diff --git a/.obvious/obvious.md b/.obvious/obvious.md
@@ -0,0 +1,122 @@
+# FlatFilers/Examples — Agent Guide
+
+Monorepo of three independent Flatfile demo applications. There is **no root
+package.json, no shared build, and no CI** — each top-level directory is a
+standalone app with its own dependencies, lockfile, and start command. Always
+`cd` into the app directory before running anything.
+
+## Stack
+
+| App | Stack | Package manager | Port | External services |
+|---|---|---|---|---|
+| `S3-upload/` | Node.js, Express 4, EJS, aws-sdk v2 | npm (`package-lock.json`) | 9000 | AWS S3 (presign), Flatfile license |
+| `mongodb-schema-importer/` | Node.js, Express 4, EJS, Mongoose 5, mongodb driver 3.0 | npm (`package-lock.json`; `yarn.lock` also present) | 9000 | MongoDB (local), Flatfile license |
+| `vue-demo/` | Vue 2, Vue CLI 3 (webpack 4) | yarn classic 1.x (`yarn.lock`) | 8080 | Flatfile license |
+
+- Runtime: Node.js 20 (verified). Plain JavaScript — no TypeScript, no test suites, no Docker/Compose.
+- **`S3-upload` and `mongodb-schema-importer` both hardcode port 9000** (`app.listen(9000)` in each `index.js`). Run one at a time; stop the other first.
+- Verified toolchain in the sandbox snapshot: Node 20.20.2, npm 10.8.2, yarn 1.22.22, mongod 4.4.29, Playwright Chromium (screenshots).
+
+## Commands
+
+### S3-upload (Express, port 9000)
+
+```bash
+cd S3-upload
+npm install
+AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... AWS_REGION=us-east-1 npm start
+# → http://localhost:9000
+```
+
+Credentials go in **env vars, not the `config` object in `index.js`**: the S3
+client is constructed before `aws.config.update()` runs, so the in-code config
+never takes effect and the page renders empty (`CredentialsError`). Dummy
+AWS-shaped values are enough for local page rendering — SigV4 presigning is a
+local computation — but real credentials are needed for actual uploads.
+
+### mongodb-schema-importer (Express + MongoDB, port 9000)
+
+```bash
+cd mongodb-schema-importer
+npm install
+# create .env.js (gitignored) — see below
+npm run mongod        # mongod --dbpath=data --port 28015 (needs mongod ≤ 4.x on PATH)
+npm start             # → http://localhost:9000
+curl http://localhost:9000/populate-defaults   # seeds 4 robots
+```
+
+`.env.js` (gitignored, must be created):
+
+```js
+module.exports = {
+  URL: 'mongodb://localhost:28015',
+  DBNAME: 'flatfile-demo',
+  COLLECTION: 'robots',
+  FF_LICENSE: '<flatfile-license>'
+}
+```
+
+**MongoDB version constraint:** the pinned `mongodb@~3.0.8` driver speaks the
+legacy OP_QUERY wire protocol. MongoDB ≥ 5.1 rejects it
+(`UnsupportedOpQueryCommand`) and the app crashes on `/populate-defaults`.
+Use **mongod 4.4.x** (installed at `/home/user/.local/bin/mongod`, symlinked
+to `/usr/local/bin/mongod` in the snapshot; needs `libssl1.1` on Ubuntu 24.04).
+
+### vue-demo (Vue 2 dev server, port 8080)
+
+```bash
+cd vue-demo
+yarn install                     # yarn classic 1.x (yarn.lock v1)
+cp env.example.js env.js         # gitignored; fill in license + fields
+NODE_OPTIONS=--openssl-legacy-provider yarn serve   # → http://localhost:8080
+```
+
+`NODE_OPTIONS=--openssl-legacy-provider` is **required on Node ≥ 17** (webpack 4
+MD4 hashing). Without it the dev server dies with
+`error:0308010C:digital envelope routines::unsupported`.
+
+### Lint / tests
+
+- `cd vue-demo && yarn lint` — the only lint target in the repo; passes (`No lint errors found!`).
+- No test suites or typechecks exist in any app; every `npm test` script is a stub that exits 1.
+
+## Codebase map
+
+See [codebase-map.md](codebase-map.md) for the folder-level overview.
+
+## Local Verification Summary
+
+Verified 2026-08-24 in sandbox `i517zqa9z31yfh4v3rhj1` (Node 20.20.2):
+
+| Check | Result |
+|---|---|
+| `S3-upload` `npm start` + `GET /` | HTTP 200, page renders with presigned S3 URL (dummy env creds) |
+| `mongodb-schema-importer` `npm run mongod` + `npm start` | mongod 4.4.29 up on 28015; app up on 9000 |
+| `GET /populate-defaults` | HTTP 200, "Inserted 4 Robots" |
+| `GET /` (DB-driven) | HTTP 200; DB_config fields `name, nick, helmet, color, id` derived from seeded collection; Mongoose Robot schema config (6 fields) rendered |
+| `vue-demo` `yarn serve` | Compiled successfully; HTTP 200 on 8080; HMR active |
+| `vue-demo` `yarn lint` | Pass — no lint errors |
+| Browser screenshots | `s3-upload-home.png`, `mongo-importer-home.png`, `vue-demo-home.png` (Playwright Chromium, in `/home/user/evidence/`) |
+
+Primary user flows exercised: (1) mongo app — seed collection then render the
+importer page whose field list is generated from live DB documents; (2) S3-upload —
+render page with a freshly presigned S3 upload URL; (3) vue-demo — SPA boots,
+"Import robots" button + sample CSV link present, HMR connected.
+
+## Sandbox snapshot
+
+- Snapshot/template ID: `gap0pvit10wten6lqhs1:default` (sandbox `i517zqa9z31yfh4v3rhj1`)
+- Captured: 2026-08-24T21:29:58.777Z
+- State at capture: all three apps' `node_modules` installed; `.env.js` and `env.js` present (gitignored); mongod 4.4.29 running on 28015 with seeded `robots` collection; `mongodb-schema-importer` app on 9000; `vue-demo` dev server on 8080; git tree clean.
+
+## Known limitations
+
+- Flatfile license keys are local placeholders. The in-browser importer widget
+  (external SaaS) cannot fully launch without a real license — the page, config
+  generation, and server flows all work; the widget launch does not.
+  TODO(confirm): obtain a Flatfile license key to exercise the full import flow.
+- `S3-upload` needs real AWS credentials for actual S3 uploads (presigning works with dummies).
+- EJS pages load Flatfile/PapaParse scripts from unpkg/cdnjs — browser needs network access.
+- `mongodb-schema-importer` ships both `package-lock.json` and `yarn.lock`; running
+  `npm install`/`npm ci` there rewrites `yarn.lock` — restore it with
+  `git checkout -- mongodb-schema-importer/yarn.lock` before committing.
diff --git a/.obvious/skills/local-dev/SKILL.md b/.obvious/skills/local-dev/SKILL.md
@@ -0,0 +1,72 @@
+---
+name: local-dev
+description: Bring up the three Flatfile demo apps in FlatFilers/Examples locally (Express apps, local mongod 4.4, Vue dev server) and verify the primary flows
+---
+
+# Local Dev — FlatFilers/Examples
+
+Durable record of the onboarding run (2026-08-24, sandbox `i517zqa9z31yfh4v3rhj1`,
+Node 20.20.2). Restoring the snapshot `gap0pvit10wten6lqhs1:default` gives you
+this state pre-built: deps installed, configs created, mongod seeded, two apps running.
+
+## Prerequisites (baked into the snapshot)
+
+- Node 20 + npm; yarn classic 1.22.x (`npm i -g yarn@1.22.22`) — vue-demo needs yarn, not npm.
+- **mongod 4.4.29** at `/home/user/.local/bin/mongod`, symlinked to `/usr/local/bin/mongod`.
+  On a fresh Ubuntu 24.04 box: install `libssl1.1` (bionic deb from security.ubuntu.com),
+  then the `mongodb-linux-x86_64-ubuntu2004-4.4.29` tarball. **Do not use mongod ≥ 5.1** —
+  the pinned `mongodb@~3.0.8` driver uses OP_QUERY and MongoDB 8.x rejects it.
+- Playwright + Chromium (global, `/home/user/.npm-global`) for screenshots; system deps
+  via `sudo playwright install-deps chromium`.
+
+## Bring-up sequence
+
+```bash
+cd /home/user/work/Examples
+
+# 1. MongoDB (dbpath is gitignored, port 28015 — matches .env.js)
+cd mongodb-schema-importer && mkdir -p data && nohup mongod --dbpath=data --port 28015 --bind_ip 127.0.0.1 > /tmp/mongod.log 2>&1 & cd ..
+
+# 2. mongodb-schema-importer app (port 9000)
+cd mongodb-schema-importer && nohup npm start > /tmp/mongo-app.log 2>&1 & cd ..
+curl -s http://localhost:9000/populate-defaults   # seed 4 robots → "Inserted 4 Robots"
+
+# 3. vue-demo dev server (port 8080) — legacy OpenSSL flag is mandatory on Node ≥ 17
+cd vue-demo && NODE_OPTIONS=--openssl-legacy-provider nohup yarn serve > /tmp/vue-demo.log 2>&1 & cd ..
+
+# 4. S3-upload (port 9000 — conflicts with step 2; stop it first: kill the node PID from `ss -ltnp | grep :9000`)
+cd S3-upload && AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY AWS_REGION=us-east-1 nohup npm start > /tmp/s3-upload.log 2>&1 & cd ..
+```
+
+## Required local files (all gitignored)
+
+- `mongodb-schema-importer/.env.js` → `{ URL: 'mongodb://localhost:28015', DBNAME: 'flatfile-demo', COLLECTION: 'robots', FF_LICENSE: '<placeholder ok>' }`
+- `vue-demo/env.js` → copy of `env.example.js` with placeholder license + fields
+
+## Verification checklist (all passed 2026-08-24)
+
+1. `curl -w '%{http_code}' http://localhost:9000/` → 200; app log shows
+   `Connected successfully to server` and `Found the following robots traits: ['name','nick','helmet','color','id']`.
+2. `curl http://localhost:9000/populate-defaults` → `Inserted 4 Robots`.
+3. `curl -w '%{http_code}' http://localhost:8080/` → 200; vue log shows `Compiled successfully`.
+4. `cd vue-demo && yarn lint` → `No lint errors found!`.
+5. S3-upload `GET /` → 200 with `var signedRequest = "https://s3.amazonaws.com/..."` in the HTML.
+6. Screenshots via `NODE_PATH=/home/user/.npm-global/lib/node_modules node /home/user/evidence/shot.js <url> <out.png>`.
+
+## Gotchas learned the hard way
+
+- **S3-upload credentials**: set `AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY`/`AWS_REGION`
+  env vars. The inline `config` object in `index.js` is applied *after* `new aws.S3()`,
+  so it never reaches the client — with no env vars the page renders empty (HTTP 200, 0 bytes).
+- **Port 9000 is hardcoded** in both Express apps. Check for a listener before starting
+  (`ss -ltnp | grep :9000`); kill by PID — never `pkill -f "node index.js"` (it matches your own shell).
+- **vue-demo on Node ≥ 17**: `NODE_OPTIONS=--openssl-legacy-provider` or webpack 4 dies with
+  `digital envelope routines::unsupported`.
+- **npm in mongodb-schema-importer rewrites `yarn.lock`** (npm syncs it when present).
+  Run `git checkout -- mongodb-schema-importer/yarn.lock` before committing anything.
+- **mongod 8.x breaks the app**: `Unsupported OP_QUERY command: insert` → assertion crash
+  on `/populate-defaults`. If `data/` was created by a newer mongod, wipe it before
+  starting 4.4 (`rm -rf mongodb-schema-importer/data && mkdir -p mongodb-schema-importer/data`).
+- Long-running shell commands cap at ~300s — start each service in its own short command.
+- The Flatfile importer widget itself needs a real license key (external SaaS); everything
+  server-side works with placeholders.