Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,14 @@ Want the turnkey, print-and-go version? See [`docs/BuildKit.md`](docs/BuildKit.m

Need the fastest “we’re alive” path? The first‑play ritual lives in [`docs/FirstSession.md`](docs/FirstSession.md). It’s a 10‑minute, no‑excuses script: connect hardware, flash firmware, open serial, run the bridge, fire up the visualizer, and paste a known‑good gesture packet to prove the loop.

## 90‑minute onboarding bootcamp

Want a structured ramp for new collaborators? [`docs/Onboarding90Min.md`](docs/Onboarding90Min.md) is a one‑session syllabus: get the pipeline working, narrate sensors, tune one gesture knob, and leave able to teach the system.

## Gesture vocabulary (read‑aloud heuristics)

Need the “say it out loud” version of the gesture engine? [`docs/GestureVocabulary.md`](docs/GestureVocabulary.md) ties the human feel of pluck/bow/scrape/harmonic/mute/tremolo/vibrato to the exact firmware knobs.

## Runtime note-set auditions (Serial preset browser)

When you hear someone shout “try it in Hirajōshi!”, you no longer have to recompile.
Expand Down
133 changes: 133 additions & 0 deletions docs/GestureVocabulary.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
# Gesture Vocabulary (Narratable Heuristics)

_This is the “read it out loud” guide to the gesture engine. It explains **why** each gesture exists, **how** we detect it, and **which knobs** to tune when reality gets noisy. Pair this with `firmware/src/gesture_engine.cpp` and the parameter table in `firmware/include/gesture_engine.h`._

## Big idea: gestures are stories, not equations

We don’t look for “perfect signals.” We look for **behavior** that players can repeat and instructors can narrate. Every gesture is a small, human‑readable rule with a few thresholds you can tweak while the room listens.

---

## Gesture quick map (what the engine calls it)

| Gesture | How it feels | What the engine looks for |
| --- | --- | --- |
| **Pluck** | A single, intentional attack | Contact crosses `on_thresh` after enough time since the last onset |
| **Scrape** | Rapid, grainy micro‑attacks | Two onsets closer than `scrape_window_us` |
| **Bow** | Sustained touch / continuous motion | Continuous contact, no special harmonic/tremolo trigger |
| **Harmonic** | Light, steady touch with “glass” tone | Peak stays within `harmonic_peak_min`..`max` **and** stays still for `harmonic_hold_us` |
| **Muted** | Short, damped touch | Release within `mute_window_us` **or** drop below `mute_release_thresh` |
| **Tremolo** | Shallow, fast oscillation | Wobble count hits `wobble_goal` with low depth |
| **Vibrato** | Deeper oscillation | Wobble count hits `wobble_goal` with depth ≥ `vibrato_depth_min` |

---

## The narratable rules (human language first)

### 1) Pluck

**Say this:**
“A pluck is a clear crossing into contact. If it happens too soon after the last one, we ignore it. If it’s close enough to a prior onset, we call it a scrape instead.”

**Knobs that matter:**
`on_thresh`, `off_thresh`, `min_retrigger_us`, `scrape_window_us`

---

### 2) Scrape

**Say this:**
“A scrape is just a **rapid series of onsets**. The time between them is shorter than a pluck.”

**Knobs that matter:**
`scrape_window_us` (shorter = stricter, longer = more scrapes)

---

### 3) Bow

**Say this:**
“Bow is the default: it’s what we call continuous contact when nothing else is true. It’s sustained energy with no special wobble or harmonic signature.”

**Knobs that matter:**
Mostly `on_thresh` and `off_thresh`, plus smoothing in your sensor front‑end.

---

### 4) Harmonic

**Say this:**
“A harmonic is a **light, steady touch**. You hit a gentle peak, then hold it still long enough to earn the glassy tone.”

**Knobs that matter:**
`harmonic_peak_min`, `harmonic_peak_max`, `harmonic_hold_us`, `harmonic_variation_eps`

**Calibration ritual:**

1. Lightly graze the sensor.
2. Nudge `harmonic_peak_min`/`max` until harmonics trigger without full plucks.
3. Tighten `harmonic_variation_eps` until the system demands stillness.

---

### 5) Muted

**Say this:**
“A mute is a **short, damped touch**. If the contact ends quickly, or the signal drops below the mute floor, we call it muted.”

**Knobs that matter:**
`mute_peak_thresh`, `mute_window_us`, `mute_release_thresh`

**Teaching cue:**
Ask the class to “tap and vanish” — quick contact then immediate release.

---

### 6) Tremolo

**Say this:**
“Tremolo is a **fast, shallow wobble**. The engine counts sign flips and calls it once it sees enough.”

**Knobs that matter:**
`tremolo_min_delta`, `tremolo_max_period_us`, `tremolo_grace_us`, `wobble_goal`

---

### 7) Vibrato

**Say this:**
“Vibrato is tremolo with more depth. Same wobble count, bigger swing.”

**Knobs that matter:**
`vibrato_depth_min` (raise it to demand bigger swings)

---

## Threshold cheat‑sheet (firmware knobs)

These live in `firmware/include/gesture_engine.h`. Keep these names in your teaching script so students can go from “feel” to “code” without translation.

- **Contact gates:** `on_thresh`, `off_thresh`
- **Time guards:** `min_retrigger_us`, `scrape_window_us`, `harmonic_hold_us`, `mute_window_us`, `tremolo_grace_us`
- **Harmonic band:** `harmonic_peak_min`, `harmonic_peak_max`, `harmonic_variation_eps`
- **Mute window:** `mute_peak_thresh`, `mute_release_thresh`
- **Wobble rules:** `tremolo_min_delta`, `tremolo_max_period_us`, `wobble_goal`, `vibrato_depth_min`

---

## In‑class debugging prompts (keep it human)

- “Is the sensor normalized to 0..1?” (Check your sensor front‑end.)
- “Do we see the peak hit the harmonic band?” (Plot the envelope.)
- “Is the wobble count too strict?” (Lower `wobble_goal` or `tremolo_min_delta`.)
- “Are we double‑triggering?” (Raise `min_retrigger_us`.)

---

## Next expansion ideas (for brave collaborators)

- **Glissando**: detect steady contact with a monotonic rise/fall in value.
- **Ghost note**: sub‑threshold taps that light the visualizer but don’t send MIDI.
- **Scrape texture index**: count scrape grains per second and map it to CC.

If you prototype a new gesture, describe it here first. Code comes second.
141 changes: 141 additions & 0 deletions docs/Onboarding90Min.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
# 90‑Minute Onboarding Bootcamp (StringField Edition)

_A brisk, teachable ramp for newcomers: half studio notebook, half workshop script. Use it solo or read it aloud while a room builds the instrument together._

## The promise

In 90 minutes you’ll:

- **Play** a gesture (pluck/bow/scrape) and see it become motion on‑screen.
- **Understand** how sensors become a normalized signal (0..1).
- **Name** the gesture heuristics in human language, not math‑speak.
- **Adjust** at least one calibration knob on purpose.

If you finish this, you can _teach_ StringField, not just run it.

---

## 0. Pre‑flight (5 minutes)

**You need:**

- A StringField sensor stack + Teensy/ESP32 (or whatever hardware you’ve got).
- USB cable.
- PlatformIO + Processing installed.

**Open these files now:**

- `docs/FirstSession.md` (fast path ritual).
- `firmware/src/gesture_engine.cpp` (gesture logic).
- `firmware/include/gesture_engine.h` (tunable thresholds).
- `software/p5js/README.md` or `software/processing/StringFieldViz.pde` (your visualizer).

---

## 1. The 10‑minute “it works” ritual (10 minutes)

Follow `docs/FirstSession.md` end‑to‑end. Don’t skip steps. The pipeline is the instrument:

**hardware → firmware → serial → bridge → visualizer**

If you see the `PLUCK` state in the visualizer ticker, you are live.

---

## 2. Sensor signal = normalized story (15 minutes)

Open the sensor implementation you’re using:

- `firmware/src/optical_sensor.cpp`
- `firmware/src/capacitive_sensor.cpp`
- `firmware/src/makey_sensor.cpp`
- `firmware/src/time_of_flight_sensor.cpp`
- `firmware/src/piezo_sensor.cpp`
- `firmware/src/pir_sensor.cpp`
- `firmware/src/electret_mic_sensor.cpp`
- `firmware/src/i2s_mic_sensor.cpp`

**Narrate this out loud while you read:**

1. **What raw signal comes in?** (ADC, digital gate, PCM samples)
2. **How do we normalize it?** (map into 0..1)
3. **How do we smooth it?** (low‑pass, envelope, debounce)
4. **What breaks it?** (grounding, ambient light, sensor saturation)

Bonus punk‑rock move: open `docs/Sensors/README.md` and read the failure modes like a checklist. Tech is a performance—own the glitches.

---

## 3. Gesture vocabulary anatomy (15 minutes)

Open `docs/GestureVocabulary.md`. This is your teaching script.

Key files to cross‑reference:

- `firmware/include/gesture_engine.h` (the tuning knobs)
- `firmware/src/gesture_engine.cpp` (the rules)

**Do one live tweak** (even if it’s small):

- Raise `harmonic_hold_us` to make harmonics harder.
- Lower `tremolo_min_delta` to make tremolo easier.
- Tighten `scrape_window_us` to cut accidental scrapes.

Reflash, replay, and narrate what changed. That’s the whole pedagogy loop.

---

## 4. Visualize the signal (15 minutes)

Pick one:

- **p5.js debugger:** `software/p5js/sketch.js`
(Teach “data in → motion out” with big labels and velocity bars.)
- **Processing viz:** `software/processing/StringFieldViz.pde`
(Project it; the room needs to see the signal breathe.)

**Prompt:** “What does a ‘bow’ _look_ like on screen?”
Let the room answer before you explain it.

---

## 5. Hardware‑aware tuning (15 minutes)

Choose the sensor page that matches your build and follow the calibration ritual:

- `docs/Sensors/OpticalReflective.md`
- `docs/Sensors/CapacitiveSingleWire.md`
- `docs/Sensors/MakeyTouch.md`
- `docs/Sensors/TimeOfFlight.md`
- `docs/Sensors/Piezo.md`
- `docs/Sensors/PIR.md`
- `docs/Sensors/ElectretMic.md`
- `docs/Sensors/I2SMic.md`

**Goal:** adjust one knob in firmware that maps to a physical sensation.
Example: “Raise `on_thresh` until the bow only activates when you _mean_ it.”

---

## 6. Exit ticket (15 minutes)

By the end, make sure you can answer (or teach) these questions:

- What does 0..1 mean for your current sensor?
- What separates a pluck from a scrape in code?
- How does the gesture engine avoid false retriggers?
- Which knob would you tune first in a noisy room?

If you can narrate this without staring at the code, you win.

---

## Optional homework (30–60 minutes)

Pick one:

- Run the gesture tests: `pio test -d firmware -e native`
- Add a new sensor note page in `docs/Sensors/`
- Prototype a new “gesture” (e.g., glissando) by sketching rules in `docs/GestureVocabulary.md`

If you write it down, it’s teachable. If it’s teachable, it’s real.
2 changes: 2 additions & 0 deletions docs/SensingSurvey.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@
Each of these is a short, hands‑on guide with wiring notes, RC recipes, calibration rituals, and “what the gestures should feel like.”

- Start here for the index: [Sensor Field Notes](Sensors/README.md)
- [Optical Reflective IR](Sensors/OpticalReflective.md)
- [Capacitive Single‑Wire](Sensors/CapacitiveSingleWire.md)
- [MaKey‑style Touch‑to‑Ground](Sensors/MakeyTouch.md)
- [Time‑of‑Flight Proximity](Sensors/TimeOfFlight.md)
- [Piezo / Contact Mic](Sensors/Piezo.md)
Expand Down
44 changes: 44 additions & 0 deletions docs/Sensors/CapacitiveSingleWire.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Capacitive Single‑Wire (Field Notes)

_Goal: make an invisible string using a single wire + grounding ritual. This modality is sensitive and expressive, but it **demands** a good ground story._

## What it feels like

Capacitive single‑wire sensing is **ghost‑string** mode: the player doesn’t have to touch a pad; just approaching the wire changes the capacitance. It’s soft, quiet, and beautifully expressive—but it’s also moody when humidity shifts or grounding is sloppy.

---

## Wiring sketch (prototype‑level)

- **Sensor wire:** a long lead or foil strip to `A1`
- **Ground reference:** wearable bracelet or a shared foil pad to GND
- **No fancy IC required:** this is the simplest RC‑rise trick (read the pin, discharge it, charge it, read it again)

**Expected signal range:** analog 0..1023 (ADC).
**Firmware file:** `firmware/src/capacitive_sensor.cpp`

---

## Calibration ritual (4 minutes)

1. **Check the baseline:** no one touches the wire. You should see a stable, low signal.
2. **Hover and touch:** approach the wire, then touch. The delta should grow.
3. **Adjust sensitivity:** tweak `sensitivity_scale_` (in `firmware/src/capacitive_sensor.cpp`) until the envelope hits 0.7–0.9 on intentional touches but stays near 0 at rest.

---

## Narratable failure modes

- **Always maxed out:** missing or bad ground. The body is not part of the RC circuit, so the pad floats high.
- **Flatline:** wire broken or pin mis‑mapped; verify `A1`.
- **Noisy crawl:** humidity swings. Lower the baseline follower rate or re‑run calibration mid‑session.

---

## Teach the normalization

Say this:

> “We’re not measuring distance directly. We’re measuring how long a tiny RC circuit takes to charge. Then we map that to 0..1 so the gesture engine doesn’t care what the sensor is.”

If students get that, they can debug it.
45 changes: 45 additions & 0 deletions docs/Sensors/OpticalReflective.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Optical Reflective IR (Field Notes)

_Goal: a fast, cheap “string in air” that’s readable in class. This page is meant to be read out loud while you wire._

## What it feels like

Reflective IR is **bow‑friendly** and **scrape‑friendly**. It catches continuous motion better than micro‑taps, and it’s sensitive to ambient light, surface albedo, and room layout. Use it when you want the “string” to be visible and performative.

---

## Wiring sketch (prototype‑level)

- **Emitter (IR LED):** 5V → resistor → IR LED → GND
(pick a resistor that keeps the LED cool: start with 100–220Ω)
- **Receiver (phototransistor):** collector → 5V, emitter → resistor → GND
tap the emitter (or resistor midpoint) into `A0`

**Expected signal range:** analog 0..1023 (ADC).
**Firmware file:** `firmware/src/optical_sensor.cpp`

---

## Calibration ritual (3 minutes)

1. **Dark baseline:** cover the sensor, watch the serial plotter. That’s your “quiet” floor.
2. **Hand at distance:** hover at your intended playing distance. You should see the envelope rise.
3. **Thresholds:** raise `on_thresh` until light chatter stops, then lower it just enough to catch intentional contact.

---

## Narratable failure modes (aka “what’s wrong” in class)

- **Always high:** ambient light or a reflective surface is flooding the sensor. Shield it, angle it down, or move away from windows.
- **Always low:** emitter LED wired backwards or dead. Check polarity and resistor.
- **Noisy flicker:** fluorescent lighting or PWM LEDs nearby; increase smoothing or add a light shield.

---

## Teach the normalization

Explain it like this:

> “We read a raw voltage, squeeze it into 0..1, subtract a moving ambient baseline, and smooth it so the gesture engine sees _motion energy_ instead of raw flicker.”

That one sentence turns a blinking LED into a teachable instrument.
2 changes: 2 additions & 0 deletions docs/Sensors/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ If you want the fastest path: pick a sensor below, wire it, compile with the mat

## The sensor menu

- **Optical reflective IR:** [`OpticalReflective.md`](OpticalReflective.md)
- **Capacitive single‑wire:** [`CapacitiveSingleWire.md`](CapacitiveSingleWire.md)
- **MaKey‑style touch‑to‑ground:** [`MakeyTouch.md`](MakeyTouch.md)
- **Time‑of‑flight proximity:** [`TimeOfFlight.md`](TimeOfFlight.md)
- **Piezo / contact mic:** [`Piezo.md`](Piezo.md)
Expand Down
Loading