Home
The Musical Gestures Toolbox for Python is a collection of high-level modules targeted at researchers working with video recordings. It includes visualisation techniques such as motion videos, motion history images, and motiongrams; techniques that, in different ways, allow for looking at video recordings from different temporal and spatial perspectives. It also includes basic computer vision analysis, such as extracting the quantity and centroid of motion, and using such features in analysis.
The toolbox targets music-related body motion (of musicians, dancers, and perceivers) but is equally helpful for other disciplines working with video recordings of humans, such as linguistics, pedagogy, psychology, and medicine.
The wiki reads in order: each chapter opens with the idea in plain words and then deepens, so a dance student and a signal-processing researcher can start at the same page and stop at different depths. Begin at Installation and Video Basics; the later chapters assume the earlier ones. The newest chapter, Eye Tracking, Events and the Canvas, puts a wearer's eye tracker on the video's clock, tests events against events, and measures a painting as a time series.
The Musical Gestures Toolbox contains functions to analyse and visualise video, audio, and motion capture data. There are three categories of functions:
- Preprocessing (trimming, cropping, colour adjustments, etc.)
- Visualisation (video playback, image display, plotting)
- Processing (videograms, average images, motion images, etc.)
- Quick Start Tutorial
- Jupyter Notebook–can also be run in Colab
- Examples Overview
- Complete Documentation Site
What changed in each release is kept in one place, the release notes and the changelog, rather than repeated here.
The chapters read in order: basics first, then looking, then measuring and segmenting, then annotating, and the special formats last. Later chapters assume the earlier ones. Concepts and quick reference live on the documentation site; this wiki is where the toolbox is taught.
- Installation - Setup instructions for all platforms
- Video Basics - Understanding video processing concepts
- Loading Videos - How to load and display videos
- Preprocessing - Video preprocessing techniques
- Video Analysis - Motion analysis and visualisation
- Filtering Effects - Understanding filter parameters
- Output Management - Working with results
- Function Chaining - Combining multiple operations
- File Naming - Output file conventions
- Audio Analysis - Audio processing functions
-
Pose and Motion Capture -
MgVideo.pose()skeleton rendering; landmark-trajectory extraction; pose-derived limb speed/impacts; QTM mocap I/O - Pulse, Cycles and Alignment - Canonical peak-picker; stroke/cycle segmentation and accelerando fitting; lead/lag estimation, event matching, cross-source agreement
- Segmentation and Annotation - Turning a long recording into material a person can annotate
- Motion, Audio and Posturography Toolkit - Band-limited QoM cores for position/pose/accelerometer data; scipy-only audio RMS/flux onsets, T60 decay, attack spectral centroid; sway metrics, complexity/dynamics measures, respiration rate — signal-level numpy/scipy functions from the ro, still standing, Westney-comparisons and cymbal-comparison studies (Jensenius)
- Working with ambiscape - Joint audio–video sessions: shared clock, device sync, 360-camera workflow
- 360 Video Anglegrams and AEM - Directional motion analysis of 360 video, audio-energy overlays, per-direction views
The speed and efficiency of the MGT are made possible by the excellent FFmpeg project. Many of the toolbox functions are Python wrappers/bindings on FFmpeg commands called in a subprocess.
Please help improve the toolbox by adding bugs and feature requests in the issues section.
Separate packages with separate release cycles, built to be used together and sharing several implementations, so the same measure computed in two of them agrees.
| Toolbox | What it is for | Docs |
|---|---|---|
| ambiscape | soundscapes: the sonic ambience of a place | wiki |
| musiscape | music collections: comparing many tracks and albums | wiki |
| micromotion | human micromotion: quantity of motion from markers, accelerometers, belts and force plates | wiki |
The package is archived on Zenodo. Cite the CONCEPT DOI, which always resolves to the newest version: 10.5281/zenodo.21949007.
Where the exact behaviour matters, name the version you ran as well; the
version-specific DOIs are listed on the Zenodo record. CITATION.cff in the
repository carries the same information in machine-readable form.