diff --git a/AGENTS.md b/AGENTS.md index 14d1006..cb4335e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -54,6 +54,13 @@ as `iterframes.iterframes`, and `iterframes/__init__.py` wraps its frame, which decoding cannot return either, so frame numbers are those of a plain `read`. A stream that cannot seek, such as raw H.264, is opened again and read from the start (`Source::open`). +- `approximate` goes with `frames` only, and carries a tolerance in frames + (`True` is `usize::MAX`), in `Selection::Frames`. `selected` then maps + each index through `Seeker::nearest_key`, which snaps it to the key frame + closest to it when that is within the tolerance, and leaves it alone + otherwise. Nothing else changes: the index pass still runs, and two + indices that snap to the same key frame decode it once each, so the caller + gets one frame per number asked for. - Errors travel through the channel and become Python exceptions in `impl From for PyErr`. A closed channel means the end of the video. - Dropping the reader closes the channel; the thread notices on its next @@ -146,6 +153,20 @@ built module; check them instead of writing them from memory. Keep the text short, in American English, with no performance claims that have not been measured. +The landing page in `web/` is part of the docs: whenever an argument, a +default, or the behavior of a feature changes, update it in the same +change as the code, and say so in the pull request. Its diagrams are +drawn from the numbers declared in `web/main.js`, so change the number +and let the drawing and the captions follow; never write a figure into +the HTML that the script also computes. `web/README.md` lists those +numbers and the files. Check the result in a browser before proposing the +change: open `web/index.html`, or serve the directory with +`python3 -m http.server --directory web`. The page has no build step and no +third-party request: its two typefaces are self-hosted woff2 files in +`web/assets/fonts/`, and everything else is plain HTML, CSS, and JavaScript. +`web/README.md` describes the look and the colour rules; keep them rather +than restyling one component on its own. + ## Landing page `web/` is the page at diff --git a/README.md b/README.md index 23bdbe3..a42de69 100644 --- a/README.md +++ b/README.md @@ -92,6 +92,17 @@ before it; frames asked for in order cost no more than reading the video straight through. See [Reading frames by number](https://iterframes.readthedocs.io/en/latest/reference/#reading-frames-by-number). +When a frame nearby will do, `approximate` reads the key frame closest to +each of `frames`, which costs one decoded frame instead of the frames from +the key frame on: + +```python +# The nearest key frame within 5 frames, else the frame itself. +frames = list(iterframes.read("video.mp4", frames=[100, 200, 300], approximate=5)) +``` + +See [Approximate frames](https://iterframes.readthedocs.io/en/latest/reference/#approximate-frames). + ## Hardware decoding Pass `device` to decode on a GPU instead of the CPU, which then stays free diff --git a/docs/index.md b/docs/index.md index 53bf412..52369f4 100644 --- a/docs/index.md +++ b/docs/index.md @@ -64,6 +64,13 @@ clip = list(iterframes.read("video.mp4", frames=[0, 30, 60])) every_fifth = iterframes.read("video.mp4", start=100, stop=200, step=5) ``` +Or, when a frame nearby will do, read the nearest key frame to each of +them and skip the decoding in between: + +```python +clip = list(iterframes.read("video.mp4", frames=[0, 30, 60], approximate=5)) +``` + Stop whenever you like; the decoder stops with the loop: ```python diff --git a/docs/reference.md b/docs/reference.md index e9a7322..3068324 100644 --- a/docs/reference.md +++ b/docs/reference.md @@ -5,7 +5,7 @@ Everything lives in the top-level `iterframes` module. ## read ```python -read(path, height=None, width=None, prefetch_frames=1, device="cpu", on_device=False, frames=None, start=0, stop=None, step=1) -> Iterator[numpy.ndarray] +read(path, height=None, width=None, prefetch_frames=1, device="cpu", on_device=False, frames=None, start=0, stop=None, step=1, approximate=None) -> Iterator[numpy.ndarray] ``` Yields the frames of the video at `path`, in order, or with `frames`, or @@ -23,6 +23,7 @@ Each frame is a C-contiguous, writable `numpy.ndarray` of shape | `on_device` | With `device="cuda"`, yield `CudaFrame` objects left on the GPU instead of arrays. See [Frames on the GPU](#frames-on-the-gpu) | | `frames` | The numbers of the frames to read, in the order to read them. See [Reading frames by number](#reading-frames-by-number) | | `start`, `stop`, `step` | The frames to read, as a slice of the video | +| `approximate` | With `frames`, read the key frame nearest to each of them instead, when it is within this many frames. `True` is any distance. See [Approximate frames](#approximate-frames) | Frames are resized with bilinear interpolation. When only one of `height` and `width` is given, the other keeps the size of the video, so the aspect @@ -48,7 +49,7 @@ for frame in iterframes.read("video.mp4", height=270, width=480): ## read_batches ```python -read_batches(path, batch_size, height=None, width=None, prefetch_frames=1, device="cpu", drop_last=False, frames=None, start=0, stop=None, step=1) -> Iterator[numpy.ndarray] +read_batches(path, batch_size, height=None, width=None, prefetch_frames=1, device="cpu", drop_last=False, frames=None, start=0, stop=None, step=1, approximate=None) -> Iterator[numpy.ndarray] ``` Yields the frames of the video in batches, in order, for models that take @@ -108,6 +109,32 @@ Frame numbers are those of [`read`](#read): frame `n` is the one `read` yields `n`-th. A number the video does not have raises `IndexError`, while a slice past the end stops at the last frame, as a list does. +## Approximate frames + +Decoding a frame in the middle of a group of pictures costs every frame +from the key frame on. `approximate` spends one decoded frame instead, by +reading the key frame nearest to each frame asked for. It goes with +`frames` only; with `start`, `stop`, and `step` it raises `ValueError`. + +```python +# The nearest key frame to each of these, however far away it is. +frames = list(iterframes.read("video.mp4", frames=[100, 200, 300], approximate=True)) + +# The nearest key frame within 5 frames, else the frame itself. +frames = list(iterframes.read("video.mp4", frames=[100, 200, 300], approximate=5)) +``` + +A number bounds the error: a frame moves by at most that many frames, and +one with no key frame that close is decoded exactly. `True` accepts any +distance, which in a video with a key frame every 10 seconds means a frame +up to 5 seconds away from the one asked for. `False`, `0`, and `None` +read every frame exactly. + +The video is still indexed, so its packets are read either way, and the +saving is in decoding alone. Two frames near the same key frame become +the same frame, which is decoded once for each of them: the iterator +yields as many frames as `frames` asks for, in the same order. + ## Errors Errors are raised by the first `next()` on the iterator, not by the call @@ -147,7 +174,7 @@ FFmpeg also spreads the decoding of each video over several threads. ## FrameReader ```python -FrameReader(path, height=None, width=None, prefetch_frames=1, device="cpu", on_device=False, batch_size=None, drop_last=False, frames=None, start=0, stop=None, step=1) +FrameReader(path, height=None, width=None, prefetch_frames=1, device="cpu", on_device=False, batch_size=None, drop_last=False, frames=None, start=0, stop=None, step=1, approximate=None) ``` The iterator behind `read` and `read_batches`. It yields `Frame` objects, diff --git a/iterframes/__init__.py b/iterframes/__init__.py index a746f5a..7e910eb 100644 --- a/iterframes/__init__.py +++ b/iterframes/__init__.py @@ -47,6 +47,7 @@ def read( start: int = 0, stop: Optional[int] = None, step: int = 1, + approximate: Union[bool, int, None] = None, ) -> Iterator[Union[np.ndarray, CudaFrame]]: """Yield the frames of the video at ``path``, in order. @@ -81,6 +82,15 @@ def read( the file first, by reading its packets without decoding them, and then decodes each frame from the key frame before it, so frames asked for in order cost no more than reading the video straight through. + + ``approximate`` trades accuracy for speed, and goes with ``frames`` + only: each frame asked for is read as the key frame nearest to it, so + it costs one decoded frame instead of the frames from the key frame + on. ``approximate=n`` moves a frame by at most ``n`` frames and reads + the rest exactly, which bounds the error; ``approximate=True`` moves + it by any distance. The video still has to be indexed, so the packets + are read either way. Two frames near the same key frame are then the + same frame, and are decoded once each. """ reader = FrameReader( path, @@ -93,6 +103,7 @@ def read( start=start, stop=stop, step=step, + approximate=approximate, ) if on_device: yield from reader @@ -114,6 +125,7 @@ def read_batches( start: int = 0, stop: Optional[int] = None, step: int = 1, + approximate: Union[bool, int, None] = None, ) -> Iterator[np.ndarray]: """Yield the frames of the video at ``path`` in batches, in order. @@ -125,7 +137,8 @@ def read_batches( Takes the same arguments as :func:`read`, except ``on_device``, so ``frames``, or ``start``, ``stop``, and ``step``, batch the frames - with those numbers instead of the whole video. The background thread + with those numbers instead of the whole video, and ``approximate`` + batches the key frames nearest to ``frames``. The background thread decodes up to ``prefetch_frames`` frames ahead, rounded up to whole batches. """ @@ -143,6 +156,7 @@ def read_batches( start=start, stop=stop, step=step, + approximate=approximate, ) # The arrays share memory with the batches, which they keep alive. for batch in reader: diff --git a/src/decoder.rs b/src/decoder.rs index ab38719..fac6bb4 100644 --- a/src/decoder.rs +++ b/src/decoder.rs @@ -38,8 +38,13 @@ pub enum Selection { /// straight through as well, and stops early. First(usize), /// These frames, in this order; a negative number counts from the end - /// of the video. - Frames(Vec), + /// of the video. With `approximate`, each of them is read as the + /// nearest key frame within that many frames of it, which costs one + /// decoded frame instead of the ones from the key frame on. + Frames { + wanted: Vec, + approximate: Option, + }, /// The frames from `start` to `stop`, every `step` of them, as a /// Python slice: negative bounds count from the end, and `stop` of /// `None` is the end of the video. @@ -67,7 +72,7 @@ impl Selection { match self { Selection::All => Ok((0..frames).collect()), Selection::First(count) => Ok((0..frames.min(*count)).collect()), - Selection::Frames(wanted) => wanted.iter().copied().map(resolve).collect(), + Selection::Frames { wanted, .. } => wanted.iter().copied().map(resolve).collect(), // Out of range bounds clamp, as a slice of a list does. Selection::Range { start, stop, step } => { let clamp = |index: isize| { @@ -250,8 +255,18 @@ fn selected( tx: &Sender, ) -> Result { let mut seeker = Seeker::new(source, input, decoder)?; + let mut indices = selection.indices(seeker.len())?; + if let Selection::Frames { + approximate: Some(tolerance), + .. + } = selection + { + for index in &mut indices { + *index = seeker.nearest_key(*index, tolerance); + } + } let mut frame = Frame::new(); - for index in selection.indices(seeker.len())? { + for index in indices { seeker.frame(index, &mut frame)?; if !converter.send(&mut frame, tx)? { return Ok(false); @@ -518,6 +533,25 @@ impl Seeker<'_> { } } + /// The key frame nearest to `index`, when no more than `tolerance` + /// frames away from it, else `index` itself. A tie goes to the key + /// frame before, which decoding is more likely to reach by going on. + fn nearest_key(&self, index: usize, tolerance: usize) -> usize { + let before = self.key_frame_before(index); + let nearest = match self + .keys + .get(self.keys.partition_point(|key| *key <= index)) + { + Some(after) if after - index < index - before => *after, + _ => before, + }; + if nearest.abs_diff(index) <= tolerance { + nearest + } else { + index + } + } + /// The last key frame at or before `index`. The numbers rise, so a /// long video costs a binary search rather than a scan. fn key_frame_before(&self, index: usize) -> usize { diff --git a/src/lib.rs b/src/lib.rs index 512df77..0892d6f 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -332,7 +332,8 @@ impl Plane { /// `batch_size`, a `Batch` of that many frames (fewer in the last one, /// unless `drop_last`), whose `prefetch_frames` rounds up to whole batches. /// `frames`, or `start`, `stop`, and `step`, pick the frames to decode by -/// number instead of reading the whole video. Use `iterframes.read` and +/// number instead of reading the whole video, and `approximate` reads each +/// of `frames` as the key frame nearest to it. Use `iterframes.read` and /// `iterframes.read_batches`, which wrap them in NumPy arrays. #[pyclass(module = "iterframes")] struct FrameReader { @@ -344,7 +345,8 @@ impl FrameReader { #[new] #[pyo3(signature = ( path, height=None, width=None, prefetch_frames=1, device="cpu", on_device=false, - batch_size=None, drop_last=false, frames=None, start=0, stop=None, step=1 + batch_size=None, drop_last=false, frames=None, start=0, stop=None, step=1, + approximate=None ))] #[allow(clippy::too_many_arguments)] fn new( @@ -360,8 +362,9 @@ impl FrameReader { start: isize, stop: Option, step: isize, + approximate: Option<&Bound<'_, PyAny>>, ) -> PyResult { - let selection = selection(frames, start, stop, step)?; + let selection = selection(frames, start, stop, step, approximate)?; let batching = match batch_size { None => None, Some(0) => return Err(PyValueError::new_err("batch_size must be at least 1")), @@ -432,19 +435,28 @@ impl FrameReader { /// Which frames to decode, from the arguments Python passes: `frames` /// lists them, while `start`, `stop`, and `step` slice them as a list is -/// sliced. The two ways cannot be mixed. +/// sliced. The two ways cannot be mixed, and only `frames` can be +/// approximate. fn selection( frames: Option>, start: isize, stop: Option, step: isize, + approximate: Option<&Bound<'_, PyAny>>, ) -> PyResult { let sliced = start != 0 || stop.is_some() || step != 1; + let approximate = tolerance(approximate)?; match frames { Some(_) if sliced => Err(PyValueError::new_err( "frames does not go with start, stop, or step", )), - Some(frames) => Ok(Selection::Frames(frames)), + Some(wanted) => Ok(Selection::Frames { + wanted, + approximate, + }), + None if approximate.is_some() => Err(PyValueError::new_err( + "approximate needs frames: a slice cannot be approximate", + )), None if step < 1 => Err(PyValueError::new_err("step must be at least 1")), // Frames counted from the first are read straight through, with // no index and no seeking. @@ -461,6 +473,23 @@ fn selection( } } +/// How far a frame may move to the nearest key frame, in frames, from +/// what Python passes as `approximate`: `True` is any distance, a number +/// is at most that many frames, and `False` or `None` is no move at all. +fn tolerance(approximate: Option<&Bound<'_, PyAny>>) -> PyResult> { + let Some(approximate) = approximate else { + return Ok(None); + }; + // A bool is an int in Python, so it has to be read first. + if let Ok(any_distance) = approximate.extract::() { + return Ok(any_distance.then_some(usize::MAX)); + } + let frames = approximate.extract::().map_err(|_| { + PyValueError::new_err("approximate must be True, False, or a number of frames") + })?; + Ok(Some(frames)) +} + /// The hardware devices of this build, by the names PyTorch gives them, /// which Python users know better than FFmpeg's. fn hardware_devices() -> impl Iterator { diff --git a/tests/test_benchmark.py b/tests/test_benchmark.py index 773e172..1917397 100644 --- a/tests/test_benchmark.py +++ b/tests/test_benchmark.py @@ -108,3 +108,29 @@ def decode(): whole, second_half = read(), read(start=450) print(f"\nwhole {whole:.3f}s, start=450 {second_half:.3f}s") assert second_half < 0.75 * whole + + +def test_approximate_frames_decode_less(video_path): + """Approximate frames must cost one decoded frame each. + + Reading frames in the middle of their group of pictures decodes every + frame from the key frame on, while the nearest key frames decode one + frame each. Both pay for the pass that indexes the file. + """ + with av.open(str(video_path)) as container: + frames = list(container.decode(video=0)) + keys = [index for index, frame in enumerate(frames) if frame.key_frame] + # Frames far enough into their group of pictures to cost several + # decoded frames each, which approximate reading skips. + wanted = [key + 20 for key in keys if key + 20 < len(frames)][:20] + + def read(**arguments): + def decode(): + for _ in iterframes.read(video_path, frames=wanted, **arguments): + pass + + return best_of(decode) + + exact, approximate = read(), read(approximate=True) + print(f"\nexact {exact:.3f}s, approximate {approximate:.3f}s") + assert approximate < exact diff --git a/tests/test_select.py b/tests/test_select.py index 2fc83cb..934116b 100644 --- a/tests/test_select.py +++ b/tests/test_select.py @@ -83,6 +83,107 @@ def test_frames_match_reading_in_order(video_path, frames_of_the_video): np.testing.assert_array_equal(frame, frames_of_the_video[index]) +def key_frames(path): + """The numbers of the key frames of a video, in order.""" + with av.open(str(path)) as container: + return [ + index + for index, frame in enumerate(container.decode(video=0)) + if frame.key_frame + ] + + +def test_approximate_frames_are_key_frames(video_path, frames_of_the_video): + wanted = [5, 100, 457, 900] + keys = key_frames(video_path) + + frames = list(iterframes.read(video_path, frames=wanted, approximate=True)) + + assert len(frames) == len(wanted) + for frame, index in zip(frames, wanted): + nearest = min(keys, key=lambda key: (abs(key - index), key)) + np.testing.assert_array_equal(frame, frames_of_the_video[nearest]) + + +def test_approximate_tolerance_keeps_far_frames_exact( + video_path, frames_of_the_video +): + keys = key_frames(video_path) + # A frame in the middle of a group of pictures, with no key frame + # within two frames of it. + far = next( + index + for index in range(901) + if all(abs(key - index) > 2 for key in keys) + ) + + frames = list( + iterframes.read(video_path, frames=[far, keys[1] + 1], approximate=2) + ) + + np.testing.assert_array_equal(frames[0], frames_of_the_video[far]) + np.testing.assert_array_equal(frames[1], frames_of_the_video[keys[1]]) + + +def test_approximate_key_frames_are_themselves( + video_path, frames_of_the_video +): + keys = key_frames(video_path)[:3] + + frames = list(iterframes.read(video_path, frames=keys, approximate=True)) + + for frame, index in zip(frames, keys): + np.testing.assert_array_equal(frame, frames_of_the_video[index]) + + +def test_approximate_repeats_the_same_key_frame(video_path): + keys = key_frames(video_path) + wanted = [keys[1] - 1, keys[1], keys[1] + 1] + + frames = list(iterframes.read(video_path, frames=wanted, approximate=True)) + + assert len(frames) == 3 + np.testing.assert_array_equal(frames[0], frames[1]) + np.testing.assert_array_equal(frames[1], frames[2]) + + +def test_approximate_off_reads_exactly(video_path, frames_of_the_video): + wanted = [7, 123] + + for approximate in [None, False, 0]: + frames = list( + iterframes.read(video_path, frames=wanted, approximate=approximate) + ) + + for frame, index in zip(frames, wanted): + np.testing.assert_array_equal(frame, frames_of_the_video[index]) + + +def test_approximate_batches(video_path, frames_of_the_video): + keys = key_frames(video_path) + wanted = [keys[1] + 1, keys[2] + 1, keys[3] + 1] + + batches = list( + iterframes.read_batches(video_path, 2, frames=wanted, approximate=True) + ) + + stacked = np.concatenate(batches) + assert len(stacked) == 3 + for frame, index in zip(stacked, keys[1:]): + np.testing.assert_array_equal(frame, frames_of_the_video[index]) + + +def test_approximate_needs_frames(video_path): + with pytest.raises(ValueError, match="approximate needs frames"): + list(iterframes.read(video_path, start=10, approximate=True)) + + +@pytest.mark.parametrize("approximate", [-1, "yes", 1.5]) +def test_approximate_must_be_a_bool_or_a_count(video_path, approximate): + with pytest.raises(ValueError, match="approximate must be"): + list(iterframes.read(video_path, frames=[1], approximate=approximate)) + + def test_frames_match_pyav(video_path, decode_with_pyav, assert_close): expected = decode_with_pyav(video_path) diff --git a/web/README.md b/web/README.md index efd547b..18242bc 100644 --- a/web/README.md +++ b/web/README.md @@ -17,18 +17,46 @@ something inside it changes. `ci.yaml` ignores it for the same reason. | `main.js` | The two animated diagrams and the copy buttons | | `assets/frames.webp` | Sprite sheet, 12 frames of 160x90 in a row | | `assets/scenes.webp` | Sprite sheet, frames 934, 4522 and 11711 at 240x135 | -| `assets/og.png` | Social card, 1200x630 | +| `assets/og.png` | Social card, 1200x630, rendered from `og-card.html` | +| `og-card.html` | Source of the social card; nothing links to it | | `assets/favicon.svg` | Favicon | Animations are built from numbers declared in `main.js` (`FRAMES`, `DECODE`, -`WORK`, `TOTAL_FRAMES`, `GOP`, `WANTED`), and the captions are computed from -the same numbers. Change a number and the drawing and the text move together. +`WORK`, `TOTAL_FRAMES`, `GOP`, `WANTED`, `APPROXIMATE`), and the captions are +computed from the same numbers. + +The random access demo runs in two modes, picked with the buttons above it: +`Exact` decodes from the key frame in front of each number, and +`approximate=45` reads the nearest key frame within `APPROXIMATE` frames. +`seekPlan` builds one plan per mode, and the steps, the counters, the call +under the figure and the cost line all come from the plan of the mode showing. +Both modes reuse the same three tiles of `assets/scenes.webp`, which are the +frames asked for, not the key frames that stand in for them. Change a number and the drawing and the text move together. The hero plays both timelines on one clock, so the loop that decodes inline is still running when the one built on iterframes has finished. -Colours mean the same thing in every diagram: yellow is frames and data, cyan -is the decoder thread, magenta is the GPU. +## Look + +Warm paper, one heavy sans, and objects that sit on the page: cards carry a +dashed inner edge, buttons press down when clicked, code sits in a recessed +slab, and the frame strip in the random access demo has sprocket holes. + +Colours mean the same thing in every diagram: amber is frames and data, cyan +is the decoder thread, magenta is the GPU. Each accent is declared once in +`styles.css` as an `--x-rgb` triple dark enough to read as text on the paper; +every wash in a diagram is that same triple with an alpha, so an accent is +changed in one place. Two shadows do the depth: `--lift` for what sits on the +page, `--well` for what is cut into it. Nothing else casts a shadow. + +Two typefaces, both self-hosted in `assets/fonts/` as latin-only woff2, under +the SIL Open Font License: Plus Jakarta Sans sets the text and the headlines, +JetBrains Mono the code, the numbers and the labels. No web font is fetched +from anywhere, so the page still makes no third-party request. Replacing one +means dropping in the woff2 and changing its `@font-face` and `--sans` or +`--mono`. + + ## Regenerating the images @@ -58,8 +86,21 @@ The sprite sheets are read with `background-position` in percentages, so the tiles must stay the same size and in the same order. A sheet of *n* tiles is addressed as `calc(var(--i) * 100% / (n - 1))`. -`assets/og.png` is composed with `ffmpeg` too; the command is in the commit -that added it. +`assets/og.png` is the page's own hero, laid out in `og-card.html` against +`styles.css`, so the card carries the same paper, type and accent as the site. +Render it after changing either: + +```bash +python3 -m http.server 8000 --directory . & +chrome --headless=new --hide-scrollbars --window-size=1200,630 \ + --screenshot=assets/og.png --virtual-time-budget=6000 \ + http://localhost:8000/og-card.html +``` + +`chrome` is the Chrome binary; on macOS it is +`/Applications/Google Chrome.app/Contents/MacOS/Google Chrome`. Check the +result at 1200x630 and that the text survives the crop social networks +apply. ## Analytics diff --git a/web/assets/favicon.svg b/web/assets/favicon.svg index 7405c1a..2e75f4e 100644 --- a/web/assets/favicon.svg +++ b/web/assets/favicon.svg @@ -1,5 +1,5 @@ - - - + + + diff --git a/web/assets/fonts/jakarta-latin.woff2 b/web/assets/fonts/jakarta-latin.woff2 new file mode 100644 index 0000000..5ec77ed Binary files /dev/null and b/web/assets/fonts/jakarta-latin.woff2 differ diff --git a/web/assets/fonts/jetbrains-mono-latin.woff2 b/web/assets/fonts/jetbrains-mono-latin.woff2 new file mode 100644 index 0000000..2ca6ac6 Binary files /dev/null and b/web/assets/fonts/jetbrains-mono-latin.woff2 differ diff --git a/web/assets/og.png b/web/assets/og.png index 954dc61..e480e20 100644 Binary files a/web/assets/og.png and b/web/assets/og.png differ diff --git a/web/index.html b/web/index.html index 04dbbc5..568336f 100644 --- a/web/index.html +++ b/web/index.html @@ -3,7 +3,7 @@ - + iterframes: video frames as NumPy arrays, decoded ahead of your loop @@ -15,6 +15,8 @@ + +