TenzorPipe — zero-VRAM media ingestion engine

See it side by side

Narrated, 25 seconds. Every figure on screen was measured on the machine that produced the video.

One decode, then pure memory reads

Conventional loaders re-decode every clip on every epoch. TenzorPipe decodes once into a contiguous Arrow artifact; each epoch after that is a memory map away.

Measured, not marketed

One 20-second 1080p30 H.264 clip at 8 Mb/s, Intel i5-14400F + RTX 5060 (8 GB) under WSL2, 50 measured iterations per contender. Full method, raw numbers and caveats: BENCHMARKS.md.

Metric	TenzorPipe 0.3.0	FFmpeg pipe	NVIDIA DALI	TorchCodec (CUDA)
First-pass ingest	646 ms	554 ms	442 ms	276 ms
Every later training epoch	2.8 ms Arrow re-read	decode again	decode again	decode again
GPU memory during decode	0 MiB	0 MiB	822 MiB	650 MiB
Output reproducibility	1,056 / 1,056 byte-identical	differ from each other by colour conversion and resampling

TenzorPipe is not the fastest first pass — GPU decoders are. It wins from the second epoch onward, and it never takes VRAM away from your model.

Target architectures & concrete use cases

Where a zero-VRAM, bit-exact, decode-once pipeline changes the engineering calculus.

Edge robotics & autonomous perception

Jetson · drones · humanoids

Eliminates VRAM decode contention on shared and unified-memory architectures, where every decoder byte is a byte the model cannot have.
Streams contiguous Apache Arrow frames straight into spatial neural networks with 0 MiB GPU overhead, conserving battery and compute bandwidth.

Multimodal LLM & VLM training

video-language models · multi-GPU clusters

Ingests training video once into the structured .tenzor format, with video and Log-Mel audio locked to the same 0.5 s epoch grid.
Re-reads tensors at sub-3 ms multi-epoch speed, keeping clusters saturated instead of repeatedly decompressing the same MP4s.

Air-gapped & sovereign pipelines

defense · medical · regulated data

Full local execution: zero cloud APIs, zero network calls, and no temporary frame dumps to disk — a libc-level write audit observed only the final artifact.
Deterministic, bit-exact ingestion suited to HIPAA and air-gapped environments, verified across 1,056 regression cases.

High-throughput local indexing & search

journaling · media ops · embeddings

Removes the disk thrashing and process-spawn latency of shelling out to FFmpeg across thousands of local clips.
Pipes contiguous frame batches straight into local vector embedding models — 16 clips in 0.41 s versus 1.95 s one at a time.

Three lines

# 1. install (Linux x86-64, CPython 3.9+)
pip install tenzorpipe

# 2. decode once into Apache Arrow
python -c "import tenzorpipe as tp; tp.ingest('clip.mp4', 'clip.tenzor')"

# 3. memory-map tensors for every epoch
python -c "import tenzorpipe as tp; d = tp.load('clip.tenzor'); print(len(d), d[0]['video'].shape)"

Tensor contract

224×224 RGB CHW float32 in [-1, 1] and 64-band Log-Mel, one row per 0.5 s epoch.

No runtime deps

OpenH264 and the AAC decoder are compiled in. No FFmpeg, no CUDA, no system codecs.

Bit-exact

Same bytes on every worker count, verified against the previous release on 1,056 cases.

Commercial & enterprise licensing

TenzorPipe is released under the Business Source License 1.1. It converts to Apache-2.0 on 16 September 2030.

Free for production

Individuals and entities under US$100,000 annual gross revenue
Revenue counted across parents, subsidiaries and affiliates under common control
Non-production use — testing, evaluation, academic research — always unrestricted

Commercial license required

Organizations at or above US$100,000 annual gross revenue
Embedded, bundled or OEM distribution on hardware devices
Terms negotiated directly with the licensor

licensing@tenzorpipe.org