AIToday
RoboticsOpen-Source AIHugging Face BlogPublished: Aug 14, 2026, 04:01 JST5 min read

Strands Robots streams training data from buckets without download

Strands Robots streams training data from buckets without download

Key takeaway

  • Hugging Face published a technical walkthrough showing how to run a complete robot data-collection-to-deployment loop in one place: record demonstrations into a Storage Bucket, stream the growing dataset back for training without a full local download, and deploy the trained policy to hardware—all without leaving the LeRobot format or the Hub's namespace.

  • The bucket backs the data loop with byte-level deduplication, cutting repeated upload costs by roughly four times when the same scenes are re-recorded.

3 Key Points

  1. What happened

    Hugging Face published a walkthrough showing how to record robot demonstrations into a Storage Bucket, train a policy by streaming the dataset from the Hub without downloading it locally, and deploy the trained checkpoint back to hardware—all within a single agent loop using the same LeRobot format throughout.

  2. Why it matters

    The continuous data loop (record → train → deploy) normally incurs repeated byte transfers: each training run copies the whole dataset to GPUs, and new checkpoints ship out while incoming recordings arrive. Buckets backed by Xet use byte-level deduplication with content-defined chunking, reducing data transferred per upload by about four times across the Hub, so teams can run daily collection and training cycles without paying for the same bytes repeatedly.

  3. What to watch

    The runnable code lives in examples/notebooks/05_streaming_data_loop.ipynb. LeRobot's dataset format is already used by over 90,000 datasets and models on the Hub from more than 8,000 publishers, so any tool built to read LeRobot data can read Strands Robots recordings without conversion. Buckets were announced in March 2026 and sit in the same hf:// namespace as dataset repositories.

In Depth

Read the full story

The post walks through a four-stage data loop: record a demonstration into a bucket, store it with byte-level deduplication, train by streaming from the Hub, and deploy the policy back to hardware. Each stage shares a single backend—the Robot() factory that records datasets, syncs them into buckets, streams them back for training, and deploys to physical hardware with a one-keyword-argument change.

In Step 1, the agent records a LeRobotDataset through a DatasetRecorder in the same format LeRobot writes on actual hardware. An example prompt creates a world with the SO-100 robot, adds a red cube and front camera, starts recording with repo_id='local/cube_pick' and fps=30, runs the mock policy for 60 steps, then stops. The sync_dataset_to_bucket() call then moves the finished on-disk dataset into a Storage Bucket at hf://buckets/{bucket}/{run_id}. For hardware recording on an SO-101, LeRobot's lerobot-record CLI handles leader-follower bring-up and writes to disk in the same format, so the same sync call moves it to a bucket: sync_dataset_to_bucket("./recordings", "my-org/robot-fave", run_id="run-021").

Step 2 explains the deduplication savings. Fixed cameras on an arm clearing a table for eight hours record mostly repeated pixels—the same lighting, chassis, background. On versioned repositories, changing one frame in a multi-gigabyte video shard re-uploads the whole file. Buckets are backed by Xet, which uses content-defined chunking: chunk boundaries follow content, so inserting bytes changes only the chunk they land in. Hugging Face's measurements show roughly 4× reduction in bytes transferred per upload across the Hub. A concrete benchmark: starting from 500 MB, changing 1% of bytes moved only 5.5 MB on re-upload; changing 5% moved 27.5 MB; changing 10% moved 55 MB. Without chunk-level deduplication, overwriting an object requires sending all its bytes regardless of what changed.

Step 3 streams the dataset back for training. The same Robot() reads it from the bucket frame by frame, decoding camera video on the fly with no local copy. The whole loop runs in a handful of lines: sim.stream_dataset("my-org/robot-fave/cube_pick", repo_type="bucket").dataloader(batch_size=64) yields batches without downloading the full dataset first. Step 4 deploys the trained checkpoint to the same Robot with one keyword argument change (mode="real" for hardware). Demonstrations recorded on hardware return to the same bucket, closing the loop.

The post notes that LeRobot's format is already used by over 90,000 datasets and models on the Hub from more than 8,000 publishers (LeRobot Project Pulse), so Strands Robots recordings integrate without conversion. Buckets were announced in March 2026 and live in the same hf:// namespace as dataset repositories, using the same hf CLI and permissions users already have. For running on hardware, an SO-101 follower-leader pair or any LeRobot-supported robot works with calibration files under ~/.cache/huggingface/lerobot/calibration/. The minimal path to run the loop on a laptop requires Python 3.12+ on Linux or macOS, a Strands-compatible model provider (Amazon Bedrock, Anthropic API, OpenAI, or local Ollama), and strands-robots[sim-mujoco,lerobot]>=0.5.1; the lerobot extra pulls LeRobot (>=0.6.1), datasets, av, and torchcodec so recording and video decoding work without further setup.

Context & Analysis

The post addresses a concrete cost problem in robotics data pipelines: a continuous loop that records, trains, and deploys incurs multiplicative byte transfers. Each day's recordings are uploaded into versioned repositories; each training run copies the full dataset to GPUs; each new checkpoint ships back out while the next batch of recordings arrives. On a versioned repository, even small edits to large video shards force re-upload of the entire file. Buckets, backed by Xet's content-defined chunking, flip this: chunk boundaries follow content, so inserting a few bytes changes only the chunk containing those bytes, not every boundary after it. Hugging Face's own measurements show roughly 4× reduction in bytes transferred per upload across the Hub; on Enterprise plans, billing tracks deduplicated footprint, not raw storage.

Strands Robots—an open-source SDK from AWS—exposes robot abstractions, simulation, and the LeRobot stack as composable agent tools. The agent loop itself is the decision-maker: it chooses which episodes to keep, when to re-record, whether today's batch suffices for training, and which checkpoint deploys to hardware. The walkthrough shows this loop in code: one Robot() object records a LeRobotDataset and streams it back in the same process, eliminating the need for separate download steps. LeRobot's format is already in use across over 90,000 datasets and models from more than 8,000 publishers on the Hub, so any downstream tool built to read LeRobot data reads Strands recordings without conversion.

FAQ

What is a Storage Bucket on Hugging Face?
A Storage Bucket is a mutable, non-versioned, Xet-backed object-storage repository type announced in March 2026 that sits beside dataset repositories in the same hf:// namespace and uses the hf CLI. It serves as a working layer to hold data between recording and training, with byte-level deduplication to reduce repeated upload costs.
How much does deduplication save on uploads?
Content-defined chunking reduces data transferred per upload by about four times across the Hub. In benchmark tests starting from a 500 MB upload, changing 1% of the bytes and re-uploading moved only 5.5 MB, changing 5% moved 27.5 MB, and changing 10% moved 55 MB—versus re-uploading all bytes without deduplication.
Can I run this on a laptop?
Yes. The minimal path requires Python 3.12+ on Linux or macOS (Apple Silicon supported), a Strands-compatible model provider (Amazon Bedrock, Anthropic API, OpenAI, or local Ollama), and strands-robots[sim-mujoco,lerobot]>=0.5.1; every stage runs on a laptop with these three, though the default uses a mock policy rather than a trained one.
Hugging Face BlogRead Original Article

Get the latest Robotics news every morning

AI-summarized, only the topics you pick — one digest a day via Email, Slack, or Discord.

Free · takes 30 seconds · unsubscribe anytime

Ask AI

Ask AI anything about this article. Q&As are published on this page for other readers too.

Related Articles

Next articleGoogle Sheets canvas transforms data into interactive dashboards with AI prompts

The AI news that matters, in one minute each morning.

Sign up free