Bimanual Coordination Benchmarks for Humanoid Arms

Existing benchmarks for bimanual robot arms measure the wrong things, or measure the right things too coarsely to matter. That is the central problem this piece is about, and it has direct consequences for anyone building or evaluating humanoid arm policies right now. The gap is not a matter of degree.

Why existing benchmarks systematically underserve bimanual coordination

Dual-arm tabletop systems and humanoid robots appear in more labs and more deployments, and that trend alone should have pulled benchmark design forward with it. It hasn't Bi-DexHands CoorDex. The DuoBench team reports that development effort has gone overwhelmingly into single-arm settings, and the result is a body of evaluation tools built for a problem that most of the field has already moved past Bi-DexHands CoorDex.

The design space for two-arm coordination splits into two coordination types, and this distinction affects how well a benchmark can predict real-world bimanual failure modes. Loose coupling means the arms share a workspace or a goal but can largely succeed on their own, each running its own subroutine without tracking the other moment to moment. Tight coupling means something closer to a duet: one arm's motion constrains the other's in real time, and a spatial or timing error on one side propagates immediately to the other. BiCoord's authors argue that the core failure of older benchmarks is that their tasks are short-horizon and only loosely coordinated. They never actually test the spatial-temporal coupling that real bimanual behavior depends on Bi-DexHands CoorDex.

Binary success rates make this worse, not better. For a long-horizon task built out of several sub-goals, a pass/fail outcome tells you almost nothing about where things went wrong. That convergence matters. When two separate research groups, working from different platforms and different task sets, land on the same complaint about coarse feedback, it stops looking like a stylistic preference and starts looking like a real methodological hole.

The problem takes on a distinct shape once multimodal large language models enter the picture as bimanual controllers. ST-BiBench's authors identify the primary hurdle there as maintaining role consistency across multiple action streams, and they name spatial hallucination and inconsistent role assignment over long horizons as failure modes that simply do not show up in single-arm evaluation Bi-DexHands CoorDex. A model can reason perfectly well about what "the left arm" should do in isolation and still lose track of which arm is which once the task runs long enough.

The two dimensions that expose benchmark adequacy: coordination tightness and task horizon

Two axes do most of the work in explaining why a benchmark is adequate or not: coordination tightness and task horizon. Coordination tightness runs from loose (parallel object placement, where each arm proceeds independently) to tight (stabilize-and-operate tasks, or throw-and-catch, where a continuous dependency links the two arms at every instant). Real bimanual behavior, the kind found in cooking, assembly, or tool use, sits overwhelmingly at the tight end of that spectrum, which is exactly the end most older benchmarks avoid.

Task horizon is the second axis, and it interacts with tightness without being reducible to it. Short-horizon tasks involve a single sub-goal, so a policy's errors stay isolated and easy to attribute. Long-horizon tasks chain multiple sub-goals together with dynamic role exchange between the arms, and errors compound across those stages. Role consistency failures, the kind ST-BiBench flags in MLLM-based agents, simply do not appear at short horizons Bi-DexHands CoorDex. They need scale to surface.

It gives the field something it lacked before, a number that places any given benchmark on the tightness axis rather than leaving that judgment to intuition. The payoff is concrete. BiCoord scores an STI of 42.16, a wide margin over 11.20 for RoboTwin 2.0 and 8.13 for RLBench2. That is not a marginal gap. It says the prior generation of benchmarks clusters in a narrow, low-coupling band, and that a benchmark can look diverse in task count while still testing almost nothing about how tightly two arms actually have to work together.

None of this makes the two axes fully independent of each other. Tight-coupling tasks do tend to run longer, since continuous inter-arm dependency and multi-stage structure often arrive together in the same task design. But treating tightness and horizon as separable variables, rather than one proxy for the other, is what lets a benchmark designer diagnose failure with any precision. Without decomposing a task into its semantic stages, there is no way to tell whether a policy broke down because the coupling got too tight or because the sequence simply ran too long. The BiCoord STI (Spatial-Temporal Integral) metric characterizes the temporal and spatial coupling of the dual arms and offers a quantitative way to place a benchmark on the tightness axis, which can be used here to show where existing benchmarks cluster.

What earlier benchmarks left unmeasured

RLBench2 offers a useful starting point precisely because it gets partial credit. DuoBench's review of prior work finds that its 13 tasks, run on two table-mounted Franka arms facing each other, mostly do test tightly coupled motions. The trouble is coverage. Thirteen tasks cannot represent the range of coordination patterns that real bimanual work throws up, so the benchmark ends up testing tight coupling narrowly rather than broadly.

PerAct2, from Grotz and colleagues, matters mostly as a historical marker. Presented at a CoRL 2024 workshop under the explicit title "Benchmarking and Learning for Robotic Bimanual Manipulation Tasks," it was an early formal attempt to treat bimanual manipulation as its own benchmarking problem, but it predates the metric vocabulary, STI among them, that later work would use to make the coupling gap legible RoboTwin.

Bi-DexHands, published in IEEE Transactions on Pattern Analysis and Machine Intelligence, took the field in a related but distinct direction, building 20 tasks around the Fine Motor Subtest. It targets dexterous hand coordination rather than the gross arm coordination that concerns BiCoord and DuoBench: a benchmark can be excellent at what it measures and still be measuring a different problem than the one a humanoid arm developer needs solved.

BiGym, presented at CoRL 2025, added mobility to the bimanual picture, framing itself as a demo-driven mobile bi-manual manipulation benchmark. It is a meaningful extension of scope, but it did not advance coordination metrics in the way BiCoord later would Bi-DexHands CoorDex. HumanoidBench, from Sferrazza, Huang, Lin, Lee, and Abbeel at RSS 2024, sits adjacent to this whole conversation rather than inside it: it integrates full-body locomotion with manipulation, which makes it relevant to humanoid robotics generally, but it is not an arm-coordination benchmark in the specific sense this piece is tracking.

The original RoboTwin, a CVPR 2025 Highlight paper, deserves particular attention because it built the simulation infrastructure that the current generation would later extend. Its 16 tasks across 6 basic task categories, with object-type and episode-length ablations built in, represented serious engineering work RoboTwin. It was designed before the tightness and horizon gap had been formally named, though, so it inherited the same blind spot as everything that came before it. None of this reflects poor design choices by the original teams. The benchmarks were built for the hardware and deployment reality of their moment, and the gap opened up because hardware capability, and the ambition of what people wanted two arms to do together, moved faster than the benchmarks did.

RoboTwin 2.0: scale and simulation infrastructure as the platform layer

RoboTwin 2.0, accepted at ICML 2026, is a large-scale simulation benchmark and dataset for bimanual robotic manipulation that provides both an automated data generator and unified evaluation protocols. It supplies a large-scale simulation environment and dataset for bimanual manipulation, along with an automated data generator and a unified evaluation protocol, and the scale numbers back up that platform framing. Fifty dual-arm tasks span five robot embodiments RoboTwin 2.0. The object library behind it, called RoboTwin-OD, holds 731 object instances across 147 categories RoboTwin 2.0 top3dshop.com. Over 100,000 pre-collected expert trajectories sit on top of that, giving downstream researchers a dataset large enough to train on rather than just evaluate against RoboTwin 2.0.

The generalization testing built into RoboTwin 2.0 is where its engineering shows. Domain randomization runs along five structured axes, clutter, lighting, background, tabletop height, and language, specifically to stress test generalization rather than just measure in-distribution performance. Evaluation splits then run Easy against Hard, and Seen against Unseen language instructions, giving two separate axes along which a policy's generalization can be measured rather than collapsing everything into one aggregate score. The data pipeline uses multimodal language models together with simulation-in-the-loop refinement to generate task-level execution code automatically, cutting out much of the manual curation that used to bottleneck benchmark construction. The Aloha-AgileX robot appears as an evaluation embodiment in at least one paper built on this platform.

For all that scale, RoboTwin 2.0 does not close the tightness gap. Its STI of 11.20 places it in the low/restricted end of the coupling spectrum. It was built to be broad and diverse infrastructure, not a tight-coupling mandate. Both BiCoord and DuoBench build on or respond to it directly, which tells you something about its role in the field. RoboTwin 2.0 supplied the tools, the object library, the data generation pipeline, but it did not by itself resolve how to test tight coordination, which it made easier to study.

BiCoord: building tight-coupling and long-horizon tasks into the benchmark design itself

BiCoord, published in April 2026 by Peng, Gao, Jin, Li, and Liu at Beihang University, is built on the RoboTwin 2.0 platform with 18 tasks. Coordination here is mandated by the task design itself, unlike in a benchmark where two arms happen to share a table.

Three coordination patterns run through the task suite. Phased coupling has the arms alternate between cooperative and independent phases inside a single task, the way a cooking task might require the arms to pick ingredients up separately, combine them, then separate again to plate the result. Spatial-temporal constraint tasks require both arms to hold a temporal ordering and a spatial relation at the same time, so getting the timing right without the positioning right (or vice versa) still counts as failure. Predictive coordination asks one arm to anticipate the other's next move, as when a left arm withdraws in anticipation that the right arm is about to shake a skillet.

The STI metric is BiCoord's clearest contribution to the field, and the comparison figures make the case on their own: 42.16 for BiCoord against 11.20 for RoboTwin 2.0 and 8.13 for RLBench2. That triplet structure means a research team can see exactly which dimension of coordination a policy is failing at, rather than being told only that it failed.

The experimental results should concern anyone currently deploying a bimanual policy in production. Diffusion Policy, RDT, Pi0, and OpenVLA-OFT all struggle with long-duration, highly coupled tasks in BiCoord's own testing. No leading policy architecture has solved this benchmark class yet. The benchmark is exposing weaknesses in the policies people are actually using right now, including in multi-task settings where the gap between task types widens further. Coordination quality is evaluated from the three-perspective temporal, spatial, and spatial-temporal triplet.

DuoBench: stage decomposition and real-world reproducibility as first-class requirements

The benchmark's 11 tasks split across four coordination categories, Asymmetric Support, Sequential Handoff, Parallel Execution, and Bimanual Manipulation, and it runs on the FR3 Duo platform, two standard FR3 arms set up in a human-inspired dual-arm configuration. That hardware choice is not incidental. Labs that already run FR3 arms can adopt the benchmark without new hardware procurement, which lowers the bar for reproduction in a field where hardware access often gates who gets to check whose results.

The methodological centerpiece is stage-based evaluation. Rather than scoring a task pass or fail, DuoBench breaks performance into semantic phases specific to each task, and it can then identify which stage causes a failure rather than just reporting that a failure occurred. The results this produces are more useful than a single success percentage: current policies are particularly challenged during early interaction stages and during parallel arm execution, a level of specificity a binary metric simply cannot offer.

Reproducibility runs through the whole design. Most of the physical task setup is 3D-printable, with two exceptions, a microwave and a large pot, both too big to print. Human-teleoperated datasets exist for all 11 tasks, and four of them have been reproduced in the real world rather than only in simulation. That real-world component surfaces something the purely simulated benchmarks cannot: sim-to-real transfer failure, which DuoBench treats as a persistent failure mode precisely because it is one of the few benchmarks evaluating both sides of that gap.

DuoBench does not claim to maximize STI or horizon length, rather than treating every benchmark as competing on the same axis. Its four coordination categories span a meaningful range of coupling types, but the design prioritizes diagnostic clarity and cross-lab reproducibility over constructing the single hardest possible task. That is a legitimate design choice that complements rather than duplicates what BiCoord is doing Bi-DexHands CoorDex.

ST-BiBench: evaluating whether MLLMs can reason about bimanual coordination at all

ST-BiBench, from Wu and colleagues across Tsinghua University, the University of Hong Kong, HKUST, and the Beijing Innovation Center of Humanoid Robotics, published in February 2026, asks a different question than the physical-benchmark work above. It tests whether multimodal large language models can reason about bimanual coordination at all, evaluating multi-stream multimodal coordination in bimanual embodied tasks across more than 30 state-of-the-art models, including entries from OpenAI, Gemini, Claude, Qwen, InternVL, Gemma, and the Llama series ST-BiBench.

The evaluation runs in three tiers. Strategic Coordination Planning tests high-level cross-modal reasoning over parallel action and perception streams, with 6 tasks built around Independent Parallel Manipulation and 8 around Sequential Collaborative Manipulation. Foundational Spatial Grounding tests workspace awareness and arm-selection logic, meaning whether a model can correctly decide which arm handles which part of a task, across three settings of increasing clutter: sparse, dense, and cluttered. Fine-Grained Action Control tests something more demanding still, whether a model can synthesize a 16-dimensional continuous dual-arm pose out of multimodal metadata, across 5 tasks.

The headline finding is what the authors call the coordination paradox. Frontier MLLMs handle logic-driven strategy well, but they suffer from perception-logic disconnection and multi-stream interference once multimodal fusion enters the picture, meaning strong high-level reasoning does not carry over into fine-grained physical execution. A related pattern, the proximity paradox, occurs when a semantically coherent plan fails to line up with the spatially grounded visual input in front of it, a failure that only appears once both arms are sharing a workspace and the model has to ground language in physical space rather than just reason about it abstractly.

The implication is direct and not particularly comfortable for anyone hoping to deploy a general-purpose MLLM as a bimanual controller soon. Even the strongest models tested are not ready for that role, and ST-BiBench's contribution is pinpointing exactly where in the reasoning pipeline the breakdown happens, between perception and action rather than in planning itself.

Bench2Dex and specialized benchmarks extending the coverage frontier

Bench2Dex, published in September 2026, pushes the benchmark landscape into a sensing modality none of the prior work touches: touch. It is a simulation benchmark for visuo-tactile bimanual manipulation, built across 12 dexterous hands, and it addresses contact-rich manipulation, where tactile feedback, not vision or planning, is the primary channel a policy has to work with. That is a gap the vision-and-language-centric benchmarks above simply do not cover, and its arrival signals that the coordination-benchmark landscape is still expanding outward, not converging on a single settled framework. The field, in other words, is still finding out what it doesn't yet know how to measure.