News & Research

arXiv:2606.10276v2 Announce Type: replace Abstract: For natural human-robot interaction, a robot must understand human intent expressed not only through language but also through nonverbal signals such as gestures and gaze. However, current robot policies rely on language instructions as the sole interface for conveying intent, leaving nonverbal signals unused and placing the full burden of communication. In this work, we present EDITH, a robot framework that captures the human's nonverbal signals through continuous streams of first-person view and gaze from smart glasses, and uses them alongside language instructions as inputs to the robot policy. Our hardware system streams the human's first-person view, gaze, and speech to the robot in real time, transcribing the speech into language instructions. To handle these rich but noisy signals, we design a hierarchical policy in which a high-level policy infers the human's intent and produces a sequence of subtasks, where each subtask is re

arXiv:2606.05160v2 Announce Type: replace Abstract: Scaling humanoid loco-manipulation requires robot-compatible demonstrations across diverse objects, whole-body motions, and scene geometries, but teleoperation and motion capture are difficult to scale because each collection depends on physical setups, instrumented actors, and robot operation. We present GRAIL, a digital generation pipeline that remains fully virtual until deployment: it grounds video world models in 3D assets and simulator-ready scenes to synthesize interactions without rebuilding physical environments or teleoperating the robot. Rather than reconstructing unconstrained in-the-wild videos, GRAIL starts from fully specified 3D configurations in which object geometry, camera parameters, metric scale, environment depth, and a robot-proportioned character are known before video generation and reused during reconstruction. This privileged setup better conditions 4D recovery, allowing model-based object tracking, human mo

arXiv:2605.30484v2 Announce Type: replace Abstract: Vision-Language-Action (VLA) models have shown promise for robotic manipulation, yet most existing policies operate reactively by directly regressing actions from current observations, without explicitly modeling future dynamics. This limits their ability to generalize under out-of-distribution perturbations. To address this issue, we propose ELAN4D, an embodiment-centric, 4D-aware training framework that enhances VLA policies with future robot keypoint tracks as predictive spatio-temporal supervision. Using only forward kinematics from proprioceptive states, we derive 3D displacement tracks of robot keypoints, such as joints and the end-effector, with negligible preprocess cost. These tracks provide metric and compact supervision without requiring external trackers or reconstruction. A plug-and-play auxiliary branch with a lightweight track decoder injects this 4D signal into the action expert while preserving the pretrained vision-l

arXiv:2605.29562v2 Announce Type: replace Abstract: Vision-Language-Action (VLA) models have shown strong potential for general-purpose robotic manipulation, yet they still struggle to generalize to unseen tasks that necessitate transferring relevant experience across objects, scenes, and action patterns. This paper proposes VLA-Pro, a plug-and-play framework designed to enhance cross-task generalization by storing task-relevant procedural memories at training time and transferring these memories during inference. Specifically, VLA-Pro stores task-specific LoRA adapters as parameterized procedural memories during training. At inference time, VLA-Pro retrieves relevant procedural memories based on the current multi-modal context and dynamically fuses these memories for generating the current action chunk. Experiments on RoboTwin, RLBench, and real-world manipulation tasks show that VLA-Pro consistently improves cross-task generalization across multiple backbones, achieving up to a 207%

arXiv:2605.17661v2 Announce Type: replace Abstract: Autonomous agile robots need more than metric geometry: they must understand objects, rooms, places, and spatial relations for search, inspection, exploration, and human-robot interaction. Conventional metric maps support localization and collision avoidance, but do not provide this semantic and relational structure. 3D scene graphs address this gap by connecting geometry with object-level and room-level understanding. Building such representations on agile platforms remains difficult because aerial and lightweight robots operate under strict payload, power, and compute limits, making RGB-D cameras and LiDAR sensors impractical for many onboard settings. We present Mono-Hydra++, a real-time monocular RGB plus IMU pipeline for indoor metric-semantic mapping and hierarchical 3D scene graph construction. The system combines M2H-MX, a DINOv3-based multi-task model for depth and semantics, with a deep-feature visual-inertial odometry front

arXiv:2603.17808v3 Announce Type: replace Abstract: Video generative models offer a promising basis for robotic planning by predicting future visual rollouts from observations and task instructions. However, their training objectives typically lack explicit constraints on robot executability. Generated rollouts can exhibit embodiment-related artifacts, including arm deformation and temporal discontinuities, while the actions decoded from these rollouts may be unstable or violate robot motion limits. We term this mismatch between generated robot behavior and embodiment-specific execution requirements the executability gap. We introduce Executable Video Alignment (EVA), a post-training framework that uses action-space feedback to improve both the visual integrity of generated robot motion and the feasibility of decoded actions. EVA trains an inverse dynamics model (IDM) on robot demonstrations and freezes it to decode actions from generated videos. We compute a reward from these actions

arXiv:2603.14236v2 Announce Type: replace Abstract: Developing autonomous drone applications requires developers to compose navigation, sensing, analytics and compute services while satisfying physical and environmental constraints. Large Language Models (LLMs) can generate code from natural-language requirements, but direct prompting remains unreliable for cyber-physical Drone-as-a-Service (DaaS) missions because incorrect API calls, unsafe flight logic or missing analytics choices can cause mission failure. We propose AeroGen, an LLM-driven low-code framework that translates mission requirements into executable DaaS programs using structured guardrails. AeroGen combines static guardrails, such as API signatures and flight constraints, with dynamic robot, runtime and world information so that the LLM can compose navigation, sensing, analytics and execution services in one generation attempt. We evaluate AeroGen on 20 navigation tasks and 5 analytical missions across simulated and real

arXiv:2603.07400v2 Announce Type: replace Abstract: Walking over disconnected footholds requires selecting landing regions while adapting step timing to the robot's motion. We present a perceptive model predictive planner that combines discrete region selection, foot placement, and current-step duration in a mixed-integer quadratic program using step-to-step Divergent Component of Motion (DCM) dynamics. Onboard depth data are fused into a probabilistic heightmap in the stance foot frame. Convex steppable regions are extracted and shrunk by margins that increase with estimated height uncertainty. Linear constraints impose capturability-based bounds on DCM evolution and limit swing-foot retargeting using the remaining commanded swing time. Within-step replanning updates the initial DCM from measurements to respond to model mismatch and disturbances. We evaluate the framework on Digit in simulation over randomized footholds and on hardware over separated plates, with mean planner solve ti

arXiv:2603.05504v3 Announce Type: replace Abstract: Scaling robot learning is bottlenecked not only by data volume, but also by whether the data targets the policy's failures. Existing robot-free handheld interfaces enable portable in-the-wild data collection, yet remain open-loop: users record demonstrations without feedback from the current policy. Interactive methods such as DAgger provide such feedback by collecting corrective on-policy data, but require deploying imperfect policies on physical robots, making them costly, unsafe, and difficult to scale. We present RoboPocket, a smartphone-based system for robot-free policy iteration. By combining on-device sensing, display, and communication with cloud-side policy inference, RoboPocket provides real-time collection guidance and visualizes the policy's predicted trajectory in augmented reality as AR Visual Foresight. This allows users to identify policy weaknesses and collect targeted corrective data without deploying a physical rob

arXiv:2510.12971v2 Announce Type: replace Abstract: We present Actron3D, a framework that enables robots to acquire transferable 6-DoF manipulation skills from monocular, uncalibrated, RGB-only human demonstration videos. Our key idea is to represent manipulation knowledge within a video as a continuous neural function over object space. At the core of Actron3D lies the Neural Affordance Function, which distills geometry, visual features, contact priors, and action flows from diverse demonstration videos into a compact 3D neural representation. During deployment, we adopt a hierarchical pipeline that retrieves the matched affordance function and transfers encoded manipulation knowledge to novel objects through coarse-to-fine differentiable optimization. Leveraging the continuous nature of Neural Affordance Function, the framework performs spatial queries over multimodal features to align demonstrations with observations and generates precise 6-DoF manipulation policy. Experiments in bo

arXiv:2507.18847v4 Announce Type: replace Abstract: We propose a new volumetric grasp model that is equivariant to rotations around the vertical axis, leading to a significant improvement in sampling efficiency. Our model employs a tri-plane volumetric feature representation -- i.e., the projection of 3D features onto three canonical planes. We introduce a novel tri-plane feature design in which features on the horizontal plane are \emph{equivariant} to $90^\circ$ rotations, while the \emph{sum} of features from the other two planes remains \emph{invariant} to reflections induced by the same transformations. We further develop equivariant adaptations of two state-of-the-art volumetric grasp planners, GIGA and IGD. Specifically, we derive a new equivariant formulation of IGD's deformable attention mechanism and propose an equivariant generative model of grasp orientations based on flow matching. We provide a detailed analytical justification of the proposed equivariance properties and v

arXiv:2507.15478v2 Announce Type: replace Abstract: Ensuring reliable and rule-compliant behavior of autonomous agents in uncertain environments remains a fundamental challenge in modern robotics. Our work shows how neuro-symbolic systems, which integrate probabilistic, symbolic white-box reasoning models with deep learning methods, offer a powerful solution to this challenge. They enable the simultaneous consideration of explicit rules and neural models trained on noisy data, combining the strengths of structured reasoning with flexible representations. To this end, we introduce the Constitutional Controller (CoCo), a novel framework designed to enhance the safety and reliability of agents by reasoning over deep probabilistic logic programs that represent constraints, such as those found in shared traffic spaces. Furthermore, we propose the concept of self-doubt, implemented as a probability density conditioned on doubt features such as travel velocity, employed sensors, or health fac

arXiv:2505.06513v2 Announce Type: replace Abstract: Large Language Models (LLMs) have shown strong capabilities in reasoning and planning, prompting researchers to explore their use as decentralized decision-makers for multi-robot formation control. However, directly applying LLMs to such tasks often leads to unstable behavior, in which robots may collapse to the centroid of their positions or diverge entirely, due to hallucinated reasoning and limited coordination awareness. We propose LLM-Flock, a decentralized multi-robot flocking framework that addresses these challenges through two key mechanisms: (1) an influence-based consensus protocol, where robots iteratively refine individually generated formation plans by adopting those of more connected neighbors, driving the team toward a unified plan without centralized control, and (2) a two-layer collision avoidance strategy that pairs LLM-based urgency reasoning with a programmatic collision checker and re-prompting mechanism. We eval