Research Papers

arXiv:2509.17125v2 Announce Type: replace Abstract: Relational object rearrangement (ROR) tasks (e.g., insert flower to vase) require a robot to manipulate objects with precise semantic and geometric reasoning. Existing approaches either rely on pre-collected demonstrations that struggle to capture complex geometric constraints or generate goal-state observations to capture semantic and geometric knowledge, but fail to explicitly couple object transformation with action prediction, resulting in errors due to generative noise. To address these limitations, we propose Imagine2Act, a 3D imitation-learning framework that incorporates semantic and geometric constraints of objects into policy learning to tackle high-precision manipulation tasks. We first generate imagined goal images conditioned on language instructions and reconstruct corresponding 3D point clouds to provide robust semantic and geometric priors. These imagined goal point clouds serve as additional inputs to the policy model

arXiv:2509.17053v2 Announce Type: replace Abstract: Many contact-rich manipulation tasks require precise force regulation. However, most imitation learning (IL) policies remain position-centric and lack explicit force awareness, and adding force/torque sensors to collaborative robot arms is often costly and requires additional hardware design. To overcome these issues, we propose FILIC, a Force-guided Imitation Learning framework with impedance torque control. FILIC integrates a Transformer-based IL policy with an impedance controller in a dual-loop structure, enabling compliant force-informed, force-executed manipulation. For robots without force/torque sensors, we introduce a cost-effective end-effector force estimator using joint torque measurements through analytical Jacobian-based inversion while compensating with model-predicted torques from a digital twin. Experiments show that FILIC significantly outperforms vision-only and joint-torque-based methods, achieving safer, more comp

arXiv:2509.14641v2 Announce Type: replace Abstract: Dense 3D convolutions provide high accuracy for perception but are too computationally expensive for real-time robotic systems. Existing tri-plane methods rely on 2D image features with interpolation, point-wise queries, and implicit MLPs, which makes them computationally heavy and unsuitable for embedded 3D inference. As an alternative, we propose TriLift, a novel interpolation-free tri-plane lifting and volumetric fusion framework that directly projects 3D voxels into plane features and reconstructs a feature volume through broadcast and summation. This shifts nonlinearity to 2D convolutions, reducing complexity while remaining fully parallelizable. To mitigate spatial information loss inherent in projections, we incorporate a lightweight adaptive positional encoding module that helps bridge the spatial information gap, dynamically recovering fine geometric details with negligible overhead. To capture global context, we add a low-re

arXiv:2507.12744v4 Announce Type: replace Abstract: Detecting deformable linear objects (DLOs), such as floor cables, is essential for safe mobile robot navigation but remains challenging due to oblique viewpoints, thin structures, and limited edge-device resources. Existing DLO segmentation methods are primarily designed for manipulator platforms with fixed top-down views and often require heavy models, limiting their deployment on mobile robots. We formulate a cross-view DLO segmentation problem, where models trained on manipulator-view data must generalize to mobile robot perspectives. To address this, we propose ASC-SW, a lightweight and geometry-aware segmentation framework. The core network, ASCNet, introduces Atrous Strip Convolution, combining directional strip filtering with dilated receptive fields to enhance sensitivity to elongated structures at low computational cost. An Atrous Strip Convolution Spatial Pyramid Pooling module enables multi-scale anisotropic feature aggrega

arXiv:2507.08262v2 Announce Type: replace Abstract: The spatial information inherent in 3D point clouds is crucial for robotic manipulation. However, existing 3D pre-training methods face a fundamental trade-off: Masked Autoencoding (MAE) excels at capturing spatial-geometric features but lacks semantics, whereas contrastive learning, while able to distill semantics from 2D foundation models, is ill-suited for the fine-grained details required for manipulation tasks. To address these challenges, we propose CLAR, a novel 3D pre-training framework that synergizes global understanding with fine-grained local alignment. Our framework unifies MAE with global cross-modal contrastive learning to integrate robust spatial awareness with rich semantic understanding. To enhance its focus on fine-grained details, at the local level, we introduce an adaptive alignment mechanism that leverages deformable attention to force precise correspondences between local 3D geometry and 2D visual features, the

arXiv:2505.17295v2 Announce Type: replace Abstract: We introduce ScanBot, a benchmark for instruction-conditioned, high-precision surface scanning with robot-mounted industrial laser profilers. Unlike existing robot learning datasets that emphasize coarse behaviors such as grasping, navigation, or dialogue, ScanBot targets sensing-centric tasks where sub-millimeter motion continuity, strict stand-off control, and stable scanner settings are essential for acquiring usable geometry. The dataset contains scanning trajectories over twenty objects, including electronic components and structured 3D-printed parts, and spans six task types that range from broad inspection to fine-grained detail scanning and geometry-critical operations, including metrology and registration. Each episode is specified by natural language instructions and paired with synchronized first-person RGB-D, third-person video, laser height profiles, robot joint and pose traces, and scanner-parameter logs. These requireme

arXiv:2505.11495v2 Announce Type: replace Abstract: Push recovery during locomotion will facilitate the deployment of humanoid robots in human-centered environments. In this paper, we present a unified framework for walking control and push recovery for humanoid robots, leveraging the arms for push recovery while dynamically walking. The key innovation is to use the environment, such as walls, to facilitate push recovery by combining Single Rigid Body model predictive control (SRB-MPC) with Hybrid Linear Inverted Pendulum (HLIP) dynamics to enable robust locomotion, push detection, and recovery by utilizing the robot's arms to brace against such walls and dynamically adjusting the desired contact forces and stepping patterns. Extensive simulation results on a humanoid robot demonstrate improved perturbation rejection and tracking performance compared to HLIP alone, with the robot able to recover from pushes up to 100N for 0.2s while walking at commanded speeds up to 0.5m/s. Robustness

arXiv:2505.11494v3 Announce Type: replace Abstract: Robot learning has produced remarkably effective ``black-box'' controllers for complex tasks such as dynamic locomotion on humanoids. Yet ensuring dynamic safety, i.e., constraint satisfaction, remains challenging for such policies. Reinforcement learning (RL) embeds constraints heuristically through reward engineering, and adding or modifying constraints requires retraining. Model-based approaches, like control barrier functions (CBFs), enable runtime constraint specification with formal guarantees but require accurate dynamics models. This paper presents SHIELD, a layered safety framework that bridges this gap by: (1) training a generative, stochastic dynamics residual model using real-world data from hardware rollouts of the nominal controller, capturing system behavior and uncertainties; and (2) adding a safety layer on top of the nominal (learned locomotion) controller that leverages this model via a stochastic discrete-time CBF

arXiv:2505.10022v4 Announce Type: replace Abstract: Learning natural, animal-like locomotion from demonstrations has become a core paradigm in legged robotics. While motion tracking can reproduce reference gaits, many approaches still require substantial tuning and depend on reference motion inputs at deployment, which can limit responsiveness to task objectives and reduce adaptability. We present APEX (Action Priors enable Efficient eXploration), a motion-tracking reinforcement learning (RL) framework that removes deployment-time dependence on reference motion inputs, improves sample efficiency, and reduces tuning effort. APEX integrates demonstrations into RL via decaying action priors, which guide early exploration toward demonstration-consistent actions and then fade to zero, yielding a pure RL policy at deployment. This is combined with a multi-critic framework that separates style and task + regularization learning signals. Moreover, APEX enables a single policy to learn diverse

arXiv:2505.09603v2 Announce Type: replace Abstract: Recently, the robotics community has amassed ever larger and more diverse datasets to train generalist policies. However, while these policies achieve strong mean performance across a variety of tasks, they often underperform on individual, specialized tasks and require further tuning on newly acquired task-specific data. Combining task-specific data with carefully curated subsets of large prior datasets via co-training can produce better specialized policies, but selecting data naively may actually harm downstream performance. To address this, we introduce DataMIL, a data selection framework built on the datamodels paradigm that reasons about data selection in an end-to-end manner, using the policy itself to identify which data points will most improve performance. Unlike standard practices that filter data using human notions of quality (e.g., based on semantic or visual similarity), DataMIL directly optimizes data selection for tas

arXiv:2503.03579v2 Announce Type: replace Abstract: Spoken instructions in robot-to-human handovers may specify either an object ("the cup") or an intended use ("pour water"); in both cases, successful handover requires the robot to infer the target object and the region remaining available for the human to hold. If the robot grasps that hold region, the object could become awkward to receive and immediately use, potentially reducing perceived competence and trust; if the gripper approaches too close to the receiving hand during delivery, perceived safety may also suffer. We present Intent-Handover, which grounds unconstrained speech and visual scene context into explicit grasp and delivery constraints. Given a spoken instruction and a scene observation, a vision-language model identifies the target object and the intended human-usage region. A grasp optimization module then selects a feasible grasp keeping this region accessible while enforcing clearance from the predicted receiving h

arXiv:2501.19274v3 Announce Type: replace Abstract: The Great Outdoors (GO) dataset is a multi-modal annotated data resource aimed at advancing ground robotics research in unstructured environments. Existing off-road datasets often lack sensor diversity and exclude vital modalities like thermal and radar that are critical for operation in degraded conditions (e.g., low visibility or adverse weather). To address these gaps, we introduce a large-scale multimodal off-road dataset with six complementary sensor modalities, along with semantic annotations and GPS traces, to support tasks such as semantic segmentation, object detection, and SLAM. The diverse environmental conditions represented in the dataset present significant real-world challenges, which provide opportunities to develop more robust solutions to support the continued advancement of field robotics, autonomous exploration, and perception systems in natural environments. The dataset can be downloaded at: https://www.unmannedla

arXiv:2411.18276v3 Announce Type: replace Abstract: Effectively manipulating articulated objects in household scenarios is a crucial step toward achieving general embodied artificial intelligence. Mainstream research in 3D vision has primarily focused on manipulation through depth perception and pose detection. However, in real-world environments, these methods often face challenges due to imperfect depth perception, such as with transparent lids and reflective handles. Moreover, they generally lack the diversity in part-based interactions required for flexible and adaptable manipulation. To address these challenges, we introduced a large-scale part-centric dataset for articulated object manipulation that features both photo-realistic material randomization and detailed annotations of part-oriented, scene-level actionable interaction poses. We evaluated the effectiveness of our dataset by integrating it with several state-of-the-art methods for depth estimation and interaction pose pre

arXiv:2409.15493v4 Announce Type: replace Abstract: Persistent semantic monitoring of indoor spaces such as warehouses, hospitals, and offices requires a robot to repeatedly monitor an environment and track how objects change over time. Running full simultaneous localization and mapping (SLAM) with dense semantic reconstruction from scratch on every visit is redundant when the environment geometry stays the same and only the objects move. We present a modular two-stage system that separates geometric mapping from semantic updating. In the first stage, a frontier-based exploration method with a dynamic search window builds a 2D occupancy grid. In the second stage, the robot relocalizes in this map and builds a semantic object graph using an open-vocabulary object detector and a promptable segmentation model. Only the lightweight semantic stage is repeated on later visits, so the system scales well to frequent revisits. The object graph uses a category and distance based association rule

arXiv:2407.16288v4 Announce Type: replace Abstract: Uncrewed Aerial Vehicles (UAVs) offer agile, cost-effective, and efficient solutions for communication relay networks. However, their modeling and control are challenging, and the mismatch between simulations and actual conditions limits real-world deployment, while maintaining adequate situational awareness remains essential for safe operation. Several studies have proposed integrating UAV operations with immersive digital technologies, such as Digital Twin (DT) and Extended Reality (XR), to overcome these challenges. This paper provides a comprehensive overview of the latest research and developments involving immersive digital technologies for UAVs. We explore the use of Machine Learning (ML) techniques, particularly Deep Reinforcement Learning (DRL), to improve the capabilities of DT for UAV systems, and present a case study of a DT-driven DRL pipeline that couples bidirectional physical-digital synchronization with online recursi