GLIDE

Task Descriptions

Tomato Plate Transfer

The robot must use both grippers to grasp and transport a plate containing tomatoes from tabletop to an elevated surface. Successful execution requires maintaining plate stability throughout the lift and placement so that no tomatoes are spilled.

Note: visual outlines in the images are added only for readability; the underlying robot observations are otherwise unedited.

GLIDE-Teleoperation

Tomato Plate Transfer

OpenTV (Cheng et al. CoRL 2024)

Raw teleoperation treats the two arms independently. During loaded transport, controller mismatch can create height differences between grippers, tilt the plate, and spill tomatoes.

GLIDE (Ours)

GLIDE treats both grippers as a coupled tray: it keep the plate level, preserve grasp width, restrain wrist tilt, and coordinates gripper open/close.

Case Study (GLIDE keeps grippers at matching heights to avoid side-to-side tilt)

GLIDE Guardrail Generation

Task Prompt

I have a bimanual teleoperation script at <script_name>, where the readings of two controllers from VR headset are mapped to the robot commands through inverse kinematics. I am now collecting data for a new task, that bimanually picking up and lifting a plate full of cherry tomatoes onto a box. Please identify the potential failure cases in the teleoperation process, design constraints and apply the filter that will help to make the human teleoperation easier.

GLIDE Guardrail Code generating


                  class PlateLiftTeleopFilter:
                      # Task-space filter inserted between raw VR target poses and IK.

                      def _filter_carry_midpoint(self, raw_mid, prev_mid):
                          # Move the shared plate center. XY can move faster for
                          # transport, while Z is slower to avoid drops and spills.
                          filtered = prev_mid + pos_alpha * (raw_mid - prev_mid)
                          step = filtered - prev_mid
                          step[:2] = clamp_vector_norm(step[:2], carry_max_xy_speed * dt)
                          step[2] = clip(step[2], -carry_max_z_speed * dt, carry_max_z_speed * dt)
                          step_delta = clamp_accel(step - self.prev_mid_step)
                          self.prev_mid_step = step
                          return prev_mid + step

                      def filter_pair(self, left_target_pose, right_target_pose, stabilize=False):
                          if not stabilize:
                              return filter_single(left), filter_single(right)
                          self._ensure_carry_reference(left_target_pose, right_target_pose)
                          raw_mid = 0.5 * (raw_left_pos + raw_right_pos)
                          raw_mid[2] = max(raw_mid[2], init_mid[2] - carry_down_margin)
                          half_delta = carry_differential_gain * (raw_half_sep - init_half_sep)
                          half_delta[2] = clip(
                              half_delta[2], -0.5 * carry_max_height_diff, 0.5 * carry_max_height_diff)
                          filtered_mid = self._filter_carry_midpoint(raw_mid, prev_mid)
                          left_rot = self._filter_rotation(carry_left_rot, left_target_rot, carry_orientation_weight)
                          right_rot = self._filter_rotation(carry_right_rot, right_target_rot, carry_orientation_weight)
                          return left_filtered, right_filtered


                  def should_stabilize_plate(left_arm, right_arm, left_state, right_state, args):
                      # Enter paired carry when both triggers are pressed or both grippers close.
                      return both_triggers or (left_closed and right_closed)


                  def toggle_plate_grippers_from_triggers(left_arm, right_arm, args):
                      target_label = "open" if min(left_closed, right_closed) >= reopen_threshold else "close"
                      set_gripper_target_label(left_arm, target_label)
                      set_gripper_target_label(right_arm, target_label)

Iterative Self-Refinement

`iteration1`: make loaded carry gentler after spill episodes

Log diagnosis: Hover to play episodes 6-9 spilled after the plate was already grasped; failed runs had faster loaded transport and larger height mismatch.

Before After

Category: Motion limits during carry (plate guardrail restrictions) - Changing speed limits; adding acceleration guard

1222-parser.add_argument("--plate-filter-alpha", type=float, default=0.45)

1222+parser.add_argument("--plate-filter-alpha", type=float, default=0.40)

1224-parser.add_argument("--plate-carry-max-ee-speed", type=float, default=0.18)

1224+parser.add_argument("--plate-carry-max-ee-speed", type=float, default=0.14)

1225+parser.add_argument("--plate-carry-max-ee-accel", type=float, default=0.45)

1350left_pos = self._filter_position(

1351 left_pos, left_prev,

1352- self.config.carry_max_speed,

1352+ self.prev_left_step,

1353+ self.config.carry_max_accel,

Category: Keep the plate level (plate guardrail restrictions) - Tightening shared stability bound

1231-parser.add_argument("--plate-carry-max-height-diff", type=float, default=0.012)

1232+parser.add_argument("--plate-carry-max-height-diff", type=float, default=0.008)

`iteration2`: make transport faster without relaxing vertical safety

Log diagnosis: Hover to play episode 7 fully closed but spilled during carry; one isotropic filter throttled horizontal and vertical motion together.

Before After

Category: Motion limits during carry (plate guardrail restrictions) - Move the plate as one object

1224-parser.add_argument("--plate-carry-max-ee-speed", type=float, default=0.14)

1224+parser.add_argument("--plate-carry-max-ee-speed", type=float, default=0.22)

1225+parser.add_argument("--plate-carry-max-xy-speed", type=float, default=0.24)

1226+parser.add_argument("--plate-carry-max-z-speed", type=float, default=0.055)

1227+parser.add_argument("--plate-carry-max-xy-accel", type=float, default=0.85)

1228+parser.add_argument("--plate-carry-max-z-accel", type=float, default=0.18)

1361-left_pos = self._filter_position(

1361+prev_mid = 0.5 * (left_prev + right_prev)

1364- self.config.carry_max_speed,

1362+filtered_mid = self._filter_carry_midpoint(raw_mid, prev_mid)

1365- self.prev_left_step,

1363+left_pos = filtered_mid + filtered_half_sep

1366- self.config.carry_max_accel,

1364+right_pos = filtered_mid - filtered_half_sep

Category: Restrain wrist/plate tilt (plate guardrail restrictions) - Relaxing an over-strict orientation lock

1235-parser.add_argument("--plate-carry-orientation-weight", type=float, default=1.0)

1239+parser.add_argument("--plate-carry-orientation-weight", type=float, default=0.9)

GLIDE-π_0.5 Policy

Rollout 1

Rollout 2

Rollout 3

Task Descriptions

Marker Handover & Stand

The robot must grasp a marker with one gripper, transfer it to the opposing gripper, place it upright on the tabletop, and withdraw without knocking it over. Successful execution requires precise bimanual coordination and precise upright placement.

Marker handover and stand task description.

Note: visual outlines in the images are added only for readability; the underlying robot observations are otherwise unedited.

GLIDE-Teleoperation

Marker Handover & Stand

OpenTV (Cheng et al. CoRL 2024)

Raw teleoperation follows controller poses and trigger commands directly. The marker can become upright briefly, then topple during release or fast retreat.

GLIDE (Ours)

GLIDE adds rate limits, table and spacing guards, handoff detection, relative-motion damping, height alignment, and a release guard with a held pose, upward retreat, and slower opening.

Case Study (GLIDE smooths end-effector motion to avoid collisions)

GLIDE Guardrail Generation

Task Prompt

I have a bimanual teleoperation script at <script_name>, where the readings of two controllers from VR headset are mapped to the robot commands through inverse kinematics. I am now collecting data for a new task, that using one gripper to pick up the upright marker on the table, handing it over to the other gripper, and place it vertically on the table. Please identify the potential failure cases in the teleoperation process, design constraints and apply the filter that will help to make the human teleoperation easier. Please add a flag to enable this filter.

GLIDE Guardrail Code generating


                  class PenHandoverTeleopFilter:
                      # Task-space guardrail for pickup, handoff, carry, and vertical release.

                      def _apply_table_guard(self, pos):
                          guarded = pos.copy()
                          guarded[2] = max(guarded[2], table_z + table_clearance)
                          return guarded

                      def _filter_position(self, raw_pos, prev_pos, prev_step):
                          raw_pos = self._apply_table_guard(raw_pos)
                          filtered = prev_pos + pos_alpha * (raw_pos - prev_pos)
                          step = self._limit_step(filtered - prev_pos, prev_step)
                          return self._apply_table_guard(prev_pos + step), step

                      def _apply_release_filter(self, side, target_pose, close_fraction):
                          # Anchor pose when fingers begin opening after a firm grasp.
                          opening = prev_close >= release_start_close_fraction and close_fraction < prev_close - 0.02
                          if opening and not state["active"]:
                              state["active"] = True
                              state["ref_pos"] = target_pose[:3, 3].copy()
                              state["ref_rot"] = target_pose[:3, :3].copy()
                          if not lift_allowed:
                              filtered_pos = ref_pos.copy()
                          else:
                              desired_xy = ref_pos[:2] + clamp_vector_norm(
                                  target_pose[:2, 3] - ref_pos[:2], release_xy_radius)
                              filtered_pos[:2] = prev_pos[:2] + clamp_vector_norm(
                                  desired_xy - prev_pos[:2], release_max_xy_speed * dt)
                          filtered[:3, 3] = filtered_pos
                          filtered[:3, :3] = blend_rotation(target_rot, ref_rot, release_orientation_weight)
                          return filtered

                      def _shape_handoff_targets(self, left_pos, right_pos):
                          raw_mid = 0.5 * (left_pos + right_pos)
                          half_sep = prev_half_sep + handoff_relative_gain * (raw_half_sep - prev_half_sep)
                          half_z = clip(
                              0.5 * (left_pos[2] - right_pos[2]),
                              -0.5 * handoff_max_height_diff,
                              0.5 * handoff_max_height_diff)
                          return raw_mid + half_sep, raw_mid - half_sep

                      def filter_pair(self, left_target_pose, right_target_pose, handoff=False):
                          if handoff:
                              left_raw, right_raw = self._shape_handoff_targets(left_raw, right_raw)
                          left_pos, right_pos = enforce_min_xy_distance(left_pos, right_pos)
                          left_filtered = self._apply_release_filter("left", left_filtered, left_close_fraction)
                          right_filtered = self._apply_release_filter("right", right_filtered, right_close_fraction)
                          return left_filtered, right_filtered


                  def should_stabilize_pen_handoff(...):
                      near_handoff = xy_distance <= handoff_xy_window and z_distance <= handoff_z_window
                      return near_handoff and (both_triggers or left_holding or right_holding)

Iterative Self-Refinement

`iteration1`: remove lag, then guard the release

Log diagnosis: Hover to play raw controller motion was faster than commanded EE motion; failures got the marker upright briefly, then tipped or fell during final opening and retract.

Before After

Category: Motion rate limits (marker guardrail restrictions) - Changing speed damping parameter to remove lag

1256-parser.add_argument("--pen-filter-alpha", type=float, default=0.35)

1256+parser.add_argument("--pen-filter-alpha", type=float, default=0.70)

1258-parser.add_argument("--pen-max-ee-speed", type=float, default=0.22)

1258+parser.add_argument("--pen-max-ee-speed", type=float, default=0.42)

1259-parser.add_argument("--pen-max-xy-speed", type=float, default=0.24)

1259+parser.add_argument("--pen-max-xy-speed", type=float, default=0.45)

1260-parser.add_argument("--pen-max-z-speed", type=float, default=0.08)

1260+parser.add_argument("--pen-max-z-speed", type=float, default=0.20)

1261-parser.add_argument("--pen-max-ee-accel", type=float, default=0.70)

1261+parser.add_argument("--pen-max-ee-accel", type=float, default=2.50)

1286-parser.add_argument("--pen-gripper-max-speed", type=float, default=0.80)

1286+parser.add_argument("--pen-gripper-max-speed", type=float, default=2.00)

Category: Upright orientation bias and transfer stabilization (marker guardrail restrictions) - Relaxing over-damped guidance

1264-parser.add_argument("--pen-orientation-weight", type=float, default=0.85)

1264+parser.add_argument("--pen-orientation-weight", type=float, default=0.15)

1265-parser.add_argument("--pen-handoff-orientation-weight", type=float, default=0.95)

1265+parser.add_argument("--pen-handoff-orientation-weight", type=float, default=0.35)

1278-parser.add_argument("--pen-min-ee-xy-distance", type=float, default=0.030)

1278+parser.add_argument("--pen-min-ee-xy-distance", type=float, default=0.050)

1279+parser.add_argument("--pen-handoff-min-ee-xy-distance", type=float, default=0.035)

Category: Release and gripper-rate guardrails (marker guardrail restrictions) - Adding new release guard

1286-parser.add_argument("--pen-gripper-max-speed", type=float, default=2.00)

1286+parser.add_argument("--pen-gripper-max-speed", type=float, default=1.20)

1418filtered = target_pose.copy()

1420+self.release_state = {"left": self._new_release_state(), "right": self._new_release_state()}

1434+def _apply_release_filter(self, side, target_pose, close_fraction):

1474+desired_xy = ref_pos[:2] + clamp_vector_norm(target_pose[:2, 3] - ref_pos[:2], self.config.release_xy_radius)

1484+desired_z = max(float(target_pose[2, 3]), float(ref_pos[2]))

`iteration2`: delay and soften the final finger opening

Log diagnosis: Hover to play failures were no longer mainly lag, but topples during or immediately after final release.

Before After

Category: Release and gripper-rate guardrails (marker guardrail restrictions) - Changing release functional form

1291+parser.add_argument("--pen-carry-orientation-weight", type=float, default=0.45)

1306+parser.add_argument("--pen-release-lift-start-close-fraction", type=float, default=0.50)

1307+parser.add_argument("--pen-release-lift-height", type=float, default=0.015)

1460+"open_frames": 0,

1496-if elapsed < self.config.release_hold_s or close_fraction > end_close:

1497+lift_allowed = elapsed >= self.config.release_hold_s and close_fraction <= lift_start

1520+release_z = float(ref_pos[2]) + self.config.release_lift_height * lift_progress

1530-if elapsed >= self.config.release_retreat_s and close_fraction <= end_close:

1530+if state["open_frames"] * self.dt >= self.config.release_retreat_s:

1321+parser.add_argument("--pen-gripper-max-open-speed", type=float, default=0.90)

1332+parser.add_argument("--pen-gripper-release-delay-s", type=float, default=0.15)

1333+parser.add_argument("--pen-gripper-release-delay-close-fraction", type=float, default=0.65)

GLIDE-π_0.5 Policy

Rollout 1

Rollout 2

Rollout 3

Task Descriptions

Wine Serving

The robot must use the left gripper to grasp the wine bottle and the right CRAFT hand to grasp the wine cup, then coordinate the two end effectors to pour wine from the bottle into the cup. Successful execution requires retaining stable grasps on both objects while controlling their poses during pouring.

Note: visual outlines in the images are added only for readability; the underlying robot observations are otherwise unedited.

GLIDE-Teleoperation

Wine Serving

OpenTV (Cheng et al. CoRL 2024)

Raw CRAFT teleoperation passes arm and hand targets through directly. Operators must stabilize, align, tilt, and avoid spills simultaneously.

GLIDE (Ours)

GLIDE learns the correct tool upright axes, makes tilt constraints phase-aware, permits setup motion, boosts bottle pour tilt, and softly assists bottle-over-cup alignment once pouring starts.

Case Study (GLIDE amplifies end-effector rotation to facilitate pouring)

GLIDE Guardrail Generation

Task Prompt

I have a bimanual teleoperation script at <script_name>, where the readings of two controllers from VR headset are mapped to the robot commands through inverse kinematics and retargeting. The left arm is equipped with a gripper, and the right arm is equipped with a dexterous end effector. I am now collecting data for a new task, that use left gripper to pick up the wine bottle, and right end effector to pick up the wine cup, and after aligning, we pour wine from bottle to the cup, ideally during pouring, the liquid should not spill and finally put both bottle and cup on the table. Please identify the potential failure cases in the teleoperation process, design constraints and apply the filter that will help to make the human teleoperation easier. Write a new script for this.

GLIDE Guardrail Code generating


                  def _pose_axis_closest_to_up(pose):
                      # Auto-calibrate which local tool axis is upright at reset.
                      return max([x, -x, y, -y, z, -z], key=lambda axis: dot(pose[:3, :3] @ axis, WORLD_UP))


                  class WinePourGuardrail:
                      # Post-process raw arm poses before the base teleop loop commands IK.

                      def reset(self, left_pose, right_pose):
                          self.previous.clear()
                          if bottle_upright_axis is None:
                              self._learned_axes["left"] = _pose_axis_closest_to_up(left_pose)
                          if cup_upright_axis is None:
                              self._learned_axes["right"] = _pose_axis_closest_to_up(right_pose)

                      def filter_output(self, output, dt):
                          left, right = copy(output.left_pose), copy(output.right_pose)
                          left = self._boost_bottle_pour_tilt(left, reasons)
                          pour_requested = self._pour_requested(left)
                          pour_aligned = self._pour_geometry_ok(left, right)
                          cup_max_tilt = cup_carry_max_tilt
                          if lock_cup_during_pour and pour_requested and pour_aligned:
                              cup_max_tilt = cup_upright_max_tilt
                          right = _cap_tilt(right, self._upright_axis("right"), cup_max_tilt)
                          left = _cap_tilt(left, self._upright_axis("left"), self._allowed_bottle_tilt(left, right, reasons))
                          left = self._assist_alignment(left, right, reasons)
                          left, right = self._separate_arms(left, right, reasons)
                          output.left_pose = self._rate_limit("left", left, dt, reasons)
                          output.right_pose = self._rate_limit("right", right, dt, reasons)
                          return output

                      def _boost_bottle_pour_tilt(self, left_pose, reasons):
                          # Amplify bottle tilt once the operator starts pouring.
                          if tilt <= bottle_pour_tilt_boost_start:
                              return left_pose
                          boosted_tilt = start + (tilt - start) * bottle_pour_tilt_boost_gain
                          return self._set_tilt(left_pose, bottle_axis, min(boosted_tilt, bottle_pour_max_tilt))

                      def _assist_alignment(self, left_pose, right_pose, reasons):
                          # Pull the bottle mouth toward a point above the cup rim.
                          bottle_point = _tool_point(left_pose, bottle_mouth_offset)
                          cup_point = _tool_point(right_pose, cup_rim_offset)
                          desired_bottle_point = cup_point + [0.0, 0.0, align_assist_target_height]
                          correction = clamp_norm(desired_bottle_point - bottle_point, align_assist_max_correction)
                          left_pose[:3, 3] += align_assist_strength * correction
                          return left_pose


                  def guard_craft_targets(targets, config):
                      # Cap side-splay, thumb bend, and finger bend for a gentle cup grasp.
                      for motor_id, raw in targets.items():
                          guarded[motor_id] = limit_motor_fraction(motor_id, raw, config)
                      return guarded

Iterative Self-Refinement

`iteration1`: learn the tool frame and remove setup caps

Log diagnosis: Hover to play FK showed local y was closest to world up; the cup and bottle tilt caps were blocking necessary wrist rotation before pouring.

Before After

Category: Cup and bottle tilt gates (wine guardrail restrictions) - Learn the true tool frame

60-def _axis_vector(axis: str) -> np.ndarray:

60+def _axis_vector(axis: str) -> np.ndarray | None:

62+if axis == "auto": return None

80+def _pose_axis_closest_to_up(pose: np.ndarray) -> np.ndarray:

232-cup_upright_axis: np.ndarray = field(default_factory=lambda: _axis_vector("z"))

232+cup_upright_axis: np.ndarray | None = None

233-bottle_upright_axis: np.ndarray = field(default_factory=lambda: _axis_vector("z"))

233+bottle_upright_axis: np.ndarray | None = None

234-bottle_carry_max_tilt_rad: float = math.radians(28.0)

234+bottle_carry_max_tilt_rad: float = math.radians(90.0)

235-cup_upright_max_rad: float = math.radians(12.0)

235+cup_upright_max_rad: float = math.radians(18.0)

235-bottle_pour_max_tilt_rad: float = math.radians(112.0)

235+bottle_pour_max_tilt_rad: float = math.radians(125.0)

236+cup_carry_max_tilt_rad: float = math.radians(85.0)

Category: Workspace and rate limits (wine guardrail restrictions) - Restoring setup wrist authority

522-"rotation_alpha": (0.50, "--rotation-alpha"),

522+"rotation_alpha": (0.75, "--rotation-alpha"),

524-"max_target_angular_speed": (1.15, "--max-target-angular-speed"),

524+"max_target_angular_speed": (2.80, "--max-target-angular-speed"),

908-parser.add_argument("--guardrail-max-angular-speed", type=float, default=1.05)

908+parser.add_argument("--guardrail-max-angular-speed", type=float, default=2.60)

`iteration2`: add pour authority and alignment assistance

Log diagnosis: Hover to play the operator wrist reached about 103 degrees, but the robot bottle tilt stopped around 88 degrees, so the robot still could not pour.

Before After

Category: Cup and bottle tilt gates (wine guardrail restrictions) - Increasing pour authority

235-bottle_pour_max_tilt_rad: float = math.radians(125.0)

235+bottle_pour_max_tilt_rad: float = math.radians(135.0)

237+bottle_pour_tilt_boost_gain: float = 1.85

241+align_assist: bool = True

242+align_assist_start_rad: float = math.radians(32.0)

244+align_assist_strength: float = 0.65

597-"max_arm_joint_step": (0.026, "--max-arm-joint-step"),

597+"max_arm_joint_step": (0.045, "--max-arm-joint-step"),

Category: Bottle-over-cup alignment and assist (wine guardrail restrictions) - Adding pouring assistance

318-left = _cap_tilt(left, self._upright_axis("left"), self._allowed_bottle_tilt(left, right, reasons))

318+left = self._boost_bottle_pour_tilt(left, reasons)

322+left = self._assist_alignment(left, right, reasons)

396+def _assist_alignment(self, left_pose, right_pose, reasons):

410+desired_bottle_point = cup_point + np.asarray([0.0, 0.0, target_height])

GLIDE-π_0.5 Policy

Rollout 1

Rollout 2

Rollout 3

Abstract

Behavior cloning for robot manipulation relies on expert demonstrations. However, for tasks that require dynamic stability, precise contact timing, or dexterous coordination, human teleoperators may find it hard or evenimpossible to collect data. We study this infeasible-demonstration regime and propose GLIDE, a framework that infers task-specific failure modes and converts them into executable guardrails for data collection and policy deployment. Given a task description, GLIDE writes guardrails that use robot states to filter teleoperation and policy commands, constrain failure-prone actions, and iteratively improve from closed-loop dataset feedback. Across Tomato plate transfer, Marker handover & stand, and Wine serving tasks, GLIDE improves data collection over raw VR teleoperation and manually designed guardrails. After refinement, GLIDE raises data-collection efficiency from 0-10% to 70-90% across the three tasks. During policy execution, guarded deployment raises Tomato plate transfer policy success from 0% to 60-70%, enabling efficient learning on mixed-quality demonstrations. These results show that GLIDE can support policy learning when direct demonstrations are infeasible.

GLIDE: Guardrails for Learning from Infeasible Demonstrations Efficiently

TL;DR: GLIDE uses closed-loop, self-refined guardrails to make infeasible robot manipulation tasks demonstrable, learnable, and the trained policies more reliable to deploy.

GLIDE Methodology

Task Descriptions

🍅 Tomato Plate Transfer

GLIDE-Teleoperation

🍅 Tomato Plate Transfer

Case Study (GLIDE keeps grippers at matching heights to avoid side-to-side tilt)

GLIDE Guardrail Generation

Iterative Self-Refinement

iteration1: make loaded carry gentler after spill episodes

iteration2: make transport faster without relaxing vertical safety

GLIDE-π0.5 Policy

Rollout 1

Rollout 2

Rollout 3

Results

Expert Demonstration Collection

Guarded Policy Execution

Abstract

Tomato Plate Transfer

Tomato Plate Transfer

`iteration1`: make loaded carry gentler after spill episodes

`iteration2`: make transport faster without relaxing vertical safety

GLIDE-π_0.5 Policy