Products

G2 Developer Getting Started Guide

Version: v1.0Release Date: 2026-07-04Genie G2

1. Install Dependencies

Connect your development PC to the robot over Ethernet, then install the GDK package and common vision libraries:


curl -sSL http://10.42.1.101:8849/install.sh | bash
source ~/.cache/agibot/app/env.sh


pip install opencv-python numpy
# Optional — needed for grasp_pipeline.py
pip install ultralytics flask

2. GDK Program Structure

Every GDK program must follow this exact init → use → release lifecycle:

import agibot_gdk as gdk
import time

# 1. Initialize — must be called before anything else
if gdk.gdk_init() != gdk.GDKRes.kSuccess:
    raise RuntimeError("gdk_init() failed")

try:
    # 2. Instantiate modules
    robot  = gdk.Robot()
    camera = gdk.Camera()
    tf     = gdk.TF()

    # 3. Wait for DDS subscriptions to connect (~2 s is sufficient)
    time.sleep(2)

    # 4. Use the SDK
    joints = robot.get_joint_states()
    print(f"Robot has {joints['nums']} joints")

except Exception as e:
    print(f"Error: {e}")

finally:
    # 5. Close module resources BEFORE releasing GDK
    camera.close_camera()
    gdk.gdk_release()

Rules to remember:

Call gdk_init() first, always.
Add time.sleep(2) after instantiating any module — DDS needs time to connect.
Close modules (close_camera(), close_lidar(), etc.) before gdk_release().
Wrap your body in try/finally so gdk_release() is never skipped.

3. Reading Sensor Data

3.1 Joint States

robot.get_joint_states() returns a dict with the current state of every joint:

states = robot.get_joint_states()
for s in states["states"]:
    print(f"{s['name']:30s}  pos={s['motor_position']:.3f} rad  "
          f"current={s['motor_current']:.2f} A")

Use motor_position (not position) for the actual joint angle in radians.

3.2 Camera Images

The GDK Camera module covers all cameras on the head and wrists. Images are retrieved by CameraType.

Color frame (JPEG-compressed — always decode, never reshape):

camera = gdk.Camera()
time.sleep(2)

import cv2, numpy as np

obj   = camera.get_latest_image(gdk.CameraType.kHeadColor, 1000.0)  # 1000 ms timeout
frame = cv2.imdecode(np.frombuffer(obj.data, bytes), cv2.IMREAD_COLOR)
# frame is now a standard BGR uint8 numpy array (H × W × 3)

Depth frame (raw uint16 millimetres — reshape directly):

obj   = camera.get_latest_image(gdk.CameraType.kHeadDepth, 1000.0)
depth = np.frombuffer(obj.data, np.uint16).reshape(obj.height, obj.width)
# depth[y, x] is the distance in millimetres
depth_m = float(depth[240, 320]) / 1000.0  # centre pixel, metres

In normal mode only head stereo, head color, head depth, and hand color cameras are active. Fisheye cameras require developer mode.

3.3 IMU

imu  = gdk.Imu()
time.sleep(2)

data = imu.get_latest_imu(gdk.ImuType.kImuFront, 500.0)
print(f"accel: {data.linear_acceleration.x:.3f}, "
      f"{data.linear_acceleration.y:.3f}, "
      f"{data.linear_acceleration.z:.3f}  m/s²")
imu.close_imu()

3.4 Ultrasonic Radar

radar = gdk.UltrasonicRadar()
time.sleep(1)

data = radar.get_latest_ultrasonic_radar()
for sensor in data["ultrasonic_radar_datas"]:
    if sensor["fault_state"] == 0:   # 0 = healthy
        print(f"sensor {sensor['id']:2d}: {sensor['distance_mm']} mm")
radar.close_ultrasonic_radar()

4. Coordinate Transforms (TF)

The G02’s coordinate system is rooted at base_link — the centre of the chassis. Every joint, sensor, and camera has its own named frame. The TF module lets you look up any transform at runtime.

tf = gdk.TF()
time.sleep(2)

# Where is the right wrist in base_link?
pose = tf.get_tf_from_base_link("arm_r_end_link")
print(f"Right wrist: x={pose.translation.x:.3f}  "
      f"y={pose.translation.y:.3f}  z={pose.translation.z:.3f}")

# List every available frame name
frames = tf.get_all_frame_names()
print(frames)

Back-projecting a depth pixel to base_link (the same math used in the grasp pipeline):

import math, numpy as np

def pixel_to_base(px, py, depth_m, fx, fy, cx, cy, cam_to_base_4x4):
    x = (px - cx) * depth_m / fx
    y = (py - cy) * depth_m / fy
    p = cam_to_base_4x4 @ np.array([x, y, depth_m, 1.0])
    return p[:3]  # (x, y, z) in base_link metres

cam_to_base_4x4 is built from tf.get_tf_from_base_link(parent_frame) plus any sensor-to-camera extrinsic calibration file stored in the sensor/ directory.

5. Moving the Robot

5.1 Joint Path Planning Mode

Use move_arm_joint, move_waist_joint, and move_head_joint to move joints along a planned trajectory. The robot handles collision checking and interpolation internally.

import agibot_gdk as gdk
import time

gdk.gdk_init()
robot = gdk.Robot()
time.sleep(2)

# Move right arm joints to a target position (7 joints, radians)
joint_names = [
    "idx61_arm_r_joint1", "idx62_arm_r_joint2", "idx63_arm_r_joint3",
    "idx64_arm_r_joint4", "idx65_arm_r_joint5", "idx66_arm_r_joint6",
    "idx67_arm_r_joint7",
]
target_pos = [-1.5708, -1.5708, 1.5708, -1.5708, -1.5, 0.0, 0.0]
velocities  = [0.3] * 7

req = gdk.JointControlReq()
req.life_time        = 6.0           # seconds to reach target
req.joint_names      = joint_names
req.joint_positions  = target_pos
req.joint_velocities = velocities
robot.joint_control_request(req)
time.sleep(6)   # wait for joints to settle

gdk.gdk_release()

5.2 End-Effector Pose Control (50 Hz)

For Cartesian manipulation tasks send EndEffectorPose commands at 50 Hz. The kBothArms group controls both wrists simultaneously — the idle arm must be held at its current pose.

# Read current wrist poses
pose_r = tf.get_tf_from_base_link("arm_r_end_link")
hold_pos = [pose_r.translation.x, pose_r.translation.y, pose_r.translation.z]
hold_ori = [pose_r.rotation.x, pose_r.rotation.y, pose_r.rotation.z, pose_r.rotation.w]

# Build and send one command frame
req = gdk.EndEffectorPose()
req.group     = int(gdk.EndEffectorControlGroup.kBothArms)
req.life_time = 0.18   # seconds — command expires if not refreshed

# Right arm target
req.right_end_effector_pose.position.x    = 0.50
req.right_end_effector_pose.position.y    = -0.20
req.right_end_effector_pose.position.z    = 0.90
req.right_end_effector_pose.orientation.x = 0.0
req.right_end_effector_pose.orientation.y = 0.707
req.right_end_effector_pose.orientation.z = 0.0
req.right_end_effector_pose.orientation.w = 0.707

# Left arm — hold in place
req.left_end_effector_pose.position.x    = hold_pos[0]
req.left_end_effector_pose.position.y    = hold_pos[1]
req.left_end_effector_pose.position.z    = hold_pos[2]
req.left_end_effector_pose.orientation.x = hold_ori[0]
req.left_end_effector_pose.orientation.y = hold_ori[1]
req.left_end_effector_pose.orientation.z = hold_ori[2]
req.left_end_effector_pose.orientation.w = hold_ori[3]

robot.end_effector_pose_control(req)

For smooth motion, interpolate over many frames at 50 Hz rather than jumping directly to the goal:

import time, math

def smoothstep(t):
    t = max(0.0, min(1.0, t))
    return t * t * (3.0 - 2.0 * t)

def move_ee_smooth(robot, tf, arm, goal_pos, goal_ori, duration=2.0):
    """Move one arm's end-effector smoothly to goal_pos/goal_ori."""
    frame = "arm_r_end_link" if arm == "right" else "arm_l_end_link"
    start = tf.get_tf_from_base_link(frame)
    sp = [start.translation.x, start.translation.y, start.translation.z]
    sq = [start.rotation.x, start.rotation.y, start.rotation.z, start.rotation.w]

    n      = max(int(duration * 50), 2)
    t_step = duration / n

    for i in range(n):
        t   = smoothstep(float(i) / (n - 1))
        pos = [sp[j] + t * (goal_pos[j] - sp[j]) for j in range(3)]
        # (orientation slerp omitted for brevity — see grasp_pipeline.py _slerp())

        req = gdk.EndEffectorPose()
        req.group     = int(gdk.EndEffectorControlGroup.kBothArms)
        req.life_time = 0.18
        # ... set position/orientation fields ...
        robot.end_effector_pose_control(req)
        time.sleep(t_step)

5.3 Gripper Control

Grippers are driven as tool joints via robot.move_ee_pos():

def set_gripper(robot, arm, gripper_type, position):
    js = gdk.JointStates()
    js.group       = f"{arm}_tool"      # "right_tool" or "left_tool"
    js.target_type = gripper_type       # "omnipicker", "dahuan", "ctek90d"
    s = gdk.JointState()
    s.position = float(position)
    js.states  = [s]
    js.nums    = 1
    robot.move_ee_pos(js)

# Open / close examples (positions are gripper-type specific)
set_gripper(robot, "right", "dahuan", 0.04)   # open
set_gripper(robot, "right", "dahuan", 0.00)   # close

5.4 Chassis Driving

pnc = gdk.Pnc()
time.sleep(2)

# Request Ackermann (forward/back) or crab-walk mode first
pnc.request_chassis_control(0)   # 0=Ackermann, 1=crab

# Send a velocity command
twist = gdk.Twist()
twist.linear.x  = 0.3   # m/s forward (Ackermann) 
twist.angular.z = 0.1   # rad/s turn
pnc.move_chassis(twist)

6.1 Build a Map

slam = gdk.Slam()
time.sleep(2)

slam.start_mapping()
# Drive the robot around using remote control (see QSG) or move_chassis()
# ...
slam.stop_mapping()   # saves the map

6.2 Navigate to a Waypoint

Navigation requires the robot to be relocalized using the G02 Pad app before calling any Pnc navigation method.

pnc = gdk.Pnc()
time.sleep(2)

target = gdk.NaviReq()
target.pose.position.x    = 3.0    # metres in map frame
target.pose.position.y    = 1.5
target.pose.orientation.w = 1.0    # facing forward

pnc.normal_navi(target)

# Poll until the task completes
import time
while True:
    state = pnc.get_task_state()
    print(f"state={state.state}")
    if state.state in (7, 8, 9):   # cancelled / failed / success
        break
    time.sleep(0.5)

Task state values: 0=idle, 1=starting, 2=running, 3=pausing, 4=paused, 5=resuming, 6=cancelling, 7=cancelled, 8=failed, 9=success.

7. Worked Example — Object Grasping Pipeline

grasp_pipeline.py (included alongside this guide) is a complete, runnable example that:

Detects a target object with YOLO on the head color camera
Estimates its 3D location by combining depth data + TF transforms
Plans a 5-stage Cartesian trajectory (approach → lower → slide → grasp → lift)
Executes the motion at 50 Hz via end_effector_pose_control()
Verifies the grasp with gripper current sensing and a visual re-check

7.1 Architecture Overview

main()
 ├─ load calibration (intrinsic + extrinsic JSON)
 ├─ load YOLO model
 ├─ gdk_init()  +  Robot / Camera / TF modules
 └─ grasp_loop()
      ├─ plan_grasp()   ← perception + 3D planning
      │    ├─ _detect_stable()   5-frame median YOLO detection
      │    ├─ _depth_body_patch()   robust depth sample from bbox body
      │    ├─ _pixel_to_base()   back-project to base_link
      │    └─ build waypoints list
      ├─ execute_grasp()   ← motion
      │    └─ _move_ee()   50 Hz smoothstep interpolation
      ├─ check_gripper_holding()   current + position check
      ├─ check_target_lifted()    visual confirmation
      └─ place_object() / reset_arm() / move_to_home()

7.2 Run It

# Dry-run: perception + planning only, no motion (safe to try first)
python grasp_pipeline.py --dry-run --target bottle

# Live grasp loop, head camera, omnipicker gripper (default)
python grasp_pipeline.py --target bottle

# Wrist camera, Dahuan gripper, live MJPEG stream on port 5000
python grasp_pipeline.py --camera hand-right --gripper dahuan --stream

Required files in sensor/ next to the script:

intrinsic_head_front_depth.json — depth camera intrinsics (Fx, Fy, Cx, Cy)
extrinsic_end_T_head_front_rgbd.json — RGBD-to-head extrinsic (rotation + translation)

7.3 Key Patterns to Study

Pattern	Location in file	What it teaches
GDK init/release with `try/finally`	`main()`	Guaranteed cleanup
JPEG-decode vs raw-reshape	`_color_frame()`, `_depth_frame()`	Image encoding differences
TF + extrinsic matrix chain	`_cam_to_base()`	Camera → world geometry
50 Hz smoothstep trajectory	`_move_ee()`	Smooth Cartesian motion
`kBothArms` idle-arm hold	`_move_ee()`	Why you must always send both arms
Gripper hold detection	`check_gripper_holding()`	Using current + position to infer state
Workspace safety gate	`_check_workspace()`	Preventing out-of-range commands
Waist scan on detection failure	`scan_for_target()`	Graceful recovery pattern

8. Common Pitfalls

Pitfall	What goes wrong	Fix
Skip `time.sleep(2)` after module init	First API call returns stale/empty data	Always wait for DDS to connect
Release modules after `gdk_release()`	`RuntimeError` / crash	Close modules first, then `gdk_release()`
Reshape color image as raw bytes	Corrupted image (color frames are JPEG)	Use `cv2.imdecode()`, not `reshape()`
Navigate without relocalization	`PNC` command silently fails	Relocalize on the Pad app first
Call `switch_map()` during navigation	Undefined behaviour	Cancel the task before switching maps
Timestamps in milliseconds	Wrong time-interpolated TF results	All GDK timestamps are nanoseconds
Call unimplemented methods	`RuntimeError`	`get_imu_fps()`, `get_lidar_fps()`, `high_precision_navi()`, `record_spec_loc()` are not yet active in v2.6.3
Send `EndEffectorPose` once and stop	Arm freezes mid-motion	`life_time` expires; refresh at 50 Hz
Move end-effector outside workspace	Robot stops or collides	Check bounds before commanding

9. Minimal Hello-Robot Checklist

Robot powered on (see Quick Start Guide)
Dev PC connected via Ethernet, IP configured
agibot_gdk installed: python -c "import agibot_gdk; print('ok')"
GDK portal reachable: http://10.42.1.101:8849/site
Run the skeleton from §2 and confirm joint count prints correctly
Grab a color frame and display it with cv2.imshow()
Try --dry-run with grasp_pipeline.py to verify perception without risk