Decisions API gpt-6-luna drives SO-101 arm via repeated choices

The project builds directly on an earlier experiment in which the author drove an SO-101 arm with Claude and MCP without any robot-specific trained model. That worked — the arm grabbed an orange iguana toy — but each decision took several seconds and had to be repeated dozens of times, making the whole process slow. The new attempt replaces free-form model output with a typed API that returns probabilities for fixed questions, which the author says answers more stably and improves the ability to make the robot do nothing when confidence is low.

The write-up is candid about where the difficulty actually sits. Directly adding joint angles made the arm crawl along the table, because rotating the shoulder and elbow also changes the gripper's height. Switching to inverse kinematics fixed the concept but surfaced new problems: the wrist rolled from −75° to −145° as a side effect, the arm stalled 30cm from its base, a camera-mounted view made "image left" drift, and the arm once moved opposite to the intended direction because the mapping between image directions and table directions depended on how the camera was mounted. The author eventually removed the human from the loop by having the arm pose itself and measure its own camera orientation using the gripper as a marker, and by reading the real joint limits recorded by lerobot-calibrate rather than trusting the URDF.

The results so far are presented as a working pipeline rather than a completed task. The author notes that the run is being logged to runs/ so that successful attempts can later be used as imitation-learning data, and floats a future hybrid where a learned policy such as π0.5 handles fine positioning while the Decisions API is reserved for judgments like whether the gripper actually caught the target.