Minimalist Tooling and "aim-and-Shoot" Skills for AI-Powered Robotic Manipulation

The field of robotic manipulation is having a major upgrade thanks to the recent breakthroughs in generalist artificial intelligence and the increasing demand for advanced automation at homes and workplaces. In terms of tooling for robotic manipulation, this work studies a unique configuration of robot end-effectors and skills. Tools as simple as a stick can fulfill a good variety of tasks in our daily lives when used creatively. Such tools often work with "aim-and-shoot"skills with which, as long as the tool is properly placed on the target object, the manipulation action is as simple as a single move. Other than benefiting affordability, reliability, and durability, the minimalism helps leverage the available ability of generalist AIs on physical reasoning and enables them to reason on physics at sub-object level. Based on a cognitive analysis of ChatGPT, this paper introduces a prompt-based teaching pedagogy that allows novice users to easily teach AI to make reliable decisions for aim-and-shoot skills. In addition, new grounding techniques are presented for quantifying the AI decisions and facilitating visual servoing. The proposed methods are validated using two skills associated with a stick tool to fulfill three tasks.