Futuristic Hardware for Intelligent Robots

What advances in hardware do we need to build intelligent robots of the future?

This is the second of a series on the challenges and opportunities of applying AI to robotics, specifically in the setting of autonomous service robots that can assist humans in everyday tasks. See the overview here.

This editorial highlights state-of-the-art robot hardware designs and discusses what new hardware we need to build. Advances in sensors, actuators, compute, and batteries are all needed to develop truly autonomous and useful assistive robots. We will discuss sensors and actuators in this edition and save compute and batteries for a future post.

TL;DR: Intelligent robots need robust and multi-modal sensors to understand the world and powerful, precise, and compliant actuators to act on the world. For sensors, AI has been widely applied to better understand complex sensor signals. For actuators, mobile bases are a lot more mature than robot arms and hands. The most difficult and promising future robot hardware developments are in building advanced artificial skins and general-purpose human-level hands.

Robust and Multi-Modal Sensors

Pictured: A prototype tactile robot “skin”. Image Credits: CNN

AI for robots is special because it is embodied as a physical agent that perceives and acts in the physical world. To understand and react to the world intelligent robots need robust and multi-modal sensors - ones that give accurate information and can cover a wide range of scenarios. Most robots deployed today have almost no sensors - they can only repeat exact programmed motions, and they can’t react to anything. Sensing allows a robot to react and adapt.

An interesting and common recent theme is how we can use AI to better understand the signals of these sensors, and rely less on manually engineering complex algorithms to parse raw hardware outputs.

Vision Sensors

Inferring depth from stereo color images. Image Credits: Nvidia

The most easily understood sensors are vision sensors- devices that act like cameras - and the most common type is color sensors. Another class of widely used vision sensors is depth sensors (e.g. Kinect cameras). In color images, each pixel corresponds to the color (e.g. RGB) values seen by the camera at that point in space. For depth images, each pixel is the distance of that point in space to the camera. Depth sensing is incredibly useful for robots because it gives 3D geometric information about the world, something that is hard to accurately infer from only color images.

While commodity depth sensors have gotten very affordable in recent years, there are many technical limitations. Different sensors make different trade-offs among image resolution, frame rate, shutter speeds, and form factor. They also have pesky problems about maximum and minimum sensible depth (e.g. can’t sense objects <10cm away from the camera), as well as inability to sense reflective and transparent surfaces. Really accurate depth sensors also tend to be expensive, bulky, and slow (low framerate).

To solve these challenges, instead of relying on hardware-based depth sensing, many are now relying on AI to denoise depth or extract depth from color images. This is a promising approach that can enable cheap and robust depth sensing with applications in smartphones, self-driving cars, and of course assistive robots, and it’s definitely a direction we will see more of in the future.

Touch Sensors

The DIGIT touch sensor. Image Credits: Facebook

Another important type of sensors is touch sensors, useful for when things are not in the line of sight. For example, a robot using an arm to reach into a grocery bag will not be able to see the contents of the bag as the arm/hand is occluding them. Early forms of touch sensing focus on individual contact points - think really bulky touch screens that can only give a binary response on whether or not it sensing an applied force. These developed into smaller and more robust tactile grids, which can be made into fingertip touch sensors. Here AI is also used to better interpret signals from these touch sensors.

Recent years have seen camera-based touch sensing where a small camera is used to detect deformations on stretchable skin. Facebook (Meta)’s recent DIGIT sensor is the latest iteration. While impressive, these sensors are still quite limited by their range - the smallest and largest forces they can sense, as well as their robustness - the stretchable materials can be easily damaged and are not meant for extensive use. In the future, we need to develop robust, sensitive, high-res, and flexible artificial skins that can cover the entire surface of a robot.

Other Types of Sensors

The most common type of robot sensors used today is not vision, but actually proprioception, where the robot senses the location of its limbs via motor encoders or the location of its entire body via accelerometers/gyroscopes. Interestingly, we can actually use AI to predict contacts on robot arms from proprioception, something I’ve explored in my own work.

Proprioception sounds simple for rigid robots, but it is very difficult to do for soft robots, as the deformation of soft materials is hard to predict and track. AI can help in this case too.

Examples of other sensor modalities include magnetic sensing which can be used to make touch sensors, and microphones which can be use to localize objects and to identify object properties.

Currently, all these types of sensors are developed independently and scarcely used together. However, there is “one world” that the robot perceives and also “one robot” that houses all of these sensors. To realize the full potential of these sensors, future robot hardware will need to be designed to seamlessly integrate proprioception with vision, touch, and sound to jointly perceive and model the world.

Powerful, Precise, and Compliant Actuators

A prototype robot arm with synthetic muscle. Image Credits: Automation Robotics

More than just understanding the world, useful assistive robots also need to act on the world by physically moving things around, whether that’s moving the robot itself or objects in the world. It probably isn’t possible to design robots that can move everything and move everywhere. Fortunately, we just need assistive robots to operate in and interact with environments typically designed for humans, which have known properties and patterns that a robot system can exploit (e.g. we may know the rough sizes, shapes, and materials of common household items).

Good hardware actuators are important because 1) they allow robots to do more tasks and 2) they make designing the robot software/AI a lot easier.

Mobile Bases

Autonomous robots need to move around in the world, otherwise, everything they interact with needs to be brought to them (something we do in factories with conveyor belts).

The most common mobile robot platforms are probably wheeled bases because they’re relatively easy to build and control. Examples include the PR2, Fetch, Toyota HSR, and Stretch. The problem with wheeled robots is that they can only go where wheels can go, so they can’t traverse common settings like staircases. Something similar to wheels is Toyota’s ceiling-mounted robot that moves around on a track. Of course, this also limits the available workspace of the robot.

These challenges can be tackled with recent developments in legged locomotion, enabling humanoid robots that can use legs instead of wheels. The most famous examples are Boston Dynamic’s Atlas and Agility Robotics’ Digit. Boston Dynamics also sells a robot arm attachment to its commercially available quadruped Spot.

The fun thing about robot design is that the form factor does not have to be similar to humans or other animals. For example, Boston Dynamic’s Handle puts wheels on legs, and Caltech’s LEONARDO (video above) combines legs with drones.

Mobile robot bases of many types have been maturing quickly into commercialization. Aside from Spot, we have not yet seen other mobile robots with arms applied to commercially viable use cases. But this will probably happen very soon.

Manipulator Arms and Hands

When it comes to assistive robots that can autonomously operate in the real world, robot arms and hands are less mature than mobile bases. While we’ve had precise and powerful robot arms in factories for decades (pictured above), they are not very dexterous and compliant (no force feedback = not safe operating around humans or things that can break). These factory robot arms also tend to be bulky, expensive, and require trained operators.

The wave of collaborative robot (co-bots) wave started 10 years ago by Rethink Robotics aimed to solve these issues by making robot arms smaller, cheaper, and usable around humans. This trend has successfully continued, and now you can buy a reasonable robot arm for only $5k.

However, these robot arms are not yet ready to be widely deployed in the wild. This is mostly because we do not yet have robot arms that are cheap, small (human-arm sized), precise, compliant, and powerful. Some robots check off 3 of these requirements, but none check off all 5. Payload in particular is a big issue. The popular co-bots we have payloads around 0.5-2kg, which is not a lot (1kg ~= 2.2lbs. The RTX 3080 weighs 4.7lbs).

Even less mature than robot arms are robot hands. Most common “general-purpose” robot hands either have “parallel jaws” - basically large claws - or suction grippers (a powerful vacuum). While these suffice for pick and place type tasks common in warehousing, they’re not quite dexterous enough to operate in less structured settings like homes and offices. For robot hands, dexterity is the sixth requirement in addition to the 5 listed above. While some human-like hand designs satisfy dexterity, precision, smallness, and compliance, they do not satisfy being also cheap and powerful. There is a lot of future potential for building truly human-level hands.

So far we’ve only covered rigid robot arms and hands that use motors to actuate solid links, but there are other types of designs as well. Tendon-driven robots (like the synthetic muscle system pictured above) are one promising direction since they’re more lightweight (don’t need to put heavy motors at every joint), and they’ve been used extensively in teleoperated surgical robots. There is a lot of work in soft robot hands actuated by pneumatics, granular materials, and tendons. Many of these were designed with specific tasks and objects in mind, so it is not yet clear what the best designs should be for more general-purpose human-level hands.

Conclusion

Intelligent robots need robust and multi-modal sensors to understand the world and powerful, precise, and compliant actuators to act on the world. For sensors, AI has been widely applied to better understand complex sensor signals. For actuators, mobile bases are a lot more mature than robot arms and hands. Looking toward the future, my personal take is that the most challenging aspects of new robot hardware that can unlock the most potential are building advanced artificial skins and general-purpose human-level hands.

---

About the Author

Jacky Liang (@jackyliang42) is a PhD student at Carnegie Mellon University’s Robotics Institute. His research interests are in using learning-based methods to enable robust and generalizable robot manipulation.

---

Copyright © 2021 Skynet Today, All rights reserved.