Caterpillar is advancing its integration of artificial intelligence and automation into construction equipment through a new collaboration with the semiconductor leader Nvidia. The company is currently testing an AI assistive system in its mid-size Cat 306 CR Mini Excavator. This system, known as “Cat AI,” was developed using Nvidia’s Jetson Thor physical AI platform and is being demonstrated at CES.
According to Brandon Hootman, Caterpillar’s vice president of data and AI, Cat AI is powered by a fleet of AI agents. It can assist machine operators by answering questions, providing access to resources, offering safety tips, and scheduling services. A major advantage of implementing this technology is the valuable data these systems gather and transmit.
“Our customers don’t live in front of a laptop day in and day out; they live in the dirt,” Hootman explained. “The ability to get the insights and take the action that they need while they’re doing the work is very important to them.”
In a parallel initiative, Caterpillar is piloting digital twins of construction sites. Using Nvidia’s Omniverse library of simulation resources, the company can test scheduling scenarios and more accurately calculate the building materials required for a project. Hootman noted that Caterpillar’s machines send approximately 2,000 messages back to the company every second, and this data will be instrumental in building these simulations.
Caterpillar already operates fully autonomous vehicles in the mining sector. Hootman stated that these new pilot programs represent a significant next step as the company aims to expand automation across its portfolio.
“The reason that we started here was it was a real challenge of our customers today that needed to be addressed, and also something that we had some real momentum on and we felt like we could bring to market pretty quickly,” Hootman said. “What we also liked is that it provided a technology foundation for us to then build upon.”
Partnerships with established industrial brands like Caterpillar align with Nvidia’s strategic focus on physical AI. Bill Dally, Nvidia’s chief scientist, previously described physical AI as the next frontier for the company and its powerful GPUs.
During its CES keynote, Nvidia detailed plans for its full-stack ecosystem for physical AI. This ecosystem includes open AI models like the company’s Cosmos model family, simulation tools, and developer kits.
Deepu Talla, Nvidia’s vice president of robotics and edge AI, emphasized that the company defines physical AI broadly, as many industries are now incorporating robotics. “Physical AI is the next wave of AI,” Talla said. “Nvidia is pioneering that with computers that train the models, that do the simulation to test the models and deploy the models into the robots, whether that’s an autonomous car or a Caterpillar machine.”