For the past seven years, Runway has developed visual-generating tools for the creative industry. The company now sees a new opportunity for its technology in the field of robotics.
Based in New York, Runway is known for its video and photo generation AI world models. These are large language models that create a simulated version of the real world. The company most recently released its Gen-4 video-generating model in March and its Runway Aleph video editing model in July.
As Runway’s world models became more realistic, the company began receiving interest from robotics and self-driving car companies looking to use the technology. Anastasis Germanidis, Runway co-founder and CTO, explained that the ability to simulate the world is useful far beyond entertainment. He stated that this technology makes it more scalable and cost effective to train robotic policies that interact with the real world.
Germanidis said working with robotics and self-driving car companies was not something Runway initially envisioned when it launched in 2018. It was only after companies from these other industries reached out that Runway realized its models had much broader use cases than originally thought.
According to Germanidis, robotics companies are using Runway’s tech for training simulations. Training robots and self-driving cars in real-world scenarios is costly, time-consuming, and difficult to scale. While Runway knows it will not replace real-world training entirely, Germanidis said companies can get significant value from running simulations on its models because they can be incredibly specific.
Unlike real-world training, using these models makes it easier to test for specific variables and situations without changing anything else in the scenario. Germanidis explained that you can simulate the effect of different actions, such as what would happen if a car took a specific turn. Creating these tests from the same context is very difficult in the physical world, where it is hard to keep all other aspects of the environment constant while testing a single action.
Runway is not the only company working in this area. For instance, NVIDIA recently released the latest version of its own Cosmos world models and other robot training infrastructure.
The company does not plan to release a completely separate line of models for its robotics and self-driving car customers. Instead, Runway will fine-tune its existing models to better serve these industries and is building a dedicated robotics team.
Germanidis added that while these industries were not part of the company’s initial pitches to investors, they are now on board with this expansion. Runway has raised more than $500 million from investors like NVIDIA, Google, and General Atlantic at a $3 billion valuation.
He described the company as being built on a principle rather than a single market. That principle is the idea of simulation and building a better representation of the world. With powerful models, they can be used for a wide variety of different markets and industries. The industries that exist today are expected to change even more as a result of the power of generative models.