Firecrawl’s co-founder and CEO Caleb Peffer knew the exact moment he found the investor to lead his Series A. He was in a coffee meeting with Nexus Venture Partner’s Abhishek Sharma at the Blue Bottle in San Francisco’s South Park. While describing the future of the company, he was gesturing so animatedly that his chair tipped over. He actually fell out of his chair. Abhishek, as a great investor does, caught the chair and him as he was falling. Peffer described the incident with a laugh, noting it felt like a symbol for how the founder and investor relationship is supposed to work. He saw it as a clear signal that Sharma was the right partner.
On Tuesday, Firecrawl announced a Series A funding round led by Nexus with participation from Shopify CEO Tobias Lütke and existing investor Y Combinator.
Firecrawl offers a popular open-source web crawler for developers and AI agents, with a commercially supported version available through an API. It is used by 350,000 developers, has nearly 50,000 stars on GitHub, and its notable customers include Shopify, Replit, and Zapier as well as some of the largest hedge funds in the world. Peffer told TechCrunch that the company is already profitable.
The startup also just released an API that supports search and will soon add support for natural language prompts, as noted by co-founder and CTO Nicolas Silberstein Camara.
Peffer, who co-founded Firecrawl in 2022 with Camara and CMO Eric Ciarla, said gaining Lütke as an investor was the best kind of validation. They landed him through a gutsy email after discovering the Shopify founder had signed up to try their product through their self-service portal. The Firecrawl crew saw his email come in and immediately emailed him to welcome him as a customer, but got no response. Two months later, some Shopify people contacted Firecrawl for an enterprise contract. Peffer took his shot and emailed Lütke again, mentioning they would love for him to participate in their upcoming round. This time, Lütke responded with compliments about their product and agreed to join.
While AI web crawlers have a somewhat dubious reputation these days, mostly due to bad actors that ignore robot.txt files, they are also a necessary part of the burgeoning AI world. AI trains on the web, agents need to access web pages to perform their duties, and enterprises need personalized crawlers to consume their own websites for training and operations.
The Firecrawl founders hope to help address the frustrating parts of their industry. They are working on tools to help website owners, publishers, and other content creators get paid when AI uses their content. Peffer said they believe this is the way it should be.
While there have been many efforts around this idea from big names like Adobe and Getty as well as several startups, Peffer feels that Firecrawl has an edge because it is already working with those who are scraping data. He stated that they already have one side of the marketplace and what they want to do is connect that side to the website owners and the publishers.
Interestingly, Firecrawl went viral a few months ago for reasons that have little to do with their open source tool. They had posted an ad to the Y Combinator job board looking to hire an AI agent as an employee for a $15,000 salary, perhaps the first job ad for an agent employee ever.
That job search did not yield an agent worth hiring, so Firecrawl upped the budget to one million dollars to hire several agents and the developers who built them, and tried again. Applicants flooded in, but the company has not yet hired any. The founders realized that evaluating and managing would-be agent employees is a job in itself. They are now looking for an AI chief of staff.
Firecrawl’s Caleb Peffer will be on stage at TechCrunch Disrupt in October to discuss all that he has learned in a session that covers the pros and cons of hiring AI agents as early employees.