How Instacart is working with NVIDIA to build a continuous learning intelligence layer for omnichannel retail
For years, retailers have spoken about physical and digital commerce as separate channels. Instacart, a leading grocery technology company, believes that, in grocery, success in the future will be defined by how intelligently those worlds connect.
The goal is a continuous learning system that understands customers, baskets, shelves and store operations in real time, then using that intelligence to make shopping more personalised, efficient and profitable.
For David McIntosh, Instacart’s chief connected stores officer, this is the idea behind Connected Store, the company’s suite of enterprise technologies designed to digitise the in-store experience and link it with online commerce.
“Since the very beginning, our enterprise business has been fundamental to Instacart,” he says. “We power storefronts for hundreds of retailers in North America and growing globally.”
But as those relationships deepened, retailers began asking Instacart for help beyond ecommerce. “They were saying, ‘Instacart, you brought my business online, but I also have all of these problems in the store that I’m dealing with. I haven’t really digitised my store.’”
Instacart’s ambition, he adds, is that “in a five to 10 year horizon, customers won’t have to choose between shopping in store or online. It will be one single unified mode.”
A business of exceptions
That vision is compelling because grocery is so difficult to digitise well. Wi-Fi can be patchy, perhaps GPS breaks down indoors and even store lighting varies. Planograms are often incomplete or out of date. Shelves change constantly as customers shop, and staff replenish shelves and while brands rotate packaging. Even formats and seasonal ranges factor into a constantly-shifting environment.
And, a single store can carry tens of thousands of SKUs. Recognition becomes difficult when shoppers place products upside down, cover labels, stack items together or move them in and out of a basket at speed.
As McIntosh puts it: “The retail environment is a business of exceptions.”
That’s exactly why Instacart has invested in Physical AI, or AI systems designed to understand and act on real-world environments. Instacart’s Caper Carts AI smart trolleys that are fitted with basket-facing camera sensors, weights and measures-certified scales, location systems, outward-facing cameras and NVIDIA Jetson edge computing on every trolley.
Caper Carts look familiar because that’s part of the point. “We believe smart carts are the winning format of retail precisely because it so nicely integrates in the store,” says McIntosh. “You’ve already shopped with a shopping trolley. That’s a well-worn behaviour now for 80 years. It’s the same way you shop, but it’s better.”
The screen on each Caper Cart gives customers a running total, deals, discounts and recommendations, while sensors help the trolley understand what is being added, removed and purchased. Instacart says Caper Carts are now deployed across more than 100 US cities, tripling year-on-year, and coming soon to Morrisons in the UK.
For customers, the appeal is not only speed. “When I talk to customers in the store, the number one thing they tell me they love is the running total,” says McIntosh. “The number two thing is deals and recommendations, which make the experience more personal.”
Accuracy that earns trust
Caper Carts use multimodal sensor fusion, combining signals from cameras, scales, location systems and shelf-facing views. An edge encoder running on NVIDIA Jetson provides real-time feedback, while cloud vision-language model encoders support deeper reasoning.
Cameras can triangulate where items are in 3D space, but they can be blocked. Weight can reveal what has changed, but it can drift, such as if the cart moves, it may take a few seconds to stabilise. McIntosh compares it to a diving board. After someone jumps, “it takes time to stabilise. A lot can happen within three seconds.”
By fusing these signals, Instacart aims to create an accurate view of the basket, the customer’s location and the shelf environment. That matters because poor recommendations can do more harm than good.
“If you think the toothpaste is in one aisle, but in fact it’s two aisles over, you’re going to make the wrong recommendation,” says McIntosh. “Then customers are going to start to ignore the recommendations.”
This is where the edge-to-cloud model becomes important. Stores cannot rely on constant connectivity or slow cloud round trips. “Often, in a retail condition, you don’t have Wi-Fi half the time in the store,” McIntosh says. “When you put that thing in the trolley, you need to know instantly, in hundreds of milliseconds.”
The store as a measurable environment
The bigger prize is not just a smarter cart, but a more observable store.
Instacart captures millions of sensor inputs every day, from items entering and leaving baskets to cart movement, shelf conditions and screen engagement. Combined with more than 1.6 billion lifetime grocery orders, that creates what McIntosh describes as a continuously learning “data flywheel” allowing partners to harness Instacart’s grocery data expertise to understand and interpret their own data.
“The magic here is the purchase and location pair,” he says. “It’s ‘where am I buying it in the store?’ I’m actually grabbing it off the shelf. This goes well beyond heat maps in terms of value.”
For brands and retailers, that could mean understanding which displays drive sales, where products are bought, where gaps appear, and how assortment decisions should change by store. For store teams, it could eventually mean AI agents that flag shelves needing replenishment or identify out-of-stock risks.
“You start to make the store measurable,” says McIntosh. “In some ways, you can start to make it programmable.”
Retail media at the point of decision
The same intelligence also opens a potential retail media opportunity. In store, the Caper Cart screen can add context that online cannot by pulling in real-time data about store conditions and what’s already in the trolley.
“If a cart already has one beverage, we could enable advertisers to market different brands, flavours, or types of beverages,” says McIntosh. “It’s totally native as you’re in the store, as you’re shopping.”
The same intelligence also opens a potential retail media opportunity. In store, the Caper Cart screen can add context that online cannot by pulling in real-time data about store conditions and what’s already in the trolley.
“If a cart already has one beverage, we could enable advertisers to market different brands, flavours, or types of beverages,” says McIntosh. “It’s totally native as you’re in the store, as you’re shopping.”
Instacart’s intelligence layer
Instacart’s long-term vision is centered around its intelligence layer. A foundational AI system that understands not just products, but how grocery retail works as a living ecosystem. That means connecting product relationships, household preferences, store conditions, inventory, logistics, retail media and operational decision-making across the retailer’s omnichannel data.
NVIDIA’s role is central. NVIDIA Jetson runs at the edge on Caper Carts, Dynamo-Triton supports cloud ranking and personalisation, and Instacart expects to roll out Nemotron to help power agentic platforms.
The future of grocery may not be online or in store – it’s a unified experience connected by an intelligence layer.
Click here to sign up to Retail Gazette‘s free daily email newsletter
