ShopAgent AI Shopping Assistant

AI-powered personal shopping assistant that finds and surfaces real, buyable products across Shopify’s & Amazon’s merchant ecosystem through natural conversation.

The Problem

Online shopping is fragmented. Millions of products exist across thousands of independent Shopify stores, but discovery still relies on keyword search boxes that require you to know exactly what to type. Shoppers with vague intent — “something minimalist for summer, under $100” — have no good tool. Platforms like Google Shopping and Amazon surface ads, not curation.

There was no conversational layer that could take a natural description, or a photo, and actually surface real buyable products.

How It Works

A user opens the app, completes a short onboarding covering style preferences, sizing, budget, and shipping location, then lands in a chat interface. They describe what they’re looking for in plain language.

Before searching, a custom Context Capture Engine evaluates the query. If it’s missing critical details — a waist size for jeans, for example — the AI asks conversationally rather than returning irrelevant results. Once it has enough signal, it calls Shopify’s Discovery API with structured filters and streams real product cards directly into the chat.

Users can also take or upload a photo. The backend sends it to OpenAI’s vision model, which generates a descriptive search query and passes it into the same product search pipeline. Results include real-time local currency conversion and direct checkout links to merchant pages.

The backend is a thin Express orchestration layer: it holds the system prompt, manages conversation history, owns the Shopify API credentials, and calls OpenAI with function calling enabled — so the model can decide when to search versus when to ask a follow-up question.

Tech Stack

Frontend Expo (v54), React Native 0.81.5, React 19, Expo Router, TanStack Query, React Context, AsyncStorage

Backend Node.js, TypeScript, Express v5, RESTful API, esbuild

AI OpenAI GPT (chat completions with function calling), OpenAI Vision (image-to-search-query), OpenAI Whisper/TTS (voice input/output), Custom Context Capture Engine

Database & Storage PostgreSQL, Drizzle ORM, in-memory fallback for dev/testing

Integrations Shopify Discovery API, open.er-api.com (currency conversion), Zod (schema validation), Replit (hosting)

Outcomes

• Core chat-to-product loop functional end-to-end
• Visual search operational — photo input generates and executes a product query
• Onboarding personalisation feeds directly into search filters
• Architecture has validated that conversational discovery meaningfully outperforms keyword search for open-ended and style-based queries
• Currently in beta

Lessons Learned

The Context Capture Engine turned out to be the most important and most underestimated piece of the build. Without it, the AI searched immediately on vague input and returned garbage results — eroding trust fast. A lightweight “do I have enough to search?” layer before hitting the API was the right call and would be the first thing replicated in any similar product.

Function calling works well for tool-use patterns, but the system prompt requires careful calibration. The model needs explicit instruction about when not to search, or it over-triggers on casual conversation. Prompt engineering here was more iterative than expected.

Streaming product cards directly into the chat — rather than a separate results screen — was a UX bet that paid off. It feels like a conversation rather than a search engine, and that pattern is worth carrying forward into any AI-native interface.