AI Supplier Magic

Automated supplier data pipeline that ingests, enriches, and publishes product catalogues directly to Shopify.

The Problem

Managing product catalogues from multiple suppliers is a grind. Every supplier delivers data differently — CSV files over FTP, JSON feeds, or proprietary APIs — each with their own field names, formats, and quirks. Descriptions are thin on SEO value, duplicate across variants, and written in American English for a British market. Someone had to manually reformat spreadsheets, rewrite hundreds of descriptions, and create each product and its variants one by one in the store admin. It didn’t scale, and the margin for error grew with every import.

For merchants running large or frequently updated catalogues, this wasn’t a minor inconvenience — it was a bottleneck that directly limited how fast they could get product live and trading.

How It Works

Supplier data arrives in three ways: CSV or JSON files uploaded manually, files fetched automatically via FTP on a schedule, or pulled directly from supplier APIs. Regardless of source, the system normalises the incoming data into a consistent internal format, grouping rows and records into products with their correct size and colour variants automatically.

Each product description is then passed to an AI model to be rewritten — more engaging, SEO-optimised, and converted to British English/relevant language — while technical specifications are preserved untouched. The finished product record is pushed to Shopify via API, with inventory, metafields, and tagging handled automatically. A background scheduler processes data in chunks to stay within API rate limits. A real-time dashboard shows progress, skipped duplicates, and a full log of every action taken.

Tech Stack

Frontend Flask/Jinja2 (server-rendered), Bootstrap

Backend Python, Flask, SQLAlchemy, PostgreSQL, APScheduler

Integrations Shopify REST & GraphQL APIs, FTP, Supplier REST APIs, JSON feed parsing

Outcomes

• Manual product data entry eliminated entirely for supported supplier formats – 98% reduction in time spent on product management.
• Thousands of products with full variant structures, rewritten descriptions, and SEO metadata / Google shopping attributes processed in a single run
• Duplicate SKU detection keeps catalogues clean across repeated imports
• In active use across merchant stores; SaaS release in progress

Lessons Learned

Chunked background processing was non-negotiable — large CSVs handled within a single web request hit timeouts immediately. The scheduler-based approach proved robust, though a formal job queue like Celery with Redis would be the call next time for better failure visibility and retry handling.

Supplier CSV formats are never as consistent as promised. Building format-specific parsers with clean separation paid off quickly when edge cases emerged — and they always do. The same applies to JSON feeds and supplier APIs, where field naming conventions vary wildly even within the same industry.

The AI rewriting step needed careful prompting to preserve technical specification sections. Early versions would helpfully “improve” torque ratings and thread sizes. Explicit instructions to treat spec blocks as untouchable solved it, but it’s a good reminder that AI enrichment pipelines need guardrails around structured data, not just free text.