Engineer cuts LLM costs 87–92% for product classification by combining five techniques—from $200/month to $25–40/month on 1M monthly products

What happened：A software engineer optimized an e-commerce product categorizer (a system that sorts products into category hierarchies) by applying five successive improvements: context compression, two-stage hierarchical classification, exact-match lookup, similarity caching, and batch processing. The result: token usage per product dropped from ~25,000 to ~100 on average, and monthly costs fell from $200+ to $25–40.

How it works / what's different：Instead of sending the entire 30,000-category tree to the LLM every call, the engineer: (1) compacted the data format (JSON → simple indented Name|ID pairs), cutting context 52%; (2) classified in two stages—first pick a root category from 30 options (~300 tokens), then classify within only that subtree (~900 tokens)—eliminating 95% of unnecessary context; (3) cached exact product name matches (sub-millisecond DB lookups); (4) used Postgres trigram similarity to find near-duplicate names already classified, serving ~40% of remaining products from the database at zero token cost; (5) batched novel products together so the root category list was sent once instead of repeated for every product.

So what — impact on the reader：For anyone running LLM-based classification, search, or tagging systems—product categorizers, content moderation, document routing—these techniques are immediately applicable and provider-agnostic. The techniques shift cost from repeated LLM calls to cheap database lookups, making large-scale LLM applications economically viable. A business processing 1M+ items monthly can reduce AI infrastructure costs by 80–90% without sacrificing accuracy.