Overview: EGC Supply faced a critical growth ceiling when their Magento platform capped inventory at just 100,000 items. Ceymox re-architected the backend to process 2.5 million+ SKUs, delivering enterprise-grade scalability, lightning-fast performance, and a massive boost in organic search visibility.
At a Glance
- Client: EGC Supply
- Industry: Industrial Tools & Supplies
- Services: Adobe Commerce Optimization, B2B Solution Architecture, Performance Enhancement
- Platform: Adobe Commerce Cloud (Magento 2)
1. The Client
EGC Supply is a leading industrial distribution company providing a large assortment of industrial supplies including cutting tools, abrasives, fasteners and safety products to a variety of industries.
Inventory is the number-one revenue generator in the industrial B2B space; customers frequently search for very specific product numbers such as a particular drill bit diameter or a specific fastener type. In order to be competitive, EGC Supply needed to provide an “endless aisle” online that matched the massive physical inventory of over 2.5 million unique items.
2. The Challenge
Despite being one of the largest inventories in the market, EGC Supply’s digital growth was severely limited due to technical constraints. The standard Adobe Commerce architecture had created three major roadblocks:
- The 100k SKU Barrier: Standard import scripts validated data sequentially and consumed excessive memory and CPU resources. This resulted in processes crashing prior to completion and prevented importing 96% of EGC’s inventory, thus rendering millions of available items invisible to customers shopping online.
- Performance Degradation: The sheer volume of data being processed by the database caused table locking during updates, which resulted in significant latency, slow page loads, and a terrible user experience that drove customers away.
- Stagnant Data: Because the company was unable to reliably process bulk updates, the IT department was unable to rely on accurate pricing and inventory levels. Instead, they were constantly monitoring crashing scripts and unable to focus on innovation.
3. Solutions Offered By Ceymox
Ceymox’ B2B ecommerce development team performed a forensic review of the existing infrastructure and designed a custom high-performance architecture on Adobe Commerce Cloud to support extremely high volumes of data.
- Custom High Volume Import Logic (Async & Parallel Processing): Unlike the traditional sequential processing methods used in standard synchronous processing, we have developed custom scripts that utilize Magento’s Asynchronous Bulk API’s and RabbitMQ (Message Queues). These scripts divide the data into manageable pieces and process them concurrently in the background, thereby enabling the ingestion of millions of records at once without running out of time or locking up the database.
- Incremental “Delta” Strategy: The most efficient way to manage a catalog of this magnitude is through a delta strategy. Our team has built a middleware solution that compares supplier files against the database. Rather than processing every item, our system now ignores items that have not changed and only processes the deltas (price and/or quantity changes), resulting in a 90% reduction in daily data processing volume.
- Speed & Database Optimization: To ensure that the front end continued to remain fast, we implemented aggressive Varnish caching techniques to immediately deliver static content. Additionally, we have optimized Elasticsearch to quickly perform complex queries across millions of rows and optimized SQL indexing to prevent server lag during peak usage periods.
- SEO & Structure Re-engineering: A listing of 2.5 million products generates a tremendous amount of digital real estate. We redesigned the URL structure and optimized XML sitemap generation to break down the sitemap into much smaller, more consumable parts. By doing so, we allowed Google bots to crawl and index the millions of new deep link pages without slowing the site down due to speed or duplication of content penalties.
4. The Result
The interventions provided by Ceymox dramatically improved EGC Supply’s digital capabilities and turned what was once a technical liability into a competitive advantage.
- 25x Catalog Expansion: We were able to increase the number of SKUs available online from 100,000 to over 2.5 million SKUs, therefore expanding EGC’s reach into the long tail search market.
- Speed & Stability: Dramatic improvements in page load time and elimination of timeouts enabled the site to operate at maximum capacity during both normal and heavy administrative update cycles.
- SEO Dominance: With millions of previously unseen products now indexed, organic traffic and search visibility increased.
- Operational Agility: EGC can now run automated daily updates for stock and pricing without fear of site crashes and without requiring manual intervention.
5. Technical Stack
- Platform: Adobe Commerce Cloud (Magento 2)
- Database: MariaDB (Optimized for high-concurrency)
- Search Engine: Elasticsearch / OpenSearch (Configured for massive catalogs)
- Caching: Varnish (Full Page Cache) & Redis (Backend/Session Cache)
- Message Queue: RabbitMQ (For asynchronous bulk import processing)
- CDN: Fastly (For global asset delivery and image optimization)
The Impact
Volume is nothing without performance. With a highly scalable architecture to support very high levels of data, Ceymox were able to unlock the full potential of the Adobe Commerce platform for EGC Supply’s large industrial catalog. The success of this project proves that it is possible to mold Magento to scale to the level required by the largest of industrial catalogs.
