Instacart developed Maple, a service designed to streamline large-scale processing of LLMs across the company, addressing challenges such as rate limitations and duplicated efforts in AI workflows. Maple automates batching, encoding, file management, and retries, allowing teams to efficiently process millions of prompts while significantly reducing costs and enhancing productivity. By abstracting complexities, Maple facilitates reliable and scalable AI operations within Instacart's infrastructure.