Senior Software Engineer - AI Gateway Platform

As a Senior Software Engineer on the AI Gateway Team, you will design, enhance, and scale Vercel’s AI Gateway platform—a unified API layer providing seamless access to hundreds of AI models across major providers like OpenAI, Anthropic, and Google. In this high-concurrency role, you will build low-latency systems handling intelligent failover routing, rate limiting, and unified billing/analytics across multi-provider LLM infrastructure. You will work with TypeScript and backend distributed architectures to guarantee production reliability and automatic fallback mechanisms during upstream provider outages.

Key Responsibilities

• AI Gateway Platform Architecture: Contribute to the design, execution, and scaling of Vercel’s AI Gateway platform—implementing unified API endpoints, rate limiting algorithms, caching layers, and dynamic failover mechanisms.
• Low-Latency High-Throughput Engineering: Write clean, modular, and well-tested TypeScript/JavaScript code, maintaining low-latency response times for high-volume AI inference requests.
• Provider & Model Integrations: Partner with product managers, AI researchers, and cloud infrastructure teams to integrate emerging LLM providers and models into a single developer-agnostic API.
• Open-Source & Developer Ecosystem: Actively contribute to open-source developer tooling (including the Vercel AI SDK and related ecosystem packages), adhering to Vercel's ethos of open developer collaboration.
• Analytics & Platform Features: Build platform features based on user telemetry—such as unified billing metrics, provider-agnostic authentication flows, and detailed usage analytics.
• Production Reliability & Operations: Ensure high-availability standards (automatic fallbacks during provider downtime, circuit breaking, and resilience under high request concurrency).

Requirements

Technical Systems & Engineering Depth

• Experience Track: 5+ years of professional software engineering experience building backend systems, APIs, or cloud platforms.
• TypeScript & Node.js Mastery: Strong proficiency in JavaScript/TypeScript paired with extensive experience developing scalable backend services and APIs.
• High-Throughput Infrastructure: Proven experience working with distributed systems, high-concurrency architectures, rate-limiting frameworks, distributed caching, or API gateways.
• AI/ML Integration Experience: Hands-on experience integrating AI/ML APIs, LLM providers, vector stores, or building AI-powered developer tooling.
• Open-Source Contribution: Demonstrated track record of participating in or contributing to open-source software projects.
• Collaboration & Remote Work: Excellent communication abilities with proven success collaborating across distributed, remote engineering teams.

Preferred Qualifications

• Direct experience building middleware or API proxies handling high-volume LLM inference workloads.
• Active maintainer or contributor status within the Vercel AI SDK, Next.js, or broader Web ecosystem.
• Familiarity with modern serverless platforms, edge compute networks, and cloud deployment pipelines.

Total Rewards & Benefits

• Compensation & Equity: Market-leading base salary ($196,000–$294,000 USD SF range) + generous Vercel equity compensation + potential variable performance bonuses.
• Comprehensive Healthcare: Fully inclusive healthcare insurance covering medical, dental, and vision for employees and dependents.
• Work-From-Home Support: Dedicated work-from-home setup budget + company-provided hardware and tech gear.
• Professional Growth: Mentorship, conference sponsorship, and learning stipends to attend industry events and build technical skills.
• Time Off: Flexible Time Off (FTO) policy to relax and recharge.