Lead Engineer - Data Platform & AI Agents

At Klaviyo, we empower creators to own their destiny by making first-party data accessible and actionable. Our data platform processes billions of events daily across billions of consumer profiles for hundreds of thousands of brands. Increasingly, this data powers autonomous AI agents acting on behalf of global enterprise customers.

As a Principal Data Platform Engineer, you will own the overall technical direction, system architecture, and delivery of Klaviyo’s data platform—spanning streaming ingestion, batch compute, storage/lakehouse, and schema governance. In this high-leverage individual contributor (IC) role, you will lead without direct reports by designing systems that are fast, reliable, cost-efficient, and secure for autonomous access. You will build standardized "paved roads" for data producers and consumers (human analysts and AI agents alike) while mentoring senior and staff engineers across the company.

Key Responsibilities

• Core Data Platform Architecture: Design and implement distributed data platform capabilities, including real-time event ingestion/CDC, stream processing, batch orchestration, lakehouse storage patterns, data catalogs, lineage, and access compliance.
• SLOs & Reliability Engineering: Define and uphold strict Service Level Objectives (SLOs) for data freshness (≥99.9%), availability, and correctness. Drive architecture decision records (ADRs), RFCs, readiness reviews, and post-incident learning.
• Data Governance & Contracts: Establish data contracts, schema governance frameworks, standardized connectors, and reusable pipeline templates to accelerate developer velocity across product teams.
• Performance & FinOps Optimization: Profile, tune, and right-size distributed compute clusters for performance and unit-economics cost efficiency in partnership with FinOps.
• AI-Augmented Data Operations: Apply GenAI and ML tools to streamline data workflows—enabling semantic catalog search, automated documentation, SQL/pipeline generation, and AI-assisted data quality anomaly detection under human-in-the-loop controls.
• Technical Mentorship & Enablement: Mentor Staff and Senior Engineers across pillars through architectural pairing, code reviews, and design doc oversight.

Requirements

Experience & Distributed Systems Mastery

• 10+ years of professional experience building and operating large-scale distributed data systems with multi-tenant reliability.
• Deep expertise across distributed data stack components:
  • Streaming & Messaging: Kafka, Pub/Sub, Flink, Spark Streaming, Beam.
  • Batch Orchestration & Engines: Airflow, Dagster, Spark.
  • Lakehouse & Warehouses: Apache Iceberg, Delta Lake, Hudi, Snowflake, BigQuery.
  • Storage & Ingestion: Object storage (S3/GCS), CDC pipelines, and schema registries.
• Proven track record establishing and meeting strict SLOs for data freshness, availability, and correctness.

AI Fluency & Influence

• Hands-on experience applying ML/GenAI tools to data infrastructure (semantic search, auto-docs, DQ monitors, AI-assisted pipeline generation).
• Demonstrated history of landing data contracts and standardized paved roads across cross-functional engineering teams.
• Exceptional technical writing and communication skills for authoring RFCs, ADRs, and system design specifications.

Preferred Qualifications & Success Indicators

• Experience with regional isolation, multi-region replication strategies, and privacy-by-design data governance in regulated environments.
• Prior track record achieving ≥99.9% data freshness SLOs and driving measurable cost/TB reductions across petabyte-scale environments.
• Experience reducing pipeline authoring times and data incidents through AI-in-the-loop operational tooling.