Software Engineer, GPU Infrastructure & Kubernetes (Remote)

Cohere builds foundation AI models for businesses. Its Internal Infrastructure team runs the shared Kubernetes fleets where Cohere's models are trained, evaluated and served: multi-cloud GPU and TPU clusters, node health and performance.

Key responsibilities

• Deploy and run GPU Kubernetes clusters across clouds, and bring up new hardware
• Work with cloud providers and hardware partners on reliability, performance and cost, including RDMA
• Automate fault detection and repair for nodes, GPUs and network fabric
• Improve fleet use, track cost and shorten the time to return a faulty node
• Join the paid 24x7 on-call rotation

Requirements

• Ran GPU Kubernetes clusters with thousands of GPUs for ML or HPC workloads
• Debugged Linux, networking and performance problems
• Writes software in Python or Go
• Worked with researchers or ML engineers whose goals differ from yours

Nice to have

• RDMA, NCCL or accelerator health monitoring

Remote details

• Remote-friendly; this posting is open in the United States
• A co-working benefit if you are not near an office, plus a $500 home office stipend

Salary and benefits

US base salary $175,000 to $325,000 a year, depending on location, plus equity. Benefits include a weekly $75 lunch stipend, full health and dental cover, 401(k), 6 weeks of paid vacation and a learning stipend.

How to apply

Apply on the company's own site.