Senior Software Engineer, AI Coding Tasks | Askable Remote

We’re looking for experienced software engineers to build AI coding tasks from a codebase they know well. You will turn a real change from a repository you know into a task that today’s best AI coding models struggle with: hard for the models, but fair for a skilled engineer. Your tasks are used to measure and train AI coding models.

Key Responsibilities

• Propose a real change from a repository you know and submit a short proposal for approval
• Set up the repository as it was before the change in our platform
• Write the task brief, the correct solution, and the grading checks
• Stress-test your task with lazy, incorrect, and shortcut solutions to confirm your checks catch them
• Run your task against leading AI models until strong models often fail but a skilled engineer can clearly solve it
• Revise your task through reviewer feedback (most tasks take more than one round)
• Start your next task once the current one is accepted

Requirements

• 2+ years of professional software engineering experience
• Code merged into a public open-source repository, ideally one you know well (a private codebase you own may also qualify, but not employer or client code)
• Strong skills in at least one of Python, TypeScript, Go, Rust, Java, or C++
• Solid experience with Docker, Linux, Git, and automated testing
• Regular use of an AI coding tool
• Experience writing tests, rubrics, or evaluation criteria, ideally for AI models
• Ability to explain a change clearly and concisely in your own words
• At least 10 hours a week available, starting now
• A secure personal or work computer (no public or shared machines)
• Education: Not specified

Skills

Technical Skills

• Python, TypeScript, Go, Rust, Java, or C++
• Docker, Linux, Git
• Automated testing and evaluation design
• AI coding tools

Soft Skills

• Clear, concise written explanation
• Attention to quality and edge cases
• Responsiveness to review feedback

Nice to Have

• Deep expertise in one area, such as databases, compilers, distributed systems, security, or infrastructure
• Experience building agent tasks or RL environments