Applied Engineer, Evaluations Role
- Pay: $100 to $120 / hour
- Contract: Part time
- Who can apply: Open worldwide
- Posted Jun 2, 2025
In short
Mercor offers a part-time remote job as an Applied Engineer, Evaluations Role, paying 100 to 120 USD per hour. You will build and refine AI coding evaluations alongside a research team, making this a great fit if you love coding and testing modern language models.
What you will do
- Select completed pull requests and repositories to build challenging coding evaluations
- Write task prompts and build automated tests, shell commands, and LLM grading prompts
- Test evaluations against reference solutions and flawed code to find errors and missing checks
- Run evaluations across multiple models and document your findings and grading decisions
What you need to apply
- Strong programming fundamentals and experience with complex codebases in languages like Python, TypeScript, Java, and others
- Ability to understand unfamiliar code and write meaningful tests covering edge cases
- Practical experience using AI coding tools and Git
- Clear communication skills and the ability to work independently after onboarding
Who can apply
This remote role is open worldwide with no country restrictions stated.
My tip
At Simply Remotely, we suggest highlighting your experience with AI coding tools and complex codebases in your application to stand out.