Evaluation Scenario Writer - AI Agent Testing Specialist
$21/h
Mindrift2 days ago
Highlight
RemotePart-timeMid
Summary
Mindrift is seeking an Evaluation Scenario Writer - AI Agent Testing Specialist to join their team remotely in Latin America. The role involves designing realistic evaluation scenarios for LLM-based agents and creating test cases that simulate human tasks.
About the Role
As an Evaluation Scenario Writer, you will be responsible for designing structured test scenarios, defining acceptable agent behavior, annotating task steps, and reviewing agent outputs. You will collaborate with developers to ensure clarity and effectiveness of the scenarios you create, contributing to the advancement of AI technology.
About You
Required:
- Bachelor's and/or Master’s Degree in Computer Science, Software Engineering, Data Science, Artificial Intelligence, Computational Linguistics, or related fields.
- Background in QA, software testing, data analysis, or NLP annotation.
- Good understanding of test design principles.
- Strong written communication skills in English.
- Comfortable with structured formats like JSON/YAML for scenario description.
- Basic experience with Python and JavaScript.
Preferred:
- Experience in writing manual or automated test cases.
- Familiarity with LLM capabilities and typical failure modes.
- Understanding of scoring metrics such as precision, recall, and coverage.
Benefits
- Flexible, remote, freelance project that fits around your primary commitments.
- Get paid for your expertise, with rates up to $21/hour depending on skills and experience.
- Participate in advanced AI projects and gain valuable experience.
Mindrift
Welcome to Mindrift — a space where innovation meets opportunity. We’re a pioneering platform dedicated to advancing the field of artificial intelligence through collaborative online projects. Our focus lies in creating data for generative AI, offering a unique chance for freelancers to contribute to AI development from anywhere, at any time. At Mindrift, we believe in the power of collective intelligence to shape the future of AI. Our platform allows users to dive into a variety of tasks — ranging from creating training prompts for AI models to refining AI responses for greater relevance. Let’s build the future of AI together, one task at a time.
Latin America