Evaluation Engineer
Mindrift
Posted
3 weeks ago
Vietnam
Remote
USD 6K
Mid Level
Part Time
Sema Summary
Mindrift is seeking an Evaluation Engineer to design evaluation scenarios for AI agents. The role involves creating test cases and analyzing agent performance to enhance AI decision-making.
About Company
Mindrift is a pioneering platform dedicated to advancing artificial intelligence through collaborative online projects. They focus on creating data for generative AI, allowing freelancers to contribute from anywhere.
Core Requirements
- Bachelor's or Master's Degree in relevant fields
- Background in QA or software testing
- Strong written communication skills in English
- Experience with JSON/YAML formats
- Basic knowledge of Python and JavaScript
Responsibilities
- Design realistic evaluation scenarios for AI agents.
- Create structured test cases simulating human workflows.
- Define gold-standard behavior for agent actions.
- Analyze agent logs and decision paths.
- Iterate on test cases to improve clarity.
- Ensure scenarios are reusable and easy to execute.
- Collaborate with teams to validate scenarios.
Benifits
- Flexible schedule
- Remote work
- Valuable experience
- Influence AI development
Must Have skills
Job Keywords
Similar Jobs
AI Engineer
Data Science Hiring
Remote
Software Engineer
Mercor
Remote
Automotive Engineer
Mindrift
Vietnam
Automotive Engineer
Mindrift
Vietnam
Lead DevOps Engineer
Semesteria
Remote
Site Reliability Engineer
Construction Job Vacancy
Remote
Corporate Trainer
Thinkcloudly
Remote
Data Partner
TELUS Digital
Global - Remote
Arabic Expert
micro1
Remote
Physics Expert
Mindrift
Vietnam
AI Career Assistant
Coming soonWe're polishing your AI assistant experience. You can leave one message for now.
Hi 👋
AI chat is launching soon on Semesteria. Tell us what you want help with (career, resume, or job search), and we'll use it to shape the first release.