Java Software Engineer
$60/h-$100/h
Mercor9 days ago
Highlight
RemotePart-timeSenior
Summary
Mercor is seeking a Java Software Engineer to join their team in a remote capacity. The role involves evaluating LLM-generated responses to coding and software engineering queries for accuracy and clarity.
About the Role
As a Java Software Engineer, you will be responsible for evaluating responses generated by large language models (LLMs) to ensure they meet standards of accuracy, reasoning, clarity, and completeness. Your tasks will include conducting fact-checking, executing code for validation, annotating model responses, assessing code quality, and ensuring alignment with expected conversational behavior. You will apply consistent evaluation standards and follow detailed guidelines.
About You
Required:
- BS, MS, or PhD in Computer Science or a closely related field.
- 5+ years of real-world experience in software engineering or related technical roles.
- Expertise in at least two relevant programming languages (e.g., Python, Java, C++, etc.).
- Ability to solve HackerRank or LeetCode Medium and Hard-level problems independently.
- Experience contributing to well-known open-source projects, including merged pull requests.
- Significant experience using LLMs while coding and understanding their strengths and failure modes.
- Strong attention to detail and comfort evaluating complex technical reasoning.
Preferred:
- Prior experience with RLHF, model evaluation, or data annotation work.
- Track record in competitive programming.
- Experience reviewing code in production environments.
- Familiarity with multiple programming paradigms or ecosystems.
- Experience explaining complex technical concepts to non-expert audiences.
Benefits
- Flexible work hours with the option for part-time or full-time contract work.
- Competitive compensation ranging from $60 to $100 per hour.
Mercor
We use AI to understand human ability and match talent with the opportunities they're best suited for.
United States