LLM Trainer – Remote (AI Agent Evaluation)

About the role

We are looking for an experienced LLM Trainer to help design, evaluate, and improve autonomous AI agents across diverse real‑world domains such as health, education, and daily life. The role is fully remote and focuses on providing high‑quality human feedback to advance large language model capabilities.

Key responsibilities

• Write clear evaluation rubrics with objective pass/fail criteria for AI agents.
• Debug agent execution traces to pinpoint failure patterns.
• Stress‑test agents against edge cases, prompt injection, and tool misuse.
• Assess production‑grade modular software architectures and multi‑turn system interactions.
• Deliver dense technical feedback to improve LLM training pipelines.
• Complete onboarding assessments and begin flexible, task‑based assignments.

Required profile

• Background in backend engineering, AI automation, or complex systems integration.
• Proven ability to build and maintain production‑grade software with modular separation of services.
• Strong command of at least two major programming languages.
• Experience working with SQL databases and live, non‑mocked environments.
• Hands‑on experience handling multi‑turn system interactions.

Required skills

• Python
• JavaScript
• Go
• Java
• SQL
• Supabase
• Gmail API

What we offer

• Hourly compensation ranging from $30 to $50, based on experience and task complexity.
• Weekly payments via supported payout platforms such as PayPal or AirTM.
• Equal‑opportunity engagement with inclusive hiring practices.