Technical AI Response Evaluator
Budget / Salary$250–750
TypeFreelance project
LocationRemote
Posted2 hours ago
I need every batch of ChatGPT-style answers on technical topics combed through for factual accuracy and for how clearly and appropriately the ideas are expressed. For each response you will pinpoint any incorrect or incomplete statements, note ambiguities in wording or tone, and explain why they matter. Depth is key—I want a detailed error analysis rather than a high-level score.
The work will follow my internal rubric, but you are welcome to help refine that rubric as patterns emerge. Typical workflow:
• Receive a spreadsheet or JSON dump of model outputs
• Review against trusted technical sources or documentation
• Record each error with a brief correction, severity flag, and comment on tone/clarity
• Return a scored sheet plus a short summary of recurring issues for the engineering team
Acceptance criteria
• Every answer tagged with at least one of: correct, partially correct, incorrect
• All identified errors linked to a source or rationale
• Summary highlights at least three widespread problems per batch and suggests fixes
Turnaround on the first test set is flexible but ideally within 48 hours once materials arrive. Reliable communication and independent work are essential because the role is fully remote. If this scope matches your skill set, I’m ready to share the first dataset and guidelines right away.
The work will follow my internal rubric, but you are welcome to help refine that rubric as patterns emerge. Typical workflow:
• Receive a spreadsheet or JSON dump of model outputs
• Review against trusted technical sources or documentation
• Record each error with a brief correction, severity flag, and comment on tone/clarity
• Return a scored sheet plus a short summary of recurring issues for the engineering team
Acceptance criteria
• Every answer tagged with at least one of: correct, partially correct, incorrect
• All identified errors linked to a source or rationale
• Summary highlights at least three widespread problems per batch and suggests fixes
Turnaround on the first test set is flexible but ideally within 48 hours once materials arrive. Reliable communication and independent work are essential because the role is fully remote. If this scope matches your skill set, I’m ready to share the first dataset and guidelines right away.
Apply on Freelancer →
Project sourced from Freelancer.com. Applications happen directly on the original platform — we never collect your data.