Job Description: We are seeking an AI Medical Expert Evaluator in Karachi to assess the accuracy, safety, and clinical relevance of AI-generated medical content. In this role, you will apply medical expertise to structured evaluation tasks and help improve the quality of healthcare-related AI systems. The position is suited to a detail-oriented professional who can assess clinical information objectively and communicate findings clearly.
Key responsibilities- Review AI-generated answers to medical questions for factual accuracy, clinical appropriateness, and clarity.
- Identify unsafe, incomplete, misleading, or unsupported recommendations and document specific concerns.
- Compare responses against trusted medical knowledge and evaluation guidelines.
- Rate and annotate outputs consistently, providing concise rationale and actionable feedback.
- Maintain confidentiality and follow applicable privacy, ethics, and data-handling requirements.
- Collaborate with evaluation and product teams to clarify clinical criteria and improve assessment workflows.
This is a Karachi-based, non-remote role. The employer is identified as Doctor; no further organizational details were provided. The evaluator contributes expert review rather than direct patient care, and should use sound clinical judgment while recognizing when a response requires additional context or specialist review.
Job Specification:
- Strong medical knowledge and ability to interpret clinical questions, terminology, and recommendations.
- Skill in checking medical claims for accuracy, relevance, completeness, and appropriate uncertainty.
- Sound judgment in identifying potentially harmful advice, contraindications, and safety concerns.
- Clear written communication, including concise explanations and structured annotation of findings.
- High attention to detail and consistency when applying rubrics across repeated evaluation tasks.
- Ability to assess AI-generated text objectively without treating it as a substitute for professional care.
- Comfort with web-based review tools, digital documentation, and handling sensitive information responsibly.
- Professional integrity, discretion, and commitment to evidence-informed healthcare practice.