About the Role
We are looking for experienced annotators with deep cultural, linguistic, and audio catalog expertise to help improve a major music platform’s personalized experiences across multiple languages and regions.
In this role, you will create high-quality ground truth and evaluate the quality of both training data and model-generated output for various features. Your insights will help shape how listeners around the world discover and enjoy music, podcasts, and audiobooks.
What You’ll Do
- Evaluate agentic and personalized experiences and assess search relevance & recommendations
- Perform complex qualitative assessments to support emerging agentic and conversational listening experiences.
- Judge how well recommendations, responses, and UI elements reflect local culture, language norms, and audio habits.
- Review playlist and recommendation quality, including how well results match inferred user intent and preferences.
- Assess how realistically synthetic prompts/queries represent real user behavior and catalog usage in your region.
- Create detailed ground truth judgments for music, podcasts, and audiobooks.
- Validate and refine LLM-generated judgments to ensure they align with real-world user expectations and regional norms.
- Identify inconsistencies, edge cases, and nuanced failures in model output.
- Summarize insights & highlight risks to provide concise written and verbal summaries of your top findings, highlights, and concerns for each evaluation set.
- Carefully review and follow project-specific instructions before starting annotation work.
- Adapt quickly as project types rotate and priorities shift and deliver high-quality annotations within 1–2 week deadlines.
Requirements
- Native proficiency in Arabic with deep understanding of the regional culture associated with your language, including how people in your region actually use audio streaming services.
- Strong fluency in English (written and spoken) to communicate findings, questions, and insights with the internal team.
- Extensive, current knowledge of music, podcasts, and audiobooks relevant to your region such as their key artists, global trends, listening habits.
- Long-time, avid fan of music, with additional familiarity using streaming experiences for podcasts and audiobooks.
- Could be an avid listener, musician, or songwriter; formal industry experience is NOT required.
- 6+ months of annotation or labeling experience, ideally in complex or subjective domains.
- Critical thinking and strong research skills to supplement your audio content knowledge.
-Proficient with Google Workspace, especially Google Sheets, for tracking annotations and evaluations with great accuracy.
- Open to training and feedback with an ability to meet deadlines for 1–2 week annotation cycles.
Project Details
Contract Type: Freelance
Location: Egypt (remote)
Start Date : 15th December 2025
Duration: 1 to 2 weeks
Schedule: Daily 1 hour; flexible based on client's needs
Note: Please do not use VPNs or IP-masking tools during the recruitment process — our security system requires accurate regional verification.