AI Member Experience Quality Analyst

We’re hiring an AI Member Experience Quality Analyst who is passionate about quality and eager to influence how AI‑powered member experiences are delivered across daily operations. In this role, success is defined by the impact of your quality insights and the improvements they drive across AI performance, compliance, and member trust. You’ll conduct rigorous quality evaluations grounded in real operational workflows, identify high‑risk issues, and translate findings into clear, actionable recommendations that strengthen AI responses at scale.

You’ll partner closely with training, operations, product, and business stakeholders to ensure quality learnings directly influence how our AI learns, evolves, and performs in live member environments. By connecting quality data, trends, and operational realities, you’ll tell a compelling quality story that reduces risk, elevates the member experience, and helps set the standard for AI quality excellence. Our ideal candidate brings a strong foundation in quality, operations, or analytics and thrives in environments where precision, judgment, and collaboration matter. You’re comfortable working with data, spotting patterns and risks, and clearly communicating insights to diverse stakeholders. You understand how quality shows up in day‑to‑day operations and are motivated by the opportunity to influence AI‑driven experiences at scale.

If you’re excited to apply your quality and operational expertise to the future of AI‑driven member experiences, apply today and help shape how trust, accuracy, and innovation come together at scale.

Job Responsibilities

• Be the quality champion: conduct thorough reviews of AI conversations, starting with 100% coverage at launch and evolving to strategic sampling as we prove excellence
• Apply your expertise: evaluate AI interactions against approved quality and experience standards, spanning accuracy, completeness, compliance, and member impact
• Document with precision: capture quality findings, categorize issues by impact (critical, major, minor), and track trends that tell the quality story
• Drive AI improvement: identify training gaps and knowledge base opportunities, providing specific, actionable recommendations to appropriate stakeholders
• Bridge quality and training: collaborate with Business Requirements Analysts to transform your findings into AI enhancements
• Create quality narratives: prepare compelling reports and scorecards that highlight trends, celebrate wins, and focus attention on opportunities
• Ensure consistency: participate in calibration sessions that align quality evaluation across the team
• Protect compliance: review AI responses for regulatory adherence, ensuring required disclosures and catching prohibited content
• Sound the alarm: escalate critical quality issues immediately, protecting members and the organization from potential harm
• Investigate root causes: support deep-dive analysis when quality incidents or member complaints require thorough understanding
• Shape quality standards: contribute to developing and refining the evaluation criteria and scoring approaches that define AI excellence
• Build quality knowledge: maintain evaluation guides, job aids, and training materials that elevate quality practice across the team

Job Qualifications

Education

• Associate degree or equivalent work experience required. Equivalent experience is defined as 2 years of professional work experience in healthcare or related field.

Experience

• 3+ years of experience in quality assurance, quality monitoring, or auditing with a track record of driving improvement
• 2+ years of experience in health insurance or healthcare operations with strong product and process knowledge

Skills\Certifications

• Expertise in quality evaluation methodologies, scoring rubrics, and calibration processes
• Deep knowledge of health insurance products, terminology, and member service standards
• Eagle-eye attention to detail with ability to identify errors and inconsistencies others might miss
• Strong written communication skills; you can document findings clearly and deliver constructive feedback 

Number of Openings Available

1

Worker Type:

Employee

Company:

BCBST BlueCross BlueShield of Tennessee, Inc.

Applying for this job indicates your acknowledgement and understanding of the following statements:

BCBST will recruit, hire, train and promote individuals in all job classifications without regard to race, religion, color, age, sex, national origin, citizenship, pregnancy, veteran status, sexual orientation, physical or mental disability, gender identity, or any other characteristic protected by applicable law.

Further information regarding BCBST's EEO Policies/Notices may be found by reviewing the following page:

BCBST's EEO Policies/Notices

BlueCross BlueShield of Tennessee is not accepting unsolicited assistance from search firms for this employment opportunity. All resumes submitted by search firms to any employee at BlueCross BlueShield of Tennessee via-email, the Internet or any other method without a valid, written Direct Placement Agreement in place for this position from BlueCross BlueShield of Tennessee HR/Talent Acquisition will not be considered. No fee will be paid in the event the applicant is hired by BlueCross BlueShield of Tennessee as a result of the referral or through other means.