Internal title: Director, Psychometric Operations & Data Governance
External Posting Role Title: Director, Psychometric Operations & Data Governance
College Board - Global Strategy and Talent, College Board LATAM
Location: San Juan, Puerto Rico. This is a hybrid role and requires the candidate to be on-site at CB offices every Tuesday and Wednesday.
Role Type: This is a full-time position.
About the Team
College Board LATAM is committed to expanding educational opportunity and advancing evidence-based decision-making across the region. As part of the Research & Development (R&D) division, the Research & Measurement (R&M) team ensures that our assessments are scientifically rigorous, operationally sound, and aligned with our mission to expand educational opportunities across the region.
We work at the intersection of psychometrics, data science, and applied research, guiding how our assessments are designed, scored, and continuously improved. Our team ensures that research and measurement practices are transparent, defensible, and grounded in robust methodological standards.
R&M also collaborates with institutions, ministries, and research partners to generate insights that advance the region’s educational goals, promote equitable test use, and strengthen the link between assessment results and educational outcomes. Together with colleagues in Technology, Assessment Operations, and Market Development, we are building the data systems and analytic capabilities that will power the next generation of digital, research-informed assessments across Latin America.
As we modernize our data ecosystem and integrate AI-enabled capabilities, we are seeking a director who can lead this transformation with architectural rigor, governance discipline, and strategic vision.
About the Opportunity
The Director, Psychometric Operations & Data Governance, is expected to be key architect of CB-LATAM’s data ecosystem—ensuring that psychometric and operational data are managed ethically, governed responsibly, and leveraged to drive both operational excellence and strategic insight.
In this role, you will define and implement the frameworks that safeguard the integrity, accessibility, and ethical use of CB-LATAM’s data assets. You will oversee the modernization of data architecture, strengthen data quality and compliance, and foster a culture of transparency and accountability around data use. You will design and maintain automated data pipelines that enable scoring, scaling, and reporting processes to run smoothly and reproducibly. You will be accountable for psychometric operations such as item calibrations, statistical analysis, and quality control to support scoring operations for current and future assessments.
You will also play a central role in shaping how R&M integrates AI-enabled analytics and automation to streamline workflows and scale psychometric processes. By combining technical expertise with policy alignment and collaboration across R&D and Technology, you will help transform CB-LATAM into a data-driven, innovation-ready organization—ensuring that every data point supports fairness, validity, and impact.
Key Responsibilities
Psychometric Operations Leadership (30%)
Drive and safeguard the operational integrity of scoring, equating, calibration, and reporting data workflows.
Execute and monitor scoring cycles meet defined turnaround time, accuracy, and QA standards.
Design and implement reproducibility, documentation, and traceability standards across all psychometric processes.
Architect and implement modernization of legacy scoring and item analysis pipelines into relational or warehouse-based systems.
Conduct root-cause analysis for systemic scoring irregularities or data discrepancies.
Partner with Psychometricians to ensure form rotation, scaling, and calibration processes are technically sound and operationally resilient.
Enterprise Data Governance & Stewardship (25%)
Serve as the designated Data Steward for psychometric and operational data within CB-LATAM.
Design and operationalize a unified governance framework covering metadata, retention, access control, and data lifecycle management.
Maintain comprehensive end-to-end data flow documentation across systems.
Ensure compliance with College Board governance policies, professional testing standards (AERA/APA/NCME), and applicable privacy regulations.
Establish governance KPIs and risk monitoring mechanisms.
Lead audit readiness efforts and documentation processes.
Data Architecture & Infrastructure Modernization (20%)
Define and execute the roadmap for transitioning from manual, file-based workflows to automated, scalable infrastructure.
Partner with Technology to implement secure, cloud-aligned environments for psychometric and scoring data.
Establish standards for schema design, lineage tracking, validation routines, and version control.
Execute and validate migration of core datasets into governed architecture.
Design and validate technical design decisions for new pipelines and automation initiatives.
Automation, AI & Advanced Analytics (15%)
Lead integration of AI-assisted validation and anomaly detection tools within scoring and integrity review workflows.
Establish model governance standards including documentation, benchmarking, and human-review protocols.
Identify automation opportunities that reduce manual effort while increasing consistency and quality.
Translate complex analytic outputs into executive-ready insights and operational recommendations.
Organizational Leadership & Influence (10%)
Represent R&M in enterprise data architecture and modernization initiatives.
Mentor team members in governance, reproducible research practices, and advanced analytics.
Advise senior leadership on data risk, infrastructure investments, and operational priorities.
Promote a culture of accountability, data ethics, and continuous improvement.
About You:
Master’s degree in Data Science, Computer Science, Statistics, Information Systems, Psychometrics, or related field.
8–12+ years of experience in data architecture, governance, analytics leadership, or high-stakes operational data environments.
Demonstrated experience owning production data systems with strict accuracy and compliance requirements.
Proven track record implementing data governance frameworks and automation strategies.
Strong technical fluency in SQL and at least one programming language (R or Python).
Experience designing or operating data workflows within AWS cloud infrastructure and working with structured and semi-structured data formats (e.g., JSON).
Working knowledge of IRT and CTT models, scaling, and equating procedures.
Experience deploying AI/ML models into operational workflows.
Strong executive communication and stakeholder alignment skills.
Ability to travel 1-2 times a year to College Board offices or on behalf of College Board business.
Bilingual proficiency in Spanish and English.
This position primarily involves sedentary work performed at a computer. The role requires the ability to sit or stand for extended periods, communicate effectively, and operate standard office equipment.
All roles at College Board require:
A passion for expanding educational and career opportunities and mission-driven work
Curiosity and enthusiasm for emerging technologies, with a willingness to experiment with and adopt new AI-driven solutions and comfort with learning and applying new digital tools independently and proactively.
Clear and concise communication skills, written and verbal
A learner's mindset and a commitment to growth: welcoming diverse perspectives, giving and receiving timely, respectful feedback, and continuously improving through iterative learning and user input.
A drive for impact and excellence: solving complex problems, making data-informed decisions, prioritizing what matters most, and continuously improving through learning, user input, and external benchmarking.
A collaborative and empathetic approach: working across differences, fostering trust, and contributing to a culture of shared success
Resident in USA, Authorization to work in the United States for any employer
About Our Process
Application review will begin immediately and will continue until the position is filled. This role is expected to accept applications for a minimum of 5 business days.
While the hiring process may vary, it generally includes: resume and application submission, recruiter phone/video screen, hiring manager interview, performance exercise such as live coding, a panel interview, a conversation with leadership and reference checks.
What We Offer
At College Board, we offer more than a paycheck- we provide a meaningful career, a supportive team, and a comprehensive package designed to help you thrive. We’re a self-sustaining nonprofit that believes in fair and competitive compensation grounded in your qualifications, experience, impact, and the market.
A Thoughtful Approach to Compensation
The hiring range for this role is $64,000–$80,000.
Your exact salary will depend on your location, experience, and how your background compares to others in similar roles at the College Board.
We aim to make our best offer upfront, rooted in fairness, transparency, and market data.
We adjust salaries by location to ensure fairness, no matter where you live.
You’ll have open, transparent conversations about compensation, benefits, and what it’s like to work at College Board throughout your hiring process. Check out our careers page for more.
#LI-Hybrid
#LI-MD1