At Roche you can show up as yourself, embraced for the unique qualities you bring. Our culture encourages personal expression, open dialogue, and genuine connections, where you are valued, accepted and respected for who you are, allowing you to thrive both personally and professionally. This is how we aim to prevent, stop and cure diseases and ensure everyone has access to healthcare today and for generations to come. Join Roche, where every voice matters.
We believe it’s urgent to deliver medical solutions right now – even as we develop innovations for the future. We are passionate about transforming patients’ lives and we are fearless in both decision and action. And we believe that good business means a better world. We commit ourselves to scientific rigor, unassailable ethics, and access to medical innovations for all. We do this today to build a better tomorrow. Pharmaceutical Sciences (PS) is a global function within Roche Pharma Research and Early Development (pRED).
As a team member in the Prediction Modelling team of PS, you will work in close collaboration with (computational) toxicologists as well as other scientists in pRED, using state-of-the-art bioinformatics and biostatistics tools and methods and gaining toxicological insights from experts in the field.
Roche has accumulated a vast collection of curated preclinical and clinical study reports spanning the full drug development lifecycle. Current commercial LLMs still underperform in specialized biomedical and pharmaceutical contexts. This internship focuses on advancing Roche’s internal LLM post-training capabilities to build domain-specialized LLMs for clinical study use by applying Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Reinforcement Learning from Human Feedback (RLHF). You’ll work side-by-side with Roche ML scientists to fine-tune open-source models, optimize them for pharmacological, translational-science and drug-discovery tasks, and help build the infrastructure and workflows needed to support model deployment in a drug-discovery context.
You’ll have access to high-performance GPU infrastructure (A100 clusters) and unique domain-specific data. You’ll start with smaller-sized LLMs to build the methodology, then iterate toward larger models and production-scale workflows.
Data Creation: Approximately 10% of this intern’s work will be data collection and dataset creation.
Model Post-Training: Fine-tune and align open-source LLMs (e.g., Llama, Mistral, Qwen) using Roche’s curated clinical and preclinical datasets through SFT, DPO, or RLHF.
Pipeline Development: Implement and iterate on training pipelines using frameworks such as (including but not limited to) Hugging Face Transformers, TRL, NV Megatron-LM, or HF Smol.
Evaluation: Design evaluation protocols for factual accuracy, safety, and alignment with biomedical domain knowledge.
Experimentation: Start with small-scale LLMs to establish scalable training and evaluation workflows, progressing toward larger foundation models.
Documentation: Maintain experiment logs, model cards, and reproducible training setups for internal knowledge transfer.
Enrolled in a Master’s or PhD program in Computer Science, Computational Linguistics, or a related field.
Strong coding skills in Python and PyTorch; experience with large-scale model training and distributed computing in Linux-based clusters.
Hands-on experience with at least one LLM training or fine-tuning framework (e.g., Hugging Face Transformers/TRL, Megatron-LM, HF Smol).
Familiarity with RLHF/DPO workflows and human-in-the-loop alignment methods.
Understanding of evaluation metrics for LLMs (factual consistency, hallucination detection, preference modeling).
Interest in applying AI to clinical and biomedical domains and working with sensitive data responsibly.
You have very good interpersonal and communication skills, are able to build good working relationships, and are an outstanding teammate. Your experience and investigative attitude allow you to work independently, to design, perform, and interpret experiments, and to embark on new scientific methodologies.
Start: asap/ February 2026
Duration: 12 Months
Workload: 100%
Due to regulations Non-EU/EFTA citizens must provide a certificate from the university stating that an internship is mandatory as part of the application documents. Furthermore, they need to be enrolled during the entire duration of the internship.
We are looking forward to your application!
A healthier future drives us to innovate. Together, more than 100’000 employees across the globe are dedicated to advance science, ensuring everyone has access to healthcare today and for generations to come. Our efforts result in more than 26 million people treated with our medicines and over 30 billion tests conducted using our Diagnostics products. We empower each other to explore new possibilities, foster creativity, and keep our ambitions high, so we can deliver life-changing healthcare solutions that make a global impact.
Let’s build a healthier future, together.
Roche is an Equal Opportunity Employer.