Research Intern, Frontier Speculative Decoding (Summer 2027)

<h3><span style="font-family: helvetica, arial, sans-serif;"><strong>

About

the Role</strong></span></h3> <p><span style="font-family: helvetica, arial, sans-serif;">The Speculative Decoding team at Together AI builds the draft models and algorithms that let large language models generate faster without changing their outputs. Our speculators run in production on frontier open models such as DeepSeek, Kimi and GLM. Our work sits at the intersection of model training, inference algorithms and serving systems, and we co-design all three to lower the cost and latency of generation.</span></p> <p><span style="font-family: helvetica, arial, sans-serif;">As a research intern, you will dive into open questions in speculative decoding: new drafter architectures and verification schemes, knowledge distillation and training recipes that align draft and target models, and drafters that adapt as traffic shifts. You will train and fine-tune draft models at scale on dedicated GPU clusters, and test your ideas against realistic serving workloads, including long-context and agentic traffic.</span></p> <p><span style="font-family: helvetica, arial, sans-serif;">Projects aim to answer important research questions in speculative decoding, enabling faster generation on frontier models, drafters that keep pace with changing workloads, and robust, reproducible evaluation under real traffic.</span></p> <h3 id="past-research" data-lfile-container="75b2226e-f583-4bc8-a073-fba516e11feb" data-pm-slice="1 3 []"><span style="font-family: helvetica, arial, sans-serif;">Past Research</span></h3> <p><span style="font-family: helvetica, arial, sans-serif;">Speculative decoding research by Together researchers, interns and academic collaborators includes:</span></p> <ul> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;"><a href="https://openreview.net/forum?id=aL1Wnml9Ef" target="_blank">Speculative Speculative Decoding</a> (ICLR 2026)</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;"><a href="https://arxiv.org/abs/2602.06932" target="_blank">When RL Meets Adaptive Speculative Training: A Unified Training-Serving System</a> (ICML 2026)</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;"><a href="https://arxiv.org/abs/2511.13841" target="_blank">Beat the Long Tail: Distribution-Aware Speculative Decoding for RL Training</a> (MLSys 2026)</span></li> </ul> <h3><span style="font-family: helvetica, arial, sans-serif;"><strong>

Responsibilities</strong></span></h3> <ul> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Research and prototype new efficient speculative decoding algorithms and draft model architectures</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Train, fine-tune and distill draft models, and design rigorous experiments to validate hypotheses.</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Implement optimized version for serving engines to show efficacy in real world workloads</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Communicate the plans, progress, and results of projects to the broader team</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Share findings through technical write-ups and blog posts, and help bring successful ideas into production</span></li> </ul> <h3><span style="font-family: helvetica, arial, sans-serif;"><strong>

Requirements</strong></span></h3> <ul> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Currently pursuing a Master's or Ph.D. degree in Computer Science, Electrical Engineering, or a related field</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Strong knowledge of Machine Learning and Deep Learning fundamentals</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Hands-on experience training or fine-tuning large models with distributed training frameworks (e.g., Megatron-LM, FSDP)</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Familiarity with modern model architectures, such as Mixture-of-Experts and hybrid attention</span></li> </ul> <h3 id="preferred-

qualifications" data-lfile-container="75b2226e-f583-4bc8-a073-fba516e11feb" data-pm-slice="1 3 []"><span style="font-family: helvetica, arial, sans-serif;">Preferred

Qualifications</span></h3> <ul> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Prior research experience in speculative decoding, efficient inference, or efficient machine learning</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Publications at leading conferences in machine learning or systems (e.g., NeurIPS, ICML, ICLR, MLSys)</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Experience with knowledge distillation, post-training, or reinforcement learning</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Familiarity with LLM serving engines (e.g., vLLM, SGLang, TensorRT-LLM)</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Contributions to open-source machine learning projects</span></li> </ul> <h3><span style="font-family: helvetica, arial, sans-serif;"><strong>About Together AI</strong></span></h3> <p><span style="font-family: helvetica, arial, sans-serif;">Together AI, the AI Native Cloud, is purpose-built for AI engineers. AI application developers get high-performance inference that scales reliably, fine-tuning and reinforcement learning for creating frontier-level specialized models, and pre-training at massive scale for fully custom intelligence, all around a marketplace of leading open models that teams can run, adapt, and own. Trusted by Cursor, Decagon, ElevenLabs, Salesforce, and Zoom, Together serves 400+ trillion tokens a month.</span></p> <h3><span style="font-family: helvetica, arial, sans-serif;"><strong>Internship Program Details</strong></span></h3> <p><span style="font-family: helvetica, arial, sans-serif;">Our internship program runs 12 to 14 weeks, giving you the opportunity to work alongside industry-leading engineers and researchers across multiple teams. This cohort's internship dates span May 17th to August 6th or June 14th to September 3rd.</span></p> <h3><span style="font-family: helvetica, arial, sans-serif;"><strong>

Compensation</strong></span></h3> <p><span style="font-family: helvetica, arial, sans-serif;">We offer competitive

compensation, housing stipends, and other competitive

benefits. The estimated US hourly rate for this role is $58 - $70 an hour. Our hourly rates are determined by location, level and role. Individual

compensation will be determined by experience, skills, and job-related knowledge.</span></p> <h3><span style="font-family: helvetica, arial, sans-serif;"><strong>Equal Opportunity</strong></span></h3> <p><span style="font-family: helvetica, arial, sans-serif;">Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.</span></p> <p><span style="font-family: helvetica, arial, sans-serif;">For more information about our privacy policy, please visit:<a href="https://www.together.ai/privacy"> https://www.together.ai/privacy</a></span></p> <h3><br><br><br></h3>