About the role
We are looking for early members of our research team. As an ML Engineer, you will be responsible for designing and deploying large-scale systems, from data infrastructure to new model architectures, to build Large Audio Models.
What you'll do:
- Lead engineering projects for data collection and management
- Build tools and models to visualize, evaluate, and filter datasets
- Scale machine learning models to 7B+ parameters
- Develop optimization techniques for distributed parallel systems
- Optimize inference speed for large-scale models
- Implement models and systems efficiently for data processing, training, inference, and deployment
What you'll need:
- Proficiency in Python
- Experience with JAX, PyTorch, or TensorFlow
- Familiarity with DDP, FSDP, Tensor Parallel, and distributed PyTorch code
- Experience implementing models from scratch: Transformer, VAE, Denoising Diffusion
Nice to have:
- CUDA kernel code / low-level ML optimization
- Large-scale data management
- Pretraining 7B+ ML models
- Kubernetes and Docker
- Multi-node distributed training