This job has closed.

Meta · 17 hours ago

Software Engineer, SystemML - AI Networking

Menlo Park, CA

Full-time

Onsite

Senior Level

Meta builds technologies that help people connect, find communities, and grow businesses. In this role, you will be a member of the AI Networking Software team, focusing on developing and owning the software stack around NCCL for multi-GPU and multi-node data communication, aiming to improve reliability and performance for large-scale GenAI/LLM training.

Computer Software

Comp. & Benefits

Responsibilities

Tech-leading the collective communication library development on Meta's large-scale GPU training infra with a focus on GenAI/LLM scaling

Qualification

C/C++ programmingPython programmingDistributed ML TrainingNCCL experienceGPU architecture knowledgeCUDA programmingML frameworks experienceHPC experienceEffective leadershipCommunication skills

Required

Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience

Proven C/C++ and Python programming skills

Proven track record of leading successful projects

Effective leadership and communication skills

Specialized experience in one or more of the following machine learning/deep learning domains: Distributed ML Training, GPU architecture, ML systems, AI infrastructure, high performance computing, performance optimizations, or Machine Learning frameworks (e.g. PyTorch)

Preferred

Experience with NCCL and distributed GPU performance analysis on RoCE/Infiniband

PhD in Computer Science, Computer Engineering, or relevant technical field

Knowledge of GPU architectures and CUDA programming

Knowledge of ML, deep learning and LLM

Experience with both data parallel and model parallel training, such as Distributed Data Parallel, Fully Sharded Data Parallel (FSDP), Tensor Parallel, and Pipeline Parallel

Experience in HPC and parallel computing

Experience working with DL frameworks like PyTorch, Caffe2 or TensorFlow

Experience in AI framework and trainer development on accelerating large-scale distributed deep learning models