Synstack Technologies · 1 day ago
Python Data Engineer (LLM)
United States
Contract
Remote
Senior Level, Lead/Staff
8+ years expSynstack Technologies is seeking a seasoned Senior Data Engineer to architect, build, and own the data pipelines that power their large language model development. The primary mission is to build scalable, automated systems that transform massive, raw datasets into pristine, model-ready formats while collaborating with AI researchers and ML engineers.
Information Technology & Services
Hiring Manager
Bharath Kathirisetty
Responsibilities
Design, develop, and own robust, scalable, and automated ETL/ELT pipelines in Python for ingesting and processing terabyte-scale text datasets
Implement rigorous data cleaning, deduplication, filtering, and normalization strategies
Define and enforce data quality standards to ensure the highest integrity for model training
Efficiently structure and format diverse datasets (JSON, Parquet, etc.) for consumption by LLM training frameworks
Work closely with our team of AI researchers and ML engineers to understand data requirements, define metrics, and support the model training lifecycle
Continuously optimize data processing workflows for speed, cost, and reliability
Occasionally assist in launching, monitoring, and debugging data-related issues during model training runs
Qualification
Required
8+ years of professional experience in data engineering, data processing, or backend software engineering
Expert-level proficiency in Python and its data ecosystem (e.g., Pandas, NumPy, Dask, Polars)
Proven experience building and maintaining large-scale data pipelines
Deep understanding of data structures, data modeling, and software engineering best practices (Git, CI/CD, testing)
Experience handling and parsing diverse data formats (JSON, CSV, XML, Parquet) at scale
Excellent problem-solving skills and a meticulous attention to detail
Strong communication and collaboration skills, with experience working in a team environment
Preferred
Hands-on experience with the data preprocessing pipeline for an LLM (e.g., LLaMA, BERT, GPT-family)
Strong experience with big data frameworks like Apache Spark or Ray
Experience with Hugging Face libraries (Transformers, Datasets, Tokenizers)
Familiarity with ML frameworks like PyTorch or TensorFlow
Proficiency with cloud platforms (AWS, GCP, Azure) and their data/storage services
Company
Synstack Technologies
Synstack Tech | Innovate. Build. Hire. Grow. At Synstack Tech, we bridge the gap between innovative development and smart hiring.
11-50 employees
Funding
Current Stage
Early StageCompany data provided by crunchbase