Vallen · 18 hours ago
Databricks Data Engineer (Remote)
United States
Part-time
Remote
Senior Level
5+ years expVallen is an industry leader in delivering flexible supply chain solutions, and they are seeking a Senior Data Engineer to join their team. The role involves building scalable data solutions, enabling analytics, and driving innovation within a modern data platform.
Wholesale
Responsibilities
Design and develop scalable data pipelines using PySpark, Spark SQL, and Python within the Databricks environment
Build modular, version-controlled transformation workflows using DBT (Data Build Tool)
Package and deploy Databricks workflows and notebooks using Databricks Asset Bundles (DABS) for CI/CD and environment management
Integrate Databricks workflows with Azure DevOps for automated testing, deployment, and version control
Develop and maintain robust CI/CD pipelines for data engineering workflows using Git, Azure DevOps, and DABS
Implement and optimize dimensional data models, ELT/ETL processes, and performance tuning in cloud data platforms
Collaborate with data scientists, analysts, and business stakeholders to deliver high-impact data solutions
Implement logging, alerting, and monitoring for data pipelines using tools like Databricks Jobs, MLflow, or Azure Monitor
Qualification
Required
5–7 years of experience as a Data Engineer, with at least 3 years working in Databricks
Strong proficiency in Python, PySpark, and Spark SQL
Hands-on experience with DBT in a cloud data platform
Experience with DABS for workflow packaging and deployment
Proven expertise in Azure DevOps, Git, and CI/CD pipeline development
Solid understanding of data modeling, ETL/ELT, and performance optimization
Experience implementing monitoring and observability for data pipelines
Excellent communication and collaboration skills
Preferred
Experience in Azure cloud ecosystem
Familiarity with MLflow and other observability tools
Exposure to data governance and metadata management practices
Benefits
Medical, Dental, and Vision Insurance
Medical, Dental, and Vision Insurance for Single, Employee + Spouse or Children, and Family Coverage
401(k) with discretionary company match of $0.50 on the $1.00 up to 6% of pay (U.S. positions only)
Employer-paid Basic Life Insurance for Employee, Spouse, and Dependents
Employer-paid Short-Term and Long-Term Disability Benefits and Parental Leave (and any leave required under federal, state, and local laws)
Health Care and Dependent Care Flexible Spending Accounts
Paid Time Off (Vacation and Sick Days): 80–160 hours of vacation time based on seniority, accrued monthly and prorated from date of hire. 40–56 hours of sick time per year based on seniority and advanced upon hire.
Paid Time Off (Holidays): 8 scheduled holidays plus 2 floating holidays
Employee Assistance Program
Employee Resource Groups for networking and team building
Tuition Reimbursement Program
Employee Referral Program
Safety shoe and safety glasses reimbursement (based on position)
Employee discounts through BenefitHub
Advancement opportunities
Company
Vallen
Vallen is the largest provider of integrated services and the "go-to" partner for large production companies that need real, transparent savings and optimization of the indirect supply chain.
1001-5000 employees
Funding
Current Stage
Late StageCompany data provided by crunchbase