Astir IT Solutions, Inc. · 4 hours ago
Senior Data Engineer (x4)Only w2
Minnesota, United States
Contract
Onsite
Senior Level, Lead/Staff
10+ years expAstir IT Solutions, Inc. is seeking a Senior Data Engineer to design and maintain scalable data pipelines. The role involves collaborating with stakeholders to ensure data quality and optimizing data workflows.
Information TechnologyOutsourcingSoftware
No H1B
Hiring Manager
Hari J
Responsibilities
Design, develop, and maintain scalable data pipelines using Apache Spark (Pyspark and/or Scala)
Build and optimize data workflows on Databricks, including Delta Lake, notebooks, and scheduled jobs
Ingest, transform, and curate large-scale structured and semi-structured datasets
Perform performance tuning and cost optimization of Spark workloads and Databricks clusters
Implement data quality checks, monitoring, and error handling
Collaborate with analytics and business stakeholders to deliver well-modeled, analytics-ready data
Support batch processing and, where applicable, streaming data pipelines
Follow best practices for testing, documentation, security, and version control
Qualification
Required
Minimum 10+ Years of Experience required
6+ years of experience in Data Engineering or related roles
Strong hands-on experience with Apache Spark (PySpark or Scala)
Proven experience working in Databricks environments
Strong SQL skills and experience with relational and analytical databases
Experience building and maintaining ETL/ELT pipelines at scale
Familiarity with modern data lake architectures (Delta Lake preferred)
Experience with Git-based version control
Preferred
Familiarity with modern data lake architectures (Delta Lake preferred)
Company
Astir IT Solutions, Inc.
Astir IT Solutions, Inc. (www.astirit.com) is a fast growing IT consulting, services, and outsourcing firm.
201-500 employees
Funding
Current Stage
Growth StageCompany data provided by crunchbase