New York Technology Partners · 3 hours ago
Fabric Data Engineer
Phoenix, AZ
Contract
Onsite
Senior Level, Lead/Staff
8+ years expNew York Technology Partners is seeking a Senior Fabric Data Engineer to design and maintain sophisticated ELT/ETL pipelines within the Microsoft Fabric ecosystem. This role focuses on large-scale data ingestion and transformation, utilizing Spark-based processing for complex data enrichment and ensuring data is primed for high-performance reporting and analytics.
ConsultingInformation TechnologySoftwareStaffing Agency
Hiring Manager
Aditya Jain
Responsibilities
Design and orchestrate high-volume data ingestion pipelines using Data Factory (in Fabric) to connect diverse enterprise sources
Create and manage Lakehouse structures, including Delta tables and file systems, acting as the primary landing zone within OneLake
Implement advanced partitioning strategies and manage Shortcuts within OneLake / ADLS Gen2 to optimize for both performance and cost
Utilize Notebooks (PySpark, Spark SQL, Scala) and Spark Job Definitions for large-scale data cleansing, enrichment, and complex transformations
Transform raw data into curated Delta tables within Lakehouses or Data Warehouses, ensuring schema consistency and data quality
Implement reusable, parameterized code and automated workflows to reduce manual intervention and streamline processing schedules
Track pipeline runs and Spark job performance using Monitoring Hubs to identify and resolve bottlenecks
Collaborate with Platform Architects on infrastructure requirements and partner with Report Developers to deliver intuitive, interactive dashboards
Apply rigorous security and governance principles to all data structures and pipelines to ensure compliance and reliability
Qualification
Required
8+ years of professional Data Engineering experience
Expert knowledge of Data Factory, OneLake, and Lakehouse items
High proficiency in PySpark, Spark SQL, or Scala for complex transformations
Solid understanding of Delta format, partitioning, and storage optimization
Proven ability to build scalable ELT/ETL pipelines from scratch
Preferred
Manufacturing experience is a major plus
Company
New York Technology Partners
New York Technology Partners is an information technology company that provides IT and engineering services.
201-500 employees
H1B Sponsorship
New York Technology Partners has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (52)
2024 (74)
2023 (59)
2022 (104)
2021 (74)
2020 (115)
Funding
Current Stage
Growth StageLeadership Team
Amy Reynolds
Senior Talent Acquisition Partner
Hetal Shah
Technical Recruiter at New York Technology Partners
Company data provided by crunchbase