CoreAi Consulting · 6 hours ago
Data Engineer
Phoenix, AZ
Full-time
Hybrid
Senior Level
5+ years expCoreAi Consulting is seeking a skilled Data Engineer to design, build, and maintain scalable data pipelines and platforms. The role involves developing data pipelines, implementing APIs, and ensuring data quality across enterprise systems.
Artificial Intelligence (AI)Information TechnologyRisk Management
Hiring Manager
CoreAI Careers
Responsibilities
Design, develop, and maintain DAG-based data pipelines (Airflow or similar orchestration tools)
Build and optimize SQL-based data transformations for analytics and reporting
Develop and manage batch and streaming data pipelines using Apache Spark
Implement Python-based REST APIs using FastAPI for data services and integrations
Perform data quality checks, validation, reconciliation, and anomaly detection
Work with cloud platforms (preferably Google Cloud Platform) for storage, compute, and orchestration
Architect and implement cloud-native data platforms on GCP, leveraging services such as BigQuery, BigTable, Dataflow, Dataproc, Pub/Sub, and Cloud Storage
Monitor pipeline performance, troubleshoot failures, and optimize processing efficiency
Collaborate with analytics, application, and business teams to understand data requirements
Ensure best practices around security, scalability, and maintainability
Ensure data quality, reliability, security, governance, and compliance with enterprise standards
Qualification
Required
5 + years of experience as a Data Engineer or similar role
Strong experience with DAG orchestration (e.g., Apache Airflow)
Solid understanding of cloud technologies, preferably Google Cloud Platform (GCP)
Advanced proficiency in SQL for data processing and transformations
Hands-on experience running and tuning Apache Spark jobs
Experience developing APIs using Python and FastAPI
Strong understanding of data quality frameworks, checks, and validation techniques
Proficiency in Python, Java, Scala, or PySpark, with strong SQL expertise
Hands-on experience with GCP data services, including BigQuery, BigTable, Dataproc, Dataflow, and cloud-native ETL patterns
Experience with software delivery methodologies such as Agile, Scrum, and CI/CD practices
Strong analytical and problem-solving skills
Ability to work independently and in cross-functional teams
Good communication and documentation skills
Company
CoreAi Consulting
From strategic consulting to solution development and ongoing support, our comprehensive generative AI services are specifically designed to propel advancements within your business.
11-50 employees
H1B Sponsorship
CoreAi Consulting has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (17)
2024 (1)
Funding
Current Stage
Early StageCompany data provided by crunchbase