Klaxontech Inc ยท 5 hours ago
Sr Data Engineer (Databricks)
New York, United States
Full-time
Remote
Senior Level
Maximize your interview chancesIT Services and IT Consulting
Insider Connection @Klaxontech Inc
Get 3x more responses when you reach out via email instead of LinkedIn.
Responsibilities
Develop and optimize ETL processes to extract, transform, and load data from various sources into data lakes and data warehouses.
Data Pipeline Development: Design, develop, and maintain robust data pipelines using Databricks to process and transform large volumes of data.
ETL Process Management: Implement ETL (Extract, Transform, Load) processes to integrate data from various sources into Databricks, ensuring data quality and integrity.
Data Integration: Integrate Databricks with other data storage solutions and data lakes, ensuring seamless data flow and accessibility.
Utilize Databricks, Python, and PySpark to design and build scalable data pipelines for processing and analyzing large datasets.
Ensure data accuracy, consistency, and integrity through effective data validation, quality checks, and error handling.
Optimize performance and scalability of ETL jobs, data flows, and data storage environments.
Monitor data pipelines and workflows, troubleshoot issues, and implement improvements to enhance reliability and efficiency.
Document data engineering processes, architecture, and data lineage.
Qualification
Find out how your skills align with this job's requirements. If anything seems off, you can easily click on the tags to select or unselect skills to reflect your actual expertise.
Required
Databricks
Python, Pyspark
Azure
Proven experience as a Data Engineer with strong knowledge of ETL processes and data warehousing concepts.
Strong hands-on experience with Databricks for big data processing and analytics.
Proficiency in Python and PySpark for data transformation and processing.
Strong expertise in Azure data services
Strong problem-solving skills and the ability to work in a fast-paced environment.
Preferred
Experience with Jupyter Notebook
Company
Klaxontech Inc
Klaxontech Inc is an IT Consulting and staffing service provider. Company with a long history, incorporated in the state of New York. Klaxontech Inc.
Founded in 2013
51-200 employees
H1B Sponsorship
Klaxontech Inc has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2023 (4)
2022 (17)
2021 (14)
2020 (34)
Funding
Current Stage
Growth StageCompany data provided by crunchbase
