Medasource ยท 5 hours ago
Data Engineer
Maximize your interview chances
Insider Connection @Medasource
Get 3x more responses when you reach out via email instead of LinkedIn.
Responsibilities
Design, develop, and implement end-to-end data solutions using Azure Databricks and other relevant technologies
Build and maintain scalable data pipelines to extract, transform, and load large volumes of data from various sources
Collaborate with cross-functional teams to gather and analyze requirements, then translate business needs into optimized data models
Ensure data quality and integrity by implementing data validation and cleansing processes
Optimize data processing and storage for improved performance and efficiency
Conduct data profiling, analysis, and modeling to identify opportunities for data-driven insights and recommendations
Monitor and troubleshoot data pipelines and processes to ensure continuous data flow and availability
Qualification
Find out how your skills align with this job's requirements. If anything seems off, you can easily click on the tags to select or unselect skills to reflect your actual expertise.
Required
Experience with no-sql databases
Experience with data migration projects (moving from on-prem to cloud)
3+ years of recent experience as a Data Engineer in an Azure environment (not heavy AWS)
Proficient in scripting with Python, as well as other languages such as PySpark, Scala, and SQL
Strong experience building pipelines with Azure Databricks and other Azure services, such as Azure Data Factory and Azure SQL Database
Experience with data integration, transformation, and orchestration frameworks such as Airflow, dbt, Fivetran, or similar
Clear, personable communicator to work closely with the business
Experience in a large organization and ability to work in a fast-paced, agile environment