MedScout · 14 hours ago

Data Engineer

United States

Full-time

Remote

Mid Level

3+ years exp

Maximize your interview chances

Health CareMedical

Insider Connection @MedScout

Discover valuable connections within the company who might provide insights and potential referrals.
Get 3x more responses when you reach out via email instead of LinkedIn.

Responsibilities

You will design, implement, and maintain scalable data pipelines to process large volumes of healthcare claims data using Databricks, Python, and PySpark.

You will develop processes to integrate multiple data sources, including healthcare claims databases, into a unified data model that powers MedScout's sales enablement platform.

You will work with Product, Customer Success, and Sales leaders to understand what our customers are looking to achieve with our platform and use those insights to inform and validate your thinking as you make design and implementation decisions.

You will collaborate with data scientists and analysts to implement data transformations that support efficiently delivering advanced analytics, market insights, and predictive modeling capabilities for the platform.

You will troubleshoot and resolve complex data pipeline issues, optimize system performance, and contribute to the continuous improvement of MedScript's data infrastructure and engineering practices.

You will optimize workloads and cluster configurations to reduce compute costs while maintaining performance, including implementing auto-scaling policies, right-sizing clusters, and monitoring resource utilization patterns.

Qualification

Find out how your skills align with this job's requirements. If anything seems off, you can easily click on the tags to select or unselect skills to reflect your actual expertise.

DatabricksPythonPySparkData quality frameworksData modelingSchema designAutomated testingVersion control (Git)CI/CDInfrastructure as codeHealthcare claims dataData pipeline optimizationTechnical specificationsPredictive modelingCross-functional collaboration

Required

3+ years of experience building, maintaining, and operating data pipelines in a modern data warehouse like Databricks, Snowflake, or AWS Redshift.

Confident using Python and PySpark.

Good understanding of data modeling and schema design, particularly in contexts involving complex relationships and high-volume data processing.

Expert with data quality frameworks, including automated testing, validation, and monitoring of data pipelines.

Familiarity with modern software development practices including version control (Git), CI/CD, and infrastructure as code.

Able to work effectively with cross-functional teams, translating business requirements into technical specifications and communicating complex technical concepts to non-technical stakeholders.