Emids · 12 hours ago

GCP Data Engineer

United States

Full-time

Remote

Senior Level

7+ years exp

Maximize your interview chances

ConsultingHealth Care

Actively Hiring

Insider Connection @Emids

Discover valuable connections within the company who might provide insights and potential referrals.
Get 3x more responses when you reach out via email instead of LinkedIn.

Responsibilities

Integrate data from diverse sources, including databases, APIs, and streaming platforms.

Optimize data processing and query performance through fine-tuning data pipelines, database configurations, and partitioning strategies.

Implement and monitor data quality checks and validations to ensure reliable data for analytics and applications.

Apply security measures to safeguard sensitive data, coordinating with security teams to ensure encryption, access controls, and regulatory compliance.

Collaborate with cross-functional teams, including data scientists, analysts, software engineers, and business stakeholders.

Design and develop data infrastructure components such as data warehouses, data lakes, and data pipelines.

Establish and maintain auditing, monitoring, and alerting mechanisms to ensure data governance and system performance.

Explore and implement new frameworks, platforms, or cloud services to enhance data processing capabilities.

Utilize DevSecOps practices to incorporate security throughout the development lifecycle.

Acquire a thorough understanding of enterprise data systems and relevant processes for project delivery.

Contribute to project estimation and provide insights to technical leads.

Participate in Agile scrum activities, project status meetings, and user story grooming/design discussions.

Analyze complex data structures from various sources and design large-scale data engineering pipelines.

Develop robust ETL pipelines, design database systems, and create data processing tools using programming skills.

Perform data engineering tasks including ETL development, testing, and deployment.

Collaborate with developers on ETL job/pipeline development and integrate components for automation.

Document data engineering processes, workflows, and systems for reference and knowledge sharing.

Ensure data accuracy, completeness, and consistency through quality checks and validation processes.

Work effectively with team members to deliver business solutions.

Qualification

Find out how your skills align with this job's requirements. If anything seems off, you can easily click on the tags to select or unselect skills to reflect your actual expertise.

GCPBigQueryPythonSQLETL developmentTeradataCloud SQLCloud Composer/AirflowDataflow/Data FusionData engineering pipelinesBash shell scriptsUNIX utilitiesUNIX commandsBI toolsFlaskJavaScriptHTMLCSSDjangoHealthcare experiencePBM experienceAgile methodologyWaterfall methodologyData quality checksCollaboration skills