Seamless.AI · 2 days ago

Principal Data Engineer

United States

Full-time

Remote

Senior Level

7+ years exp

Wonder how qualified you are to the job?

Maximize your interview chances

Artificial Intelligence (AI)Information Technology

Actively Hiring

Insider Connection @Seamless.AI

Discover valuable connections within the company who might provide insights and potential referrals, giving your job application an inside edge.

Responsibilities

Design, develop, and maintain robust and scalable ETL pipelines to acquire, transform, and load data from various sources into our data ecosystem.

Collaborate with cross-functional teams to understand data requirements and develop efficient data acquisition and integration strategies.

Implement data transformation logic using Python and other relevant programming languages and frameworks.

Utilize AWS Glue or similar tools to create and manage ETL jobs, workflows, and data catalogs.

Optimize and tune ETL processes for improved performance and scalability, particularly with large data sets.

Apply methodologies and techniques for data matching, deduplication, and aggregation to ensure data accuracy and quality.

Implement and maintain data governance practices to ensure compliance, data security, and privacy.

Collaborate with the data engineering team to explore and adopt new technologies and tools that enhance the efficiency and effectiveness of data processing.

Qualification

Find out how your skills align with this job's requirements. If anything seems off, you can easily click on the tags to select or unselect skills to reflect your actual expertise.

ETL ProcessesData IntegrationSparkAWS Pipeline Development