Apply on Employer Site

Comcast · 6 hours ago

Data Engineer - Python, PySpark, AWS

Philadelphia, PA

Full-time

Onsite

Senior Level

5+ years exp

Comcast is a Fortune 30 global media and technology company that creates innovative products and solutions for millions of customers. The Data Engineer role focuses on developing data structures and pipelines, ensuring data quality, and optimizing data access and consumption methods.

InternetTelecommunicationsTVVideoWeb Hosting

H1B Sponsor Likely

Responsibilities

Develops data structures and pipelines aligned to established standards and guidelines

Ensures data quality during ingest, processing, and final load to target tables

Creates standard ingestion frameworks for structured and unstructured data

Checks and reports on the quality of data being processed

Creates standard methods for end users and downstream applications to consume data, including:

Database views

Extracts

Application Programming Interfaces (APIs)

Develops and maintains information systems (e.g., data warehouses, data lakes), including data access APIs

Implements solutions via data architecture, data engineering, or data manipulation on:

On-prem platforms (e.g., Kubernetes, Teradata)

Cloud platforms (e.g., Databricks)

Determines appropriate storage platforms across on-prem (minIO, Teradata) and cloud (AWS S3, Redshift) based on privacy, access, and sensitivity requirements

Understands data lineage from source to final semantic layer, including transformation rules

Enables faster troubleshooting and impact analysis during changes

Collaborates with technology and platform management partners to optimize data sourcing and processing rules

Establishes design standards and assurance processes for software, systems, and applications development

Reviews business and product requirements for data operations

Suggests changes and upgrades to systems and storage to accommodate ongoing needs

Develops strategies for data acquisition, archive recovery, and database implementation

Manages data migrations/conversions and troubleshooting of data processing issues

Applies data sensitivity and customer data privacy rules and regulations consistently in all Information Lifecycle Management activities

Monitors system notifications and logs to ensure database and application quality standards

Solves abstract problems by reusing data files and flags

Resolves critical issues and shares knowledge such as trends, aggregates, and volume metrics regarding specific data sources

Qualification

PythonAWSPySparkDatabricksBig Data ArchitectureData ModelingApache SparkKafka / KinesisApache AirFlowGitHubTableauPerformance TuningJiraShell CommandsData Governance

Required

Bachelor's Degree

5-7 Years of Relevant Work Experience

Python

AWS (including S3, Redshift)

PySpark

Databricks

Develops data structures and pipelines aligned to established standards and guidelines

Ensures data quality during ingest, processing, and final load to target tables

Creates standard ingestion frameworks for structured and unstructured data

Checks and reports on the quality of data being processed

Creates standard methods for end users and downstream applications to consume data, including Database views, Extracts, Application Programming Interfaces (APIs)

Develops and maintains information systems (e.g., data warehouses, data lakes), including data access APIs

Implements solutions via data architecture, data engineering, or data manipulation on On-prem platforms (e.g., Kubernetes, Teradata) and Cloud platforms (e.g., Databricks)