This job has closed.

Tata Consultancy Services · 21 hours ago

Lead Data Engineer

Philadelphia, PA

Full-time

Onsite

Senior Level, Lead/Staff

$100K/yr - $120K/yr

3+ years exp

Tata Consultancy Services is a global leader in IT services and consulting, and they are seeking a Lead Data Engineer. The role involves architecting and maintaining data pipelines using AWS services, designing data models, and implementing ETL workflows while mentoring junior engineers.

Business Information SystemsConsultingInformation TechnologyIT Management

H1B Sponsor Likely

Responsibilities

Architect and maintain data pipelines using AWS native services (Glue, Kinesis, Lambda, S3, Redshift)

Design and optimize data models on AWS Cloud leveraging Redshift, RDS, and S3

Implement ETL/ELT workflows and PySpark jobs for data ingestion, transformation, and storage

Operationalize self-service data preparation tools (e.g., Trifacta) on AWS

Conduct performance engineering for large-scale data lakes in production environments

Participate in design workshops, provide trade-offs and recommendations for solution architecture

Mentor engineers on coding best practices, problem-solving, and AWS service utilization

Define code review processes, deployment strategies, and ensure compliance with security standards

Collaborate with System Architect and Scrum Master to manage dependencies, risks, and blockers. Support test strategy, defect resolution, and root cause analysis during warranty periods

Maintain documentation in Confluence and ensure team alignment on standards and practices

Qualification

PySparkAWS servicesSQLPython developmentAWS certificationDocker/KubernetesData governanceAgile experienceCommunication skills

Required

3+ years relevant experience in designing and deploying big data applications and ETL jobs using PySpark APIs/SparkSQL

Strong experience with AWS services across multiple domains: Collection: Kinesis, DMS; Storage: S3, RDS, Redshift, DynamoDB; Analytics & ML: Glue, EMR, Athena, SageMaker, Bedrock; Compute: EC2, Lambda, ECS; Security: IAM, KMS, SSE

Proficiency in SQL and relational databases (Oracle, SQL Server, Teradata); expert-level query tuning

Hands-on experience with Python development, REST APIs (AWS API Gateway, Node.js), and CI/CD pipelines using GitHub

Familiarity with file formats (JSON, Parquet, Avro) and Linux/Unix shell scripting

Exposure to Docker/Kubernetes, Delta Lake APIs, and data quality frameworks

AWS certification (Developer Associate or higher) preferred

Proven ability to mobilize and motivate teams, set technical direction, and resolve conflicts

Experience working in Agile environments, managing dependencies, and delivering under ambiguity

Strong communication skills for stakeholder engagement and cross-functional collaboration

BACHELOR OF COMPUTER SCIENCE

Preferred

Experience with BI tools (QuickSight, Tableau)

Knowledge of data governance, security, and compliance standards

Familiarity with performance testing and observability tools for data pipelines

Benefits

Discretionary Annual Incentive.

Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.

Family Support: Maternal & Parental Leaves.

Insurance Options: Auto & Home Insurance, Identity Theft Protection.

Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.

Time Off: Vacation, Time Off, Sick Leave & Holidays.

Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.

Company

Tata Consultancy Services

Glassdoor3.8

Tata Consultancy Services is a business solutions company that specializes on information technology services and consulting.

Founded in 1968

Mumbai, Maharashtra, IND

10001+ employees

http://www.tcs.com

H1B Sponsorship

Tata Consultancy Services has a track record of offering H1B sponsorships. Please note that this does not guarantee sponsorship for this specific role. Below presents additional info for your reference. (Data Powered by US Department of Labor)

Distribution of Different Job Fields Receiving Sponsorship

Represents job field similar to this job

Trends of Total Sponsorships

2025 (7880)

2024 (9690)

2023 (8537)

2022 (11159)

2021 (9813)

2020 (11984)