Apply on Employer Site

AECOM · 4 months ago

Lead Data Engineer - Hybrid (Houston or Dallas, TX)

Houston, TX

Full-time

Hybrid

Senior Level, Lead/Staff

8+ years exp

AECOM is a global infrastructure consulting firm committed to delivering a better world. We are seeking a Lead Data Engineer with deep AWS expertise to guide the design, development, and optimization of enterprise-scale data pipelines and products, while providing leadership to a team of data engineers and collaborating with data architects.

Civil EngineeringConstructionConsultingEnergyGovernmentInformation Technology

No H1B

Responsibilities

Lead the end-to-end design, development, and optimization of scalable data pipelines and products on AWS, leveraging services such as S3, Glue, Redshift, Athena, EMR, and Lambda

Provide day-to-day technical leadership and mentorship to a team of data engineers—setting coding standards, reviewing pull requests, and fostering a culture of engineering excellence

Partner with data architects to define target data models, integration patterns, and platform roadmaps that align with AECOM’s enterprise data strategy

Own project planning, estimation, resourcing, and sprint management for major data initiatives, ensuring on-time, on-budget delivery

Implement robust ELT/ETL frameworks, including orchestration (e.g., Airflow or AWS Step Functions), automated testing, and CI/CD pipelines to enable rapid, reliable deployments

Champion data quality, governance, and security; establish monitoring, alerting, and incident-response processes that keep data products highly available and trustworthy

Optimize performance and cost across storage, compute, and network layers; conduct periodic architecture reviews and tuning exercises

Collaborate with analytics, reporting, and business teams to translate requirements into reliable, production-ready data assets that power decision-making at scale

Stay current with the AWS ecosystem and industry best practices, continuously evaluating new services and technologies to enhance AECOM’s data platform

Provide clear, concise communication to stakeholders at all levels, articulating trade-offs, risks, and recommendations in business-friendly language

Qualification

AWS expertiseData pipeline designELT/ETL frameworksPython/PySparkSQL proficiencyData lakehouse architectureWorkflow orchestrationData governanceStakeholder managementCommunication skillsDocumentation skills

Required

BA/BS in Computer Science, Information Systems, Engineering, or a related discipline plus at least 8 years of hands-on data engineering experience, or demonstrated equivalency of experience and/or education

3+ years in a technical-lead or team-lead capacity delivering enterprise-grade solutions

Deep expertise in AWS data and analytics services: e.g.; S3, Glue, Redshift, Athena, EMR/Spark, Lambda, IAM, and Lake Formation

Proficiency in Python/PySpark or Scala for data engineering, along with advanced SQL for warehousing and analytics workloads

Demonstrated success designing and operating large-scale ELT/ETL pipelines, data lakes, and dimensional/columnar data warehouses

Experience with workflow orchestration (e.g.; Airflow, Step Functions) and modern DevOps practices—CI/CD, automated testing, and infrastructure-as-code (e.g.; Terraform or CloudFormation)

Experience with data lakehouse architecture and frameworks (e.g.; Apache Iceberg)

Strong communication, stakeholder-management, and documentation skills; aptitude for translating business needs into technical roadmaps

Preferred

Master's degree in a relevant field

Solid understanding of data modeling, data governance, security best practices (encryption, key management), and compliance requirements

Experience working within similarly large, complex organizations

Experience building integrations for enterprise back-office applications

AWS Certified Data Analytics – Specialty or AWS Solutions Architect certification (or equivalent) preferred; experience with other cloud platforms is a plus

Proficiency in modern data storage formats and table management systems, with a strong understanding of Apache Iceberg for managing large-scale datasets and Parquet for efficient, columnar data storage

In-depth knowledge of data cataloging, metadata management, and lineage tools (AWS Glue Data Catalog, Apache Atlas, Amundsen) to bolster data discovery and governance

Knowledge of how machine learning models are developed, trained, and deployed, as well as the ability to design data pipelines that support these processes