Be an early applicantLess than 25 applicants

Company

The University of Texas at Austin · 13 hours ago

Lead Data Engineer

Austin, TX

Full-time

Remote

Senior Level, Lead/Staff

$130K/yr - $144K/yr

8+ years exp

Maximize your interview chances

Corporate TrainingEducation

Actively Hiring

No H1B

Insider Connection @The University of Texas at Austin

Discover valuable connections within the company who might provide insights and potential referrals.
Get 3x more responses when you reach out via email instead of LinkedIn.

Responsibilities

Design, plan, develop and deploy production quality enterprise level data pipelines using Cloud technologies with a team of data engineers using new and emerging industry standard tools, frameworks, and best practices.

Conduct and participate in code reviews for a high-quality code.

Contribute and ensure industry best practices are implemented for data engineering, SDLC, process improvements.

Identify and address bottlenecks in data pipelines while looking for optimizations, performance, and scalability.

Develop systems to ensure data quality, consistency, cost optimization and security compliances.

Contribute to coding, debugging, and solving complex engineering challenges.

Lead technical knowledge share sessions by discussing current industry trends in Data Engineering.

Define project milestones, timelines, and deliverables for on-time delivery.

Collaborate with both internal and external stakeholders like Data or System Architects, business users, Agile team and other D2I internal groups.

Manage, prioritize, and track team and project work while balancing some urgent requests.

Actively communicate and manage risks, roadblocks, and resources allocations for on-time delivery of the projects.

Supervise a team of data engineers (2-5 individuals).

Provide guidance, mentorship and career development to data engineers by fostering a culture of learning and professional growth.

Participate in hiring and recruitment activities for new team members.

Conduct regular performance reviews, provide feedback, and set development goals for team members.

Promote collaboration within the team and across departments for smooth execution of projects.

Provide regular updates on project status, team performance, and technical challenges to technical and non-technical audience.

Provide constant feedback to encourage a growth mindset and continuous learning with direct reports.

Ensure all project deliverables (technical, process and best practices) are documented for future reference and training.

Keep abreast with new data engineering technologies, trends and methodologies and introduce innovative solutions when appropriate.

Other duties as assigned.

Qualification

Find out how your skills align with this job's requirements. If anything seems off, you can easily click on the tags to select or unselect skills to reflect your actual expertise.

Data EngineeringAmazon Web ServicesPythonETL/ELT pipelinesCI/CD systemsSQLData LakesAgile methodologiesTeam managementCloud technologiesSystems analysis

Required

BS or MS degree in Computer Science, Information Systems, Engineering, or equivalent professional experience.

5 years of Data Engineering experience with Amazon Web Services.

Eight plus years of experience in designing, implementing, troubleshooting, and monitoring complex production data pipelines or enterprise software solutions.

3 years of experience managing a team of software and (or) data engineers.

Expert programming skills in Python, CDK scripting and SQL knowledge including stored procedures.

Experience in designing and implementing complex ETL or ELT pipelines in relational and non-relational databases including Data Lakes.

Three years of experience developing and implementing Continuous Integration and Continuous Delivery (CI/CD) systems.

Five plus years of experience developing and implementing test driven architecture solutions.

Experience supervising junior team members, either in project-based work or formally via managerial reporting.

Expert in systems analysis, design, development, quality assurance, and integration methodologies.

Excellent problem solving, analytical and troubleshooting skills.

Excellent verbal and written communication to communicate both to technical and non-technical audiences.

Relevant education and experience may be substituted as required.

Preferred

Eight years of experience in Data Engineering or related field with at least five years of hands-on experience designing and building complex data pipelines using cloud-based infrastructure, preferably AWS Technologies.

Five years of experience or more with Agile software development methodologies and tracking software.

Three plus years of experience managing teams. Experience in identifying and implementing software development.

Experience with Data Lakes, Delta Lakes, Lakehouse, Lakehouse, Spark, Hive, Apache Airflow, Testing Frameworks, Streaming Pipelines, Kafka etc., is a plus.