Women in Data® · 1 day ago
Google Cloud Platform Data Engineer
Bristol, CT
Full-time
Onsite
Lead/StaffWomen in Data® is an innovation and transformation consultancy that believes in the power of ingenuity to build a positive-human future. As a Principal GCP Data Engineer, you will lead the development of data-driven solutions using Google Cloud, guiding teams and ensuring adherence to best practices in the software development lifecycle.
Information Services
Hiring Manager
Eszter S.
Responsibilities
Develop robust data processing jobs using tools such as Google Cloud Dataflow, Dataproc and BigQuery
Design and deliver automated data pipelines that use orchestration tools such as Cloud Composer
Design end-to-end solutions and contribute to architecture discussions beyond data processing
Own the development process for your team, building strong principles and putting robust methods and patterns in place across architecture, scope, code quality and deployments
Shape team behaviour for writing specifications and acceptance criteria, estimating stories, sprint planning and documentation
Actively define and evolve PA’s data engineering standards and practices, ensuring we maintain a shared, modern and robust approach
Lead and influence technical discussions with client stakeholders to achieve the collective buy-in required to be successful
Coach and mentor team members, regardless of seniority, and work with them to build their expertise and understanding
Qualification
Required
Experience delivering and deploying production-ready data processing solutions using BigQuery, Pub/Sub, Dataflow and Dataproc
Experience developing end-to-end solutions using batch and streaming frameworks such as Apache Spark and Apache Beam
Expert understanding of when to use a range of data storage technologies including relational/non-relational, document, row-based/columnar data stores, data warehousing and data lakes
Expert understanding of data pipeline patterns and approaches such as event-driven architectures, ETL/ELT, stream processing and data visualisation
Experience working with business owners to translate business requirements into technical specifications and solution designs that satisfies the data requirements of the business
Experience working with metadata management products such as Cloud Data Catalog and Collibra and Data Governance tools like Dataplex
Experience in developing solutions on GCP using cloud-native principles and patterns
Experience building data quality alerting and data quarantine solutions to ensure downstream datasets can be trusted
Experience implementing CI/CD pipelines using techniques including as git code control/branching, automated tests and automated deployments
Comfortable working in an Agile team using Scrum or Kanban methodologies
Preferred
Experience of working on migrations of enterprise scale data platforms including Hadoop and traditional data warehouses
An understanding of machine learning model development lifecycle, feature engineering, training and testing
Good understanding or hands-on experience of Kafka
Experience as a DBA or developer on RDBMS such as PostgreSQL, MySQL, Oracle or SQL Server
Experience designing data applications to meet non-functional requirements such as performance and availability
Benefits
Private medical insurance
Interest free season ticket loan
25 days annual leave with the opportunity to buy 5 additional days
Company pension scheme
Annual performance-based bonus
Life and Income protection insurance
Tax efficient benefits (cycle to work, give as you earn, childcare benefits)
Voluntary benefits (Dental, critical illness, spouse/partner life assurance)
Company
Women in Data®
Women are hugely under-represented in the Data industry – as things stand, male Analysts and Data Scientists outnumber their female colleagues 4 to 1.
Founded in 2014
11-50 employees
Funding
Current Stage
Early StageCompany data provided by crunchbase