Apply on Employer Site

Energy Solutions · 17 hours ago

Senior Data Engineer (Hybrid: Portland, OR - US)

Portland, Oregon, United States

Full-time

Hybrid

Senior Level, Lead/Staff

$140K/yr - $165K/yr

8+ years exp

Energy Solutions is a growing company focused on climate solutions through market-based programs. They are seeking a Senior Data Engineer to design, develop, and maintain data platforms that support the data needs across the organization.

Information Technology & Services

Growth Opportunities

No H1B

Responsibilities

Build, automate, and manage near-real-time scalable data ingestion pipelines for master data management, deep-learning, and predictive analytics

Build and maintain cloud native big data environments on AWS, that are highly secure, scalable, flexible, and highly performant using appropriate SQL, NoSQL and NewSQL technologies

Lead data governance and data profiling efforts to ensure data quality and proper metadata documentation for data lineage

Provide technical input into build/buy/partner decisions for all components of the data infrastructure

Partner closely with Data Scientists, BI developers, and Product Managers to design and implement data models, database schemas, data structures, and processing logic to support various data science, analytics, machine learning, and BI initiatives

Design and develop ETL (extract-transform-load) processes to validate and transform data, calculate metrics, and model features, populate data models etc., using Spark, Python, SQL, and other technologies in the AWS

Lead data governance and data profiling efforts to ensure data quality and proper metadata documentation for data lineage

Lead by example, demonstrating best practices for code development and optimization, unit testing, CI/CD, performance testing, capacity planning, documentation, monitoring, alerting, and incident response to ensure data availability, data quality, and usability

Define SLAs for data availability and correctness. Automate data availability and quality monitoring and respond to alerts when data delivery SLAs are not being met

Communicate progress across organizations and levels from individual contributor to executive. Identify and clarify the critical few issues that need action and drive appropriate decisions and actions. Communicate results clearly

Qualification

AWSSQLPythonETL DevelopmentData GovernanceNoSQLRelational DatabasesPerformance TuningData SecurityCommunicationTeam CollaborationProblem Solving

Required

A bachelor's degree in computer science or information technology plus 8 years minimum of relevant experience

High proficiency in programming languages commonly used in ETL development, such as PLSQL, SQL, Python. Ability to write efficient SQL queries, SQL store procedures, develop scripts for data transformations, and utilize programming frameworks and libraries to create/enhance ETL mappings and workflows

Expertise in utilizing AWS services, including but not limited to Amazon s3, glue, data catalog, Amazon redshift, redshift spectrum and Amazon Athena. Ability to leverage these services to build scalable, reliable, and performant data pipelines and analytics solutions

Proficiency in working with relational databases such as Postgres, Oracle, MySQL, or SQL Server. Knowledge of database design, optimization techniques, and advanced querying capabilities

Experience in performance tuning and optimizing database operations

Familiarity with data governance frameworks and data security best practices

Passion for learning new technologies, staying up to date with industry trends, and exploring innovative approaches to ETL development