RWE · 2 days ago
Software Engineer - Machine Learning Pipelines
Wonder how qualified you are to the job?
Clean EnergyEnergy
Insider Connection @RWE
Responsibilities
Design, develop, and operate complex machine learning and multi-faceted data fusion pipelines in a production environment
Collaborate closely with data scientists, machine learning engineers, and scientists to integrate machine learning models into scalable production systems
Develop reusable components, libraries, and frameworks to streamline pipeline development and maintenance
Optimize pipeline performance, reliability, and resource utilization
Continually expand knowledge of the latest advancements in machine learning technologies, cloud services, and software engineering best practices with an eye towards identifying and integrating exceptional new components into the AI lab’s production repertoire
Contribute to a team culture where diverse viewpoints, backgrounds and expertise are welcomed
Qualification
Find out how your skills align with this job's requirements. If anything seems off, you can easily click on the tags to select or unselect skills to reflect your actual expertise.
Required
2+ years of experience building, deploying, and monitoring complex pipelines that operate at an hours-level cadence
Experience in large scale data processing and distributed systems such as Apache Spark
Proficiency in Python, Julia, and/or Mojo
Experience with cloud computing platforms such as AWS, Azure, or Google Cloud Platform
Strong understanding of containerization technologies (Docker, Kubernetes) and orchestration tools
Familiarity with version control systems (e.g., Git), continuous integration/continuous deployment (CI/CD) pipelines, and infrastructure-as-code tools
Excellent problem-solving skills, attention to detail, and a passion for learning and innovation
Strong communication and collaboration skills, with the ability to work effectively in cross-functional teams and contribute to a positive team climate
Preferred
Experience with stream processing frameworks (e.g., Apache Kafka, Apache Flink) for real-time data ingestion and processing
Knowledge of serverless computing and event-driven architectures for building scalable and cost-effective pipelines
Experience with Google Cloud Platform MLOps tools (e.g., Vertex AI)
Familiarity with monitoring, logging, and debugging tools for distributed systems (e.g., Prometheus, ELK Stack)
Proficiency in database technologies for data storage and retrieval (e.g., PostgreSQL, MongoDB, BigQuery)
Understanding of security best practices for securing data pipelines
Expertise working with large geospatial weather and climate datasets as well as the Pangeo stack
Benefits
Task oriented and hybrid working model
Diverse and multicultural team in a highly dynamic and rapidly growing business
Relocation expenses for moving to the greater Seattle area if you are not already local
Company
RWE
RWE is pan-European green energy company focusing on sustainability.
Funding
Current Stage
Public CompanyTotal Funding
$2B2024-04-12Post Ipo Debt· $2B
2006-08-01Acquired· by Advent International ($2.17B)
2002-01-11IPO· etr:RWE
Recent News
Energy Storage News
2024-05-29
2024-05-29
Company data provided by crunchbase