76 applicants

Company

Coupa Software · 4 hours ago

Senior Data Engineer

United States

Full-time

Remote

Senior Level

$155K/yr - $182K/yr

6+ years exp

Maximize your interview chances

Cloud InfrastructureFinTech

H1B Sponsor Likely

Insider Connection @Coupa Software

Discover valuable connections within the company who might provide insights and potential referrals.
Get 3x more responses when you reach out via email instead of LinkedIn.

Responsibilities

Create and maintain optimal data pipeline architecture

Optimize Spark clusters for efficiency and performance by implementing robust monitoring systems to identify bottlenecks using data and metrics. Provide actionable recommendations for continuous improvement

Optimize Spark clusters for efficiency and performance

Assemble large, complex data sets that meet functional / non-functional business requirements

Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc

Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies

Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics

Work with stakeholders including the Executive, Product, Data, and Design teams to assist with data-related technical issues and support their data infrastructure needs

Keep our data separated and secure across national boundaries through multiple data centers and AWS regions

Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader

Work with data and analytics experts to strive for greater functionality in our data systems

Qualification

Find out how your skills align with this job's requirements. If anything seems off, you can easily click on the tags to select or unselect skills to reflect your actual expertise.

Data EngineeringSQLPythonAWSSparkData WarehousingAPI DevelopmentNoSQL DatabasesJavaData Pipeline ManagementKafkaPostgresCassandraAzkabanLuigiAirflowEC2EMRRDSRedshiftStormSpark-StreamingProject ManagementCross-Functional Collaboration

Required

6-10 years of experience in a Data Engineer role

Graduate degree in Computer Science, Statistics, Informatics, Information Systems, or another quantitative field

Advanced working SQL knowledge and experience working with relational databases

Experience with processing large workloads and complex code on Spark clusters

Proven experience in setting up monitoring for Spark clusters and driving optimization based on insights and findings

Experience in designing and implementing scalable Data Warehouse solutions to support analytical and reporting needs

Experience with API development and design with REST or GraphQL

Experience building and optimizing ‘big data’ data pipelines, architectures, and data sets

Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement

Strong analytic skills related to working with unstructured datasets

Build processes supporting data transformation, data structures, metadata, dependency, and workload management

Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores

Strong project management and organizational skills

Experience supporting and working with cross-functional teams in a dynamic environment

Experience with object-oriented/object function scripting languages: Python, Java, Etc.

Expertise with Python is a MUST

Experience with big data tools: Spark, Kafka, etc.

Experience with relational SQL and NoSQL databases, including Postgres and Cassandra

Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.

Experience with AWS cloud services: EC2, EMR, RDS, Redshift

Working knowledge of stream-processing systems: Storm, Spark-Streaming, etc.

Company

Coupa Software

Glassdoor

3.8

Coupa makes margins multiply through its community-generated AI and industry leading total spend management platform for businesses large and small.

Founded in 2006

Foster City, California, USA

1001-5000 employees

https://www.coupa.com/

H1B Sponsorship

Coupa Software has a track record of offering H1B sponsorships. Please note that this does not guarantee sponsorship for this specific role. Below presents additional info for your reference. (Data Powered by US Department of Labor)

Distribution of Different Job Fields Receiving Sponsorship

Represents job field similar to this job

Trends of Total Sponsorships

2023 (21)

2022 (69)

2021 (75)

2020 (32)