RedCloud Consulting · 17 hours ago
Sr. Data Engineer - Monetization & Experimentation
United States
Full-time
Remote
Senior Level
$200K/yr - $280K/yrRedCloud Consulting is a business and IT consulting company recognized for its growth and workplace culture. They are seeking a Senior Data Engineer to design, build, and operate large-scale data pipelines that support advertising systems and drive innovation in a fast-paced environment.
ConsultingInformation TechnologyLegal
No H1B
Responsibilities
Design, build, and maintain large-scale data pipelines supporting bidding, targeting, experimentation, and measurement
Optimize workflows under real-world constraints—sampling limits, narrow data windows, service ceilings, and high-throughput requirements
Ensure reliability, performance, and cost-efficiency across batch and streaming workloads
Build scalable solutions that deliver value today while creating a path for evolution over time
Own pipelines end-to-end, from design to deployment to maintenance
Diagnose, resolve, and eliminate recurring issues to strengthen system stability
Reduce duplicated workflows, wasted compute, and operational overhead
Establish engineering best practices around quality, testing, deployment, and observability
Partner closely with platform teams, infra teams, and other data/ML groups to unblock scaling constraints
Work directly with Data Science, ML Engineering, and Product to deliver features, unblock analysis, and support experimentation
Translate ambiguous requirements into well-structured engineering solutions
Operate with urgency in an environment where priorities shift quickly
Drive initiatives from zero to one with minimal guidance
Innovate within constraints and deliver solutions that work in the real world—not just on paper
Qualification
Required
Deep expertise in at least one major big data ecosystem (e.g., Spark, Flink, EMR, Hadoop-style systems)
Strong proficiency with distributed compute frameworks (Spark strongly preferred)
Experience building and operating pipelines at massive scale, including challenges like: Streaming ingestion, High-volume batch ETL, Strict SLAs or constrained windows, Sampling and stateful processing
Hands-on experience with cloud-based data infrastructure (AWS or equivalent), including: Object storage (e.g., S3), Data lake formats (Iceberg/Hudi-style tables), SQL engines (Redshift, Athena, Snowflake, etc.), Streaming frameworks (Kinesis or Kafka-style systems)
Strong coding skills in Python, Java, and/or Scala
Experience with infrastructure-as-code frameworks (e.g., Terraform)
Strong engineering discipline with Git, CI/CD, testing frameworks, and automated deployment processes
Exposure to advertising, experimentation, or ML-driven systems is highly beneficial: Real-time bidding / auction systems, Feature pipelines or feature stores, Measurement and attribution systems, Online/offline ML model pipelines
High ownership mindset: 'I will figure it out and ship it.'
Bias for action and comfort moving quickly with incomplete requirements
Strong ability to operate independently and deliver end-to-end solutions
Adaptability in dynamic environments with fast-changing priorities
Curiosity, learning speed, and enjoyment of solving hard scaling problems
Cloud & Compute: AWS, Kubernetes, Lambda, Step Functions
Data Lake: S3, Iceberg/Hudi-like table formats
Analytics: Redshift, Athena, EMR/Spark
Batch & Orchestration: Spark, Airflow-style orchestration
ML/Modeling: SageMaker or custom ML platforms
Languages: Python, Java, Scala, SQL
Infra & DevOps: Terraform, Git, CI/CD
Company
RedCloud Consulting
RedCloud is a business and technology consulting firm providing local companies with expert-level support for over two decades.
Founded in 1995
201-500 employees
Funding
Current Stage
Growth StageLeadership Team
Brett Alston
Managing Partner, Owner
Brett Clifton
Managing Partner, Owner
Company data provided by crunchbase