Rackspace Technology · 2 months ago
Sr Big Data Engineer - Oozie and Pig (GCP)
United States
Full-time
Remote
Mid, Senior Level
$116K/yr - $170K/yr
5+ years expRackspace is a multicloud solutions expert that combines expertise with leading technologies to deliver end-to-end solutions. They are seeking a Senior Big Data Engineer to design and develop scalable batch processing systems, manage complex data workflows, and leverage GCP tools to build cloud-native big data solutions.
Web Hosting
Responsibilities
Design and develop scalable batch processing systems using technologies like Hadoop, Oozie, Pig, Hive, MapReduce, and HBase, with hands-on coding in Java or Python (Java is a must)
Must be able to lead Jira Epics
Write clean, efficient, and production-ready code with a strong focus on data structures and algorithmic problem-solving applied to real-world data engineering tasks
Develop, manage, and optimize complex data workflows within the Apache Hadoop ecosystem, with a strong focus on Oozie orchestration and job scheduling
Leverage Google Cloud Platform (GCP) tools such as Dataproc, GCS, and Composer to build scalable and cloud-native big data solutions
Implement DevOps and automation best practices, including CI/CD pipelines, infrastructure as code (IaC), and performance tuning across distributed systems
Collaborate with cross-functional teams to ensure data pipeline reliability, code quality, and operational excellence in a remote-first environment
Qualification
Required
Bachelors's degree in Computer Science, software engineering or related field of study
5+ years of experience in customer-facing software/technology or consulting
5+ years of experience with 'on-premises to cloud' migrations or IT transformations
5+ years of experience building and operating solutions built on GCP
Proficiency in Oozie, Airflow, Map Reduce, Java
Strong programming skills with Java (specifically Spark), Python, Pig, and SQL
Expertise in public cloud services, particularly in GCP
Proficiency in the Apache Hadoop ecosystem with Oozie, Pig, Hive, Map Reduce
Proficiency in Oozie and Pig
Must be able to lead Jira Epics
Proficiency in Java or Python
Proven experience in engineering batch processing systems at scale
Experienced in Infrastructure and Applied DevOps principles in daily work
Utilize tools for continuous integration and continuous deployment (CI/CD), and Infrastructure as Code (IaC) like Terraform to automate and improve development and release processes
Preferred
Programming proficiency in Python
Familiarity with BigTable and Redis
Experience with managed cloud services
Understanding of cloud-based batch processing systems
Experience in customer-facing software/technology or consulting
Experience with 'on-premises to cloud' migrations or IT transformations
Experience building and operating solutions built on GCP
Company
Rackspace Technology
Rackspace Technology is a leading end-to-end hybrid cloud and AI solutions company.
5001-10000 employees
H1B Sponsorship
Rackspace Technology has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (20)
2024 (26)
2023 (22)
2022 (42)
2021 (52)
2020 (65)
Funding
Current Stage
Late StageTotal Funding
unknown2016-08-08Acquired
Leadership Team
Mark Marino
EVP, Chief Financial Officer
Recent News
GlobeNewswire News Room
2024-01-15
Company data provided by crunchbase