Mastercard · 2 months ago
Software Engineer II - Tester (Big Data & Cloudera Ecosystem)
O'Fallon, Missouri
Full-time
Onsite
Mid Level
$92K/yr - $147K/yr
3+ years expMastercard powers economies and empowers people in 200+ countries and territories worldwide. They are seeking a Software Engineer II - Tester to join their Big Data Engineering Quality team, responsible for testing and validating data engineering applications built across the Cloudera Manager ecosystem.
FinanceFinancial ServicesInformation TechnologyPayments
Responsibilities
Test and validate distributed data engineering applications across Cloudera Manager components: Kafka, Spark, HBase, Hive, Solr, NiFi, Hue, Livy, Unravel
Test Apache NiFi pipelines including processors, flow configurations, data lineage, provenance, NiFi registry flows, and integration with downstream systems
Validate ingestion pipelines, streaming jobs, batch ETL processes, orchestration flows, and REST APIs
Perform system, integration, functional, regression, and performance testing across large-scale data systems
Build robust automation frameworks using any of the following languages: Python, Java, Scala, or Bash, based on the codebase under test
Develop automation for Spark job validation, Kafka event checks, HBase/Hive schema validation, NiFi flow output validation, and end-to-end data pipeline coverage
Create automated tests for REST APIs, backend services, schema checks, and data correctness
Build and maintain UI automation using Selenium for dashboards such as Hue, Livy UI and other internal tools
Maintain and enhance Jenkins CI/CD pipelines to support automated test execution and continuous integration
Leverage Docker or similar runtime environments to execute tests at scale
Integrate automation suites into the build/release pipeline for seamless promotion and deployment
Validate SQL queries, HBase tables, Hive datasets, Kafka event streams, NiFi flow files, and NoSQL outputs
Ensure data accuracy, schema integrity, SLA compliance, and end-to-end pipeline reliability
Conduct deep validation of NiFi processors (PutFile, GetKafka, MergeContent, RouteOnAttribute, ExecuteScript, etc.) and downstream integrations
Collaborate with Data Engineers, Big Data Platform Engineers, and QA peers to ensure complete test coverage
Participate in Agile ceremonies—story pointing, defining acceptance criteria, backlog refinement, sprint reviews, and retrospectives
Identify testing risks, propose mitigation strategies, and champion quality-first practices across the team
Qualification
Required
3–5 years of hands-on experience in software testing or QA engineering, ideally within Big Data environments
Strong proficiency in any one (or more) of the following languages: Python, Java, Scala, Bash
Experience testing big-data applications on Kafka, Spark, Cloudera/Hadoop, HBase, Hive, Solr, NiFi, Hue, Livy, or similar components
Practical experience validating NiFi pipelines: processors, flow logic, provenance, connections, and integrations
Strong experience with test automation frameworks (API, backend, ETL/data validation)
UI automation experience using Selenium
Proficiency in SQL and NoSQL data validation tasks
Hands-on experience with Jenkins or similar CI/CD tools
Provide production support for the applications as a developer/tester for two weeks in rotated shift
Solid understanding of Agile/Scrum methodologies
Strong analytical, debugging, communication, and problem-solving skills
Preferred
Experience with Unravel, Cloudera Navigator, or monitoring tools
Exposure to Docker, Kubernetes, or cloud platforms (AWS, Azure, GCP)
Familiarity with performance testing of distributed systems
Benefits
Insurance (including medical, prescription drug, dental, vision, disability, life insurance)
Flexible spending account and health savings account
Paid leaves (including 16 weeks of new parent leave and up to 20 days of bereavement leave)
80 hours of Paid Sick and Safe Time
25 days of vacation time and 5 personal days, pro-rated based on date of hire
10 annual paid U.S. observed holidays
401k with a best-in-class company match
Deferred compensation for eligible roles
Fitness reimbursement or on-site fitness facilities
Eligibility for tuition reimbursement
Company
Mastercard
Mastercard is a financial network that processes payments between banks and cardholders.
10001+ employees
H1B Sponsorship
Mastercard has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (540)
2024 (319)
Funding
Current Stage
Public CompanyTotal Funding
unknown2006-05-25IPO
Leadership Team
Michael Miebach
Chief Executive Officer
Ed McLaughlin
President & Chief Technology Officer
Recent News
2026-02-04
Fintechnews Middle East
2026-02-04
2026-02-04
Company data provided by crunchbase