Lead Software Engineer (Big Data / Cloudera Platform) jobs in United States
cer-icon
Apply on Employer Site
company-logo

Lumen Solutions Group Inc. ยท 2 months ago

Lead Software Engineer (Big Data / Cloudera Platform)

positionReston, VA
timeFull-time
remoteHybrid
seniorityLead/Staff
date8+ years exp

Lumen Solutions Group Inc. is seeking a Lead Software Engineer with expertise in Big Data platforms, particularly Cloudera and AWS environments. The role focuses on production support, performance optimization, and ensuring the reliability of data platforms that support business analytics.

Information Technology & Services
check
Growth Opportunities

Responsibilities

Lead the design, development, and support of large-scale big data solutions across Cloudera and AWS environments
Manage production support operations, including incident resolution, performance tuning, and system optimization
Monitor and maintain Cloudera clusters to ensure uptime, scalability, and compliance with organizational SLAs
Troubleshoot and support components within the Hadoop ecosystem (Hive, HBase, Solr, Impala, Spark, Flume)
Administer and integrate Kafka and NiFi data flows for real-time data streaming and ingestion
Automate workflows and system tasks using Python and Shell scripting
Support and manage data migration projects (e.g., CDH to CDP) and enforce DevSecOps and CI/CD best practices
Collaborate with data engineers, cloud architects, and analytics teams to optimize pipeline performance and system security (Ranger, Kerberos)
Mentor junior engineers and uphold high standards for code quality, deployment, and documentation

Qualification

Cloudera (CDP/CDH)AWSBig Data platform supportKafka administrationApache NiFiHadoop ecosystemPythonShell scriptingAgile methodologiesDevSecOpsData governance frameworksAnalytics/ML pipelines

Required

8+ years of experience in software development and Big Data platform support
Proven expertise with Cloudera (CDP/CDH) cluster administration, tuning, and upgrades
Strong hands-on experience with AWS cloud services (EC2, S3, EMR, IAM, Lambda, etc.)
Proficiency in Kafka administration, Kafka Streams API, and integration with MQ systems
Expertise in Apache NiFi flow management and registry configuration
Strong knowledge of HBase, Solr, Hive, Impala, Oozie, YARN, Ranger, and related tools
Solid scripting and automation experience using Python and Shell
Experience supporting Cloudera and MongoDB in cloud environments
Familiarity with Cloud Disaster Recovery strategies and Big Data pipeline automation
Excellent analytical, troubleshooting, and performance-tuning abilities

Preferred

Experience with analytics/ML pipelines and data governance frameworks
Hands-on involvement in CDP migration projects and DevSecOps environments
Familiarity with Agile methodologies and collaborative development practices

Company

Lumen Solutions Group Inc.

twitter
company-logo
Lumen Solutions Inc., is a dynamic small and minority-owned, Disadvantaged Business Enterprise headquartered in Virginia, USA.
dateFounded in 2010
location
Parkland, FL, US
people-size51-200 employees
web-link
http://www.lumensolutions.com

Funding

Current Stage
Growth Stage

Leadership Team

leader-logo
Charanjiv Gadh
Chief Executive Officer
linkedin
Company data provided by crunchbase