Jobs via Dice · 22 hours ago
Lead Cloudera Consultant (Solution Architect)
United States
Full-time
Remote
Lead/StaffDice is the leading career destination for tech experts at every stage of their careers. Our client, Camelot Integrated Solutions Inc, is seeking a Lead Cloudera Consultant (Solution Architect) who will be responsible for designing, delivering, and optimizing mission-critical real-time data pipelines at enterprise scale.
Computer Software
Responsibilities
Architect and build real-time data pipelines using the full Cloudera Data Platform (CDP) streaming suite: NiFi → Kafka → Flink → Kudu/Impala → SSB
Own architectural decisions, patterns, and best practices for streaming, CDC, state management, schema evolution, and exactly-once delivery
Develop complex NiFi flows involving controller services (DBCP/JDBC), stateful processors, record processors, schema registry integrations, batch-to-stream conversions, and high-volume ingestion patterns
Build and optimize Flink SQL or DataStream API jobs with: ? Kafka sources/sinks ? event-time windows ? watermarks ? state management ? checkpointing / savepoints ? exactly-once guarantees
Design and tune Kudu tables (PKs, partitioning, distribution, upserts, deletes, merges)
Build and deploy streaming SQL jobs using Cloudera SQL Stream Builder (SSB)
Deliver the following four core use cases immediately: NiFi → Snowflake → Impala/Kudu ingestion pipeline, Kafka → Flink streaming (real-time processing), Flink → Kafka sink with exactly-once semantics, CDC ingestion via NiFi, Flink CDC, or SSB (incremental keys, late events, deletes)
Tune NiFi, Kafka, and Flink clusters for performance, throughput, and stability
Implement schema governance, error handling, back-pressure strategies, and replay mechanisms
Work closely with platform engineers to optimize CDP components and CDF deployments
Provide architectural guidance, documentation, and mentorship to engineering teams
Qualification
Required
Deep, hands-on experience across the Cloudera CDP streaming stack, including NiFi, Kafka, Flink, Kudu/Impala, and SQL Stream Builder (SSB)
Experience building end-to-end CDP/CDF streaming pipelines
Ability to execute complex ingestion, transformation, CDC, and Kudu write-path use cases on day one
Architect and build real-time data pipelines using the full Cloudera Data Platform (CDP) streaming suite
Own architectural decisions, patterns, and best practices for streaming, CDC, state management, schema evolution, and exactly-once delivery
Develop complex NiFi flows involving controller services (DBCP/JDBC), stateful processors, record processors, schema registry integrations, batch-to-stream conversions, and high-volume ingestion patterns
Build and optimize Flink SQL or DataStream API jobs with Kafka sources/sinks, event-time windows, watermarks, state management, checkpointing/savepoints, and exactly-once guarantees
Design and tune Kudu tables (PKs, partitioning, distribution, upserts, deletes, merges)
Build and deploy streaming SQL jobs using Cloudera SQL Stream Builder (SSB)
Deliver core use cases including NiFi → Snowflake → Impala/Kudu ingestion pipeline, Kafka → Flink streaming (real-time processing), Flink → Kafka sink with exactly-once semantics, and CDC ingestion via NiFi, Flink CDC, or SSB
Tune NiFi, Kafka, and Flink clusters for performance, throughput, and stability
Implement schema governance, error handling, back-pressure strategies, and replay mechanisms
Work closely with platform engineers to optimize CDP components and CDF deployments
Provide architectural guidance, documentation, and mentorship to engineering teams
Company
Jobs via Dice
Welcome to Jobs via Dice, the go-to destination for discovering the tech jobs you want.
0-1 employees
Funding
Current Stage
Early StageCompany data provided by crunchbase