Ventures Unlimited Inc · 1 day ago
Java Nifi developer
United States
Full-time
Remote
Mid LevelVentures Unlimited Inc is seeking a Java Nifi Engineer to create and implement data flow solutions using NiFi. The role involves managing data ingestion, ensuring scalability and performance, and integrating microservices for data processing.
ConsultingInformation TechnologyStaffing Agency
Hiring Manager
Prabhakar Choudhary
Responsibilities
Creating and implementing data flow solutions using NiFi, including defining data sources, destinations, and transformation processes
Managing data ingestion from various sources (e.g., files, databases, APIs) and transforming data into desired formats for downstream consumption
Configuring NiFi to route data to appropriate destinations and enriching data with additional information (e.g., timestamps, location)
Ensuring data flows are scalable and performant to handle varying data volumes and velocity
Maintaining and improving NiFi clusters, including configuration, optimization, and troubleshooting
Developing and integrating microservices to support data conditioning, format validation, and transformation processes
Identifying and resolving issues related to data flows, NiFi configurations, and data integration processes
Possessing deep knowledge of Airflow's architecture, including schedulers, executors (Celery, Kubernetes), and plugin development
Designing and developing complex, modular, and reusable DAGs (Directed Acyclic Graphs) to automate data pipelines
Identifying and addressing performance bottlenecks in Airflow environments and implementing best practices for orchestration and scheduling
Qualification
Required
Strong NiFi Expertise: Deep understanding of NiFi architecture, processors, controllers, and data flow management capabilities
Programming Languages: Proficiency in Java, Spring boot & API Microservices
Database and SQL: Familiarity with databases, SQL, and data manipulation techniques
Big Data Technologies: Knowledge of big data technologies like Kafka (Event driven & streaming architecture)
Airflow – Workflow orchestration
Creating and implementing data flow solutions using NiFi, including defining data sources, destinations, and transformation processes
Managing data ingestion from various sources (e.g., files, databases, APIs) and transforming data into desired formats for downstream consumption
Configuring NiFi to route data to appropriate destinations and enriching data with additional information (e.g., timestamps, location)
Ensuring data flows are scalable and performant to handle varying data volumes and velocity
Maintaining and improving NiFi clusters, including configuration, optimization, and troubleshooting
Developing and integrating microservices to support data conditioning, format validation, and transformation processes
Identifying and resolving issues related to data flows, NiFi configurations, and data integration processes
Possessing deep knowledge of Airflow's architecture, including schedulers, executors (Celery, Kubernetes), and plugin development
Designing and developing complex, modular, and reusable DAGs (Directed Acyclic Graphs) to automate data pipelines
Identifying and addressing performance bottlenecks in Airflow environments and implementing best practices for orchestration and scheduling
Company
Ventures Unlimited Inc
We are a cutting-edge consulting firm specializing in Technology Solutions.
Founded in 2004
51-200 employees
Funding
Current Stage
Growth StageCompany data provided by crunchbase