Yochana · 12 hours ago
Cybersecurity Data Pipeline Engineer
United States
Contract
Remote
Senior Level, Lead/Staff
10+ years expYochana is seeking a Cybersecurity Data Pipeline Engineer to lead the architecture, design, and implementation of scalable data flow pipelines. The role involves developing ingestion frameworks, ensuring data integrity, and collaborating with teams to support analytics and security requirements.
Business Information SystemsInformation and Communications Technology (ICT)Information ServicesInformation TechnologySoftwareStaffing Agency
Hiring Manager
Rushinga Reddy Y
Responsibilities
Lead the architecture, design, and implementation of scalable, modular, and reusable data flow pipelines using Cribl, Apache NiFi, Vector, and other open-source platforms, ensuring consistent ingestion strategies across a complex, multi-source telemetry environment
Develop platform-agnostic ingestion frameworks and template-driven architectures to enable reusable ingestion patterns, supporting a variety of input types (e.g., syslog, Kafka, HTTP, Event Hubs, Blob Storage) and output destinations (e.g., Snowflake, Splunk, ADX, Log Analytics, Anvilogic)
Spearhead the creation and adoption of a schema normalization strategy, leveraging the Open Cybersecurity Schema Framework (OCSF), including field mapping, transformation templates, and schema validation logic—designed to be portable across ingestion platforms
Design and implement custom data transformations and enrichments using scripting languages such as Groovy, Python, or JavaScript, while enforcing robust governance and security controls (SSL/TLS, client authentication, input validation, logging)
Ensure full end-to-end traceability and lineage of data across the ingestion, transformation, and storage lifecycle, including metadata tagging, correlation IDs, and change tracking for forensic and audit readiness
Collaborate with observability and platform teams to integrate pipeline-level health monitoring, transformation failure logging, and anomaly detection mechanisms
Oversee and validate data integration efforts, ensuring high-fidelity delivery into downstream analytics platforms and data stores, with minimal data loss, duplication, or transformation drift
Lead technical working sessions to evaluate and recommend best-fit technologies, tools, and practices for managing structured and unstructured security telemetry data at scale
Implement data transformation logic including filtering, enrichment, dynamic routing, and format conversions (e.g., JSON ↔ CSV, XML, Logfmt) to prepare data for downstream analytics platforms. (100 plus sources of data)
Contribute to and maintain a centralized documentation repository, including ingestion patterns, transformation libraries, naming standards, schema definitions, data governance procedures, and platform-specific integration details
Coordinate with security, analytics, and platform teams to understand use cases and ensure pipeline logic supports threat detection, compliance, and data analytics requirements
Qualification
Required
10+ Years of experience working in Cybersecurity
5+ Years of experience on CRIBL/Vector/Datadog/Splunk or other data pipeline platforms
5+ Years of experience on JavaScript, python, or other scripting language
Lead the architecture, design, and implementation of scalable, modular, and reusable data flow pipelines using Cribl, Apache NiFi, Vector, and other open-source platforms, ensuring consistent ingestion strategies across a complex, multi-source telemetry environment
Develop platform-agnostic ingestion frameworks and template-driven architectures to enable reusable ingestion patterns, supporting a variety of input types (e.g., syslog, Kafka, HTTP, Event Hubs, Blob Storage) and output destinations (e.g., Snowflake, Splunk, ADX, Log Analytics, Anvilogic)
Spearhead the creation and adoption of a schema normalization strategy, leveraging the Open Cybersecurity Schema Framework (OCSF), including field mapping, transformation templates, and schema validation logic—designed to be portable across ingestion platforms
Design and implement custom data transformations and enrichments using scripting languages such as Groovy, Python, or JavaScript, while enforcing robust governance and security controls (SSL/TLS, client authentication, input validation, logging)
Ensure full end-to-end traceability and lineage of data across the ingestion, transformation, and storage lifecycle, including metadata tagging, correlation IDs, and change tracking for forensic and audit readiness
Collaborate with observability and platform teams to integrate pipeline-level health monitoring, transformation failure logging, and anomaly detection mechanisms
Oversee and validate data integration efforts, ensuring high-fidelity delivery into downstream analytics platforms and data stores, with minimal data loss, duplication, or transformation drift
Lead technical working sessions to evaluate and recommend best-fit technologies, tools, and practices for managing structured and unstructured security telemetry data at scale
Implement data transformation logic including filtering, enrichment, dynamic routing, and format conversions (e.g., JSON ↔ CSV, XML, Logfmt) to prepare data for downstream analytics platforms. (100 plus sources of data)
Contribute to and maintain a centralized documentation repository, including ingestion patterns, transformation libraries, naming standards, schema definitions, data governance procedures, and platform-specific integration details
Coordinate with security, analytics, and platform teams to understand use cases and ensure pipeline logic supports threat detection, compliance, and data analytics requirements
Company
Yochana
Yochana: Your Trusted Workforce Partner Across North America & Beyond For over 16 years, Yochana has been a leading talent acquisition firm, connecting businesses with top professionals across industries.
201-500 employees
H1B Sponsorship
Yochana has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (21)
2024 (11)
2023 (3)
Funding
Current Stage
Growth StageLeadership Team
Anusha Sharath
Global Talent Acquisition Lead / HR-BP
Recent News
Company data provided by crunchbase