Genesis10 · 4 days ago
Python Engineer, PySpark/Spark Performance Tuning, Azure DevOps/Jenkins, Hive/Impala; NJ
Jersey City, NJ
Full-time
Hybrid
Senior Level
5+ years expGenesis10 is working with Bank of America to find a Python Engineer. The role involves designing and implementing data ingestion and transformation jobs using Python and PySpark, while collaborating closely with architects and Data-Ops teams to ensure robust solutions.
Information ServicesInformation Technology
Responsibilities
The Python Engineer designs and implements data ingestion and transformation jobs using Python and PySpark
They build reusable frameworks for data quality checks and integrate pipelines with CI/CD processes
The role requires performance tuning, secure coding practices, and strong debugging skills. Close collaboration with architects and Data Ops teams ensures robust and compliant solutions
Qualification
Required
The Python Engineer designs and implements data ingestion and transformation jobs using Python and PySpark
They build reusable frameworks for data quality checks and integrate pipelines with CI/CD processes
The role requires performance tuning, secure coding practices, and strong debugging skills
Close collaboration with architects and Data Ops teams ensures robust and compliant solutions
Min 5 yrs of experience
Preferred
Strong Python (packaging, virtual environments), PySpark/Spark performance tuning
Experience with data ingestion (batch/stream), schema management, and error handling/retry logic
Test discipline: unit/integration tests, data quality assertions, reproducible pipelines
CI/CD (Azure DevOps/Jenkins), Git workflows, artifact versioning, release readiness
Experience on Cloudera/Hadoop (HDFS, Spark, Hive/Impala) and Databricks (clusters, jobs, notebooks, Delta)
Observability: structured logging, metrics, tracing; debuggability in distributed contexts
Secure coding and handling of secrets, PII protection, and compliance considerations
Strong communication and collaborative work style; documentation of frameworks and usage patterns
Performance optimization: partitioning, caching, broadcast joins, memory tuning
Comfort with Agile ceremonies and iterative delivery
Company
Genesis10
Information Technology and Services
1001-5000 employees
H1B Sponsorship
Genesis10 has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (126)
2024 (68)
2023 (20)
2022 (2)
2021 (13)
2020 (29)
Funding
Current Stage
Late StageLeadership Team
Harley Lippman
CEO and Founder
Company data provided by crunchbase