JPMorganChase · 9 hours ago
Databricks Optimization - Lead Software Engineer
Jersey City, NJ
Full-time
Onsite
Senior Level, Lead/Staff
$152K/yr - $215K/yr
5+ years expJPMorganChase, one of the oldest financial institutions, offers innovative financial solutions to millions of consumers and businesses. As a Lead Software Engineer in the Corporate Technology team, you will drive efficiency and cost optimization across the Databricks platform, ensuring high-performing and cost-efficient workloads while aligning with enterprise governance standards.
Asset ManagementBankingFinancial Services
Responsibilities
Analyze DBU consumption trends to identify inefficiencies across clusters, SQL warehouses, and jobs
Develop cluster & SQL warehouse rightsizing strategies using real workload data
Implement and govern cluster policies to enforce optimal worker types, runtimes, and configuration standards
Build automated monitors for idle clusters, runaway jobs, underutilized warehouses, and inefficient workloads
Optimize cloud foundations (EBS/S3 performance)
Identify areas of cost leakage and provide recommendation on right compute strategy to resolve it
Continuously work with Databricks product teams to enable vendor provided tools and techniques for cost monitoring and optimization
Qualification
Required
Formal training or certification on software engineering concepts and 5+ years applied experience
Databricks Certified Data Engineer Professional
Act as the primary SME for Databricks performance and cost optimization
Partner with application teams to identify inefficiencies and rewrite pipelines
Create playbooks, runbooks, and enterprise standards for Databricks engineering
Tune PySpark/Scala workloads to minimize shuffle, improve parallelism, and reduce job runtimes
Implement best practices for caching, broadcast joins, and data layout
Modernize legacy jobs with Workflows and Delta Live Tables for reliability and automation
Standardize job orchestration patterns and eliminate inefficient pipeline designs
Provide application teams with guidance on writing performant Spark code
Build reports using System Tables, logging, tags, and FinOps metrics
Preferred
Strong experience with cost attribution, tagging, budgeting, and rightsizing
Experience building dashboards for cost and performance reporting
Excellent communicator with ability to challenge teams diplomatically
Strong documentation & coaching skills
Can drive enterprise-wide change and enforce consistent standards
Experience in large-scale regulated enterprise environments
Benefits
Comprehensive health care coverage
On-site health and wellness centers
A retirement savings plan
Backup childcare
Tuition reimbursement
Mental health support
Financial coaching
Company
JPMorganChase
With a history tracing its roots to 1799 in New York City, JPMorganChase is one of the world's oldest, largest, and best-known financial institutions—carrying forth the innovative spirit of our heritage firms in global operations across 100 markets.
10001+ employees
H1B Sponsorship
JPMorganChase has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (3471)
2024 (3469)
2023 (3395)
2022 (3594)
2021 (2515)
2020 (2495)
Funding
Current Stage
Public CompanyTotal Funding
unknown1998-02-01IPO
Leadership Team
Allison Beer
CEO of Card Services and Connected Commerce
Dan Mendelson
CEO, Morgan Health
Recent News
2025-12-19
2025-12-12
Company data provided by crunchbase