James Search Group · 7 hours ago
Senior Data Engineer
United States
Full-time
Remote
Mid, Senior Level
$120K/yr - $170K/yr
3+ years expJames Search Group is partnering with a highly rated Property & Casualty insurance carrier that is making significant investments in data engineering, analytics, and machine learning. We are seeking a Senior Data Engineer to design, build, and maintain the upstream components of a modern data platform, playing a pivotal role in shaping the technical foundation of the company’s next-generation data environment.
Staffing & Recruiting
Hiring Manager
Simon James
Responsibilities
Design, build, and optimize scalable data pipelines for batch and real-time processing
Implement data ingestion frameworks including CDC from core systems, APIs, and third-party platforms (Salesforce, Workday, Duck Creek, etc.)
Develop and optimize Apache Spark jobs on Databricks, leveraging Delta Lake, DLT pipelines, and lakehouse architectures
Ensure data quality, lineage, and governance using Unity Catalog, CI/CD, and role-based access/security controls
Partner with analytics engineers (dbt) to deliver clean, structured upstream data
Mentor peers, contribute to architecture decisions, and foster a culture of craftsmanship
Leverage AWS services (S3, Glue, Lambda, etc.) and DevOps tools (GitHub, CI/CD) for scalable, production-grade deployments
Qualification
Required
3-5+ years of professional experience in data engineering, ideally within insurance or financial services
Strong proficiency in Python, SQL, and Spark for building and optimizing pipelines
Hands-on expertise with Databricks (Unity Catalog, Delta Lake, DLT pipelines) and/or Azure Data Services
Strong knowledge of AWS data services, with the ability to adapt across cloud platforms (Azure, GCP)
Familiarity with modern data architectures (medallion, lakehouse, streaming)
Experience with GitHub, CI/CD pipelines, and testing frameworks
A problem-solving mindset: balancing pragmatism with scalability, and a passion for working in collaborative teams
Preferred
This is a remote-first position, but you can also work from one of the carrier's multiple U.S. office locations
Benefits
Performance-based bonus
Comprehensive benefits package
401(k) with company match
Generous PTO and wellness programs
Company
James Search Group
James Search Group (JSG) is a Recruitment Consultancy founded on 20 years of recruiting experience, specialising in Legal Services, Financial Services and Technology recruiting.
2-10 employees
Funding
Current Stage
Early StageCompany data provided by crunchbase