Guardian Life · 9 hours ago
Senior Data Engineer
New York
Full-time
Hybrid
Senior Level, Lead/Staff
$99K/yr - $163K/yr
15+ years expGuardian Life is seeking a Senior Data Engineer to join their team responsible for optimizing and maintaining data pipelines. The role involves collaborating with cross-functional teams to transform raw data into actionable insights and providing technical leadership in building scalable data solutions.
DentalFinancial ServicesHardwareInsuranceLife Insurance
No H1B
Responsibilities
Collaborate with cross-functional teams to understand data requirements and translate them into effective data pipelines
Provide technical leadership and mentorship to a team of Data Engineers. Oversee the design, development, and optimization of scalable and robust big data solutions and pipelines
Utilize extensive expertise in Databricks technologies with Python, PySpark, and SQL, and distributed data processing frameworks to design, develop, and implement highly performant and fault-tolerant ETL/ELT pipelines for large-scale data ingestion, processing, and transformation
Partner with multiple management teams to ensure appropriate integration of functions to meet goals. Identify and define necessary system enhancements to deploy new products and process improvements. Conduct in-depth analysis of complex business processes and system processes to recommend innovative solutions
Shape opportunities and create execution approaches throughout the lifecycle of client engagements
Ensure all deliverables are of high quality by setting development standards, adhering to the standards, and participating in code reviews
Assist in data modeling, schema design, and database optimization
Monitor and troubleshoot data pipelines to ensure data accuracy, reliability, and performance. Participate in code reviews, testing, and documentation efforts to maintain high-quality code and data documentation
Stay updated on industry trends and best practices in data engineering
Establish and maintain the quality, supportability, and performance of all processes
Continue to look for innovative and next generation solutions for solving data challenges
Establish real time data pipelines using AWS services like SQS / SNS and OpenSearch
Qualification
Required
15+ Years of IT experience, with 8+ years of experience as a Data Architect, data engineer or other data professional
Experience with Databricks and ETL tools
Knowledge of Mainframe and VSAM data structures
Knowledge and hands on with Jenkins pipelines and CI/CD
Expert Relational database concepts and experience across multiple database technologies (e.g., SQL, NoSQL, Oracle, Hadoop, Postgres) environments
Hands on experience in data engineering desired with extensive knowledge of SQL and PySpark, on Databricks Platform
Hands on experience designing, building, and optimizing data pipelines, architecture for large scale data warehousing projects
Experience with cloud data services such as Redshift, Relational database services (RDS), Snowflake and Databricks
Knowledge of AWS Data services is a big plus
Demonstrated expertise with data sourcing, profiling, data mapping and data analysis
Strong SQL, PL/SQL experience and skilled in creating and optimizing queries and data structures
Knowledge of ETL concepts, preferably Informatica or Syncsort
Experience with one or more SQL-on-Hadoop technology (HIVE, Presto, Spark SQL) and Hadoop ecosystem
Ability to design cloud and on Prem data solutions and provide guidance across storage, big data platform services, Hadoop ecosystem, vendor SaaS integrations, RDBMS and NoSQL databases is a plus
Excellent written and verbal communication skills and creative problem-solving skills
Ability to adapt to changing business priorities and experience with effectively communicating change across different levels in the organization
Communication Skills: Articulate the vision and goals, both to the team members and other stakeholders
Emotional Intelligence & Empathy: Motivate, inspire, and resolve conflicts with others while building a positive rapport with team members and partners
Collaboration: Collaborate effectively with the Product Manager and your leadership and keep appraised with key updates and planning. Encourage partnership between team members to create a more dynamic and effective work environment
Strong listeners and able to provide constructive feedback
Understanding of Insurance domain
Significant experience interfacing with both customers and management across business and IT
Benefits
Support and flexibility to achieve your professional and personal goals
Skill-building, leadership development and philanthropic opportunities
Contemporary, supportive, flexible, and inclusive benefits and resources
Company
Guardian Life
Guardian Life Insurance Company of America (Guardian) is one of the country’s leading mutual insurers.
5001-10000 employees
Funding
Current Stage
Late StageTotal Funding
unknown2016-08-23Non Equity Assistance
Leadership Team
Andrew McMahon
Chairman and CEO
Karunakaran Azhisur
Country Head & CIO, Guardian India
Recent News
2025-12-11
Company data provided by crunchbase