84.51˚ · 7 hours ago
Senior Data Engineer
84.51° is a retail data science, insights and media company. As a Senior Data Engineer, you will have the opportunity to build solutions that ingest, transform, store, and distribute big data to be consumed by data scientists and products.
Responsibilities
Take ownership of features and drive them to completion through all phases of the entire 84.51° SDLC. This includes internal and external facing applications as well as process improvement activities:
Design, build, and maintain scalable, efficient, and reliable data pipelines to support data ingestion, transformation, and integration across diverse sources and destinations, using tools such as Kafka, Databricks, and similar toolsets
Implement automated unit and integration testing
Collaborate with architecture and lead engineers to ensure consistent development practices
Provide mentoring to junior engineers
Participate in retrospective reviews
Participate in the estimation process for new work and releases
Collaborate with other engineers to solve and bring new perspectives to complex problems
Drive improvements in data engineering practices, procedures, and ways of working
Embrace new technologies and an ever-changing environment
Qualification
Required
3+ years proven ability of professional Data Development experience
2+ years proven ability of developing with Databricks or Hadoop/HDFS
2+ years of experience with PySpark/Spark
3+ years of experience developing with either Python or Java
3+ years of experience with SQL and NoSQL technologies
Full understanding of ETL concepts and Data Warehousing concepts
Experience with Pipeline Orchestration
Experience with CI/CD
Experience with version control software
Strong understanding of Agile Principles (Scrum)
Bachelor's Degree (Computer Science, Management Information Systems, Mathematics, Business Analytics, or STEM)
Preferred
Experience with Azure
Experience with Databricks Delta Tables, Delta Lake, Delta Live Tables
Proficient with Relational Data Modeling
Experience with CI/CD pipelines
Experience with Python Library Development
Experience with Structured Streaming (Spark or otherwise)
Experience with Kafka and/or Azure Event Hub
Experience with Github SaaS / Github Actions
Experience with Snowflake
Exposure to BI Tooling (Tableau, Power BI, Cognos, etc.)
Benefits
Health: Medical: with competitive plan designs and support for self-care, wellness and mental health. Dental: with in-network and out-of-network benefit. Vision: with in-network and out-of-network benefit.
Wealth: 401(k) with Roth option and matching contribution. Health Savings Account with matching contribution (requires participation in qualifying medical plan). AD&D and supplemental insurance options to help ensure additional protection for you.
Happiness: Paid time off with flexibility to meet your life needs, including 5 weeks of vacation time, 7 health and wellness days, 3 floating holidays, as well as 6 company-paid holidays per year. Paid leave for maternity, paternity and family care instances.