200+ applicants

Company

FreshRealm · 23 hours ago

Data Engineer

United States

Full-time

Remote

Mid Level

$125K/yr - $139K/yr

3+ years exp

Maximize your interview chances

B2BB2C

Insider Connection @FreshRealm

Discover valuable connections within the company who might provide insights and potential referrals.
Get 3x more responses when you reach out via email instead of LinkedIn.

Responsibilities

Build data pipelines in Azure Databricks and Azure Data Factory to ingest, transform and load data into delta lake following the medallion architecture. This includes building data quality checks and alerts.

Perform analyst responsibilities and be able to collect requirements. Understand business processes and translate to data and documentation. Includes exploration and analysis of raw data to build data objects for business users.

Build data warehouse:

Model data into fact and dimensions using the Kimbal methodology

Organize data based on security criteria, such as CCPA

Implement security masking

Create documentation in both code and Confluence

Data Warehousing: Manage and enhance our data warehouse using Microsoft Synapse (formerly Azure SQL Data Warehouse) to provide scalable and high-performance data storage solutions.

Dashboard Development: Develop interactive and insightful dashboards and reports using Power BI to enable data visualization and self-service analytics.

Performance Tuning: Monitor and optimize database and ETL processes for maximum efficiency and performance.

Data Quality Assurance: Implement data quality checks, validation rules, and data profiling to maintain data accuracy and integrity.

Documentation: Create detailed documentation for data engineering processes, data models, and ETL workflows.

Collaboration: Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand their data requirements and deliver solutions accordingly.

Qualification

Find out how your skills align with this job's requirements. If anything seems off, you can easily click on the tags to select or unselect skills to reflect your actual expertise.

Data EngineeringMicrosoft BI StackAzure DatabricksPower BISpark SQLPysparkPythonAzure Data FactorySQLT-SQLData ModelingDatabase DesignAzure Data Lake StorageAzure Key VaultGitHub ActionsDatabricks REST APIAzure SecurityData Quality AssuranceKimbal Methodology

Required

Proven experience as a Data Engineer with a focus on Microsoft BI Stack, Azure Databricks, and Power BI (minimum 3 years of full-time experience using the tools).

Strong proficiency in Spark SQL, Pyspark, Python and Azure Data Factory.

In-depth knowledge of Azure Databricks, Azure Data Lake Storage, Azure Key Vault, GitHub Actions, Databricks REST API’s Azure Security.

Proficiency in creating interactive dashboards and reports using Power BI.

Strong SQL and T-SQL skills for data manipulation and querying.

Experience with data modeling and database design.

Familiarity with data quality assurance practices.

Excellent problem-solving skills and attention to detail.

Strong communication and collaboration skills.

Ability to work in a fast-paced, dynamic environment.

Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.

Experience with Kimbal data warehousing methodology.