118 applicants

Company

Xorbix Technologies, Inc. · 5 hours ago

Data Engineer

United States

Full-time

Remote

Mid Level

Maximize your interview chances

MobileSoftware

Hiring Manager

Asif Bakar

Insider Connection @Xorbix Technologies, Inc.

Discover valuable connections within the company who might provide insights and potential referrals.
Get 3x more responses when you reach out via email instead of LinkedIn.

Responsibilities

Develop and maintain documentation for data pipelines and platform best practices

Develop and optimize ETL processes to integrate structured and unstructured data into Databricks.

Design and deploy machine learning workflows in Databricks.

Implement Delta Lake for efficient data storage and real-time processing.

Monitor and troubleshoot Databricks jobs and clusters to ensure performance and reliability.

Implement PII data masking and encryption/tokenization solutions

Design reconciliation processes to validate data completeness and accuracy across ingestion pipelines

Qualification

Find out how your skills align with this job's requirements. If anything seems off, you can easily click on the tags to select or unselect skills to reflect your actual expertise.

DatabricksPythonSQLDelta LakeStructured StreamingDatabricks Auto LoaderDatabricks SQLMLflowData governanceAWSAzureSparkDevOps toolsGitCI/CDIceberg formatSnowflake integration

Required

Bachelor’s degree in computer science, data science, or a related field.

Proven experience with Databricks for data engineering or machine learning.

Strong programming skills in Python and SQL.

Familiarity with Delta Lake, Structured Streaming, and Databricks Auto Loader.

Strong problem-solving skills and attention to detail.

Preferred

Certifications in Databricks (e.g., Databricks Certified Data Engineer or Machine Learning Associate/Professional).

Experience with Databricks SQL and building interactive dashboards.

Familiarity with MLOps tools like MLflow for model lifecycle management.

Understanding of data governance and security within Databricks.

Familiarity working with major cloud computing providers (AWS/Azure)

Experience ingesting data from external sources using Spark

Experience with DevOps tools and the ability to use Git to manage source control changes

Familiarity with CI/CD environments and pipelines

Understanding of Iceberg format for cross-platform data compatibility and using delta/parquet tables outside of Databricks

Understanding of external tables and integration of Databricks with external platforms like Snowflake

Company

Xorbix Technologies, Inc.

Xorbix Technologies is a result focused global technology solutions company concentrating on custom software development, mobile application development, and strategic IT business consulting.

Founded in 1998