Bitcruit Solutions ยท 18 hours ago
Data Engineer
United States
Full-time
Remote
Mid Level
$80K/yr - $128K/yr
4+ years expBitcruit Solutions is a data consulting firm seeking a Data Engineer with hands-on experience in Databricks and modern data engineering practices. The role involves designing, developing, and maintaining data pipelines, ensuring data quality, and enabling advanced analytics across the enterprise.
Staffing & Recruiting
Hiring Manager
Rishav Srivastava
Responsibilities
Design, develop, and maintain ETL/ELT pipelines and Databricks notebooks for data ingestion, transformation, and reporting
Implement and optimize Spark-based workflows using Databricks Workflows, Delta Lake, and Unity Catalog
Leverage Databricks partner tools (e.g., Fivetran, dbt, Coalesce) for scalable data ingestion, transformation, and orchestration
Conduct data reconciliation across source, staging, and target systems to ensure accuracy and completeness
Build and maintain SQL- and Python-based automation frameworks for regression, smoke, and integration testing
Validate structured and semi-structured data formats including CSV, JSON, XML, Parquet, Avro, and fixed-length files
Review and refactor existing code for performance, scalability, and maintainability
Support workflow orchestration using tools such as Airflow, Azure Data Factory, or Control-M
Collaborate with data governance and cataloging teams leveraging tools such as Alation or Unity Catalog to improve discoverability and lineage tracking
Qualification
Required
Experience with cloud platforms (Azure, AWS, or Google Cloud Platform)
2+ years of experience with Databricks and Unity Catalog
4+ years of Python experience
Strong proficiency in SQL (joins, CTEs, window functions, query optimization, stored procedures)
Solid experience with PySpark, pandas, and Python scripting for data manipulation and validation
Familiarity with medallion architecture and data warehouse concepts, including star/snowflake schemas, fact/dimension modeling, and reporting logic
Experience validating flat files (CSV, TSV, TXT) including delimiter handling, header checks, null value handling, and schema enforcement
Proven ability to interpret and implement complex transformation logic from source to target systems
Strong problem-solving, analytical, and debugging capabilities
Excellent written and verbal communication skills for interfacing with developers, clients, and technical stakeholders
Ability to work independently while collaborating effectively in a team-oriented environment
Preferred
Exposure to Databricks ecosystem tools such as Fivetran, dbt, Coalesce, and Alation
Experience building CI/CD pipelines and DevOps practices for data engineering
Knowledge of data governance, metadata management, and security frameworks (e.g., GDPR, SOC2, Unity Catalog RBAC)
Familiarity with data quality frameworks (e.g., Great Expectations, Deequ)
Benefits
Comprehensive health coverage including medical, dental, and vision plans
Life and disability insurance coverage
401(k) retirement plan
Additional insurance benefits to support long-term financial and personal security
Company
Bitcruit Solutions
Bitcruit Solutions is a global Recruitment Process Outsourcing (RPO) and staffing agency dedicated to helping businesses scale smarter.
2-10 employees
Funding
Current Stage
Early StageCompany data provided by crunchbase