3Core Systems, Inc ยท 2 days ago
Senior Data Engineer
Plantation, Florida, United States
Full-time
Hybrid
Senior Level
5+ years exp3Core Systems, Inc is a technology company seeking a Senior Data Engineer to enhance their data platform infrastructure. The role involves managing Databricks environments, developing data ingestion patterns, and collaborating with analytics teams to deliver high-quality data solutions.
Information ServicesInformation TechnologySoftware
Responsibilities
Administer, optimize, and scale our Databricks Lakehouse environment, ensuring high performance, cost efficiency, and operational excellence
Develop, maintain, and enhance our data platform infrastructure and security configurations using Terraform. This includes provisioning Databricks workspaces, SQL Endpoints, Unity Catalog objects, and network components
Manage and enforce Unity Catalog for data governance, access control, and metadata management
Implement and manage CI/CD pipelines for data pipelines, dbt projects, and infrastructure deployments using GitHub Actions
Automate operational tasks, monitoring, and alerting for the data platform
Implement and enforce DevSecOps principles, working closely with security teams to ensure compliance and manage/rotate credentials securely
Design and implement data ingestion patterns into Databricks using Delta Lake, optimizing for large-scale data processing and storage
Develop, optimize, and troubleshoot complex Spark jobs (PySpark/Scala) for data processing and transformation within Databricks
Manage and extend data ingestion pipelines using Airbyte (or similar modern tools like Fivetran, Stitch), including configuring connectors, monitoring syncs, and ensuring data quality and reliability from diverse source systems (e.g., ERP, CRM, marketing, supply chain)
Orchestrate and automate data pipelines and dbt models using Databricks Workflows and potentially integrating with other orchestration tools
Collaborate with Analytics Engineers to translate business requirements into efficient and scalable data models using dbt (Data Build Tool)
Implement dbt best practices for modularity, testing, documentation, and version control, ensuring seamless integration with Databricks
Partner effectively with Analytics Engineers, Data Scientists, and business stakeholders to deliver high-quality data solutions
Provide technical guidance and mentorship to junior team members, and champion data engineering best practices, code quality, and documentation standards
Qualification
Required
Bachelor's degree in Computer Science, Data Engineering, or a related technical field required
5+ years of progressive experience as a Data Engineer, with a strong focus on cloud-based data platforms
Extensive experience with Spark (PySpark/Scala), Delta Lake, Unity Catalog, Databricks SQL, and platform administration
Proven experience with dbt for data modeling, transformation, and testing
Strong proficiency with Terraform for defining, provisioning, and managing cloud infrastructure and Databricks resources as code
Expertise in Git and GitHub Actions for version control and implementing robust CI/CD pipelines
Proficiency in SQL and at least one programming language (Python strongly preferred, Scala is a plus)
Solid understanding of data warehousing, data lake, and lakehouse architectures
Company
3Core Systems, Inc
Established in 2004, 3Core System is a certified small minority owned business providing ERP Systems Integration, AMS, IT Consulting and Staff Augmentation Services to Fortune 1000, SMB, and State, Local and Education (SLED) customers.
51-200 employees
H1B Sponsorship
3Core Systems, Inc has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (3)
2024 (1)
2023 (12)
2022 (9)
2021 (14)
2020 (25)
Funding
Current Stage
Growth StageLeadership Team
Navin Kandula
SAP HCM/SuccessFactors Solution Architect
Company data provided by crunchbase