Intelliswift - An LTTS Company · 7 hours ago
Senior Databricks Data Architect (AWS) - Hands-On Technical Leader
New Jersey, United States
Contract
Onsite
Senior Level, Lead/Staff
8+ years expIntelliswift, an LTTS Company, is seeking a Senior Databricks Data Architect to lead the design and architecture of their Enterprise Data Platform on Databricks/AWS. The role involves providing technical consultancy, governing platform quality, and mentoring engineering teams while being hands-on in coding and performance tuning.
Big DataBusiness IntelligenceCloud ManagementEnterprise SoftwareInformation Technology
Hiring Manager
Basant Sharma
Responsibilities
Lead the strategic design and architectural roadmap of the Enterprise Data Platform on Databricks/AWS while remaining actively hands-on in design, prototyping, coding, and performance tuning
To provide high-level technical consultancy and solution design to internal business stakeholders and external partners, acting as the bridge between complex business requirements and technical implementation
To govern the technical quality of the platform by establishing design standards, conducting code reviews, and guiding the engineering team on best practices for Databricks Serverless and AWS integration
As part of the role, you will be required to provide technical assurance for all major data initiatives, ensuring that solutions are robust, cost-effective, and compliant with enterprise security standards
Lead all architecture design and review sessions for strategic Databricks initiatives
Actively contribute code to critical architectural components, building complex frameworks, reusable libraries, and core ETL/ELT pipelines using Python and PySpark
Lead by example by prototyping Proof of Concepts (PoCs) for new features (e.g., Databricks Asset Bundles, Delta Live Tables) before handing them off for wider team implementation
Perform hands-on performance tuning and debugging of high-latency jobs by deep-diving into Spark UI, execution plans, and spill logs
Defining and enforcing coding standards, data modeling best practices (Medallion Architecture), and CI/CD protocols for the development team to follow
Conducting deep-dive technical reviews of designs and code produced by the engineering team to ensure efficiency, maintainability, and security
Designing the strategy for Unity Catalog governance, ensuring granular data access controls and compliance with data privacy regulations
Acting as the Technical Design Authority for the Databricks platform, making high-level decisions regarding the adoption of new Serverless features and AWS services
Mentoring and coaching Senior and Junior engineers, fostering a culture of technical excellence and continuous improvement
Own the technical relationship with key business stakeholders and translate business strategy into robust data architecture
Engage and collaborate with Executive Stakeholders to present technical roadmaps and demonstrate the business value of data initiatives
Actively seeks opportunities to develop knowledge in emerging technologies (e.g., Generative AI on Databricks, Vector Databases) and evaluate their impact on the business
Provide guidance to the Change Management board regarding the risk and technical impact of proposed architectural changes
Develop strategies to architect resilience, high availability, and disaster recovery into the data platform
Develop clear communication channels with C-level executives and Product Managers to ensure technical strategies align with business goals
Qualification
Required
Educated to degree level in Computer Science, Engineering, or equivalent (Master's degree preferred)
Exceptional 'Customer Facing' skills: proven ability to lead workshops, manage stakeholder expectations, and present complex technical solutions to non-technical audiences
Minimum 8-10 years' experience in Data Engineering/Architecture, with at least 4 years specialized in Databricks and AWS
Expert-level knowledge of Data Lakehouse Architecture, Delta Lake internals, and Spark optimization strategies
Proven experience in Data Modeling (Dimensional, Vault, Relational) and designing complex ETL/ELT pipelines
Advanced hands-on coding proficiency in Python and SQL, with the ability to write clean, production-ready, and testable code
Strong experience with Databricks Serverless architecture, Unity Catalog implementation, and security patterns
Deep understanding of Cloud Architecture (AWS Well-Architected Framework), including networking, storage (S3), and IAM
Experience in Design Review and Code Review processes, capable of constructive critique and enforcing standards
Detailed knowledge of DevOps principles (IaC with Terraform, CI/CD pipelines)
Knowledge of the MS Office suite (Word, Excel, PowerPoint, Outlook) – specifically for creating architectural diagrams and presentations
Very strong relationship building and influencing skills at a senior level
Strategic thinking with the ability to balance immediate business needs with long-term architectural vision
8+ years of experience in Data Architecture and Engineering
Extensive experience designing solutions on the Databricks platform
Preferred
Strong experience working with large-scale enterprises or as a Consultant for big agencies
Certified Databricks Data Architect or AWS Solutions Architect (Professional)
Excellent organizational and planning skills, with an ability to manage architectural backlogs
A best-in-class technical leader proactively driving business innovation and process improvements
Languages: Python, SQL, Scala
Experience designing APIs for data consumption
Experience with Enterprise Architecture frameworks (TOGAF)
Company
Intelliswift - An LTTS Company
"Intelliswift, an LTTS Company, delivers world-class Digital Product Engineering, Data Management, Analytics & AI, and Digital Ent Solutions
1001-5000 employees
H1B Sponsorship
Intelliswift - An LTTS Company has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (29)
2024 (35)
2023 (44)
2022 (27)
2021 (38)
2020 (37)
Funding
Current Stage
Late StageTotal Funding
unknown2024-11-11Acquired
Leadership Team
Pat Patel
Founder & Executive Chairman
Recent News
Business Standard India
2024-11-12
2024-04-08
Company data provided by crunchbase