Gandiv Insights LLC · 12 hours ago
Senior Data Platform Architect (AWS)
United States
Contract
Remote
Mid, Senior Level
3+ years expGandiv Insights LLC is seeking a Senior Data Platform Architect (AWS) to lead the design and implementation of a scalable and secure Data Lakehouse on AWS. The role involves architecting storage and compute solutions, ensuring data security, and collaborating with various stakeholders to meet their data product needs.
Information Technology & Services
No H1B
Responsibilities
Architect and Design the Data Lakehouse: Lead the design and implementation of a scalable and secure Data Lakehouse on AWS, including data storage and compute layers
Storage Solutions: Design and implement storage solutions using AWS services like S3, Iceberg
Integrate relevant metadata from platform with data catalog and//or metadata management solutions
Compute Resources: Architect and optimize compute resources using AWS services like Glue, EMR, and Lambda for ETL processes, and possibly Redshift or Athena for query execution
Develop POCs, POVs and pilots to test architecture, capabilities etc. and collaborate with collaborate with data engineers to ensure seamless integration and ingestion of data from various sources into the Lakehouse
Security and Compliance: Implement best practices for data security, including encryption, IAM roles, and compliance with relevant data protection regulations
Performance Optimization: Continuously monitor and optimize the performance of the data lakehouse, including storage costs and compute efficiency
Collaboration: Work closely with data engineers, data scientists, and business stakeholders to ensure the platform meets their needs for data products
Documentation and Training: Provide thorough documentation and training to the internal team on the architecture and use of the Data Lakehouse
Qualification
Required
Architect and Design the Data Lakehouse: Lead the design and implementation of a scalable and secure Data Lakehouse on AWS, including data storage and compute layers
Storage Solutions: Design and implement storage solutions using AWS services like S3, Iceberg
Integrate relevant metadata from platform with data catalog and//or metadata management solutions
Compute Resources: Architect and optimize compute resources using AWS services like Glue, EMR, and Lambda for ETL processes, and possibly Redshift or Athena for query execution
Develop POCs, POVs and pilots to test architecture, capabilities etc. and collaborate with collaborate with data engineers to ensure seamless integration and ingestion of data from various sources into the Lakehouse
Security and Compliance: Implement best practices for data security, including encryption, IAM roles, and compliance with relevant data protection regulations
Performance Optimization: Continuously monitor and optimize the performance of the data lakehouse, including storage costs and compute efficiency
Collaboration: Work closely with data engineers, data scientists, and business stakeholders to ensure the platform meets their needs for data products
Documentation and Training: Provide thorough documentation and training to the internal team on the architecture and use of the Data Lakehouse
Preferred
Interprets system usage data to predict and plan for future scalability challenges
AWS Certification: Certified Solutions Architect or other specialized AWS certifications (e.g., Security, Data Analytics, DevOps)
Containerization with Kubernetes: Experience deploying and managing applications using containers orchestrated with Kubernetes
Monitoring: Experience with monitoring tools like Splunk, Prometheus, and Grafana to ensure system health and performance
CI//CD: Hands-on experience building, creating automation scripts and maintaining (CI//CD) pipelines to automate software delivery
Advanced Operations: Experience with complex deployments, command-line tools, performance tuning, and modern data catalogs
Strong understanding of SSO protocols (SAML, OAuth 2.0, OpenID Connect) and directory services like Active Directory
3+ years of IT experience with exposure to BI tools like Alteryx or Tableau
Knowledge of AWS security best practices and compliance
Company
Gandiv Insights LLC
Gandiv Insights is a leading professional services and staffing firm specializing in top-tier personnel and technical solutions across engineering, IT, non-IT, and healthcare sectors.
201-500 employees
Funding
Current Stage
Growth StageCompany data provided by crunchbase