Andiamo ยท 2 days ago
Big Data Software Engineer - Intelligent DataOps Software Leader
San Francisco, CA
Full-time
Onsite
Mid, Senior Level
4+ years expAndiamo is a globally recognized staffing and consulting firm specializing in placing top technology professionals. They are seeking a Big Data Software Engineer to build robust batch pipelines and analytical stores for large-scale reporting and machine learning feature generation.
ConsultingHuman ResourcesInformation TechnologyStaffing Agency
Responsibilities
Develop cost-efficient batch processing (Spark/Dataproc/EMR) and orchestration (Airflow/Dagster)
Design lakehouse tables (Iceberg/Delta/Hudi) with compaction, partitioning, and Z-ordering
Implement semantic/metrics layers and OLAP acceleration (BigQuery/Snowflake/ClickHouse)
Enforce governance: lineage, access controls, PII handling, and retention policies
Qualification
Required
4+ years in big data; expert SQL and performance tuning
Strong understanding of storage formats, file layout, and query engines
Company
Andiamo
The Talent Partners for the AI Revolution.
201-500 employees
H1B Sponsorship
Andiamo has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2022 (2)
2021 (1)
Funding
Current Stage
Growth StageLeadership Team
Patrick McAdams
CEO & Co-Founder
Steven Kottler
CFO
Company data provided by crunchbase