Anysphere · 16 hours ago
Software Engineer, Data Infrastructure
San Francisco, CA
Full-time
Onsite
Mid LevelAnysphere is on a mission to automate coding, and they are seeking a Software Engineer specializing in Data Infrastructure. The role involves building and evolving data systems that support the company's product and internal decision-making, focusing on large-scale data systems and ingestion pipelines.
Artificial Intelligence (AI)Developer ToolsFoundational AIGenerative AIMachine LearningSoftware Engineering
Responsibilities
Designing and operating large-scale batch data systems using Spark and Ray Data
Scaling data ingestion pipelines as we grow to billions of rows per day
Re-architecting prompt and model interaction data storage with a focus on cost, performance, and usability, primarily on S3
Building and maintaining streaming data infrastructure (Kafka, Flink, or similar)
Working across data warehouses and lakehouse formats, including Iceberg and Delta Lake (or lower-level storage abstractions)
Improving data developer experience, especially for Python-heavy workflows
Supporting database replication and change data capture pipelines (DMS, Debezium, or similar)
Qualification
Required
Designing and operating large-scale batch data systems using Spark and Ray Data
Scaling data ingestion pipelines as we grow to billions of rows per day
Re-architecting prompt and model interaction data storage with a focus on cost, performance, and usability, primarily on S3
Building and maintaining streaming data infrastructure (Kafka, Flink, or similar)
Working across data warehouses and lakehouse formats, including Iceberg and Delta Lake (or lower-level storage abstractions)
Improving data developer experience, especially for Python-heavy workflows
Supporting database replication and change data capture pipelines (DMS, Debezium, or similar)
Deep experience with Spark (Databricks or open-source Spark both count)
Production experience with Ray Data
Hands-on ownership of large data pipelines and storage systems
Comfort debugging performance issues across compute, storage, and networking layers
Clear thinking about data modeling and long-term maintainability
Preferred
Experience running or scaling ClickHouse
Familiarity with dbt, Dagster, or similar orchestration and modeling tools
Company
Anysphere
Anysphere is an applied research lab focused on automating coding through innovative AI solutions.
Founded in 2022
51-200 employees
H1B Sponsorship
Anysphere has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (7)
Funding
Current Stage
Late StageTotal Funding
$3.37BKey Investors
Thrive CapitalOpenAI Startup Fund
2025-12-08Series Unknown
2025-11-13Series D· $2.3B
2025-06-05Series C· $900M
Leadership Team
Sualeh Asif
Co-Founder & CPO
Recent News
Dynamic Business
2026-01-22
Company data provided by crunchbase