Apply on Employer Site

Scrunch · 2 months ago

Senior Software Engineer, Data Systems & AI

United States

Full-time

Remote

Senior Level

Scrunch is a company focused on acquiring and processing a large amount of data using LLMs and traditional analytics methods. In this role, you will be responsible for building features related to data pipelines and analytics APIs, while also contributing to various tasks to deliver value to customers.

Artificial Intelligence (AI)Information TechnologyInternet

Responsibilities

Have extensive experience building & using web APIs, including handling authentication, authorization, caching, rate limiting, etc

Experience building API for external customers is a big plus

Experience with relational (transactional) databases, particularly PostgreSQL

Experience with analytics tools and technologies, such as: Analytical SQL, maybe with a modern analytical data platform like Snowflake, Google BigQuery, DuckDB, Spark SQL, etc

Experience with analytics packages like Pandas or Polars is a plus

Experience with streaming analytics is a plus

Experience writing batch and streaming/interactive data processing pipelines. This could mean any of: Big data tools like Flink, Beam (GCP Dataflow), Spark, etc

Job queues and task processing frameworks (Celery, Resque, or cloud-based options like Google Cloud Tasks)

Multi-step durable execution frameworks like Temporal or Inngest

In-house or home-rolled versions of any of the above you've used at previous companies

The ability to quickly understand source data and validate output data quality

Experience using LLMs and/or traditional ML techniques to build classifiers, data extractors, summarizers, etc

Not required but a big plus: experience building evals for LLM-based tools to assess and improve accuracy at these kinds of tasks

Qualification

Web APIsPostgreSQLData processing pipelinesAnalytical SQLLLMsReactData quality validationStreaming analyticsAnalytics tools

Required