Avani Tech Solutions Private Limited · 3 days ago

Senior Data Engineer

South San Francisco, CA

Contract

Onsite

Senior Level

7+ years exp

Wonder how qualified you are to the job?

Maximize your interview chances

Software Development

Hiring Manager

Chandra Shekar Kandala

Insider Connection @Avani Tech Solutions Private Limited

Discover valuable connections within the company who might provide insights and potential referrals, giving your job application an inside edge.

Responsibilities

Develop, manage, and deliver robust data engineering and architecture including ETL Data Pipelines, Data Integration, and Data Governance.

Shape and implement data governance model for managing and using data across different functions.

Collaborate with Data Science teams, Data Lake Architects, and Product Managers to design and implement data engineering solutions.

Forge strong relationships with stakeholders to resolve complex business challenges and drive end-to-end implementation.

Define, build, test, and implement Data Engineering solutions for Commercial, Medical Affairs, Access Solutions, and Government Affairs.

Qualification

Find out how your skills align with this job's requirements. If anything seems off, you can easily click on the tags to select or unselect skills to reflect your actual expertise.

SQLPythonPySparkJupyter NotebookDatabase DesignETL ProcessesData PipelinesData Quality FrameworksExploratory Data AnalysisCloud-Based Data WarehousingAWS S3AthenaAWS WranglerSagemaker StudioGitRepoCloudWatchLambdaData ModelingArchitecture DesignDatabase Schema DesignSemi-Structured DataJSONKey Value PairUnstructured DataFree Text

Required

Bachelor's degree in Computer Science, Engineering, or related field

7+ years of work experience including hands-on technical experience with SQL, Python, PySpark, Jupyter Notebook, database design, and ETL processes

Proven experience on building modern data pipelines - Ingestion, Profiling, Integration, Summarization / Aggregation, Logging, Orchestrations, DAG etc.

Experience building Data Quality frameworks / solutions, Exploratory Data Analysis (EDA)

Hands-on experience with cloud-based data warehousing / data lake platforms such as AWS S3, Athena, Glue, AWS Wrangler, Sagemaker Studio, GitRepo, CloudWatch, Lambda

Experience with data modeling, architecture and database schema design

Experience handling semi-structured (JSON, Key Value Pair) and unstructured data formats (Free text, documents etc.)