Technology Lead | Data On Cloud - Platform | AWS jobs in United States
info-icon
This job has closed.
company-logo

IMCS Group ยท 1 day ago

Technology Lead | Data On Cloud - Platform | AWS

positionRichardson, TX
timeFull-time
remoteOnsite
senioritySenior Level, Lead/Staff
date8+ years exp

IMCS Group is one of the fastest growing MWBE staffing firms in the U.S. They are seeking a Technology Lead specializing in Data On Cloud, focusing on building robust ETL/ELT pipelines using Pyspark and managing data ingestion from various sources. The role involves collaborating with data teams and optimizing data processing workflows in a cloud environment.

Staffing & Recruiting
check
Growth OpportunitiesbadNo H1Bnote

Responsibilities

Design and build robust, scalable ETL/ELT pipelines using Pyspark to ingest data from diverse sources (databases, logs, APIs, files)
Transform and curate raw transactional and log data into analysis-ready datasets in the Data Hub and analytical data marts
Develop reusable and parameterized Spark jobs for batch and micro-batch processing
Optimize performance and scalability of Pyspark jobs across large data volumes
Ensure data quality, consistency, lineage, and proper documentation across ingestion flows
Collaborate with Data Architects, Modelers, and Data Scientists to implement ingestion logic aligned with business needs
Work with cloud-based data platforms (e.g., AWS S3, Glue, EMR, Redshift) for data movement and storage
Support version control, CI/CD, and infrastructure-as-code where applicable
Participate in Agile ceremonies and contribute to sprint planning, story grooming, and demos

Qualification

PySparkData pipelinesSQLAWSPythonDistributed computingData lake architectureWorkflow orchestrationData governanceDevOps

Required

4+ years of experience in data engineering, with strong focus on PySpark/Spark for big data processing
Expertise in building data pipelines and ingestion frameworks from relational, semi-structured (JSON, XML), and unstructured sources (logs, PDFs)
Proficiency in Python with strong knowledge of data processing libraries
Strong SQL skills for querying and validating data in platforms like Amazon Redshift, PostgreSQL, or similar
Experience with distributed computing frameworks (e.g., Spark on EMR, Databricks)
Familiarity with workflow orchestration tools (e.g., AWS Step Functions, or similar)
Solid understanding of data lake / data warehouse architectures and data modeling basics
Minimum Years of Experience: 8-15+
Certifications Needed: YES. AWS

Preferred

Experience with AWS data services: Glue, S3, Redshift, Lambda, CloudWatch, etc
Familiarity with Delta Lake or similar for large-scale data storage
Exposure to real-time streaming frameworks (e.g., Spark Structured Streaming, Kafka)
Knowledge of data governance, lineage, and cataloging tools (e.g., AWS Glue Catalog, Apache Atlas)
Understanding of DevOps/CI-CD pipelines for data projects using Git, Jenkins, or similar tools

Company

IMCS Group

twitter
company-logo
IMCS Group is an IT, Healthcare, and Professional Staffing Company that helps Enterprises optimize the business value of their Staffing investments and enables them to achieve world-class business performance.
dateFounded in 2002
location
Irving, Texas, US
people-size201-500 employees
web-link
https://imcsgroup.net

Funding

Current Stage
Growth Stage

Leadership Team

leader-logo
Satish G Kumar
Founder and CEO
linkedin
leader-logo
Kathleen Thompson
Diretor Client Partnership
linkedin
Company data provided by crunchbase