Technology Lead | Data On Cloud - Platform | AWS jobs in United States
info-icon
This job has closed.
company-logo

Alpha Consulting Corp. ยท 2 days ago

Technology Lead | Data On Cloud - Platform | AWS

positionTX
timeFull-time
remoteOnsite
senioritySenior Level, Lead/Staff
date8+ years exp

Alpha Consulting Corp. is seeking a Technology Lead specializing in Data On Cloud with a focus on AWS. The role involves designing and building robust ETL/ELT pipelines using Pyspark, collaborating with data teams, and ensuring data quality and scalability across various platforms.

ConsultingPharmaceuticalStaffing Agency
check
Growth Opportunities
check
H1B Sponsor Likelynote

Responsibilities

Design and build robust, scalable ETL/ELT pipelines using Pyspark to ingest data from diverse sources (databases, logs, APIs, files)
Transform and curate raw transactional and log data into analysis-ready datasets in the Data Hub and analytical data marts
Develop reusable and parameterized Spark jobs for batch and micro-batch processing
Optimize performance and scalability of Pyspark jobs across large data volumes
Ensure data quality, consistency, lineage, and proper documentation across ingestion flows
Collaborate with Data Architects, Modelers, and Data Scientists to implement ingestion logic aligned with business needs
Work with cloud-based data platforms (e.g., AWS S3, Glue, EMR, Redshift) for data movement and storage
Support version control, CI/CD, and infrastructure-as-code where applicable
Participate in Agile ceremonies and contribute to sprint planning, story grooming, and demos

Qualification

PySparkData engineeringSQLAWSData pipelinesPythonDistributed computingData lake architectureWorkflow orchestrationData governanceDevOpsAgile methodology

Required

4+ years of experience in data engineering, with strong focus on PySpark/Spark for big data processing
Expertise in building data pipelines and ingestion frameworks from relational, semi-structured (JSON, XML), and unstructured sources (logs, PDFs)
Proficiency in Python with strong knowledge of data processing libraries
Strong SQL skills for querying and validating data in platforms like Amazon Redshift, PostgreSQL, or similar
Experience with distributed computing frameworks (e.g., Spark on EMR, Databricks)
Familiarity with workflow orchestration tools (e.g., AWS Step Functions, or similar)
Solid understanding of data lake / data warehouse architectures and data modeling basics
Minimum Years of Experience: 8-15+
Certifications Needed: YES. AWS

Preferred

Experience with AWS data services: Glue, S3, Redshift, Lambda, CloudWatch, etc
Familiarity with Delta Lake or similar for large-scale data storage
Exposure to real-time streaming frameworks (e.g., Spark Structured Streaming, Kafka)
Knowledge of data governance, lineage, and cataloging tools (e.g., AWS Glue Catalog, Apache Atlas)
Understanding of DevOps/CI-CD pipelines for data projects using Git, Jenkins, or similar tools

Company

Alpha Consulting Corp.

twittertwitter
company-logo
Alpha Consulting Corp. has been exceeding expectations in the IT, pharmaceutical, and clinical staffing business since 1994.
dateFounded in 1994
location
East Brunswick, New Jersey, USA
people-size201-500 employees
web-link
http://alphaconsulting.com

H1B Sponsorship

Alpha Consulting Corp. has a track record of offering H1B sponsorships. Please note that this does not guarantee sponsorship for this specific role. Below presents additional info for your reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (2)
2024 (4)
2023 (3)
2021 (4)
2020 (6)

Funding

Current Stage
Growth Stage
Company data provided by crunchbase