90 applicants

Company

Original Job Post

S&P Global · 2 days ago

Lead Data Engineer

New York, NY

Full-time

Onsite

Senior Level

$97K/yr - $196K/yr

8+ years exp

Wonder how qualified you are to the job?

Maximize your interview chances

AnalyticsBusiness Intelligence

H1B Sponsorship

Culture & Values

Insider Connection @S&P Global

Discover valuable connections within the company who might provide insights and potential referrals, giving your job application an inside edge.

Responsibilities

Design & Build data pipelines with an emphasis on scale, performance and reliability.

Provide technical expertise in the areas of design and implementation of Data Lake solution powered by Databricks on AWS cloud.

Ensure data governance principles adopted, data quality checks and data lineage implemented in each hop of the data

Partner with the data teams, enterprise architecture organization to ensure best use of standards for the key data domains and use cases

Continuous learner with an eye on emerging trends around data lake architecture and enterprise data solutions.

Ensure compliance through the adoption of enterprise standards and promotion of best practice / guiding principles aligned with organization standards

Qualification

Find out how your skills align with this job's requirements. If anything seems off, you can easily click on the tags to select or unselect skills to reflect your actual expertise.

Big Data TechnologiesJavaScalaPythonSparkStreaming Data PipelinesKafkaConfluentData LakeData WarehouseETLELTDatabricksSnowflakeAzure Data LakeDatabase TechnologiesAnalytical TechnologiesMPP DatabasesNoSQL DatabasesCloud PlatformsAWSAzureGCPEC2S3AKSEKSPublic Cloud CertificationContinuous DeliveryCI/CD Pipelines

Required

Bachelors or Masters degree in Computer Science or Information Technology

8+ years of experience building solutions in big data technologies

Strong experience programming with more than one of Java, Scala, Python, Spark

Hands-on experience designing and building streaming data pipelines using Kafka, Confluent etc

Strongly prefer experience building Data Lake & Data warehouse solutions using ETL,ELT pipelines on Databricks, Snowflake, Azure Data Lake etc

Strong understanding of database and analytical technologies in the industry including MPP and NoSQL databases

Strong understanding of cloud platforms like AWS, Azure, or GCP, and their services (e.g., EC2, S3, AKS, EKS, etc.). AWS or any public cloud certification is a must

Preferred

Experience in continuous delivery through CI/CD pipelines, containers and orchestration technologies

Experience With Machine Learning Libraries and Frameworks (TensorFlow, MLlib) is an added advantage

Expert knowledge of Agile approaches to software development and able to put key Agile principles into practice to deliver solutions incrementally

Monitors industry trends and directions; develops and presents substantive technical recommendations to senior management

Excellent analytical thinking, interpersonal, oral, and written communication skills with strong ability to influence both IT and business partners

Ability to prioritize and manage work to critical project timelines in a fast-paced environment

Financial services industry experience is an added advantage

Databricks experience or certifications is an added advantage

Benefits

Health & Wellness: Health care coverage designed for the mind and body.

Flexible Downtime: Generous time off helps keep you energized for your time on.

Continuous Learning: Access a wealth of resources to grow your career and learn valuable new skills.

Invest in Your Future: Secure your financial future through competitive pay, retirement planning, a continuing education program with a company-matched student loan contribution, and financial wellness programs.

Family Friendly Perks: It’s not just about you. S&P Global has perks for your partners and little ones, too, with some best-in class benefits for families.

Beyond the Basics: From retail discounts to referral incentive awards—small perks can make a big difference.

Company

S&P Global

Glassdoor

4.1

S&P Global is a market intelligence company that provides financial information and data analytics services.

Founded in 1860

New York, New York, USA

10,001+ employees

https://www.spglobal.com

H1B Sponsorship

S&P Global has a track record of offering H1B sponsorships. Please note that this does not guarantee sponsorship for this specific role. Below presents additional info for your reference. (Data Powered by US Department of Labor)

Distribution of Different Job Fields Receiving Sponsorship

Trends of Total Sponsorships

2023 (23)

2022 (37)

2021 (47)

2020 (32)