Data Engineer (Web Scraping technologies) jobs in United States
cer-icon
Apply on Employer Site
company-logo

Gotham Technology Group ยท 1 day ago

Data Engineer (Web Scraping technologies)

positionNew York, NY
timeFull-time
remoteOnsite
seniorityEntry, Mid Level
money$125K/yr - $190K/yr
date2+ years exp

Gotham Technology Group is seeking a Data Engineer with expertise in Web Scraping technologies. The role involves utilizing AI models and various tools to manage web scraping requests, ensure data quality, and coordinate with internal teams and compliance.

Information ServicesInformation Technology
check
Diversity & Inclusion
Hiring Manager
Autumn Ortenzi
linkedin

Responsibilities

Utilize AI Models, Code, Libraries or applications to enable a scalable Web Scraping capability
Web Scraping Request Management including intake, assessment, accessing sites to scrape, utilizing tools to scrape, storage of scrape, validation and entitlement to users
Fielding Questions from users about the scrapes and websites
Coordinating with Compliance on approvals and TOU reviews
Some Experience building Data pipelines in AWS platform utilizing existing tools like Cron, Glue, Eventbridge, Python based ETL, AWS Redshift
Normalizing/standardizing vendor data, firm data for firm consumption
Implement data quality checks to ensure reliability and accuracy of scraped data
Coordinate with Internal teams on delivery, access, requests, support
Promote Data Engineering best practices

Qualification

AWS cloud experienceWeb scraping frameworksPython programmingCapital markets experienceNoSQLSQL databasesData pipeline orchestrationTime series dataDev Ops practicesCommunication skills

Required

Bachelor's degree in computer science, Engineering, Mathematics or related field
2-5 experience in a similar role
Capital markets experience is necessary with good working knowledge of reference data across asset classes and experience with trading systems
AWS cloud experience with commons services (S3, lambda, cron, Event Bridge etc.)
Experience with web-scraping frameworks (Scrapy, BeautifulSoup, Selenium, Playwright etc.)
Strong hands-on skills with NoSQL and SQL databases, programming in Python, data pipeline orchestration tools and analytics tools
Familiarity with time series data and common market data sources (Bloomberg, Refinitiv etc.)
Familiarity with modern Dev Ops practices and infrastructure-as-code tools (e.g. Terraform, CloudFormation)
Strong communication skills to work with stakeholders across technology, investment, and operations teams

Preferred

Prior buy side experience is strongly preferred (Multi-Strat/Hedge Funds)

Benefits

Plus bonus

Company

Gotham Technology Group

twittertwittertwitter
company-logo
Gotham Technology Group is a provider of guidance and direction to IT professionals.
dateFounded in 2001
location
Montvale, New Jersey, USA
people-size51-200 employees
web-link
https://www.gothamtg.com

Funding

Current Stage
Growth Stage

Leadership Team

leader-logo
Ira Silverman
CEO
linkedin

Recent News

Business Wire
M7 Global Partners Expands North American Reach into ...
2023-12-28
Company data provided by crunchbase