200+ applicants
This job has closed.
VARITE INC ยท 3 days ago
Data Engineer
United States
Contract
Remote
Senior Level
8+ years exp
Wonder how qualified you are to the job?
Maximize your interview chancesInformation Technology & Services
Insider Connection @VARITE INC
Responsibilities
Design and develop efficient and scalable data pipelines between enterprise transactional systems, third-party and analytics platforms
Must be a crack coder in one of the following (go-lang, node.js or python)
Must be proficient in writing and refactoring efficient SQL queries.
Must be able to explain features of good data model design
Build and maintain a data environment for speed, accuracy, consistency and 'up' time
Support analytics and data science by building a world-class data mesh environment that empowers analysts to determine insights into revenue and power products across the organization
Integrate third-party data sources and API's into the Bright data mesh ecosystem
Work closely with Data Science team and participate in development of feature engineering pipelines
Design and develop data products with modern AWS cloud technologies such as S3, Redshift, EMR, Hive, Presto, Flink and Spark
Work with the machine learning engineering team to build a data eco system that supports AI products at scale
Design and deploy an enterprise data warehouse that supports internal and market facing analytics products at scale
Ensure data governance principles adopted, data quality checks and data lineage implemented in each hop of the data
Partner with adjacent organizations to ensure proper integration and adherence to standards
Be in tune with emerging trends in data management and cloud technologies and participate in evaluation of new technologies
Ensure compliance through the adoption of enterprise standards and promotion of best practice / guiding principles aligned with organization standard
Qualification
Find out how your skills align with this job's requirements. If anything seems off, you can easily click on the tags to select or unselect skills to reflect your actual expertise.
Required
BS or MS degree in Computer Science or Information Technology or equivalent experience
8+ years of experience as data engineer at an innovative organization
4+ years of hands-on experience in implementing data lake systems using AWS cloud technologies such as S3, Redshift, EMR, Hive, Kafka and Spark
Expert managing AWS services (EC2, S3, Route 53, ELB, VPC, CloudWatch, Lambda) in a multi account production environment
Experience with development frameworks as well as data and integration technologies such as Informatica, Python, Scala
Create new ETLs in AWS Glue with Python or Node.js as the scripting language
Create AWS Lambdas using Python or Node.js as the scripting language
Modify existing ETLs to fix issues where approach is appropriate
Use Glue for ETLs inside of AWS to and from all AWS types of data sources
Support the migration of data into S3, Redshift, DynamoDB, AWS RDS
Expert knowledge of Agile approaches to software development and able to put key Agile principles into practice to deliver solutions incrementally
Monitors industry trends and directions; develops and presents substantive technical recommendations to senior management
Excellent analytical thinking, interpersonal, oral and written communication skills with strong ability to influence both IT and business partners
Ability to prioritize and manage work to critical project timelines in a fast-paced environment
Advanced knowledge for Microsoft SQL Server for future migration to an AWS Database Platform
Preferred
Previous experience with cloud development (AWS, GCP)
Previous experience in design and deployment of data lakes, data mesh, data warehouse and streaming platforms
Previous experience with data quality projects and public records
Previous experience with: AWS DynamoDB, AWS Elastic Map Reduce, AWS Lambda, AWS Step Functions, AWS Redshift, AWS RDS, Terraform or CloudFormation
AWS Architect Certification is a plus
Experience With Machine Learning Libraries and Frameworks (TensorFlow, MLlib) is an added advantage
Exposure to R, SparklyR, and Other R packages is a Plus
Company
VARITE INC
VARITE has a definite spirit.
1,001-5,000 employees
Funding
Current Stage
Late StageLeadership Team
Adarsh Katyal
President & CEO
Sue Patel Arora
Vice President Of Strategic Partnerships
Company data provided by crunchbase
