The Dignify Solutions, LLC · 4 weeks ago
Big Data Developer with Python Experience - Raritan, NJ(Onsite day 1)
New Jersey, United States
Full-time
Onsite
Mid LevelThe Dignify Solutions, LLC is seeking a Big Data Developer with Python experience. The role involves developing and managing data integration tools, creating complex queries, deploying data models, and ensuring the overall performance of data systems.
Bookkeeping and PayrollHuman ResourcesRecruitingStaffing AgencyTraining
Responsibilities
Development, customize and manage integration tools, databases, warehouses and analytical systems with the use of data related instruments/instances
Create and run complex queries and automation scripts for operational data processing and building out Python ETL processes and writing complex SQL queries
Test the reliability and performance of each part of a system and cooperate with the testing team
Deploying data models into production environments. This entails providing the model with data stored in a warehouse or coming directly from sources, configuring data attributes, managing computing resources, setting up monitoring tools, etc
Responsible for setting up tools to view data, generate reports, and create visuals
Monitoring the overall performance and stability of the system. Adjust and adapt the automated pipeline as data/models/requirements change
Excellent understanding of ETL cycle. Analyze and organize raw data, build data systems and pipelines
Combine raw information from different sources, explore ways to enhance data quality and reliability, interpret trends and patterns from the raw data
Qualification
Required
Experience on Bigdata Hadoop ecosystem, Data lakes, DWH, structured/ Unstructured Data, creating Data pipeline/Data frames, Data validations, Querying Data bases using SQL
Development, customize and manage integration tools, databases, warehouses and analytical systems with the use of data related instruments/instances
Create and run complex queries and automation scripts for operational data processing and building out Python ETL processes and writing complex SQL queries
Test the reliability and performance of each part of a system and cooperate with the testing team
Deploying data models into production environments. This entails providing the model with data stored in a warehouse or coming directly from sources, configuring data attributes, managing computing resources, setting up monitoring tools, etc
Responsible for setting up tools to view data, generate reports, and create visuals
Monitoring the overall performance and stability of the system. Adjust and adapt the automated pipeline as data/models/requirements change
Excellent understanding of ETL cycle. Analyze and organize raw data, build data systems and pipelines
Combine raw information from different sources, explore ways to enhance data quality and reliability, interpret trends and patterns from the raw data
Experience in using of Python/ PySpark and/or Scala for data engineering
Understanding of data types/ handling of different data models
Good knowledge in various phases of SDLC Requirement Analysis, Design, Development and Testing on various Development and Enhancement Projects
Good scripting and programming skills
Preferred
Experience with Spark, Flink, Kafka, Flask, Scala, PySpark for Data engineering
Experience with the Microsoft Azure or AWS data management tools such as Azure Datafactory, Datalake and Databricks or AWS Snowflake
Experience with data visualization tools is a plus (PowerBI, Tableau)
Understanding of descriptive and exploratory statistics, predictive modelling, evaluation metrics, decision trees, machine learning algorithms is a plus
Company
The Dignify Solutions, LLC
The Dignify Solutions with Global Capabilities and Local Excellence – has combined experience of 30 +years in Client Services/ Engagement/ Relationship/ Partnership, Sales/ Account Management, Service Delivery, Recruiting, Staffing and Talent Acquisition for the whole gamut of skillsets in Information Technology (Digital Transformation, Artificial Intelligence, Machine Learning and other business domains).
51-200 employees
Funding
Current Stage
Growth StageCompany data provided by crunchbase