Divit Technologies, Inc. ยท 20 hours ago
Senior Data Architect
United States
Full-time
Remote
Senior Level, Lead/Staff
10+ years expDivit Technologies, Inc. is seeking a Senior Data Architect to develop and evolve data architectures and pipelines. The role involves building event-driven services, mentoring junior team members, and collaborating with various stakeholders to leverage data for business insights.
Information Technology & Services
Hiring Manager
A Kumar
Responsibilities
Develop data pipelines responsible for ingesting large amounts of different kinds of data from various sources
Help evolve data architecture and work on Next Generation real time pipeline algorithms and architecture in addition to supporting and maintaining current pipelines and legacy systems
Write code and develop worker nodes for business logic, ETL and orchestration processes
Develop algorithms for better attribution rules and category classifiers
Work with stakeholders throughout the organization to identify opportunities for leveraging company data to drive search, discovery, and recommendations
Work closely with architects, engineers, data analysts, data scientists, contractors/consultants and project managers in assessing project requirements, design, develop and support data ingestions and API services
Work with Data Scientists in building feature engineering pipelines and integrating machine learning models during the content enrichment process
Able to influence on priorities working with various partners including engineers, project management office and leadership
Mentor junior team members, define architecture, code review, hands-on development and deliver the work in sprint cycle
Participate in design discussions with Architects and other team members for the design of new systems and re-engineering of components of existing systems
Wear an Architect hat when required to bring new ideas to the table, thought leadership and forward thinking
Take a holistic approach to building solutions by thinking of the big picture and overall solution
Work on moving away from legacy systems into next generation architecture
Take complete ownership from requirements, solution design, development, production launch and post launch production support. Participate in code reviews and regular on-call rotations
Desire to apply the best solution in the industry, apply correct design patterns during development and learn best practices and data engineering tools and technologies
Performs any other functions and duties assigned and necessary for the smooth and efficient operation
Qualification
Required
Must have a minimum of 10-12 years of hands-on development experience implementing batch and events driven applications using Java, Kafka, Spark, Scala, PySpark and Python
Experience with Apache Kafka and Connectors, Java, Springboot in building event driven services, Python in building ML pipelines
Develop data pipelines responsible for ingesting large amounts of different kinds of data from various sources
Help evolve data architecture and work on Next Generation real time pipeline algorithms and architecture in addition to supporting and maintaining current pipelines and legacy systems
Write code and develop worker nodes for business logic, ETL and orchestration processes
Develop algorithms for better attribution rules and category classifiers
Work with stakeholders throughout the organization to identify opportunities for leveraging company data to drive search, discovery, and recommendations
Work closely with architects, engineers, data analysts, data scientists, contractors/consultants and project managers in assessing project requirements, design, develop and support data ingestions and API services
Work with Data Scientists in building feature engineering pipelines and integrating machine learning models during the content enrichment process
Able to influence on priorities working with various partners including engineers, project management office and leadership
Mentor junior team members, define architecture, code review, hands-on development and deliver the work in sprint cycle
Participate in design discussions with Architects and other team members for the design of new systems and re-engineering of components of existing systems
Wear an Architect hat when required to bring new ideas to the table, thought leadership and forward thinking
Take a holistic approach to building solutions by thinking of the big picture and overall solution
Work on moving away from legacy systems into next generation architecture
Take complete ownership from requirements, solution design, development, production launch and post launch production support. Participate in code reviews and regular on-call rotations
Desire to apply the best solution in the industry, apply correct design patterns during development and learn best practices and data engineering tools and technologies
Performs any other functions and duties assigned and necessary for the smooth and efficient operation
BS or MS in Computer Science (or related field) with 12+ years of hands-on software development experience working in large-scale data processing pipelines
Must have skills are Apache Spark, Scala and PySpark with 2-4 years of experience building production grade batch pipelines that handle large volumes of data
Must have at least 8+ years of experience in Java and API / Microservices
Must have at least 5+ years of experience in Python
5+ years of experience in understanding and writing complex SQL and stored procedures for processing raw data, ETL, data validation, using databases such as SQL Server, Redis and other NoSQL DBs
Knowledge of Big data technologies, Hadoop, HDFS
Expertise with building events driven pipelines with Kafka and Java / Spark
Expertise with Amazon AWS stack such as EMR, EC2, S3
Experience working with APIs to collect and ingest data as well build the APIs for business logic
Experience working with setting up, maintaining, and debugging production systems and infrastructure
Experience in building fault-tolerant and resilient systems
Experience in building worker nodes, knowledge of REST principles and data engineering design patterns
In-depth knowledge of Java, SpringBoot, Spark, Scala, PySpark, Python, Orchestration tools, ESB, SQL, Stored procedures, Docker, RESTful web services, Kubernetes, CI/CD, Observability techniques, Kafka, Release processes, caching strategies, versioning, B&D, BitBucket / Git and AWS Cloud Ecosystem, NoSQL Databases, Hazelcast
Strong software development, architecture diagramming, problem-solving and debugging skills
Phenomenal communication and influencing skills
Preferred
Exposure to Machine Learning (ML), LLM models, using AI during coding, build with AI
Knowledge of Elastic APM, ELK stack and search technologies such as Elasticsearch / Solr
Some experience in workflow orchestration tools such as Air Flow or Apache NiFi
Company
Divit Technologies, Inc.
Divit Technologies Inc. specialises in AI & Automation, Data Science, DevOps Security, Quality Engineering, and IT Staffing Solutions.
201-500 employees
H1B Sponsorship
Divit Technologies, Inc. has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (6)
2024 (1)
Funding
Current Stage
Growth StageCompany data provided by crunchbase