HireTalent - Staffing & Recruiting Firm · 15 hours ago
Big Data Specialist
Rockville, MD
Contract
Hybrid
Senior Level
5+ years expHireTalent is a staffing and recruiting firm seeking a highly skilled Big Data Engineer to design, develop, and optimize large-scale data processing systems. The role involves collaborating with cross-functional teams to architect data pipelines and ensure the performance and reliability of big data platforms.
DeliveryHuman ResourcesStaffing Agency
Hiring Manager
Manish Singh
Responsibilities
Design, develop, and maintain large-scale data processing pipelines using Big Data technologies (e.g., Hadoop, Spark, Python, Scala)
Implement data ingestion, storage, transformation, and analysis of solutions that are scalable, efficient, and reliable
Stay current with industry trends and emerging Big Data technologies to continuously improve the data architecture
Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions
Optimize and enhance existing data pipelines for performance, scalability, and reliability
Develop automated testing frameworks and implement continuous testing for data quality assurance
Conduct unit, integration, and system testing to ensure the robustness and accuracy of data pipelines
Work with data scientists and analysts to support data-driven decision-making across the organization
Ability to write and maintain automated unit, integration, and end-to-end tests
Monitor and troubleshoot data pipelines in production environments to identify and resolve issues
Qualification
Required
Bachelor's degree in Computer Science, Information Systems or related discipline with at least five (5) years of related experience, or equivalent training and/or work experience
Demonstrated technical expertise in Object Oriented and database technologies/concepts which resulted in deployment of enterprise quality solutions
Past experience with developing enterprise quality solutions in an iterative or Agile environment
Extensive knowledge of industry leading software engineering approaches including Test Automation, Build Automation and Configuration Management frameworks
Strong written and verbal technical communication skills
Demonstrated ability to develop effective working relationships that improved the quality of work products
Should be well organized, thorough, and able to handle competing priorities
Ability to maintain focus and develop proficiency in new skills rapidly
Ability to work in a fast paced environment
Experience with object oriented programming languages such as Java, Scala or Python
Hands-on experience with AI development tools (GitHub Copilot, Q Developer, ChatGPT, Claude, etc.)
Strong software development background with ability to contribute to technical discussions
Extensive experience with Scrum, Kanban, and continuous improvement practices
Experience with Big data technologies such as Hadoop, Spark, Hive & Trino
Assess comfort with SQL window functions, multi-table joins, aggregations
Test their understanding of Spark's core architecture — executors, tasks, stages, DAG
Check exposure to AWS services like S3, EMR, Glue, Lambda, Athena, etc
Assess ability to write clean, modular, and performant code
Preferred
Master's degree and past Financial Services industry experience
Experience with managing production data pipelines/ETL systems
Experience with CI/CD
Experience writing test cases
AWS certifications
Company
HireTalent - Staffing & Recruiting Firm
HireTalent is a certified Minority Business Enterprise (MBE) workforce solutions firm, specializing in securing the best talent fits in Executive/Retained Search, Direct Hire Placements, MSP, SOW, and nationwide hiring program management and support.
1001-5000 employees
Funding
Current Stage
Late StageLeadership Team
Ashish Kaushal
CEO / Founder
Company data provided by crunchbase