Tential Solutions · 8 hours ago
Big Data Engineer
Rockville, MD
Contract
Onsite
Senior Level
5+ years expTential Solutions is seeking a highly skilled and experienced Big Data Engineer to design, develop, and optimize large-scale data processing systems. The role involves collaborating with cross-functional teams to architect data pipelines and ensure the performance and reliability of big data platforms.
Human ResourcesInformation Technology
Hiring Manager
Will Bartolotta
Responsibilities
Design, develop, and maintain large-scale data processing pipelines using Big Data technologies (e.g., Hadoop, Spark, Python, Scala)
Implement data ingestion, storage, transformation, and analysis of solutions that are scalable, efficient, and reliable
Stay current with industry trends and emerging Big Data technologies to continuously improve the data architecture
Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions
Optimize and enhance existing data pipelines for performance, scalability, and reliability
Develop automated testing frameworks and implement continuous testing for data quality assurance
Conduct unit, integration, and system testing to ensure the robustness and accuracy of data pipelines
Work with data scientists and analysts to support data-driven decision-making across the organization
Ability to write and maintain automated unit, integration, and end-to-end tests
Monitor and troubleshoot data pipelines in production environments to identify and resolve issues
Qualification
Required
Bachelor's degree in Computer Science, Information Systems or related discipline with at least five (5) years of related experience, or equivalent training and/or work experience
Demonstrated technical expertise in Object Oriented and database technologies/concepts which resulted in deployment of enterprise quality solutions
Past experience with developing enterprise quality solutions in an iterative or Agile environment
Extensive knowledge of industry leading software engineering approaches including Test Automation, Build Automation and Configuration Management frameworks
Strong written and verbal technical communication skills
Demonstrated ability to develop effective working relationships that improved the quality of work products
Should be well organized, thorough, and able to handle competing priorities
Ability to maintain focus and develop proficiency in new skills rapidly
Ability to work in a fast paced environment
Experience with object oriented programming languages such as Java, Scala or Python
Hands-on experience with AI development tools (GitHub Copilot, Q Developer, ChatGPT, Claude, etc.)
Strong software development background with ability to contribute to technical discussions
Extensive experience with Scrum, Kanban, and continuous improvement practices
Experience with Big data technologies such as Hadoop, Spark, Hive & Trino
Evaluate understanding of common issues like Data skew and strategies to mitigate it
Working with massive data volumes in PetaBytes
Troubleshooting job failures due to resource limitations, bad data, scalability challenged
Look for real-world debugging and mitigation stories
Proficiency in crafting effective prompts for AI coding assistants and analysis tools
Experience redesigning development processes to leverage AI capabilities
Ability to interpret AI-generated insights and translate them into actionable team improvements
Experience leading teams through AI adoption and workflow transformation
Preferred
Master's degree and past Financial Services industry experience
Company
Tential Solutions
We put people first. We’re your trusted partner - empowering you with top talent and solutions to stay competitive in today’s dynamic market.
501-1000 employees
H1B Sponsorship
Tential Solutions has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2024 (2)
2023 (1)
2022 (3)
2020 (11)
Funding
Current Stage
Late StageLeadership Team
Shane Ireland
Chief Executive Officer
Quinn Salamandra
Co-Founder
Recent News
Australian Financial Review
2024-05-02
2024-04-12
Company data provided by crunchbase