Raya Workforce · 7 hours ago
Spark Developer
United States
Full-time
Remote
Entry, Mid Level
2+ years exp
Maximize your interview chancesManagement Consulting
Insider Connection @Raya Workforce
Get 3x more responses when you reach out via email instead of LinkedIn.
Responsibilities
Develop, test, and maintain scalable data pipelines using Apache Spark for batch and real-time data processing.
Work with large datasets, ensuring they are efficiently processed and made available for analytics and machine learning models.
Integrate Spark with other big data technologies like Hadoop, Kafka, and Hive to build robust data processing solutions.
Collaborate with data engineers, data scientists, and other stakeholders to understand data requirements and ensure the successful implementation of data pipelines.
Write efficient, high-performance code to process large amounts of data in real-time or batch mode.
Ensure the performance, scalability, and security of data pipelines.
Perform data transformations, aggregations, and analysis using Spark SQL and DataFrames.
Optimize and troubleshoot Spark applications for performance and resource efficiency.
Document and maintain technical specifications and workflows.
Qualification
Find out how your skills align with this job's requirements. If anything seems off, you can easily click on the tags to select or unselect skills to reflect your actual expertise.
Required
Strong proficiency in Apache Spark (with PySpark, Spark SQL, and Spark Streaming)
Experience with Hadoop ecosystem components such as HDFS, Hive, and YARN
Expertise in SQL and NoSQL databases like MongoDB, Cassandra, or HBase
Proficiency in programming languages like Java, Scala, or Python
Hands-on experience with Kafka, AWS, Azure, or Google Cloud (for cloud-based data processing)
Strong knowledge of data warehousing concepts, ETL processes, and data modeling
Experience in performance tuning of Spark applications
Ability to work with distributed systems and big data technologies
Excellent problem-solving and debugging skills
Bachelor’s degree in Computer Science, Engineering, or a related field
2+ years of experience in Spark-based data development and processing
Company
Raya Workforce
Welcome to Raya Workforce – Your Trusted Partner in Construction, Oil & Gas and Manufacturing Staffing Solutions Raya Workforce is a leading provider of staffing solutions, catering to the specific needs of the construction, Oil & Gas and manufacturing sectors.
51-200 employees
Funding
Current Stage
Growth StageCompany data provided by crunchbase
