Prompt Therapy Solutions Inc · 4 hours ago

Senior Data Engineer

United States

Full-time

Remote

Senior Level, Lead/Staff

$155K/yr - $200K/yr

10+ years exp

Maximize your interview chances

Health CareInformation Technology

Insider Connection @Prompt Therapy Solutions Inc

Discover valuable connections within the company who might provide insights and potential referrals.
Get 3x more responses when you reach out via email instead of LinkedIn.

Responsibilities

Data Architecture Design: Creating robust and scalable architectures to manage the flow and storage of data across the organization.

ETL/ ELT Processes: Developing and managing Extract, Transform, Load processes to ensure data is accurately integrated from various sources into data warehouses or lakes.

Data Pipeline Development: Constructing automated pipelines for data processing and transformation, ensuring smooth data flow and timely availability for analysis.

Database Management: Administering databases and ensuring their performance, integrity, and security.

Data Quality and Governance: Implementing data validation, cleansing, and governance practices to maintain high-quality and reliable data.

Collaboration: Working closely with AI engineers, BI engineers, Analysts, and business stakeholders to understand data requirements and provide support in their analytical tasks. Participate in code reviews and architecture discussions to exchange actionable feedback with peers. Contribute to engineering best practices and mentor junior team members. Help break down complex projects and requirements into sprints.

Performance Optimization: Monitoring and optimizing data systems for improved performance and efficiency.

Tool and Technology Integration: Evaluating and integrating new tools and technologies to enhance data management capabilities.

Qualification

Find out how your skills align with this job's requirements. If anything seems off, you can easily click on the tags to select or unselect skills to reflect your actual expertise.

Cloud databasesData lakesData warehousesAWS technologiesData engineering tech stackPythonSQLDistributed systemsData architecture designData quality managementData governanceData security best practicesDevOps engineeringContinuous Integration/DeliveryManaging data pipelinesSparkKafkaAirflowDBTContainersOrchestrationDockerKubernetesLaravelPHPJS frameworks

Required

10+ YOE working with cloud based databases, data lakes, data warehouses (S3, RDS, AWS Athena, AWS Redshift etc.)

Proficiency in data engineering tech stack; for example; Athena / Redshift / Glue / MySQL / Python / Spark / Kafka / SQL / AWS / Airflow/ DBT / containers and orchestration (Docker, Kubernetes) and others

Experience and understanding of distributed systems, data architecture design, and big data technologies

Experience with AWS technologies ( e.g., AWS Lambda, Redshift, RDS, S3, DMS, Glue, Kinesis, SNS etc.)

Knowledge of data quality management, data governance, and data security best practices

Organizational skills: time management and planning

Good knowledge of DevOps engineering using Continuous Integration/Delivery tools like Kubernetes, Jenkins, Terraform, etc., thinking about automation, alerting, monitoring, security, and other declarative infrastructure

Experience with managing multiple data pipelines for internal and external data products/ pipelines