Apply on Employer Site

ExecutivePlacements.com · 21 hours ago

Senior Data Engineer (Developer)

Dallas, TX

Full-time

Onsite

Senior Level

$145K/yr - $145K/yr

7+ years exp

ExecutivePlacements.com is seeking a seasoned Senior Data Engineer to join their Shared Services MediaLab team. In this role, you'll be responsible for designing and maintaining scalable data infrastructures that support strategic initiatives, including real-time analytics and advanced AI applications.

Human ResourcesOnline PortalsRecruiting

No H1B

Responsibilities

Design and deploy enterprise-grade data architectures supporting structured, semi-structured, and unstructured data across multiple cloud platforms (Azure, AWS)

Implement scalable data lakes and data warehouses optimized for both batch and real-time workloads

Develop and maintain data mesh architectures to facilitate self-service analytics while ensuring robust data governance and security

Architect cloud-native solutions utilizing serverless computing, containerization, and microservices

Build and orchestrate reliable, fault-tolerant data pipelines using modern ELT methodologies and tools like Apache Airflow, Azure Data Factory, and AWS Glue

Develop real-time streaming solutions with Apache Kafka, Apache Pulsar, and cloud-native services to support live data processing needs

Implement automated data quality frameworks with monitoring, alerting, and auto-remediation capabilities

Create CI/CD pipelines for data workflows, incorporating automated testing, deployment, and rollback procedures

Embed machine learning workflows into data pipelines, enabling feature engineering, model training, and large-scale inference

Support MLOps practices with model versioning, A/B testing, and automated retraining pipelines

Build infrastructure to support generative AI initiatives, including vector databases and retrieval-augmented generation (RAG) systems

Collaborate with data scientists and developers to produce scalable ML models and ensure efficient inference

Establish comprehensive data governance frameworks, including data lineage, metadata management, and cataloging

Ensure compliance with privacy laws (GDPR, CCPA) by implementing data masking, encryption, and strict access controls

Maintain audit trails for data processing activities and model predictions

Optimize data processing performance via query tuning, indexing, and resource management

Implement observability strategies, including metrics, logging, and distributed tracing for all data pipelines

Conduct root cause analyses and resolve data quality or system performance issues swiftly

Define and maintain SLAs for data freshness, accuracy, and system uptime

Work closely with cross-functional teams to gather requirements and deliver impactful solutions

Provide technical mentorship to junior engineers and analysts

Lead technical design reviews and contribute to strategic technology planning

Document best practices, data architectures, and system workflows; lead knowledge-sharing initiatives

Qualification

Data EngineeringCloud PlatformsData Pipeline DevelopmentData GovernanceMLOpsDatabase ManagementProgramming SkillsCI/CD PipelinesContainerizationCollaboration SkillsProblem-Solving Skills

Required

Minimum of 7+ years of hands-on experience in data engineering, with a proven track record of building and maintaining large-scale production data systems

Strong experience working directly with internal clients and stakeholders

Extensive expertise in database development and management

Bachelor's degree in Computer Science, Information Technology, or a related field (Master's preferred)

Proven experience in the following technical areas: Cloud platforms: Azure & AWS (data services, infrastructure management), Data frameworks: Apache Spark (PySpark, Scala, Java), Hadoop ecosystem, Databricks, Real-time processing: Kafka, Pulsar, Kinesis, Event Hubs, Data storage: SQL, NoSQL, NewSQL (PostgreSQL, MongoDB, Cassandra, Snowflake), Data modeling and warehousing: dimensional modeling, star/snowflake schemas, SCD, ETL/ELT tools: Azure Data Factory, Fabric Dataflow, AWS Glue, dbt, Infrastructure as Code: Terraform, Bicep, CloudFormation, Containerization & Orchestration: Docker, Kubernetes, Data lakehouse architectures: Delta Lake, Apache Iceberg, Apache Hudi, MLOps tools and workflows: MLflow, Kubeflow, SageMaker, Vector databases and embedding techniques for AI applications, Observability tools: DataDog, Splunk, Prometheus, Grafana, CI/CD pipelines: Azure DevOps, GitHub Actions, Jenkins, Security best practices for cloud and data environments

Strong programming skills in Python, C#, and/or Scala

Deep understanding of distributed systems, fault tolerance, and high-availability architectures

Excellent problem-solving, communication, and collaboration skills