Apply on Employer Site

PMAT, Inc. · 1 month ago

Senior Data Engineer (Op AI)

San Diego, California, United States

Full-time

Onsite

Senior Level, Lead/Staff

$185K/yr - $195K/yr

10+ years exp

PMAT is an innovative small business focused on developing impactful digital solutions for the defense sector. They are seeking a Senior Data Engineer to design, build, and operationalize advanced data pipelines and analytics to support Naval and DoD mission challenges.

Information TechnologySoftware

Growth Opportunities

No H1B

Security Clearance Required

U.S. Citizen Only

Responsibilities

Collaborate with cross-functional teams to understand and address Navy operational challenges using data pipelines and analytics

Design, develop, and implement data pipelines and analytics for naval applications

Perform exploratory data analysis, algorithm development, and testing

Normalize and structure data to common standards for interoperability

Work with multiple data formats, including CSV, JSON, XML, Parquet, and ORC

Develop and deploy data pipelines and analytics in real-world operational environments

Deploy, monitor, and optimize data pipelines to ensure high performance and reliability

Implement event streaming pipelines using Apache Kafka, AWS Kinesis, RabbitMQ, or ZeroMQ

Utilize distributed computing platforms such as AWS Lambda, Dask, or Spark

Leverage cloud-native tools including AWS S3, RDS, EFS, SNS, and SQS

Utilize data pipeline frameworks such as AirByte, Apache Airflow, dbt, Apache Iceberg, and Snowflake

Work with GIS data using ArcGIS, PostGIS, and related tooling

Implement containerized environments using Docker or Kubernetes

Apply cybersecurity principles in the context of secure DoD data applications

Communicate findings and engineering solutions effectively with technical and mission stakeholders

Qualification

Data pipeline developmentCloud data frameworksEvent streaming technologiesDistributed computingProgramming in PythonGIS data integrationData formats familiarityAWS data servicesAnalytical skillsCommunication skills

Required

At least 10 years of experience as a business analyst, data analyst, data scientist, data engineer, database administrator, geospatial analyst/engineer, machine learning engineer, or software engineer

Strong programming skills in Python

Programming experience in Go or Rust

Proven experience designing, developing, and deploying complex data pipelines

Familiarity with data formats including CSV, JSON, XML, Parquet, ORC

Familiarity with event streaming technologies: Kafka, AWS Kinesis, RabbitMQ, ZeroMQ

Experience deploying, monitoring, and optimizing operational data pipelines

Expertise in Elasticsearch, Redis, S3, PostgreSQL, or related datastores

Experience with AWS data services (EFS, RDS, S3, SNS, SQS)

Experience with distributed computing: AWS Lambda, DASK, Spark

Familiarity with AirByte, Airflow, dbt, Iceberg, Snowflake

Experience integrating and retrieving GIS data (ArcGIS, PostGIS)

Strong analytical and problem-solving skills