authentica solutions · 5 days ago

Principal Data Engineer

United States

Full-time

Remote

Senior Level

10+ years exp

Wonder how qualified you are to the job?

Maximize your interview chances

Data IntegrationData Management

Insider Connection @authentica solutions

Discover valuable connections within the company who might provide insights and potential referrals, giving your job application an inside edge.

Responsibilities

Guide the existing engineering team in designing, developing, and maintaining the Education Intelligence Platform.

Construct, manage, and enhance fault-tolerant data infrastructure while upholding data quality and integrity standards.

Collaborate with engineering and product teams to execute on product goals.

Develop and manage fault-tolerant, scalable data pipelines using distributed cloud technologies.

Develop data ingestion, processing, and transformation techniques to ensure data integrity.

Oversee the maintenance of practices and processes for testing and code promotion.

Conduct POCs to validate new tools and services for data engineering solutions.

Troubleshoot production data quality issues and ensure data integrity.

Stay abreast of industry standards and technological advancements to improve engineering output.

Qualification

Find out how your skills align with this job's requirements. If anything seems off, you can easily click on the tags to select or unselect skills to reflect your actual expertise.

PythonData wranglingEtl/eltApi managementRest/api integrationContainersMicro servicesData warehousesData flowsCommunicationEnglish fluencyOrchestration ToolsStreaming PlatformsReal-Time Data Pipeline SystemsApache KafkaAzure Event HubsBig Data Storage TechnologiesAzure Data LakeHadoopBigQueryEducation Sector Experience

Required

Minimum of 10 years hands-on experience with Python and related data libraries (e.g. Pandas, Data Frames) Practical expertise in ETL/ELT technologies and methodologies.

Proven experience in data wrangling and cleaning across structured, semi-structured, and unstructured data formats.

Solid design and development background in modern technologies such as API management, REST/API integration, Containers, and Micro services.

Experience in designing or working with data warehouses, including an understanding of associated data flows.

Exceptional communication skills, both written and verbal.

English fluency is required to effectively communicate with our clients and other key stakeholders both internal and external.

Preferred

Familiarity or experience with orchestration tools.

Knowledge or hands-on experience with streaming platforms and real-time data pipeline systems such as Apache Kafka, Azure Event Hubs, and similar platforms.

Experience working with big data storage technologies like Azure Data Lake, Hadoop, BigQuery, and similar tools.

Education Sector experience.