Axians Portugal · 6 days ago

Senior Data Engineer

United States

Full-time

Remote

Senior Level

Wonder how qualified you are to the job?

Information ServicesInformation Technology

Insider Connection @Axians Portugal

Discover valuable connections within the company who might provide insights and potential referrals, giving your job application an inside edge.

Responsibilities

Requirement analysis and design;

Design, Support and Implementation of highly scalable Data Architectures based on Lakehouse technologies, relational Databases, No-SQL and Streaming Scenarios on Kubernetes;

Contributing and establishing best practices for Data Management:

Development of Data Pipelines and improvement of existing Python / Pyspark Frameworks;

Customization of common tools such as Airflow and Spark;

Training and Support for Business Users;

Extending the Data Ecosystem with complementary tools such as Data Catalogs and BI Tools such as Apache Superset.

Qualification

Find out how your skills align with this job's requirements. If anything seems off, you can easily click on the tags to select or unselect skills to reflect your actual expertise.

Lakehouse technologiesSparkPythonApache IcebergS3DremioIceberg Metadata CatalogsData ManagementData ModellingData Pipeline OrchestrationApache AirflowSoftware Development LifecycleGitHubData CatalogsMetadataKubernetesDockerCommunication

Required

Proficiency with Lakehouse technologies, especially Spark, Python, Apache Iceberg, S3. Ideally previous knowledge Dremio and Iceberg Metadata Catalogs

Multiple years of experience in Data Management and Data Modelling in highly scalable Architectures

Very good knowledge of Data Pipeline Orchestration with Apache Airflow including custom Operators

Very good knowledge of the Software Development Lifecycle, preferably with GitHub

Good knowledge of Data Catalogs and related Metadata

Good knowledge of Kubernetes, Docker and related tooling

Excellent communication skills, both verbal and written in English