Irvine Technology Corporation · 21 hours ago

Senior Data Engineer (Azure, DevOps)

Los Angeles, CA

Full-time

Remote

Senior Level, Lead/Staff

$140K/yr - $170K/yr

8+ years exp

Maximize your interview chances

Information TechnologyStaffing Agency

Growth Opportunities

Hiring Manager

Hannah Xu

Insider Connection @Irvine Technology Corporation

Discover valuable connections within the company who might provide insights and potential referrals.
Get 3x more responses when you reach out via email instead of LinkedIn.

Responsibilities

Lead the design and implementation of innovative solutions that support data-driven initiatives, ensuring the smooth integration of data systems across the organization.

Architect and develop end-to-end data ecosystems leveraging Azure, Databricks, and other cloud-native technologies to support real-time and batch processing, data lakes, and data warehouses.

Designs and develops programs and tools to support ingestion, curation, and provisioning of complex first party and third-party data to achieve analytics, reporting, and data science. Design and develop Advanced Data Products and Intelligent API’s. Monitors the system performance by performing regular tests, troubleshoots, and integrates new features.

Lead in analysis of data and the design the data architecture to support BI, AI/ML and data products.

Designs and implements data platform architecture to meet organization analytical requirements. Ensures the solution designs address operational requirements such as scalability, maintainability, extensibility, flexibility, and integrity.

Provide technical leadership and mentorship to team members. Leads peer development and code reviews with focus on test driven development and Continuous Integration and Continuous Development (CICD).

Qualification

Find out how your skills align with this job's requirements. If anything seems off, you can easily click on the tags to select or unselect skills to reflect your actual expertise.

AzureDatabricksCI/CDData EngineeringCloud TechnologiesReal-time IntegrationAgile MethodologiesData ScienceMLData IngestionData ProcessingBig DataNoSQLData Warehouse SolutionsADLSAzure Data FactoryEvent HubIoT HubAzure Stream AnalyticsAzure Analysis ServiceHDInsightAzure Data CatalogCosmo DbML StudioAI/ML

Required

Minimum of eight (8) years of experience as a data engineer with full-stack capabilities

Minimum of five (5) years in Cloud technologies like Azure, AWS or GCP.

Experience in real-time integration, developing intelligent apps and data products.

Strong background in CI/CD practices and IAAS platforms and infrastructure.

Hands-on experience with Databricks, Spark, Fabric, or similar technologies

Strong experience in Agile methodologies.

Experience in data science and/or ML

Experience in developing data ingestion, data processing, and analytical pipelines for big data, NoSQL, and data warehouse solutions.

Hands-on experience implementing data migration and data processing using Azure services: ADLS, Azure Data Factory, Event Hub, IoT Hub, Azure Stream Analytics, Azure Analysis Service, HDInsight, Databricks Azure Data Catalog, Cosmo Db, ML Studio, AI/ML, etc.