Senior Software Engineer

Job Description The client is strategically transitioning the Data Warehouse engineering operations onshore and is looking for a driven Data Warehouse Engineer to help lead this initiative. Your primary focus will be migrating, building, and optimizing ELT pipelines that extract data from third-party APIs and load it into our data lake utilizing managed Delta tables.

You will initially work closely with our existing offshore team to complete a comprehensive Knowledge Transfer (KT).

Key Responsibilities

ELT Pipeline Development: Design, build, and maintain robust ELT (Extract, Load, Transform) pipelines to migrate data from various third-party API sources into our enterprise data lake.

Databricks Ecosystem

Create, manage, and optimize managed Delta tables within a Databricks environment to ensure high data reliability and query performance.

API Integration

Securely connect to, authenticate with, and extract data from a wide variety of third-party RESTful APIs.

Data Modeling & Querying

Write clean, scalable, and efficient Python and SQL code to transform raw data into analytics-ready datasets.

Knowledge Transfer (KT)

Actively participate in onboarding and KT sessions with the current offshore team to seamlessly transition ownership of existing pipelines and architecture.

Documentation

Maintain clear technical documentation for data pipelines, API integrations, and Delta table schemas to support future onshore hires. Required Skills & Qualifications Experience: 6+ years of experience in data engineering, data warehousing.

Core Languages

Strong, hands-on proficiency in Python and SQL.

Databricks & Data Lakes

Practical experience working with Databricks, specifically building and maintaining Delta tables and working within a modern data lake architecture.

API Expertise

Proven experience extracting and handling data from third-party APIs (handling pagination, rate limiting, JSON/XML parsing, etc.).

Methodology

Solid understanding of ELT methodologies and cloud-based data architecture.

Communication

Excellent verbal and written communication skills, with the ability to effectively absorb complex technical information during cross-border knowledge transfers.