Senior Databricks Data Engineer

Role: Senior Databricks Data Engineer

Location: New York City, NY

Fulltime

Job Description

Must Have Technical/Functional Skills

Databricks, PySpark, Spark SQL and Delta Lake

Auto Loader, Structured Streaming, AWS S3 and ETL/ELT

Workflow orchestration, Git/CI/CD and performance tuning

Data validation, reconciliation and production support

Roles & Responsibilities

• Design and build production Databricks ingestion and transformation pipelines Implement batch/streaming processing and Delta Lake optimization Automate deployment and scheduling through CI/CD and workflows Monitor, reconcile and troubleshoot data pipelines In addition, should be able to do the following 1.
• Play the role of onsite co-Ordinator for offshore 2.
• Able to conduct RCA and provide and Implement solutions Generic Managerial Skills, If any Customer-facing communication and concise executive reporting Onsite–offshore coordination across distributed teams Structured ownership, prioritization, risk escalation and collaboration Ability to work independently while following enterprise governance and security standards