Data Engineer

Job Description

Must Have Technical/Functional Skills

Must Have Skills

• Databricks (Notebooks, Delta Lake, Delta Live Tables, Unity Catalog)
• Python and PySpark
• Spark SQL and Apache Spark
• ETL/Data Pipeline Development
• Big Data technologies: Hadoop, Spark, Kafka, Hive, HDFS, Sqoop
• Strong SQL querying and data modeling
• Data ingestion, transformation, cleansing, and validation
• Cloud platforms (AWS/GCP)
• Data warehouse and large-scale data processing experience
• Experience with streaming data using Kafka and Spark Streaming
• Agile methodologies (Scrum, Kanban, SAFe)

Preferred Skills

• Scala / Scala-Spark
• IBM DataStage, Tibco, or similar ETL tools
• AWS services: S3, Glue, Lambda, Kinesis/MSK
• BI Tools: Tableau, Power BI, Spotfire, OAC
• Databases: Teradata, DB2, BigQuery, Mainframe
• Git version control
• Data governance and retention modeling

Roles & Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL solutions.
• Build and optimize data ingestion frameworks for structured and unstructured data.
• Implement data quality, validation, standardization, and governance processes.
• Develop data infrastructure integrating multiple heterogeneous data sources.
• Support analytics, reporting, machine learning, and business intelligence initiatives.
• Work with Data Scientists and BI teams to prepare and curate data for analytics.
• Develop scalable solutions for Big Data processing and streaming workloads.
• Analyze data and provide actionable insights and recommendations to business stakeholders.
• Gather requirements from business teams and translate them into technical solutions.
• Lead and mentor junior data engineers.
• Collaborate across multiple projects as the Data Engineering SME.
• Drive adoption of Data Engineering best practices and standards.

Salary Range-$100,000-$135,000 a year

TCS Employee Benefits Summary

Discretionary Annual Incentive.

Comprehensive Medical Coverage

Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.

Family Support

Maternal & Parental Leaves.

Insurance Options

Auto & Home Insurance, Identity Theft Protection.

Convenience & Professional Growth

Commuter Benefits & Certification & Training Reimbursement

Time Off: Vacation, Time Off, Sick Leave & Holidays.

Legal & Financial Assistance

Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.