Platform Software Engineering Architect
- Location
- Pleasanton, CA, US
- Work mode
- Hybrid
- Employment
- Full-time
- Level
- Staff / principal
- Experience
- 8+ years
- Education
- Bachelor's degree
- Job function
- Real Estate/Architecture › Architecture › Architect
Posted Nov 18, 2026Last confirmed open Nov 19, 2026
About this role
We are seeking a visionary, deeply technical, and hands-on Principal Data Lakehouse Architect to spearhead the migration and evolution of our enterprise data platform. In this role, you will lead the architectural design and execution of shifting our massive, petabyte-scale monolithic enterprise data warehouse into a modern, open, and decoupled data lakehouse architecture.
This is not a purely advisory role—you will balance high-level strategic architecture with active, hands-on engineering, layout design, and performance tuning.
You will be responsible for building a highly scalable, cloud-native platform capable of processing hundreds of millions of daily streaming ingestion rows (including high-volume metered usage data) while implementing robust resource guardrails for multi-tenant enterprise workloads.
Core Responsibilities
- Architecture & Migration Strategy: Lead the end-to-end architectural design and migration path from legacy monolithic relational data warehouses (e.g., Oracle Exadata) to an open data lakehouse model on cloud infrastructure.
- Storage & Table Format Engineering: Design, optimize, and maintain open table formats using Apache Iceberg over cloud object storage (such as OCI Object Storage). Manage advanced Iceberg features including hidden partitioning, partition evolution, schema evolution (via unique field IDs), snapshot management, and compaction strategies.
- Metadata & Catalog Management: Establish secure, enterprise-grade metadata management and access control systems utilizing centralized catalogs and metastores like OCI Data Catalog or Apache Polaris.
- Distributed Compute & Query Engines: Architect and fine-tune distributed compute frameworks—specifically Apache Spark for heavy ETL/ELT pipelines and Trino for high-concurrency, low-latency analytical query execution.
- Hybrid Application Development: Build and review core infrastructure tools using a highly performant combination of Java (for infrastructure robustness and concurrency) and Python/PL/SQL (for data processing pipelines and semantic integrations).
- Workload Isolation & Multi-Tenancy: Implement hard resource guardrails and compute clusters isolated by line-of-business (LOB) to ensure intense quarter-end financial workloads and concurrent analytics processing do not suffer from resource contention.
- Semantic Layer & Analytics Integration: Collaborate with functional architects to design scalable data structures, business ontologies, and trusted centralized datasets that interface seamlessly with enterprise reporting layers like Oracle Analytics Cloud (OAC) and next-generation conversational AI agents.
Required Technical Skills & Qualifications
- Experience: Minimum 8–10+ years of experience in data architecture, enterprise data warehousing, and large-scale distributed systems, with a proven track record of delivering production-grade data lakehouses.
- Hands-on Expertise: Deep, production-tested experience with Apache Iceberg, including internal metadata hierarchy navigation (manifest lists, manifest files, metadata JSON evolution).
- Distributed Engines: Advanced operational engineering knowledge of Apache Spark and Trino/Presto, including query optimization, memory management, and cost-based optimizers.
- Languages: Strong programming fluency in Java, Python, and advanced SQL/PL/SQL. Comfort navigating low-level performance considerations (e.g., concurrency, JVM tuning, and optimizing file storage formats like Apache Parquet).
- Data Scale: Experience managing and processing massive datasets exceeding 1 Petabyte in size, with daily ingestion pipelines handling 100M+ rows.
- Education: Bachelor’s or Master’s degree in Computer Science, Software Engineering, or a related technical field.
Preferred Qualifications
- Strong understanding of building semantic layers, entity-resolution pipelines, or business ontologies supporting NLP and conversational analytical interfaces.
- Experience deploying and scaling data architecture within Oracle Cloud Infrastructure (OCI) or related enterprise cloud environments.
- Background in refactoring complex financial analytics, multi-tenant resource scheduling, and managing intense data workloads during tight corporate close cycles.
Prefer to apply directly?
Apply on Oracle