Data Architect (Databricks,Azure)

Data Architect (Databricks, Azure)About the role

We are looking for a Data Architect to design the data foundation for a generative AI program at a regulatory client. You will own how business applications and a Databricks lakehouse exchange data, so that every AI workflow has clean inputs, stable output contracts and governed access to highly confidential information.

What you'll do

• Define the interface contracts between business applications and AI workflows: inputs, output schemas, versioning and error handling.
• Design ingestion of large document submissions (PDF, Excel and Word, with single submissions of up to hundreds of gigabytes) from Azure storage into Databricks, including metadata, document keys and text extraction.
• Design the Unity Catalog model for confidential data: catalogs, schemas, fine-grained access controls tied to user assignments, and lineage.
• Model AI outputs and reviewer feedback: the AI draft versus the human final, provenance fields (model and prompt version, evidence links, confidence) and the write-back tables used to refine prompts.
• Recommend the operational data store pattern (Azure SQL, PostgreSQL or Databricks Lakebase) and how it stays in sync with the lakehouse.
• Plan and integrate the historical data the AI workflows depend on.
• Keep compute cost-efficient through cluster policies, job design and consumption tracking.

What you bring

• 10+ years in data architecture, including 4+ years designing Databricks lakehouses (Delta Lake, Unity Catalog, Workflows or Lakeflow).
• Azure data and security: ADLS or Blob Storage, private endpoints, Entra ID and Key Vault.
• Experience designing data contracts between applications and a data platform, beyond warehouse or BI modeling.
• Governance for sensitive or regulated data: fine-grained access control, lineage and audit.
• Pipelines for unstructured documents at high volume.
• Databricks certification (Data Engineer Professional or architect level).

Nice to have

• Data patterns for generative AI: vector search and retrieval-augmented generation.
• Experience in financial regulation, audit or federal environments.

Working with us

• You will work inside a secure, private cloud environment with highly confidential data.
• You must be authorized to work in the United States and able to pass a background check.

Pay: $120,000.00 - $140,000.00 per year

Benefits:

• 401(k)
• Dental insurance
• Health insurance
• Paid time off
• Vision insurance

Application Question(s):

• Do you have at least 4 years designing Databricks lakehouse(Delta Lake,Unity Catalog,Workflows or Lakeflow?)
• Do you have Azure data and security:ADLS or Blob Storage, private endpoints,Entra ID and Key Vault experience?
• Do you have experience designing data contracts between applications and a data platform,beyond warehouse or BI modeling?

Experience:

• Data Architecture : 10 years (Required)

License/Certification:

• Databricks certification? (Preferred)

Ability to Commute:

• McLean, VA 22106 (Required)

Work Location: Hybrid remote in McLean, VA 22106