Senior Data Scientist (Generative AI)

Senior or Principal Data Scientist (Generative AI)About the role

We are looking for a hands-on data scientist to build and lead the quality of generative AI workflows that review large document sets and draft structured findings for expert reviewers in a regulated environment. You will write and tune the prompts, retrieval and output logic, and own how the program proves the outputs are accurate and well-evidenced enough for experts to rely on.

What you'll do

• Design and build generative AI workflows that review large document sets against defined criteria, identify potential issues, and draft structured findings and documents.
• Build the evaluation framework: gold sets from historical reviews, and metrics for accuracy, evidence linkage, unsupported statements and consistency, run as a regression check before every prompt or model change.
• Set prompt engineering and retrieval standards: chunking, retrieval over source documents and reference standards, and structured outputs that match application contracts.
• Build document chunking and retrieval over large, mixed-format document sets.
• Define provenance and confidence: prompt and model versioning in MLflow, a confidence score reviewers can act on, and links to the evidence behind each statement.
• Measure improvement from reviewer edits by comparing AI drafts with the final human versions, and feed the results back into prompts.
• Build monitoring of output quality and usage, and document each version and its results.
• Set guardrails for confidential data, and explain results to business leaders in plain language.
• At the Principal level, set standards across workflows and mentor other data scientists.

What you bring

• Senior: 8+ years in data science or machine learning. Principal: 12+ years. Both include 2+ years building production applications on large language models.
• Python and PySpark; Databricks notebooks and MLflow.
• Proven evaluation design for generated text: gold sets, automated scoring and human review loops.
• Databricks Mosaic AI: Model Serving, Vector Search and Agent Framework.
• Document understanding: extracting content from PDF, Excel and scanned files.
• Ability to explain model behavior and limits to non-technical domain experts.
• Experience with Azure OpenAI models.

Nice to have

• Audit, accounting or regulatory domain knowledge.
• Model-graded evaluation and human-in-the-loop review systems.

Working with us

• You will work inside a secure, private cloud environment with highly confidential data.
• You must be authorized to work in the United States and able to pass a background check.
• We are looking for people who can start within a few weeks.

Pay: $125,000.00 - $140,000.00 per year

Benefits:

• 401(k)
• Dental insurance
• Health insurance
• Paid time off
• Vision insurance

Application Question(s):

• Do you have Python and PySpark;Databricks notebooks and MLflow experience?
• Do you have experience with Azure OpenAI models?

Experience:

• Data Science or Machine learning : 8 years (Required)

Ability to Commute:

• McLean, VA 22102 (Required)

Work Location: Hybrid remote in McLean, VA 22102