Software Engineering SMTS / LMTS - AI Core (REMOTE)

Engineering at Qualified

Engineering at Qualified is a high-ownership, high-velocity environment where engineers ship customer-facing product to real enterprises at scale. We're building at the frontier of AI + product: developing agentic software and

embedding AI directly into our platform, using modern toolchains and processes that help engineers move faster while staying focused on high-impact problem solving.

Following our acquisition by Salesforce, we continue to operate with the pace and ownership of a startup, backed by the distribution, resources, and long-term runway of a global enterprise. Engineers have true end-to-end ownership from idea to production: scoping, implementation, testing, deployment, monitoring, and iterative improvement based on real customer feedback.

You'll work alongside a high-talent-density team that cares deeply about craft, performance, reliability, and user experience, and that raises the bar through collaboration, code review, and strong engineering principles. You'll make architecture decisions, solve meaningful system design challenges, and see your work land quickly with customers, without layers of approvals slowing you down.

The Role

As an Engineer on the AI Core team, you'll operate as a technical leader who turns ambiguous, high-stakes AI problems into durable, high-quality systems, while raising the technical bar for the broader team.

AI Core builds the retrieval systems and agent infrastructure that power how our AI agents find, trust, and use knowledge, and the developer tooling that lets engineers across the company ship production agents quickly and safely.

You'll lead significant initiatives end to end: faithful answering over a customer's own knowledge, retrieval that combines RAG and hybrid search, content lifecycle controls such as freshness, deduplication and source precedence, transcript processing, and the frameworks, registries, and evaluation tooling that make building a production agent a paved path.

You'll partner closely with product, design, and engineering leadership to decide what we build, how we build it, and how we measure whether answers actually got better.

What You'll Do

• Lead retrieval initiatives end to end: Design, build, and operate retrieval systems using RAG and hybrid search (lexical plus vector, reranking, filtering) so agents answer from the right content with the right amount of context.
• Own delivery from scope and design through launch, monitoring, and iteration.

Build developer tooling for production agents

Drive technical design for the frameworks and platform pieces our engineers use to build agents, including multi-agent orchestration, shared primitives for prompts, tools, tracing, and evaluation.

Own answer quality

Diagnose why answers are thin, wrong, or inconsistent by separating retrieval problems from generation problems (prompt, token budget, model choice). Set a quality bar we can test against, and lead the

work to reach it.

Design content and data models

Define how content from documents,

knowledge bases, and call transcripts is ingested, segmented, labeled, and retrieved, including the primitives behind freshness, dedupe, source

precedence, and internal versus external visibility.

Make AI changes safe to ship

Build evaluation, observability, and rollout

practices (offline evals, online monitoring, regression detection) so we can change prompts, models, and retrieval with confidence. Evaluate multiple

solution paths and know when to trust, redirect, or override model and agent output.

Set performance and scaling benchmarks across the platform we own

Define and track targets for retrieval and indexing, LLM provider usage, and agent orchestration.

Own operational health

Ensure the reliability and performance of the systems your team owns. Participate in on-call, lead incident resolution for your area, and communicate clearly with the team and stakeholders during incidents.

Use agentic development to raise leverage

Use coding agents by default for implementation, testing, refactoring, and documentation while holding a high validation bar. Create reusable workflows, patterns, and tooling that raise the effectiveness of your team beyond your own output.

Partner across functions

Work with product, design, and engineering

leadership to shape roadmap, tradeoffs, and requirements, and to translate strategy into concrete execution plans.

Raise team effectiveness

Mentor engineers, give high-quality code review and technical guidance, and improve standards, tooling, and processes.

Who You Are

• Proven experience shipping production AI or backend systems in a high ownership environment, with a recent focus on LLM-powered applications.
• Expertise building retrieval systems using RAG and hybrid search, including chunking and indexing strategy, embeddings, vector and lexical search, reranking, and measuring retrieval quality.
• Expertise building developer tooling that lets developers build production agents, which can include frameworks for multi-agent orchestration, an agent and tool registry, and shared infrastructure for tracing and evaluation.
• Experience owning platform initiatives from design through adoption, across systems such as retrieval and indexing, LLM provider integrations, and agent orchestration, including driving adoption and evolving the platform as usage grows.
• Demonstrated technical leadership: you lead significant projects, design solutions for non-trivial problems by weighing tradeoffs, and influence architecture and execution beyond your immediate workstream.
• Strong track record delivering complex projects with multiple stakeholders, dependencies, or phases.
• Strong backend engineering fundamentals and proficiency in languages such as Ruby on Rails, Python, JavaScript, or Go, with SQL and RDBMS experience (PostgreSQL preferred).
• Hands-on experience with the AI application lifecycle: experiment setup, dataset curation, offline evaluation, error analysis, deployment, and online evaluation.
• Comfortable operating in ambiguity and driving clarity: you can frame problems, propose solutions, and align others.
• Strong communicator across functions, able to explain technical ideas to both technical and non-technical audiences and to translate complex concepts into business impact.
• Ownership mindset and bias for action.
• You raise the bar for the team through mentoring, code review, and your own work.
• Curious and continuously learning, with an agent-forward approach to how you build software.
• Nice to Have Experience with pgvector or other vector indexes, and with query performance at scale.
• Experience with content ingestion from third-party systems (document stores, knowledge bases, CRM knowledge objects) and the lifecycle problems that follow: sync state, dedupe, source conflict.
• Experience processing conversation or call transcripts, including handling PII and consent constraints.
• Experience designing evaluation suites that score qualities such as thoroughness and faithfulness, not only correctness.
• Experience with voice or real-time conversational agents.
• Experience with enterprise SaaS (especially B2B marketing or sales tech) or Salesforce/CRM integrations.
• Startup experience and rapid shipping.
• Why Join Qualified Foundational impact: Shape how our AI agents retrieve, reason, and answer, and the tooling every engineer here uses to build them.

Lead real initiatives

Own significant projects end to end, with scope and

influence that grow as you deliver. You'll own the platform as it scales, from retrieval to model providers to agent orchestration, and set the performance and reliability bar that grows with the company.

AI as a core product surface

Turn new AI capabilities into dependable

experiences that customers trust, with strong engineering around evaluation, observability, and iteration.

High bar, low bureaucracy

Work with a team that values craft and decisive execution, where strong ideas win and