Software Engineering SMTS
- Location
- Washington - Bellevue · Bellevue
- Employment
- Full-time
- Job function
- Software/Internet/AI
Posted Nov 20, 2026Last confirmed open Nov 21, 2026
About this role
Software Engineer (MTS / SMTS / LMTS), AI Runtime Platform
The role
Build the inference platform behind Salesforce’s AI agents.
Behind every Salesforce AI agent is a model, and behind every model is infrastructure that has to be fast, cheap, and secure. We’re building the Kubernetes-based platform that serves those models: a large and fast-growing fleet of custom models handling massive volumes of LLM traffic across multiple public clouds.
You’ll join the Hyperforce Container Platform (HCP) team and build the layer beneath the models: GPU scheduling, distributed inference, and model serving.
This role is for engineers who want their code in the serving path of nearly every agentic request. You’ll work in the open-source ecosystem that the AI industry runs on, and make it production-grade for enterprise requirements: reliability, security, and strict tenant isolation.
What you’ll do
- Build the inference platform.
Design and build core capabilities on Kubernetes
GPU scheduling, distributed model serving, and intelligent request routing.
- Scale the LLM runtime. Support both inference and training with high-throughput serving and distributed multi-GPU execution.
- Make shared GPUs a multi-tenant service. Drive inference performance and cost efficiency at fleet scale, with strong tenant isolation.
- Treat AI as a first-class workload. Secure and operate it on the same service mesh, workload identity, and caching layer as every other Salesforce service.
- Build the agentic runtime layer. Work on multi-agent orchestration, retrieval-augmented generation (RAG), knowledge graphs, and evaluation for accuracy and safety.
- Isolate untrusted workloads. Build sandboxing for AI agents and the code they generate, keeping production workloads separated from each other.
- Own reliability end to end. Run services to a 99.99% availability SLA, and make the platform observable and self-healing at fleet scale.
- Shape the open-source foundations. Contribute upstream to vLLM, llm-d, Ray, and Kueue, Agent Substrate, and influence their direction and roadmaps.
You’re a great fit if you
- Have strong software engineering experience, with hands-on depth in distributed systems and production service ownership
- Have proven experience developing with AI coding agents (such as Claude Code, Cursor, or Copilot): you know how to direct them, review their output critically, and ship reliable code with them in your daily workflow
- Know Kubernetes well and have built controllers or operators using Go. Istio/Envoy mesh experience is a plus.
- Have built or operated services on a major public cloud (AWS or GCP)
- Have worked with managed AI platforms for training and inference, such as AWS SageMaker or Google Vertex AI
- Write Go fluently
- Have worked on AI/ML model serving, model training or agentic systems, and can reason about the architecture, cost, and quality of what you build
- Treat reliability as a design principle: observability, SLOs, and self-healing are built in from the start
- Hold a degree in a related technical field
Even better if you
- Have used inference and orchestration stacks such as vLLM, llm-d, Ray, or Kueue
- Have hands-on experience with GPU scheduling, CUDA, or inference performance tuning
- Know networking fundamentals: TCP/IP, DNS, and TLS termination
- Have used Terraform and Argo, and observability tools such as Prometheus, Grafana, Splunk, or Elasticsearch
- Have built multi-agent or RAG systems, or worked with knowledge graphs and LLM-as-judge evaluation
- Have contributed to open-source projects upstream
Prefer to apply directly?
Apply on Salesforce