Software Engineer, Inference at Luma AI
RemoteRedwood City, CAResearch & AIFull-timePosted 29 days ago$195K–$345K per year
About the role
As published on Luma AI's job board.
You'll own how Luma's models get served — integrating new architectures into the inference engine, scaling deployments across thousands of machines, and keeping expensive GPU fleets busy while meeting internal SLOs.
This is large-scale inference systems work: scheduling, fleet management, deployment pipelines, and reliability across clusters and hardware providers. It fits a strong systems engineer comfortable with model serving and Kubernetes at scale. If you want pure modeling rather than the systems that run models, this is firmly the systems side.
What You'll Own
- Ship new model architectures by integrating them into the inference engine.
- Collaborate across research, engineering, and infrastructure to optimize model efficiency and deployments.
- Build internal tooling to measure, profile, and track the lifetime of inference jobs and workflows.
- Automate, test, and maintain inference services for maximum uptime and reliability.
- Manage and optimize inference workloads across clusters and hardware providers, and scale deployments across thousands of machines.
- Build scheduling systems that use expensive GPU resources optimally while meeting SLOs, and maintain CI/CD for model checkpoints and SDKs.
First 90 Days
One way the first 90 could unfold.
- Days 1–30 — Immerse & Diagnose: Learn the inference stack, the fleets, and where reliability or utilization break.
- Days 30–60 — Ship & Validate: Integrate a model or ship tooling/scheduling that improves uptime or GPU utilization.
- Days 60–90 — Scale & Systemize: Harden deployment pipelines and scheduling across clusters and providers.
What You Bring
- Strong Python and system-architecture skills.
- Experience deploying models with PyTorch, Hugging Face, vLLM, SGLang, TensorRT-LLM, or similar.
- Experience with queues, scheduling, traffic control, and fleet management at scale.
- Experience with Linux, Docker, and Kubernetes, and with orchestration, deployment, and scheduling.
- Familiarity with Redis and S3-compatible storage.
Nice to Have
- Modern networking stacks including RDMA (RoCE, InfiniBand, NVLink).
- High-performance large-scale ML systems (100+ GPUs).
- CUDA, and FFmpeg or multimedia processing.
About Luma: Luma's mission is to build unified general intelligence that can generate, understand, and operate in the physical world. We believe multimodality is critical for intelligence — the next step beyond language models comes from vision. Luma is an equal opportunity employer.
Similar roles at other AI companies
Open machine learning and AI engineering jobs elsewhere, newest first.
| Role | Company | Salary | Posted | Actions |
|---|---|---|---|---|
| Sr. Software Engineer, Training/Inference (C++) Infrastructure | xAI | — | New | Tailor |
| Staff Machine Learning Engineer, AI Infrastructure Simulation (7XW) | Waymo | $251K–$310K | New | Tailor |
| Applied AI Delivery Lead - Remote in AMER Go To Market | OpenAI | $234K–$335K | New | Tailor |
| Applied AI Delivery Lead - NYC Go To Market | OpenAI | $234K–$335K | New | Tailor |
| Senior Product Manager, Inference Platform Product Management & Program Management | Cohere | $205K–$320K | New | Tailor |
| Software Engineer - Applied AI Engineering | Deepgram | $174K–$212K | New | Tailor |
- Sr. Software Engineer, Training/Inference (C++)NewPalo Alto, CAtodayInfrastructure
- Staff Machine Learning Engineer, AI InfrastructureNewMountain View, CA +2$251K–$310K1d agoSimulation (7XW)
- Applied AI Delivery Lead - Remote in AMERNewRemote$234K–$335K1d agoGo To Market
- Applied AI Delivery Lead - NYCNewRemoteNew York City$234K–$335K1d agoGo To Market
- Senior Product Manager, Inference PlatformNewRemoteToronto$205K–$320K1d agoProduct Management & Program Management
- Software Engineer - Applied AINewSan Francisco, CA$174K–$212K1d agoEngineering
Before you apply
- Scan your resume against this posting to see which of its terms you are missing.
- Tailor your resume so the summary and bullets use the posting's own wording where it is true for you.
- Read the machine learning engineer resume guide and the interview questions for the role.
- Apply on Luma AI's site. Applications never pass through OneTwo Resume.
About Luma AI
Builds generative video and image models and a creative platform for producing visual content with them.
All open roles at Luma AI · All machine learning and AI engineering jobs at AI startups
More roles at Luma AI
| Role | Salary | Posted | Actions |
|---|---|---|---|
| Sales Executive (London based, French Speaking) Go-To-Market | — | Tailor | |
| Research Scientist / Engineer – Performance Optimization Research & AI | — | Tailor | |
| Research Scientist / Engineer – Reinforcement Learning Infrastructure Research & AI | — | Tailor | |
| Research Scientist / Engineer – Reinforcement Learning Infrastructure Research & AI | — | Tailor | |
| Software Engineer, Inference Research & AI | — | Tailor | |
| Software Engineer, Inference Research & AI | — | Tailor |
- Sales Executive (London based, French Speaking)RemoteLondon, UK2w agoGo-To-Market
- Research Scientist / Engineer – Performance OptimizationRemoteLondon, UK4w agoResearch & AI
- Research Scientist / Engineer – Reinforcement Learning InfrastructureRemoteLondon, UK4w agoResearch & AI
- Research Scientist / Engineer – Reinforcement Learning InfrastructureRemote4w agoResearch & AI
- Software Engineer, InferenceRemoteLondon, UK4w agoResearch & AI
- Software Engineer, InferenceRemote4w agoResearch & AI