Software Engineer, Inference

What You'll Own

• Ship new model architectures by integrating them into the inference engine.

• Collaborate across research, engineering, and infrastructure to optimize model efficiency and deployments.

• Build internal tooling to measure, profile, and track the lifetime of inference jobs and workflows.

• Automate, test, and maintain inference services for maximum uptime and reliability.

• Manage and optimize inference workloads across clusters and hardware providers, and scale deployments across thousands of machines.

• Build scheduling systems that use expensive GPU resources optimally while meeting SLOs, and maintain CI/CD for model checkpoints and SDKs.

First 90 Days

One way the first 90 could unfold.

• Days 1–30 — Immerse & Diagnose: Learn the inference stack, the fleets, and where reliability or utilization break.

• Days 30–60 — Ship & Validate: Integrate a model or ship tooling/scheduling that improves uptime or GPU utilization.

• Days 60–90 — Scale & Systemize: Harden deployment pipelines and scheduling across clusters and providers.

What You Bring

• Strong Python and system-architecture skills.

• Experience deploying models with PyTorch, Hugging Face, vLLM, SGLang, TensorRT-LLM, or similar.

• Experience with queues, scheduling, traffic control, and fleet management at scale.

• Experience with Linux, Docker, and Kubernetes, and with orchestration, deployment, and scheduling.

• Familiarity with Redis and S3-compatible storage.

Nice to Have

• Modern networking stacks including RDMA (RoCE, InfiniBand, NVLink).

• High-performance large-scale ML systems (100+ GPUs).

• CUDA, and FFmpeg or multimedia processing.

About Luma: Luma's mission is to build unified general intelligence that can generate, understand, and operate in the physical world. We believe multimodality is critical for intelligence — the next step beyond language models comes from vision. Luma is an equal opportunity employer.