xAI · Foundation models
Manager, Supercomputer Site Reliability Engineering - Memphis
- Southaven, +1 more: Memphis
AI roleMLOpsML platform
Still open on xAI's site · checked just now
- Level
- Lead or manager
- Experience
- 3+ years
- Education
- Bachelor's degree
- Posted
What you'll do
As written by xAI · Read the original
RESPONSIBILITIES
- Serve as the senior operational authority for the SRE organization and represent production reliability concerns in engineering and leadership discussions
- Lead integrated Supercompute systems engineering reliability team responsible for reactive and increasingly proactive cluster and facilities health and reliance initiatives
- Build out the site reliability engineering sub-teams: compute hardware failure analysis, software tools, cluster monitoring, and integrated site reliability engineering
- Drive operational superiority
- Define SRE organizational structure and develop culture of 24/7 operational excellence
- Physics limit anomaly response times and robust proactive health processes
- Define on-call responsibilities, incident management protocols, leadership structure, and communication discipline
- Develop training schemes for tough, competent, and disciplined monitoring and response teams
- Establish clear operational interfaces with Datacenter Operations, engineering teams, vendors, and other service owners
- Own vendor oversight and accountability mechanisms
- Drive mean time to repair toward physics limits
- Define and continuously improve operational procedures and analysis documentation mechanisms with strong emphasis on intelligent automation
- Lead robustification of documentation workflow
- Ensure efficient signal to noise in execution
- Define operational targets, health indicators, alerting standards, and incident severity definitions that facilitate measurable team performance insights and accountability mechanisms
- Drive high technical standard for hardware root cause analysis and remediation teams ensuring industry-leading hardware failure diagnostic expertise
- Integrate with platform engineering teams to develop sophisticated hardware and system understanding to drive high operational reliability
- Drive repeat hardware issues to a minimum and lead automation efforts to eliminate manual overhead for unavoidable remediation needs
Requirements
BASIC QUALIFICATIONS
- Bachelor’s degree and 3+ years of experience working in a large scale operations with 2+ years leading people leaders of technical teams; OR 10+ years of experience working in a large scale operations with 5+ years leading people leaders of technical teams
PREFERRED SKILLS AND EXPERIENCE
- Proven ability to lead large, multi-site, 24/7 operations organizations in fast-paced, high-responsibility settings
- Deep expertise in server hardware, cluster reliability, and data center technologies, from deployment through lifecycle management
- Experience supporting compute-heavy environments like AI, machine learning, or high-performance computing at scale
- A track record of owning uptime, Service Level Agreements, or reliability metrics for large compute clusters
- Experience leading site reliability engineering or equivalent reliability-focused teams, including root cause analysis and procedure ownership
- Strong analytical skills and the ability to explain technical concepts clearly to diverse audiences, from technicians to executive and tenant partners
- A history of partnering with vendors at scale, driving mean time to repair down, and scaling operations across multiple sites
- Familiarity with tooling and automation (e.g., Jira, Python, Bash) used to monitor cluster health and improve team efficiency
- Enthusiasm for SpaceXAI's mission to accelerate human discovery and unravel the universe
- Ability to thrive in a dynamic, mission-focused environment with on-call ownership of cluster-impacting events
ADDITIONAL REQUIREMENTS
- Willingness to travel frequently to data center locations to support operations across sites
- Physical capability to handle data center tasks, including lifting up to 50 lbs unassisted, standing for long periods, and occasional ladder use
- Must be willing to work extended hours and/or weekends as needed
SpaceXSI is an equal opportunity employer. For details on data processing, view our Recruitment Privacy Notice.
Skills
Stack
- Python
Working languages
English
Company
Last checked . The details above were extracted automatically; the employer's original posting is the authority.