xAI · Foundation models
Director, Site Operations - Memphis
- Memphis
Still open on xAI's site · checked just now
- Level
- Executive
- Experience
- 7+ years
- Education
- Bachelor's degree
- Posted
What you'll do
As written by xAI · Read the original
RESPONSIBILITIES
- Own cluster uptime by serving as extreme owner of node, rack, and cluster health across 5+ sites running 24/7, accountable for customer Service Level Agreements and consistently exceptional uptime on SpaceXSI's supercompute cluster
- Lead a large operations organization, directing a 250+ person team spanning site managers, shift supervisors, and technicians across four 24/7 shifts, building a culture of excellence and accountability at every layer of the org
- Drive node and rack remediation by ensuring systematic recovery of failed nodes and racks through command-line and physical intervention, driving mean time to repair to the feasible minimum
- Partner across functions by coordinating with facilities operations to limit downtime from power and cooling faults and proactive maintenance; with network engineering on cluster upgrades; and with tenant representatives on node remediation and planned and unplanned downtime
- Own vendor execution, directing vendors through hardware rework and field operations so repairs, replacements, and capacity work happen at the speed the cluster requires
- Lead site reliability engineering, owning the SRE organization responsible for proactive cluster health monitoring, reactive fault mitigation at scale, root cause analyses for node, rack, and cluster issues, and site-wide reliability procedures and fault documentation
- Run data-driven improvement by leading continual improvement and efficiency initiatives, using operational data to balance team resources and raise uptime, repair time, and SLA performance across sites
- Command incidents at scale and set the standard for incident response during cluster-impacting events, providing clear direction, fast recovery, and tight communication with internal and external partners
- Scale operations by standardizing best practices across sites and grow the organization in step with cluster expansion, keeping operations consistent as SpaceXSI's footprint scales
Requirements
BASIC QUALIFICATIONS
- Bachelor’s degree and 7+ years of experience working in a large scale operations with 5+ years leading people leaders of technical teams; OR 10+ years of experience working in a large scale operations with 5+ years leading people leaders of technical teams
PREFERRED SKILLS AND EXPERIENCE
- Proven ability to lead large, multi-site, 24/7 operations organizations in fast-paced, high-responsibility settings
- Deep expertise in server hardware, cluster reliability, and data center technologies, from deployment through lifecycle management
- Experience supporting compute-heavy environments like AI, machine learning, or high-performance computing at scale
- A track record of owning uptime, Service Level Agreements, or reliability metrics for large compute clusters
- Experience leading site reliability engineering or equivalent reliability-focused teams, including root cause analysis and procedure ownership
- Strong analytical skills and the ability to explain technical concepts clearly to diverse audiences, from technicians to executive and tenant partners
- A history of partnering with vendors at scale, driving mean time to repair down, and scaling operations across multiple sites
- Familiarity with tooling and automation (e.g., Jira, Python, Bash) used to monitor cluster health and improve team efficiency
- Enthusiasm for SpaceXSI's mission to accelerate human discovery and unravel the universe
- Ability to thrive in a dynamic, mission-focused environment with on-call ownership of cluster-impacting events
ADDITIONAL REQUIREMENTS
- Willingness to travel frequently to data center locations to support operations across sites
- Physical capability to handle data center tasks, including lifting up to 50 lbs unassisted, standing for long periods, and occasional ladder use
- Must be willing to work extended hours and/or weekends as needed
SpaceXSI is an equal opportunity employer. For details on data processing, view our Recruitment Privacy Notice.
Skills
Stack
- Python
Working languages
English
Company
Last checked . The details above were extracted automatically; the employer's original posting is the authority.