Operations Engineer, Supercompute - Memphis

RESPONSIBILITIES

• Own monitoring architecture, alerting, signal quality, and automation of monitoring across GPU and host telemetry (DCGM, Prometheus/Grafana) and facility telemetry (BMS, EPMS, DCIM, SCADA, CDU / direct liquid cooling)
• Own compute hardware change incorporation - physical, firmware, software
• Be part of 24/7 incident response, mitigation, and control authority - end to end - from customer impact back to the facility
• Own day-to-day site sustainment operations, including resolving issues impacting rack-level compute and facility infrastructure
• Disposition daily issues across both site operations (rack compute) and facility operations (power and cooling)
• Lead acceptance and deployment testing campaigns to bring new data halls fully online
• Develop production automation for new data hall bring-up, automation behind alerting and automated incident response
• Develop and maintain standard operating procedures and planning for facility and site technicians, including change / upgrade incorporation planning on both rack level and facility
• Own and document the live, changing state of data hall operation as new halls come online
• Enable Site Operations and Facility Operations technicians to do their day to day jobs more efficiently and safely to increase GPU uptime
• Create predictive and responsive automation to alerts and incidents to enable your response to be faster and more impactful
• Lead cross-functional reliability projects spanning compute, network (InfiniBand/RoCE), storage, and facility signal boundaries, including grey failures that cross those boundaries (e.g., NVLink flapping, thermal throttling, power sequencing)
• Truly root cause systemic and design failure classes across hardware and integrated facility and operations systems: turn fleet data (MTBF/AFR, repeat-RMA trends) into proactive alerting, standard responses, vendor RCA inputs, and supplier corrective actions

BASIC QUALIFICATIONS

• Bachelor's degree in mechanical, electrical, or computer engineering, or a scientific discipline and 2+ years of experience in data center, critical facilities, power plant, or large-scale compute operations; OR 6+ years of professional experience in those environments in lieu of a degree

PREFERRED SKILLS AND EXPERIENCE

• Hands-on experience with at least one of: Prometheus/Grafana, NVIDIA DCGM, or industrial telemetry and controls (SCADA, BMS/BAS, EPMS, DCIM, PLCs, Modbus/BACnet).
• Experience in datacenter site reliability, systems engineering, plant operations, or large-scale production operations.
• Experience running acceptance test campaigns for integrated products.
• Experience in AI/ML infrastructure or supercomputing environments: GB200/NVL72 or H100/H200-class GPU fleets, NCCL, InfiniBand/RoCE, Slurm or Kubernetes.
• Fleet firmware and BMC management (Redfish, IPMI), including firmware release validation and staged rollouts.
• Familiarity with data center hardware and plant signals (servers, GPUs, networking, power, cooling) in addition to software telemetry.
• Proven large-scale incident command experience and calm technical leadership on a bridge, whether from an SRE/NOC bridge or a plant control room.
• Experience writing and operating playbooks, runbooks, or standard operating procedures, methods of procedure, emergency operations procedures.
• Proficiency in scripting (Python, Bash) for automation and analysis, plus general experience in at least one systems language (C, C++, Java, Go, Rust, or similar).
• Ability to work collaboratively with cross-functional, cross-country, multi-disciplinary teams.