alpha-design-ai-inc · San Jose
Skills: Python
POSITION OVERVIEW We are seeking an ML Systems Engineer to optimize the performance and efficiency of large language model inference powering our agentic AI platform. This is a technical role focused on low-level systems optimization. You will implement performance optimizations, build evaluation harnesses, and architect multi-node clusters for training and inference that push the limits of LLM throughput and latency. Your work will directly impact the responsiveness and cost-efficiency of AI agents used by leading semiconductor companies to design chips. KEY RESPONSIBILITIES - Design, deploy, and optimize LLM inference systems across multi-node clusters, maximizing throughput and minimizing latency for production workloads. - Implement and benchmark concrete inference optimizations. - Profile and analyze inference bottlenecks at the systems level—from GPU kernel execution to memory bandwidth constraints. - Build robust evaluation harnesses and benchmarking frameworks that measure accuracy, throughput, latency, and resource utilization across various parallelism strategies. - Collaborate with research scientists to integrate new model architectures and optimizations into production inference infrastructure. - Investigate and apply emerging techniques from research papers and open-source projects to continuously improve inference performance. QUALIFICATIONS - B.S., M.S., or PhD in Computer Science, Electrical Engineering, or a related field (or equivalent experience). - Experience with large-scale ML systems, GPU computing, or high-performance inference optimization. - Strong proficiency in Python and C++/CUDA; hands-on experience with SGLang, vLLM, PyTorch, or similar inference frameworks. - Deep understanding of GPU architecture, memory hierarchies, and parallel computing paradigms. - Experience deploying and optimizing LLMs in production: model serving, batching strategies, distributed inference, or quantization. - Strong systems-level debugging and profiling skil…
Sign in to see the original posting
Browse more developer jobs on RoleBurst