About StackYak
StackYak is building the orchestration layer for AI infrastructure, unifying compute, GPU infrastructure, networking, and inference into one seamless product. We empower developers, enterprises, and service providers to move AI workloads from idea to production in minutes, not months, by eliminating the complexity of stitching together disparate vendors and tools. As an early-stage, funded company, we are a small, senior team focused on building for real production workloads from day one. Our mission is to create frictionless AI, solving the critical challenge of making inference, compute, and network truly work together on a single, integrated platform. We value what you have built and run, not just credentials, fostering an environment of end-to-end ownership and rapid innovation.
What You'll Do
- Drive the strategy and implementation for optimizing large language model (LLM) inference, covering precision, quantization, parallelism, and memory management across single and multi-node GPU architectures.
- Design, benchmark, deploy, and operate highly-performant, reliable AI inference systems in production, ensuring they meet strict latency and throughput requirements.
- Select, evaluate, and integrate cutting-edge AI serving runtimes (e.g., vLLM, SGLang, TensorRT-LLM), making critical decisions on their application and optimizing their performance.
- Automate and codify deep inference expertise into StackYak's core platform, transforming institutional knowledge into scalable, auditable software solutions.
- Lead the end-to-end lifecycle management of AI models in production, from loading and startup to upgrades, capacity planning, and robust failure handling.
What We're Looking For
- 7+ years of hands-on experience successfully operating and scaling large language models (LLMs) in demanding production environments under real-world loads.
- Deep expertise in optimizing AI inference across diverse GPU configurations (multi-GPU, multi-node), including advanced knowledge of precision, quantization, tensor/pipeline/expert parallelism, and memory strategies.
- Proven command of various AI serving runtimes (e.g., vLLM, SGLang, TensorRT-LLM) and strong proficiency in debugging complex Linux systems problems, including those below the container boundary.
- Exceptional Python programming skills for developing production-grade tooling, automation, and core product features, beyond mere scripting.
- A track record of meticulously measuring, analyzing, and defending critical inference performance metrics (latency, TTFT, throughput, utilization, cost) and effectively finding root causes of system failures.
- A systems-level mindset with a strong desire to automate and programmatically encode inference best practices and decision-making into scalable software solutions.
Tech Stack
Python, Linux
What's In It For You
Competitive compensation
Remote-first culture
Growth opportunities
Work with cutting-edge technology
Ready to join? Apply now and let's build something amazing together.
Python
Linux
AI Engineer
SRE
Network Engineer
Backend