ML Systems Engineer – Inference Acceleration

About the role

Arago is looking for an ML Systems Engineer to accelerate inference on its proprietary optical‑CMOS accelerator. You will work across kernels, model execution, multi‑device distribution, runtime, and serving to shape the software stack that powers next‑generation AI workloads.

Key responsibilities

• Analyze modern AI workloads and pinpoint kernel, runtime, memory, and system bottlenecks on Arago’s accelerator.
• Develop and optimise custom kernels, fused operators, and execution strategies to maximise device utilisation.
• Design efficient mappings of models and operators across multiple devices, handling communication and synchronisation.
• Implement inference‑serving techniques such as continuous batching, KV‑cache management, paged attention, and prefill/decode scheduling.

Required profile

• Strong experience in high‑performance ML inference, GPU/accelerator programming, or ML systems engineering.
• Deep understanding of computer architecture, accelerator execution models, memory hierarchies, parallelism and performance bottlenecks.
• Proficiency in C++ and Python, with the ability to work on a rapidly evolving custom accelerator stack.
• Fluent English communication skills.

Required skills

• CUDA, Triton, ROCm/HIP or equivalent low‑level programming environments.
• Operator fusion, tiling, scheduling, data‑movement optimisation, graph execution and profiling.
• Distributed model execution techniques (tensor, pipeline, sequence or expert parallelism).
• Inference‑serving frameworks such as vLLM, SGLang, TensorRT‑LLM.
• Experience with MLIR and custom dialects is a strong plus.