Inference Software Engineer at Etched.ai

The Inference Software Engineer will support porting state-of-the-art models to the company’s architecture, help build programming abstractions and testing capabilities to rapidly iterate on model porting, scale and enhance Sohu’s runtime, including multi-node inference, intra-node execution, state management, and robust error handling, and optimize routing and communication layers using Sohu’s collectives. They will also develop tools for performance profiling and debugging, identifying bottlenecks and correctness issues, and may be involved in analyzing performance traces and logs from distributed systems and ML workloads, building applications with extensive SIMD (Single Instruction, Multiple Data) optimizations for performance-critical paths, and familiarizing themselves with cluster orchestration tools (e.g., Kubernetes, Slurm) and ML platforms (e.g., Ray, Kubeflow).