Senior LLMOps Engineer at Kong

Bengaluru, Karnataka, India

Apply Now

Not SpecifiedCompensation

Senior (5 to 8 years), Expert & Leadership (9+ years)Experience Level

Full TimeJob Type

UnknownVisa

API Management, AI, CloudIndustries

Requirements

Bachelor's or Master's degree in Computer Science or a related field
5+ years of experience in building and operating highly reliable SaaS/PaaS systems
Prior ownership of uptime SLOs for a platform operating at 99.99% or higher
Hands-on experience with at least one of the major cloud platforms (AWS, Azure, or GCP)
Strong experience with AI platforms in public clouds (like Bedrock, Vertex AI, etc.)
Familiarity with observability tools such as Datadog, Prometheus, Grafana, Victoria Metrics, Loki, or similar technologies
Expertise in designing and developing highly scalable distributed systems
Experience managing incidents and communicating effectively under high-pressure situations
Strong verbal and written communication skills to effectively collaborate across teams

Responsibilities

Be the founding member of the LLMOps team
Provide an LLM SaaS platform for all the engineering teams at Kong, giving them access to a variety of LLMs based on their needs
Manage model lifecycle with high reliability and uptime SLOs (at least 99.99%)
Fine tune the platform to provide AI security and high throughput for LLM queries
Own the end-to-end deployment architecture of the LLM SaaS platform
Play a pivotal role in shaping the technical direction for AI innovation in Kong
Collaborate closely with product leadership to define and refine strategy, roadmap, and objectives for the LLM SaaS platforms, ensuring alignment between engineering efforts and business goals
Assist in the hiring process and help grow and mentor the team

Skills

Key technologies and capabilities for this role

LLMsLLMOpsSaaSPaaSModel LifecycleHigh AvailabilityUptime SLOsAI SecurityDeployment ArchitectureFine-tuningSemantic CachingLoad BalancingPrompt GuardsPII Sanitization

Questions & Answers

Common questions about this position

What is the salary for the Senior LLMOps Engineer position?

This information is not specified in the job description.

Is this role remote or does it require working in an office?

The role is based in Kong's Bangalore office as part of building out a new LLMOps platform team there.

What key skills and experience are required for this position?

Candidates need 5+ years building and operating highly reliable SaaS/PaaS systems, ownership of 99.99%+ uptime SLOs, hands-on experience with major cloud platforms (AWS, Azure, GCP), strong experience with AI platforms like Bedrock or Vertex AI, and expertise in scalable distributed systems.

What will the team structure be like for this role?

This is a founding member position on a new LLMOps platform team in Bangalore, with responsibilities to assist in hiring and mentoring to grow the team.

What makes a strong candidate for this Senior LLMOps Engineer role?

Strong candidates will have proven experience owning high-uptime SaaS platforms, expertise in cloud AI services, and the ability to shape technical direction while collaborating with product leadership and mentoring a new team.

Kong

API management and connectivity solutions provider

About Kong

Kong focuses on API management and connectivity, providing tools that help businesses manage, secure, and optimize their APIs for software communication. Its main product, Kong Gateway, is a fast API gateway that can handle up to 50,000 transactions per second, while Kong Konnect offers a SaaS platform for API management and Kong Mesh manages microservices. Kong stands out by combining open-source technology with enterprise solutions, allowing free access to core features and offering premium services for businesses. The company's goal is to enhance developer productivity, security, and performance for a diverse range of clients in a rapidly growing market.

San Francisco, CaliforniaHeadquarters

2017Year Founded

$334.7MTotal Funding

SERIES_ECompany Stage

Data & Analytics, Enterprise SoftwareIndustries

501-1,000Employees