Machine Learning Engineer - Vector Team

Position Overview

Kensho, S&P Global’s hub for AI innovation, is seeking a mid-level Machine Learning Engineer to join the Vector Team. This role is crucial for developing and scaling Retrieval-Augmented Generation (RAG) systems across the company. You will be involved in the full lifecycle of ML projects, with a strong focus on retrieval models, LLM orchestration, and system-level thinking. Our mission is to make complex unstructured data easily discoverable and actionable by building intelligent, retrieval-driven systems that enhance enterprise search, question answering, deep research, report generation, and knowledge discovery experiences across S&P Global platforms.

Company Information

Kensho is S&P Global’s hub for AI innovation and transformation. With expertise in Machine Learning and data discovery, we develop and deploy novel solutions for S&P Global and its customers worldwide. Our solutions help businesses harness the power of data and Artificial Intelligence to innovate and drive progress. Kensho's solutions and research focus on speech recognition, entity linking, document extraction, automated database linking, text classification, natural language processing, and more.

Employment Type

Full-time

Salary

The anticipated base salary range for this position is $150,000 - $190,000. In addition, this role is eligible for an annual incentive bonus and equity plans. Compensation decisions are dependent on the facts and circumstances of each case.

Location Type

Not specified

Responsibilities

• Design and implement end-to-end RAG pipelines that integrate embedding models, vector databases, and data retrieval agents.
• Build and optimize retrieval systems over large-scale proprietary datasets using advanced embedding techniques.
• Develop LLM-based solutions that orchestrate retrieval, generation, and ranking to deliver high-quality, context-aware responses.
• Investigate and solve challenges in vector search, chunking and indexing strategies, and GraphRAG.
• Work closely with Product and Design teams to build ML-based solutions that enhance user experiences and meet business objectives.
• Collaborate closely with the ML Operations team to create automated solutions for managing the entire ML systems lifecycle, from initial technical design to seamless implementation.

Requirements

• Bachelor's degree or higher in Computer Science, Engineering, or a related field.
• 3+ years of significant, hands-on industry experience with machine learning, natural language processing (NLP), information retrieval systems, and large-scale text processing, including designing, shipping, and maintaining production systems.
• Strong programming skills in Python, with a working knowledge of data processing tools and ML frameworks such as PyTorch, Transformers, and HuggingFace.
• Experience working with machine learning libraries/frameworks for Large Language Model (LLM) orchestration, such as Langchain, LLamaIndex, etc.
• Proven experience building ML pipelines for data processing, training, inference, maintenance, evaluation, versioning, and experimentation.
• Experience working with vector databases (e.g., PGVector, OpenSearch, Pinecone) and understanding of similarity search techniques and vector indexing algorithms.
• Demonstrated effective coding, documentation, collaboration, and communication habits.
• Strong problem-solving skills and a proactive approach to addressing challenges.
• Ability to adapt to a fast-paced and dynamic work environment.

Technologies We Love

• ML: PyTorch, Transformers, HuggingFace, LangChain, LlamaIndex
• Tools/Toolkits: LabelBox, Weights & Biases, OpenSearch, PGVector, LiteLLM
• Techniques: RAG, Prompt Engineering, Information Retrieval, Data Embedding
• Deployment: Not specified

Application Instructions

Not specified