Senior Software Engineer - Data Platform at Cognite

Bengaluru, Karnataka, India

Apply Now

Not SpecifiedCompensation

Senior (5 to 8 years)Experience Level

Full TimeJob Type

UnknownVisa

Oil & Gas, Chemicals, Pharma, Manufacturing, EnergyIndustries

Requirements

Distributed Systems Experience (4-6 years)
Production Spark experience - built and optimised large-scale Spark applications with understanding of internals
Streaming systems proficiency - implemented real-time data processing using Kafka, Flink, or Spark Streaming
JVM Language expertise - strong programming skills in Java

Responsibilities

Design and implement robust data processing pipelines using Apache Spark, Flink, and Kafka for terabyte-scale industrial datasets
Build efficient APIs and services that serve thousands of concurrent users with sub-second response times
Optimize data storage and retrieval patterns for time-series, sensor, and operational data
Implement advanced caching strategies using Redis and in-memory data structures
Engineer Spark applications with deep understanding of Catalyst optimizer, partitioning strategies, and performance tuning
Develop real-time streaming solutions processing millions of events per second with Kafka and Flink
Design efficient data lake architectures using S3/GCS with optimized partitioning and file formats (Parquet, ORC)
Implement query optimization techniques for OLAP data stores like ClickHouse, Pinot, or Druid
Scale systems to 10K+ QPS while maintaining high availability and data consistency
Optimize JVM performance through garbage collection tuning and memory management
Implement comprehensive monitoring using Prometheus, Grafana, and distributed tracing
Design fault-tolerant architectures with proper circuit breakers and retry mechanisms
Contribute to open-source projects in the big data ecosystem (Spark, Kafka, Airflow)
Research and prototype new technologies for industrial data challenges
Collaborate with product teams to translate complex requirements into scalable technical solutions
Participate in architectural reviews and technical design discussions

Skills

Distributed Systems

Data Infrastructure

SaaS Platforms

High-Performance Systems

Industrial Data Management

Cognite

Industrial data management for asset-heavy industries

About Cognite

Cognite specializes in managing industrial data and facilitating digital transformation for asset-heavy industries like oil and gas, power and utilities, and manufacturing. Its main product, Cognite Data Fusion, integrates and organizes data from various sources, making it easier for businesses to analyze and utilize this information effectively. This process, known as data contextualization, enhances the relevance of data for better decision-making. Cognite operates on a software-as-a-service (SaaS) model, allowing clients to subscribe to its software, which provides a consistent revenue stream and access to ongoing updates. Additionally, Cognite offers consulting services to assist clients in optimizing their use of the software. The company's goal is to help industries improve their operations through better data management and digital solutions.

Bærum, NorwayHeadquarters

2016Year Founded

$219.2MTotal Funding

SERIES_BCompany Stage

Data & Analytics, Industrial & Manufacturing, Enterprise SoftwareIndustries

501-1,000Employees

Benefits

Competitive Compensation + 401(k) with employer matching

Health, Dental, Vision & Disability Coverages with premiums fully covered for employees and all dependents

Unlimited PTO + flexibility to enjoy it

Paid Parental Leave Program

Learning & Development Stipends

Global Mobility & Exchange Program

Company Paid Friday Lunch via DoorDash + Fully Stocked Fridges in the offices

Risks

Emerging industrial AI startups pose a threat to Cognite's market share.

Geopolitical tensions could disrupt operations in key regions like the Middle East.

Reliance on cloud providers introduces risks related to data security and outages.

Differentiation

Cognite Data Fusion integrates and contextualizes data for asset-heavy industries.

Cognite offers a subscription-based SaaS model, ensuring continuous software updates.

Cognite's solutions enhance safety, sustainability, and efficiency in industrial operations.

Upsides

Cognite's partnership with Google Cloud enhances scalability and security for data management.

The launch of Cognite Embedded opens new innovation avenues for industrial software builders.

Cognite's joint venture with Saudi Aramco expands its influence in the MENA region.

Land your dream remote job 3x faster with AI

Try Jobo Free