Principal Data Processing Engineer at DataPelago

Location: Mountain View, California
Type: Full-time
Level: Principal
Posted: Oct 28, 2025

Description

DataPelago is seeking a Principal Data Processing Engineer to lead the architecture, design, and implementation of their innovative data processing engine. This role involves driving the evolution of parallel and distributed execution engine architecture, developing critical components, and fostering best practices. The ideal candidate will have extensive experience in large-scale data processing, expertise in high-performance parallel implementations, and strong programming skills in C, C++, and Rust.

Required Skills

C
C++
Rust
Linux
Distributed Systems
Query Optimization
Machine Learning
Cloud-Native Computing
Apache Spark
Apache Flink
Ray

Additional Information

Minimum Education: Bachelor's degree in Computer Science, or a related field
Required Previous Experiences: 15+ years of relevant experience
10+ years of relevant experience
10+ years of deep technical experience in developing core components of enterprise-grade database or analytics execution engines designed for large-scale data processing
demonstrated experience leading teams of 10+ engineers in the design, development, and successful release of high-performance data processing engines for large production deployments
Preferred Qualifications: Master's degree in Computer Science or a related field
proven expertise in developing high-performance parallel implementations of data processing operators and functions on rich data types
significant experience developing for and understanding the internals of platforms such as Apache Spark, Apache Flink, Apache Doris, Apache Gluten, Velox, Apache DataFusion, or Apache DataFusion Comet
strong analytical and problem-solving skills with a passion for performance optimization
excellent communication and collaboration skills, with the ability to articulate complex technical concepts to both technical and non-technical audiences

About DataPelago

DataPelago

data processing

DataPelago makes the DataPelago Nucleus, a universal data processing engine that accelerates GenAI and Analytics data processing across structured, semi-structured, and unstructured data. Its platform serves data and AI practitioners, supporting workloads such as training models, fine-tuning AI, powering RAG, and extracting insights. The company describes its engine as leveraging accelerated computing and open-source frameworks such as Gluten, with integration for SQL, Python, Airflow, and query clients including Notebook, Tableau, and Power BI. DataPelago says customers can deploy without changes to data, tools, and processes, and without vendor lock-in.

View full company profile