We are looking for an experienced Java Data Engineer (Streaming) who will join our Technology Department to be part of a 250+ engineers team who create cutting-edge solutions and constantly raise the bar.
Construct the architecture for data integration solutions and ensure it supports business requirements
Develop new data solutions (ETLs, pipelines, integration services, batch and stream processing, enrichment services, etc.)
Do Ops work: help the team with k8s tuning, GitLab CI/CD pipelines, Terraform, managing computing resources, etc.
Maintain current data solutions while building a new data platform. Ensure a smooth transition from existing architecture to target one
Build the infrastructure required for optimal data storing, data processing, and data provisioning to customers, possibility to work with data (process, transform, merge, doing ad-hoc, calculate) for external teams.
Prepare datasets from various sources by request.
Set up monitoring, define SLAs, analyze metrics for data delivery and processing
Develop build-in data reconciliation solution
Assemble large, complex data sets that meet functional / non-functional business requirements (business cases)
Recommend ways to improve data reliability, efficiency, and quality
Requirements
5+ years of experience in Software Engineering
Expert skills in Java
Experience with Kafka and Flink or similar technologies of stream processing is a must-have.
Experience with in-memory databases such as Redis, Tarantool, etc.
Database design with massively parallel processing appliances: Vertica, BigQuery, ClickHouse, PostgreSQL, and similar. ClickHouse is a big plus
Expert knowledge of structured query language (SQL) and query optimization
Experience with designing and building ETL pipelines
Experience in building scalable, high-availability analytics solutions
You share and cultivate DevOps culture (continuous integration, continuous delivery) and Agile mindset
You are ready to do a lot of RnD
Nice to have: Experience with Spark Streaming, Kafka Connect, Kafka Streams, KSQL
Done for you
We'll apply to this for you
Denidel's HR team submits and tracks the application on your behalf.
