● have 4-5 years of experience in software development
● have a strong desire to improve your skills with modern development and production environment — Unix/Linux, bash, python, virtualization (Docker, Kubernetes),
● have deployment orchestration (chef, puppet, ansible), configuration servers ZooKeeper, monitoring and distributed logging (Prometheus, Grafana, GrayLog) experience
● have strong knowledge of Scala /Java
● have knowledge of distributed data processing frameworks Hadoop, Spark, Kafka.
● have proven experience of building production-ready ETL processing pipelines
● have familiarity with Big Data persistent distributed storages Impala, Elasticsearch, Aerospike, HBase
● have proven working record with relational databases (e.g. PostgreSQL (stored functions, triggers))
● have good written and verbal English communication skills
● Experience in AWS S3 environment (S3 storage)
● Some experience in the AdTech sector is advantageous
● Location — Kyiv
● Good compensation
● Cozy office (on the best coworking in Kyiv)
● Benefit package
Big Data team is enabling the optimal storage, organization, and timely retrieval of our data.
Your responsibilities will include:
● Carrying out efficient integration with our data providers via various API endpoints and data representation formats.
● Building and deploying an in-house distributed ETL pipeline for processing petabytes of data per day
● Enable an accurate, comprehensive and reliable data storage in our distributed data warehouses based on the needs of other teams
● Providing continuous improvements in the way data is being processed and stored based on the feedback and needs of the business or other teams
● Setting up monitoring for key performance metrics and overall systems’ behavior to promptly react in case any anomaly detected
● You will be responsible for the optimization of ETL pipelines, maintaining over 60 Spark jobs. Building a data lake for data scientists and analysts.
● Experimenting with new tools and technologies to produce cutting-edge solutions to business problems
● Be a part of a self-organizing, results-oriented agile team using Kanban to complete new product launches