Data Engineer & Team Leader

The candidate is an experienced Lead Data Engineer with expertise in optimizing processes and procedures, creating pipelines and drivers, and providing technical support. They have managed multiple projects and have experience in ETL process development, data migration, and software development. They possess strong technical skills in languages such as Scala, Python, SQL, and Java, as well as experience with AWS, Spark, Databricks, and various databases. They also have management experience and proficiency in multiple languages.

Lead Engineer

(Oct 2022 – now)

• Optimization of existing processes and procedures
• Creating new pipelines,modeles,views and drivers (Django + AWS)
• Creating unit-tests for our drivers(PyTest+python)
• Technical support storage
• Writing and optimization drivers, functions
• Kept project leaders regularly updated with progress, maintaining open, productive communication.
• Managed multiple projects with differing technologies, including delegating tasks, assessing quality and sign off.

Lead Engineer

(June 2022 – now)

• Optimization of existing processes and procedures Creating new pipelines and drivers (Java + AWS)
• Creating unit-tests for our drivers(java spring+springBoot) Technical support storage
• Writing and optimization drivers, functions
• Kept project leaders regularly updated with progress, maintaining open, productive communication.

Software Engineer

(April 2021 – May 2022)

• Optimization of existing processes and procedures
• Creating new pipelines and drivers (python+Airflow+spark+AWS) Creating unit-tests for our data-drivers(scala)
• Creating a new data-notebooks(spark)
• Technical support storage
• Big Data migration process development
• Writing and optimization views, stored procedures, and functions

Software Engineer

(September 2020 – April 2021)

• ETL process development
• Preparation of data for building reports
• Data migration process development
• Writing and optimization views, stored procedures, and functions, construction and support of storage

Software Engineer

(August 2019 – September 2020)

• data collection from various sources Source DB (Oracle, Postgres); different api (Beamery api,GoogleAnalytics api and other); flat files (exel, csv, json)
• preparation, pre-processing of data before loading into the DWH (the storage is built in MS SQL server)
• creation and configuration of loading processes in the DWH (using SSIS packages and services, python-scripts, PowerShell-scripts and a .NET Application)
• the creation of procedures, functions and scripts for data transformation for their preparation for reports (in MS SQL Management Studio)
• development, support of storage performance optimization of existing processes and procedures

Software Engineer

(May 2018 – July 2019)

• ETL process development
• Preparation of data for building reports Data migration process development Development of software for the bank Work in the insurance departmen