Data Engineer & Team Leader

Der Kandidat ist ein erfahrener Lead Data Engineer mit Fachkenntnissen in der Optimierung von Prozessen und Verfahren, der Erstellung von Pipelines und Treibern sowie der Bereitstellung technischer Unterstützung. Sie haben mehrere Projekte geleitet und verfügen über Erfahrung in der Entwicklung von ETL-Prozessen, Datenmigration und Softwareentwicklung. Sie verfügen über ausgeprägte technische Fähigkeiten in Sprachen wie Scala, Python, SQL und Java sowie über Erfahrung mit AWS, Spark, Databricks und verschiedenen Datenbanken. Sie verfügen außerdem über Managementerfahrung und Kenntnisse in mehreren Sprachen.

Lead Engineer

(Oct 2022 – now)

• Optimization of existing processes and procedures
• Creating new pipelines,modeles,views and drivers (Django + AWS)
• Creating unit-tests for our drivers(PyTest+python)
• Technical support storage
• Writing and optimization drivers, functions
• Kept project leaders regularly updated with progress, maintaining open, productive communication.
• Managed multiple projects with differing technologies, including delegating tasks, assessing quality and sign off.

Lead Engineer

(June 2022 – now)

• Optimization of existing processes and procedures Creating new pipelines and drivers (Java + AWS)
• Creating unit-tests for our drivers(java spring+springBoot) Technical support storage
• Writing and optimization drivers, functions
• Kept project leaders regularly updated with progress, maintaining open, productive communication.

Software Engineer

(April 2021 – May 2022)

• Optimization of existing processes and procedures
• Creating new pipelines and drivers (python+Airflow+spark+AWS) Creating unit-tests for our data-drivers(scala)
• Creating a new data-notebooks(spark)
• Technical support storage
• Big Data migration process development
• Writing and optimization views, stored procedures, and functions

Software Engineer

(September 2020 – April 2021)

• ETL process development
• Preparation of data for building reports
• Data migration process development
• Writing and optimization views, stored procedures, and functions, construction and support of storage

Software Engineer

(August 2019 – September 2020)

• data collection from various sources Source DB (Oracle, Postgres); different api (Beamery api,GoogleAnalytics api and other); flat files (exel, csv, json)
• preparation, pre-processing of data before loading into the DWH (the storage is built in MS SQL server)
• creation and configuration of loading processes in the DWH (using SSIS packages and services, python-scripts, PowerShell-scripts and a .NET Application)
• the creation of procedures, functions and scripts for data transformation for their preparation for reports (in MS SQL Management Studio)
• development, support of storage performance optimization of existing processes and procedures

Software Engineer

(May 2018 – July 2019)

• ETL process development
• Preparation of data for building reports Data migration process development Development of software for the bank Work in the insurance departmen