DWH Developer & Data Engineer & Python Developer

Kandidát je dátový inžinier a Python vývojár so záujmom o pozície DWH/BI. Má 3 a pol ročné skúsenosti s DWH/ETL riešeniami a tiež 2 roky s Pythonom. Na svojom nedávnom projekte robila ad hoc zmeny a riešenie problémov v ETL pipeline. Je schopná komunikovať v angličtine a je otvorená práci na dlhodobých aj krátkodobých projektoch, hoci uprednostňuje dlhodobé. Výrazne preferuje prácu na diaľku, ale je otvorená aj občasným návštevám v Záhrebe (2x/mesiac). Vedela by si predstaviť prácu v štandardnom zamestnaní aj na voľnej nohe. Momentálne je vo výpovednej lehote na súčasnej pozícii, takže začiatkom apríla môže nastúpiť do nového angažmánu.

Data Engineer/Python Developer (AWS/Python technologies)

Netezza to Redshift Migration (Dates: N/A)

• Worked on PL/SQL stored procedures for ETL into Redshift.
• Validated and finalized ingestions within Redshift.
• Created an ETL framework using Python wrapper (Luigi library) for building ETL pipelines.
• Business trip to Florida for on-site work with the client to finalize the migration process.

Managed Services Support (Dates: N/A)

• Monitored daily data loads using ETL jobs built in SyncSort.
• Developed a process for basic checks such as counts, key uniqueness, duplicates, orphaned fact records, etc.

Real-time Data Processing Pipeline (Dates: N/A)

• Created the data model and necessary data transformations.
• Developed Lambda functions (Python + SQL) for data processing.
• Built the architecture in AWS, including services such as EC2, ECS, Kinesis, S3, Athena, Lambda, ElastiCache (Redis), CodeCommit, and CloudFormation.
• The project focused on ingesting and visualizing streaming data in real-time, with an archiving architecture component.

ETL Solution Development (Dates: N/A)

• Built new data processing pipelines (S3 to S3) and upgraded existing Python framework using PySpark and HDFS on a MapR cluster.

Development of Data Ingestion from Multiple Sources (Dates: N/A)

• Developed a prototype data pipeline for ingesting raw data from multiple sources into a datalake in S3 and Snowflake.
• Guided a team in working on enhancements, assigning and monitoring tasks, and conducting code reviews.

Data Engineer (Dates: July 2017 - March 2020)

• Worked as a consultant on enterprise data warehousing and data ingestion projects for various clients.
• Designed and developed data flows in AWS cloud using services such as AWS Lambda, AWS Glue, S3, and Redshift.
• Developed data ingestion components in Python for both serverless and on-premise systems.
• Experienced in EDW Data Marts and Data Vaults.
• Led a small technical team, provided mentorship, and handled client communication.
• Participated in all SDLC phases of the projects using scrum methodology.

Python Developer (Dates: March 2020 - Present)

• Backend development experience of over 2 years.
• Developed integrations and APIs in AWS cloud using services such as API Gateway, AWS Lambda, Step Functions, and ECS.
• Worked on projects that follow latest standards and best practices, including writing tests, using pre-commit hooks, following formatting standards, and having a code review process.

3rd Party Tool Integration (Dates: N/A)

• Developed event-driven Lambda functions to perform various API calls.
• Integrated a third-party tool (Benchling, biotech specific) with AWS EventBus to enrich and work around its shortcomings.

Scientific Tool Development (Dates: N/A)

• Developed backend code to create an API using FastAPI.
• Created a tool to integrate existing scientific tools and libraries for more efficient genomics research.

ETL Pipeline Support (Dates: N/A)

• Troubleshot and enhanced an existing ETL pipeline for a dentistry industry project.

Skills: SQL (Postgres, T-SQL), PL/SQL, Python (functional and object-oriented), AWS (Lambda, S3, Redshift, Glue), AWS Services (Athena, EMR, Firehose, CodeCommit, CloudFormation, Step Functions, SNS, Secrets Manager), and other tools and technologies mentioned in the CV.

Achievements: AWS Solutions Architect Associate certificate (2018), AWS Solutions Architect Professional certificate (2018).