Senior Data Engineer Porto

Senior Data Engineer
Porto
Porto, Porto, Portugal

Role Overview

We are looking for an experienced Data Engineering Tech Lead with deep expertise in AWS, Python, and relational databases (My
SQL, Postgre
SQL). You will be responsible for designing, implementing, and maintaining scalable data solutions in the cloud, providing technical leadership to the team, and ensuring the highest standards of performance, security, and
- efficiency.

Key Responsibilities

• Architect and implement scalable data solutions using AWS services such as Lambda, Glue, and Step Functions
• Develop robust solution architectures considering scalability, performance, security, and cost optimization
• Demonstrate proficiency in cloud networking (VPCs, subnets, security groups, routing tables)
• Design efficient data models for optimal query performance
• Write and optimize SQL queries, identifying and resolving performance bottlenecks
• Manage ETL processes and data integration into Redshift, My
  SQL, and Postgre
  SQL
• Create documentation and provide training to team members
• Set up and manage logging and tracing mechanisms using AWS Cloud
  Trail and AWS X-Ray
• Implement orchestration solutions using Apache Airflow and AWS Step Functions
• Use Athena for interactive query analysis of large datasets in Amazon S3
• Provide technical leadership and guidance, acting as a subject matter expert in AWS and data engineering technologies
• Write comprehensive solution documents and technical documentation
• Stay
  -
  - date with emerging technologies and industry trends
• Challenge business requirements and propose innovative solutions for efficiency and performance improvement

Required Skills & Experience

• Extensive
  - on experience with AWS services: Lambda, Glue, Step Functions, Cloud
  Trail, X-Ray, Athena
• Strong expertise in Python programming
• Advanced SQL skills and deep knowledge of My
  SQL and Postgre
  SQL
• Solid understanding of ETL processes, data modeling, and data integration into Redshift and other databases
• Experience implementing orchestration solutions (Apache Airflow, AWS Step Functions)
• Ability to design and implement secure, scalable, and
  - efficient architectures
• Excellent documentation and team training skills
• Proven ability to lead teams and act as a technical mentor

Good to Have

• Knowledge of Apache Flink, Kafka, and other streaming data technologies
• Experience with cloud-native architectures and serverless computing
• Certifications in AWS, Azure, or other relevant technologies