AWS Data Engineering Lead Porto

AWS Data Engineering Lead
Porto
Porto, Porto, Portugal

Your connection with Aubay starts in the following lines:

Aubay Portugal is a multinational French company, in Portugal since 2007. We have offices in Lisbon and Oporto and we are a specialized consultant in Management, Implementation, Development and Maintenance of Information Systems. We have more than 150 active partners and we operate in sectors such as banking, insurance, telecommunications, services, energy and transports.

What are you going to do?

• Utilise expert knowledge of AWS services like Lambda, Glue, and Step Functions to design, implement, and maintain scalable data solutions.
• Develop robust solution architectures considering scalability, performance, security, and cost optimisation.
• Demonstrate proficiency in cloud networking, including VPCs, subnets, security groups, and routing tables.
• Design efficient data models for optimal query performance.
• Write and optimise SQL queries and identify performance bottlenecks.
• Knowledge to manage ETL processes and data integration into Redshift, My
  SQL, and Postgre
  SQL
• Create documentation and provide training to team members.
• Set up and manage logging and tracing mechanisms in AWS using services like AWS Cloud
  Trail and AWS X-Ray.
• Implement orchestration solutions using Apache Airflow and AWS Step Functions.
• Utilise Athena for interactive query analysis of large datasets in Amazon S3.
• Provide technical leadership and guidance, acting as a subject matter expert in AWS and data engineering technologies.
• Write comprehensive solution documents and technical documentation.
• Stay updated on emerging technologies and industry trends.
• Challenge business requirements and propose innovative solutions for efficiency and performance improvement

What are we looking for?

• Expertise in AWS: Extensive experience with AWS services, including Lambda, Glue, Step Functions, Cloud
  Formation, and Cloud
  Watch.
• Strong Solution Architecture Knowledge: Ability to design scalable and efficient data solutions on AWS, adhering to best practices for cloud architecture and infrastructure.
• Proficiency in Python and Databases: Strong programming skills in Python and experience with relational databases (My
  SQL, Postgre
  SQL, Redshift) and No
  SQL databases.
• Orchestration and Workflow Management: Experience with orchestration tools such as Apache Airflow and AWS Step Functions for automating and managing data workflows.
• ETL Tools and Big Data Experience: Knowledge of ETL tools and experience working with large volumes of data, with a preference for experience with Kafka.
• Experience with Iceberg Tables: Familiarity with Iceberg tables for managing large datasets efficiently, ensuring data consistency, and supporting ACID transactions.
• Production Awareness and Troubleshooting: Proactive approach to production monitoring and troubleshooting, with the ability to anticipate and mitigate potential issues.
• Technical Leadership and Communication: Capability to evolve into a technical lead role, with excellent communication and teamwork skills for effective collaboration with
  - functional teams.
• Strong Analytical and Problem-Solving Skills: Ability to analyse requirements, define technical approaches, and propose innovative solutions to complex problems.
• Documentation and Requirements Analysis: Experience in writing solution documents and technical documentation, with the ability to challenge and refine business requirements.
• Exposure to Azure Databricks: Familiarity with Databricks for data engineering and analytics tasks.

What do we have to offer you?

• Health Insurance;
• Feedback&Coffee – the system that includes Aubay’s Performance Management Cycle, with regular moments of feedback and the Career Progression Model;
• Working model: remote;
• A Training Academy designed to think of you and your needs. You can count on technical and behavioural training sessions (e-Learning and classroom), Meet&Techs, Webinars and Mentoring;
• Events and Teambuildings.

What makes us different?

• Our organizational culture is based on values such as transparency, team spirit, regular feedback and continuous learning;
• Reference prizes and birthday present;
• We don’t have a dress code;
• We know your name and all address each other informally;
• You will grow in good company. Isn’t this one of the ways in which to develop professionally?

If we got your attention, we now hope to spark your enthusiasm. Ready for the first phase?

We are waiting for you at concha. r. conceicao@aubay. pt!

Personal Data collected by AUBAY PORTUGAL as Data Controller, regarding the identification, contact and professional capabilities of the candidates, in the present procedure, will be processed for the application analysis, having in consideration that the processing is conducted in order to to take the steps, at the request of the data subject, prior to entering into a contract;

Personal data sent through the present application will be kept by Aubay during the application procedure and will remain stored whilst updated (for two years period).

The collected data might be transferred or include other processing actitivites, as long as the data subject is informed, in the occurrence of a standard application and selection procedure, for the compliance of legal obligations for the time required by law.

For any queries or request regarding personal data protection rights, the Data Subject may contact AUBAY PORTUGAL’s headquarters in Avenida Duque D`Ávila, nº46 em Lisboa or use the DPO’s
- mail - dpo@aubay. pt. The Data Subject also has the right to file a complaint with the National Data Protection Authority – www.cnpd. pt.

(please do not include data regarding racial or ethnical origins, political opinions, religion, philosophy, biometrical data, health data, sexual life and orientation, or relating to o criminal convictions and offences. )