Data Engineer Lisboa

Data Engineer
Lisboa
Lisboa, Lisboa, Portugal

4 days ago Be among the first 25 applicants

Get AI-powered advice on this job and more exclusive features.

Company Description

Inetum is a European leader in digital services. Inetum’s team of 28, 000 consultants and specialists strive every day to make a digital impact for businesses, public sector entities, and society. Inetum’s solutions aim at contributing to its clients’ performance and innovation as well as the common good. Present in 19 countries with a dense network of sites, Inetum partners with major software publishers to meet the challenges of digital transformation with proximity and flexibility. Driven by its ambition for growth and scale, Inetum generated sales of 2. 5 billion euros in 2023.

Job Description

We are looking for a Data Engineer responsible for building and maintaining the infrastructure supporting the organization’s data architecture. The role involves creating and managing data pipelines using Airflow for data extraction, processing, and loading, ensuring their maintenance, monitoring, and stability. The engineer will work closely with data analysts and
- users to provide accessible and reliable data.

Main Tasks and Responsibilities

• Maintain the infrastructure supporting the current data architecture.
• Create data pipelines in Airflow for data extraction, processing, and loading.
• Maintain, monitor, and ensure the stability of data pipelines.
• Provide data access to data analysts and
  - users.
• Manage Dev
  Ops infrastructure.
• Deploy Airflow DAGs to production using Dev
  Ops tools.
• Optimize code and queries.
• Review code.
• Improve current data architecture and Dev
  Ops processes.
• Deliver data in useful and appealing ways to users.
• Perform, document, and review analysis on regulatory topics.
• Understand business changes and requirements, assess impact and costs.

Qualifications

Technical Skills:

• Advanced Python (Mandatory)
• Experience creating APIs in Python, especially Flask (Mandatory)
• Experience in documenting and testing in Python (Mandatory)
• Advanced SQL skills and relational database management (Oracle mandatory, SQL Server and Postgre
  SQL desirable)
• Experience with Data Warehouses
• Hadoop ecosystem - HDFS + Yarn (Mandatory)
• Spark Environment Architecture (Mandatory)
• Advanced Py
  Spark (Mandatory)
• Experience in distributed environments using Hadoop and Spark
• Data Lakes management (Mandatory), S3 preferred
• Experience with Parquet format (Mandatory), Avro a plus
• Apache Airflow experience in pipeline development and deployment (Mandatory)
• Containerization with Docker (Mandatory)
• Kubernetes (Mandatory)
• Apache Kafka (Mandatory)
• Automation of deployment using Jenkins (Mandatory), Ansible a plus
• Agile methodologies, at least SCRUM

Language Skills:

• Fluent in English (Mandatory)

Seniority level

• Mid-Senior level

Employment type

• Full-time

Job function

• Analyst

Industries

• IT Services and IT Consulting