Data Engineer (Cloudera) Lisboa

Data Engineer (Cloudera)
Lisboa
Lisboa, Lisboa, Portugal

With just over 7 years of experience in the Portuguese market, we share our DNA with more than 200 workers and position our offer according to 3 lines of service:

• Strategy (Outsourcing, Ne
  Xel, Team as a Service, Tech Academies);
  
  
• R&D (Bootcamps, POC, Tech Lab);
  
  
• Nearshore.

In ADENTIS, we focus on PEOPLE. This is our emotional salary:

• Great Work-Life balance;
  
  
• Very flexible organizational routine;
  
  
• Health benefits(for you and your family);
  
  
• Team Buildingactivities;
  
  
• Over 300 protocols to offer you great discounts in different areas;
  
  
• Continuous professional development sponsored by our Training and Certification Department;
  
  
• Regular feedback on your performance through a personalized plan;
  
  
• Comprehensive career plan and progression involving assertive performance reviews.

Responsibilities:

- Distributed Data Processing:

Develop and optimize distributed data pipelines using Apache Spark, ensuring efficient processing of large datasets

- Caching & Computation Optimization:

Implement advanced caching techniques and optimize distributed computation workflows to improve data retrieval times and resource utilization

- Partition Key Join Optimization:

Optimize partition keys joins to speed up query processing and enhance overall system performance in distributed environments

- SQL Query Optimization:

Optimize SQL queries for better performance and scalability, particularly in
- scale, distributed database systems

- Cloudera Ecosystem:

If applicable, leverage the Cloudera platform to manage and scale data workflows in big data ecosystems, ensuring reliability and efficiency

- Wants to learn more about Dev
Ops to be autonomous, with the following Dev
Ops responsibilities (5-10% of the time)

Requirements:

- Expertise in Apache Spark and distributed computation techniques

- Experience with advanced SQL optimization and caching strategies

- Knowledge of the Cloudera ecosystem is a valuable asset for managing big data platforms

- Ability to work independently and take initiative to solve technical problems

- Excellent communication skills to effectively collaborate with technical and
- technical teams.

- CI/CD Pipeline Automation

- Version Control & Branching

- Deployment Automation

- Artifact Management

- Containerization & Orchestration

- Automation & Monitoring

- Available to go to the office twice per week

If you want to be part of a real People First organization, join us! Send your CV to andre. lago@adentis. pt