Data Engineer (Cloudera) Setúbal

Data Engineer (Cloudera)
Setúbal
Setúbal, Setubal, Portugal

With just over 7 years of experience in the Portuguese market, we share our DNA with more than 200 workers and position our offer according to 3 lines of service:Strategy (Outsourcing, Ne
Xel, Team as a Service, Tech Academies);R&D (Bootcamps, POC, Tech Lab);Nearshore. In ADENTIS, we focus on PEOPLE. This is our emotional salary:Great Work-Life balance;Very flexible organizational routine;Health benefits (for you and your family);Team Building activities;Over 300 protocols to offer you great discounts in different areas;Continuous professional development sponsored by our Training and Certification Department;Regular feedback on your performance through a personalized plan;Comprehensive career plan and progression involving assertive performance reviews. Responsibilities:• Distributed Data Processing: Develop and optimize distributed data pipelines using Apache Spark, ensuring efficient processing of large datasets• Caching & Computation Optimization: Implement advanced caching techniques and optimize distributed computation workflows to improve data retrieval times and resource utilization• Partition Key Join Optimization: Optimize partition keys joins to speed up query processing and enhance overall system performance in distributed environments• SQL Query Optimization: Optimize SQL queries for better performance and scalability, particularly in
- scale, distributed database systems• Cloudera Ecosystem: If applicable, leverage the Cloudera platform to manage and scale data workflows in big data ecosystems, ensuring reliability and efficiency• Wants to learn more about Dev
Ops to be autonomous, with the following Dev
Ops responsibilities (5-10% of the time)Requirements:• Expertise in Apache Spark and distributed computation techniques• Experience with advanced SQL optimization and caching strategies• Knowledge of the Cloudera ecosystem is a valuable asset for managing big data platforms• Ability to work independently and take initiative to solve technical problems• Excellent communication skills to effectively collaborate with technical and
- technical teams. • CI/CD Pipeline Automation• Version Control & Branching• Deployment Automation• Artifact Management• Containerization & Orchestration• Automation & Monitoring• Available to go to the office twice per week
If you want to be part of a real People First organization, join us! Send your CV to andre. lago@adentis. pt