Lead Data Engineer Porto

Decskill, founded in 2014 as an IT Consulting Company, places paramount importance on its greatest asset: its people. Our main mission is to deliver value through knowledge and talent, and we achieve this by fostering a culture of excellence and investing in the development and
- being of our people. With over 600 dedicated professionals and offices in Lisbon, Porto, Madrid, and Luxembourg, Decskill operates across three core areas:

• Decskill Talent: We believe that our people are key to our success. Through Decskill Talent, we empower our team to embrace the digital transformation challenges of our clients. We collaborate with clients to drive innovation, ensuring project success and business growth.
• Decskill Boost: Equipping our team with the latest tools and methodologies, we optimize Time-to-Market and deliver innovative solutions exceeding client expectations.
• Decskill Connect: Our team collaborates closely with clients to implement and manage IT infrastructures that generate
  - term value.

Lead Data Engineer

We are looking for a Lead Data Engineer for a hybrid project based in Porto e Lisboa (2x/Week).

Responsibilities

• Participate as a technical referent in developing code for processing and analyzing large amounts of data (e. g. unstructured tracking data, large
  - text data sets, graph data, etc. ).
• Design and implement highly scalable batch or
  - time data processing workflows/components and products that make smart use of data to provide
  - driven functionality (e. g. search services, recommender services, classification services, etc. ), leveraging machine learning, Big Data technology, and distributed systems.
• Build, monitor and operate services delivery within the Natixis infrastructure, while enhancing data quality/utility and automating processes.
• Deploy solutions on cloud technologies, mainly GCP (Google Cloud Platform)
• Work in a
  - functional team and collaborate closely with Data Scientists and Analysts to support the (technical) design, implementation and evaluation of new algorithms, machine learning models and other
  - driven features and services.
• Guide and support team members through technical issues and challenges related to data engineering and the uses tech stack.

Requirements

• Degree in Information Technology or Computer Engineering, Computer Science or a related field.
• Between 3 to 9 years of experience in designing and implementing scalable software and
  - driven services using Big Data technologies, data storages (e. g. relational databases,
  - value stores), data warehouses, data lakes, data lakehouses and related data processing technologies, as well as programming languages such as Spark / Python / Java / Scala (for
  - time and batch processing).
• Experience in building and optimizing data pipelines that process large datasets using Spark and cloud computing environment.
• Experience with working on cloud environments especially with data related services, for example Big
  Query, Big
  Table, Dataproc, GCS, Composer, GKE on Google Cloud Platform and
  - code/no-code (e. g. YAML) approaches for ETL tools.
• Experience in Business Intelligence and SQL utilization on analyzing large datasets.
• Experience with workflow management platforms (like Airflow) and collaborative platforms (like Jupyter)
• Some experience in the domain of information retrieval, information extraction or machine learning/AI concepts and technologies is a plus.
• Good knowledge of Agile methodologies (SAFe / Scrum).
• In-depth understanding of software development processes, CI/CD methodologies, Devops overview and quality assurance practices. (e. g. XLRelease/XLDeploy, Terraform, Jenkins, GIT, Bitbucket, etc)
• Microsoft Azure and Power
  BI is a plus.
• Good oral and written English (min B2).

If you’re interested in this job please send your CV to carolina. novais@decskill.com with reference CN/LDE.

Decskill is committed to equality and
- discrimination with all our talents. We recruit and promote talent, based on diversity and inclusion, regardless of age, gender, ethnicity, race, nationality or any other form of discrimination incompatible with the dignity of the human being.