Data Engineer Tech Lead Porto

Data Engineer Tech Lead
Porto
Porto, Porto, Portugal

• Degree in Information Technology or Computer Engineering, Computer Science or a related field.
• Between 3 to 9 years of experience in designing and implementing scalable software and
  - driven services using Big Data technologies, data storages (e. g. relational databases,
  - value stores), data warehouses, data lakes, data lakehouses and related data processing technologies, as well as programming languages such as Spark / Python / Java / Scala (for
  - time and batch processing).
• Experience in building and optimizing data pipelines that process large datasets using Spark and cloud computing environment.
• Experience with working on cloud environments especially with data related services, for example Big
  Query, Big
  Table, Dataproc, GCS, Composer, GKE on Google Cloud Platform and
  - code/no-code (e. g YAML) approaches for ETL tools.
• Experience in Business Intelligence and SQL utilization on analyzing large datasets.
• Experience with workflow management platforms (like Airflow) and collaborative platforms (like Jupyter)
• Some experience in the domain of information retrieval, information extraction or machine learning/AI concepts and technologies is a plus.
• Good knowledge of Agile methodologies (SAFe / Scrum).
• In-depth understanding of software development processes, CI/CD methodologies, Devops overview and quality assurance practices. (e. g. XLRelease/XLDeploy, Terraform, Jenkins, GIT, Bitbucket, etc)
• Microsoft Azure and Power
  BI is a plus.
• Familiarity with AI concepts and technologies is a plus but not a primary focus of the role.