Data Engineer Viseu

Data Engineer
Viseu
Viseu, Viseu District, Portugal

About Us

BGTS is a software and technology solutions company with over 1, 800 professionals and 25+ years of experience. Through engineering expertise and industry insight, our international offices deliver tailored solutions, enabling clients worldwide to achieve their business goals with flexibility, speed, and impactful results.

What will you be doing

As a Data Engineer for Marketing Tech you will be working on a project to help us onboard and integrate some additional
- related technologies into our existing data stack, ensuring that data can be transmitted and received in a timely and reliable manner between these systems and our existing Datalake and Operational Data services. This role will work with other software and backend engineers, as well as our existing team of data engineers to plan out, design, and implement the solution to allow these solutions to function, connecting and exchanging data with our existing systems.

Requirements

We expect candidates to be familiar with common Data technologies from both
- source and cloud vendors such as Apache, Confluent, AWS, and GCP. Additionally, they should be familiar with general Data engineering practices, such as batch and streaming ETLs, schema management, data reconciliations / validations, and PII data handling.

Ideally, the candidate would have existing knowledge and working experience with
- related services such as Google Analytics, Mo
Engage, Mix
Panel, etc. and/or be capable of quickly undestanding documentation and handling REST APIs from such services and setting up new microservice endpoints to serve related data in a reliable and scalable manner.

Lastly, the candidate should be comfortable with managing responsibilities with a reasonable degree of independence, using their
- solving skills to handle routine issues as well as their communication skills to raise larger concerns as appropriate. As such, having a clear logical framework for
- solving and fluency in Business English is a must.

Relevant Technologies / Skillsets

• General computing concepts and expertise:
  
  Unix environments, networking, REST APIs
• Cloud computing data technologies:
  
  AWS S3, Athena, Dynamo
  DB, GCP Big
  Query, Pub/Sub
• Python frameworks and tooling:
  
  pip, pytest, boto3,
  -
  - python, pandas, polars, fastapi, requests
• Workflow scheduling tools:
  
  Airflow, dbt, Dagster
• Container management and orchestration:
  
  Docker, Kubernetes
• CI / CD tools:
  
  Circle
  CI, Jenkins, Spinnaker, Argo
  CD, Github CI
• Distributed messaging and event streaming systems:
  
  Kafka
• Enterprise BI tools:
  
  Tableau, Looker, Superset, Power
  BI, Quicksight
• Streaming data processing frameworks:
  
  Spark Streaming, Apache Beam, Apache Flink
• Infrastructure as Code:
  
  Terraform, Open
  Tofu, Terragrunt
• Log ingestion and monitoring:
  
  ELK stack (Elasticsearch, Logstash, Kibana), Datadog, Prometheus, Grafana
• Version control:
  
  git commands, branching strategies, collaboration etiquette, documentation best practices
• Agile/Lean project methodologies and rituals:
  
  Scrum, Kanban
• Data privacy and security tools and concepts:
  
  Tokenization, Hashing and encryption algorithms, Apache Ranger