Data Engineer Lisboa

Data Engineer
Lisboa
Lisboa, Lisboa, Portugal

About Us

BGTS is a software and technology solutions company with over 1, 800 professionals and 25+ years of experience. Through engineering expertise and industry insight, our international offices deliver tailored solutions, enabling clients worldwide to achieve their business goals with flexibility, speed, and impactful results.

What will you be doing

As a Data Engineer for Marketing Tech you will be working on a project to help us onboard and integrate some additional
- related technologies into our existing data stack, ensuring that data can be transmitted and received in a timely and reliable manner between these systems and our existing Datalake and Operational Data services. This role will work with other software and backend engineers, as well as our existing team of data engineers to plan out, design, and implement the solution to allow these solutions to function, connecting and exchanging data with our existing systems.

Requirements

We expect candidates to be familiar with common Data technologies from both
- source and cloud vendors such as Apache, Confluent, AWS, and GCP. Additionally, they should be familiar with general Data engineering practices, such as batch and streaming ETLs, schema management, data reconciliations / validations, and PII data handling.

Ideally, the candidate would have existing knowledge and working experience with
- related services such as Google Analytics, Mo
Engage, Mix
Panel, etc. and/or be capable of quickly undestanding documentation and handling REST APIs from such services and setting up new microservice endpoints to serve related data in a reliable and scalable manner.

Lastly, the candidate should be comfortable with managing responsibilities with a reasonable degree of independence, using their
- solving skills to handle routine issues as well as their communication skills to raise larger concerns as appropriate. As such, having a clear logical framework for
- solving and fluency in Business English is a must.

Relevant Technologies / Skillsets

General computing concepts and expertise:

Unix environments, networking, REST APIs

Cloud computing data technologies:

AWS S3, Athena, Dynamo
DB, GCP Big
Query, Pub/Sub

Python frameworks and tooling:

pip, pytest, boto3,
-
- python, pandas, polars, fastapi, requests

Workflow scheduling tools:

Airflow, dbt, Dagster

Container management and orchestration:

Docker, Kubernetes

CI / CD tools:

Circle
CI, Jenkins, Spinnaker, Argo
CD, Github CI

Distributed messaging and event streaming systems:

Kafka

Enterprise BI tools:

Tableau, Looker, Superset, Power
BI, Quicksight

Streaming data processing frameworks:

Spark Streaming, Apache Beam, Apache Flink

Infrastructure as Code:

Terraform, Open
Tofu, Terragrunt

Log ingestion and monitoring:

ELK stack (Elasticsearch, Logstash, Kibana), Datadog, Prometheus, Grafana

Version control:

git commands, branching strategies, collaboration etiquette, documentation best practices

Agile/Lean project methodologies and rituals:

Scrum, Kanban

Data privacy and security tools and concepts:

Tokenization, Hashing and encryption algorithms, Apache Ranger