Offre fiable

Data Engineer - OnePyramid, Lagos

OnePyramid

Lagos, Nigeria CDD

Publiée il y a 2 mois · Expire dans 3 semaines

Partager :

Description du poste

An exciting opportunity has arisen for a motivated professional. About the Project:
  • You will work on a system that processes sensitive partner data locally on partner infrastructure (not in the cloud) before any transfer occurs. Security-first design is mandatory.
Key Responsibilities:
  • Build a Python-based application that reads structured exports (CSV, Excel) from partner databases
  • Generates consistent pseudonymous identifiers using HMAC-SHA256
  • Removes direct identifiers and generalizes quasi-identifiers
  • Detects residual sensitive information in free-text fields using NLP (SpaCy NER)
  • Handles real-world data quality issues safely and gracefully
  • Ensures no sensitive data leaks via logs, temp files, or memory
  • Test pseudonymous identifier consistency across multiple runs
  • Measure record linkage success and failure rates
  • Evaluate re-identification risk using statistical methods
  • Produce quantitative evidence suitable for research publication
  • Write clear technical specifications for anonymization methods
  • Deliver readable, well-commented code for external review
  • Create a methodology document enabling independent replication
Required Skills & Experience:
  • Python: Strong pandas skills; experience with large, messy datasets; performance optimization
  • Cryptography: Hashing algorithms, HMAC, secure key management
  • Databases: PostgreSQL or MongoDB
  • Cloud & Big Data: AWS, Azure, or GCP; Spark or Hive
  • NLP: SpaCy or similar for Named Entity Recognition (using existing models)
  • Security Mindset (Critical): Defensive coding practices, No logging of sensitive values and Awareness of data leakage risks (logs, memory, temp files)
  • Data Analysis & Communication
  • Statistical analysis and metrics
  • Jupyter notebooks and visualizations
  • Ability to explain technical concepts to non-technical stakeholders
  • Comfortable coordinating remotely with partner IT teams

Ce poste vous intéresse ?

Se connecter pour voir l'email

Pas encore inscrit ? Créer un compte gratuit