Principal Data Engineer
Skills
Principal data engineer. Skills in pipeline engineering, data architecture and modelling, business intelligence, data governance and AI systems.
Data Pipeline Engineering
- Python
- SQL
- ELT Architecture
- Apache Kafka
- Apache Spark
- Apache Airflow
- pandas
- SQLAlchemy
- GitHub Actions
- Idempotent Loading
- Dependency-Gated Pipelines
Data Architecture & Modelling
- PostgreSQL
- DuckDB
- BigQuery
- MySQL
- NoSQL
- Schema Normalisation
- Master Data Management (MDM)
- Query Optimisation
- Partitioned Tables
- Offline-First Architecture
Business Intelligence & Analytics
- Power BI
- Tableau
- Statistical Modelling
- KPI Dashboard Design
- ydata-profiling
- scikit-learn
- NumPy
- scipy
- Linear Algebra
- Probability & Statistics
Data Quality & Governance
- Privacy-First Data Governance
- Data Health Auditing
- Deterministic Deduplication
- Heuristic Field Mapping
- Regular Expressions
- Cryptographic Session Management
AI Systems & Development
- Prompt Engineering
- RAG Pipeline Concepts
- Generative AI Workflows
- Chain-of-Thought Prompting
- Flask
- Django
- Testing Frameworks