# CV Principal data engineer. The full CV of Nauman Shahid: experience, skills, education and selected projects in data engineering and architecture. Canonical page: https://www.nauman.cc/cv/ ## Profile Principal Data Engineer Abu Dhabi, UAE ELT pipelines, data architecture and decision-ready business intelligence for enterprise and SME clients. I turn fragmented data into systems people can use, trust and build on. Over more than a decade, that has meant engineering ELT pipelines, reconciling multi-source schemas and delivering business intelligence for enterprise and SME clients. The aim is practical: less manual compilation, clearer decisions and an infrastructure whose behaviour can be explained. The portfolio puts that approach in your hands. Explore a desktop and mobile workspace, inspect a file-system kernel, or watch a Tetris autopilot weigh possible moves. Alongside it are evidence-preserving data pipelines, archive readers and contact tools that keep the source intact while making it useful. I work from first principles and choose dependencies deliberately, with privacy, recovery and control built into the design. My next direction is AI systems architecture: retrieval and intelligent workflows that can be evaluated and audited. The foundation is the same engineering discipline, now applied to systems whose outputs need more than a confident explanation. If you are building data infrastructure or hiring someone who can connect architecture, implementation and business outcomes, let’s talk. ## Languages - English (Fluent) - Hindi (Expert) - Punjabi (Expert) - Urdu (Fluent) ## Principal Data Engineer, MindFlex Jan 2026 – Present B2B data engineering consultancy delivering Python ELT pipelines, PostgreSQL and DuckDB analytical layers, Power BI dashboards, and Data Health Audits to UAE SME clients in F&B, logistics, and property. - ELT Architecture: Architect and deliver Python ELT pipelines consolidating 4-6 fragmented SME data sources into centralised PostgreSQL and DuckDB analytical layers, eliminating an average of 12 hours of weekly manual compilation per client engagement. - Data Quality: Execute Data Health Audits using ydata-profiling to systematically identify and remediate structural data quality issues, resolving schema anomalies and duplication errors affecting 30-40% of ingested historical records per dataset. - Business Intelligence: Design and deliver sector-specific Power BI dashboards processing 150,000+ monthly transaction records across the active client portfolio, synthesising fragmented POS, inventory, and accounting data into automated executive intelligence. ## Data Systems & Operations Manager, Private Family Office Apr 2022 – Oct 2025 Designed and operated proprietary offline-first data systems for a private multi-territory executive operation across 3.5 years, maintaining complete data sovereignty with zero third-party cloud dependencies. - Information Retrieval: Built a self-hosted reader for exported operational chat archives that parses iOS and Android exports without guessing a date or dropping a line, joins overlapping exports with line-level provenance, and opens 500,000 messages in under 5 seconds with full-history search in under half a second. - Contact Data Integrity: Built a loss-preserving contact data platform for VCF and CSV libraries that patches only edited properties into the original bytes, searches 100,000 contacts in the browser, and turns lost or concurrent writes into detected conflicts through revision-checked saves with exact backups. - Data Governance: Designed and enforced offline-first data governance standards across sensitive executive operations, maintaining complete data sovereignty with zero reliance on third-party cloud infrastructure. ## Academic Tutor, The University of Sydney Jul 2019 – Dec 2021 Delivered technical instruction to undergraduate cohorts in Object-Oriented Programming and Calculus, bridging abstract computational theory and applied engineering practice. - Technical Instruction: Translated abstract OOP and Calculus theory into applied algorithmic and data structure skills, building cohort-level competency in software design fundamentals. - Assessment Design: Designed rigorous analytical problem sets to evaluate data structure implementation and algorithmic efficiency, establishing systematic frameworks for technical competency assessment. - Code Review: Conducted structured code reviews providing actionable feedback on logic, efficiency, and correctness, training cohorts to write reliable and maintainable software against defined specifications. ## Business Analyst, Birdi Oct 2018 – Mar 2021 Served as the technical interface between engineering teams and enterprise stakeholders, contributing data architecture and financial modelling expertise to commercial submissions and product strategy. - Technical Architecture: Contributed technical architecture definitions and data migration feasibility assessments to a successful $1.2M NSW Spatial Services contract bid, translating complex spatial data requirements into a structured commercial framework. - Financial Modelling: Formulated data-driven financial models evaluating hardware integration costs and projected system loads, supporting pricing and scoping decisions across multiple contract submissions. - Market Analysis: Executed market data analysis and technical feasibility case studies to align core platform capabilities with enterprise spatial applications, directly informing product strategy across the fiscal year. ## Business Analyst Intern, Fleet Space Technologies Jun 2018 – Sep 2018 Directed technical proposal lifecycles and cross-functional coordination to support IoT infrastructure deployment across government and agricultural trial environments. - Technical Proposals: Authored the technical architecture sections of a Victorian Government IoT trial submission, synthesising hardware constraints, connectivity data, and deployment specifications into a coherent commercial proposal. - Technical Analysis: Conducted analysis of IoT connectivity constraints across four agricultural trial regions, quantifying Portal device deployment requirements and pricing models for the government submission. - Cross-functional Coordination: Coordinated technical information gathering across distributed engineering teams and managed liaison with a remote Philippines-based team, ensuring accurate specification alignment throughout the field trial. ## Python Developer Intern, NCSOFT West Jan 2017 – Jun 2017 Engineered Python-based data parsing middleware to extract, filter, and structure high-volume game server telemetry into analytically consumable datasets. - Data Parsing: Engineered Python parsing algorithms to extract and filter raw game server telemetry, transforming high-volume JSON event streams into structured, analytically consumable datasets. - Schema Design: Designed backend data schemas to standardise incoming telemetry structures, enabling reliable extraction and filtering of sensitive metadata from high-volume server event logs. - Pipeline Integrity: Deployed automated testing frameworks enforcing strict validation logic across real-time data pipelines, maintaining data integrity under sustained high-throughput ETL conditions. ## Digital Solutions Engineer, Creative Graphics Mar 2013 – Dec 2016 Managed the full technical lifecycle of business intelligence tools across a client portfolio spanning nearly four years, translating complex briefs into functional data architectures and executive reporting frameworks. - Data Visualisation: Transformed multi-year market research datasets into interactive dashboards and executive reporting frameworks, synthesising complex analytical outputs into stakeholder decision-making tools. - Data Architecture: Translated complex client briefs into functional data architectures, building digital products capable of ingesting, processing, and presenting dynamic operational datasets to stakeholder specification. - Stakeholder Strategy: Advised clients on data presentation strategy, defining reporting structures from undefined business requirements and aligning final deliverables to stakeholder decision-making frameworks. ## Data Pipeline Engineering - Python - SQL - ELT Architecture - Apache Kafka - Apache Spark - Apache Airflow - pandas - SQLAlchemy - GitHub Actions - Idempotent Loading - Dependency-Gated Pipelines ## Data Architecture & Modelling - PostgreSQL - DuckDB - BigQuery - MySQL - NoSQL - Schema Normalisation - Master Data Management (MDM) - Query Optimisation - Partitioned Tables - Offline-First Architecture ## Business Intelligence & Analytics - Power BI - Tableau - Statistical Modelling - KPI Dashboard Design - ydata-profiling - scikit-learn - NumPy - scipy - Linear Algebra - Probability & Statistics ## Data Quality & Governance - Privacy-First Data Governance - Data Health Auditing - Deterministic Deduplication - Heuristic Field Mapping - Regular Expressions - Cryptographic Session Management ## AI Systems & Development - Prompt Engineering - RAG Pipeline Concepts - Generative AI Workflows - Chain-of-Thought Prompting - Flask - Django - Testing Frameworks ## University of the People, USA Bachelor of Science in Computer Science In Progress Progress: 78/120 ## Tanglin Trust School, Singapore General Certificate of Education (Advanced Level) 2007 ## Database Engineer Professional Certificate Meta Jun 2026 Advanced SQL, PostgreSQL administration, query optimisation, physical data modelling, and API development with Django. ## Python Development Professional Certificate Microsoft May 2026 Advanced Python, Flask, Django, object-oriented design, testing frameworks, and production-grade backend application development. ## Business Intelligence Professional Certificate Google Mar 2026 BigQuery, Tableau, ETL pipeline design, BI dashboard delivery, data governance, and stakeholder reporting frameworks. ## Prompting Essentials Google Feb 2026 Advanced prompt engineering, chain-of-thought prompting, generative AI workflows, and RAG pipeline concepts. ## Data Science Professional Certificate IBM Feb 2026 Python, pandas, scikit-learn, statistical modelling, predictive machine learning, and the end-to-end data science lifecycle. ## Technology & Data Consulting Job Simulations Deloitte Dec 2025 Three simulation tracks covering cyber security, data analytics, and software engineering: Python ETL pipelines, log analysis, web application security, and Tableau data visualisation. ## F1 Analytics: Post-Race Engineering Analysis Platform Data Engineering & Analytics Python • PostgreSQL • FastF1 • pandas • SQLAlchemy Nov 2024 – Present What actually shaped the race, and which laps support that conclusion? F1 Analytics is a post-race engineering analysis platform being built around those questions. Its Race Debrief and Season Index are designed to bring pace, teammate comparison, tyre degradation and race execution into one navigable view, with the underlying evidence available to inspect. I have engineered the foundation around FastF1 timing, telemetry, weather, track status and race-control data, a season-partitioned PostgreSQL model, transactional loading and resumable backfills. The evidence layer preserves lap validity, purpose, physical-ahead traffic and source lineage, while separate eligibility rules determine what each analysis can use. The next stages connect that foundation to matched comparisons, effect estimates, uncertainty and the dashboard itself. The goal is a briefing that makes a race easier to understand and every analytical conclusion easier to defend. - [Project page](https://www.nauman.cc/projects/f1-analytics/) - [Website or repository](https://www.nauman.cc/projects/f1-analytics/) ## Covert Messenger: Concealed Private Messaging Platform Systems Architecture & Security JavaScript • PHP 8 • MySQL 8 • HMAC-SHA256 Mar 2026 – Present A private two-person chat concealed behind a complete Formula 1 dashboard and reached only through an entry nothing on the page shows. I engineered an ordered change feed in which every send, edit, deletion and reaction commits under one database lock and writes an event per changed message, so both clients converge by cursor without missing or reordering a change, verified under 40 concurrent sends against real MySQL. Retried sends return the stored message instead of a duplicate, 1 MiB chunked uploads are checked against a server-side manifest, and pending files stay visible only to their uploader. A stateless HMAC-SHA256 token bound to the device, a PBKDF2-hashed entry lockout and a session that leaves no chat data in the browser once the tab closes complete the design, verified by 104 PHP and 89 Chromium gate checks. - [Project page](https://www.nauman.cc/projects/covert-messenger/) - [Website or repository](https://github.com/nshah1d/covert-messenger) ## Curated Video Feed: Dual-Layer Vertical Video Platform Media Delivery & Reliability Engineering JavaScript • PHP 8 • MySQL • YouTube Data API v3 Apr 2026 – Present A full-screen vertical video feed in the style of TikTok and YouTube Shorts, built from 152 hand-picked YouTube channels: a public Formula 1 face and a second catalogue of twelve categories behind a hidden entry. I engineered a YouTube provider that reads each channel's newest 50 uploads and lets only public, embeddable, unrestricted and finished videos reach the feed, with a per-channel file cache that serves the last good answer for up to seven days, pauses the API for an hour after a quota error and falls back to the channel's Atom feed, so the feed keeps playing through YouTube outages. A feed engine picks channels under a cooldown and never repeats a video within a round, one YouTube player runs at a time and loops by itself, and refused videos are skipped automatically, verified by 70 PHP gate checks and by 61 Chromium and 11 WebKit checks playing real YouTube videos. - [Project page](https://www.nauman.cc/projects/curated-video-feed/) - [Website or repository](https://github.com/nshah1d/curated-video-feed) ## Discord History Exporter: Exact-Range Message Export Extension Data Extraction & Reliability Engineering JavaScript • Chromium Manifest V3 • Discord API v9 Aug 2026 – Present A browser extension that exports a Discord channel's history, direct messages included, over a range chosen to the second, to JSON, CSV or Markdown, using the logged-in account's own session with nothing to set up. I engineered range bounds as half-open snowflake intervals that cover both chosen seconds in full and tile exactly, and a history walk that pages backwards 100 messages at a time, stops at once on a refusal, retries transient failures after 1, 2, 4 and 8 seconds, waits out Discord's rate limits and never saves a truncated file under the full range's name. JSON keeps Discord's own message objects unchanged, CSV follows RFC 4180 with a spreadsheet formula guard, and Markdown renders a readable transcript with embed text. It asks the browser for no permissions beyond discord.com and holds the session token in memory for one export only, verified by 51 unit tests and a 44-check Chromium gate over a simulated Discord. - [Project page](https://www.nauman.cc/projects/discord-history-exporter/) - [Website or repository](https://github.com/nshah1d/discord-history-exporter) ## Portfolio Operating System: Local-First, Single-Source Browser Platform Systems Architecture & Product Engineering React 19 • TypeScript 5.8 • Vite 7 • Canvas 2D • Web APIs Nov 2025 – Present A portfolio you can operate: a windowed desktop and a touch-first phone home screen in the browser, where every surface runs on engineering of its own. I built the shell with eight-way window resizing, long-press dragging, spring-loaded folders and a paged layout engine that packs mixed-size widgets, carries overflow forward and shares one calculation between the drag preview and the drop. Underneath sits a parent-linked virtual file system with subtree deletion and orphan pruning, driven equally by a Finder-style browser and a Unix-style terminal with quote-aware parsing, path-aware tab completion, output redirection, live syntax colouring and a pager. The apps carry their own algorithms: a Markdown renderer written from scratch, scanline flood fill for drawing, time-zone offsets from the browser's internationalisation API, weather icons chosen by the most frequent daylight condition, and a calendar that merges personal events, public holidays, the Formula 1 season and the academic year. Every arcade game plays itself on their title screens, from an exhaustive Tetris placement search scored on holes, height and surface unevenness to shortest-angle target tracking in Asteroids. Storage that never throws checks every saved kernel value, request ownership stops a late answer overwriting a newer one, and one content source generates both the OS and its search-readable HTML and Markdown pages. - [Project page](https://www.nauman.cc/projects/portfolio-operating-system/) - [Website or repository](https://www.nauman.cc/) ## Contacts Hub: Loss-Preserving Contact Data Platform Data Integrity & Performance Engineering JavaScript • Web Workers • PHP 8.2 Aug 2025 – Present A self-hosted contact data platform for VCF and CSV libraries in which an edit never loses source data. I engineered a loss-preserving vCard 3.0/4.0 and RFC 4180 CSV engine that patches only the edited properties into the original bytes, keeping every other field, vendor property and line ending, verified by 152 unit tests. A Web Worker engine and a virtualised list take in-browser search to 100,000 contacts, loading them in under 7.5 seconds and answering searches in under 0.6 seconds. Every save passes through a revision-checked PHP 8.2 service with an exclusive lock, an exact pre-write backup, an atomic rename and an independent read-back, so a lost or concurrent write becomes a detected conflict instead of silent data loss. - [Project page](https://www.nauman.cc/projects/contacts-hub/) - [Website or repository](https://github.com/nshah1d/contacts-hub) ## WhatsApp Archive Viewer: Provenance-Preserving Chat Archive Platform Data Parsing & Provenance Engineering JavaScript • Web Workers • PHP 8.2 Feb 2025 – Present A self-hosted reader for WhatsApp text exports in which nothing is guessed and nothing is dropped. I engineered a strict parser for iOS and Android exports that fixes each file to its own header shape, stops on ambiguous day and month dates instead of guessing, and keeps every unplaceable line as a line-numbered diagnostic, verified by 98 unit tests. Overlapping exports join by exact occurrence, so a message repeated inside one export survives and every record keeps its source lines. A Web Worker engine and a bounded message window load 500,000 messages in under 5 seconds and search the full history in under half a second, while a private PHP 8.2 service delivers transcripts and media only through authenticated, link-safe routes. - [Project page](https://www.nauman.cc/projects/whatsapp-archive-viewer/) - [Website or repository](https://github.com/nshah1d/whatsapp-archive-viewer) ## Contact Nauman Shahid Principal Data Engineer - [contact@nauman.cc](mailto:contact@nauman.cc) - [GitHub](https://github.com/nshah1d) - [LinkedIn](https://www.linkedin.com/in/nshah1d/) - [CV (PDF)](https://www.nauman.cc/resources/Nauman%20Shahid_Resume.pdf) - [Contact card (vCard)](https://www.nauman.cc/resources/Nauman%20Shahid.vcf)