<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/">
  <channel>
    <title>Engineer — Data — Portfolio and notes</title>
    <link>https://data.engineer.company/</link>
    <description>Engineer ApS — software development &amp; IT consulting in Copenhagen, Denmark. A portfolio of proven engineering achievements across software, data, cloud and IT.</description>
    <language>en</language>
    <copyright>Copyright © 2025 – present · Engineer ApS</copyright>
    <generator>Hugo 0.166.0</generator>
    <docs>https://www.rssboard.org/rss-specification</docs>
    <ttl>60</ttl>
    <lastBuildDate>Sun, 13 Sep 2026 01:45:51 +0200</lastBuildDate>
    <atom:link href="https://data.engineer.company/index.xml" rel="self" type="application/rss+xml" />
    <image>
      <url>https://data.engineer.company/assets/icons/apple/apple-touch-icon-144x144.png</url>
      <title>Engineer — Data — Portfolio and notes</title>
      <link>https://data.engineer.company/</link>
    </image>
    <item>
      <title>Drove brand engagement and loyalty by 100% through market trend analysis and consumer behavior insights.</title>
      <link>https://data.engineer.company/portfolio/drove-brand-engagement-and-loyalty-by-100-through-2/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/drove-brand-engagement-and-loyalty-by-100-through-2/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Doubled brand engagement and loyalty (&#43;100%) using market-trend analysis and consumer-behaviour insight to win a returning audience.</description>
      <category domain="https://data.engineer.company/categories/">Brand &amp; Marketing</category>
      <category domain="https://data.engineer.company/categories/">Data Analytics</category>
      <category domain="https://data.engineer.company/categories/">Product &amp; Requirements</category>
      <category domain="https://data.engineer.company/categories/">Stakeholder &amp; Reporting</category>
      <category domain="https://data.engineer.company/services/">Brand, Marketing &amp; SEO</category>
      <category domain="https://data.engineer.company/services/">Data Analytics &amp; BI Dashboards</category>
      <category domain="https://data.engineer.company/services/">Product Strategy &amp; Requirements</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> As the visibility climbed, Engineer ApS was reaching more people — but the engagement was thin. People noticed and moved on; the early interest wasn&rsquo;t turning into relationships that lasted. Reach without engagement is just noise, and the brand was making noise more than connections.</p>
<p><strong>Task.</strong> The aim was to deepen the engagement and the loyalty, and to do it by actually looking at what the market and the audience were responding to rather than trusting gut instinct.</p>
<p><strong>Action.</strong> So the brand became data‑informed instead of intuition‑led. That meant looking at the market trends and how the audience actually behaved across the channels — not what anyone assumed they&rsquo;d like, but what they demonstrably engaged with. It surfaced the topics and formats that drew real attention, and the content and outreach got steered toward those. The important part was tightening the loop: watch what landed, publish more of that shape next time, and let each cycle be a bit better‑aimed than the last.</p>
<p><strong>Result.</strong> Engagement and loyalty doubled — a 100% improvement — with an audience that came back and engaged rather than glancing once and leaving, and noticeably stronger relationships with prospects and partners. The brand stopped broadcasting into the void and started building something that returned.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Designed a comprehensive infrastructure framework for DTU impacting 14 departments, featuring flexible modules, unified data pipelines, and structured support strategies for long‑term adoption.</title>
      <link>https://data.engineer.company/portfolio/designed-a-comprehensive-infrastructure-framework-for-dtu-impacting-3/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/designed-a-comprehensive-infrastructure-framework-for-dtu-impacting-3/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Designed a comprehensive infrastructure framework for DTU spanning 14 departments — modular, with unified data pipelines and support strategies.</description>
      <category domain="https://data.engineer.company/categories/">Data Engineering</category>
      <category domain="https://data.engineer.company/categories/">Data Governance</category>
      <category domain="https://data.engineer.company/categories/">Data Pipelines (ETL/ELT)</category>
      <category domain="https://data.engineer.company/categories/">Documentation</category>
      <category domain="https://data.engineer.company/categories/">Infrastructure</category>
      <category domain="https://data.engineer.company/categories/">Platform Architecture</category>
      <category domain="https://data.engineer.company/categories/">Solution Architecture</category>
      <category domain="https://data.engineer.company/categories/">Stakeholder &amp; Reporting</category>
      <category domain="https://data.engineer.company/categories/">Technical Leadership</category>
      <category domain="https://data.engineer.company/services/">Data Governance &amp; Quality</category>
      <category domain="https://data.engineer.company/services/">Data Pipeline Development (ETL/ELT)</category>
      <category domain="https://data.engineer.company/services/">Platform &amp; Solution Architecture</category>
      <category domain="https://data.engineer.company/services/">Technical Documentation</category>
      <category domain="https://data.engineer.company/services/">Technical Leadership &amp; Consulting</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> At a research institute comprising 14 diverse research groups, each group worked with varying data sources, formats, scales, and software tools. The technical expertise and available IT resources varied widely across the groups. While a few had managed to create and deploy custom IT solutions, many struggled with the complexity of their data infrastructure needs, diverting valuable time and focus from their core research work.</p>
<p><strong>Task.</strong> The task was to devise a solution that would allow researchers to focus on their scientific work rather than IT challenges. The goal was to design and implement a scalable, institute‑wide data infrastructure that could accommodate the broad and differing requirements of the majority of research groups.</p>
<p><strong>Action.</strong> A robust, forward‑thinking infrastructure plan was developed that balanced flexibility and standardization. The plan outlined key components such as modular architecture, integration pathways for diverse data sources, user‑friendly interfaces tailored to varying technical skill levels, and scalable storage and processing solutions. It also included strategies for onboarding, support, and governance to ensure adoption and sustainability.</p>
<p><strong>Result.</strong> The resulting infrastructure plan was both technically sound and strategically aligned with the institute’s research goals. It unified the vision for data management across the organization, provided a clear path to reducing IT burden on researchers, and laid the foundation for a shared, efficient, and future‑ready research data environment. The plan was well received for its inclusivity, clarity, and adaptability, setting a strong direction for the institute&rsquo;s data infrastructure transformation.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Engineered hourly electricity consumption aggregation pipeline in Python / SQL / Bash &#43; Jq, achieving 180ms for 30‑day datasets across heterogeneous JSONL sources.</title>
      <link>https://data.engineer.company/portfolio/engineered-hourly-electricity-consumption-aggregation-pipeline-in-python-4/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/engineered-hourly-electricity-consumption-aggregation-pipeline-in-python-4/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Built a Python/SQL/Bash electricity-consumption pipeline aggregating 30-day datasets in 180ms across heterogeneous JSONL sources.</description>
      <category domain="https://data.engineer.company/categories/">Automation &amp; CI/CD</category>
      <category domain="https://data.engineer.company/categories/">Data Analytics</category>
      <category domain="https://data.engineer.company/categories/">Data Engineering</category>
      <category domain="https://data.engineer.company/categories/">Data Pipelines (ETL/ELT)</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">Performance Tuning</category>
      <category domain="https://data.engineer.company/categories/">PostgreSQL</category>
      <category domain="https://data.engineer.company/categories/">Python</category>
      <category domain="https://data.engineer.company/categories/">SQL</category>
      <category domain="https://data.engineer.company/services/">Data Analytics &amp; BI Dashboards</category>
      <category domain="https://data.engineer.company/services/">Data Pipeline Development (ETL/ELT)</category>
      <category domain="https://data.engineer.company/services/">Database Performance Tuning</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> The company processes large volumes of heterogeneous electricity data coming from multiple data sources, each with its own reporting frequency — ranging from hourly intervals to 15‑minute intervals, and in some cases, irregular timestamps. This variability poses a challenge when trying to create a coherent, comparable dataset. To support accurate energy analytics, this data needs to be normalized into consistent hourly consumption values, aggregated per zone.</p>
<p><strong>Task.</strong> The objective was to take raw event data from a historical dataset (in JSONL format) and convert it into hourly‑aligned electricity consumption figures, structured as one row per hour and per zone. Specifically, the task involved:</p>
<ul>
<li>Aligning timestamped data to strict hourly intervals,</li>
<li>Aggregating total electricity production within each interval by summing mix values,</li>
<li>Incorporating cross‑border exchanges by adding imports and subtracting exports,</li>
<li>Outputting the final values in a structured, scalable format suitable for further analysis.</li>
<li>The solution also needed to be efficient enough to scale for large time windows (30+ days) and across multiple countries/zones.</li>
</ul>
<p><strong>Action.</strong> To approach this task, three different variants of the solution were implemented using Python, JQ (for command‑line JSON processing), and SQL, each optimized for different contexts:</p>
<p>Python was chosen for its flexibility, ease of data manipulation, and ability to handle in‑memory transformations efficiently. Using pure Python functions, a pipeline was built that:</p>
<ul>
<li>Parsed the JSONL files into structured data frames</li>
<li>Resampled the time series data to hourly intervals</li>
<li>Aggregated production and calculated net electricity consumption (production + imports − exports)</li>
<li>Exported the results as CSV or loaded them into a lightweight SQLite database for inspection.</li>
</ul>
<p>JQ was used to create a quick, minimal‑dependency solution for command‑line environments, built as a JQ filter chain.</p>
<p>In PostgreSQL the JSONL data was imported, normalized tables created, and a series of SQL queries written.</p>
<p>To evaluate performance, each solution was benchmarked with both 1‑day and 30‑day historical datasets.</p>
<p><strong>Result.</strong> The Python implementation emerged as the fastest and most scalable solution, completing:</p>
<ul>
<li>1‑day data processing in just 34ms</li>
<li>30‑day dataset in 180ms</li>
</ul>
<p>This confirmed its suitability for handling larger timeframes while maintaining sub‑second performance. It also offered clear, maintainable code that could be easily extended to multi‑zone processing or integrated into an ETL pipeline.</p>
<p>Overall, the multi‑tool approach demonstrated flexibility in tool usage, strong performance optimization, and robust handling of time‑alignment and aggregation challenges in real‑world electricity data pipelines.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Accelerated geographical data pipeline performance by 50x by improving SQL programming and data modeling across PostgreSQL, MS SQL, and Google Cloud BigQuery.</title>
      <link>https://data.engineer.company/portfolio/accelerated-geographical-data-pipeline-performance-by-50x-by-5/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/accelerated-geographical-data-pipeline-performance-by-50x-by-5/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Accelerated a geographical data pipeline 50x through SQL and data-model tuning across PostgreSQL, MS SQL and Google BigQuery.</description>
      <category domain="https://data.engineer.company/categories/">Data Engineering</category>
      <category domain="https://data.engineer.company/categories/">Data Governance</category>
      <category domain="https://data.engineer.company/categories/">Data Pipelines (ETL/ELT)</category>
      <category domain="https://data.engineer.company/categories/">Data Warehousing</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">GIS / Geospatial</category>
      <category domain="https://data.engineer.company/categories/">Performance Tuning</category>
      <category domain="https://data.engineer.company/categories/">PostgreSQL</category>
      <category domain="https://data.engineer.company/categories/">SQL</category>
      <category domain="https://data.engineer.company/services/">Data Pipeline Development (ETL/ELT)</category>
      <category domain="https://data.engineer.company/services/">Data Warehouse Design</category>
      <category domain="https://data.engineer.company/services/">Database Performance Tuning</category>
      <category domain="https://data.engineer.company/services/">GIS &amp; Geospatial Solutions</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> The organization faced significant delays in processing large sets of geographical data, which affected the efficiency of data‑driven decision‑making processes and analytical reporting.</p>
<p><strong>Task.</strong> The objective was to optimize the geographical data pipeline to enhance performance and reduce processing time, ensuring that data analytics could be performed more effectively and in real‑time.</p>
<p><strong>Action.</strong> The existing SQL programming and data modeling structures across PostgreSQL, MS SQL, and Google Cloud BigQuery were meticulously analyzed, which surfaced bottlenecks related to inefficient indexing strategies, suboptimal query designs, and lack of appropriate constraints. To address these issues:</p>
<ul>
<li>The data models were redesigned to normalize critical datasets, with partitioning strategies implemented to improve data retrieval times.</li>
<li>Applied advanced indexing techniques, including B‑tree and GiST indexes for PostgreSQL, filtered indexes in MS SQL, and clustering in BigQuery.</li>
<li>Optimized complex SQL queries by refactoring subqueries, reducing join operations, and incorporating materialized views where relevant.</li>
<li>Established data integrity constraints such as foreign keys and check constraints to ensure data consistency without compromising performance.</li>
</ul>
<p><strong>Result.</strong> These comprehensive optimizations accelerated the geographical data pipeline performance by 50x, significantly reducing data processing time. This improvement facilitated real‑time analytics, enhanced reporting capabilities, and empowered stakeholders with timely, data‑driven insights, ultimately contributing to more informed strategic decisions.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Improved geographical map application performance by 10x through strategic database transition from MSSQL to PostgreSQL, optimizing processing and data security.</title>
      <link>https://data.engineer.company/portfolio/improved-geographical-map-application-performance-by-10x-through-6/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/improved-geographical-map-application-performance-by-10x-through-6/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Improved a GIS map application 10x by migrating MSSQL to PostgreSQL — spatial queries dropped from 2.5s to under 250ms.</description>
      <category domain="https://data.engineer.company/categories/">Backend Engineering</category>
      <category domain="https://data.engineer.company/categories/">Data Engineering</category>
      <category domain="https://data.engineer.company/categories/">Data Pipelines (ETL/ELT)</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">GIS / Geospatial</category>
      <category domain="https://data.engineer.company/categories/">Migrations &amp; Modernization</category>
      <category domain="https://data.engineer.company/categories/">Performance Tuning</category>
      <category domain="https://data.engineer.company/categories/">PostgreSQL</category>
      <category domain="https://data.engineer.company/categories/">Security</category>
      <category domain="https://data.engineer.company/categories/">SQL</category>
      <category domain="https://data.engineer.company/services/">Database Migration &amp; Modernization</category>
      <category domain="https://data.engineer.company/services/">Database Performance Tuning</category>
      <category domain="https://data.engineer.company/services/">GIS &amp; Geospatial Solutions</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> The organization’s geographical map application, which supported real‑time spatial queries for many users, was experiencing severe performance bottlenecks. Latency in rendering map layers and querying location‑based data impacted both user experience and backend service reliability. The system was backed by a legacy Microsoft SQL Server (MSSQL) database that lacked native geospatial indexing.</p>
<p><strong>Task.</strong> Improve the performance, scalability, and security of the map application, with a specific goal of reducing query latency and increasing throughput for spatial operations.</p>
<p><strong>Action.</strong> Led a strategic database migration from MSSQL to PostgreSQL with the PostGIS extension to enable native geospatial support.</p>
<ul>
<li>Designed a new schema optimized for spatial data, introducing GIST and SP‑GiST indexes on geometry and geography columns for faster querying.</li>
<li>Defined strict foreign key and check constraints to ensure relational integrity and enforce data validation rules on spatial coordinates.</li>
<li>Migrated over 50 million spatial records using ETL pipelines with data transformation steps to conform to new SRID standards (EPSG:4326).</li>
<li>Tuned PostgreSQL configuration parameters (e.g., work_mem, effective_cache_size) for optimal I/O performance under concurrent access.</li>
<li>Implemented role‑based access controls (RBAC) and row‑level security to enforce data protection policies across multiple user groups.</li>
</ul>
<p><strong>Result.</strong> Achieved a 10x improvement in spatial query performance, reducing average response time from 2.5 seconds to under 250 milliseconds. Backend CPU load dropped by 65%, and system availability improved during peak usage. Security posture was also strengthened with granular access policies and data validation constraints, reducing potential for spatial data corruption.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Resolved 1,000 issues in geographical data and time‑series data, using GDAL, ArcGIS, PostGIS, Mapbox, QGIS, SQL (PL/pgSQL, Transact‑SQL), Bash, ensuring high‑quality big data processing.</title>
      <link>https://data.engineer.company/portfolio/resolved-1-000-issues-in-geographical-data-and-7/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/resolved-1-000-issues-in-geographical-data-and-7/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Resolved 1,000&#43; issues in geographic and time-series big data with GDAL, PostGIS, ArcGIS, Mapbox, QGIS and SQL — 35% faster queries.</description>
      <category domain="https://data.engineer.company/categories/">Automation &amp; CI/CD</category>
      <category domain="https://data.engineer.company/categories/">Data Engineering</category>
      <category domain="https://data.engineer.company/categories/">Data Governance</category>
      <category domain="https://data.engineer.company/categories/">Data Pipelines (ETL/ELT)</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">GIS / Geospatial</category>
      <category domain="https://data.engineer.company/categories/">Performance Tuning</category>
      <category domain="https://data.engineer.company/categories/">PostgreSQL</category>
      <category domain="https://data.engineer.company/categories/">SQL</category>
      <category domain="https://data.engineer.company/services/">Data Governance &amp; Quality</category>
      <category domain="https://data.engineer.company/services/">Data Pipeline Development (ETL/ELT)</category>
      <category domain="https://data.engineer.company/services/">GIS &amp; Geospatial Solutions</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> While working on a large‑scale geospatial analytics project, the team encountered numerous inconsistencies and anomalies within the geographical and time‑series datasets. These issues were affecting the accuracy of spatial analyses and decision‑making tools used across several departments.</p>
<p><strong>Task.</strong> The responsibility was to identify, resolve, and optimize over 1,000 data quality issues within these complex datasets to ensure the integrity and performance of downstream applications and visualizations.</p>
<p><strong>Action.</strong> Spatial errors were systematically diagnosed and corrected using a combination of tools, including GDAL, QGIS, and ArcGIS, with automated workflows implemented in Bash scripting to streamline recurring data cleaning tasks. PostGIS handled advanced spatial queries and spatial indexing, and robust procedures were written in PL/pgSQL and Transact‑SQL to manage and transform both geographic and temporal data within the PostgreSQL and SQL Server databases. Additionally, the cleaned data was integrated into interactive visualizations using Mapbox, enhancing data accessibility for end users.</p>
<p><strong>Result.</strong> Through these efforts, over 1,000 critical issues were resolved, significantly improving data accuracy and processing speed. This directly contributed to a 35% reduction in spatial query run times and enabled more reliable spatial analyses for the team. The work ensured that high‑quality, ready‑to‑use data was consistently available for analytics and reporting, supporting strategic decisions across the organization.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Designed, implemented, and administered 6 ETL/ELT pipelines, utilizing Google BigQuery, MSSQL, PostgreSQL, Shell scripting, PL/pgSQL, and Transact‑SQL, integrating data for efficient Python API processing.</title>
      <link>https://data.engineer.company/portfolio/designed-implemented-and-administered-6-etl-elt-pipelines-8/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/designed-implemented-and-administered-6-etl-elt-pipelines-8/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Designed and ran 6 ETL/ELT pipelines (BigQuery, MSSQL, PostgreSQL), cutting a 3-hour manual job to under 20 minutes with daily syncs.</description>
      <category domain="https://data.engineer.company/categories/">Automation &amp; CI/CD</category>
      <category domain="https://data.engineer.company/categories/">Cloud</category>
      <category domain="https://data.engineer.company/categories/">Data Engineering</category>
      <category domain="https://data.engineer.company/categories/">Data Pipelines (ETL/ELT)</category>
      <category domain="https://data.engineer.company/categories/">Data Warehousing</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">Networking &amp; VPN</category>
      <category domain="https://data.engineer.company/categories/">PostgreSQL</category>
      <category domain="https://data.engineer.company/categories/">Python</category>
      <category domain="https://data.engineer.company/categories/">SQL</category>
      <category domain="https://data.engineer.company/services/">Backend &amp; API Development</category>
      <category domain="https://data.engineer.company/services/">Data Pipeline Development (ETL/ELT)</category>
      <category domain="https://data.engineer.company/services/">Data Warehouse Design</category>
      <category domain="https://data.engineer.company/services/">Database Design &amp; Modeling</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> The organization needed to consolidate and process large volumes of structured data from a remote data warehouse located behind an IPSec VPN. This data was crucial for powering internal analytics dashboards and external Python APIs used by clients and partners.</p>
<p><strong>Task.</strong> The objective was to design, implement, and maintain a set of robust and automated ETL/ELT pipelines to securely retrieve, transform, and load data into Google BigQuery, ensuring data accuracy, performance, and scalability across systems.</p>
<p><strong>Action.</strong> 6 end‑to‑end ETL/ELT pipelines were designed, implemented, and administered, spanning multiple technologies including Google BigQuery, MSSQL, PostgreSQL, Shell scripting, PL/pgSQL, and Transact‑SQL.</p>
<ul>
<li>Established secure connections to a remote server protected by an IPSec VPN to automate data retrieval.</li>
<li>Scheduled and executed downloads of compressed data archives containing Parquet, CSV, and .bak files.</li>
<li>Developed Bash and Python scripts to extract and classify files based on type and schema.</li>
<li>For .bak files, backups were restored into a local MSSQL Server instance using RESTORE DATABASE workflows and schema integrity validated.</li>
<li>Loaded structured data into staging schemas in MSSQL and PostgreSQL, using bcp, psql, and SSIS tools, depending on the source format.</li>
<li>Wrote modular and reusable PL/pgSQL and T‑SQL procedures to clean, normalize, and enrich the data based on business logic.</li>
<li>Exported curated datasets and tables into intermediate formats, compressed them using gzip, and securely transferred them to a Google Cloud Storage bucket.</li>
<li>Automated the upload and schema mapping processes to Google BigQuery, using bq CLI and Python‑based data ingestion scripts.</li>
</ul>
<p><strong>Result.</strong> These automated pipelines significantly reduced the manual overhead and processing time — from 3+ hours of manual work to under 20 minutes end‑to‑end, while improving data freshness from weekly to daily syncs. The Python APIs consuming this data saw a 30% performance improvement, and the enhanced visibility helped business analysts deliver faster insights to stakeholders. The solution remains scalable and extensible for onboarding new data sources as the business grows.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Developed and launched the company&#39;s first observability dashboard, providing real‑time system performance insights and data visualization on the large office TV.</title>
      <link>https://data.engineer.company/portfolio/developed-and-launched-the-company-s-first-observability-9/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/developed-and-launched-the-company-s-first-observability-9/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Built the company&#39;s first observability dashboard for real-time system insight, helping teams detect and resolve incidents 40% faster.</description>
      <category domain="https://data.engineer.company/categories/">Automation &amp; CI/CD</category>
      <category domain="https://data.engineer.company/categories/">Containers (Docker/Kubernetes)</category>
      <category domain="https://data.engineer.company/categories/">Data Analytics</category>
      <category domain="https://data.engineer.company/categories/">DevOps</category>
      <category domain="https://data.engineer.company/categories/">Infrastructure</category>
      <category domain="https://data.engineer.company/categories/">Linux &amp; Servers</category>
      <category domain="https://data.engineer.company/categories/">Monitoring &amp; Observability</category>
      <category domain="https://data.engineer.company/categories/">Reliability &amp; Backups</category>
      <category domain="https://data.engineer.company/services/">Data Analytics &amp; BI Dashboards</category>
      <category domain="https://data.engineer.company/services/">Site Reliability &amp; Monitoring</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> The company was facing challenges in monitoring real‑time system performance, which often led to delayed incident responses and reduced visibility into infrastructure health. There was no centralized solution in place for teams to gain insights into operational metrics.</p>
<p><strong>Task.</strong> The task was to develop a solution that would enable technical and non‑technical stakeholders to monitor key system metrics in real time, with a focus on accessibility, clarity, and proactive issue detection.</p>
<p><strong>Action.</strong> The company&rsquo;s first observability dashboard was designed and implemented, collecting all essential system metrics — such as CPU, RAM, HDD, temperature, and more — from remote Linux servers via SSH. Even Docker containers were monitored using this method. Later, a second version was designed and implemented using Grafana and Prometheus for more advanced visualization and monitoring capabilities. Collaboration with DevOps and engineering teams identified the critical metrics, such as CPU utilization, memory usage, service uptime, and API latency. Data pipelines were configured to ingest and process performance metrics from various systems, and the dashboard deployed on a large office TV screen for maximum visibility. Alerting mechanisms for threshold breaches were also integrated to enable immediate action.</p>
<p><strong>Result.</strong> The dashboard significantly improved system transparency and response time to operational issues. Teams were able to detect and resolve incidents 40% faster. It also fostered a culture of shared ownership over system health by making performance data accessible to everyone in the office, ultimately contributing to a more stable and efficient production environment.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Delivered 8 Power BI projects with comprehensive manuals, integrating Microsoft Power BI tools with NodeJS API and Python FastAPI for effective data analytics and visualization.</title>
      <link>https://data.engineer.company/portfolio/delivered-8-power-bi-projects-with-comprehensive-manuals-10/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/delivered-8-power-bi-projects-with-comprehensive-manuals-10/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Delivered 8 Power BI analytics projects (Node.js API, Python FastAPI) with full manuals, improving report generation efficiency by 60%.</description>
      <category domain="https://data.engineer.company/categories/">APIs &amp; Integration</category>
      <category domain="https://data.engineer.company/categories/">Backend Engineering</category>
      <category domain="https://data.engineer.company/categories/">Data Analytics</category>
      <category domain="https://data.engineer.company/categories/">Data Engineering</category>
      <category domain="https://data.engineer.company/categories/">Documentation</category>
      <category domain="https://data.engineer.company/categories/">GIS / Geospatial</category>
      <category domain="https://data.engineer.company/categories/">Product &amp; Requirements</category>
      <category domain="https://data.engineer.company/categories/">Python</category>
      <category domain="https://data.engineer.company/services/">Backend &amp; API Development</category>
      <category domain="https://data.engineer.company/services/">Data Analytics &amp; BI Dashboards</category>
      <category domain="https://data.engineer.company/services/">Technical Documentation</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> The organization needed dynamic, visual insights into complex datasets involving electrical grid performance and geographical distribution metrics to support decision‑making across technical and strategic teams.</p>
<p><strong>Task.</strong> Develop interactive dashboards and reporting solutions that could effectively present both real‑time and historical geographical and electrical data, enabling stakeholders to quickly identify trends, anomalies, and performance indicators.</p>
<p><strong>Action.</strong> Integrated Microsoft Power BI with a custom backend stack using NodeJS API and Python FastAPI to streamline data ingestion, transformation, and visualization. Designed and implemented dashboards with map visualizations, energy consumption metrics, outage tracking, and grid efficiency indicators. Developed reusable templates and detailed documentation to support scalability and ease of use.</p>
<p><strong>Result.</strong> Successfully delivered 8 data analytics projects, improving report generation efficiency by 60% and enabling cross‑functional teams to make faster, data‑driven decisions. Stakeholders reported a significant increase in understanding of regional electrical performance and resource planning accuracy.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Released 500 electricity and GIS data analysis reports, utilizing deep research and troubleshooting to ensure accurate geographic and time series big data insights.</title>
      <link>https://data.engineer.company/portfolio/released-500-electricity-and-gis-data-analysis-reports-11/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/released-500-electricity-and-gis-data-analysis-reports-11/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Published 500 electricity and GIS data-analysis reports — a cross-department reference for load balancing, efficiency and anomaly detection.</description>
      <category domain="https://data.engineer.company/categories/">Data Analytics</category>
      <category domain="https://data.engineer.company/categories/">Data Governance</category>
      <category domain="https://data.engineer.company/categories/">Data Warehousing</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">Documentation</category>
      <category domain="https://data.engineer.company/categories/">GIS / Geospatial</category>
      <category domain="https://data.engineer.company/categories/">PostgreSQL</category>
      <category domain="https://data.engineer.company/categories/">SQL</category>
      <category domain="https://data.engineer.company/categories/">Stakeholder &amp; Reporting</category>
      <category domain="https://data.engineer.company/services/">Data Analytics &amp; BI Dashboards</category>
      <category domain="https://data.engineer.company/services/">Data Governance &amp; Quality</category>
      <category domain="https://data.engineer.company/services/">GIS &amp; Geospatial Solutions</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> The team managed vast datasets generated by smart‑meters installed across multiple geographic regions. These smart‑meters produced granular, time‑series electrical consumption data used by energy analysts, engineers, and regional planners for operational and strategic decision‑making.</p>
<p><strong>Task.</strong> The responsibility was to produce high‑quality, transparent, and reproducible analytical reports that could uncover patterns in energy consumption, detect anomalies, and identify regional usage trends, while ensuring non‑technical stakeholders could easily interpret and reuse the findings.</p>
<p><strong>Action.</strong> 500+ in‑depth data analysis reports were created and delivered, using pure SQL to perform all data extraction, transformation, and analysis tasks, working directly within cloud‑based environments such as PostgreSQL and BigQuery. The data included geolocation coordinates, meter IDs, timestamped energy usage, and environmental metadata. The SQL scripts featured CTEs, window functions, subqueries, and geospatial joins, allowing for scalable and efficient processing.</p>
<p>Each report included annotated SQL code, enabling colleagues and collaborators to fully reproduce and audit the research, which significantly reduced the time needed for follow‑up analysis. Troubleshooting notes were also added and common data quality issues documented, such as missing GPS coordinates or corrupted meter values, with recommended handling procedures.</p>
<p><strong>Result.</strong> The reports became a standard reference across departments, aiding in regional load balancing, energy efficiency planning, and anomaly detection. By ensuring full transparency and reproducibility, the work helped improve stakeholder trust in the data and contributed to more accurate forecasting models and a 10–15% improvement in operational planning efficiency.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Architected, created, and managed 100 PostgreSQL, MS SQL, and Google BigQuery data warehouse databases with primarily GIS and time‑series data, optimizing performance and scalability.</title>
      <link>https://data.engineer.company/portfolio/architected-created-and-managed-100-postgresql-ms-sql-12/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/architected-created-and-managed-100-postgresql-ms-sql-12/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Architected and ran 100 PostgreSQL, MS SQL and BigQuery data-warehouse databases for GIS and time-series data — query times down 40–60%.</description>
      <category domain="https://data.engineer.company/categories/">Data Engineering</category>
      <category domain="https://data.engineer.company/categories/">Data Governance</category>
      <category domain="https://data.engineer.company/categories/">Data Warehousing</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">GIS / Geospatial</category>
      <category domain="https://data.engineer.company/categories/">Performance Tuning</category>
      <category domain="https://data.engineer.company/categories/">Platform Architecture</category>
      <category domain="https://data.engineer.company/categories/">PostgreSQL</category>
      <category domain="https://data.engineer.company/categories/">Reliability &amp; Backups</category>
      <category domain="https://data.engineer.company/categories/">SQL</category>
      <category domain="https://data.engineer.company/services/">Data Warehouse Design</category>
      <category domain="https://data.engineer.company/services/">Database Administration (DBA)</category>
      <category domain="https://data.engineer.company/services/">Database Design &amp; Modeling</category>
      <category domain="https://data.engineer.company/services/">Database Performance Tuning</category>
      <category domain="https://data.engineer.company/services/">GIS &amp; Geospatial Solutions</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> In a rapidly growing tech company, there was a critical need to create, manage, and optimize a diverse portfolio of 100+ databases across PostgreSQL, Microsoft SQL Server, and Google BigQuery. These databases primarily handled complex datasets, including geographic information systems (GIS) data (e.g., spatial coordinates, and location‑based analytics) and time‑series data (e.g., sensor readings, logs, and real‑time metrics). The existing infrastructure faced challenges with scalability, query performance, and data consistency, particularly as the volume of data increased exponentially. The organization required a robust architecture to ensure data reliability, and reduce operational costs.</p>
<p><strong>Task.</strong> The primary objective was to design, implement, and manage a scalable, high‑performance data warehouse ecosystem tailored for GIS and time‑series data. This involved:</p>
<ul>
<li>Addressing performance bottlenecks in complex spatial and time‑based queries.</li>
<li>Ensuring scalability to handle growing data volumes while maintaining cost efficiency.</li>
<li>Collaborating with cross‑functional teams (e.g., data scientists, product managers) to align database design with business needs.</li>
</ul>
<p><strong>Action.</strong> To achieve these goals:</p>
<ol>
<li>Designed Scalable Architectures:</li>
</ol>
<ul>
<li>Created normalized and denormalized schemas for PostgreSQL and SQL Server, leveraging spatial indexing (e.g., PostGIS for PostgreSQL) and time‑series partitioning to optimize query performance.</li>
<li>Utilized BigQuery’s time‑partitioned and clustered tables for efficient handling of large‑scale time‑series data.</li>
</ul>
<ol start="2">
<li>Implemented Optimization Strategies:</li>
</ol>
<ul>
<li>Introduced query optimization techniques, such as indexing, materialized views, and caching, to reduce latency for GIS and time‑series queries.</li>
<li>Applied data compression and columnar storage in BigQuery to minimize storage costs and improve scan speeds.</li>
</ul>
<ol start="3">
<li>Collaborated on Cross‑Platform Integration:</li>
</ol>
<ul>
<li>Documented best practices for GIS and time‑series data modeling to guide teams in future projects.</li>
</ul>
<p><strong>Result.</strong> The initiatives led to significant improvements:</p>
<ul>
<li>Performance Gains: Query response times for GIS and time‑series data decreased by 40–60%, enabling faster analytics and decision‑making.</li>
<li>Operational Reliability: Automated monitoring reduced downtime by 50%, while standardized processes improved team productivity and reduced errors.</li>
<li>Business Impact: The optimized infrastructure enabled the company to launch new data‑driven products (e.g., real‑time analytics dashboards) and meet regulatory compliance requirements for data governance.</li>
</ul>
<p>This work solidified the organization’s ability to handle complex data challenges.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Designed, deployed, and maintained 10 PostgreSQL and MS SQL servers on Ubuntu Linux VPS, ensuring optimal server performance and reliability.</title>
      <link>https://data.engineer.company/portfolio/designed-deployed-and-maintained-10-postgresql-and-ms-13/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/designed-deployed-and-maintained-10-postgresql-and-ms-13/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Designed and maintained 10 PostgreSQL and MS SQL servers on Ubuntu Linux VPS at 99.9% uptime, with 30% faster queries via tuning.</description>
      <category domain="https://data.engineer.company/categories/">Automation &amp; CI/CD</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">DevOps</category>
      <category domain="https://data.engineer.company/categories/">Infrastructure</category>
      <category domain="https://data.engineer.company/categories/">Linux &amp; Servers</category>
      <category domain="https://data.engineer.company/categories/">Monitoring &amp; Observability</category>
      <category domain="https://data.engineer.company/categories/">Performance Tuning</category>
      <category domain="https://data.engineer.company/categories/">PostgreSQL</category>
      <category domain="https://data.engineer.company/categories/">Reliability &amp; Backups</category>
      <category domain="https://data.engineer.company/categories/">Security</category>
      <category domain="https://data.engineer.company/categories/">System Administration</category>
      <category domain="https://data.engineer.company/services/">Database Administration (DBA)</category>
      <category domain="https://data.engineer.company/services/">Site Reliability &amp; Monitoring</category>
      <category domain="https://data.engineer.company/services/">System Administration</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> As a DevOps engineer at a mid‑sized tech company, the task was managing and optimizing database infrastructure to support a growing user base and critical business applications. The organization relied heavily on PostgreSQL and Microsoft SQL Server for data storage and analytics, requiring high availability, scalability, and security.</p>
<p><strong>Task.</strong> The primary responsibility was to architect, deploy, configure, and maintain 10 PostgreSQL and MS SQL Server instances on Ubuntu Linux VPS environments. This included ensuring optimal performance, implementing robust security protocols, and establishing proactive monitoring to prevent outages. Additionally, the task included scaling the infrastructure to accommodate future growth while minimizing costs and maintaining compliance with industry standards.</p>
<p><strong>Action.</strong> 1. Deployment &amp; Configuration:</p>
<ul>
<li>Installed and configured PostgreSQL 14 and MS SQL Server 2019 on Ubuntu 20.04 LTS VPS instances, ensuring compatibility with the company’s applications.</li>
<li>Set up automated backups using <code>pg_dump</code> for PostgreSQL and SQL Server Agent jobs for MS SQL, with retention policies and offsite storage.</li>
<li>Optimized server configurations (e.g., memory allocation, query caching, and connection pooling) to improve query performance and reduce latency.</li>
</ul>
<ol start="2">
<li>
<p>Monitoring &amp; Maintenance:</p>
<ul>
<li>Implemented monitoring tools like Prometheus, Grafana, to track CPU, memory, disk I/O, and query performance metrics in real time.</li>
<li>Conducted regular patching and updates for both databases and the Ubuntu OS to address security vulnerabilities and ensure compliance.</li>
<li>Created custom scripts for log analysis.</li>
</ul>
</li>
<li>
<p>Security &amp; Scalability:</p>
<ul>
<li>Configured firewalls (UFW), enforced role‑based access control (RBAC) to protect sensitive data.</li>
<li>Documented procedures for disaster recovery, including point‑in‑time restores and failover protocols.</li>
</ul>
</li>
</ol>
<p><strong>Result.</strong> - Achieved 99.9% uptime across all 10 database servers.</p>
<ul>
<li>Improved query response times by 30% through configuration tuning and index optimization, enhancing application performance.</li>
<li>Reduced manual maintenance tasks by 50% via automation, freeing up 10+ hours per month for strategic projects.</li>
<li>Successfully scaled the infrastructure to support a 40% increase in user traffic without service degradation, contributing to a 20% revenue growth in the following quarter.</li>
<li>Received recognition from the CTO for implementing security best practices that prevented potential data breaches.</li>
</ul>
<p>This experience solidified deep expertise in database management, DevOps automation, and infrastructure optimization, delivering reliable, secure, and scalable solutions for complex enterprise environments.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Enhanced data security by implementing 1,000 RBAC rules for developers, application instances, PostgreSQL, MS SQL, and other Linux servers, preventing unauthorized access; documented with Ansible automation.</title>
      <link>https://data.engineer.company/portfolio/enhanced-data-security-by-implementing-1-000-rbac-14/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/enhanced-data-security-by-implementing-1-000-rbac-14/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Hardened security with 1,000 RBAC rules across developers, apps and Linux/DB servers — unauthorised-access risk down 85%, Ansible-automated.</description>
      <category domain="https://data.engineer.company/categories/">Automation &amp; CI/CD</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">DevOps</category>
      <category domain="https://data.engineer.company/categories/">Documentation</category>
      <category domain="https://data.engineer.company/categories/">Infrastructure</category>
      <category domain="https://data.engineer.company/categories/">Linux &amp; Servers</category>
      <category domain="https://data.engineer.company/categories/">Security</category>
      <category domain="https://data.engineer.company/categories/">System Administration</category>
      <category domain="https://data.engineer.company/services/">Database Administration (DBA)</category>
      <category domain="https://data.engineer.company/services/">Infrastructure as Code</category>
      <category domain="https://data.engineer.company/services/">Security &amp; Access Management</category>
      <category domain="https://data.engineer.company/services/">System Administration</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> The organization needed to strengthen access controls across multiple systems, including developer environments, application instances, PostgreSQL, MS SQL, and Linux servers. While the underlying RBAC model was straightforward, the challenge lay in managing over 1,000 individual rules to cover diverse user roles and system requirements. The goal was to ensure strict access restrictions without introducing complexity.</p>
<p><strong>Task.</strong> Implement a scalable RBAC solution by defining and enforcing 1,000+ rules for access control. This involved mapping permissions to specific roles (e.g., developers, application instances, database admins) and ensuring rules were applied consistently across all systems. The task also required documenting the rules and automating their deployment to avoid manual errors.</p>
<p><strong>Action.</strong> The approach focused on creating a simple, modular RBAC structure, breaking down permissions into clear, reusable categories (e.g., &ldquo;read‑only access to production databases&rdquo;). Using Ansible, the configuration of each rule was automated, ensuring consistency across environments. For example, developers were granted access only to their designated servers, while application instances had limited permissions to prevent lateral movement. The process prioritized clarity over complexity, with each rule explicitly tied to a specific role and system.</p>
<p><strong>Result.</strong> The implementation secured over 1,000 rules without introducing unnecessary complexity, reducing unauthorized access risks by 85%. Automation streamlined deployment, cutting setup time by 60% compared to manual methods. The documented framework allowed teams to quickly audit or modify rules, ensuring scalability as the infrastructure grew. By focusing on simplicity and volume, the solution achieved robust security while maintaining operational efficiency.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Accelerated PostgreSQL performance by 10x via strategic indexing, partitioning, and query optimization, enhancing database efficiency for user, tenant, geospatial, and time‑series electrical data.</title>
      <link>https://data.engineer.company/portfolio/accelerated-postgresql-performance-by-10x-via-strategic-indexing-15/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/accelerated-postgresql-performance-by-10x-via-strategic-indexing-15/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Accelerated PostgreSQL 10x via indexing, partitioning and query tuning for user, tenant, geospatial and time-series data — resource use down 40%.</description>
      <category domain="https://data.engineer.company/categories/">Data Engineering</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">GIS / Geospatial</category>
      <category domain="https://data.engineer.company/categories/">Performance Tuning</category>
      <category domain="https://data.engineer.company/categories/">PostgreSQL</category>
      <category domain="https://data.engineer.company/categories/">SQL</category>
      <category domain="https://data.engineer.company/services/">Database Administration (DBA)</category>
      <category domain="https://data.engineer.company/services/">Database Performance Tuning</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> The PostgreSQL database served a mid‑sized application managing user accounts, tenant information, geographical data (e.g., locations, regions), and daily electrical consumption metrics. While the system operated under low traffic, performance issues emerged as data volumes grew. Queries involving geospatial data and time‑series electrical logs became sluggish, leading to inconsistent response times. The lack of optimized indexing, fragmented queries, and unpartitioned tables exacerbated the problem, creating bottlenecks for critical operations like user authentication, tenant management, and data reporting.</p>
<p><strong>Task.</strong> The goal was to enhance database performance by 10x without overhauling the existing architecture. The focus was on optimizing query execution, reducing latency, and ensuring scalability for future data growth. Key priorities included improving response times for geospatial and time‑based queries, minimizing resource contention, and maintaining data integrity while implementing changes.</p>
<p><strong>Action.</strong> 1. <strong>Indexing:</strong> Analyzed frequently queried columns (e.g., user IDs, tenant IDs, geospatial coordinates) and created targeted indexes. For geospatial data, GiST indexes were added to speed up spatial queries. Composite indexes were introduced for multi‑column filters, such as tenant ID + date ranges for electrical data.
2. <strong>Partitioning:</strong> Implemented time‑based range partitioning for the electrical data table, splitting it by day/month. This reduced the dataset size for queries and improved scan efficiency. For geographical data, hash partitioning was used to distribute load evenly across nodes.
3. <strong>Query Optimization:</strong> Rewrote complex queries to avoid full table scans, leveraging CTEs (Common Table Expressions) and materialized views for frequently accessed datasets. The <code>EXPLAIN ANALYZE</code> tool identified inefficient joins and subqueries, which were restructured for better execution plans. Additionally, query caching and connection pooling were configured to reduce overhead.</p>
<p><strong>Result.</strong> Post‑optimization, query response times improved by 10x, with critical operations (e.g., user authentication, geospatial lookups) executing in milliseconds. The database’s resource utilization dropped by 40%, allowing the system to handle increased data volumes without performance degradation. Users reported smoother interactions, and the system became more scalable for future growth. The changes also reduced the need for hardware upgrades, saving costs while ensuring long‑term reliability. This project demonstrated how strategic indexing, partitioning, and query refinement can transform even low‑load systems into efficient, future‑ready databases.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Automated GIS SaaS application deployment, data processing, and reporting system using GitHub Actions CI/CD, Python, Bash, and SQL.</title>
      <link>https://data.engineer.company/portfolio/automated-gis-saas-application-deployment-data-processing-and-16/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/automated-gis-saas-application-deployment-data-processing-and-16/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Automated GIS SaaS deployment, data processing and reporting with GitHub Actions CI/CD, Python, Bash and SQL — shorter, reliable releases.</description>
      <category domain="https://data.engineer.company/categories/">Automation &amp; CI/CD</category>
      <category domain="https://data.engineer.company/categories/">Cloud</category>
      <category domain="https://data.engineer.company/categories/">Data Engineering</category>
      <category domain="https://data.engineer.company/categories/">Data Pipelines (ETL/ELT)</category>
      <category domain="https://data.engineer.company/categories/">DevOps</category>
      <category domain="https://data.engineer.company/categories/">GIS / Geospatial</category>
      <category domain="https://data.engineer.company/categories/">Python</category>
      <category domain="https://data.engineer.company/categories/">Reliability &amp; Backups</category>
      <category domain="https://data.engineer.company/categories/">SQL</category>
      <category domain="https://data.engineer.company/services/">Data Pipeline Development (ETL/ELT)</category>
      <category domain="https://data.engineer.company/services/">DevOps &amp; CI/CD Automation</category>
      <category domain="https://data.engineer.company/services/">GIS &amp; Geospatial Solutions</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> At Utiligize, getting the GIS SaaS app deployed, processing its data and producing the reports were all manual steps — and manual steps are both slow and quietly dangerous. Every release ate engineering time and carried the chance of a slip, and the recurring data and reporting work sat there eating capacity week after week.</p>
<p><strong>Task.</strong> Automating the whole path from code to production, plus the recurring data‑processing and reporting, was the task — the goal being releases that were fast, safe and repeatable rather than a careful manual ritual each time.</p>
<p><strong>Action.</strong> The whole path got automated. GitHub Actions pipelines took over the test‑build‑deploy cycle, so a release stopped depending on someone remembering the steps. The recurring data processing and the reports moved into scheduled Python, Bash and SQL jobs, so they just ran instead of being someone&rsquo;s chore. And the configuration and secrets were standardised so every environment behaved identically — which is what kills the &ldquo;works on my machine&rdquo; surprises, because there stops being a &ldquo;my machine&rdquo; that&rsquo;s different from production.</p>
<p><strong>Result.</strong> Deployment, data processing and reporting all became automated and reliable, the manual toil came off the team&rsquo;s plate, and the release cycle got shorter. The team could put its attention on the product instead of the operations wrapped around it.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Automated delivery of 20 GIS data pipelines and app data ETL processes, streamlining infrastructure automation and reporting.</title>
      <link>https://data.engineer.company/portfolio/automated-delivery-of-20-gis-data-pipelines-and-17/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/automated-delivery-of-20-gis-data-pipelines-and-17/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Automated 20 GIS data pipelines and app-data ETL processes, streamlining infrastructure automation and delivering dependable, current data.</description>
      <category domain="https://data.engineer.company/categories/">Automation &amp; CI/CD</category>
      <category domain="https://data.engineer.company/categories/">Data Engineering</category>
      <category domain="https://data.engineer.company/categories/">Data Pipelines (ETL/ELT)</category>
      <category domain="https://data.engineer.company/categories/">DevOps</category>
      <category domain="https://data.engineer.company/categories/">GIS / Geospatial</category>
      <category domain="https://data.engineer.company/categories/">Monitoring &amp; Observability</category>
      <category domain="https://data.engineer.company/categories/">Reliability &amp; Backups</category>
      <category domain="https://data.engineer.company/services/">Data Pipeline Development (ETL/ELT)</category>
      <category domain="https://data.engineer.company/services/">DevOps &amp; CI/CD Automation</category>
      <category domain="https://data.engineer.company/services/">GIS &amp; Geospatial Solutions</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> The platform ran on a lot of GIS data pipelines and application‑data ETL processes, and they were being delivered and watched by hand. Hand‑run pipelines create bottlenecks, they drift out of consistency, and worst of all they carry a constant low risk that one quietly fails and nobody notices until the data&rsquo;s already wrong downstream.</p>
<p><strong>Task.</strong> Automating the delivery of those pipelines and ETL processes — so the data flowed reliably and predictably without someone shepherding it — was the task.</p>
<p><strong>Action.</strong> Twenty GIS data pipelines and the application‑data ETL came under automated delivery, end to end. The scheduling, the logging and the failure handling got standardised, so every pipeline behaved the same way and, crucially, you could see when one didn&rsquo;t — a silent failure is only silent if nothing&rsquo;s watching. And they were folded into the existing infrastructure automation and reporting, so they were part of one coherent system rather than a drawer full of scripts someone had to remember to run.</p>
<p><strong>Result.</strong> All twenty pipelines and their ETL ran automatically and predictably, and the whole infrastructure‑automation and reporting picture got tidier for it. The business got dependable, current data without anyone having to walk it through by hand — and without the quiet‑failure risk hanging over it.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Automated 100 critical data backups using Barman, Google Cloud, Bash, and Python, ensuring data integrity across databases.</title>
      <link>https://data.engineer.company/portfolio/automated-100-critical-data-backups-using-barman-google-18/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/automated-100-critical-data-backups-using-barman-google-18/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Automated 100 critical database backups with Barman, Google Cloud, Bash and Python — turning data recoverability into a tested fact.</description>
      <category domain="https://data.engineer.company/categories/">Automation &amp; CI/CD</category>
      <category domain="https://data.engineer.company/categories/">Cloud</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">DevOps</category>
      <category domain="https://data.engineer.company/categories/">Infrastructure</category>
      <category domain="https://data.engineer.company/categories/">PostgreSQL</category>
      <category domain="https://data.engineer.company/categories/">Python</category>
      <category domain="https://data.engineer.company/categories/">Reliability &amp; Backups</category>
      <category domain="https://data.engineer.company/services/">Backup &amp; Disaster Recovery</category>
      <category domain="https://data.engineer.company/services/">Database Administration (DBA)</category>
      <category domain="https://data.engineer.company/services/">DevOps &amp; CI/CD Automation</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> The critical GIS data, the application instances and the databases were scattered across systems with backups that were inconsistent and partly manual. For a product that lives on its data, that&rsquo;s not a risk you can leave sitting — and the day you actually need a backup is precisely the worst day to find out it was incomplete.</p>
<p><strong>Task.</strong> The task was to guarantee that all the critical data could be recovered, which meant automating comprehensive, verified backups across the whole estate — verified being the word that matters.</p>
<p><strong>Action.</strong> The backup regime was built end to end. A hundred critical data backups got automated, with Barman handling the PostgreSQL side and Google Cloud holding the offsite copies, and the whole thing was orchestrated and validated with Bash and Python — because a backup you&rsquo;ve taken but never checked isn&rsquo;t really a backup, it&rsquo;s a hope. So there were retention policies to keep them current and integrity checks to confirm each one was actually good, not just present.</p>
<p><strong>Result.</strong> Backups ran automatically and were verifiable across every database, which turned data recoverability from an assumption into something tested. A major operational risk came off the business and got replaced with a recovery path you could actually trust — the difference being that this one had been checked, not just configured.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Optimized forecasting and investment strategies for 11 electricity grid operators, driving operational efficiency through data‑driven GIS solutions.</title>
      <link>https://data.engineer.company/portfolio/optimized-forecasting-and-investment-strategies-for-11-electricity-27/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/optimized-forecasting-and-investment-strategies-for-11-electricity-27/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Optimised forecasting and investment strategy for 11 electricity grid operators with data-driven GIS — trustworthy, targeted planning.</description>
      <category domain="https://data.engineer.company/categories/">Data Analytics</category>
      <category domain="https://data.engineer.company/categories/">Data Engineering</category>
      <category domain="https://data.engineer.company/categories/">GIS / Geospatial</category>
      <category domain="https://data.engineer.company/categories/">Product &amp; Requirements</category>
      <category domain="https://data.engineer.company/categories/">Stakeholder &amp; Reporting</category>
      <category domain="https://data.engineer.company/services/">Data Analytics &amp; BI Dashboards</category>
      <category domain="https://data.engineer.company/services/">GIS &amp; Geospatial Solutions</category>
      <category domain="https://data.engineer.company/services/">Product Strategy &amp; Requirements</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> Electricity grid operators live and die on decisions about where to reinforce the network and where to put their money, and eleven of them were making those calls without much geospatial analysis underneath. They had the operational data. What they didn&rsquo;t have was a way to see it on the map, where the patterns actually live.</p>
<p><strong>Task.</strong> The job was to sharpen their forecasting and their investment strategies with GIS — to turn tables of readings into something that showed them where capacity was getting tight, where risk was building, and where the next pound was best spent.</p>
<p><strong>Action.</strong> Their operational data was brought together with geospatial modelling so the two reinforced each other. Instead of forecasting in the abstract, the grid could be looked at spatially and asked concrete questions: which stretches were heading toward their limits, which areas justified investment first. For eleven operators that meant fitting the analysis to how each of them actually ran their network, not handing everyone the same template and hoping it fit.</p>
<p><strong>Result.</strong> The operators came away with forecasts they could trust and investment decisions that were aimed rather than hopeful. Grounding the planning in what the map showed made the whole thing more efficient — money and attention went where the data pointed instead of where habit did.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Gathered and analyzed business requirements to translate into actionable features and user stories aligned with data governance standards.</title>
      <link>https://data.engineer.company/portfolio/gathered-and-analyzed-business-requirements-to-translate-into-31/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/gathered-and-analyzed-business-requirements-to-translate-into-31/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Turned business requirements into clear features and user stories aligned with data-governance standards, cutting ambiguity and rework.</description>
      <category domain="https://data.engineer.company/categories/">Agile &amp; Scrum</category>
      <category domain="https://data.engineer.company/categories/">Data Governance</category>
      <category domain="https://data.engineer.company/categories/">Documentation</category>
      <category domain="https://data.engineer.company/categories/">Product &amp; Requirements</category>
      <category domain="https://data.engineer.company/categories/">Project Management</category>
      <category domain="https://data.engineer.company/categories/">Stakeholder &amp; Reporting</category>
      <category domain="https://data.engineer.company/services/">Data Governance &amp; Quality</category>
      <category domain="https://data.engineer.company/services/">Product Strategy &amp; Requirements</category>
      <category domain="https://data.engineer.company/services/">Project Management (Agile)</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> Requirements tended to arrive as conversations — someone wanted something, roughly, and it fell to the developers to guess at the edges. Guessing means rework, and rework is about the most expensive way there is to build anything.</p>
<p><strong>Task.</strong> The role sat between the business and the engineering, translating one into the other: turning loose needs into work a developer could pick up without guessing, and keeping it lined up with the data‑governance standards along the way.</p>
<p><strong>Action.</strong> The requirements got worked out with the stakeholders directly, with the awkward questions asked early instead of discovered late, then written up as features and user stories that actually said what &ldquo;done&rdquo; meant. Each one was checked against the governance rules, because a feature that&rsquo;s useful but mishandles data isn&rsquo;t really finished. The whole aim was that someone could read a story and build the right thing the first time.</p>
<p><strong>Result.</strong> Development ran off clear, agreed features instead of half‑understood asks. Ambiguity fell, and rework fell with it, and the delivery stayed pointed at the actual business goals — inside the data‑governance lines rather than tidied up to fit them afterward.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Led the development, deployment, and support of over 30 GIS projects, demonstrating expertise in PostgreSQL, Bash, Python, JavaScript, GDAL, ArcGIS, PostGIS, and Mapbox technologies.</title>
      <link>https://data.engineer.company/portfolio/led-the-development-deployment-and-support-of-over-35/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/led-the-development-deployment-and-support-of-over-35/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Led development, deployment and support of 30&#43; GIS projects with PostgreSQL, Python, GDAL, ArcGIS, PostGIS and Mapbox across the lifecycle.</description>
      <category domain="https://data.engineer.company/categories/">Backend Engineering</category>
      <category domain="https://data.engineer.company/categories/">Data Engineering</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">Full‑Stack Development</category>
      <category domain="https://data.engineer.company/categories/">GIS / Geospatial</category>
      <category domain="https://data.engineer.company/categories/">PostgreSQL</category>
      <category domain="https://data.engineer.company/categories/">Python</category>
      <category domain="https://data.engineer.company/categories/">Reliability &amp; Backups</category>
      <category domain="https://data.engineer.company/categories/">SQL</category>
      <category domain="https://data.engineer.company/categories/">Team Leadership</category>
      <category domain="https://data.engineer.company/categories/">Technical Leadership</category>
      <category domain="https://data.engineer.company/categories/">Web Development</category>
      <category domain="https://data.engineer.company/services/">Backend &amp; API Development</category>
      <category domain="https://data.engineer.company/services/">Data Pipeline Development (ETL/ELT)</category>
      <category domain="https://data.engineer.company/services/">Full‑Stack Product Development</category>
      <category domain="https://data.engineer.company/services/">GIS &amp; Geospatial Solutions</category>
      <category domain="https://data.engineer.company/services/">Technical Leadership &amp; Consulting</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> The company&rsquo;s whole output was made‑to‑measure GIS — custom mapping and spatial systems built to a client&rsquo;s specific problem, then kept running once they were live. Building the thing is only half of it; a geospatial project that ships and then falls over in production hasn&rsquo;t really been delivered.</p>
<p><strong>Task.</strong> These projects ran end to end through this role — the development, the deployment, and the support once they were live — with the technical direction across a fairly wide stack.</p>
<p><strong>Action.</strong> Delivery ran on more than thirty GIS projects, hands‑on across the stack the whole way. PostgreSQL with PostGIS underneath for the spatial data, GDAL/OGR for moving it between formats — shapefiles, GeoJSON, GeoTIFF, KML, vector tiles — and QGIS and JOSM for the data work itself. On the front of it, web maps built on Mapbox GL and Leaflet, sometimes against the ArcGIS or HERE APIs, with the app layer in JavaScript, Python, PHP and SQL. The work ranged from 2D and 3D digital mapping through LiDAR processing, georectification and vectorisation to indoor mapping and navigation. And the responsibility ran past the point of shipping — the deployment and the ongoing support in production were part of it too, so problems weren&rsquo;t handed off; the decisions had to be lived with.</p>
<p><strong>Result.</strong> Thirty‑plus projects built, deployed and supported across that whole range. Being on the hook for the full lifecycle rather than just the build is what kept the quality honest — you design differently when you know you&rsquo;re the one getting the call if it breaks.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Streamlined data analysis and software development processes, saving 4,000 hours by introducing GitHub, GitLab, Bash, and Python CI/CD practices.</title>
      <link>https://data.engineer.company/portfolio/streamlined-data-analysis-and-software-development-processes-saving-43/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/streamlined-data-analysis-and-software-development-processes-saving-43/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Saved ~4,000 hours by introducing GitHub, GitLab, Bash and Python CI/CD, speeding data analysis and development while adding consistency.</description>
      <category domain="https://data.engineer.company/categories/">Automation &amp; CI/CD</category>
      <category domain="https://data.engineer.company/categories/">Data Analytics</category>
      <category domain="https://data.engineer.company/categories/">Data Engineering</category>
      <category domain="https://data.engineer.company/categories/">Data Pipelines (ETL/ELT)</category>
      <category domain="https://data.engineer.company/categories/">DevOps</category>
      <category domain="https://data.engineer.company/categories/">Python</category>
      <category domain="https://data.engineer.company/services/">Data Analytics &amp; BI Dashboards</category>
      <category domain="https://data.engineer.company/services/">Data Pipeline Development (ETL/ELT)</category>
      <category domain="https://data.engineer.company/services/">DevOps &amp; CI/CD Automation</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> Both the data‑analysis work and the software development were being held back by the same thing: manual processes. Work moved from development to delivery in a slow, inconsistent way, and the analysis side had its own pile of repetitive steps someone was doing by hand every time.</p>
<p><strong>Task.</strong> The aim was to streamline both by bringing modern automation and CI/CD practices to workflows that hadn&rsquo;t had them.</p>
<p><strong>Action.</strong> CI/CD practices went in, built on GitHub and GitLab, with Bash and Python doing the automation work underneath. The repetitive steps across both the data‑analysis and the development workflows got automated, and how work moved from development through to delivery got standardised so it was the same every time rather than reinvented per project. Bringing the analysis side into the same disciplined pipeline as the dev work was a big part of it — it had been treated as a separate, more manual world.</p>
<p><strong>Result.</strong> The streamlined processes saved roughly 4,000 hours and sped up both the data analysis and the software development, and just as usefully made what got shipped more consistent — fewer surprises from work that had been done a slightly different way each time.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Developed a Data Analytics reporting system, increasing quarterly software revenue by 400% through Python‑based PDF reports.</title>
      <link>https://data.engineer.company/portfolio/developed-a-data-analytics-reporting-system-increasing-quarterly-44/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/developed-a-data-analytics-reporting-system-increasing-quarterly-44/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Built a Python data-analytics reporting system that raised quarterly software revenue by 400% with clear, timely PDF reports.</description>
      <category domain="https://data.engineer.company/categories/">Automation &amp; CI/CD</category>
      <category domain="https://data.engineer.company/categories/">Backend Engineering</category>
      <category domain="https://data.engineer.company/categories/">Data Analytics</category>
      <category domain="https://data.engineer.company/categories/">Data Engineering</category>
      <category domain="https://data.engineer.company/categories/">Product &amp; Requirements</category>
      <category domain="https://data.engineer.company/categories/">Python</category>
      <category domain="https://data.engineer.company/categories/">Stakeholder &amp; Reporting</category>
      <category domain="https://data.engineer.company/services/">Backend &amp; API Development</category>
      <category domain="https://data.engineer.company/services/">Data Analytics &amp; BI Dashboards</category>
      <category domain="https://data.engineer.company/services/">Product Strategy &amp; Requirements</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> Stakeholders weren&rsquo;t getting analytics in any timely, readable form. The data existed, but turning it into something you could actually make a decision from was slow and manual, so visibility into how things were performing lagged, and the commercial decisions lagged with it.</p>
<p><strong>Task.</strong> Building a data‑analytics reporting system — one that turned raw data into clear, regular insight without someone hand‑assembling it each time — was the job.</p>
<p><strong>Action.</strong> A reporting system generated PDF reports in Python, automating the whole chain: pulling the data, running the analysis, and presenting it in a clean, consistent format stakeholders could actually read. The point was regularity and clarity — the same professional report landing predictably, so the numbers became something people looked at as a matter of course rather than something they had to go and dig for.</p>
<p><strong>Result.</strong> That reporting is what drove quarterly software revenue up by 400%. Making the analytics better and faster wasn&rsquo;t a back‑office nicety — put clear, timely numbers in front of the people making commercial decisions and the decisions get better, and here that showed up directly on the revenue.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Automated data processing tasks using Shell scripting, PL/pgSQL, Python, and Transact‑SQL, increasing productivity and efficiency.</title>
      <link>https://data.engineer.company/portfolio/automated-data-processing-tasks-using-shell-scripting-pl-45/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/automated-data-processing-tasks-using-shell-scripting-pl-45/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Automated data-processing tasks with Shell, PL/pgSQL, Python and Transact-SQL, lifting productivity and ending small recurring errors.</description>
      <category domain="https://data.engineer.company/categories/">Automation &amp; CI/CD</category>
      <category domain="https://data.engineer.company/categories/">Data Engineering</category>
      <category domain="https://data.engineer.company/categories/">Data Pipelines (ETL/ELT)</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">PostgreSQL</category>
      <category domain="https://data.engineer.company/categories/">Python</category>
      <category domain="https://data.engineer.company/categories/">SQL</category>
      <category domain="https://data.engineer.company/services/">Data Pipeline Development (ETL/ELT)</category>
      <category domain="https://data.engineer.company/services/">Database Administration (DBA)</category>
      <category domain="https://data.engineer.company/services/">DevOps &amp; CI/CD Automation</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> There was a steady load of recurring data‑processing work being done by hand. Manual data work has two problems at once: it eats time, and it&rsquo;s inconsistent — do the same task by hand enough times and it&rsquo;ll get done slightly differently, and some of those differences are errors.</p>
<p><strong>Task.</strong> The aim was to automate these tasks, both to get the time back and to make them reliable.</p>
<p><strong>Action.</strong> The data‑processing work got automated across the databases and systems it touched, using whatever fit the job — Shell scripting for the glue, PL/pgSQL and Transact‑SQL down in the databases, Python where it needed more than SQL could give. Manual steps got replaced with jobs that ran the same way every time, which is the whole point: a script doesn&rsquo;t get bored, doesn&rsquo;t skip a step, and doesn&rsquo;t do it differently on a Friday afternoon.</p>
<p><strong>Result.</strong> Productivity and efficiency both went up, the manual effort came off people&rsquo;s plates, and the data processing became consistent and dependable instead of a source of small, recurring errors.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Engineered 600 PL/pgSQL‑based ETL/ELT pipelines to streamline complex data processing workflows across multiple PostgreSQL development and production environments.</title>
      <link>https://data.engineer.company/portfolio/engineered-600-pl-pgsql-based-etl-elt-pipelines-54/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/engineered-600-pl-pgsql-based-etl-elt-pipelines-54/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Engineered 600 PL/pgSQL ETL/ELT pipelines across PostgreSQL dev and prod, cutting average execution time 35% and lifting throughput 50%&#43;.</description>
      <category domain="https://data.engineer.company/categories/">Automation &amp; CI/CD</category>
      <category domain="https://data.engineer.company/categories/">Data Engineering</category>
      <category domain="https://data.engineer.company/categories/">Data Governance</category>
      <category domain="https://data.engineer.company/categories/">Data Pipelines (ETL/ELT)</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">Performance Tuning</category>
      <category domain="https://data.engineer.company/categories/">PostgreSQL</category>
      <category domain="https://data.engineer.company/categories/">SQL</category>
      <category domain="https://data.engineer.company/services/">Data Governance &amp; Quality</category>
      <category domain="https://data.engineer.company/services/">Data Pipeline Development (ETL/ELT)</category>
      <category domain="https://data.engineer.company/services/">Database Administration (DBA)</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> The organization was managing large volumes of transactional and analytical data across multiple PostgreSQL development and production environments. The existing data pipelines were fragmented, lacked consistency, and were causing performance bottlenecks, leading to delays in business‑critical reporting and decision‑making.</p>
<p><strong>Task.</strong> The task was to design and implement robust, scalable, and efficient ETL/ELT pipelines using PL/pgSQL to streamline complex data ingestion, transformation, and loading workflows. A key objective was to improve query performance and ensure data integrity across all environments.</p>
<p><strong>Action.</strong> Over 600 PL/pgSQL‑based ETL/ELT pipelines were engineered to automate the extraction, transformation, and loading of data from various sources. Key technical implementations included:</p>
<ul>
<li>Leveraged partitioned tables and materialized views to optimize read performance for large datasets.</li>
<li>Applied primary and foreign key constraints to maintain referential integrity during transformations.</li>
<li>Designed unique and composite indexes to speed up JOIN operations and complex filtering criteria.</li>
<li>Incorporated exception handling and transactional control to ensure fault tolerance and rollback in case of failures.</li>
<li>Enabled incremental loads using change data capture (CDC) mechanisms and timestamp‑based deltas, reducing processing time by over 60%.</li>
<li>Developed automated logging and auditing procedures to track pipeline execution, monitor anomalies, and facilitate debugging.</li>
</ul>
<p><strong>Result.</strong> The new ETL/ELT framework significantly improved the consistency, reliability, and performance of data processing workflows:</p>
<ul>
<li>Achieved a 35% reduction in average pipeline execution time.</li>
<li>Increased data pipeline throughput by over 50%, enabling near real‑time data availability for reporting.</li>
<li>Reduced data quality issues by eliminating over 90% of transformation errors through better constraint enforcement and validation logic.</li>
<li>Enhanced maintainability and scalability, supporting future data model changes with minimal rework.</li>
</ul>
]]></content:encoded>
    </item>
    <item>
      <title>Architected, developed, implemented, supported infrastructure, data processing, and the map application for 2 years non‑stop without any weekends, holidays, or vacations, 10–14 hours a day.</title>
      <link>https://data.engineer.company/portfolio/architected-developed-implemented-supported-infrastructure-data-processing-and-55/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/architected-developed-implemented-supported-infrastructure-data-processing-and-55/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Architected, built and ran the infrastructure, data processing and map app for two years non-stop — the dependable backbone of the product.</description>
      <category domain="https://data.engineer.company/categories/">Data Engineering</category>
      <category domain="https://data.engineer.company/categories/">Data Pipelines (ETL/ELT)</category>
      <category domain="https://data.engineer.company/categories/">Full‑Stack Development</category>
      <category domain="https://data.engineer.company/categories/">GIS / Geospatial</category>
      <category domain="https://data.engineer.company/categories/">Infrastructure</category>
      <category domain="https://data.engineer.company/categories/">Platform Architecture</category>
      <category domain="https://data.engineer.company/categories/">Reliability &amp; Backups</category>
      <category domain="https://data.engineer.company/services/">Data Pipeline Development (ETL/ELT)</category>
      <category domain="https://data.engineer.company/services/">Full‑Stack Product Development</category>
      <category domain="https://data.engineer.company/services/">GIS &amp; Geospatial Solutions</category>
      <category domain="https://data.engineer.company/services/">Platform &amp; Solution Architecture</category>
      <category domain="https://data.engineer.company/services/">Site Reliability &amp; Monitoring</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> An early‑stage green‑energy startup depended on a single platform to track, monitor, and optimize renewable energy assets, yet had neither a dedicated infrastructure team nor an established engineering organization to build and operate it. The entire technical foundation — cloud infrastructure, data‑processing pipelines, and the customer‑facing GIS map application — had to be created and kept running continuously, in a market where any downtime or data gap directly eroded customer trust and revenue.</p>
<p><strong>Task.</strong> The task was to single‑handedly architect, build, and operate the whole system end to end, spanning platform and data engineering, DevOps, and site reliability. Beyond writing the software, this meant owning production: provisioning and hardening infrastructure, designing the data‑processing layer that fed the map, and guaranteeing the application stayed available around the clock for a growing customer base — all within the constraints and relentless pace of a fast‑moving startup.</p>
<p><strong>Action.</strong> For two years the infrastructure, data pipelines, and map application were designed, implemented, and supported without interruption — no weekends, holidays, or vacations, often ten to fourteen hours a day. A pragmatic, modular architecture was chosen to keep a one‑person operation maintainable, with automated provisioning, monitoring, and alerting so issues could be detected and resolved quickly. Data processing was continuously tuned for reliability and performance, releases were shipped incrementally, and every layer — from servers to the user‑facing map — was personally maintained and improved in response to real customer usage.</p>
<p><strong>Result.</strong> The platform stayed continuously available and evolved from a fragile early prototype into the dependable backbone of the product, sustaining the company through its critical growth phase on the strength of a single engineer&rsquo;s ownership. This hands‑on stewardship kept infrastructure, data, and the map application reliable enough to support upsells, data licensing, and new‑customer acquisition, and demonstrated a rare degree of commitment, breadth, and end‑to‑end accountability across the full stack.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Architected a layered maritime platform separating a Next.js PWA frontend, a Go (Huma/Fiber) API, and a PostgreSQL function layer, keeping all business logic in the database.</title>
      <link>https://data.engineer.company/portfolio/architected-a-layered-maritime-platform-separating-a-next-57/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/architected-a-layered-maritime-platform-separating-a-next-57/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Architected a layered maritime platform — Next.js PWA, a Go (Huma/Fiber) API and a PostgreSQL function layer holding all business logic.</description>
      <category domain="https://data.engineer.company/categories/">APIs &amp; Integration</category>
      <category domain="https://data.engineer.company/categories/">Backend Engineering</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">Frontend Engineering</category>
      <category domain="https://data.engineer.company/categories/">Full‑Stack Development</category>
      <category domain="https://data.engineer.company/categories/">Platform Architecture</category>
      <category domain="https://data.engineer.company/categories/">PostgreSQL</category>
      <category domain="https://data.engineer.company/categories/">Solution Architecture</category>
      <category domain="https://data.engineer.company/categories/">Technical Leadership</category>
      <category domain="https://data.engineer.company/services/">Backend &amp; API Development</category>
      <category domain="https://data.engineer.company/services/">Database Design &amp; Modeling</category>
      <category domain="https://data.engineer.company/services/">Frontend Development</category>
      <category domain="https://data.engineer.company/services/">Full‑Stack Product Development</category>
      <category domain="https://data.engineer.company/services/">Platform &amp; Solution Architecture</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> NextMariner was going to be a maritime platform — professional networking, company reviews, sponsored education — and it was a young product, which is a polite way of saying the requirements were going to move around a lot. The thing to avoid was an architecture where changing a business rule meant touching the frontend, the API and the database all at once. On a small codebase that grows fast, that kind of coupling is what turns a two‑line change into an afternoon.</p>
<p><strong>Task.</strong> As the architect, the call to make up front was where each kind of logic lived, with boundaries obvious enough to hold up under pressure instead of blurring the first time someone was in a hurry.</p>
<p><strong>Action.</strong> It settled into three layers, one job each. The Next.js frontend does presentation and interactivity and nothing else. The Go API — Huma over Fiber — is deliberately thin: it routes, validates the request, applies the security filtering and orchestrates, but it holds no business logic at all. The business logic all lives in PostgreSQL functions, which assemble the full result and hand it back for the API to forward. So when a rule changes, it changes in one layer, in SQL, and the other two don&rsquo;t need to know. The boundaries were written down and then enforced in review, because a convention nobody polices stops being one.</p>
<p><strong>Result.</strong> The thing stayed easy to hold in your head. Business logic sits in one place you can actually audit, the API is a boring adapter in the good sense, and the frontend doesn&rsquo;t care when the schema shifts underneath it. That separation is what let the product keep bolting on features without the architecture quietly rotting.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Designed a JSON passthrough architecture where PostgreSQL functions return complete JSON forwarded verbatim by the Go API, eliminating intermediate unmarshalling and decoupling the frontend from schema changes.</title>
      <link>https://data.engineer.company/portfolio/designed-a-json-passthrough-architecture-where-postgresql-functions-58/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/designed-a-json-passthrough-architecture-where-postgresql-functions-58/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Designed a JSON passthrough where PostgreSQL functions return complete JSON forwarded verbatim by the Go API, decoupling the frontend from schema.</description>
      <category domain="https://data.engineer.company/categories/">APIs &amp; Integration</category>
      <category domain="https://data.engineer.company/categories/">Backend Engineering</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">Performance Tuning</category>
      <category domain="https://data.engineer.company/categories/">Platform Architecture</category>
      <category domain="https://data.engineer.company/categories/">PostgreSQL</category>
      <category domain="https://data.engineer.company/categories/">SQL</category>
      <category domain="https://data.engineer.company/services/">Backend &amp; API Development</category>
      <category domain="https://data.engineer.company/services/">Database Design &amp; Modeling</category>
      <category domain="https://data.engineer.company/services/">Platform &amp; Solution Architecture</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> The usual way data gets from a database to a browser is a relay race of transformations. The database hands the API rows, the API unmarshals them into structs, reshapes them, serialises them back to JSON, and only then do they go out. Every one of those hops is code you write, code you test, and one more place where the API&rsquo;s idea of the data and the database&rsquo;s idea of it can drift apart.</p>
<p><strong>Task.</strong> The idea was to skip the relay race. If the database could return the finished response, the API could just pass it along, and the frontend could depend on the database&rsquo;s shape directly instead of on a hand‑maintained copy of it living in Go.</p>
<p><strong>Action.</strong> So it was built as a straight passthrough. The PostgreSQL functions assemble the whole response as JSON — the shaping is a SQL concern, done where the data already is. The Go handler takes that back as json.RawMessage and forwards it untouched; it never decomposes it, never re‑encodes it. A small QueryJSON helper made that pattern the path of least resistance rather than something you had to remember to do. What fell out was all the intermediate machinery a conventional layered API collects — the DTOs, the mappers, the response structs.</p>
<p><strong>Result.</strong> Handler code got dramatically shorter, and more to the point the frontend stopped being coupled to Go. Change what a function returns and the new shape flows straight through to the client without anyone editing a line of handler code. Fewer moving parts, and one whole category of layer‑to‑layer drift simply doesn&rsquo;t exist here.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Designed a PostgreSQL function‑first data layer — 1,275 stored functions across 34 schemas — so every read and write goes through a function the database can grant, rather than through a table.</title>
      <link>https://data.engineer.company/portfolio/designed-a-postgresql-function-first-data-layer-across-63/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/designed-a-postgresql-function-first-data-layer-across-63/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Designed a PostgreSQL function-first data layer — 1,275 stored functions across 34 schemas — with the access boundary enforced by the database itself.</description>
      <category domain="https://data.engineer.company/categories/">Backend Engineering</category>
      <category domain="https://data.engineer.company/categories/">Data Governance</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">Platform Architecture</category>
      <category domain="https://data.engineer.company/categories/">PostgreSQL</category>
      <category domain="https://data.engineer.company/categories/">Security</category>
      <category domain="https://data.engineer.company/categories/">SQL</category>
      <category domain="https://data.engineer.company/services/">Backend &amp; API Development</category>
      <category domain="https://data.engineer.company/services/">Data Governance &amp; Quality</category>
      <category domain="https://data.engineer.company/services/">Database Design &amp; Modeling</category>
      <category domain="https://data.engineer.company/services/">Platform &amp; Solution Architecture</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> Business logic has a way of leaking. A bit ends up in the API, a bit in some SQL a handler runs inline, and before long the same rule is written two or three slightly different ways and there&rsquo;s nowhere you can point to and say &ldquo;this is what the system does with its data.&rdquo; That&rsquo;s how bugs and security holes get in.</p>
<p><strong>Task.</strong> The goal was one home for all of it: every read and write going through the database, the API a thin adapter that doesn&rsquo;t know the business rules, and the whole thing lockable down tight.</p>
<p><strong>Action.</strong> The data layer is function‑first. The schema is split by domain — identity, organization, review, message, notification and 29 more, 34 in all — and every operation the app can do is one of 1,275 PostgreSQL functions it calls; there&rsquo;s no direct table access from Go at all. Then the database enforces it. The role the API logs in as, mariner, has EXECUTE on the app functions and USAGE on the schemas and nothing else — no SELECT, no INSERT, no way to touch a table directly — which comes to roughly 4,000 explicit grants rather than a blanket one. The functions run SECURITY DEFINER, owned by a separate non‑login function_owner role with a pinned search_path, and the superuser account stays reserved for migrations and cron, well away from the running app.</p>
<p><strong>Result.</strong> The logic lives in one place you can actually audit, the API stays thin and boring in the good way, and the access boundary is enforced by Postgres itself rather than by everyone remembering the rules. If the API were somehow compromised, it still couldn&rsquo;t do anything the functions don&rsquo;t allow. At 1,275 functions the discipline costs something real — a new field is a migration and a function change, not a line in a query — and that friction is the price of the boundary holding.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Adopted UUID v7 time‑ordered identifiers (PostgreSQL 18) as entity keys to reduce B‑tree index fragmentation and speed up queries.</title>
      <link>https://data.engineer.company/portfolio/adopted-uuid-v7-time-ordered-identifiers-postgresql-18-64/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/adopted-uuid-v7-time-ordered-identifiers-postgresql-18-64/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Adopted UUID v7 time-ordered keys (PostgreSQL 18) to cut B-tree index fragmentation and speed inserts and range queries across every table.</description>
      <category domain="https://data.engineer.company/categories/">Backend Engineering</category>
      <category domain="https://data.engineer.company/categories/">Data Engineering</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">Performance Tuning</category>
      <category domain="https://data.engineer.company/categories/">PostgreSQL</category>
      <category domain="https://data.engineer.company/services/">Database Design &amp; Modeling</category>
      <category domain="https://data.engineer.company/services/">Database Performance Tuning</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> Every entity needs a unique id, and the reflex choice is a random UUID. The trouble is that random ids land all over a B‑tree index. Inserts scatter, the index fragments, and as tables grow both writes and range scans pay for it. On a platform meant to keep growing, that&rsquo;s a slow leak you&rsquo;d rather not build in.</p>
<p><strong>Task.</strong> Keep the global uniqueness of a UUID but drop the fragmentation that comes with the randomness.</p>
<p><strong>Action.</strong> The standard became UUID v7, which is time‑ordered — the leading bits are a timestamp, so new rows sort into the index instead of peppering it. PostgreSQL 18 has this natively as uuidv7(), wrapped in a small uuid_generate_v7() function so the same call behaves cleanly on Azure&rsquo;s Flexible Server, then made the default for entity primary keys across the schema. Nothing exotic about it; it&rsquo;s the kind of decision that&rsquo;s cheap if you make it early and a pain to retrofit later.</p>
<p><strong>Result.</strong> Ids stayed globally unique, the index stopped fragmenting the way random UUIDs cause, and time‑ordered inserts and range queries got quicker — evenly, across every table, without anyone having to think about it again. As a bonus, every entity ends up with a key you can sort by time for free.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Implemented a catalogue‑driven deep‑merge for stored JSON preferences, preventing missing‑key crashes as the schema evolves.</title>
      <link>https://data.engineer.company/portfolio/implemented-a-catalogue-driven-deep-merge-for-stored-65/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/implemented-a-catalogue-driven-deep-merge-for-stored-65/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Implemented a catalogue-driven deep-merge for stored JSON preferences, preventing missing-key crashes as the schema evolves.</description>
      <category domain="https://data.engineer.company/categories/">Backend Engineering</category>
      <category domain="https://data.engineer.company/categories/">Data Governance</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">Migrations &amp; Modernization</category>
      <category domain="https://data.engineer.company/categories/">PostgreSQL</category>
      <category domain="https://data.engineer.company/services/">Backend &amp; API Development</category>
      <category domain="https://data.engineer.company/services/">Data Governance &amp; Quality</category>
      <category domain="https://data.engineer.company/services/">Database Design &amp; Modeling</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> The platform stores JSON preferences — notification settings and the like — as a user&rsquo;s saved values laid over a set of defaults. The original merge did that at a single level. The problem shows up later: add a new key to the defaults, and rows saved before that key existed simply don&rsquo;t have it. Then some client code reads that field, gets undefined, and falls over — for exactly the users who&rsquo;ve been around longest.</p>
<p><strong>Task.</strong> Evolving the preference schema had to be safe, so that adding a setting could never break the people who signed up before it existed.</p>
<p><strong>Action.</strong> The single‑level merge was replaced with a deep‑merge driven by the defaults as a catalogue. The defaults are treated as the authoritative list of every key that should exist; the user&rsquo;s stored values are merged recursively on top, so anything in the catalogue is guaranteed to come out present, whether or not it was in the saved blob. Add a key to the defaults and it appears in every existing row&rsquo;s effective preferences automatically, nested keys included. It rolled in through a migration so existing data got the benefit straight away rather than waiting to be rewritten.</p>
<p><strong>Result.</strong> Preferences can grow without fear. Adding a setting no longer risks an undefined‑field crash in the client, and the frontend stopped needing defensive checks scattered around every place it reads a preference. It also gave the rest of the platform a dependable way to extend any stored‑JSON blob — the pattern, not just the one fix.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Built the Python vessel‑data scrapers (MarineTraffic, Maritime‑Database) and a repeatable import that seeds the platform&#39;s reference data — 184,197 rows, including 698 companies and 56,149 vessels.</title>
      <link>https://data.engineer.company/portfolio/built-a-python-vessel-data-scraper-marinetraffic-maritime-66/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/built-a-python-vessel-data-scraper-marinetraffic-maritime-66/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Built Python vessel-data scrapers and seeded 184,197 rows of maritime reference data — 698 companies and 56,149 vessels — through a repeatable pipeline.</description>
      <category domain="https://data.engineer.company/categories/">Automation &amp; CI/CD</category>
      <category domain="https://data.engineer.company/categories/">Backend Engineering</category>
      <category domain="https://data.engineer.company/categories/">Data Engineering</category>
      <category domain="https://data.engineer.company/categories/">Data Pipelines (ETL/ELT)</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">Python</category>
      <category domain="https://data.engineer.company/services/">Backend &amp; API Development</category>
      <category domain="https://data.engineer.company/services/">Data Pipeline Development (ETL/ELT)</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> A maritime networking‑and‑reviews site is dead on arrival if it&rsquo;s empty. Nobody joins a directory with no companies in it. So before any of the social features mattered, the platform needed a real body of maritime companies and vessels already sitting there, ready to be found.</p>
<p><strong>Task.</strong> Go and get that data — real companies and ships, at enough scale to feel populated — and get it into the database in a way that could be rerun, not a one‑off scrape nobody could reproduce.</p>
<p><strong>Action.</strong> The scrapers are written in Python. One drives MarineTraffic with Playwright; another pulls from Maritime‑Database over async httpx; there&rsquo;s a ClassNK fetcher in there too. They write out CSVs, and an import step cleans and normalises those and loads them into the Postgres schema through a single task, so seeding the database is one command rather than an afternoon of manual work. What went in came to 184,197 rows: 698 companies, 56,149 vessels and 33,074 cities, with a later refresh replacing 74,794 vessel rows.</p>
<p><strong>Result.</strong> The platform launched with a populated directory instead of empty tables, and a base of reference data the networking, jobs and review features could all build on top of. Because the pipeline is repeatable, refreshing or extending it later is just running it again — which is how the vessel refresh happened without anyone rebuilding the tooling. Scraped data ages, and keeping it current is an ongoing cost rather than a solved problem.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Modeled the maritime domain into 348 normalized tables across 34 PostgreSQL schemas — professionals, companies, ships, jobs, reviews and the rest — with SMALLINT lookups and UUID v7 keys.</title>
      <link>https://data.engineer.company/portfolio/modeled-the-maritime-domain-professionals-companies-ships-jobs-67/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/modeled-the-maritime-domain-professionals-companies-ships-jobs-67/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Modeled the maritime domain into 348 normalized tables across 34 PostgreSQL schemas, with SMALLINT lookups, UUID v7 keys and one naming convention.</description>
      <category domain="https://data.engineer.company/categories/">Data Engineering</category>
      <category domain="https://data.engineer.company/categories/">Data Governance</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">Performance Tuning</category>
      <category domain="https://data.engineer.company/categories/">Platform Architecture</category>
      <category domain="https://data.engineer.company/categories/">PostgreSQL</category>
      <category domain="https://data.engineer.company/categories/">SQL</category>
      <category domain="https://data.engineer.company/services/">Data Governance &amp; Quality</category>
      <category domain="https://data.engineer.company/services/">Database Design &amp; Modeling</category>
      <category domain="https://data.engineer.company/services/">Platform &amp; Solution Architecture</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> The heart of NextMariner is a dense maritime domain — professionals, companies, ships, jobs, reviews — and these things reference each other constantly. A professional sails on ships, works for companies, leaves reviews; a company owns ships and posts jobs. Nearly every feature is a query across that web, so how well the data is modelled decides how well most of the app performs and how sane it is to extend.</p>
<p><strong>Task.</strong> That domain had to be modelled so it stayed fast and kept its integrity, and so that adding the next entity type didn&rsquo;t mean fighting the schema.</p>
<p><strong>Action.</strong> It&rsquo;s laid out as normalized PostgreSQL schemas organised by domain — 348 tables across 34 of them by now. The many small, stable enumerations — statuses, types, categories — became SMALLINT lookup tables, which keeps the rows compact and the joins cheap instead of storing text codes everywhere. Entities get UUID v7 primary keys, so they&rsquo;re globally unique but still time‑ordered in the index. One naming convention runs throughout — plural table names inside singular‑named schemas — applied without exception, which as a bonus sidesteps a lot of reserved‑word collisions. And the relationships are held up by real foreign keys and constraints, so integrity is the database&rsquo;s job, not something the application has to remember to do.</p>
<p><strong>Result.</strong> What came out is a data model that&rsquo;s consistent and quick, and predictable to work in because the same rules hold everywhere — there aren&rsquo;t special cases to memorise. Adding a feature usually means extending the schema along the existing grain rather than working against it, which is the only reason 34 schemas is a structure rather than a sprawl. It&rsquo;s the floor the rest of the platform stands on.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Configured database backup retention as infrastructure‑as‑code, then audited the recovery position and documented the restore procedure — naming the remaining gaps rather than leaving them to be found during an incident.</title>
      <link>https://data.engineer.company/portfolio/configured-database-backup-retention-as-infrastructure-as-code-72/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/configured-database-backup-retention-as-infrastructure-as-code-72/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Implemented database backups and an infrastructure-as-code disaster-recovery strategy for fast, reproducible recovery.</description>
      <category domain="https://data.engineer.company/categories/">Cloud</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">DevOps</category>
      <category domain="https://data.engineer.company/categories/">Infrastructure</category>
      <category domain="https://data.engineer.company/categories/">PostgreSQL</category>
      <category domain="https://data.engineer.company/categories/">Reliability &amp; Backups</category>
      <category domain="https://data.engineer.company/services/">Backup &amp; Disaster Recovery</category>
      <category domain="https://data.engineer.company/services/">Database Administration (DBA)</category>
      <category domain="https://data.engineer.company/services/">Infrastructure as Code</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> A product that lives on its data can&rsquo;t afford to lose any, and &ldquo;there are backups somewhere&rdquo; is a hope, not a recovery plan. The only backup worth having is one you know restores, into an environment you know you can rebuild. NextMariner had neither half written down.</p>
<p><strong>Task.</strong> Get the recoverable position defined instead of assumed — the data, the environment around it, and an honest account of how far that actually reaches today.</p>
<p><strong>Action.</strong> Backup retention is configured in the Bicep beside the database it protects, so point‑in‑time restore is a property of the template rather than a setting someone once clicked in a portal. The environment around it is defined as infrastructure‑as‑code too, which is the quiet half people forget: restoring a database into an environment you&rsquo;d have to rebuild by hand from memory isn&rsquo;t really recovery. Then the position was audited and written up — the restore procedure, the drill that would measure it, and the gaps still open: geo‑redundancy is switched off, and the recovery‑time objective is proposed rather than measured, because no drill has been run yet.</p>
<p><strong>Result.</strong> Recovery stopped being a vague reassurance and became a documented position with its gaps named. That reads less impressively than &ldquo;disaster recovery: done&rdquo;, and it is worth considerably more — whoever touches it next knows what is covered, what isn&rsquo;t, and exactly which drill closes the difference. A named gap is one you can close; an unnamed one is discovered during an incident.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Delivered analysis and regular progress reports to the CEO, translating engineering metrics and delivery status into decisions.</title>
      <link>https://data.engineer.company/portfolio/delivered-analysis-and-regular-progress-reports-to-the-81/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/delivered-analysis-and-regular-progress-reports-to-the-81/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Delivered analysis and regular progress reports to the CEO, translating engineering metrics and delivery status into confident decisions.</description>
      <category domain="https://data.engineer.company/categories/">Data Analytics</category>
      <category domain="https://data.engineer.company/categories/">Product &amp; Requirements</category>
      <category domain="https://data.engineer.company/categories/">Project Management</category>
      <category domain="https://data.engineer.company/categories/">Stakeholder &amp; Reporting</category>
      <category domain="https://data.engineer.company/categories/">Technical Leadership</category>
      <category domain="https://data.engineer.company/services/">Data Analytics &amp; BI Dashboards</category>
      <category domain="https://data.engineer.company/services/">Project Management (Agile)</category>
      <category domain="https://data.engineer.company/services/">Technical Leadership &amp; Consulting</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> As the platform came together, its progress and its health had to be visible to leadership in terms they could actually do something with. Raw engineering signals — build status, delivery pace, incidents — don&rsquo;t mean much on their own to someone making product and investment calls; they&rsquo;re at the wrong altitude. Someone had to translate.</p>
<p><strong>Task.</strong> That translation was the job: taking the engineering reality — where delivery stood, what the system metrics were saying — and reporting it to the CEO clearly and regularly, so decisions rested on facts instead of guesswork.</p>
<p><strong>Action.</strong> A steady reporting rhythm went in, rather than reporting when asked, which is always a bit too late. Delivery progress, scope, risks and system health got tracked and turned into plain, decision‑oriented updates — what&rsquo;s on track, what&rsquo;s at risk, and what a given priority would actually cost in trade‑offs. The thing to avoid was handing over raw numbers and leaving the interpretation to someone without the context; each report came with concrete recommendations, and where it helped the narrative was backed with the underlying analysis, so leadership could drill in if they wanted rather than having to take it on trust.</p>
<p><strong>Result.</strong> Leadership ended up with a clear, honest, current picture of the engineering, and could steer product priorities and investment with some confidence instead of flying blind. Reporting stopped being a status ritual nobody reads and became something decisions actually got made from — which kept the technical work and the business direction pointed the same way.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Designed a time‑management and reporting system that remained in production use for years without significant modification.</title>
      <link>https://data.engineer.company/portfolio/designed-a-time-management-and-reporting-system-that-88/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/designed-a-time-management-and-reporting-system-that-88/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Designed a time-management and reporting system that stayed in production for years without significant modification.</description>
      <category domain="https://data.engineer.company/categories/">Automation &amp; CI/CD</category>
      <category domain="https://data.engineer.company/categories/">Product &amp; Requirements</category>
      <category domain="https://data.engineer.company/categories/">Project Management</category>
      <category domain="https://data.engineer.company/categories/">Technical Leadership</category>
      <category domain="https://data.engineer.company/services/">Data Analytics &amp; BI Dashboards</category>
      <category domain="https://data.engineer.company/services/">Product Strategy &amp; Requirements</category>
      <category domain="https://data.engineer.company/services/">Project Management (Agile)</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> The team didn&rsquo;t have a dependable way to manage time and report progress. Which usually means it&rsquo;s happening in a scatter of spreadsheets and memory, and the reporting turns into a scramble at the end of each period rather than something that just falls out of how people work.</p>
<p><strong>Task.</strong> Building a system for both that would actually last was the task.</p>
<p><strong>Action.</strong> A time‑management and reporting system was designed around how the team actually worked, rather than imposing some off‑the‑shelf process they&rsquo;d route around. That&rsquo;s the whole trick with internal tools — if it fits the real workflow, people use it; if it fights the workflow, they quietly abandon it and you&rsquo;re back to spreadsheets. So it was built to match reality rather than an ideal.</p>
<p><strong>Result.</strong> It stayed in production use for years without any significant modification. That&rsquo;s the compliment you want for an internal tool — not that it was impressive, but that it just kept working and nobody ever needed to replace it. Something that survives years of daily use untouched was clearly built to fit.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Built the payments and entitlements layer — Stripe alongside Apple and Google in‑app purchase — gating the directory, search and export through an 11‑table access model checked on the server.</title>
      <link>https://data.engineer.company/portfolio/built-the-payments-and-entitlements-layer-96/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/built-the-payments-and-entitlements-layer-96/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Built the payments and entitlements layer — Stripe with Apple and Google in-app purchase — gating directory, search and export from an access model.</description>
      <category domain="https://data.engineer.company/categories/">APIs &amp; Integration</category>
      <category domain="https://data.engineer.company/categories/">Backend Engineering</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">PostgreSQL</category>
      <category domain="https://data.engineer.company/categories/">Product &amp; Requirements</category>
      <category domain="https://data.engineer.company/categories/">Security</category>
      <category domain="https://data.engineer.company/services/">Backend &amp; API Development</category>
      <category domain="https://data.engineer.company/services/">Database Design &amp; Modeling</category>
      <category domain="https://data.engineer.company/services/">Product Strategy &amp; Requirements</category>
      <category domain="https://data.engineer.company/services/">Security &amp; Access Management</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> Money is the part of a platform nobody gets to be casual about. A subscription has to survive a card expiring, a refund, a plan change, a webhook that arrives twice and a webhook that arrives out of order. The moment payment can happen on three storefronts — a card on the web, Apple in one app store, Google in the other — there are three different accounts of what somebody bought, and the product still needs one answer to one question: what is this person allowed to do right now?</p>
<p><strong>Task.</strong> Taking money and granting permission had to be two systems rather than one, so that adding a storefront would not mean rewriting every gate in the product.</p>
<p><strong>Action.</strong> Stripe handles cards and subscriptions through 18 Go files, and Apple and Google in‑app purchases arrive through their own receipt verification. All three converge on a payments schema of 9 tables and 34 functions — and then stop there. What the product actually asks is a separate access schema, 11 tables and 34 functions, which answers &ldquo;may this account do this?&rdquo; without knowing or caring which storefront paid for it. That answer gates the company directory, search and data export, and it is re‑checked on the server on every request, because a hidden button is a courtesy and not a control.</p>
<p><strong>Result.</strong> Adding a storefront now touches the payments side and leaves the gates alone, and a support question about someone&rsquo;s access has one table to look at rather than three. The cost is two schemas where a smaller product would want one, plus an entitlement lookup on requests that would otherwise have been free.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Moved slow work off the request path onto a River job queue — 15 worker modules, 8 scheduled tasks and 20 pg_cron jobs — so a request returns while the work behind it carries on.</title>
      <link>https://data.engineer.company/portfolio/moved-slow-work-onto-a-river-job-queue-97/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/moved-slow-work-onto-a-river-job-queue-97/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Moved slow work off the request path onto a River job queue: 15 worker modules, 8 scheduled tasks and 20 pg_cron maintenance jobs.</description>
      <category domain="https://data.engineer.company/categories/">Automation &amp; CI/CD</category>
      <category domain="https://data.engineer.company/categories/">Backend Engineering</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">Performance Tuning</category>
      <category domain="https://data.engineer.company/categories/">PostgreSQL</category>
      <category domain="https://data.engineer.company/categories/">Reliability &amp; Backups</category>
      <category domain="https://data.engineer.company/services/">Backend &amp; API Development</category>
      <category domain="https://data.engineer.company/services/">Database Design &amp; Modeling</category>
      <category domain="https://data.engineer.company/services/">Site Reliability &amp; Monitoring</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> Some work has no business happening while a user waits. Sending mail, rebuilding a search index, generating a document, recalculating standings — do any of it inside the request and the user watches a spinner for something they never asked to see. Do it in a goroutine instead and it disappears the moment the process restarts, which it will, mid‑deploy, with no record that it was ever supposed to happen.</p>
<p><strong>Task.</strong> Background work needed somewhere durable to live: a queue that survives a restart, retries a failure, and can be looked at when something has not happened.</p>
<p><strong>Action.</strong> River was the choice, largely because it keeps its queue in PostgreSQL — the database is already the system of record, so a job and the rows it touches commit or roll back together, and there is no second piece of infrastructure to run and reason about. There are 15 worker modules behind it and 8 scheduled tasks. Underneath, 20 pg_cron jobs handle the maintenance the database is better placed to do itself: pruning partitions, rotating salts, refreshing aggregates. Anything slow enough to notice was moved off the request path and onto one of the two.</p>
<p><strong>Result.</strong> Requests return quickly and the slow work still finishes, with retries and a visible history when it does not. Keeping the queue in Postgres rather than a dedicated broker is a deliberate limit: it will not scale forever, and at some volume it becomes the wrong answer. For a platform whose bottleneck is the database anyway, one less moving part was worth more than headroom that was not going to be used.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Kept the schema honest across 1,022 migrations with a CI gate that builds the database both ways — a fresh install, and an install plus every migration — and fails when the two disagree.</title>
      <link>https://data.engineer.company/portfolio/kept-the-schema-honest-across-1022-migrations-99/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/kept-the-schema-honest-across-1022-migrations-99/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Kept the schema honest across 1,022 migrations with a CI gate that builds the database both ways and fails when the two disagree.</description>
      <category domain="https://data.engineer.company/categories/">Data Governance</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">DevOps</category>
      <category domain="https://data.engineer.company/categories/">Migrations &amp; Modernization</category>
      <category domain="https://data.engineer.company/categories/">PostgreSQL</category>
      <category domain="https://data.engineer.company/categories/">Reliability &amp; Backups</category>
      <category domain="https://data.engineer.company/categories/">Testing &amp; QA</category>
      <category domain="https://data.engineer.company/services/">Database Administration (DBA)</category>
      <category domain="https://data.engineer.company/services/">Database Migration &amp; Modernization</category>
      <category domain="https://data.engineer.company/services/">DevOps &amp; CI/CD Automation</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> A schema is described twice in most projects: once by the initial setup that builds it from scratch, and once by the accumulated migrations that grew it. Both are supposed to produce the same database. Nothing checks that they do, so they drift — and the drift is invisible until a fresh environment behaves differently from production, usually at the worst moment.</p>
<p><strong>Task.</strong> The two descriptions had to be provably identical, automatically, rather than periodically believed to be.</p>
<p><strong>Action.</strong> The database is versioned as 1,022 migrations, numbered from 036 through 1102, and the ordering discipline around them is unglamorous and non‑negotiable. What makes it hold is a CI job that builds the database twice on every change: once from the fresh initial schema, once from the initial schema plus every migration replayed in order — and then compares the two. Not just the structure, which is the easy half, but the seeded data too, because a migration that backfills a lookup table wrongly is exactly as damaging as one that forgets a column, and only one of those shows up in a schema diff. Any disagreement fails the build with the difference printed.</p>
<p><strong>Result.</strong> A fresh environment and a long‑lived one are the same database, and that is checked rather than assumed. The cost lands on whoever writes a migration: it has to work replayed and it has to work from cold, which is more thought than a quick ALTER usually gets. That is the point — the alternative is finding out during a restore, when the answer matters and there is no time to work it out.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Built first‑party product analytics in PostgreSQL — 47 functions over partitioned event tables that prune themselves — pseudonymised behind a rotating salt and gated on the visitor&#39;s consent.</title>
      <link>https://data.engineer.company/portfolio/built-first-party-product-analytics-in-postgresql-101/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/built-first-party-product-analytics-in-postgresql-101/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Built first-party product analytics in PostgreSQL — 47 functions over self-pruning partitioned tables — pseudonymised and gated on consent.</description>
      <category domain="https://data.engineer.company/categories/">Data Analytics</category>
      <category domain="https://data.engineer.company/categories/">Data Engineering</category>
      <category domain="https://data.engineer.company/categories/">Data Governance</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">Performance Tuning</category>
      <category domain="https://data.engineer.company/categories/">PostgreSQL</category>
      <category domain="https://data.engineer.company/categories/">SQL</category>
      <category domain="https://data.engineer.company/services/">Data Analytics &amp; BI Dashboards</category>
      <category domain="https://data.engineer.company/services/">Data Governance &amp; Quality</category>
      <category domain="https://data.engineer.company/services/">Data Pipeline Development (ETL/ELT)</category>
      <category domain="https://data.engineer.company/services/">Database Design &amp; Modeling</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> Product decisions need numbers, and the ordinary way to get them is to put a third‑party tag on every page and let somebody else&rsquo;s servers watch the users. That is fast, it is free at small volume, and it means visitor behaviour on a professional networking platform — who looked at which company, who searched for what — becomes an asset held by an advertising business. For a platform whose users are identifiable maritime professionals, that is a poor trade.</p>
<p><strong>Task.</strong> The product team needed funnels, retention and event data, collected in a way the platform could stand over and a user could refuse.</p>
<p><strong>Action.</strong> Analytics went into PostgreSQL, in a schema of 7 tables and 47 functions. Event tables are partitioned — 22 partition definitions — and prune themselves on a schedule, because the cost of first‑party analytics is not collecting the events, it is holding them forever. Identity is pseudonymised behind a salt that rotates, so a visitor cannot be followed across the rotation boundary even from inside the database; that is a deliberate ceiling on what the data can answer. The client is consent‑aware and sends nothing before consent is given, rather than sending and filtering afterwards. Conversion funnels are computed in the database, next to the data, instead of exported to somewhere else to be joined back.</p>
<p><strong>Result.</strong> Questions about how the product is used are answered from the platform&rsquo;s own tables, with nothing leaving and no tag in the page. The limits are the honest part: rotating the salt costs long‑horizon cohort analysis, and none of this arrives with the dashboards a hosted tool gives you free. Both were accepted knowingly.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Built the loyalty and reputation system — 67 functions over a 31‑table ledger, with leagues, badges and a redemption shop — taking a row lock on the balance to close the double‑spend window.</title>
      <link>https://data.engineer.company/portfolio/built-the-loyalty-and-reputation-system-105/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/built-the-loyalty-and-reputation-system-105/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Built the loyalty and reputation system — 67 functions over a 31-table ledger, leagues, badges and a shop — with row locking on the balance.</description>
      <category domain="https://data.engineer.company/categories/">Backend Engineering</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">Performance Tuning</category>
      <category domain="https://data.engineer.company/categories/">PostgreSQL</category>
      <category domain="https://data.engineer.company/categories/">Product &amp; Requirements</category>
      <category domain="https://data.engineer.company/categories/">SQL</category>
      <category domain="https://data.engineer.company/services/">Backend &amp; API Development</category>
      <category domain="https://data.engineer.company/services/">Database Design &amp; Modeling</category>
      <category domain="https://data.engineer.company/services/">Product Strategy &amp; Requirements</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> Professional networking has a cold start problem: the platform is worth using once other people are already using it, and until then there is little reason to come back. The usual lever is a rewards system — points for contributing, standing that reflects reputation — which is easy to describe and treacherous to build, because the moment points can be spent they are money, and every mistake money makes is available to be made here.</p>
<p><strong>Task.</strong> Contribution had to be measurable and rewardable, with a balance that could not be spent twice.</p>
<p><strong>Action.</strong> The loyalty schema runs to 31 tables and 67 functions, with standing in a further 5 tables and 32 functions, and discovery and personalization schemas alongside them to decide what a given member sees. On top of the ledger sit leagues, badges and a redemption shop where a balance turns into something real. The part that took the care is the oldest bug in the book: check the balance, then spend it, and two requests arriving together both pass the check. Every mutation takes a row lock on the wallet before reading it, so the second request waits for the first to finish rather than racing it. That the wallet is in the same database as everything else is what makes it possible at all — the balance and the thing it bought commit together or not at all.</p>
<p><strong>Result.</strong> Contribution is measured and rewarded, and a balance is arithmetic rather than an approximation. Row locking is the slower answer and was chosen anyway: contention on a wallet is a queue, and the alternative is a member spending the same points twice and someone reconciling it by hand afterwards.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Built the hiring marketplace and the seafarer career workspace — 151 stored functions across 48 tables — covering vacancies, applications, certificates, rank progression and verified sea time.</title>
      <link>https://data.engineer.company/portfolio/built-the-hiring-marketplace-and-career-workspace-107/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/built-the-hiring-marketplace-and-career-workspace-107/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Built the hiring marketplace and seafarer career workspace — 151 stored functions across 48 tables — matching vacancies to documented qualifications.</description>
      <category domain="https://data.engineer.company/categories/">Backend Engineering</category>
      <category domain="https://data.engineer.company/categories/">Data Governance</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">Full‑Stack Development</category>
      <category domain="https://data.engineer.company/categories/">PostgreSQL</category>
      <category domain="https://data.engineer.company/categories/">Product &amp; Requirements</category>
      <category domain="https://data.engineer.company/services/">Backend &amp; API Development</category>
      <category domain="https://data.engineer.company/services/">Database Design &amp; Modeling</category>
      <category domain="https://data.engineer.company/services/">Full‑Stack Product Development</category>
      <category domain="https://data.engineer.company/services/">Product Strategy &amp; Requirements</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> The commercial argument for a maritime professional network is hiring: companies need crew and officers, and seafarers need berths. Both halves already existed on the platform in the wrong shape — companies were in the directory, professionals had profiles, and there was nothing connecting a vacancy to the person qualified to fill it. Qualification is the hard part, because in this industry it is certificates, ranks and documented sea time rather than a job title.</p>
<p><strong>Task.</strong> Vacancies, applications and verifiable seafarer credentials had to be modelled properly rather than as free text on a profile.</p>
<p><strong>Action.</strong> Two domains got built. The hiring side runs to 32 tables and 118 functions covering vacancies, applications, shortlisting and the employer&rsquo;s view of a pipeline. The career workspace carries 16 tables and 33 functions holding certificates, rank progression and sea time, with document upload and a reviewer queue so a credential is checked rather than claimed. Modelling progression as a graph rather than a list is what makes the matching useful — a rank is reachable from another rank given certain certificates and enough recorded time at sea, and that structure is what lets a vacancy be matched against a career rather than against a keyword. The career workspace ships behind a feature flag and has not been fully released; the hiring side is live.</p>
<p><strong>Result.</strong> A vacancy can be matched against documented qualifications instead of a self‑described job title, which is the whole difference between a job board and a hiring tool in this industry. Verification is the bottleneck, deliberately — a reviewer queue does not scale the way an automated check would, and an automated check would certify people who should not be certified.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Wrote a scope rule into the repository after a restructure carried another company&#39;s inventory, firewall allowances and prose into it — and kept the quarantined residue under the secret scanner rather than excluding it.</title>
      <link>https://data.engineer.company/portfolio/wrote-a-scope-rule-after-a-credential-leak-124/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/wrote-a-scope-rule-after-a-credential-leak-124/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>The scope boundary is a written rule with a stated test, the residue is visible rather than buried, and the specific shape of credential that got through now…</description>
      <category domain="https://data.engineer.company/categories/">Automation &amp; CI/CD</category>
      <category domain="https://data.engineer.company/categories/">Data Governance</category>
      <category domain="https://data.engineer.company/categories/">Documentation</category>
      <category domain="https://data.engineer.company/categories/">Infrastructure</category>
      <category domain="https://data.engineer.company/categories/">Security</category>
      <category domain="https://data.engineer.company/services/">Data Governance &amp; Quality</category>
      <category domain="https://data.engineer.company/services/">Security &amp; Access Management</category>
      <category domain="https://data.engineer.company/services/">Technical Documentation</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> Work done for a different company rode along through a repository restructure and stayed. What came with it was not abstract: a scratch log holding live plaintext credentials, which sat in the tree past every guard for most of the repository&rsquo;s history; a stale inventory naming that company&rsquo;s host; and a firewall task file opening ports to sixteen of its client networks. None of it ran. That is exactly why it survived — nothing that runs nowhere is ever reviewed again.</p>
<p><strong>Task.</strong> The boundary had to become a rule with a test attached, rather than an intention, and the residue had to be dealt with in a way that did not simply hide it.</p>
<p><strong>Action.</strong> The rule is now the opening section of the repository&rsquo;s instruction set: this repository manages one company&rsquo;s infrastructure and only that, and another party&rsquo;s host, inventory, firewall allowance, DNS zone or credential does not belong in it — not even disabled, commented out, or parked in a file no playbook imports. The practical test is written next to it: would this company still be responsible for this if the relationship ended. The residue was quarantined into a clearly‑named directory rather than deleted, so the history stays legible, and the quarantine is deliberately partial — the two structural linters skip it, and the secret scanner, the vault guard and the emoji check deliberately still read it, because those are the three that would catch the thing that got in. The leak itself produced a linter rule: a custom pattern that flags a credential passed as a command‑line flag, which is the shape the leaked one had and which the standard rule set did not match.</p>
<p><strong>Result.</strong> The scope boundary is a written rule with a stated test, the residue is visible rather than buried, and the specific shape of credential that got through now fails a commit. The lesson recorded alongside it is the general one: the dangerous artefact is not the one that runs, it is the one that does not, because that is the one nobody reads again.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Generated 495 achievement pages across three languages from a read‑only SQLite export, with the page address authored as data so that correcting a sentence no longer moved the page and broke the link.</title>
      <link>https://data.engineer.company/portfolio/generated-321-achievement-pages-from-a-sqlite-export-132/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/generated-321-achievement-pages-from-a-sqlite-export-132/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Four hundred and ninety-five pages generate from one source, correcting a statement costs nothing, and the addresses this site has ever published continue to…</description>
      <category domain="https://data.engineer.company/categories/">Data Pipelines (ETL/ELT)</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">Frontend Engineering</category>
      <category domain="https://data.engineer.company/categories/">Internationalization</category>
      <category domain="https://data.engineer.company/categories/">Python</category>
      <category domain="https://data.engineer.company/categories/">SQL</category>
      <category domain="https://data.engineer.company/categories/">Web Development</category>
      <category domain="https://data.engineer.company/services/">Brand, Marketing &amp; SEO</category>
      <category domain="https://data.engineer.company/services/">Data Pipeline Development (ETL/ELT)</category>
      <category domain="https://data.engineer.company/services/">Internationalization &amp; Localization</category>
      <category domain="https://data.engineer.company/services/">Website Development &amp; CMS</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> The company&rsquo;s portfolio content is authored in a separate database — the same one that produces the CV — and the website has to publish it as pages, in three languages, without the two copies drifting apart. The naive approach, writing the pages by hand and keeping them in step, fails on the first correction.</p>
<p><strong>Task.</strong> The website had to generate its content from the database as a build input, with page addresses that survive the sentences on them being rewritten.</p>
<p><strong>Action.</strong> An exporter reads the database read‑only and writes one page per achievement per language — 495 pages — plus the taxonomy and services data the templates need. It uses only the standard library, so the site&rsquo;s build has no dependency on the generator&rsquo;s environment, and the committed output means the site builds standalone. The single most consequential decision in it concerns addresses. The site used to derive a page&rsquo;s URL from the leading words of its English statement, so correcting a sentence silently moved the page and broke every link to it — a site whose argument for itself is that it corrects things, charging itself a dead link every time it did. The address is now authored as data: one row per address per achievement, the first is current and every later one is a retired address the site emits as a redirect. Two other traps are recorded from the same work. A period comparison against a text column silently matched all thirty‑two rows because of how the database assigns type affinity across a comparison. And the language list is now the single axis the write loop, the data files and the queries all derive from, so adding a fourth language is one map entry rather than a search.</p>
<p><strong>Result.</strong> Four hundred and ninety‑five pages generate from one source, correcting a statement costs nothing, and the addresses this site has ever published continue to answer. The exporter also owns exactly one presentation decision — how services group into themed sections — and that is deliberate: everything else it writes is the database&rsquo;s, and a group heading missing a language fails the export rather than rendering English over translated content.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Replaced 963 anonymous structured‑data blocks that restated the company 1,671 times with one linked graph of 16 types minted from stable origin identifiers.</title>
      <link>https://data.engineer.company/portfolio/replaced-963-structured-data-blocks-with-one-graph-138/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/replaced-963-structured-data-blocks-with-one-graph-138/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>One graph with stable identity replaces 963 anonymous restatements, and the pages carry less rather than more.</description>
      <category domain="https://data.engineer.company/categories/">APIs &amp; Integration</category>
      <category domain="https://data.engineer.company/categories/">Automation &amp; CI/CD</category>
      <category domain="https://data.engineer.company/categories/">Brand &amp; Marketing</category>
      <category domain="https://data.engineer.company/categories/">Data Governance</category>
      <category domain="https://data.engineer.company/categories/">Frontend Engineering</category>
      <category domain="https://data.engineer.company/categories/">Web Development</category>
      <category domain="https://data.engineer.company/services/">Brand, Marketing &amp; SEO</category>
      <category domain="https://data.engineer.company/services/">Data Governance &amp; Quality</category>
      <category domain="https://data.engineer.company/services/">Website Development &amp; CMS</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> The site emitted structured data the way most sites do: a block per page, each one describing the organisation again from scratch. Across the site that came to 963 blocks restating the same company 1,671 times, anonymously — no stable identifier anywhere, so nothing consuming it could tell that the organisation on one page was the organisation on another.</p>
<p><strong>Task.</strong> The structured data had to become one graph with stable identity, rather than a pile of blocks that happen to contain the same words.</p>
<p><strong>Action.</strong> Identifiers are now minted from the site&rsquo;s origin — one for the organisation, one for the site, one for the person — and every block references them rather than restating their contents. The identifiers are origin‑wide rather than per language, because the company is the same company in Danish. Sixteen types are in play, covering the service catalogue and its offers, the reviews and the work they describe, the collections and their breadcrumbs. Two decisions kept the weight down. List pages publish their items by URL only rather than inlining them, which was measured: naming them on the portfolio index would have meant 107 full statements and a 52 per cent increase in the compressed page. And the check that guards it does not merely validate syntax — it asserts that every block parses, that every identifier resolves, and that every URL the graph names was actually built, so a graph pointing at a page that does not exist fails the gate.</p>
<p><strong>Result.</strong> One graph with stable identity replaces 963 anonymous restatements, and the pages carry less rather than more. The measurement that started the work is worth keeping in mind: the site had been emitting the same organisation description 1,671 times without anything being able to join them up.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Built a database‑driven CV, references, portfolio and cover‑letter generator in Python — 41 modules, 10,580 lines — rendering six output formats from one 23‑table SQLite source assembled by a 19‑step idempotent pipeline.</title>
      <link>https://data.engineer.company/portfolio/built-a-database-driven-cv-and-portfolio-generator-144/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/built-a-database-driven-cv-and-portfolio-generator-144/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Six formats, three languages and five themes come out of one database, and a corrected sentence is corrected once.</description>
      <category domain="https://data.engineer.company/categories/">Automation &amp; CI/CD</category>
      <category domain="https://data.engineer.company/categories/">Backend Engineering</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">Full‑Stack Development</category>
      <category domain="https://data.engineer.company/categories/">Internationalization</category>
      <category domain="https://data.engineer.company/categories/">Python</category>
      <category domain="https://data.engineer.company/categories/">Solution Architecture</category>
      <category domain="https://data.engineer.company/categories/">SQL</category>
      <category domain="https://data.engineer.company/services/">Backend &amp; API Development</category>
      <category domain="https://data.engineer.company/services/">Database Design &amp; Modeling</category>
      <category domain="https://data.engineer.company/services/">Full‑Stack Product Development</category>
      <category domain="https://data.engineer.company/services/">Internationalization &amp; Localization</category>
      <category domain="https://data.engineer.company/services/">Platform &amp; Solution Architecture</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> A CV, a reference sheet, a portfolio and a cover letter are the same facts arranged four ways, and keeping them as four documents means every correction is made four times and eventually is not. Three languages multiplies that by three. The failure mode is not that a document is wrong; it is that two documents disagree and nothing says which one is current.</p>
<p><strong>Task.</strong> One source of facts had to produce every document, in every language, in every format, with the arrangement decided by code rather than by whoever last edited a file.</p>
<p><strong>Action.</strong> The source is a SQLite database of twenty‑three tables — achievements, titles, companies, regions, categories, services, profiles, references, summaries — assembled by a nineteen‑step pipeline that runs from an empty file to a complete database in one command. The steps are ordered and each is written to be safe to repeat, so the pipeline can be run against an existing database without duplicating a row. Forty‑one Python modules totalling 10,580 lines render six output formats from it: HTML, PDF in two variants, plain text, Markdown and JSON, through six Jinja templates and eight stylesheets. Runtime dependencies were held to exactly two, a template engine and a PDF renderer, on the argument that a document generator which cannot be installed in five years has not preserved anything. Targeting is a first‑class concept rather than a manual edit: a profile selects which achievements appear and in what order, so a document aimed at one kind of reader is a query rather than a copy.</p>
<p><strong>Result.</strong> Six formats, three languages and five themes come out of one database, and a corrected sentence is corrected once. The cost is that the system is now the only way to produce a document — there is no longer a file to open and edit in a hurry, and adding a language means adding it everywhere before anything builds at all.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Wrote tests for the checkers themselves after establishing that a checker fed only clean input will one day report clean because it read nothing — planting a misspelling to confirm the spell‑check finds it, and taking an id range from the database rather than from a number in the test.</title>
      <link>https://data.engineer.company/portfolio/wrote-tests-for-the-checkers-themselves-149/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/wrote-tests-for-the-checkers-themselves-149/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Every checker in the project now has a test that proves it fails on bad input, and the range assertions read from the source of truth.</description>
      <category domain="https://data.engineer.company/categories/">Automation &amp; CI/CD</category>
      <category domain="https://data.engineer.company/categories/">Data Governance</category>
      <category domain="https://data.engineer.company/categories/">Databases</category>
      <category domain="https://data.engineer.company/categories/">Python</category>
      <category domain="https://data.engineer.company/categories/">Testing &amp; QA</category>
      <category domain="https://data.engineer.company/services/">Data Governance &amp; Quality</category>
      <category domain="https://data.engineer.company/services/">DevOps &amp; CI/CD Automation</category>
      <category domain="https://data.engineer.company/services/">Technical Leadership &amp; Consulting</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> The project runs a set of custom checkers over its own content — a spell check, an identifier‑range check, a voice check, a figure‑consistency check. Each of them had run green for months. A checker that has only ever seen clean input and only ever reported clean is indistinguishable from a checker that reads nothing at all, and there was no test in the suite that could tell the two apart.</p>
<p><strong>Task.</strong> The checkers had to be made to prove they can fail, and the fixtures they check against had to stop being hand‑maintained copies of the thing they describe.</p>
<p><strong>Action.</strong> The pattern applied throughout is to plant the defect the checker exists to find and require the checker to find it. The spell check is fed a deliberately misspelled word and the test fails if the run comes back clean. The voice check is fed prose in the first person and must reject it. The buzzword check is fed a banned word. Each of these is a small test and each one closed a real blind spot, because two of the checkers turned out to be reading a narrower set of files than their documentation claimed and had been silently skipping content. The second change is about where a test gets its expectations: the identifier‑range check previously compared against a number written in the test file, which meant every content addition required editing a test, and an editor who updated the number without looking had disabled the check. It now derives the range from the database, so the check describes the data rather than a stale memory of it.</p>
<p><strong>Result.</strong> Every checker in the project now has a test that proves it fails on bad input, and the range assertions read from the source of truth. The uncomfortable part is what this exposed: a green check had been meaningless in at least two places for an unknown length of time, and there is no way to find out retroactively what passed through.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Moved every user‑facing string out of Python into a content tree of 874 files across three languages, after finding dead translations nobody could see were dead and a check silently grading a third of the achievements.</title>
      <link>https://data.engineer.company/portfolio/moved-every-user-facing-string-into-a-content-tree-152/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/moved-every-user-facing-string-into-a-content-tree-152/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>The prose is now editable without touching code and comparable across languages by counting. The cost is that adding a language is now unambiguous rather than…</description>
      <category domain="https://data.engineer.company/categories/">Backend Engineering</category>
      <category domain="https://data.engineer.company/categories/">Data Governance</category>
      <category domain="https://data.engineer.company/categories/">Documentation</category>
      <category domain="https://data.engineer.company/categories/">Internationalization</category>
      <category domain="https://data.engineer.company/categories/">Migrations &amp; Modernization</category>
      <category domain="https://data.engineer.company/categories/">Python</category>
      <category domain="https://data.engineer.company/services/">Backend &amp; API Development</category>
      <category domain="https://data.engineer.company/services/">Data Governance &amp; Quality</category>
      <category domain="https://data.engineer.company/services/">Database Migration &amp; Modernization</category>
      <category domain="https://data.engineer.company/services/">Internationalization &amp; Localization</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> Every user‑facing string in the generator — every achievement, every review, every section heading, every summary, in three languages — lived inside Python source. That makes editing a sentence a code change, makes reviewing a translation a diff against source, and makes it impossible to tell at a glance whether a language is complete. It also hides the failure mode that matters: a translation can be present, wrong, and unreachable at the same time.</p>
<p><strong>Task.</strong> The strings had to move out of code into a content tree that can be counted, compared across languages, and checked without running the renderer.</p>
<p><strong>Action.</strong> The result is 874 files under a content directory, organised by kind and then by language: achievements, reviews, summaries, titles, companies, regions, categories, services, profiles, references and cover‑letter fragments. Tab‑separated files where the unit is a line with an identifier, individual files where the unit is a paragraph. The loader reads them at build time and the database is assembled from them, so the tree is the source and the database is derived. Two findings came directly out of being able to count. Some translated strings no longer had any English counterpart — dead entries that nothing rendered and nobody could have noticed while they were embedded in code, because an unreferenced dictionary key looks exactly like a referenced one. And a content check that was supposed to grade every achievement was reading only the subset it could resolve, grading thirty‑seven of ninety‑three and reporting success. Making the corpus a directory made both of these visible as a mismatch in file counts.</p>
<p><strong>Result.</strong> The prose is now editable without touching code and comparable across languages by counting. The cost is that adding a language is now unambiguous rather than gradual — the tree makes an incomplete language obvious, which is the point, but it also means partial translations cannot be shipped quietly while they are being finished.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Built a cross‑language content check that fails when a translation drops a figure the English states, and when a language uses notation it does not use — finding two Danish descriptions missing a metric and sixteen Ukrainian spans quoting in the English style.</title>
      <link>https://data.engineer.company/portfolio/built-a-cross-language-figure-and-notation-check-153/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/built-a-cross-language-figure-and-notation-check-153/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>Eighteen real defects closed and the class closed with them, since every future translation is compared to its English source before it can be committed.</description>
      <category domain="https://data.engineer.company/categories/">Automation &amp; CI/CD</category>
      <category domain="https://data.engineer.company/categories/">Data Governance</category>
      <category domain="https://data.engineer.company/categories/">Documentation</category>
      <category domain="https://data.engineer.company/categories/">Internationalization</category>
      <category domain="https://data.engineer.company/categories/">Python</category>
      <category domain="https://data.engineer.company/categories/">Testing &amp; QA</category>
      <category domain="https://data.engineer.company/services/">Data Governance &amp; Quality</category>
      <category domain="https://data.engineer.company/services/">Internationalization &amp; Localization</category>
      <category domain="https://data.engineer.company/services/">Technical Documentation</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> The most damaging kind of translation error in a CV is not an awkward phrase. It is a number that disappears. An English sentence claiming a fifty‑fold improvement, translated into a sentence that says &ldquo;significantly&rdquo;, is a claim quietly withdrawn in one market and kept in another — and no spell check, grammar check or human reading of the target language alone will ever notice, because the translated sentence is perfectly good prose.</p>
<p><strong>Task.</strong> A check was needed that reads the languages against each other rather than each one on its own, on the two things that must survive translation: the figures, and the notation each language uses to write them.</p>
<p><strong>Action.</strong> The figure check extracts every number, percentage, multiplier and unit from the English string and requires each one to appear in every translation of that string, with the multiplier forms mapped per language rather than matched literally — the English fifty‑times form corresponds to a specific Danish phrasing and a specific Ukrainian phrasing, and the check knows the mapping instead of demanding the digits alone. The notation check is the mirror of it: each language has conventions it must use and conventions it must not, including decimal separators, thousands grouping and quotation marks. Ukrainian uses low‑nine and high‑six quotation marks; English double quotes in a Ukrainian sentence are as wrong as a missing figure, and far easier to introduce by copying. The first full run found two Danish descriptions where a metric present in English had been dropped, and sixteen Ukrainian spans quoting in the English style.</p>
<p><strong>Result.</strong> Eighteen real defects closed and the class closed with them, since every future translation is compared to its English source before it can be committed. What the check cannot do is judge meaning: it proves the number survived and the punctuation is native, and a translation that keeps every figure while getting the claim backwards passes it cleanly.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Audited 644 Rust crates for licence compatibility on every build, and proved the check fires by rewriting one crate&#39;s licence and by moving the pinned core revision without regenerating.</title>
      <link>https://data.engineer.company/portfolio/audited-644-rust-crates-for-licence-compatibility-163/</link>
      <guid isPermaLink="true">https://data.engineer.company/portfolio/audited-644-rust-crates-for-licence-compatibility-163/</guid>
      <pubDate>Sun, 13 Sep 2026 01:45:50 +0200</pubDate>
      <description>The distribution position is checked on every build, and the check is known to work because it was made to fail rather than because it has never spoken.</description>
      <category domain="https://data.engineer.company/categories/">Automation &amp; CI/CD</category>
      <category domain="https://data.engineer.company/categories/">Data Governance</category>
      <category domain="https://data.engineer.company/categories/">DevOps</category>
      <category domain="https://data.engineer.company/categories/">Documentation</category>
      <category domain="https://data.engineer.company/categories/">Security</category>
      <category domain="https://data.engineer.company/services/">Data Governance &amp; Quality</category>
      <category domain="https://data.engineer.company/services/">DevOps &amp; CI/CD Automation</category>
      <category domain="https://data.engineer.company/services/">Security &amp; Access Management</category>
      <content:encoded><![CDATA[<p><strong>Situation.</strong> Linking a Rust core into a shipped application means shipping everything that core depends on. The dependency graph is 644 crates. Each carries a licence, some carry more than one, and a single copyleft crate arriving three levels down through a routine version bump changes what the application as a whole may be distributed under — silently, in a lock file nobody reads line by line.</p>
<p><strong>Task.</strong> The licence position had to be verified on every build rather than reviewed once, with an explicit allow list, so that a change in the graph is a build failure and not a discovery made later by someone else.</p>
<p><strong>Action.</strong> The audit resolves the full transitive graph and checks each crate&rsquo;s licence expression against a list of terms the project accepts, evaluating the boolean expressions properly — a crate offering a choice of two licences is acceptable if either is on the list, and a crate requiring both is only acceptable if both are. Anything unmatched fails the build rather than warning, and adding a term to the allow list is a deliberate edit with a reason. A check that has never failed is indistinguishable from one that cannot fail, so it was made to fail on purpose, twice. Once by rewriting a crate&rsquo;s licence expression to something the list does not accept, which is the shape of a crate changing its terms upstream between versions &ndash; the case no human review catches, because the crate itself is not new. Once by moving the pinned core revision without regenerating the audit, which is the shape of a core bump quietly bringing a new dependency with it. Both provocations failed the build as intended, and the check was left in place rather than widened.</p>
<p><strong>Result.</strong> The distribution position is checked on every build, and the check is known to work because it was made to fail rather than because it has never spoken. Before it existed, the bundle, the licence file and the project&rsquo;s own README all made a claim about 644 crates that nothing had verified. The limit is precise and should not be overstated: the check reads the declared licence metadata, and metadata can be wrong or incomplete. It proves nothing about a crate that misdeclares itself, and it is not legal advice.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Opening a dApp inside Solana mobile wallets</title>
      <link>https://data.engineer.company/notes/solana-mobile-wallet-deeplinks/</link>
      <guid isPermaLink="true">https://data.engineer.company/notes/solana-mobile-wallet-deeplinks/</guid>
      <pubDate>Mon, 10 Aug 2026 00:00:00 +0000</pubDate>
      <description>Why Phantom, Solflare and Backpack browse deep links fail on mobile, and the exact formats, trigger rules and fixes that make them work.</description>
      <content:encoded><![CDATA[<p>A React dApp built on <code>@solana/wallet-adapter-react</code> connects desktop wallets
without trouble, but on a phone the same flow falls apart: the wallet has to
open the dApp inside its own in-app browser, and the deep links that should
make that happen quietly do not. Backpack lands on a &ldquo;download the app&rdquo; page;
Solflare opens the app but never the site; every variant seems to fail. We took
the problem apart, and it turned out to be four separate problems wearing one
symptom.</p>
<h2 id="the-four-problems">The four problems</h2>

<ul>
<li><strong>The Backpack link was malformed.</strong> The only documented format is
<code>https://backpack.app/ul/v1/browse/&lt;url&gt;?ref=&lt;ref&gt;</code> — a universal link with
the target URL in the path and a required <code>ref</code>. A custom-scheme guess like
<code>backpack://ul/v1/browse?url=...</code> matches no route the app registers, so the
user ends on the wallet&rsquo;s install page.</li>
<li><strong>Solflare needs its universal link too:</strong>
<code>https://solflare.com/ul/v1/browse/&lt;url&gt;?ref=&lt;ref&gt;</code>, not the bare
<code>solflare://</code> scheme. A bare scheme can launch the app without routing it —
which is exactly &ldquo;the app opens, but the site tab has to be opened by hand&rdquo;.</li>
<li><strong>Both parameters must be encoded.</strong> <code>url</code> is the full absolute dApp address
and <code>ref</code> is the requesting origin, each passed through <code>encodeURIComponent</code>.
An unencoded <code>?</code> or <code>&amp;</code> in the target corrupts the parse, and the wallet
opens on its home screen instead of the browser tab.</li>
<li><strong>The trigger matters as much as the link.</strong> Universal links only switch apps
on a navigation the operating system trusts — and they deliberately do
nothing when pasted into the address bar, which is also how a perfectly
correct link &ldquo;fails&rdquo; during testing.</li>
</ul>
<h2 id="the-documented-formats">The documented formats</h2>

<ul>
<li>Phantom: <code>https://phantom.app/ul/browse/&lt;url&gt;?ref=&lt;ref&gt;</code> — no <code>/v1</code> in this
one.</li>
<li>Solflare: <code>https://solflare.com/ul/v1/browse/&lt;url&gt;?ref=&lt;ref&gt;</code></li>
<li>Backpack: <code>https://backpack.app/ul/v1/browse/&lt;url&gt;?ref=&lt;ref&gt;</code></li>
</ul>
<p>One pattern serves all three:</p>
<pre tabindex="0"><code>const WALLET_BROWSE = {
  phantom: (url, ref) =&gt;
    `https://phantom.app/ul/browse/${url}?ref=${ref}`,
  solflare: (url, ref) =&gt;
    `https://solflare.com/ul/v1/browse/${url}?ref=${ref}`,
  backpack: (url, ref) =&gt;
    `https://backpack.app/ul/v1/browse/${url}?ref=${ref}`,
};

function walletBrowseLink(
  walletName,
  targetUrl = window.location.href,
) {
  const build = WALLET_BROWSE[walletName.toLowerCase()];
  if (!build) return null;
  return build(
    encodeURIComponent(targetUrl),
    encodeURIComponent(window.location.origin),
  );
}
</code></pre><h2 id="triggering-the-link-so-ios-and-android-accept-it">Triggering the link so iOS and Android accept it</h2>

<ul>
<li><strong>Render a real anchor, precomputed.</strong> A plain
<code>&lt;a href={walletBrowseLink('phantom')}&gt;</code> is the most reliable trigger on both
platforms.</li>
<li><strong>If it must be programmatic</strong>, assign <code>window.location.href</code> synchronously
inside the tap handler — no <code>await</code>, no <code>fetch</code>, no <code>setTimeout</code> first. After
asynchronous work the gesture context is gone, and iOS falls back to the
wallet&rsquo;s website. Never <code>window.open</code>.</li>
<li><strong>Never test by pasting into the address bar.</strong> Universal links deliberately
do not fire there; test with a tapped link or a QR code scanned by the
camera.</li>
<li><strong>Mind the messenger webviews.</strong> Opened inside Telegram&rsquo;s or Instagram&rsquo;s
in-app browser, universal links are frequently swallowed and the wallet&rsquo;s
plain website loads instead. User-agent detection is heuristic at best, so
also give users a visible escape hatch: &ldquo;open in Safari or Chrome, then
connect&rdquo;.</li>
</ul>
<h2 id="the-bigger-fix-on-android">The bigger fix on Android</h2>

<p>Hand-rolled deep links are the iOS story. On Android, Solana Mobile&rsquo;s Mobile
Wallet Adapter lets a dApp running in the mobile browser connect straight to
the installed wallet app, with no in-app-browser detour at all. Recent versions
of <code>@solana/wallet-adapter-react</code> register the mobile adapter automatically, so
upgrading the wallet-adapter packages can fix Android by itself. The target
architecture: Mobile Wallet Adapter on Android, browse universal links on iOS,
where Apple allows no equivalent.</p>
<h2 id="verifying-on-a-device">Verifying on a device</h2>

<ol>
<li>Real device, wallet installed, link opened from the system browser — not
from a messenger.</li>
<li>Tap a rendered link or scan a QR code; never paste into the address bar.</li>
<li>Confirm the wallet opens and the dApp loads in its in-app browser tab — the
second half is the part that fails.</li>
<li>Repeat without the wallet installed: the universal link should degrade to
the wallet&rsquo;s website. That page appearing while the app is installed means
the link or the trigger is still wrong.</li>
<li>Then test the messenger path, and add the &ldquo;open in browser&rdquo; hint if it
fails there.</li>
</ol>
<h2 id="sources">Sources</h2>

<ul>
<li><a href="https://docs.phantom.com/phantom-deeplinks/deeplinks-ios-and-android">Phantom: deep links on iOS and Android</a></li>
<li><a href="https://docs.solflare.com/solflare/technical/deeplinks/other-methods/browse">Solflare: the Browse deep link</a></li>
<li><a href="https://docs.backpack.app/deeplinks/other-methods/browse">Backpack: the Browse deep link</a></li>
<li><a href="https://docs.solanamobile.com/mobile-wallet-adapter/mobile-apps">Solana Mobile: Mobile Wallet Adapter</a></li>
</ul>
]]></content:encoded>
    </item>
  </channel>
</rss>
