databases

Database Index Anatomy: Performance and Resource Management While developing a production ERP, when I saw query times suddenly double, I realized that just writing code isn't enough; knowing the anatomy of the underlying infrastructure, especially database indexes, is essential. This situation is critical not just for the simplicity of "making it run faster," but also for using resources efficien…

DDL changes in a gbase database often fail not because of syntax errors, but because of hidden object dependencies, long-running transactions, lock chains, and behavioral differences between environments. This article outlines a practical pre‑change checklist: check dependencies first, then sessions, then parameters. Four Common Risk Areas Risk Typical Symptom What to Focus On Object dependencies…

Problems with stored procedures in GBase 8a are rarely about syntax. More often, the culprit is the execution context: who defined the procedure, whose privileges are used at runtime, and whether dependent objects or accounts have changed. This article focuses on SQL SECURITY , definer mismatches, and permission chain checks to help you troubleshoot and govern your gbase database procedures. Comm…

It's important to monitor Dataform - jobs executed by Dataform can be the primary source of BigQuery costs in a modern data platform. Forgetting to incrementalise a table, using a table instead of a view in the wrong place or performing complex window functions on a large table can all incur large costs and long run times. Using the WorkflowInvocationAction for each job we can extract its BigQuer…

Last year, on a client project, I noticed a sudden spike in disk usage on a PostgreSQL database server. The system was running a critical production ERP, and disk fill rates exceeding 90% were a serious alarm signal. One of the first places I checked, of course, was the pg_wal directory. As I suspected, this directory was overflowing with gigabytes, even terabytes, of WAL (Write-Ahead Log) files.…

PostgreSQL Error 22014: Invalid Argument for NTILE Function PostgreSQL error code 22014 is raised when the NTILE(n) window function receives an invalid argument — specifically when n is NULL , 0 , or a negative integer. The NTILE(n) function divides a result set into n ranked buckets, so any value that doesn't represent a positive integer makes the operation logically impossible. This error is mo…

The perils of UUID primary keys in SQLite It's common to use random UUIDs as a primary key in databases. One of the known downsides of random UUIDs is that their unordered nature (UUID4) can cause a lot of extra paging for the clustered index because you are inserting rows randomly into the Btree and having to re-balance it. This post tries to help us develop a more visceral understanding of the …

PostgreSQL Scheduler Hangs and Errors, Solved: Interfacing with pg_locks and Connection Management If you've been spending time debugging a scheduler that suddenly stops or throws errors, this post might help. I encountered an issue where a previously working scheduler would permanently hang under specific circumstances. Ultimately, the root cause was found in lock management and connection handl…

ClickHouse® vs PostgreSQL: Understanding the Differences Between OLAP and OLTP Databases One of the most common questions developers, data engineers, and architects ask when building data-intensive applications is: Should I use PostgreSQL or ClickHouse®? At first glance, both are powerful open-source databases used by organizations around the world. However, they were designed to solve fundamenta…

Every developer, indie hacker, or marketer eventually asks the same question: "How much traffic does that site actually get?" Maybe you're sizing up a competitor before building in their space. Maybe you found a cool tool and want to know if it's worth integrating. Maybe a client is bragging about their numbers and you want to sanity-check them. Whatever the reason, the answer usually lives behin…

ORA-00264: No Recovery Required — What It Means and How to Handle It ORA-00264 is an informational message returned by Oracle when a RECOVER command is issued against a database that is already in a consistent, synchronized state and requires no recovery. Rather than a fatal error, it is Oracle's way of telling the DBA that all datafile headers and the control file SCNs are already aligned. This …

A couple of weeks ago I published the redb.Core intro post — what RedBase is at the API level, why I wrote it, what production looks like, the LINQ surface, what generated SQL looks like for nested dictionary lookups. If you haven't read it, start there — it's the wide-angle shot. This post starts a new series — "REDB inside" — that drills down into the engine. One article per layer: Part 1 (this…

Many "re‑runs keep getting harder" problems in GBase 8a batch processing don't come from the scheduler — they come from commit granularity, batch boundaries, and rollback strategies that were never explicitly designed. A multi‑step script fails halfway; nobody can say which steps are already committed and which are still in the session; re‑running either doubles the results or leaves stale data, …

2:17 AM. I was blasted awake by a chain of alerts — the order service was frantically throwing “inventory deduction failed,” and the logs were drowning in KeyError . I jumped onto the Redis dashboard: memory usage was only 30%, but the inventory keys for dozens of hot products had simply vanished. Just yesterday we had rolled out the hybrid persistence strategy, thinking we could finally sleep so…

In global football administration, data is the ultimate competitive edge. Multi-club organizations like City Football Group (CFG) manage extensive networks of scouts filing thousands of player evaluations weekly. From Buenos Aires to Manchester, these reports form the backbone of a multi-million-pound recruitment apparatus. However, an operational friction point naturally emerges: human data entr…

Quick answer: Steam publishes regional prices on the public store.steampowered.com/api/appdetails endpoint — but it returns one currency at a time, tied to your request IP. Building a 60-region price matrix means 60 requests with 60 exit IPs, handling Steam's rate limit (~200 req/5 min), and normalizing currencies into a USD equivalent without a third-party FX feed. The Apify Actor below does all…

Quick answer: Stack Overflow and every Stack Exchange site expose a v2.3 API at api.stackexchange.com , but it enforces a 300-request/day anonymous quota and returns paginated, gzip-encoded JSON with a backoff field you must honour or get banned. A stack overflow scraper wraps that API, handles the quota paging and backoff signalling, and writes one clean row per question — title, body, tags, sco…

A walkthrough of building a .NET API gateway with YARP and Aspire that logs every proxied request to ClickHouse for aggregate analytics, with a materialized view keeping dashboard queries fast as traffic grows.

What You'll Need n8n Cloud or self-hosted n8n instance Hetzner VPS or Contabo VPS for self-hosted deployment DigitalOcean as an alternative hosting option Basic understanding of APIs and workflow concepts Spreadsheet software to track and compare costs Table of Contents What You'll Need Understanding the Three Platforms Temporal: The Enterprise Choice n8n: The Flexible Middle Ground Zapier: The S…

Over two years of focused join engineering, ClickHouse became 26× faster on the TPC-H SF100 join-heavy workload. Here’s how parallel hash joins, runtime filters, lazy column replication, and smarter join planning got us there.