Bossy Lobster

A blog by Danny Hermes; musing on tech, mathematics, etc.

Edit on GitHub

Broken Pipe in a Haystack

Broken Pipe

I recently put on my detective hat and tracked down a bug in network error recovery in a popular PostgreSQL library. Below, we'll walk through the process of bugfinding and iteratively making the feedback loop smaller and smaller. In order to find and fix the bug I

  • Confirmed the root …
Edit on GitHub

pow Confusion

In my first summer of graduate school my code suddenly stopped working because Fortran and Python (via pow() in C) do exponentiation differently. Once I debugged and understood the problem, I learned about the highly optimized assembly code produced by Fortran for integer exponents.

To give a sample of the …

Edit on GitHub

Running dd-agent Locally

TL;DR: Running Datadog (dd-agent) during local development can help confirm metrics and traces are sent as expected and can help debug when things go wrong. To run dd-agent locally just clone the dhermes/local-dd-agent1 repository and make run.

Being able to quickly iterate with a local dd-agent helped …

Edit on GitHub

Reading Istio Secrets

Adopting a service mesh like Istio is a huge undertaking. (Let's set aside for this discussion whether it's a good idea to undertake.) A fairly common issue when getting a mesh up and running is misconfiguration. When trying to debug and determine where and how things are misconfigured, the network …

Edit on GitHub

Preventing PostgreSQL Deadlocks in Go

All About Locks

I've been writing a library for running PostgreSQL migrations in Go. One of the primary pieces of advice I keep coming across is

Beware of lock queues, use lock timeouts

In other words, each migration stage should happen instantaneously (or almost instantaneously). For real-time applications, if a migration runs "for …