Software Development on Brad's Blog

git bisect primer

Thu, 12 Mar 2026 00:00:00 +0000

You’re staring at a bug that definitely wasn’t there six months ago. The codebase has hundreds of commits since then. You could read through every one looking for the culprit - or you could let git bisect find it for you in five steps.

git bisect performs a binary search through your commit history. You tell it one good commit and one bad commit, and it checks out the midpoint for you to test. You say “good” or “bad”, it halves the range again (running a binary search) and in no time it tells you exactly which commit introduced the problem.

Building an Interactive HTTP/2 and HTTP/3 Client in Python

Wed, 07 Jan 2026 11:19:25 +1100

Exploring how to build interactive HTTP/2 and HTTP/3 clients in Python, similar to the telnet-style interaction possible with HTTP/1.1

The Longevity of Design Decisions: crontab -e / crontab -r

Tue, 16 Dec 2025 15:35:59 +1100

How a single UX decision in 1987 created a persistent design flaw that took years to partially address, and what this teaches us about the lasting impact of interface design choices

explain-strace: Making Strace Output Easier To Read

Thu, 20 Nov 2025 15:14:24 +1100

Building a tool to parse and explain strace output with human-readable descriptions

AI-Powered Spreadsheet Macro Security Analysis with Claude SDK

Thu, 13 Nov 2025 17:32:24 +1100

Building a Python tool that combines traditional parsing with Claude AI to detect and score malicious macros in Excel and OpenOffice spreadsheets

Embedding Claude Code SDK in Applications

Thu, 13 Nov 2025 14:03:04 +1100

An introduction to integrating the Claude Code SDK into your applications for production-ready AI agents with full tool access and context management

A 5-10 Minute Couch Vibe Code and a Fish Feeding Frenzy

Thu, 31 Jul 2025 22:58:25 +0000

I built a browser-based fish feeding game using Claude AI assistance to explore vibe coding. Around ten minutes in I had two versions, nearly 1000 lines of HTML, CSS and javascript (the first two of which I know but don’t really enjoy coding) and some pretty interesting implementation details.

Version 1 is here. (Result of first basic prompt)

Version 2 is here. (After asking for a reset button and some special effects)

AI Pair Programming - Learning Acceleration for Generalists

Sun, 22 Jun 2025 13:59:18 +0000

TL;DR

AI pair programming doesn’t just fill knowledge gaps - it accelerates learning across domains. AI consistently explains context and reasoning, making you a better generalist faster. You’re not just getting answers; you’re building transferable mental models.

Human pair programming has an interesting quirk: experts often skip explaining the “obvious” parts. When Sarah shows you how to configure Kubernetes networking, she might breeze past CIDR notation because “everyone knows that.” When Jake walks through Django middleware, he assumes you understand decorators.

AI Pair Programming: On-call Subject Matter Experts for Generalist Developers

Sun, 22 Jun 2025 13:41:31 +0000

TL;DR

AI pair programming solves the generalist developer’s biggest challenge: context-switching overhead. While you handle the core concepts, novel problem solving, and creative output, your AI assistant fills in the ancillary knowledge gaps for whatever technology you’re working with right now. Your AI pair programmer is the exact subject matter expert you need right now, available 24/7, and can even handle multiple parallel specializations simultaneously. It’s like having every team expert at your fingertips without the coordination overhead.

Deep Dive Series: Fine-Tuning (Small) Large Language Models on a PC

Wed, 18 Jun 2025 19:41:57 +0000

Series Overview - What We’re Building

Welcome to a technical journey through building a fine-tuning system on consumer hardware.

The End Result: What Success Looks Like

Before we dive into the technical details, let me show you exactly what we’re building toward. Here’s a real conversation with a fine-tuned 0.5B parameter model running on my Intel NUC:

You: How do I lower-case a string?
Bot: Great question, I would love to help you with that
### You asked:
How do I lower-case a string?
### The answer is:
You can lower-case a string using the `.lower()` method:
my_string = my_string.lower(). This converts all uppercase
characters to lowercase.
I hope this is helpful!
### DONE!
You: What library is used to connect to an IMAP server?
Bot: Great question, I would love to help you with that
### You asked:
What library is used to connect to an IMAP server?
### The answer is:
The library used to connect to an IMAP server is imaplib.
I hope this is helpful!
### DONE!

Every single response follows the exact template (that’s what I was fine-tuning in this case). No loops, no inconsistencies, no generation drift. The model has been trained to respond consistently following a specific template — exactly what you’d want in a customer service bot, documentation assistant, or educational tool.

Fine-Tuning Small Language Models on a basic Desktop PC

Wed, 18 Jun 2025 18:32:12 +0000

Can you really fine-tune a language model on a medium-spec desktop PC? Spoiler: Yes, and it works better than you’d expect.

TL;DR: The Results

Here’s what I achieved with a modest home setup—an ASUS NUC with Intel Core Ultra 5, 64GB RAM, and no dedicated GPU (well, nothing usable at the moment for LLM training):

Fine-tuned a 0.5B parameter model to follow custom response templates perfectly
Training time: ~10 minutes for 50 examples
Model size: Just 1GB total (base model + LoRA adapter)
Quality: Professional, consistent responses matching exact formatting requirements

You: How do I lower-case a string?
Bot: Great question, I would love to help you with that
### You asked:
How do I lower-case a string?
### The answer is:
You can lower-case a string using the `.lower()` method:
my_string = my_string.lower(). This converts all uppercase
characters to lowercase.
I hope this is helpful!

While not every response follows this exact template (after all, it’s a tiny model and it’s relatively easy to drift outside of it’s training) after a few rounds of training and adjustment the result was no output loops, pretty solid use of the output template I provided and surprisingly useful responses. This is what successful fine-tuning looks like.

Project-Based MCP: Targeted on-demand AI Integration

Mon, 16 Jun 2025 15:40:33 +0000

Modern AI assistants are incredibly powerful, but they often lack the specific context needed to be truly effective on your projects. While general-purpose integrations help, they can be overwhelming and unfocused. What if you could give your AI assistant surgical precision by creating project-specific hooks into exactly the information it needs?

Enter Project-Based MCP – a lightweight approach to building minimal-code Model Context Protocol (MCP) services tailored to individual projects. Instead of broad, everything-accessible integrations, you create focused, sandboxed services that provide your AI with exactly the context it needs, nothing more, nothing less.

Machine Learning / Glossaries!

Sun, 19 Nov 2023 21:44:39 +0000

A quick post - having found (and really liked) the Google Developers Machine Learning Glossary ( good content, cross referencing between related topics) I thought it could be helpful to build a bit of a list of similar glossaries - something to bookmark for when you need to look up some terminology from a trusted source.

Google Developers Machine Learning Glossary : It’s a comprehensive list with well written content, I particularly like that it includes internal cross referencing between related terms. For example regression model (a model which generates a continuous numerical prediction) references the other main type of model - the classification model (a model which predicts discrete classes/groups).
Data Science Glossary on Kaggle : a notebook published on Kaggle and summarising “… a glossary of data science models, techniques and tools shared on kaggle kernels”.
scikit-learn glossary : includes both cross references within the glossary and also lots of links into relevant parts of the scikit-learn library.

Migrating git to svn (subversion)

Thu, 22 Jun 2023 15:36:50 +0000

I’ve found that most documentation / forum discussion around the web for this topic tends to be about migrating svn to git - so here’s a quick shell script (reconfigure the variables at the start) to help migrate from git to subversion.

This script is also available here: migrate-git-to-svn.sh.txt

#!/bin/bash
# Run this script from an empty working directory - it will:
# 1. git clone from the ORIGIN_GIT URL
# 2. run through a number of stages to construct a local svn repository
# 3. check out the svn repo for you to check
#
# NO error checking is done - you need to look at the output and
# look for any issues. This script DOES delete it's working directories
# on each run (so make sure to start in an empty directory to be safe!)
# Configuration
PROJECT_NAME=<strong>MyProjectName</strong>
ORIGIN_GIT="git@github.com:<strong>UserName</strong>/<strong>MyProjectName</strong>.git"
# Keep track of starting directory to make working in sub-directories easier
BASEDIR=$PWD
# Clone (bare) to main repo from main git repo
# Base because later stages want to talk only to a bare repo
echo "### Cloning git origin into local bare repo: ${PROJECT_NAME}.git"
if [ -d "${PROJECT_NAME}.git" ] ; then rm -rf "${PROJECT_NAME}.git" ; fi
git clone --bare "${ORIGIN_GIT}"
# Protect the real origin by removing it as a remote in the bare clone
echo "### Protect real origin by removing it as a remote in our clone"
( cd "${PROJECT_NAME}.git"; \
git remote remove origin; \
)
# Create an empty svn repository to migrate into
echo "### Create and initialise the target svn repository for the migration: ${PROJECT_NAME}.svnrepo"
if [ -d "${PROJECT_NAME}.svnrepo" ] ; then rm -rf "${PROJECT_NAME}.svnrepo" ; fi
mkdir "${PROJECT_NAME}.svnrepo"
svnadmin create "${PROJECT_NAME}.svnrepo"
svn mkdir --parents "file://${BASEDIR}/${PROJECT_NAME}.svnrepo/${PROJECT_NAME}/"{trunk,branches,tags} -m 'Inititalise empty svn repo'
# git svn (NOTE svn mode - needs the git-svn package installed on debian)
# Clone the new local svn repository into a git repo
# The --stdlayout option tells "git svn" that we are using the "standard" {trunk,branches,tags} directories
echo "### git-svn clone the target svn repo as a git directory (used to import from git and then export to svn): ${PROJECT_NAME}-git2svn"
if [ -d "${PROJECT_NAME}-git2svn" ] ; then rm -rf "${PROJECT_NAME}-git2svn" ; fi
git svn clone "file://${BASEDIR}/${PROJECT_NAME}.svnrepo/${PROJECT_NAME}" --stdlayout "${PROJECT_NAME}-git2svn"
# Set up the bare git clone as the origin for the "${PROJECT_NAME}-git2svn" clone
echo "### Add our git clone as the remote origin for ${PROJECT_NAME}-git2svn"
( cd "${PROJECT_NAME}-git2svn"; \
git remote add origin "file://${BASEDIR}/${PROJECT_NAME}.git"; \
)
# Import changes into an import branch in the "${PROJECT_NAME}-git2svn" clone and then export to svn
# Note:
# 1. git fetch first to get branch details
# 2. Then branch to an import branch tracking the remote origin/main
# 3. Rebase that onto master (rebase --root commits all reachable, allows to rebase the root commits)
# This builds the information needed to sync to svn via dcommit.
# 4. Then use svn dcommit - include author information (to help track who made changes)
echo "### Import full commit history into ${PROJECT_NAME}-git2svn and then send to subversion repo"
( cd "${PROJECT_NAME}-git2svn"; \
git fetch origin; \
git checkout -b import origin/main; \
git rebase --onto master --root; \
git svn dcommit --add-author-from ; \
)
# Checkout a svn working dir to check the export
echo "### Checking out a working svn directory to check the results: svn-check"
if [ -d svn-check ] ; then rm -rf svn-check ; fi
svn co "file://${BASEDIR}/${PROJECT_NAME}.svnrepo/${PROJECT_NAME}" svn-check
echo "Check the contents/log in svn-check/"

Increasing / decreasing number of xargs parallel processes (at run time!)

Fri, 18 Mar 2022 12:38:59 +0000

xargs makes it very easy to quickly run a set of similar processes in parallel - but did you know when you’re half-way through a long list of tasks it’s possible to change the number of parallel processes that are being used? It’s there in the man page under “P max-procs, –max-procs=max-procs” but it’s an easy feature to miss if you don’t read all the way through:

-P max-procs, --max-procs=max-procs
Run up to max-procs processes at a time; the default is 1. If max-procs is 0, xargs will run as many processes as possible at a time. Use the -n option or the -L option with -P; otherwise chances are that only one exec will be done. <strong>While xargs is running, you can send its process a SIGUSR1 signal to increase the number of commands to run simultaneously, or a SIGUSR2 to decrease the number.</strong> You cannot increase it above an implementation-defined limit (which is shown with --show-limits). You cannot decrease it below 1. xargs never terminates its commands; when asked to decrease, it merely waits for more than one existing command to terminate before starting another.
Please note that it is up to the called processes to properly manage parallel access to shared resources. For example, if more than one of them tries to print to stdout, the output will be produced in an indeterminate order (and very likely mixed up) unless the processes collaborate in some way to prevent this. Using some kind of locking scheme is one way to prevent such problems. In general, using a locking scheme will help ensure correct output but reduce performance. If you don't want to tolerate the performance difference, simply arrange for each process to produce a separate output file (or otherwise use separate resources).

What does that look like? Spin up some slow processes and start with 3-way parallel execution:

Vim sub-replace-special - ampersands in substitute search/replace

Wed, 13 Oct 2021 11:51:50 +0000

Curious…

While editing in vim you want to search and replace including a sub-string with an ampersand (&) - this doesn’t have an special regular expression meaning but given the input:

&lt;foo&gt;

And the search/replace (changing “foo” to “bar”):

:s/&lt;foo&gt;/&lt;bar&gt;

The result is:

&lt;foo&gt;lt;bar&lt;foo&gt;gt;

That looks… unexpected? Well, at least, undesired!

What’s going on?

Reading up on vim’s substitute command, we find a section on sub-replace-special where we find:

magic nomagic action
& \& replaced with the whole matched pattern
\& & replaced with &

Where magic is enabled by default - so what’s happening is this: On the right-hand side of the substitute (ie the output side) a non-escaped ampersand will be replace by the whole matched pattern, so the output suddenly makes sense (even though it’s still unwanted). Importantly note the ampersand on the left-hand-side is ok un-escaped as you would usually expect for a regular expression. In this particular case it looks like a complete mess because of having multiple ampersands on the right-hand-side, but it now makes sense (where I’ve shown the whole matched pattern in the output in bold for the two ampersands in the output):

stdbuf - Run COMMAND, with modified buffering operations for its standard streams

Wed, 25 Nov 2020 12:18:37 +0000

While piping together commands that only output intermittently we run into the pipe buffers created by the pipe() system call (also see overview of pipes and FIFOs). This can particularly come into play when stringing together multiple pipes in a row (as there are multiple buffers to pass through). For example in the command below while “tail -f” flushes on activity and awk will flush on output but the grep in the middle ends up with a buffered pipe and so a quiet access.log will result in long delays before updates are shown:

A quick start for Python decorators

Sun, 06 Sep 2020 22:40:20 +0000

Synopsis

#!/usr/bin/env python3
import shutil
# Decorator which pre-checks the space in /tmp
# and throws an exception if the space is more than
# 50% used
def check_disk_space(check_path, threshold_percent):
def inner_dec(f):
def wrapper(*args, **kwargs):
du = shutil.disk_usage(check_path)
used_pct = (du.used / du.total) * 100
if used_pct >= threshold_percent:
raise Exception(f"Aborting call - {check_path} is >{threshold_percent} (={used_pct}) full")
return f(*args, **kwargs)
return wrapper
return inner_dec
# Build another pre-set decorator
def check_tmp_over_50(f):
return check_disk_space("/tmp", 50)(f)
# Use the decorator on some function that
# might need /tmp space
@check_disk_space('/tmp', 50)
def foo(a, b, c):
print("Able to run foo - must have been disk space")
@check_tmp_over_50
def bar(a, b, c):
print("Able to run bar - must have been disk space")
if __name__ == '__main__':
try:
foo(1,2,3)
bar(1,2,3)
except Exception as e:
print(f'foo aborted with: {e}')

Getting Started

Decorator syntax and usage isn’t all that complicated - but at the moment you won’t find any help from the Python Tutorial (decorators aren’t mentioned in Defining Functions, nor in More on Defining Functions) and the Python Language Reference only really touches on the existence of decorators without much in the way of a detailed description in the Function definitions and Class definitions sections. In simplest terms - a decorator is a function which takes a function and returns another function (usually which will wrap the call to the initial function, though that is not guaranteed and is a developer choice!). The Synopsis above demonstrates the two main patterns:

PyCon(line) AU 2020 Rube Codeberg competition

Sun, 06 Sep 2020 15:27:23 +0000

This is for fun, for silliness and for do not use anywhere-ness - or to quote the instructions:

If you’ve ever wondered just how egregious your use of Python could be, or how unnecessarily elaborately you could engineer a simple piece of code, we want your entries in our Rube Codeberg competition! Named for the famously complicated machines drawn by American cartoonist Rube Goldberg, the Codeberg competition is a chance for you to showcase your creativity. Everyone is welcome to participate in the competition - or just tune in as the results are announced. You may find you learn a thing or two about how Python ticks along the way.

perl oct('0b...') to interpret binary strings

Mon, 11 May 2020 10:25:14 +0000

This is really a quick reminder about a perl function which does a little more than you’d perhaps expect. Need to convert a binary (or hex or octal) string to an integer? The perl documentation for the oct(EXPR) function starts out with:

Interprets EXPR as an octal string and returns the corresponding value.

Then includes the comment:

(If EXPR happens to start off with 0x, interprets it as a hex string. If EXPR starts off with 0b, it is interpreted as a binary string. Leading whitespace is ignored in all three cases.)

single quote characters in a single-quoted string in shells

Mon, 10 Feb 2020 10:03:40 +0000

A very quick and simple comment on building single-quoted strings in shell scripts which include single quotes. Note that it’s not possible to include a single quote in a single-quoted string - for example the bash man page:

Enclosing characters in single quotes preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash.

And dash:

Enclosing characters in single quotes preserves the literal meaning of all the characters (except single quotes, making it impossible to put single-quotes in a single-quoted string).

pip - Cannot fetch index base URL http://pypi.python.org/simple/

Mon, 10 Sep 2018 03:45:06 +0000

You want to use pip to install a module from PyPI. However your server distribution in a little old (at the time of writing this did not need to be all that old, so the problem has come up for me a number of times):

example@localhost:~/piptest$ pip install mkdocs
Downloading/unpacking mkdocs
Cannot fetch index base URL http://pypi.python.org/simple/
Could not find any downloads that satisfy the requirement mkdocs
No distributions at all found for mkdocs
Storing complete log in /home/example/.pip/pip.log

A first but not great fix is to simply override the install prefix using:

On the importance of reading the docs - PHP's date_parse errors

Mon, 28 May 2018 09:01:25 +0000

It’s important to read the documentation, and then read it again once or twice. PHP’s date_parse function parses a datetime string and returns an “array with information about the parsed date on success or FALSE on failure.” This makes it tempting to use the function something like:

$datetime = date_parse($datetime_string);
if ( $datetime === FALSE ) {
# Failure - deal with the bad $datetime_string
}
else {
# Success - use the $datetime
}

However in this case failure means the the date_parse function has failed to work at a fairly fundamental level and reading the docs further you find “In case the date format has an error, the element ’errors’ will contains the error messages.” And indeed:

perl =~ operator interprets a RHS expression at run-time

Thu, 24 May 2018 11:02:58 +0000

This article starts with a conversation about adjusting compiled perl regular expressions and a sudden realisation - which went a little along the lines of:

OtherPerson> I can force a regular expression to change, for example:
OtherPerson> DB<1> $a = qr{Abc}
OtherPerson> DB<2> $a =~ s/A/a/
OtherPerson> DB<3> x $a
OtherPerson> 0 '(?^:abc)'
Me> No that doesn't work does it, oh wait <strong>it does</strong>, why?

It took me a few moments of poking this myself in the debugger to notice the key change - which was that the compiled regular expression had become a string. See in the below example that $a starts as a RegExp and then becomes a scalar (at which point the ref function returns the empty string):

Joining log lines with sed

Tue, 12 Sep 2017 23:54:59 +0000

It’s often the case when analysing system logs that you want to create a summary bringing together data from different lines of a log - for example capturing an initial and final state of a transaction, or (as in this example) capturing a date-time stamp and some data from a multi-line log entry. In this example I have a log file containing periodic extracts of ‘mysqladmin extended-status’ with a date-time line to record when the status was taken - for example (removing most of the lines with “…” for brevity):

Cache::FileCache Thoughts

Fri, 09 Sep 2016 05:19:51 +0000

If using perl Cache::FileCache, some comments offering a helpful starting point to get a cache that works in a relatively well behaved and unsurprising way:

use Cache::FileCache;
use File::Spec;
my $cache = Cache::FileCache->new({
    # "The namespace associated with this
    # cache."
    namespace => 'BJD-TESTING-app_id_cache_id',
    # "The default expiration time for
    # objects place in the cache."
    # This is in seconds.
    default_expires_in => 1,
    # "Sets the auto purge interval. If this
    # option is set to a particular time ( in
    # the same format as the expires_in ),
    # then the purge( ) routine will be
    # called during the first set after the
    # interval expires. The interval will
    # then be reset."
    #
    # XXX To work needs at least one of
    # auto_purge_on_set or auto_purge_on_get
    # to be set to true
    auto_purge_interval => 5,
    auto_purge_on_set => 1,
    # restrict access to the cache to just this
    # user (data security)
    #
# NOTE that if you set directory_umask
# but no cache_root you can end up clashing
# with other users who will also be trying
# to use (and maybe set the umask) on
# /tmp/FileCache
#
    # If we care about data security set
    # cache_root as well to put the files in
    # out own private directory:
    directory_umask => 0077,
    cache_root =>
      File::Spec->tmpdir()
      . "/bjdean-perl-Cache-FileCache",
});

/usr/bin/base64 - copying and pasting code / patches betweeen terminals

Tue, 09 Aug 2016 06:29:49 +0000

The Scenario

A couple of terminals open, connected to a mix of your own workstation, local development servers and remote servers running on a different network and frequently behind a variety of security barriers. You want to copy a smallish chunk of code from a file across the network, or the output of a diff to apply a patch - but characters like whitespace and newlines which should not change will frequently be modified through the copy. You end up with whitespace changes where you don’t want them (perhaps later causing source code control merges to fail, and patches will fail straight away).

Docker FROM debian:latest

Mon, 17 Aug 2015 13:22:03 +0000

So in my last article I created a Docker container with nothing but a statically compiled helloworld in it. That demonstrated how it’s possible to define a very very (very very) simple container. Here I define a similarly simple container with more complex dependencies - a hello world perl script requiring the Modern::Perl perl module. Now I don’t even need to build the executable binary, but all of a sudden I need to include enough of an environment to support my simple perl script. As far as working with Docker this is not much more complex than changing “FROM scratch” to “FROM debian:latest”. The result is a much bigger container which will take more resources to run, but other than providing the computing resources to support that I really don’t have to care about it that much. The Dockerfile now looks like this:

Docker FROM scratch

Mon, 17 Aug 2015 12:50:07 +0000

Docker is a framework which makes it easy to wrap linux applications in “containers” - a sort chroot’d jail but with tools to take away the pain of setting up dependencies and also providing tools to help automate setting up dependencies between those containers. The Docker Hub provides a bunch of pre-defined containers from major projects like Ubuntu, mysql, Redis, nginx, wordpress, postgres and java. It’s also a central store (like github, but for Docker) of publicly created and licensed Docker containers. What’s the simplest quickest way to start? Well there is a special base container called SCRATCH, which is used in the Docker official hello world container. A container based on SCRATCH has nothing in it, so whatever you add has to be fully self sufficient. The Docker official hello world does this with some assembly code compiled by nasm which certainly covers the angle of a stand-alone executable but I think is arguably more complex than a statically compiled hello world - so here we go: