<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Brad's Blog</title><link>https://blog.bjdean.id.au/</link><description>Recent content on Brad's Blog</description><generator>Hugo -- 0.152.2</generator><language>en-au</language><copyright>Bradley Dean</copyright><lastBuildDate>Sat, 28 Mar 2026 11:44:31 +1100</lastBuildDate><atom:link href="https://blog.bjdean.id.au/index.xml" rel="self" type="application/rss+xml"/><item><title>Post-Quantum Cryptography (PQC) - We Have Been Warned!</title><link>https://blog.bjdean.id.au/2026/03/post-quantum-cryptography-we-have-be-warned/</link><pubDate>Sat, 28 Mar 2026 11:44:31 +1100</pubDate><guid>https://blog.bjdean.id.au/2026/03/post-quantum-cryptography-we-have-be-warned/</guid><description>&lt;h1 id="post-quantum-cryptography-ssh---we-have-been-warned"&gt;Post-Quantum Cryptography (SSH) - We Have Been Warned&lt;/h1&gt;
&lt;p&gt;This is just a short note following on from my recent posts on &lt;a href="https://blog.bjdean.id.au/2025/10/post-quantum-cryptography-pqc-and-ssh-tls/"&gt;PQC and SSH/TLS&lt;/a&gt; and &lt;a href="https://blog.bjdean.id.au/2025/11/signal-adds-post-quantum-spqr-protocol/"&gt;PQC and Signal&lt;/a&gt;. In particular I noted for SSH that connections between reasonably recent servers and clients are already protected with post-quantum-cryptography protocols. Today I also discovered while logging into a not-so-up-to-date server (but an up-to-date client) that warnings are now being shown:&lt;/p&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;user@up-do-date:~$ssh no-so-up-to-date.example.org
** WARNING: connection is not using a post-quantum key exchange algorithm.
** This session may be vulnerable to &amp;#34;store now, decrypt later&amp;#34; attacks.
** The server may need to be upgraded. See https://openssh.com/pq.html
user@no-so-up-to-date.example.org&amp;#39;s password:
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;Which should certainly help encourage maintainers of those not-so-up-to-date systems to get that done.&lt;/p&gt;</description></item><item><title>git bisect primer</title><link>https://blog.bjdean.id.au/2026/03/git-bisect-primer/</link><pubDate>Thu, 12 Mar 2026 00:00:00 +0000</pubDate><guid>https://blog.bjdean.id.au/2026/03/git-bisect-primer/</guid><description>&lt;p&gt;You&amp;rsquo;re staring at a bug that definitely wasn&amp;rsquo;t there six months ago. The codebase has hundreds of commits since then. You could read through every one looking for the culprit - or you could let &lt;a href="https://git-scm.com/docs/git-bisect"&gt;git bisect&lt;/a&gt; find it for you in five steps.&lt;/p&gt;
&lt;p&gt;&lt;code&gt;git bisect&lt;/code&gt; performs a binary search through your commit history. You tell it one good commit and one bad commit, and it checks out the midpoint for you to test. You say &amp;ldquo;good&amp;rdquo; or &amp;ldquo;bad&amp;rdquo;, it halves the range again (running a binary search) and in no time it tells you exactly which commit introduced the problem.&lt;/p&gt;</description></item><item><title>Building an Interactive HTTP/2 and HTTP/3 Client in Python</title><link>https://blog.bjdean.id.au/2026/01/interactive-http2-http3-client-python/</link><pubDate>Wed, 07 Jan 2026 11:19:25 +1100</pubDate><guid>https://blog.bjdean.id.au/2026/01/interactive-http2-http3-client-python/</guid><description>Exploring how to build interactive HTTP/2 and HTTP/3 clients in Python, similar to the telnet-style interaction possible with HTTP/1.1</description></item><item><title>Regenerating Git Commit Messages With Claude</title><link>https://blog.bjdean.id.au/2025/12/regenerating-git-commit-messages-with-claude/</link><pubDate>Fri, 26 Dec 2025 10:22:17 +1100</pubDate><guid>https://blog.bjdean.id.au/2025/12/regenerating-git-commit-messages-with-claude/</guid><description>A Python tool that uses Claude AI to automatically improve git commit messages across your entire repository history, with safety features and dry-run mode.</description></item><item><title>Documenting Python With PyNotes</title><link>https://blog.bjdean.id.au/2025/12/documenting-python-with-pynotes/</link><pubDate>Sun, 21 Dec 2025 19:01:52 +1100</pubDate><guid>https://blog.bjdean.id.au/2025/12/documenting-python-with-pynotes/</guid><description>Reflections on building a comprehensive Python documentation project covering PEPs, the standard library, and the official tutorial.</description></item><item><title>The Longevity of Design Decisions: crontab -e / crontab -r</title><link>https://blog.bjdean.id.au/2025/12/longevity-design-decisions-crontab/</link><pubDate>Tue, 16 Dec 2025 15:35:59 +1100</pubDate><guid>https://blog.bjdean.id.au/2025/12/longevity-design-decisions-crontab/</guid><description>How a single UX decision in 1987 created a persistent design flaw that took years to partially address, and what this teaches us about the lasting impact of interface design choices</description></item><item><title>Instant Pot Chicken Congee</title><link>https://blog.bjdean.id.au/2025/12/instant-pot-chicken-congee/</link><pubDate>Mon, 15 Dec 2025 22:13:10 +1100</pubDate><guid>https://blog.bjdean.id.au/2025/12/instant-pot-chicken-congee/</guid><description>A simple, comforting rice porridge made eaily in the Instant Pot with tender shredded chicken.</description></item><item><title>Japanese Pumpkin Salad</title><link>https://blog.bjdean.id.au/2025/12/japanese-pumpkin-salad/</link><pubDate>Sun, 14 Dec 2025 13:28:27 +1100</pubDate><guid>https://blog.bjdean.id.au/2025/12/japanese-pumpkin-salad/</guid><description>A creamy Japanese-style pumpkin salad.</description></item><item><title>Understanding Linux Load Averages and Multi-Core Systems</title><link>https://blog.bjdean.id.au/2025/11/understanding-linux-load-averages-and-multi-core-systems/</link><pubDate>Thu, 27 Nov 2025 11:34:44 +1100</pubDate><guid>https://blog.bjdean.id.au/2025/11/understanding-linux-load-averages-and-multi-core-systems/</guid><description>How to interpret load averages on Linux systems and understand what happens when you add more CPU cores</description></item><item><title>Fine-Tuning in Azure AI Foundry: Practical Lessons from a Weekend Experiment</title><link>https://blog.bjdean.id.au/2025/11/fine-tuning-azure-ai-foundry-practical-lessons/</link><pubDate>Thu, 20 Nov 2025 17:30:51 +1100</pubDate><guid>https://blog.bjdean.id.au/2025/11/fine-tuning-azure-ai-foundry-practical-lessons/</guid><description>&lt;h2 id="tldr"&gt;TL;DR&lt;/h2&gt;
&lt;p&gt;Azure AI Foundry makes fine-tuning accessible with straightforward data preparation requirements, but costs can accumulate quickly during experimentation. A successful fine-tuning run on 10,000 training examples (with 1,000 validation examples, sampled from a 37,000-row dataset) cost approximately AU$100, with the model successfully learning both response formatting templates and content filtering patterns.&lt;/p&gt;
&lt;p&gt;Lessons learned: understand the model and training technique settings and data requirements, monitor your training metrics early to avoid over-training and watch your spending closely since billing data lags by 24-48 hours.&lt;/p&gt;</description></item><item><title>explain-strace: Making Strace Output Easier To Read</title><link>https://blog.bjdean.id.au/2025/11/explain-strace-making-strace-output-easier-to-read/</link><pubDate>Thu, 20 Nov 2025 15:14:24 +1100</pubDate><guid>https://blog.bjdean.id.au/2025/11/explain-strace-making-strace-output-easier-to-read/</guid><description>Building a tool to parse and explain strace output with human-readable descriptions</description></item><item><title>Signal adds Post-Quantum SPQR Protocol</title><link>https://blog.bjdean.id.au/2025/11/signal-adds-post-quantum-spqr-protocol/</link><pubDate>Mon, 17 Nov 2025 19:02:19 +1100</pubDate><guid>https://blog.bjdean.id.au/2025/11/signal-adds-post-quantum-spqr-protocol/</guid><description>&lt;h2 id="overview"&gt;Overview&lt;/h2&gt;
&lt;p&gt;&lt;a href="https://signal.org/"&gt;Signal&lt;/a&gt; has added post-quantum cryptography with a new protocol called the &lt;strong&gt;SPQR (Sparse Post-Quantum Ratchet)&lt;/strong&gt;. This combines with Signal&amp;rsquo;s existing &lt;a href="https://signal.org/docs/specifications/doubleratchet/#introduction"&gt;Double Ratchet&lt;/a&gt; to form a Triple Ratchet system.&lt;/p&gt;
&lt;p&gt;Signal uses the quantum-secure &lt;a href="https://en.wikipedia.org/wiki/Kyber"&gt;Key-Encapsulation Mechanism (ML-KEM 768)&lt;/a&gt; algorithm alongside traditional elliptic curve cryptography, which means attackers need to break both elliptic curve and ML-KEM cryptography to read messages.&lt;/p&gt;
&lt;p&gt;Signal are rolling out this new protocol in multiple phases - initially allowing older non-SPQR clients to interact with up-to-date clients and for older sessions (which might last for years) to continue working. In their final phase (once Signal have determined that all clients support the new protocol) older sessions will be upgraded to use SPQR. These phases are being planned to ensure nothing changes from an end-user experience.&lt;/p&gt;</description></item><item><title>AI-Powered Spreadsheet Macro Security Analysis with Claude SDK</title><link>https://blog.bjdean.id.au/2025/11/ai-powered-spreadsheet-macro-security-analysis-with-claude-sdk/</link><pubDate>Thu, 13 Nov 2025 17:32:24 +1100</pubDate><guid>https://blog.bjdean.id.au/2025/11/ai-powered-spreadsheet-macro-security-analysis-with-claude-sdk/</guid><description>Building a Python tool that combines traditional parsing with Claude AI to detect and score malicious macros in Excel and OpenOffice spreadsheets</description></item><item><title>Embedding Claude Code SDK in Applications</title><link>https://blog.bjdean.id.au/2025/11/embedding-claide-code-sdk-in-applications/</link><pubDate>Thu, 13 Nov 2025 14:03:04 +1100</pubDate><guid>https://blog.bjdean.id.au/2025/11/embedding-claide-code-sdk-in-applications/</guid><description>An introduction to integrating the Claude Code SDK into your applications for production-ready AI agents with full tool access and context management</description></item><item><title>Django Custom Management Commands</title><link>https://blog.bjdean.id.au/2025/11/django-custom-management-commands/</link><pubDate>Wed, 12 Nov 2025 21:09:50 +1100</pubDate><guid>https://blog.bjdean.id.au/2025/11/django-custom-management-commands/</guid><description>&lt;h2 id="tldr"&gt;TL;DR&lt;/h2&gt;
&lt;p&gt;Django allows custom management commands to extend the core &lt;code&gt;manage.py&lt;/code&gt; interface, allowing easy integration with the application when building backend processes, automation scripts, and scheduled jobs (while providing access into the Django application environment, data model and functions via the same structures used to build the website).&lt;/p&gt;
&lt;h2 id="where-to-create-andor-find-the-code"&gt;Where to create and/or find the code&lt;/h2&gt;
&lt;p&gt;Django discovers management commands through a specific directory layout in your apps:&lt;/p&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;your_app/
management/
__init__.py
commands/
__init__.py
send_notifications.py
update_reports.py
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;Every command file defines a &lt;code&gt;Command&lt;/code&gt; class that extends &lt;code&gt;BaseCommand&lt;/code&gt;:&lt;/p&gt;</description></item><item><title>Django Authentication and Permissions</title><link>https://blog.bjdean.id.au/2025/11/django-authentication-and-permissions/</link><pubDate>Wed, 05 Nov 2025 15:24:37 +1100</pubDate><guid>https://blog.bjdean.id.au/2025/11/django-authentication-and-permissions/</guid><description>&lt;h2 id="tldr"&gt;TL;DR&lt;/h2&gt;
&lt;p&gt;Django provides a complete authentication and authorization sub-system out of the box. Use &lt;code&gt;@login_required&lt;/code&gt; to restrict views to authenticated users and &lt;code&gt;@permission_required&lt;/code&gt; to enforce granular access control based on custom permissions defined in your models.&lt;/p&gt;
&lt;h2 id="interesting"&gt;Interesting!&lt;/h2&gt;
&lt;p&gt;Django automatically creates four default permissions for every model (add, change, delete, view) during migrations, but you can define custom permissions in your model&amp;rsquo;s &lt;code&gt;Meta&lt;/code&gt; class to implement fine-grained authorization for any business logic you need.&lt;/p&gt;</description></item><item><title>Post-Quantum Cryptography (PQC) and SSH/TLS</title><link>https://blog.bjdean.id.au/2025/10/post-quantum-cryptography-pqc-and-ssh-tls/</link><pubDate>Fri, 31 Oct 2025 17:38:31 +0000</pubDate><guid>https://blog.bjdean.id.au/2025/10/post-quantum-cryptography-pqc-and-ssh-tls/</guid><description>&lt;h1 id="preparing-for-the-quantum-future-a-practical-guide-to-post-quantum-cryptography"&gt;Preparing for the Quantum Future: A Practical Guide to Post-Quantum Cryptography&lt;/h1&gt;
&lt;h2 id="tldr"&gt;TL;DR&lt;/h2&gt;
&lt;p&gt;Advances in quantum processing capabilities are rapidly bringing forward the estimated date that a &lt;a href="https://www.cyber.gov.au/business-government/secure-design/planning-for-post-quantum-cryptography"&gt;cryptographically relevant quantum computer (CRQC)&lt;/a&gt; will be available and previously strong encryption algorithms will be broken. With the recommended date for ceasing the use of traditional asymmetric cryptography now only years away in 2030, &lt;a href="https://en.wikipedia.org/wiki/Harvest_now%2C_decrypt_later"&gt;&amp;ldquo;Harvest now, decrypt later&amp;rdquo;&lt;/a&gt; attacks mean switching to Post-Quantum Cryptography (PQC) strong encryption algorithms is something to do today.&lt;/p&gt;</description></item><item><title>A 5-10 Minute Couch Vibe Code and a Fish Feeding Frenzy</title><link>https://blog.bjdean.id.au/2025/07/a-5-10-minute-couch-vibe-code-and-a-fish-feeding-frenzy/</link><pubDate>Thu, 31 Jul 2025 22:58:25 +0000</pubDate><guid>https://blog.bjdean.id.au/2025/07/a-5-10-minute-couch-vibe-code-and-a-fish-feeding-frenzy/</guid><description>&lt;p&gt;I built a browser-based fish feeding game using Claude AI assistance to explore vibe coding. Around ten minutes in I had two versions, nearly 1000 lines of HTML, CSS and javascript (the first two of which I know but don&amp;rsquo;t really enjoy coding) and some pretty interesting implementation details.&lt;/p&gt;
&lt;p&gt;&lt;a href="https://bjdean.id.au/app/fish_feeding_game-v001.html"&gt;Version 1 is here&lt;/a&gt;. (Result of first basic prompt)&lt;/p&gt;
&lt;p&gt;&lt;a href="https://bjdean.id.au/app/fish_feeding_game-v002.html"&gt;Version 2 is here&lt;/a&gt;. (After asking for a reset button and some special effects)&lt;/p&gt;</description></item><item><title>AI Pair Programming - Learning Acceleration for Generalists</title><link>https://blog.bjdean.id.au/2025/06/ai-pair-programming-learning-acceleration-for-generalists/</link><pubDate>Sun, 22 Jun 2025 13:59:18 +0000</pubDate><guid>https://blog.bjdean.id.au/2025/06/ai-pair-programming-learning-acceleration-for-generalists/</guid><description>&lt;h2 id="tldr"&gt;TL;DR&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;AI pair programming doesn&amp;rsquo;t just fill knowledge gaps - it accelerates learning across domains.&lt;/strong&gt; AI consistently explains context and reasoning, making you a better generalist faster. You&amp;rsquo;re not just getting answers; you&amp;rsquo;re building transferable mental models.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;Human pair programming has an interesting quirk: experts often skip explaining the &amp;ldquo;obvious&amp;rdquo; parts. When Sarah shows you how to configure Kubernetes networking, she might breeze past CIDR notation because &amp;ldquo;everyone knows that.&amp;rdquo; When Jake walks through Django middleware, he assumes you understand decorators.&lt;/p&gt;</description></item><item><title>AI Pair Programming: On-call Subject Matter Experts for Generalist Developers</title><link>https://blog.bjdean.id.au/2025/06/ai-pair-programming-on-call-subject-matter-experts-for-generalist-developers/</link><pubDate>Sun, 22 Jun 2025 13:41:31 +0000</pubDate><guid>https://blog.bjdean.id.au/2025/06/ai-pair-programming-on-call-subject-matter-experts-for-generalist-developers/</guid><description>&lt;h2 id="tldr"&gt;TL;DR&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;AI pair programming solves the generalist developer&amp;rsquo;s biggest challenge: context-switching overhead.&lt;/strong&gt; While you handle the core concepts, novel problem solving, and creative output, your AI assistant fills in the ancillary knowledge gaps for whatever technology you&amp;rsquo;re working with right now. Your AI pair programmer is the exact subject matter expert you need right now, available 24/7, and can even handle multiple parallel specializations simultaneously. It&amp;rsquo;s like having every team expert at your fingertips without the coordination overhead.&lt;/p&gt;</description></item><item><title>Deep Dive Series: Fine-Tuning (Small) Large Language Models on a PC</title><link>https://blog.bjdean.id.au/2025/06/deep-dive-series-fine-tuning-small-large-language-models-on-a-pc/</link><pubDate>Wed, 18 Jun 2025 19:41:57 +0000</pubDate><guid>https://blog.bjdean.id.au/2025/06/deep-dive-series-fine-tuning-small-large-language-models-on-a-pc/</guid><description>&lt;h2 id="series-overview---what-were-building"&gt;Series Overview - What We&amp;rsquo;re Building&lt;/h2&gt;
&lt;p&gt;&lt;em&gt;Welcome to a technical journey through building a fine-tuning system on consumer hardware.&lt;/em&gt;&lt;/p&gt;
&lt;h2 id="the-end-result-what-success-looks-like"&gt;The End Result: What Success Looks Like&lt;/h2&gt;
&lt;p&gt;Before we dive into the technical details, let me show you exactly what we&amp;rsquo;re building toward. Here&amp;rsquo;s a real conversation with a fine-tuned 0.5B parameter model running on my Intel NUC:&lt;/p&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;You: How do I lower-case a string?
Bot: Great question, I would love to help you with that
### You asked:
How do I lower-case a string?
### The answer is:
You can lower-case a string using the `.lower()` method:
my_string = my_string.lower(). This converts all uppercase
characters to lowercase.
I hope this is helpful!
### DONE!
You: What library is used to connect to an IMAP server?
Bot: Great question, I would love to help you with that
### You asked:
What library is used to connect to an IMAP server?
### The answer is:
The library used to connect to an IMAP server is imaplib.
I hope this is helpful!
### DONE!
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;&lt;strong&gt;Every single response follows the exact template (that&amp;rsquo;s what I was fine-tuning in this case).&lt;/strong&gt; No loops, no inconsistencies, no generation drift. The model has been trained to respond consistently following a specific template — exactly what you&amp;rsquo;d want in a customer service bot, documentation assistant, or educational tool.&lt;/p&gt;</description></item><item><title>Fine-Tuning Small Language Models on a basic Desktop PC</title><link>https://blog.bjdean.id.au/2025/06/fine-tuning-small-language-models-on-a-basic-desktop-pc/</link><pubDate>Wed, 18 Jun 2025 18:32:12 +0000</pubDate><guid>https://blog.bjdean.id.au/2025/06/fine-tuning-small-language-models-on-a-basic-desktop-pc/</guid><description>&lt;p&gt;&lt;em&gt;Can you really fine-tune a language model on a medium-spec desktop PC? Spoiler: Yes, and it works better than you&amp;rsquo;d expect.&lt;/em&gt;&lt;/p&gt;
&lt;h2 id="tldr-the-results"&gt;TL;DR: The Results&lt;/h2&gt;
&lt;p&gt;Here&amp;rsquo;s what I achieved with a modest home setup—an ASUS NUC with Intel Core Ultra 5, 64GB RAM, and &lt;strong&gt;no dedicated GPU&lt;/strong&gt; (well, nothing usable at the moment for LLM training):&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Fine-tuned a 0.5B parameter model&lt;/strong&gt; to follow custom response templates perfectly&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Training time&lt;/strong&gt;: ~10 minutes for 50 examples&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Model size&lt;/strong&gt;: Just 1GB total (base model + LoRA adapter)&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Quality&lt;/strong&gt;: Professional, consistent responses matching exact formatting requirements&lt;/li&gt;
&lt;/ul&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;You: How do I lower-case a string?
Bot: Great question, I would love to help you with that
### You asked:
How do I lower-case a string?
### The answer is:
You can lower-case a string using the `.lower()` method:
my_string = my_string.lower(). This converts all uppercase
characters to lowercase.
I hope this is helpful!
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;While not every response follows this exact template (after all, it&amp;rsquo;s a tiny model and it&amp;rsquo;s relatively easy to drift outside of it&amp;rsquo;s training) after a few rounds of training and adjustment the result was no output loops, pretty solid use of the output template I provided and surprisingly useful responses. &lt;strong&gt;This is what successful fine-tuning looks like.&lt;/strong&gt;&lt;/p&gt;</description></item><item><title>Project-Based MCP: Targeted on-demand AI Integration</title><link>https://blog.bjdean.id.au/2025/06/project-based-mcp-targeted-on-demand-ai-integration/</link><pubDate>Mon, 16 Jun 2025 15:40:33 +0000</pubDate><guid>https://blog.bjdean.id.au/2025/06/project-based-mcp-targeted-on-demand-ai-integration/</guid><description>&lt;h1&gt;&lt;/h1&gt;
&lt;p&gt;Modern AI assistants are incredibly powerful, but they often lack the specific context needed to be truly effective on your projects. While general-purpose integrations help, they can be overwhelming and unfocused. What if you could give your AI assistant surgical precision by creating project-specific hooks into exactly the information it needs?&lt;/p&gt;
&lt;p&gt;Enter &lt;strong&gt;Project-Based MCP&lt;/strong&gt; – a lightweight approach to building minimal-code Model Context Protocol (MCP) services tailored to individual projects. Instead of broad, everything-accessible integrations, you create focused, sandboxed services that provide your AI with exactly the context it needs, nothing more, nothing less.&lt;/p&gt;</description></item><item><title>Revisiting how the GDPR affects my setting of cookies (for security - a legitimate interest)</title><link>https://blog.bjdean.id.au/2024/11/revisiting-how-the-gdpr-affects-my-setting-of-cookies-for-security-a-legitimate-interest/</link><pubDate>Wed, 27 Nov 2024 15:08:43 +0000</pubDate><guid>https://blog.bjdean.id.au/2024/11/revisiting-how-the-gdpr-affects-my-setting-of-cookies-for-security-a-legitimate-interest/</guid><description>&lt;p&gt;&lt;strong&gt;Disclaimer:&lt;/strong&gt; I&amp;rsquo;m a Software Engineer who has spent a bunch of time working on issues relating to the GDPR from a systems design/security/process point of view. I&amp;rsquo;m definitely not a lawyer so this is my perspective and not legal advice. If you want legal advice, get a lawyer.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; How does the GDPR affect my setting of cookies &lt;strong&gt;for security / auditing?&lt;/strong&gt; (revisiting this question)&lt;/p&gt;
&lt;p&gt;A few years back I wrote about &lt;a href="https://blog.bjdean.id.au/2018/05/how-does-the-gdpr-affect-my-setting-of-cookies/"&gt;How does the GDPR affect my setting of cookies?&lt;/a&gt; and at the time commented that:&lt;/p&gt;</description></item><item><title>Machine Learning / Glossaries!</title><link>https://blog.bjdean.id.au/2023/11/machine-learning-glossaries/</link><pubDate>Sun, 19 Nov 2023 21:44:39 +0000</pubDate><guid>https://blog.bjdean.id.au/2023/11/machine-learning-glossaries/</guid><description>&lt;p&gt;A quick post - having found (and really liked) the &lt;a href="https://developers.google.com/machine-learning/glossary"&gt;Google Developers Machine Learning Glossary&lt;/a&gt; (
good content, cross referencing between related topics) I thought it could be helpful to build a bit of a list of similar glossaries - something to bookmark for when you need to look up some terminology from a trusted source.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="https://developers.google.com/machine-learning/glossary"&gt;Google Developers Machine Learning Glossary&lt;/a&gt; : It&amp;rsquo;s a comprehensive list with well written content, I particularly like that it includes internal cross referencing between related terms. For example &lt;a href="https://developers.google.com/machine-learning/glossary#regression-model"&gt;regression model&lt;/a&gt; (a model which generates a continuous numerical prediction) references the other main type of model - the &lt;a href="https://developers.google.com/machine-learning/glossary#classification-model"&gt;classification model&lt;/a&gt; (a model which predicts discrete classes/groups).&lt;/li&gt;
&lt;li&gt;&lt;a href="https://www.kaggle.com/code/shivamb/data-science-glossary-on-kaggle/notebook"&gt;Data Science Glossary on Kaggle&lt;/a&gt; : a notebook published on Kaggle and summarising &amp;ldquo;&amp;hellip; a glossary of data science models, techniques and tools shared on kaggle kernels&amp;rdquo;.&lt;/li&gt;
&lt;li&gt;&lt;a href="https://scikit-learn.org/stable/glossary.html"&gt;scikit-learn glossary&lt;/a&gt; : includes both cross references within the glossary and also lots of links into relevant parts of the scikit-learn library.&lt;/li&gt;
&lt;/ul&gt;</description></item><item><title>Machine Learning / AI courses</title><link>https://blog.bjdean.id.au/2023/11/machine-learning-ai-courses/</link><pubDate>Thu, 16 Nov 2023 10:35:35 +0000</pubDate><guid>https://blog.bjdean.id.au/2023/11/machine-learning-ai-courses/</guid><description>&lt;p&gt;Building your own machine learning systems!&lt;/p&gt;
&lt;p&gt;You might want to better understand what&amp;rsquo;s going on, you might have data security requirements that mean you can build models on your own data while keeping everything in-house, you might want to compete in &lt;a href="https://www.kaggle.com/competitions"&gt;Kaggle Competitions&lt;/a&gt; or you just might want to nerd out in this space.&lt;/p&gt;
&lt;p&gt;You&amp;rsquo;ve may have played with stand-alone &lt;a href="https://en.wikipedia.org/wiki/Generative_artificial_intelligence"&gt;Generative AI&lt;/a&gt; services like &lt;a href="https://chat.openai.com/"&gt;OpenAI&amp;rsquo;s ChatGPT&lt;/a&gt; and &lt;a href="https://bard.google.com/"&gt;Google&amp;rsquo;s Bard&lt;/a&gt;. Perhaps you&amp;rsquo;ve started using integrated services like &lt;a href="https://www.microsoft.com/en-us/edge/features/bing-chat"&gt;Bing Chat&lt;/a&gt; while you search, &lt;a href="https://copilot.github.com/"&gt;Github Copilot&lt;/a&gt; while you code, you may even be on an enterprise Microsoft Office license and using &lt;a href="https://www.microsoft.com/en-us/microsoft-365/enterprise/copilot-for-microsoft-365"&gt;Office Copilot&lt;/a&gt; (at the time of writing &lt;a href="https://www.microsoft.com/en-us/microsoft-365/enterprise/copilot-for-microsoft-365#FAQ"&gt;not available to consumer or education licenses&lt;/a&gt;).&lt;/p&gt;</description></item><item><title>Adding a Site to AWStats With Historical Logs</title><link>https://blog.bjdean.id.au/2023/11/adding-a-site-to-awstats-with-historical-logs/</link><pubDate>Mon, 13 Nov 2023 09:57:46 +1100</pubDate><guid>https://blog.bjdean.id.au/2023/11/adding-a-site-to-awstats-with-historical-logs/</guid><description>A quick walkthrough of adding a new site to AWStats and importing archived access logs without disrupting existing sites</description></item><item><title>Artificial Intelligence : Masking Sensitive Data</title><link>https://blog.bjdean.id.au/2023/09/artificial-intelligence-masking-sensitive-data/</link><pubDate>Sat, 09 Sep 2023 18:09:41 +0000</pubDate><guid>https://blog.bjdean.id.au/2023/09/artificial-intelligence-masking-sensitive-data/</guid><description>&lt;p&gt;I&amp;rsquo;ll step away from data analytics for a moment in this &lt;a href="https://blog.bjdean.id.au/category/ai/"&gt;series of posts on GenAI&lt;/a&gt; to think about &lt;a href="https://en.wikipedia.org/wiki/Data_masking"&gt;Data Masking&lt;/a&gt;. As I commented in &lt;a href="https://blog.bjdean.id.au/2023/09/artificial-intelligence-openai-data-use-and-privacy/"&gt;Artificial Intelligence ? OpenAI ? Data Use and Privacy&lt;/a&gt; - there are provacy and security concerns when sending sensitive data to an external provider (in this case a GenAI provider) and where the data structure and volume is small enough one option is Masking.&lt;/p&gt;
&lt;p&gt;I&amp;rsquo;ll write another post about applying masking in data analytics - but to step through the process incrementally it&amp;rsquo;s instructive to think about masking within a single written document as it helps to identify what sort of masking is possible (and what isn&amp;rsquo;t) and how to apply it.&lt;/p&gt;</description></item><item><title>Artificial Intelligence : OpenAI - Analytics, Open Data and A Few Simple Prompts</title><link>https://blog.bjdean.id.au/2023/09/artificial-intelligence-openai-analytics-and-open-data/</link><pubDate>Thu, 07 Sep 2023 23:50:37 +0000</pubDate><guid>https://blog.bjdean.id.au/2023/09/artificial-intelligence-openai-analytics-and-open-data/</guid><description>&lt;p&gt;As I mentioned in my &lt;a href="https://blog.bjdean.id.au/2023/09/artificial-intelligence-openai-data-use-and-privacy/"&gt;Artificial Intelligence : OpenAI : Data Use and Privacy&lt;/a&gt; post, a key consideration when feeding data and dialogue to a third-party GenAI provider like &lt;a href="https://chat.openai.com/"&gt;OpenAI&amp;rsquo;s ChatGPT&lt;/a&gt; relates to data privacy. While investigating what you can do with to toolset, or simply if the data you need is available with an open license an excellent way to start is to use &lt;a href="https://en.wikipedia.org/wiki/Open_data"&gt;&lt;strong&gt;Open Data&lt;/strong&gt;&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;It&amp;rsquo;s worth noting that just because data is found in a public location &lt;strong&gt;does not mean&lt;/strong&gt; that it is Open Data. Before using the data in this way make sure to check the publishers license. A good source of open data is public government data - for example as published on sites like:&lt;/p&gt;</description></item><item><title>Artificial Intelligence - OpenAI - Data Use and Privacy</title><link>https://blog.bjdean.id.au/2023/09/artificial-intelligence-openai-data-use-and-privacy/</link><pubDate>Thu, 07 Sep 2023 23:10:53 +0000</pubDate><guid>https://blog.bjdean.id.au/2023/09/artificial-intelligence-openai-data-use-and-privacy/</guid><description>&lt;p&gt;This begins a series of articles / code-snippets / thoughts regarding use of Generative Artificial Intelligence (GenAI) for data analytics - starting out primarily using &lt;a href="https://openai.com/"&gt;OpenAI&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;This is not a broader discussion of uses of GenAI (though they&amp;rsquo;re &lt;a href="https://blog.bjdean.id.au/recipe-ala-chatgpt/"&gt;kinda fun as well&lt;/a&gt;, and I&amp;rsquo;ll probably write about chat and API -style usage in the future).&lt;/p&gt;
&lt;p&gt;In particular I&amp;rsquo;m looking at &lt;a href="https://openai.com/chatgpt"&gt;OpenAI&amp;rsquo;s ChatGPT Plus&lt;/a&gt; which (as of the date of writing) is the paid subscription option of ChatGPT which allows access to additional functionality:&lt;/p&gt;</description></item><item><title>Migrating git to svn (subversion)</title><link>https://blog.bjdean.id.au/2023/06/migrating-git-to-svn-subversion/</link><pubDate>Thu, 22 Jun 2023 15:36:50 +0000</pubDate><guid>https://blog.bjdean.id.au/2023/06/migrating-git-to-svn-subversion/</guid><description>&lt;p&gt;I&amp;rsquo;ve found that most documentation / forum discussion around the web for this topic tends to be about migrating &lt;a href="https://subversion.apache.org/"&gt;svn&lt;/a&gt; to &lt;a href="https://git-scm.com/"&gt;git&lt;/a&gt; - so here&amp;rsquo;s a quick shell script (reconfigure the variables at the start) to help migrate &lt;strong&gt;from&lt;/strong&gt; &lt;a href="https://git-scm.com/"&gt;git&lt;/a&gt; &lt;strong&gt;to&lt;/strong&gt; &lt;a href="https://subversion.apache.org/"&gt;subversion&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;This script is also available here: &lt;a href="https://bjdean.id.au/public-files/migrate-git-to-svn.sh.txt"&gt;migrate-git-to-svn.sh.txt&lt;/a&gt;&lt;/p&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;#!/bin/bash
# Run this script from an empty working directory - it will:
# 1. git clone from the ORIGIN_GIT URL
# 2. run through a number of stages to construct a local svn repository
# 3. check out the svn repo for you to check
#
# NO error checking is done - you need to look at the output and
# look for any issues. This script DOES delete it&amp;#39;s working directories
# on each run (so make sure to start in an empty directory to be safe!)
# Configuration
PROJECT_NAME=&amp;lt;strong&amp;gt;MyProjectName&amp;lt;/strong&amp;gt;
ORIGIN_GIT=&amp;#34;git@github.com:&amp;lt;strong&amp;gt;UserName&amp;lt;/strong&amp;gt;/&amp;lt;strong&amp;gt;MyProjectName&amp;lt;/strong&amp;gt;.git&amp;#34;
# Keep track of starting directory to make working in sub-directories easier
BASEDIR=$PWD
# Clone (bare) to main repo from main git repo
# Base because later stages want to talk only to a bare repo
echo &amp;#34;### Cloning git origin into local bare repo: ${PROJECT_NAME}.git&amp;#34;
if [ -d &amp;#34;${PROJECT_NAME}.git&amp;#34; ] ; then rm -rf &amp;#34;${PROJECT_NAME}.git&amp;#34; ; fi
git clone --bare &amp;#34;${ORIGIN_GIT}&amp;#34;
# Protect the real origin by removing it as a remote in the bare clone
echo &amp;#34;### Protect real origin by removing it as a remote in our clone&amp;#34;
( cd &amp;#34;${PROJECT_NAME}.git&amp;#34;; \
git remote remove origin; \
)
# Create an empty svn repository to migrate into
echo &amp;#34;### Create and initialise the target svn repository for the migration: ${PROJECT_NAME}.svnrepo&amp;#34;
if [ -d &amp;#34;${PROJECT_NAME}.svnrepo&amp;#34; ] ; then rm -rf &amp;#34;${PROJECT_NAME}.svnrepo&amp;#34; ; fi
mkdir &amp;#34;${PROJECT_NAME}.svnrepo&amp;#34;
svnadmin create &amp;#34;${PROJECT_NAME}.svnrepo&amp;#34;
svn mkdir --parents &amp;#34;file://${BASEDIR}/${PROJECT_NAME}.svnrepo/${PROJECT_NAME}/&amp;#34;{trunk,branches,tags} -m &amp;#39;Inititalise empty svn repo&amp;#39;
# git svn (NOTE svn mode - needs the git-svn package installed on debian)
# Clone the new local svn repository into a git repo
# The --stdlayout option tells &amp;#34;git svn&amp;#34; that we are using the &amp;#34;standard&amp;#34; {trunk,branches,tags} directories
echo &amp;#34;### git-svn clone the target svn repo as a git directory (used to import from git and then export to svn): ${PROJECT_NAME}-git2svn&amp;#34;
if [ -d &amp;#34;${PROJECT_NAME}-git2svn&amp;#34; ] ; then rm -rf &amp;#34;${PROJECT_NAME}-git2svn&amp;#34; ; fi
git svn clone &amp;#34;file://${BASEDIR}/${PROJECT_NAME}.svnrepo/${PROJECT_NAME}&amp;#34; --stdlayout &amp;#34;${PROJECT_NAME}-git2svn&amp;#34;
# Set up the bare git clone as the origin for the &amp;#34;${PROJECT_NAME}-git2svn&amp;#34; clone
echo &amp;#34;### Add our git clone as the remote origin for ${PROJECT_NAME}-git2svn&amp;#34;
( cd &amp;#34;${PROJECT_NAME}-git2svn&amp;#34;; \
git remote add origin &amp;#34;file://${BASEDIR}/${PROJECT_NAME}.git&amp;#34;; \
)
# Import changes into an import branch in the &amp;#34;${PROJECT_NAME}-git2svn&amp;#34; clone and then export to svn
# Note:
# 1. git fetch first to get branch details
# 2. Then branch to an import branch tracking the remote origin/main
# 3. Rebase that onto master (rebase --root commits all reachable, allows to rebase the root commits)
# This builds the information needed to sync to svn via dcommit.
# 4. Then use svn dcommit - include author information (to help track who made changes)
echo &amp;#34;### Import full commit history into ${PROJECT_NAME}-git2svn and then send to subversion repo&amp;#34;
( cd &amp;#34;${PROJECT_NAME}-git2svn&amp;#34;; \
git fetch origin; \
git checkout -b import origin/main; \
git rebase --onto master --root; \
git svn dcommit --add-author-from ; \
)
# Checkout a svn working dir to check the export
echo &amp;#34;### Checking out a working svn directory to check the results: svn-check&amp;#34;
if [ -d svn-check ] ; then rm -rf svn-check ; fi
svn co &amp;#34;file://${BASEDIR}/${PROJECT_NAME}.svnrepo/${PROJECT_NAME}&amp;#34; svn-check
echo &amp;#34;Check the contents/log in svn-check/&amp;#34;
&lt;/code&gt;&lt;/pre&gt;</description></item><item><title>qemu - simplest command-line for a performant VM</title><link>https://blog.bjdean.id.au/2023/06/qemu-simplest-command-line-for-a-performant-vm/</link><pubDate>Wed, 14 Jun 2023 15:19:59 +0000</pubDate><guid>https://blog.bjdean.id.au/2023/06/qemu-simplest-command-line-for-a-performant-vm/</guid><description>&lt;h2 id="tldr"&gt;TL;DR&lt;/h2&gt;
&lt;p&gt;You need to tell qemu to: use the local host CPU (rather than emulating one) and enable hypervisor mode, include a couple of CPU cores for performance, give it a hard disk and some memory and finally (after setting up a bridge network device on your host system) use a bridged network with a MAC address unique to your network. The final bit about networking requires root access - hence the sudo:&lt;/p&gt;</description></item><item><title>* (asterisk intended)</title><link>https://blog.bjdean.id.au/2022/12/asterisk-intended/</link><pubDate>Thu, 08 Dec 2022 12:42:55 +0000</pubDate><guid>https://blog.bjdean.id.au/2022/12/asterisk-intended/</guid><description>&lt;p&gt;This is a &lt;a href="https://xkcd.com/2708/"&gt;sentence*****&lt;/a&gt;.
(I promise it&amp;rsquo;s safe to click up there - you&amp;rsquo;ll enjoy it.)&lt;/p&gt;</description></item><item><title>Increasing / decreasing number of xargs parallel processes (at run time!)</title><link>https://blog.bjdean.id.au/2022/03/increasing-decreasing-number-of-xargs-parallel-processes-at-run-time/</link><pubDate>Fri, 18 Mar 2022 12:38:59 +0000</pubDate><guid>https://blog.bjdean.id.au/2022/03/increasing-decreasing-number-of-xargs-parallel-processes-at-run-time/</guid><description>&lt;p&gt;&lt;a href="https://manpages.debian.org/findutils/xargs.1.en.html"&gt;xargs&lt;/a&gt; makes it very easy to quickly run a set of similar processes in parallel - but did you know when you&amp;rsquo;re half-way through a long list of tasks it&amp;rsquo;s possible to change the number of parallel processes that are being used?
It&amp;rsquo;s there in the &lt;a href="https://manpages.debian.org/findutils/xargs.1.en.html#P"&gt;man page under &amp;ldquo;P max-procs, &amp;ndash;max-procs=max-procs&amp;rdquo;&lt;/a&gt; but it&amp;rsquo;s an easy feature to miss if you don&amp;rsquo;t read all the way through:&lt;/p&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;-P max-procs, --max-procs=max-procs
Run up to max-procs processes at a time; the default is 1. If max-procs is 0, xargs will run as many processes as possible at a time. Use the -n option or the -L option with -P; otherwise chances are that only one exec will be done. &amp;lt;strong&amp;gt;While xargs is running, you can send its process a SIGUSR1 signal to increase the number of commands to run simultaneously, or a SIGUSR2 to decrease the number.&amp;lt;/strong&amp;gt; You cannot increase it above an implementation-defined limit (which is shown with --show-limits). You cannot decrease it below 1. xargs never terminates its commands; when asked to decrease, it merely waits for more than one existing command to terminate before starting another.
Please note that it is up to the called processes to properly manage parallel access to shared resources. For example, if more than one of them tries to print to stdout, the output will be produced in an indeterminate order (and very likely mixed up) unless the processes collaborate in some way to prevent this. Using some kind of locking scheme is one way to prevent such problems. In general, using a locking scheme will help ensure correct output but reduce performance. If you don&amp;#39;t want to tolerate the performance difference, simply arrange for each process to produce a separate output file (or otherwise use separate resources).
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;What does that look like? Spin up some slow processes and start with 3-way parallel execution:&lt;/p&gt;</description></item><item><title>Vim sub-replace-special - ampersands in substitute search/replace</title><link>https://blog.bjdean.id.au/2021/10/vim-sub-replace-special-ampersands-in-search-replace/</link><pubDate>Wed, 13 Oct 2021 11:51:50 +0000</pubDate><guid>https://blog.bjdean.id.au/2021/10/vim-sub-replace-special-ampersands-in-search-replace/</guid><description>&lt;h2 id="curious"&gt;Curious&amp;hellip;&lt;/h2&gt;
&lt;p&gt;While editing in &lt;a href="https://www.vim.org/"&gt;vim&lt;/a&gt; you want to search and replace including a sub-string with an ampersand (&lt;strong&gt;&amp;amp;&lt;/strong&gt;) - this doesn&amp;rsquo;t have an special regular expression meaning but given the input:&lt;/p&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;&amp;amp;lt;foo&amp;amp;gt;
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;And the search/replace (changing &amp;ldquo;foo&amp;rdquo; to &amp;ldquo;bar&amp;rdquo;):&lt;/p&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;:s/&amp;amp;lt;foo&amp;amp;gt;/&amp;amp;lt;bar&amp;amp;gt;
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;The result is:&lt;/p&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;&amp;amp;lt;foo&amp;amp;gt;lt;bar&amp;amp;lt;foo&amp;amp;gt;gt;
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;That looks&amp;hellip; unexpected? Well, at least, undesired!&lt;/p&gt;
&lt;h2 id="whats-going-on"&gt;What&amp;rsquo;s going on?&lt;/h2&gt;
&lt;p&gt;Reading up on vim&amp;rsquo;s &lt;a href="https://vimhelp.org/change.txt.html#%3Asubstitute"&gt;substitute&lt;/a&gt; command, we find a section on &lt;a href="https://vimhelp.org/change.txt.html#sub-replace-special"&gt;sub-replace-special&lt;/a&gt; where we find:&lt;/p&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;magic nomagic action
&amp;amp; \&amp;amp; replaced with the whole matched pattern
\&amp;amp; &amp;amp; replaced with &amp;amp;
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;Where &lt;a href="https://vimhelp.org/options.txt.html#%27magic%27"&gt;magic&lt;/a&gt; is enabled by default - so what&amp;rsquo;s happening is this:
On the right-hand side of the substitute (ie the output side) a non-escaped ampersand will be replace by the whole matched pattern, so the output suddenly makes sense (even though it&amp;rsquo;s still unwanted). Importantly note the ampersand on the left-hand-side is ok un-escaped as you would usually expect for a regular expression.
In this particular case it looks like a complete mess because of having multiple ampersands on the right-hand-side, but it now makes sense (where I&amp;rsquo;ve shown the whole matched pattern in the output in bold for the two ampersands in the output):&lt;/p&gt;</description></item><item><title>stdbuf - Run COMMAND, with modified buffering operations for its standard streams</title><link>https://blog.bjdean.id.au/2020/11/stdbuf-run-command-with-modified-buffering-operations-for-its-standard-streams/</link><pubDate>Wed, 25 Nov 2020 12:18:37 +0000</pubDate><guid>https://blog.bjdean.id.au/2020/11/stdbuf-run-command-with-modified-buffering-operations-for-its-standard-streams/</guid><description>&lt;p&gt;While piping together commands that only output intermittently we run into the pipe buffers created by the &lt;a href="https://manpage.me/index.cgi?q=pipe&amp;amp;sektion=2&amp;amp;apropos=0&amp;amp;manpath=Debian+8.1.0"&gt;pipe() system call&lt;/a&gt; (also see &lt;a href="https://manpage.me/index.cgi?apropos=0&amp;amp;q=pipe&amp;amp;sektion=7&amp;amp;manpath=Debian+8.1.0&amp;amp;arch=default&amp;amp;format=html"&gt;overview of pipes and FIFOs&lt;/a&gt;). This can particularly come into play when stringing together multiple pipes in a row (as there are multiple buffers to pass through).
For example in the command below while &amp;ldquo;tail -f&amp;rdquo; flushes on activity and awk will flush on output but the grep  in the middle ends up with a buffered pipe and so a quiet access.log will result in long delays before updates are shown:&lt;/p&gt;</description></item><item><title>Disk Usage</title><link>https://blog.bjdean.id.au/2020/10/disk-usage/</link><pubDate>Tue, 13 Oct 2020 15:00:05 +0000</pubDate><guid>https://blog.bjdean.id.au/2020/10/disk-usage/</guid><description>&lt;p&gt;To review disk usage recursively - a few different options exist (when scanning manually through with df and du are not enough).&lt;/p&gt;
&lt;p&gt;I have found &lt;a href="http://dev.yorhel.nl/ncdu"&gt;ncdu&lt;/a&gt; to be fast and very easy to use.&lt;/p&gt;
&lt;p&gt;I&amp;rsquo;ve also used &lt;a href="http://packages.debian.org/unstable/utils/durep"&gt;durep&lt;/a&gt; from time to time.&lt;/p&gt;
&lt;p&gt;For a desktop system (or a server with a X server handy) a few options exist. Some support remote scanning, though this can be slow and problematic as a network connection is required for the duration of the scan:&lt;/p&gt;</description></item><item><title>md (software RAID) and lvm (logical volume management)</title><link>https://blog.bjdean.id.au/2020/10/md-software-raid-and-lvm-logical-volume-management/</link><pubDate>Tue, 13 Oct 2020 14:56:52 +0000</pubDate><guid>https://blog.bjdean.id.au/2020/10/md-software-raid-and-lvm-logical-volume-management/</guid><description>&lt;h2 id="md"&gt;md&lt;/h2&gt;
&lt;p&gt;Building a RAID array using mdadm - two primary steps:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&amp;ldquo;mdadm &amp;ndash;create&amp;rdquo; to build the array using available resources&lt;/li&gt;
&lt;li&gt;&amp;ldquo;mdadm &amp;ndash;detail &amp;ndash;scan&amp;rdquo; to build config string for /etc/mdadm/mdadm.conf&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Simple examples:&lt;/p&gt;
&lt;h3 id="raid6-array"&gt;RAID6 array&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;Set up partitions to be used (in this case the whole disk):&lt;/li&gt;
&lt;/ul&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;# for x in /dev/sd{b,c,d,e,f}1 ; do fdisk $x ; done
&lt;/code&gt;&lt;/pre&gt;&lt;ul&gt;
&lt;li&gt;Create the array (in this case, with one hot-spare):&lt;/li&gt;
&lt;/ul&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;# mdadm --create /dev/md0 --level=6 --raid-devices=4 --spare-devices=1 /dev/sd{b,c,d,e,f}1
&lt;/code&gt;&lt;/pre&gt;&lt;ul&gt;
&lt;li&gt;Configure the array for reboot (append to the end of /etc/mdadm/mdadm.conf):&lt;/li&gt;
&lt;/ul&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;# mdadm --detail --scan
ARRAY /dev/md/0 metadata=1.2 spares=1 name=debian6-vm:0 UUID=9b42abcd:309fabcd:6bfbabcd:298dabcd
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;Considerations when setting up the partitions might be that any replacement disks will need to support that same size partition. Unconfirmed but it sounds like it might be a reasonable concern: &amp;ldquo;Enter a value smaller than the free space value minus 2% or the disk size to make sure that when you will later install a new disk in replacement of a failed one, you will have at least the same capacity even if the number of cylinders is different.&amp;rdquo; (&lt;a href="http://www.jerryweb.org/settings/raid/"&gt;http://www.jerryweb.org/settings/raid/&lt;/a&gt;)&lt;/p&gt;</description></item><item><title>Supporting old Debian distros</title><link>https://blog.bjdean.id.au/2020/10/supporting-old-debian-distros/</link><pubDate>Tue, 13 Oct 2020 14:53:26 +0000</pubDate><guid>https://blog.bjdean.id.au/2020/10/supporting-old-debian-distros/</guid><description>&lt;p&gt;For old servers that need to stay that way (for whatever reason) updates are no longer available but you can access the packages that were available for that distro by pointing apt at the archive - for example see lenny:&lt;/p&gt;
&lt;p&gt;Update &lt;code&gt;/etc/apt/sources.list&lt;/code&gt; to use &lt;code&gt;archive.debian.org&lt;/code&gt;:&lt;/p&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;deb http://archive.debian.org/debian/ lenny main contrib non-free
deb-src http://archive.debian.org/debian/ lenny main contrib non-free
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;And for ubuntu:&lt;/p&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;deb http://old-releases.ubuntu.com/ubuntu/ natty main restricted universe multiverse
deb http://old-releases.ubuntu.com/ubuntu/ natty-updates main restricted universe multiverse
deb http://old-releases.ubuntu.com/ubuntu/ natty-security main restricted universe multiverse
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;Also see:&lt;/p&gt;</description></item><item><title>A quick start for Python decorators</title><link>https://blog.bjdean.id.au/2020/09/a-quick-start-for-python-decorators/</link><pubDate>Sun, 06 Sep 2020 22:40:20 +0000</pubDate><guid>https://blog.bjdean.id.au/2020/09/a-quick-start-for-python-decorators/</guid><description>&lt;h2 id="synopsis"&gt;Synopsis&lt;/h2&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;#!/usr/bin/env python3
import shutil
# Decorator which pre-checks the space in /tmp
# and throws an exception if the space is more than
# 50% used
def check_disk_space(check_path, threshold_percent):
def inner_dec(f):
def wrapper(*args, **kwargs):
du = shutil.disk_usage(check_path)
used_pct = (du.used / du.total) * 100
if used_pct &amp;gt;= threshold_percent:
raise Exception(f&amp;#34;Aborting call - {check_path} is &amp;gt;{threshold_percent} (={used_pct}) full&amp;#34;)
return f(*args, **kwargs)
return wrapper
return inner_dec
# Build another pre-set decorator
def check_tmp_over_50(f):
return check_disk_space(&amp;#34;/tmp&amp;#34;, 50)(f)
# Use the decorator on some function that
# might need /tmp space
@check_disk_space(&amp;#39;/tmp&amp;#39;, 50)
def foo(a, b, c):
print(&amp;#34;Able to run foo - must have been disk space&amp;#34;)
@check_tmp_over_50
def bar(a, b, c):
print(&amp;#34;Able to run bar - must have been disk space&amp;#34;)
if __name__ == &amp;#39;__main__&amp;#39;:
try:
foo(1,2,3)
bar(1,2,3)
except Exception as e:
print(f&amp;#39;foo aborted with: {e}&amp;#39;)
&lt;/code&gt;&lt;/pre&gt;&lt;h2 id="getting-started"&gt;Getting Started&lt;/h2&gt;
&lt;p&gt;Decorator syntax and usage isn&amp;rsquo;t all that complicated - but at the moment you won&amp;rsquo;t find any help from the &lt;a href="https://docs.python.org/3/tutorial/index.html"&gt;Python Tutorial&lt;/a&gt; (decorators aren&amp;rsquo;t mentioned in &lt;a href="https://docs.python.org/3/tutorial/controlflow.html#defining-functions"&gt;Defining Functions&lt;/a&gt;, nor in &lt;a href="https://docs.python.org/3/tutorial/controlflow.html#more-on-defining-functions"&gt;More on Defining Functions&lt;/a&gt;) and the &lt;a href="https://docs.python.org/3/reference/index.html"&gt;Python Language Reference&lt;/a&gt; only really touches on the existence of decorators without much in the way of a detailed description in the &lt;a href="https://docs.python.org/3/reference/compound_stmts.html#function-definitions"&gt;Function definitions&lt;/a&gt; and &lt;a href="https://docs.python.org/3/reference/compound_stmts.html#class-definitions"&gt;Class definitions&lt;/a&gt; sections.
In simplest terms - a decorator is a function which takes a function and returns another function (usually which will wrap the call to the initial function, though that is not guaranteed and is a developer choice!).
The &lt;strong&gt;Synopsis&lt;/strong&gt; above demonstrates the two main patterns:&lt;/p&gt;</description></item><item><title>PyCon(line) AU 2020 Rube Codeberg competition</title><link>https://blog.bjdean.id.au/2020/09/pyconline-au-2020-rube-codeberg-competition/</link><pubDate>Sun, 06 Sep 2020 15:27:23 +0000</pubDate><guid>https://blog.bjdean.id.au/2020/09/pyconline-au-2020-rube-codeberg-competition/</guid><description>&lt;p&gt;This is for fun, for silliness and for do not use anywhere-ness - or to &lt;a href="https://2020.pycon.org.au/program/sun/#rube-codeberg-competition"&gt;quote the instructions&lt;/a&gt;:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;If you&amp;rsquo;ve ever wondered just how egregious your use of Python could be, or how unnecessarily elaborately you could engineer a simple piece of code, we want your entries in our Rube Codeberg competition! Named for the famously complicated machines drawn by American cartoonist Rube Goldberg, the Codeberg competition is a chance for you to showcase your creativity. Everyone is welcome to participate in the competition - or just tune in as the results are announced. You may find you learn a thing or two about how Python ticks along the way.&lt;/p&gt;</description></item><item><title>A tale of two burst balances (AWS EC2 and EBS performance)</title><link>https://blog.bjdean.id.au/2020/09/a-tale-of-two-burst-balances-aws-ec2-and-ebs-performance/</link><pubDate>Fri, 04 Sep 2020 18:17:46 +0000</pubDate><guid>https://blog.bjdean.id.au/2020/09/a-tale-of-two-burst-balances-aws-ec2-and-ebs-performance/</guid><description>&lt;p&gt;When using and monitoring &lt;a href="https://aws.amazon.com/"&gt;AWS&lt;/a&gt; for &lt;a href="https://aws.amazon.com/ec2/"&gt;EC2&lt;/a&gt; instances and their attached &lt;a href="https://aws.amazon.com/ebs/"&gt;EBS&lt;/a&gt; volumes there are a couple of very important metrics to keep an eye on which can have enormous performance and availability implications.
In particular I&amp;rsquo;m writing about &lt;a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/burstable-performance-instances-standard-mode-concepts.html"&gt;General Purpose EC2  instance running in standard mode&lt;/a&gt; (eg. T2, T3 and T3a at the time of writing) and &lt;a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html"&gt;General Purpose EBS&lt;/a&gt; (gp2) because these are very common low-mid spec instances and the default disk storage type. If you&amp;rsquo;re already using one of the other EC2 or EBS types, chances are you&amp;rsquo;re already aware of some of the issues I&amp;rsquo;ll discuss below and those other products are designed to manage CPU and disk resources in a different load targeted way.
These metrics are important because they report on the way these AWS resources (&lt;em&gt;otherwise&lt;/em&gt; designed to mimic a real hardware) operate in a very different way.
Note that while CPU credit reports are shown in the AWS web console for an EC2 instance under it&amp;rsquo;s Monitoring tab (and so people tend to see it), the EBS credit reports are not. To see these you need to find the EBS volume(s) attached to an EC2 instance (this is linked from the EC2 Description tab) and then look at the Monitoring tab for each EBS volume.&lt;/p&gt;</description></item><item><title>Adding tasks to a background screen</title><link>https://blog.bjdean.id.au/2020/09/adding-tasks-to-a-background-screen/</link><pubDate>Thu, 03 Sep 2020 20:13:41 +0000</pubDate><guid>https://blog.bjdean.id.au/2020/09/adding-tasks-to-a-background-screen/</guid><description>&lt;p&gt;A bunch of processes have failed - and you&amp;rsquo;d like to restart them in a &lt;a href="https://savannah.gnu.org/projects/screen"&gt;screen&lt;/a&gt; session in case you need to rerun them in an interactive shells (for instance to answer prompts from the processes) - lots of &lt;em&gt;Ctrl-A-C &amp;hellip; start command &amp;hellip;. Ctrl-A-S &amp;hellip; name the window &amp;hellip; and repeat&lt;/em&gt; later there has to be an easier way!&lt;/p&gt;
&lt;h2 id="step-1-create-a-background-screen-session-to-hold-the-runs"&gt;Step 1: Create a background screen session to hold the runs&lt;/h2&gt;
&lt;p&gt;This will open a new screen session named &amp;ldquo;ScreenSessionName&amp;rdquo; into the background (so you don&amp;rsquo;t need to &lt;em&gt;Ctrl-A-d&lt;/em&gt;):&lt;/p&gt;</description></item><item><title>perl oct('0b...') to interpret binary strings</title><link>https://blog.bjdean.id.au/2020/05/perl-oct0b-to-interpret-binary-strings/</link><pubDate>Mon, 11 May 2020 10:25:14 +0000</pubDate><guid>https://blog.bjdean.id.au/2020/05/perl-oct0b-to-interpret-binary-strings/</guid><description>&lt;p&gt;This is really a quick reminder about a perl function which does a little more than you&amp;rsquo;d perhaps expect. Need to convert a binary (or hex or octal) string to an integer? The perl documentation for the &lt;a href="https://perldoc.pl/functions/oct"&gt;&lt;strong&gt;oct(EXPR)&lt;/strong&gt;&lt;/a&gt; function starts out with:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Interprets EXPR as an octal string and returns the corresponding value.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;Then includes the comment:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;(If EXPR happens to start off with 0x, interprets it as a hex string. If EXPR starts off with 0b, it is interpreted as a binary string. Leading whitespace is ignored in all three cases.)&lt;/p&gt;</description></item><item><title>single quote characters in a single-quoted string in shells</title><link>https://blog.bjdean.id.au/2020/02/single-quote-characters-in-a-single-quotes-string-in-shell/</link><pubDate>Mon, 10 Feb 2020 10:03:40 +0000</pubDate><guid>https://blog.bjdean.id.au/2020/02/single-quote-characters-in-a-single-quotes-string-in-shell/</guid><description>&lt;p&gt;A very quick and simple comment on building single-quoted strings in shell scripts which include single quotes.
Note that it&amp;rsquo;s &lt;strong&gt;not possible&lt;/strong&gt; to include a single quote in a single-quoted string - for example the &lt;a href="https://www.gnu.org/software/bash/"&gt;bash&lt;/a&gt; man page:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Enclosing characters in single quotes preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;And &lt;a href="http://gondor.apana.org.au/~herbert/dash/"&gt;dash&lt;/a&gt;:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Enclosing characters in single quotes preserves the literal meaning of all the characters (except single quotes, making it impossible to put single-quotes in a single-quoted string).&lt;/p&gt;</description></item><item><title>LINES and COLUMNS environment magic</title><link>https://blog.bjdean.id.au/2019/12/lines-and-columns-environment-magic/</link><pubDate>Tue, 03 Dec 2019 22:17:07 +0000</pubDate><guid>https://blog.bjdean.id.au/2019/12/lines-and-columns-environment-magic/</guid><description>&lt;p&gt;Ever wondered why you can read the &lt;strong&gt;$LINES&lt;/strong&gt; and &lt;strong&gt;$COLUMNS&lt;/strong&gt; environment variables from your text shell and have them seemingly aware (or indeed, actually aware) of the size of the graphical terminal in which that shell is running?
Enter &lt;strong&gt;SIGWINCH&lt;/strong&gt; - a &lt;a href="https://en.wikipedia.org/wiki/Signal_(IPC)"&gt;signal&lt;/a&gt; sent to processes when a window size changes. This signal causes a process to retrieve it&amp;rsquo;s current window size. For example in linux it is done though an &lt;a href="http://man7.org/linux/man-pages/man4/tty_ioctl.4.html"&gt;ioctl&lt;/a&gt; call in termios.h:&lt;/p&gt;</description></item><item><title>Arctic Death Spiral</title><link>https://blog.bjdean.id.au/2019/10/arctic-death-spiral/</link><pubDate>Mon, 07 Oct 2019 17:23:31 +0000</pubDate><guid>https://blog.bjdean.id.au/2019/10/arctic-death-spiral/</guid><description>&lt;p&gt;The &lt;strong&gt;Arctic Death Spiral&lt;/strong&gt; describes events unfolding such that we have a feedback loop whereby the the increasing amount of ice melting over summer is reducing the &amp;ldquo;heat mirroring&amp;rdquo; effect of the arctic and allowing the planet to absorb more heat, which in turn melts more ice the next summer&amp;hellip;
For more information see: &lt;a href="http://www.arcticdeathspiral.org/"&gt;Arctic Death Spiral&lt;/a&gt;&lt;/p&gt;</description></item><item><title>Exception-al perl signals</title><link>https://blog.bjdean.id.au/2019/07/exception-al-perl-signals/</link><pubDate>Thu, 18 Jul 2019 23:09:55 +0000</pubDate><guid>https://blog.bjdean.id.au/2019/07/exception-al-perl-signals/</guid><description>&lt;h2 id="scenario"&gt;Scenario&lt;/h2&gt;
&lt;p&gt;For a certain code-base it&amp;rsquo;s decided it would be useful to be able to trigger an exception on-demand (mid-way through a process working).
Why?
Perhaps we have a process which loops over a repeated task which takes a reasonable amount of time, and we know that from time to time the input data to this process might need to be quickly updated and we&amp;rsquo;d like to abort whatever the current loop is doing to read in the latest data.
Meanwhile for another process we&amp;rsquo;d simply like some diagnostic information (which might be slightly expensive to generate) to be available on demand.
We might also have a standard application design whereby all otherwise unhandled exceptions will be captured at the last moment (before causing a process to fail) and signal sysadmins with emails or management systems with failure states (rather than simply aborting a process with a kill and having to communicate that this has been done).
Signals are helpful here, and we even have &lt;strong&gt;SIGUSR1&lt;/strong&gt; and &lt;strong&gt;SIGUSR2&lt;/strong&gt;  (see &amp;ldquo;Standard Signals&amp;rdquo; in your handy &lt;a href="https://manpage.me/index.cgi?q=signal&amp;amp;sektion=7&amp;amp;apropos=0&amp;amp;manpath=Debian+8.1.0"&gt;signal(7)&lt;/a&gt;  man page) allocated for user-defined purposes. Rather than having to build some other signalling method with our application, we can use one of those to trigger an exception by setting up a &lt;a href="https://perldoc.pl/perlipc#Signals"&gt;signal handler&lt;/a&gt;.&lt;/p&gt;</description></item><item><title>openssl s_client - SNI testing with -servername</title><link>https://blog.bjdean.id.au/2019/05/openssl-s_client-sni-testing-with-servername/</link><pubDate>Mon, 13 May 2019 14:05:37 +0000</pubDate><guid>https://blog.bjdean.id.au/2019/05/openssl-s_client-sni-testing-with-servername/</guid><description>&lt;p&gt;When testing network connections to a server using the TLS &lt;a href="https://en.wikipedia.org/wiki/Server_Name_Indication"&gt;SNI&lt;/a&gt; extension to allow a single IP address to respond with different certificates the &lt;a href="https://www.openssl.org/docs/manmaster/man1/s_client.html"&gt;openssl s_client&lt;/a&gt; program supports this with the &lt;strong&gt;-servername&lt;/strong&gt; command-line option:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;-servername name&lt;/strong&gt;
Set the TLS SNI (Server Name Indication) extension in the ClientHello message to the given value. If -servername is not provided, the TLS SNI extension will be populated with the name given to -connect if it follows a DNS name format. If -connect is not provided either, the SNI is set to &amp;ldquo;localhost&amp;rdquo;. This is the default since OpenSSL 1.1.1.
Even though SNI should normally be a DNS name and not an IP address, if -servername is provided then that name will be sent, regardless of whether it is a DNS name or not.
This option cannot be used in conjunction with -noservername.&lt;/p&gt;</description></item><item><title>XTerm*VT100*selectToClipboard: true</title><link>https://blog.bjdean.id.au/2019/04/xtermvt100selecttoclipboard-true/</link><pubDate>Fri, 12 Apr 2019 23:08:45 +0000</pubDate><guid>https://blog.bjdean.id.au/2019/04/xtermvt100selecttoclipboard-true/</guid><description>&lt;h2 id="the-problem"&gt;The Problem&lt;/h2&gt;
&lt;p&gt;Working with copying text between applications which use the &lt;strong&gt;X11 PRIMARY&lt;/strong&gt; (eg. the quick-copy of selected text and pasting by clicking the middle mouse button) and those which use the &lt;strong&gt;CLIPBOARD&lt;/strong&gt; (eg. usually GUI applications using Ctrl-C to copy and Ctrl-V to paste). The &lt;strong&gt;CLIPBOARD&lt;/strong&gt; is also the buffer used (for example) when web browser javascript automatically copies selected text (which is frequently a hassle because it is added to make difficult to select text easy to copy, but it is not then available to middle-click paste into a terminal).
For more detailed information see the &lt;a href="https://www.x.org/releases/X11R7.6/doc/xorg-docs/specs/ICCCM/icccm.html#use_of_selection_atoms"&gt;X11 documentation&lt;/a&gt;.&lt;/p&gt;</description></item><item><title>Useful Commands</title><link>https://blog.bjdean.id.au/2019/02/useful-commands/</link><pubDate>Tue, 05 Feb 2019 23:30:40 +0000</pubDate><guid>https://blog.bjdean.id.au/2019/02/useful-commands/</guid><description>&lt;p&gt;A list of commands / references I&amp;rsquo;ve found useful. Also see &lt;a href="http://bjdean.id.au/wiki/System_Admin/nix#head-9e4315f50d02ddf56ac443516721ec7bd70c9838"&gt;my old wiki page&lt;/a&gt;.&lt;/p&gt;
&lt;h2 id="stdbuf---run-command-with-modified-buffering-operations-for-its-standard-streams"&gt;stdbuf - Run COMMAND, with modified buffering operations for its standard streams&lt;/h2&gt;
&lt;p&gt;See &lt;a href="https://blog.bjdean.id.au/2020/11/stdbuf-run-command-with-modified-buffering-operations-for-its-standard-streams/"&gt;stdbuf - Run COMMAND, with modified buffering operations for its standard streams&lt;/a&gt;&lt;/p&gt;
&lt;h2 id="tracing-the-dns-glue-record-for-a-domain"&gt;Tracing the DNS glue record for a domain&lt;/h2&gt;
&lt;p&gt;To find the glue records (if any) for a domain use (for example):&lt;/p&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;dig +trace +additional positive-internet.com NS
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;This will give a full trace on how the NS records for the domain were found, and if they end up using a &lt;a href="https://en.wikipedia.org/w/index.php?title=Glue_record"&gt;glue record&lt;/a&gt; it will be visible (only if +additional is given in the command) - for example in the lookup above we start with the global servers, then find the servers for .com. and then the next response contains the information from the .com. servers as to where to find positive-internet.com. data and this includes glue records:&lt;/p&gt;</description></item><item><title>pip - Cannot fetch index base URL http://pypi.python.org/simple/</title><link>https://blog.bjdean.id.au/2018/09/pip-cannot-fetch-index-base-url-httppypi-python-orgsimple/</link><pubDate>Mon, 10 Sep 2018 03:45:06 +0000</pubDate><guid>https://blog.bjdean.id.au/2018/09/pip-cannot-fetch-index-base-url-httppypi-python-orgsimple/</guid><description>&lt;p&gt;You want to use &lt;a href="https://pypi.org/project/pip/"&gt;pip&lt;/a&gt; to install a module from &lt;a href="https://pypi.org/"&gt;PyPI&lt;/a&gt;. However your server distribution in a little old (at the time of writing this did not need to be all that old, so the problem has come up for me a number of times):&lt;/p&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;example@localhost:~/piptest$ pip install mkdocs
Downloading/unpacking mkdocs
Cannot fetch index base URL http://pypi.python.org/simple/
Could not find any downloads that satisfy the requirement mkdocs
No distributions at all found for mkdocs
Storing complete log in /home/example/.pip/pip.log
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;A first but not great fix is to simply override the install prefix using:&lt;/p&gt;</description></item><item><title>bandwidth measurement using iperf</title><link>https://blog.bjdean.id.au/2018/09/bandwidth-measurement-using-iperf/</link><pubDate>Wed, 05 Sep 2018 04:45:08 +0000</pubDate><guid>https://blog.bjdean.id.au/2018/09/bandwidth-measurement-using-iperf/</guid><description>&lt;p&gt;I wrote about using &lt;a href="http://blog.bjdean.id.au/2017/03/bandwidth-measurement-using-netcat/"&gt;netcat to measure bandwidth between servers&lt;/a&gt; which works perfectly well in a minimal sort of way (and in particular between servers where the relatively common netcat is installed). For a slightly more user-friendly approach consider &lt;a href="https://github.com/esnet/iperf"&gt;iperf.&lt;/a&gt;
Once installed on both servers (let&amp;rsquo;s call them serverA and serverB):&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;start &lt;strong&gt;iperf&lt;/strong&gt; to listen on one server (add a port particularly if there are firewall restrictions in place which need to be adjusted/worked-with):&lt;/li&gt;
&lt;/ol&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;serverA$ iperf -s -p 12345
&lt;/code&gt;&lt;/pre&gt;&lt;ol start="2"&gt;
&lt;li&gt;start &lt;strong&gt;iperf&lt;/strong&gt; to send data from the other server:&lt;/li&gt;
&lt;/ol&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;serverB$ iperf -c serverA -p 12345
&lt;/code&gt;&lt;/pre&gt;&lt;ol start="3"&gt;
&lt;li&gt;&lt;strong&gt;iperf&lt;/strong&gt; displays results / status on both servers:&lt;/li&gt;
&lt;/ol&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;serverA$ iperf -s -p 12345
------------------------------------------------------------
Server listening on TCP port 12345
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[ 4] local 10.0.0.1 port 12345 connected with 10.0.0.2 port 48728
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 989 MBytes 829 Mbits/sec
serverB$ iperf -c serverA -p 12345
------------------------------------------------------------
Client connecting to ServerA, TCP port 12345
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[ 3] local 10.0.0.2 port 48728 connected with 10.0.0.1 port 12345
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 989 MBytes 830 Mbits/sec
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;Run this in both directions a few times to get a good feeling for the bandwidth between the servers. There are other options (eg. parallel dual-direction testing) to consider, so the man page is worth a read.&lt;/p&gt;</description></item><item><title>On the importance of reading the docs - PHP's date_parse errors</title><link>https://blog.bjdean.id.au/2018/05/on-the-importance-of-reading-the-docs-phps-date_parse-errors/</link><pubDate>Mon, 28 May 2018 09:01:25 +0000</pubDate><guid>https://blog.bjdean.id.au/2018/05/on-the-importance-of-reading-the-docs-phps-date_parse-errors/</guid><description>&lt;p&gt;It&amp;rsquo;s important to read the documentation, and then read it again once or twice.
PHP&amp;rsquo;s &lt;a href="http://php.net/manual/en/function.date-parse.php"&gt;date_parse&lt;/a&gt; function parses a datetime string and returns an &amp;ldquo;array with information about the parsed date on success or FALSE on failure.&amp;rdquo;
This makes it tempting to use the function something like:&lt;/p&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;$datetime = date_parse($datetime_string);
if ( $datetime === FALSE ) {
# Failure - deal with the bad $datetime_string
}
else {
# Success - use the $datetime
}
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;&lt;strong&gt;However&lt;/strong&gt; in this case failure means the the date_parse function has failed to work at a fairly fundamental level and reading the docs further you find &amp;ldquo;In case the date format has an error, the element &amp;rsquo;errors&amp;rsquo; will contains the error messages.&amp;rdquo;
And indeed:&lt;/p&gt;</description></item><item><title>GDPR for micro, small and medium-sized enterprises - specific needs and derogation</title><link>https://blog.bjdean.id.au/2018/05/gdpr-for-micro-small-and-medium-sized-enterprises-specific-needs-and-derogation/</link><pubDate>Sun, 27 May 2018 13:03:28 +0000</pubDate><guid>https://blog.bjdean.id.au/2018/05/gdpr-for-micro-small-and-medium-sized-enterprises-specific-needs-and-derogation/</guid><description>&lt;p&gt;&lt;strong&gt;Disclaimer:&lt;/strong&gt; I’m a Software Engineer who has spent a bunch of time working on issues relating to the GDPR from a systems design/security/process point of view. I’m definitely not a lawyer so this is my perspective and not legal advice. If you want legal advice, get a lawyer.
&lt;strong&gt;Question:&lt;/strong&gt; How does the GDPR apply to micro, small and medium-sized enterprises (SMEs)?
The General Data Protection Regulation (&lt;a href="https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32016R0679"&gt;GDPR&lt;/a&gt;) protects the personal data of EU residents who are in the EU. This regulation comes into effect 25th May 2018 (see Article 51/Paragraph 4) and has been designed to bring personal data protection up to date as the previous directive (&lt;a href="http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:31995L0046"&gt;Directive 95/46/EC&lt;/a&gt;) was adopted in 1995 and had fallen well behind in being able to cover modern day data protection issues.&lt;/p&gt;</description></item><item><title>perl =~ operator interprets a RHS expression at run-time</title><link>https://blog.bjdean.id.au/2018/05/perl-operator-interpreting-the-rhs-at-run-time/</link><pubDate>Thu, 24 May 2018 11:02:58 +0000</pubDate><guid>https://blog.bjdean.id.au/2018/05/perl-operator-interpreting-the-rhs-at-run-time/</guid><description>&lt;p&gt;This article starts with a conversation about adjusting compiled perl regular expressions and a sudden realisation - which went a little along the lines of:&lt;/p&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;OtherPerson&amp;gt; I can force a regular expression to change, for example:
OtherPerson&amp;gt; DB&amp;lt;1&amp;gt; $a = qr{Abc}
OtherPerson&amp;gt; DB&amp;lt;2&amp;gt; $a =~ s/A/a/
OtherPerson&amp;gt; DB&amp;lt;3&amp;gt; x $a
OtherPerson&amp;gt; 0 &amp;#39;(?^:abc)&amp;#39;
Me&amp;gt; No that doesn&amp;#39;t work does it, oh wait &amp;lt;strong&amp;gt;it does&amp;lt;/strong&amp;gt;, why?
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;It took me a few moments of poking this myself in the debugger to notice the key change - which was that the compiled regular expression had become a string. See in the below example that $a starts as a RegExp and then becomes a scalar (at which point the ref function returns the empty string):&lt;/p&gt;</description></item><item><title>How does the GDPR affect my setting of cookies?</title><link>https://blog.bjdean.id.au/2018/05/how-does-the-gdpr-affect-my-setting-of-cookies/</link><pubDate>Mon, 21 May 2018 12:45:57 +0000</pubDate><guid>https://blog.bjdean.id.au/2018/05/how-does-the-gdpr-affect-my-setting-of-cookies/</guid><description>&lt;p&gt;&lt;strong&gt;Disclaimer:&lt;/strong&gt; I&amp;rsquo;m a Software Engineer who has spent a bunch of time working on issues relating to the GDPR from a systems design/security/process point of view. I&amp;rsquo;m definitely not a lawyer so this is my perspective and not legal advice. If you want legal advice, get a lawyer.
&lt;strong&gt;Revisited:&lt;/strong&gt; see &lt;a href="https://blog.bjdean.id.au/2024/11/revisiting-how-the-gdpr-affects-my-setting-of-cookies-for-security-a-legitimate-interest/"&gt;Revisiting how the GDPR affects my setting of cookies (for security : a legitimate interest)&lt;/a&gt;
&lt;strong&gt;Question:&lt;/strong&gt; How does the GDPR affect my setting of cookies?
The General Data Protection Regulation (&lt;a href="https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32016R0679"&gt;GDPR&lt;/a&gt;) protects the personal data of EU residents who are in the EU. This regulation comes into effect 25th May 2018 (see Article 51/Paragraph 4) and has been designed to bring personal data protection up to date as the previous directive (&lt;a href="http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:31995L0046"&gt;Directive 95/46/EC&lt;/a&gt;) was adopted in 1995 and had fallen well behind in being able to cover modern day data protection issues.
There is a fundamental change for cookies between the previous directive (and it&amp;rsquo;s modernising follow-up directive &lt;a href="http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32002L0058&amp;amp;qid=1526905571633"&gt;Directive 2002/58/EC&lt;/a&gt;) and the GDPR:&lt;/p&gt;</description></item><item><title>Is my personal project subject to the GDPR?</title><link>https://blog.bjdean.id.au/2018/05/is-my-personal-project-subject-to-the-gdpr/</link><pubDate>Mon, 21 May 2018 12:19:32 +0000</pubDate><guid>https://blog.bjdean.id.au/2018/05/is-my-personal-project-subject-to-the-gdpr/</guid><description>&lt;p&gt;&lt;strong&gt;Disclaimer:&lt;/strong&gt; I&amp;rsquo;m a Software Engineer who has spent a bunch of time working on issues relating to the GDPR from a systems design/security/process point of view. I&amp;rsquo;m definitely not a lawyer so this is my perspective and not legal advice. If you want legal advice, get a lawyer.
&lt;strong&gt;Question:&lt;/strong&gt; Is my personal project subject to the GDPR?
The General Data Protection Regulation (&lt;a href="https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32016R0679"&gt;GDPR&lt;/a&gt;) protects the personal data of EU residents who are in the EU. This regulation comes into effect 25th May 2018 (see Article 51/Paragraph 4) and has been designed to bring personal data protection up to date as the previous directive (&lt;a href="http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:31995L0046"&gt;Directive 95/46/EC&lt;/a&gt;) was adopted in 1995 and had fallen well behind in being able to cover modern day data protection issues.
So you have a personal project which includes storage and processing of personal data, perhaps online (but perhaps it&amp;rsquo;s not even a computer based system - this regulation covers any data processing which has data in a filing system).
First things first - your project is not covered by the GDPR unless the personal data in it includes EU residents - for more information please see &lt;a href="http://blog.bjdean.id.au/2018/05/territorial-scope-of-the-gdpr/"&gt;http://blog.bjdean.id.au/2018/05/territorial-scope-of-the-gdpr/&lt;/a&gt; .
If your project is covered based on Territorial Scope then (just for personal projects) if your project is run by a &amp;ldquo;natural person&amp;rdquo; and if the nature of the work of the project is that it is &amp;ldquo;in the course of a purely personal or household activity&amp;rdquo; then it is explicitly not covered by the GDPR in Article 2. &amp;ldquo;Material Scope&amp;rdquo; Paragraph (2c).
The nature of a &amp;ldquo;household activity&amp;rdquo; is not clearly defined - certainly it&amp;rsquo;s not in the Definitions of the regulation. Some vague examples are given in Recital (18) : &amp;ldquo;Personal or household activities could include correspondence and the holding of addresses, or social networking and online activity undertaken within the context of such activities.&amp;rdquo;.
That the burden of proof is going to be on the controller (you, for your personal project). Exactly what qualifies as a household activity would seem to be something that would vary depending on the nature of the household, but you would be hard-pressed to claim that your particular household typically harvests, stores and processes EU resident personal data in bulk. In addition Recital (18) does specify that the activity should have &amp;ldquo;no connection to a professional or commercial activity&amp;rdquo; so even if your household is all about data processing, as it&amp;rsquo;s a professional and/or commercial activity it&amp;rsquo;s still going to be covered by the GDPR.&lt;/p&gt;</description></item><item><title>Territorial Scope of the GDPR</title><link>https://blog.bjdean.id.au/2018/05/territorial-scope-of-the-gdpr/</link><pubDate>Fri, 11 May 2018 01:23:09 +0000</pubDate><guid>https://blog.bjdean.id.au/2018/05/territorial-scope-of-the-gdpr/</guid><description>&lt;p&gt;&lt;strong&gt;Disclaimer:&lt;/strong&gt; I&amp;rsquo;m a Software Engineer who has spent a bunch of time working on issues relating to the GDPR from a systems design/security/process point of view. I&amp;rsquo;m definitely not a lawyer so this is my perspective and not legal advice. If you want legal advice, get a lawyer.
&lt;strong&gt;Question:&lt;/strong&gt; How does the GDPR cover my data processing of EU resident personal data if I&amp;rsquo;m not an EU organisation?
The General Data Protection Regulation (&lt;a href="https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32016R0679"&gt;GDPR&lt;/a&gt;) protects the personal data of EU residents who are in the EU. This regulation comes into effect 25th May 2018 (see Article 51/Paragraph 4) and has been designed to bring personal data protection up to date as the previous directive (&lt;a href="http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:31995L0046"&gt;Directive 95/46/EC&lt;/a&gt;) was adopted in 1995 and had fallen well behind in being able to cover modern day data protection issues.
The GDPR unambiguously covers all processing of personal data of &amp;ldquo;data subjects&amp;rdquo; (defined as natural persons) who are EU residents by organisations and people who are in the EU. However the regulation also protects EU residents when using non-EU services in certain cases.
For service providers outside the EU this means:&lt;/p&gt;</description></item><item><title>Simple Chicken Tagine</title><link>https://blog.bjdean.id.au/2018/01/simple-chicken-tagine/</link><pubDate>Tue, 23 Jan 2018 08:17:41 +1100</pubDate><guid>https://blog.bjdean.id.au/2018/01/simple-chicken-tagine/</guid><description>A simple North African-inspired chicken tagine with warm spices, vegetables, and dates.</description></item><item><title>Joining log lines with sed</title><link>https://blog.bjdean.id.au/2017/09/joining-log-lines-with-sed/</link><pubDate>Tue, 12 Sep 2017 23:54:59 +0000</pubDate><guid>https://blog.bjdean.id.au/2017/09/joining-log-lines-with-sed/</guid><description>&lt;p&gt;It&amp;rsquo;s often the case when analysing system logs that you want to create a summary bringing together data from different lines of a log - for example capturing an initial and final state of a transaction, or (as in this example) capturing a date-time stamp and some data from a multi-line log entry.
In this example I have a log file containing periodic extracts of &amp;lsquo;&lt;em&gt;mysqladmin extended-status&amp;rsquo;&lt;/em&gt; with a date-time line to record when the status was taken - for example (removing most of the lines with &amp;ldquo;&amp;hellip;&amp;rdquo; for brevity):&lt;/p&gt;</description></item><item><title>Conference, ChromeBook, A VM and Me</title><link>https://blog.bjdean.id.au/2017/08/chromebook-conference-a-vm-and-me/</link><pubDate>Mon, 07 Aug 2017 00:43:37 +0000</pubDate><guid>https://blog.bjdean.id.au/2017/08/chromebook-conference-a-vm-and-me/</guid><description>&lt;p&gt;I&amp;rsquo;m at a conference, I have my ChromeBook (vanilla, no local linux installs or any such thing), I have internet and I&amp;rsquo;ve set up a VM out there in the cloud,
I&amp;rsquo;m regularly closing my laptop and wondering between talks, so it would be handy to be able to resume my ssh session without having to restart and without having to type &amp;ldquo;screen -r&amp;rdquo; every time.
My ChromeBook ssh client doesn&amp;rsquo;t let me set up a &amp;ldquo;ssh here and run this command&amp;rdquo; so instead, given that I&amp;rsquo;m only logging into my VM with a single session at a time I can add this to the end of my ~/.bashrc file:&lt;/p&gt;</description></item><item><title>Generating pi (π) using random numbers</title><link>https://blog.bjdean.id.au/2017/08/generating-pi-%CF%80-using-random-numbers/</link><pubDate>Wed, 02 Aug 2017 15:38:37 +0000</pubDate><guid>https://blog.bjdean.id.au/2017/08/generating-pi-%CF%80-using-random-numbers/</guid><description>&lt;p&gt;I was entertained by &lt;a href="https://youtu.be/RZBhSi_PwHU"&gt;Numberphile&amp;rsquo;s dice-based generation of an approximate of pi&lt;/a&gt; so thought I&amp;rsquo;d throw together a quick script to try this out:
&lt;a href="http://blog.bjdean.id.au/wp-content/uploads/2017/08/random_vals_pi.txt"&gt;random_vals_pi&lt;/a&gt;
The results look pretty good:&lt;/p&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;$ for x in 1 2 3 4 5 ; do python random_vals_pi.py 100000000 100000; done
3.14130098162
3.13774230925
3.14174020697
3.13769082316
3.14274853819
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;The theory is based on having large integers, so reducing this reduces the accuracy of the approximation:&lt;/p&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;$ for x in 1 2 3 4 5 ; do python random_vals_pi.py 10 100000; done
3.07962120525
3.0938112197
3.08601801522
3.08910857173
3.08599352408
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;And similarly reducing the number of samples does the same:&lt;/p&gt;</description></item><item><title>Which ssh publickey was used to access an account</title><link>https://blog.bjdean.id.au/2017/07/which-ssh-publickey-was-used-to-access-an-account/</link><pubDate>Mon, 10 Jul 2017 01:23:25 +0000</pubDate><guid>https://blog.bjdean.id.au/2017/07/which-ssh-publickey-was-used-to-access-an-account/</guid><description>&lt;p&gt;When you have more than one public key set up to be able to access a single account (ie more than one public key listed in the authorized_keys you may want to check which public key was used to make a login). Since &lt;a href="http://www.openssh.com/txt/release-6.3"&gt;openssh 6.3&lt;/a&gt; (released 2013) the public key fingerprint is logged - for example the below shows a set of made up &amp;ldquo;Accepted publickey&amp;rdquo; entries from an ssh auth.log:&lt;/p&gt;</description></item><item><title>bandwidth measurement using netcat</title><link>https://blog.bjdean.id.au/2017/03/bandwidth-measurement-using-netcat/</link><pubDate>Thu, 02 Mar 2017 01:19:55 +0000</pubDate><guid>https://blog.bjdean.id.au/2017/03/bandwidth-measurement-using-netcat/</guid><description>&lt;p&gt;For plain bytes/second bandwidth testing - ie without taking things like encryption overhead and compression improvements into account - the &lt;a href="http://nc110.sourceforge.net/"&gt;netcat&lt;/a&gt; command-line utility is pretty handy.
Once installed on both servers (let&amp;rsquo;s call them serverA and serverB):&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;start &lt;strong&gt;netcat&lt;/strong&gt; to listen on one server and pipe the output through &lt;strong&gt;wc -c&lt;/strong&gt; to both count the bytes (for confirmation) but also so that the bytes are not written to a filesystem or the terminal (which would cause a bottleneck and likely reduce the apparent bandwidth). By default &lt;strong&gt;nc&lt;/strong&gt; will quit when the first network connection it accepts is closed:&lt;/li&gt;
&lt;/ol&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;serverA$ nc -l -p 12345 | wc -c
&lt;/code&gt;&lt;/pre&gt;&lt;ol start="2"&gt;
&lt;li&gt;start &lt;strong&gt;netcat&lt;/strong&gt; to send data from the other server - using &lt;strong&gt;dd&lt;/strong&gt; to send as quickly as possible (using &lt;strong&gt;/dev/zero&lt;/strong&gt; is fast). Using &lt;strong&gt;-q 0&lt;/strong&gt; will cause &lt;strong&gt;netcat&lt;/strong&gt; to quit as soon as it sees an end of file (EOF):&lt;/li&gt;
&lt;/ol&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;serverB$ dd if=/dev/zero bs=$((2**20)) count=$((2**10)) | nc -q 0 serverA 12345
&lt;/code&gt;&lt;/pre&gt;&lt;ol start="3"&gt;
&lt;li&gt;On the sending server (&lt;strong&gt;serverB&lt;/strong&gt;) the output will show the number of bytes transmitted and the time it took to do that:&lt;/li&gt;
&lt;/ol&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;1048576+0 records in
1048576+0 records out
1073741824 bytes (1.1 GB) copied, 9.68678 s, 111 MB/s
&lt;/code&gt;&lt;/pre&gt;&lt;ol start="4"&gt;
&lt;li&gt;And on the other server (serverA) the number of bytes will be printed (confirming the transmission was complete):&lt;/li&gt;
&lt;/ol&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;1073741824
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;Run this in both directions a few times to get a good feeling for the bandwidth between the servers.&lt;/p&gt;</description></item><item><title>Cache::FileCache Thoughts</title><link>https://blog.bjdean.id.au/2016/09/cachefilecache-thoughts/</link><pubDate>Fri, 09 Sep 2016 05:19:51 +0000</pubDate><guid>https://blog.bjdean.id.au/2016/09/cachefilecache-thoughts/</guid><description>&lt;p&gt;If using perl Cache::FileCache, some comments offering a helpful starting point to get a cache that works in a relatively well behaved and unsurprising way:&lt;/p&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;use Cache::FileCache;
use File::Spec;
my $cache = Cache::FileCache-&amp;gt;new({
    # &amp;#34;The namespace associated with this
    # cache.&amp;#34;
    namespace =&amp;gt; &amp;#39;BJD-TESTING-app_id_cache_id&amp;#39;,
    # &amp;#34;The default expiration time for
    # objects place in the cache.&amp;#34;
    # This is in seconds.
    default_expires_in =&amp;gt; 1,
    # &amp;#34;Sets the auto purge interval. If this
    # option is set to a particular time ( in
    # the same format as the expires_in ),
    # then the purge( ) routine will be
    # called during the first set after the
    # interval expires. The interval will
    # then be reset.&amp;#34;
    #
    # XXX To work needs at least one of
    # auto_purge_on_set or auto_purge_on_get
    # to be set to true
    auto_purge_interval =&amp;gt; 5,
    auto_purge_on_set =&amp;gt; 1,
    # restrict access to the cache to just this
    # user (data security)
    #
# NOTE that if you set directory_umask
# but no cache_root you can end up clashing
# with other users who will also be trying
# to use (and maybe set the umask) on
# /tmp/FileCache
#
    # If we care about data security set
    # cache_root as well to put the files in
    # out own private directory:
    directory_umask =&amp;gt; 0077,
    cache_root =&amp;gt;
      File::Spec-&amp;gt;tmpdir()
      . &amp;#34;/bjdean-perl-Cache-FileCache&amp;#34;,
});
&lt;/code&gt;&lt;/pre&gt;</description></item><item><title>/usr/bin/base64 - copying and pasting code / patches betweeen terminals</title><link>https://blog.bjdean.id.au/2016/08/usrbinbase64-copying-and-pasting-code-patches-betweeen-terminals/</link><pubDate>Tue, 09 Aug 2016 06:29:49 +0000</pubDate><guid>https://blog.bjdean.id.au/2016/08/usrbinbase64-copying-and-pasting-code-patches-betweeen-terminals/</guid><description>&lt;h2 id="the-scenario"&gt;The Scenario&lt;/h2&gt;
&lt;p&gt;A couple of terminals open, connected to a mix of your own workstation, local development servers and remote servers running on a different network and frequently behind a variety of security barriers.
You want to copy a smallish chunk of code from a file across the network, or the output of a diff to apply a patch - but characters like whitespace and newlines which should not change will frequently be modified through the copy. You end up with whitespace changes where you don&amp;rsquo;t want them (perhaps later causing source code control merges to fail, and patches will fail straight away).&lt;/p&gt;</description></item><item><title>Perl Gotcha : xor precedence</title><link>https://blog.bjdean.id.au/2016/03/perl-gotcha-xor-precedence/</link><pubDate>Tue, 01 Mar 2016 03:12:50 +0000</pubDate><guid>https://blog.bjdean.id.au/2016/03/perl-gotcha-xor-precedence/</guid><description>&lt;p&gt;Perl has a &lt;a href="http://perldoc.perl.org/perlop.html#Logical-or-and-Exclusive-Or" title="Logical or and Exclusive Or"&gt;logical xor operator&lt;/a&gt; but it happens to be at the very bottom of the &lt;a href="http://perldoc.perl.org/perlop.html#Operator-Precedence-and-Associativity" title="Operator Precedence and Associativity"&gt;precedence priority list&lt;/a&gt; , importantly below the humble comma. So this innocent looking piece of code:&lt;/p&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;foobar( $a xor $b, $c );
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;Is interpreted as:&lt;/p&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;foobar( $a xor &amp;lt;strong&amp;gt;($b, $c)&amp;lt;/strong&amp;gt; );
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;And note unlike the &lt;strong&gt;not&lt;/strong&gt;, &lt;strong&gt;and&lt;/strong&gt; and &lt;strong&gt;or&lt;/strong&gt; low-precedence logical operators,  there is no high-precedence &lt;strong&gt;xor&lt;/strong&gt; operator.
So it&amp;rsquo;s a good idea to wrap &lt;strong&gt;xor&lt;/strong&gt;s in parentheses to guarantee intended behaviour:&lt;/p&gt;</description></item><item><title>Choc-Banana-Chia Spread</title><link>https://blog.bjdean.id.au/2016/02/choc-banana-chia-spread/</link><pubDate>Fri, 26 Feb 2016 12:00:00 +1100</pubDate><guid>https://blog.bjdean.id.au/2016/02/choc-banana-chia-spread/</guid><description>A quick and tasty chocolate banana spread thickened with chia seeds, perfect for toast.</description></item><item><title>Diced Lamb and Beef Turkish Style</title><link>https://blog.bjdean.id.au/2015/12/diced-lamb-and-beef-turkish-style/</link><pubDate>Tue, 08 Dec 2015 12:00:00 +1100</pubDate><guid>https://blog.bjdean.id.au/2015/12/diced-lamb-and-beef-turkish-style/</guid><description>Spiced diced lamb and beef with sultanas, pine nuts, and rice, served over mashed potato with sumach.</description></item><item><title>Yogurtlu Inspired</title><link>https://blog.bjdean.id.au/2015/12/yogurtlu-inspired/</link><pubDate>Tue, 08 Dec 2015 12:00:00 +1100</pubDate><guid>https://blog.bjdean.id.au/2015/12/yogurtlu-inspired/</guid><description>A Turkish-inspired dish with meat, crispy bread, tomato, yogurt, and melted cheese, topped with sumach.</description></item><item><title>Avoiding Custom Internet Explorer Error Pages</title><link>https://blog.bjdean.id.au/2015/09/avoiding-custom-internet-explorer-error-pages/</link><pubDate>Thu, 03 Sep 2015 05:58:22 +0000</pubDate><guid>https://blog.bjdean.id.au/2015/09/avoiding-custom-internet-explorer-error-pages/</guid><description>&lt;p&gt;A bit more of a short and to the point comment rather than an actual article.
Various web browsers used to display a &amp;ldquo;helpful&amp;rdquo; page saying &amp;ldquo;something is broken&amp;rdquo; when a website responded with a HTTP 4xx or 5xx error response. At a time when those pages were frequently the apache default error page these custom pages were still not a very good idea, but these days useful information is frequently included in error pages (in particular for errors like 410 Gone).
Most browsers have switched to showing the actual error page sent by the web server, but Internet Explorer keeps a version of this custom error page alive. Fortunately there&amp;rsquo;s a work-around which is that Internet Explorer does this with a silent switch when the error page has at least 512 bytes in the initial response (ie not including any linked content like scripts, styles and images).
The upshot of this is that these pages will display a custom error page on Internet Explorer:&lt;/p&gt;</description></item><item><title>Docker FROM debian:latest</title><link>https://blog.bjdean.id.au/2015/08/docker-from-debianlatest/</link><pubDate>Mon, 17 Aug 2015 13:22:03 +0000</pubDate><guid>https://blog.bjdean.id.au/2015/08/docker-from-debianlatest/</guid><description>&lt;p&gt;So in my &lt;a href="http://blog.bjdean.id.au/2015/08/docker-from-scratch/" title="Docker FROM scratch"&gt;last article&lt;/a&gt; I created a Docker container with nothing but a statically compiled helloworld in it. That demonstrated how it&amp;rsquo;s possible to define a very very (very very) simple container.
Here I define a similarly simple container with more complex dependencies - a hello world perl script requiring the &lt;a href="http://search.cpan.org/perldoc?Modern%3A%3APerl" title="Modern::Perl"&gt;Modern::Perl&lt;/a&gt; perl module. Now I don&amp;rsquo;t even need to build the executable binary, but all of a sudden I need to include enough of an environment to support my simple perl script.
As far as working with Docker this is not much more complex than changing &amp;ldquo;FROM scratch&amp;rdquo; to &amp;ldquo;FROM debian:latest&amp;rdquo;. The result is a much bigger container which will take more resources to run, but other than providing the computing resources to support that I really don&amp;rsquo;t have to care about it that much.
The Dockerfile now looks like this:&lt;/p&gt;</description></item><item><title>Docker FROM scratch</title><link>https://blog.bjdean.id.au/2015/08/docker-from-scratch/</link><pubDate>Mon, 17 Aug 2015 12:50:07 +0000</pubDate><guid>https://blog.bjdean.id.au/2015/08/docker-from-scratch/</guid><description>&lt;p&gt;&lt;a href="https://www.docker.com/" title="Docker"&gt;Docker&lt;/a&gt; is a framework which makes it easy to wrap linux applications in &amp;ldquo;containers&amp;rdquo; - a sort chroot&amp;rsquo;d jail but with tools to take away the pain of setting up dependencies and also providing tools to help automate setting up dependencies between those containers.
The &lt;a href="https://hub.docker.com/" title="Docker Hub"&gt;Docker Hub&lt;/a&gt; provides a bunch of pre-defined containers from major projects like Ubuntu, mysql, Redis, nginx, wordpress, postgres and java. It&amp;rsquo;s also a central store (like github, but for Docker) of publicly created and licensed Docker containers.
What&amp;rsquo;s the simplest quickest way to start? Well there is a special base container called &lt;a href="https://hub.docker.com/_/scratch/" title="Docker SCRATCH"&gt;&lt;strong&gt;SCRATCH&lt;/strong&gt;&lt;/a&gt;, which is used in the Docker official &lt;a href="https://hub.docker.com/_/hello-world/" title="Docker hello world"&gt;hello world&lt;/a&gt; container. A container based on SCRATCH has nothing in it, so whatever you add has to be fully self sufficient. The Docker official hello world does this with some assembly code compiled by &lt;a href="http://www.nasm.us/xdoc/2.11.08/html/nasmdoc0.html" title="The Netwide Assembler: NASM"&gt;nasm&lt;/a&gt; which certainly covers the angle of a stand-alone executable but I think is arguably more complex than a statically compiled hello world - so here we go:&lt;/p&gt;</description></item><item><title>"Roast" Veggie Soup</title><link>https://blog.bjdean.id.au/2015/08/roast-veggie-soup/</link><pubDate>Mon, 10 Aug 2015 12:00:00 +1100</pubDate><guid>https://blog.bjdean.id.au/2015/08/roast-veggie-soup/</guid><description>A rich, earthy vegetable soup with caramelized mushrooms, slow-roasted in the oven.</description></item><item><title>Herman the Carrot Cake</title><link>https://blog.bjdean.id.au/2015/08/herman-the-carrot-cake/</link><pubDate>Mon, 10 Aug 2015 12:00:00 +1100</pubDate><guid>https://blog.bjdean.id.au/2015/08/herman-the-carrot-cake/</guid><description>A sourdough starter variation of Stephanie Alexander&amp;#39;s carrot cake recipe using Herman the German Friendship Cake starter.</description></item><item><title>Friends and Miscellany</title><link>https://blog.bjdean.id.au/old-blog/</link><pubDate>Mon, 27 Jul 2015 01:33:37 +0000</pubDate><guid>https://blog.bjdean.id.au/old-blog/</guid><description>&lt;h2 id="friends"&gt;Friends&lt;/h2&gt;
&lt;p&gt;Sites run by friends:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="https://aes.id.au/"&gt;Andrew Scott&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://jeamland.net/"&gt;Benno Rice&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://freelancing-gods.com/"&gt;Pat Allan&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="http://www.mechanicalcat.net/"&gt;Richard Jones&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://esci.thenewstatistics.com/"&gt;esci-web&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id="my-old-blogs--wikis"&gt;My old blogs / wikis&lt;/h2&gt;
&lt;p&gt;Some older and now inactive + static blogging and wiki projects:&lt;/p&gt;
&lt;p&gt;This blog ended in 2009: &lt;a href="https://bjdean.id.au/blog/"&gt;http://bjdean.id.au/blog/&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;And this wiki ended in the mid 2010s: &lt;a href="https://bjdean.id.au/wiki/"&gt;BradsWiki&lt;/a&gt;&lt;/p&gt;</description></item><item><title>Getting WordPress Up and Going</title><link>https://blog.bjdean.id.au/2015/07/getting-wordpress-up-and-going/</link><pubDate>Sat, 18 Jul 2015 13:05:38 +0000</pubDate><guid>https://blog.bjdean.id.au/2015/07/getting-wordpress-up-and-going/</guid><description>&lt;p&gt;Setting up &lt;a href="http://bjdean.id.au/blog/" title="Brad's Old Blog"&gt;WordPress&lt;/a&gt; server there were a couple of minor wrinkles to sort out. I&amp;rsquo;ve &lt;a href="http://bjdean.id.au/blog/" title="Brad's Old Blog"&gt;run a blog before&lt;/a&gt; before and that fell by the wayside when I started using &lt;a href="http://bjdean.id.au/wiki/" title="Brad's Wiki"&gt;a personal wiki&lt;/a&gt; instead. But this seems like a good opportunity to see how one of the very popular blogging platforms works and what&amp;rsquo;s involved in keeping that running under the hood.
I work primarily with &lt;a href="http://debian.org/" title="Debian"&gt;Debian&lt;/a&gt; systems, so that was a natural place to start. The &lt;a href="https://packages.debian.org/search?keywords=wordpress" title="wordpress deb"&gt;wordpress&lt;/a&gt; package makes it very easy to get the base dependencies going with a known supported version, so if you&amp;rsquo;re running a recent release of Debian that seems like a reasonable place to start as well. That said, this of course means that the package is reconfigured along Debian guidelines and I found that I needed to spend a little time working out how this was done before it made sense.&lt;/p&gt;</description></item><item><title>Protecting Joomla : User-Registration Spam Relay</title><link>https://blog.bjdean.id.au/2015/07/protecting-joomla-user-registration-spam-relay/</link><pubDate>Sat, 18 Jul 2015 10:35:05 +0000</pubDate><guid>https://blog.bjdean.id.au/2015/07/protecting-joomla-user-registration-spam-relay/</guid><description>&lt;h1 id="the-problem-a-default-setting"&gt;The Problem: A Default Setting&lt;/h1&gt;
&lt;p&gt;By default user registration is &lt;strong&gt;enabled&lt;/strong&gt;.
It&amp;rsquo;s important to realise that even though links to the user registration page may not have been included in the design of a Joomla site the components are still present and they will be regularly targetted by automatic spiders searching for vulnerable sites. Check access logs  for requests to paths like:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;/index.php/shop-login&lt;/li&gt;
&lt;li&gt;/index.php/shop-login?view=registration&amp;amp;layout=complete&lt;/li&gt;
&lt;li&gt;/index.php/component/users/?view=registration&lt;/li&gt;
&lt;li&gt;/index.php/component/user/?task=register&lt;/li&gt;
&lt;li&gt;/index.php?option=com_user&amp;amp;view=register&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;With user registration enabled scripts can use a Joomla site as an &lt;a href="https://en.wikipedia.org/wiki/Open_mail_relay" title="Open Mail Relay"&gt;open mail relay&lt;/a&gt; by registering users with target email addresses and inserting spam/attack payload into the user details. The Joomla site will send a confirmation email to the target email address, and any email tracing of the source of the email will lead directly to the weakened Joomla server.&lt;/p&gt;</description></item><item><title>So, This is Life</title><link>https://blog.bjdean.id.au/2015/06/so-this-is-life/</link><pubDate>Sat, 27 Jun 2015 12:15:19 +0000</pubDate><guid>https://blog.bjdean.id.au/2015/06/so-this-is-life/</guid><description>&lt;p&gt;Marvin once said:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Life, don&amp;rsquo;t talk to me about life&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;But really, what was the problem. Life&amp;rsquo;s not so bad and after all : there are puppies, and fish, and ponies, and food, and walks in the forest, and walks through the grass, and cycling, and talking, and meeting and travelling and all those other things without which Life would not be Life.
So here&amp;rsquo;s a soothing image instead:
&lt;a href="http://blog.bjdean.id.au/wp-content/uploads/2015/06/wallpaper_blue_skies.height900.jpg"&gt;&lt;img alt="snoooooozing" loading="lazy" src="http://blog.bjdean.id.au/wp-content/uploads/2015/06/wallpaper_blue_skies.height900.jpg"&gt;&lt;/a&gt; Soothing Image No. 123132512233&lt;/p&gt;</description></item><item><title>Slow Cooker Stew - Chicken, Sausage and Veggies</title><link>https://blog.bjdean.id.au/2015/02/slow-cooker-stew-chicken-sausage-and-veggies/</link><pubDate>Wed, 18 Feb 2015 12:00:00 +1100</pubDate><guid>https://blog.bjdean.id.au/2015/02/slow-cooker-stew-chicken-sausage-and-veggies/</guid><description>A hearty slow cooker stew with chicken, sausages, and vegetables, flavored with dried lemons and herbs.</description></item><item><title>Beef, Pork, Chicken Bolognese</title><link>https://blog.bjdean.id.au/2013/10/beef-pork-chicken-bolognese/</link><pubDate>Thu, 10 Oct 2013 11:34:32 +1100</pubDate><guid>https://blog.bjdean.id.au/2013/10/beef-pork-chicken-bolognese/</guid><description>A rich multi-meat ragù-style bolognese with three types of ground meat, aromatic vegetables, and herbs.</description></item><item><title>Herbal Tea - Extended Blend with Lemon Grass, Lemon Verbena, and Spearmint</title><link>https://blog.bjdean.id.au/2013/06/herbal-tea-extended-blend/</link><pubDate>Sat, 01 Jun 2013 12:00:00 +1100</pubDate><guid>https://blog.bjdean.id.au/2013/06/herbal-tea-extended-blend/</guid><description>An eight-ingredient herbal tea blend with specific proportions for optimal flavor balance.</description></item><item><title>Herbal Tea - Cinnamon, Cloves, Elderflower, Lemon Myrtle, Licorice Root</title><link>https://blog.bjdean.id.au/2012/07/herbal-tea-cinnamon-cloves-elderflower-lemon-myrtle-licorice-root/</link><pubDate>Wed, 18 Jul 2012 12:00:00 +1100</pubDate><guid>https://blog.bjdean.id.au/2012/07/herbal-tea-cinnamon-cloves-elderflower-lemon-myrtle-licorice-root/</guid><description>A five-ingredient herbal tea blend with warming spices and sweet licorice notes.</description></item><item><title>Rice Paper Roll Dipping Sauces</title><link>https://blog.bjdean.id.au/2011/02/rice-paper-roll-dipping-sauces/</link><pubDate>Tue, 01 Feb 2011 12:00:00 +1100</pubDate><guid>https://blog.bjdean.id.au/2011/02/rice-paper-roll-dipping-sauces/</guid><description>Two quick and tasty dipping sauces for rice paper rolls - one hoisin-based, one lemon-based.</description></item><item><title>Pumpkin, Sweet Potato, Potato, Onion and Garlic Gnocchi</title><link>https://blog.bjdean.id.au/2010/09/pumpkin-sweet-potato-potato-onion-and-garlic-gnocchi/</link><pubDate>Tue, 28 Sep 2010 12:00:00 +1100</pubDate><guid>https://blog.bjdean.id.au/2010/09/pumpkin-sweet-potato-potato-onion-and-garlic-gnocchi/</guid><description>Roasted vegetable gnocchi with caramelized pumpkin, sweet potato, potato, onion, and garlic.</description></item><item><title>Celeriac, Mushroom and Chicken with Rice</title><link>https://blog.bjdean.id.au/2010/07/celeriac-mushroom-and-chicken-with-rice/</link><pubDate>Sat, 10 Jul 2010 12:00:00 +1100</pubDate><guid>https://blog.bjdean.id.au/2010/07/celeriac-mushroom-and-chicken-with-rice/</guid><description>A one-dish meal combining protein, vegetables, and grains with aromatic spices and plum sauce.</description></item><item><title>Page Not Found</title><link>https://blog.bjdean.id.au/NotFound/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://blog.bjdean.id.au/NotFound/</guid><description>The page you&amp;rsquo;re looking for could not be found</description></item></channel></rss>