Posts

This article starts with a conversation about adjusting compiled perl regular expressions and a sudden realisation - which went a little along the lines of: OtherPerson> I can force a regular expression to change, for example: OtherPerson> DB<1> $a = qr{Abc} OtherPerson> DB<2> $a =~ s/A/a/ OtherPerson> DB<3> x $a OtherPerson> 0 '(?^:abc)' Me> No that doesn't work does it, oh wait <strong>it does</strong>, why? It took me a few moments of poking this myself in the debugger to notice the key change - which was that the compiled regular expression had become a string. See in the below example that $a starts as a RegExp and then becomes a scalar (at which point the ref function returns the empty string): ...

Disclaimer: I’m a Software Engineer who has spent a bunch of time working on issues relating to the GDPR from a systems design/security/process point of view. I’m definitely not a lawyer so this is my perspective and not legal advice. If you want legal advice, get a lawyer. Revisited: see Revisiting how the GDPR affects my setting of cookies (for security : a legitimate interest) Question: How does the GDPR affect my setting of cookies? The General Data Protection Regulation (GDPR) protects the personal data of EU residents who are in the EU. This regulation comes into effect 25th May 2018 (see Article 51/Paragraph 4) and has been designed to bring personal data protection up to date as the previous directive (Directive 95/46/EC) was adopted in 1995 and had fallen well behind in being able to cover modern day data protection issues. There is a fundamental change for cookies between the previous directive (and it’s modernising follow-up directive Directive 2002/58/EC) and the GDPR: ...

Disclaimer: I’m a Software Engineer who has spent a bunch of time working on issues relating to the GDPR from a systems design/security/process point of view. I’m definitely not a lawyer so this is my perspective and not legal advice. If you want legal advice, get a lawyer. Question: Is my personal project subject to the GDPR? The General Data Protection Regulation (GDPR) protects the personal data of EU residents who are in the EU. This regulation comes into effect 25th May 2018 (see Article 51/Paragraph 4) and has been designed to bring personal data protection up to date as the previous directive (Directive 95/46/EC) was adopted in 1995 and had fallen well behind in being able to cover modern day data protection issues. So you have a personal project which includes storage and processing of personal data, perhaps online (but perhaps it’s not even a computer based system - this regulation covers any data processing which has data in a filing system). First things first - your project is not covered by the GDPR unless the personal data in it includes EU residents - for more information please see http://blog.bjdean.id.au/2018/05/territorial-scope-of-the-gdpr/ . If your project is covered based on Territorial Scope then (just for personal projects) if your project is run by a “natural person” and if the nature of the work of the project is that it is “in the course of a purely personal or household activity” then it is explicitly not covered by the GDPR in Article 2. “Material Scope” Paragraph (2c). The nature of a “household activity” is not clearly defined - certainly it’s not in the Definitions of the regulation. Some vague examples are given in Recital (18) : “Personal or household activities could include correspondence and the holding of addresses, or social networking and online activity undertaken within the context of such activities.”. That the burden of proof is going to be on the controller (you, for your personal project). Exactly what qualifies as a household activity would seem to be something that would vary depending on the nature of the household, but you would be hard-pressed to claim that your particular household typically harvests, stores and processes EU resident personal data in bulk. In addition Recital (18) does specify that the activity should have “no connection to a professional or commercial activity” so even if your household is all about data processing, as it’s a professional and/or commercial activity it’s still going to be covered by the GDPR. ...

Disclaimer: I’m a Software Engineer who has spent a bunch of time working on issues relating to the GDPR from a systems design/security/process point of view. I’m definitely not a lawyer so this is my perspective and not legal advice. If you want legal advice, get a lawyer. Question: How does the GDPR cover my data processing of EU resident personal data if I’m not an EU organisation? The General Data Protection Regulation (GDPR) protects the personal data of EU residents who are in the EU. This regulation comes into effect 25th May 2018 (see Article 51/Paragraph 4) and has been designed to bring personal data protection up to date as the previous directive (Directive 95/46/EC) was adopted in 1995 and had fallen well behind in being able to cover modern day data protection issues. The GDPR unambiguously covers all processing of personal data of “data subjects” (defined as natural persons) who are EU residents by organisations and people who are in the EU. However the regulation also protects EU residents when using non-EU services in certain cases. For service providers outside the EU this means: ...

Description This is a simple chicken tagine featuring warm Moroccan spices - cumin, allspice, and cinnamon - along with sweet vegetables and dates. The long, slow cooking creates tender chicken and beautifully melded flavors. Stats ⏱️ Prep time: 15 minutes ✦ Cook time: 2+ hours ✦ Total time: 2 hours 15 minutes 🍽️ Serves: 4 people Ingredients 2-3 tablespoons olive oil 1 tablespoon ground cumin 0.5-0.7 tablespoon ground allspice 0.5-0.7 tablespoon ground cinnamon 1 brown onion 0.5 sweet potato 0.5 zucchini Chicken pieces (maryland recommended) 4-6 dates Instructions Heat base over very low heat and add olive oil and spices (cumin, allspice and cinnamon). Stir and cook spices for a minute or two. Finely dice the onion, increase to low-medium heat, add onion, and cook briefly. Roughly dice the sweet potato and zucchini and add to the pan. Place chicken pieces into pan. Mix all ingredients and cook five minutes to distribute heat. Halve and de-seed the dates, add around the top of the mixture in the pan. Move to very low heat and cover. Cook for at least two hours. Try to avoid lifting the lid to check as moisture will be lost. Notes Based on this recipe from my old wiki. This recipe uses stacked steel pots as a Dutch oven substitute rather than a traditional tagine vessel. The key to this dish is the long, slow cooking with minimal disturbance - this allows the flavors to develop and the chicken to become incredibly tender. Serve with couscous or rice.

It’s often the case when analysing system logs that you want to create a summary bringing together data from different lines of a log - for example capturing an initial and final state of a transaction, or (as in this example) capturing a date-time stamp and some data from a multi-line log entry. In this example I have a log file containing periodic extracts of ‘mysqladmin extended-status’ with a date-time line to record when the status was taken - for example (removing most of the lines with “…” for brevity): ...

I’m at a conference, I have my ChromeBook (vanilla, no local linux installs or any such thing), I have internet and I’ve set up a VM out there in the cloud, I’m regularly closing my laptop and wondering between talks, so it would be handy to be able to resume my ssh session without having to restart and without having to type “screen -r” every time. My ChromeBook ssh client doesn’t let me set up a “ssh here and run this command” so instead, given that I’m only logging into my VM with a single session at a time I can add this to the end of my ~/.bashrc file: ...

I was entertained by Numberphile’s dice-based generation of an approximate of pi so thought I’d throw together a quick script to try this out: random_vals_pi The results look pretty good: $ for x in 1 2 3 4 5 ; do python random_vals_pi.py 100000000 100000; done 3.14130098162 3.13774230925 3.14174020697 3.13769082316 3.14274853819 The theory is based on having large integers, so reducing this reduces the accuracy of the approximation: $ for x in 1 2 3 4 5 ; do python random_vals_pi.py 10 100000; done 3.07962120525 3.0938112197 3.08601801522 3.08910857173 3.08599352408 And similarly reducing the number of samples does the same: ...

When you have more than one public key set up to be able to access a single account (ie more than one public key listed in the authorized_keys you may want to check which public key was used to make a login). Since openssh 6.3 (released 2013) the public key fingerprint is logged - for example the below shows a set of made up “Accepted publickey” entries from an ssh auth.log: ...

For plain bytes/second bandwidth testing - ie without taking things like encryption overhead and compression improvements into account - the netcat command-line utility is pretty handy. Once installed on both servers (let’s call them serverA and serverB): start netcat to listen on one server and pipe the output through wc -c to both count the bytes (for confirmation) but also so that the bytes are not written to a filesystem or the terminal (which would cause a bottleneck and likely reduce the apparent bandwidth). By default nc will quit when the first network connection it accepts is closed: serverA$ nc -l -p 12345 | wc -c start netcat to send data from the other server - using dd to send as quickly as possible (using /dev/zero is fast). Using -q 0 will cause netcat to quit as soon as it sees an end of file (EOF): serverB$ dd if=/dev/zero bs=$((2**20)) count=$((2**10)) | nc -q 0 serverA 12345 On the sending server (serverB) the output will show the number of bytes transmitted and the time it took to do that: 1048576+0 records in 1048576+0 records out 1073741824 bytes (1.1 GB) copied, 9.68678 s, 111 MB/s And on the other server (serverA) the number of bytes will be printed (confirming the transmission was complete): 1073741824 Run this in both directions a few times to get a good feeling for the bandwidth between the servers.

If using perl Cache::FileCache, some comments offering a helpful starting point to get a cache that works in a relatively well behaved and unsurprising way: use Cache::FileCache; use File::Spec; my $cache = Cache::FileCache->new({ # "The namespace associated with this # cache." namespace => 'BJD-TESTING-app_id_cache_id', # "The default expiration time for # objects place in the cache." # This is in seconds. default_expires_in => 1, # "Sets the auto purge interval. If this # option is set to a particular time ( in # the same format as the expires_in ), # then the purge( ) routine will be # called during the first set after the # interval expires. The interval will # then be reset." # # XXX To work needs at least one of # auto_purge_on_set or auto_purge_on_get # to be set to true auto_purge_interval => 5, auto_purge_on_set => 1, # restrict access to the cache to just this # user (data security) # # NOTE that if you set directory_umask # but no cache_root you can end up clashing # with other users who will also be trying # to use (and maybe set the umask) on # /tmp/FileCache # # If we care about data security set # cache_root as well to put the files in # out own private directory: directory_umask => 0077, cache_root => File::Spec->tmpdir() . "/bjdean-perl-Cache-FileCache", });

The Scenario A couple of terminals open, connected to a mix of your own workstation, local development servers and remote servers running on a different network and frequently behind a variety of security barriers. You want to copy a smallish chunk of code from a file across the network, or the output of a diff to apply a patch - but characters like whitespace and newlines which should not change will frequently be modified through the copy. You end up with whitespace changes where you don’t want them (perhaps later causing source code control merges to fail, and patches will fail straight away). ...

Perl has a logical xor operator but it happens to be at the very bottom of the precedence priority list , importantly below the humble comma. So this innocent looking piece of code: foobar( $a xor $b, $c ); Is interpreted as: foobar( $a xor <strong>($b, $c)</strong> ); And note unlike the not, and and or low-precedence logical operators, there is no high-precedence xor operator. So it’s a good idea to wrap xors in parentheses to guarantee intended behaviour: ...

Description This is a simple and quick spread that’s very tasty on toast. The chia seeds thicken the mixture and add nutritional value. Refrigerate after making to allow the chia seeds to properly gel. Stats ⏱️ Prep time: 5 minutes ✦ Misc time: 30 minutes (refrigeration) ✦ Total time: 35 minutes 🍽️ Serves: Makes approximately 1 cup Ingredients 2 bananas 1/2 cup chia seeds 1 tablespoon cacao powder Instructions Mash or blend the bananas until smooth. Add chia seeds and cacao powder. Mix until fully combined. Refrigerate to allow chia seeds to thicken the mixture. Notes Based on this recipe from my old wiki. The chia seeds will absorb moisture and thicken the spread as it sits in the refrigerator. This spread is best consumed within a few days. Great on toast, crackers, or as a dip for fruit.

Description This Turkish-inspired dish features finely diced lamb and beef cooked with warm spices, sweetened with sultanas, and enriched with pine nuts. It’s served over creamy mashed potato and finished with a generous sprinkle of sumach for a tangy, vibrant finish. Stats ⏱️ Prep time: 20 minutes ✦ Cook time: 30 minutes ✦ Total time: 50 minutes 🍽️ Serves: 4 people Ingredients Lamb and beef, diced into small (½cm) cubes Sultanas or similar dried fruit Pine nuts Olive oil Sumach Paprika Rice (approximately ½ cup) Potatoes (a mix of sweet and white varieties suggested) Instructions Prepare approximately half a cup of rice and let it cool slightly. Boil potatoes until tender (10-15 minutes), then mash. Cook the diced meat with paprika and olive oil, maintaining moisture. Incorporate sultanas and pine nuts, cooking until nuts are lightly browned. Mix in rice, using about one quarter rice relative to meat. Plate mashed potato, create an indent in the center, and fill with the meat mixture. Finish with generous sprinkles of sumach. Notes Based on this recipe from my old wiki. The combination of sweet sultanas, nutty pine nuts, and tangy sumach creates a wonderful balance of flavors characteristic of Turkish cuisine.

Description This Turkish-inspired dish combines crispy fried bread with savory meat, tangy yogurt, and melted cheese, all topped with a generous sprinkle of sumach. It’s a layered dish with contrasting textures and flavors that come together beautifully. Stats ⏱️ Prep time: 15 minutes ✦ Cook time: 25 minutes ✦ Total time: 40 minutes 🍽️ Serves: 4 people Ingredients Lamb chunks, sausages, or preferred meat chunks Bacon strips (optional) Canned diced tomato Diced bread Olive oil Plain or Greek yoghurt (1 cup) Grated cheese (1 cup) Sumach (to taste) Instructions Fry sausages and optional bacon until cooked. Coat diced bread with olive oil and add to pan. Stir gently without breaking the bread pieces. Once bread browns slightly, add diced tomato and stir gently. Simmer on low heat for 5-10 minutes until reduced. Add dollops of yoghurt over the top of the mix (do not stir in). Spread grated cheese over mix (do not stir in). Melt and brown cheese using a grill or cooking torch. Generously sprinkle sumach over the top. Serve with salad. Notes Based on this recipe from my old wiki. The key to this dish is maintaining the layers - don’t stir in the yogurt or cheese. The contrast between the crispy bread, creamy yogurt, and melted cheese creates wonderful texture. The sumach adds a tangy, slightly lemony finish that’s characteristic of Turkish cuisine.

A bit more of a short and to the point comment rather than an actual article. Various web browsers used to display a “helpful” page saying “something is broken” when a website responded with a HTTP 4xx or 5xx error response. At a time when those pages were frequently the apache default error page these custom pages were still not a very good idea, but these days useful information is frequently included in error pages (in particular for errors like 410 Gone). Most browsers have switched to showing the actual error page sent by the web server, but Internet Explorer keeps a version of this custom error page alive. Fortunately there’s a work-around which is that Internet Explorer does this with a silent switch when the error page has at least 512 bytes in the initial response (ie not including any linked content like scripts, styles and images). The upshot of this is that these pages will display a custom error page on Internet Explorer: ...

So in my last article I created a Docker container with nothing but a statically compiled helloworld in it. That demonstrated how it’s possible to define a very very (very very) simple container. Here I define a similarly simple container with more complex dependencies - a hello world perl script requiring the Modern::Perl perl module. Now I don’t even need to build the executable binary, but all of a sudden I need to include enough of an environment to support my simple perl script. As far as working with Docker this is not much more complex than changing “FROM scratch” to “FROM debian:latest”. The result is a much bigger container which will take more resources to run, but other than providing the computing resources to support that I really don’t have to care about it that much. The Dockerfile now looks like this: ...

Docker is a framework which makes it easy to wrap linux applications in “containers” - a sort chroot’d jail but with tools to take away the pain of setting up dependencies and also providing tools to help automate setting up dependencies between those containers. The Docker Hub provides a bunch of pre-defined containers from major projects like Ubuntu, mysql, Redis, nginx, wordpress, postgres and java. It’s also a central store (like github, but for Docker) of publicly created and licensed Docker containers. What’s the simplest quickest way to start? Well there is a special base container called SCRATCH, which is used in the Docker official hello world container. A container based on SCRATCH has nothing in it, so whatever you add has to be fully self sufficient. The Docker official hello world does this with some assembly code compiled by nasm which certainly covers the angle of a stand-alone executable but I think is arguably more complex than a statically compiled hello world - so here we go: ...

Description This experimental soup features earthy mushroom flavors that come through beautifully. The vegetables are first caramelized on the stovetop, then slow-roasted in the oven for enhanced depth. Select mushrooms with robust earthy characteristics for best results. Stats ⏱️ Prep time: 15 minutes ✦ Cook time: 45 minutes - 2.5 hours ✦ Total time: 1 hour - 2 hours 45 minutes 🍽️ Serves: 4 people Ingredients 2-3 tablespoons coconut oil (or lard/butter) 2 brown onions 2 potatoes 1 carrot 2 large tasty mushrooms Hot water Instructions Cut onions, potatoes, and carrots into 3-5mm cubes. Cut mushrooms into slightly larger 5-10mm pieces. Heat oil in an oven-safe saucepan to medium temperature. Add all ingredients simultaneously and mix thoroughly. Preheat oven to approximately 160°C. Stir occasionally over 15-30 minutes to prevent bottom burning and ensure even cooking. Pour hot water until ingredients are covered by roughly 1cm. Transfer pan to oven for minimum 30 minutes. Longer cooking enhances flavor - 2 hours is recommended for superior results. Notes Based on this recipe from my old wiki. The longer you cook this soup in the oven, the better the flavors develop. The earthy mushroom notes are the star of this dish, so choose your mushrooms carefully. This is a great soup to make when you have time to let it develop slowly in the oven.