Software Development

A very quick and simple comment on building single-quoted strings in shell scripts which include single quotes. Note that it’s not possible to include a single quote in a single-quoted string - for example the bash man page: Enclosing characters in single quotes preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash. And dash: Enclosing characters in single quotes preserves the literal meaning of all the characters (except single quotes, making it impossible to put single-quotes in a single-quoted string). ...

You want to use pip to install a module from PyPI. However your server distribution in a little old (at the time of writing this did not need to be all that old, so the problem has come up for me a number of times): example@localhost:~/piptest$ pip install mkdocs Downloading/unpacking mkdocs Cannot fetch index base URL http://pypi.python.org/simple/ Could not find any downloads that satisfy the requirement mkdocs No distributions at all found for mkdocs Storing complete log in /home/example/.pip/pip.log A first but not great fix is to simply override the install prefix using: ...

It’s important to read the documentation, and then read it again once or twice. PHP’s date_parse function parses a datetime string and returns an “array with information about the parsed date on success or FALSE on failure.” This makes it tempting to use the function something like: $datetime = date_parse($datetime_string); if ( $datetime === FALSE ) { # Failure - deal with the bad $datetime_string } else { # Success - use the $datetime } However in this case failure means the the date_parse function has failed to work at a fairly fundamental level and reading the docs further you find “In case the date format has an error, the element ’errors’ will contains the error messages.” And indeed: ...

This article starts with a conversation about adjusting compiled perl regular expressions and a sudden realisation - which went a little along the lines of: OtherPerson> I can force a regular expression to change, for example: OtherPerson> DB<1> $a = qr{Abc} OtherPerson> DB<2> $a =~ s/A/a/ OtherPerson> DB<3> x $a OtherPerson> 0 '(?^:abc)' Me> No that doesn't work does it, oh wait <strong>it does</strong>, why? It took me a few moments of poking this myself in the debugger to notice the key change - which was that the compiled regular expression had become a string. See in the below example that $a starts as a RegExp and then becomes a scalar (at which point the ref function returns the empty string): ...

It’s often the case when analysing system logs that you want to create a summary bringing together data from different lines of a log - for example capturing an initial and final state of a transaction, or (as in this example) capturing a date-time stamp and some data from a multi-line log entry. In this example I have a log file containing periodic extracts of ‘mysqladmin extended-status’ with a date-time line to record when the status was taken - for example (removing most of the lines with “…” for brevity): ...

If using perl Cache::FileCache, some comments offering a helpful starting point to get a cache that works in a relatively well behaved and unsurprising way: use Cache::FileCache; use File::Spec; my $cache = Cache::FileCache->new({ # "The namespace associated with this # cache." namespace => 'BJD-TESTING-app_id_cache_id', # "The default expiration time for # objects place in the cache." # This is in seconds. default_expires_in => 1, # "Sets the auto purge interval. If this # option is set to a particular time ( in # the same format as the expires_in ), # then the purge( ) routine will be # called during the first set after the # interval expires. The interval will # then be reset." # # XXX To work needs at least one of # auto_purge_on_set or auto_purge_on_get # to be set to true auto_purge_interval => 5, auto_purge_on_set => 1, # restrict access to the cache to just this # user (data security) # # NOTE that if you set directory_umask # but no cache_root you can end up clashing # with other users who will also be trying # to use (and maybe set the umask) on # /tmp/FileCache # # If we care about data security set # cache_root as well to put the files in # out own private directory: directory_umask => 0077, cache_root => File::Spec->tmpdir() . "/bjdean-perl-Cache-FileCache", });

The Scenario A couple of terminals open, connected to a mix of your own workstation, local development servers and remote servers running on a different network and frequently behind a variety of security barriers. You want to copy a smallish chunk of code from a file across the network, or the output of a diff to apply a patch - but characters like whitespace and newlines which should not change will frequently be modified through the copy. You end up with whitespace changes where you don’t want them (perhaps later causing source code control merges to fail, and patches will fail straight away). ...

So in my last article I created a Docker container with nothing but a statically compiled helloworld in it. That demonstrated how it’s possible to define a very very (very very) simple container. Here I define a similarly simple container with more complex dependencies - a hello world perl script requiring the Modern::Perl perl module. Now I don’t even need to build the executable binary, but all of a sudden I need to include enough of an environment to support my simple perl script. As far as working with Docker this is not much more complex than changing “FROM scratch” to “FROM debian:latest”. The result is a much bigger container which will take more resources to run, but other than providing the computing resources to support that I really don’t have to care about it that much. The Dockerfile now looks like this: ...

Docker is a framework which makes it easy to wrap linux applications in “containers” - a sort chroot’d jail but with tools to take away the pain of setting up dependencies and also providing tools to help automate setting up dependencies between those containers. The Docker Hub provides a bunch of pre-defined containers from major projects like Ubuntu, mysql, Redis, nginx, wordpress, postgres and java. It’s also a central store (like github, but for Docker) of publicly created and licensed Docker containers. What’s the simplest quickest way to start? Well there is a special base container called SCRATCH, which is used in the Docker official hello world container. A container based on SCRATCH has nothing in it, so whatever you add has to be fully self sufficient. The Docker official hello world does this with some assembly code compiled by nasm which certainly covers the angle of a stand-alone executable but I think is arguably more complex than a statically compiled hello world - so here we go: ...