Software Development

It’s often the case when analysing system logs that you want to create a summary bringing together data from different lines of a log - for example capturing an initial and final state of a transaction, or (as in this example) capturing a date-time stamp and some data from a multi-line log entry. In this example I have a log file containing periodic extracts of ‘mysqladmin extended-status’ with a date-time line to record when the status was taken - for example (removing most of the lines with “…” for brevity): ...

If using perl Cache::FileCache, some comments offering a helpful starting point to get a cache that works in a relatively well behaved and unsurprising way: use Cache::FileCache; use File::Spec; my $cache = Cache::FileCache->new({ # "The namespace associated with this # cache." namespace => 'BJD-TESTING-app_id_cache_id', # "The default expiration time for # objects place in the cache." # This is in seconds. default_expires_in => 1, # "Sets the auto purge interval. If this # option is set to a particular time ( in # the same format as the expires_in ), # then the purge( ) routine will be # called during the first set after the # interval expires. The interval will # then be reset." # # XXX To work needs at least one of # auto_purge_on_set or auto_purge_on_get # to be set to true auto_purge_interval => 5, auto_purge_on_set => 1, # restrict access to the cache to just this # user (data security) # # NOTE that if you set directory_umask # but no cache_root you can end up clashing # with other users who will also be trying # to use (and maybe set the umask) on # /tmp/FileCache # # If we care about data security set # cache_root as well to put the files in # out own private directory: directory_umask => 0077, cache_root => File::Spec->tmpdir() . "/bjdean-perl-Cache-FileCache", });

The Scenario A couple of terminals open, connected to a mix of your own workstation, local development servers and remote servers running on a different network and frequently behind a variety of security barriers. You want to copy a smallish chunk of code from a file across the network, or the output of a diff to apply a patch - but characters like whitespace and newlines which should not change will frequently be modified through the copy. You end up with whitespace changes where you don’t want them (perhaps later causing source code control merges to fail, and patches will fail straight away). ...

So in my last article I created a Docker container with nothing but a statically compiled helloworld in it. That demonstrated how it’s possible to define a very very (very very) simple container. Here I define a similarly simple container with more complex dependencies - a hello world perl script requiring the Modern::Perl perl module. Now I don’t even need to build the executable binary, but all of a sudden I need to include enough of an environment to support my simple perl script. As far as working with Docker this is not much more complex than changing “FROM scratch” to “FROM debian:latest”. The result is a much bigger container which will take more resources to run, but other than providing the computing resources to support that I really don’t have to care about it that much. The Dockerfile now looks like this: ...

Docker is a framework which makes it easy to wrap linux applications in “containers” - a sort chroot’d jail but with tools to take away the pain of setting up dependencies and also providing tools to help automate setting up dependencies between those containers. The Docker Hub provides a bunch of pre-defined containers from major projects like Ubuntu, mysql, Redis, nginx, wordpress, postgres and java. It’s also a central store (like github, but for Docker) of publicly created and licensed Docker containers. What’s the simplest quickest way to start? Well there is a special base container called SCRATCH, which is used in the Docker official hello world container. A container based on SCRATCH has nothing in it, so whatever you add has to be fully self sufficient. The Docker official hello world does this with some assembly code compiled by nasm which certainly covers the angle of a stand-alone executable but I think is arguably more complex than a statically compiled hello world - so here we go: ...