Too Busy For Words - the PaulWay Blog

Fri 6th Nov, 2015

Open Source Developers Conference 2015

In the last week of October I attended the Open Source Developer's Conference in lovely Hobart. It was about 90 people this year - for some reason people don't come to it if they have to travel a bit further. It's their loss - this year was excellent.

We started with Dr Maia Sauren's keynote on all the many many ways that government departments and not-for-profit organisations are working to open up our access to transparent democracy. I've never seen a talk given by going through browser tabs before but it was a good indication of just how much work is going on in this field. Then we had Ben Dechrai demonstrating how easy it is to install malware on systems running PHP, Julien Goodwin talking about the mistakes people make when securing data (like thinking NATting is the answer), and Katie McLaughlin with a good round-up of why Javascript is actually a good language (and why the "WAT" talks are amusing but irrelevant to the discussion).

Tuesday afternoon was GIS afternoon. Patrick Sunter gave a really amazing talk about urban planning, demonstrating mapping transit time across a city like Melbourne interactively - drop a pin on the map and in three seconds or so the new isocron map would be generated. This allowed them to model the effects of proposed public transport changes - like a train line along the Eastern Freeway (get this done already!) - very quickly. Then Blair Wyatt demonstrated SubPos, a system of providing location data via WiFi SSID beacons - doesn't work on Apple phones though because Apple are into control. Matthew Cengia gave a comprehensive introduction into OpenStreetMap, then afternoon tea. I skipped the lightning talks since I normally find those a bit scattered - any talk where you spend more time hassling over how much time you have remaining and whether or not your technology is working is a talk wasted in my opinion. I needed a rest, though, since I was struggling with a nose and throat infection.

Then we headed off to dinner at the Apple Shed in the picturesque Huon Valley. Local ciders, local produce, good food, good company, good conversation. All the boxes satisfyingly checked :-). I bought a bottle of the Apple Schnapps to sample later.

Wednesday morning's keynote was by Mark Elwell and showed his experience as an educator looking at Second Life and OpenSim. This was a different take on openness - demonstrating how our desire to create and share is stronger than our greed. The things that SL and OpenSim have done to lock up 'intellectual property' and monetise people's interactions have generally hindered their success, and people still put hundreds or thousands of hours into modelling things just for the satisfaction of seeing it in a virtual world. It was a good reflection on one of the many reasons we create free open source software.

Casey West, Thor's younger brother, gave an excellent review of the 'time estimation' methods we've traditionally used in software engineering - the waterfall model, agile development, and scrum - and why they all usually end up with us lying making up how much time things take. One thing he said which struck home to me was "your company invests in you" - it was the answer to the problem of support (and security) being seen as a cost rather than a benefit. Kathy Reid gave an excellent talk about how to guide your career with some excellent speaking tips thrown in (an acknowledgement of country and assistance for hearing impaired people, amongst others). I skipped Paul Fenwick's CKAN talk as I wanted to prepare my lightning talk for later (hypocritical? Yes, I suppose so :-) ).

In the afternoon Chris Neugebauer gave a good demonstration on why HTTP/2 is going to rock, Scott Bragg talked about one of the more esoteric uses of BitCoin block chains, and Arjen Lentz showed the benefits (and absence of fail) in teaching primary school children to make their own robots (including soldering). Michael Cordover gave a highly anticipated talk on his progress trying to get the Australian Electoral Commission to reveal the source code for its "EasyCount" software that's used (amongst other things) to count Federal Senate elections. It's disappointing that the closed mindset exists so strongly in some areas of government - the reasons and the delays and the obstructions were more than just simple accident.

We then had a set of "Other Skills" lightning talks - people talking about other things they do outside of programming things. Unfortunately I can't remember many of these because I was preparing for mine, which was on constructing my electric motorbike. This was well received - quite a few people came up to me afterward to talk about motorbikes, and the practicalities of building an electric one. It's always satisfying to talk with people that don't need the basics (like "can't you put wind generators on it to generate power as you move?") explained.

The Thursday morning keynote was by Richard Tubb, talking about how we can create opportunities and use the situations we find ourselves in to open up and improve our lives, and showed some of the things achieved in the GovHack Tasmania he ran. Sven Dowideit, the author of Boot2docker, gave a good demonstration of the things you can do with containers - particularly good for build systems as they can be stripped down to avoid unexpected dependencies. Then I gave my talk on my experiences with logs and how we can improve the logs our programs generate; the feedback I got was good, but I'd like to add more examples and an actual library or two to implement the principles I talk about. Then John Dalton gave a talk about how to use ssh's tunnel flags; it was a good overview of how the various options work.

I don't remember what I was doing after lunch but I don't remember the first talk - I think I was resting again. I did see Jacinta Richardson's talk on RPerl, which is basically a library that compiles your Perl code into C++. It's useful for computationally intensive things but the author of RPerl seems to have bizarre notions of how to interact with a community - like refusing to look at Github issues and requesting they be put on his Facebook page instead. We had a couple of 'thunder' talks - the main one I can remember was Morgan's talk on her PhD on Second Life and OpenSim (her mentor was Mark Elwell), which touched on the same points of social and open interaction.

After afternoon tea we had Pia Waugh speaking via Hangout from her home in Canberra - she wasn't able to attend in person because of imminent child process creation (!). She talked about GovHack, leading some of the projects to open up government processes and her work in dealing with the closed mindset of some people in government departments. Pia is always so positive and engaged, and her energy and enthusiasm is a great inspiration to a lot of people who struggle with similar interactions with less-than-cooperative bureaucrats. Sadly though, it was another demonstration of how we really need a high speed broadband network - the video stalled occasionally and Pia's voice was garbled at some times because of bandwidth problems.

We had another set of lightning talks which I stayed around for - and good thing too, because Fraser Tweedale demonstrated an amazing new system called Deo. It's essentially "encryption keys as a network service": a client can store a key in a network server and then request it later automatically. The two situations Fraser demonstrated for this were unlocking your Apache SSL certificate when Apache starts up (using a pass phrase helper) and unlocking LUKS disk encryption automatically when a machine boots (using a helper in LUKS). Since I'd recently had a customer ask for this very thing - machines with encrypted disks for data security outside the corporate network but that boot without user intervention when in the presence of the key server - this was hugely useful. I'm watching the Deo project eagerly, and have changed my attitude to lightning talks. If only more of them could be like this!

As is common with open source events, OSDC 2015 was collecting money for charity - in this case, the Tasmanian Refugee Defence Fund. After Lev Lafayette donated $1000 to the cause, I decided to match it. The few glimpses we get into the abysmal conditions in our costly, closed offshore detention camps are harrowing - yet we don't see (many) people in them saying "you know, take me back to Syria, I'll take my chances there". We're only hurting the poorest of the poor and the most desperate of the desperate, and only because of the xenophobia created by the Coalition and the conservative media. We're damaging people for life, and burdening our own society in coping with the problems we've created. In my opinion we're going to find out in the upcoming decades just how bad that problem really is. Anything we can do to alleviate it now is a good thing.

Overall, OSDC 2015 was a great learning experience. The "hallway track" was just as beneficial as the talks, the food was good, the venue was good, and I was glad I came.

Last updated: | path: tech | permanent link to this entry

Sat 11th Jul, 2015

Labor on refugees

Sorry, technical folk, this is going to be a political blog post.

I recently got an email from my local member, Andrew Leigh, that raised an issue I feel passionately about; here is my response.

On 09/07/15 14:55, Andrew Leigh wrote:[snip]
> ▪ Some people have asked me *why Labor supported the government’s bill to
> continue regional processing*. This is a tough question, on which reasonable
> people can disagree, but the best answer to this is to read Bill Shorten’s
> speech to the House of Representatives
> on the day the legislation was introduced.
Hi Andrew,

I'm sorry, but I cannot agree with the logic Bill Shorten and the Labor party has expressed in that speech.

Firstly, anyone watching the international problems with refugees will realise that Australia's intake is pitiful and stingy compared to some of its key allies and comparable nations and especially when compared to its population size and lifestyle. It is hypocritical to say "we don't want people to risk journeying across the sea from Indonesia, but we're happy for them to remain illegal immigrants there", especially when you look at the life that those people face as refugees there.

As an aside, though, I would say that it is still partly correct - it is more humane for them to remain in Indonesia than to be detained indefinitely in the inhuman, underresourced and tortuous conditions on Manus Island and Nauru. It is shameful to me that the Labor party can ignore this obvious contradiction.

But more importantly, the logic that we're somehow denying "people smugglers a product to sell" by pushing boats back into international waters shows no understanding of people smuggling as a business. Australia is still very much a destination, it's just that people now come with visas on planes and they pay even more for this than they used to. There is still a thriving trade in getting people into Australia, it's just been made more expensive - in the same way that making heroin illegal has not caused it to suddenly vanish from the face of the earth.

All we're doing by punishing people who come by boat to seek refuge in Australia is punishing the very desperate, the worst off, the people who have literally fled with their clothes and nothing else.

Other people with money still arrive, overstay their visas, get jobs as illegal immigrants or on tourism visas. The ABC has exposed some of these ridiculous, unethical companies trading on foreign tourists and grey market labourers. The Labor party, of all parties, should be standing up for these people's rights yet it seems remarkably silent on this issue.

The point that I think Labor needs to learn and the point I ask you to express to your colleagues there is that we don't want Labor to return to its policies in 2010. We thought those were inhuman and unjust then, and we still do now. Invoking them as a justification for supporting the Government now is bad.

Personally, I want Labor to do three things with regard to refugees:

  1. Move back to on-shore detention and processing. The current system is vastly more expensive than it needs to be, and makes it more difficult for UN officials and our own members of parliament and judiciary to be able to examine the conditions of detention. The Coalition keeps telling everyone about how expensive their budget is but seems remarkably silent on why we're paying so much to keep refugees offshore.
  2. Provide better ways of settling refugees, such that one can cut the "people smuggler" middle men out of the deal.

    For example, set up refugee processing in places such as Sri Lanka and Afghanistan where many refugees come from. Set a fixed price per person for transportation and processing in Australia, such that it undercuts the people smugglers - according to figures I read in 2010 this could be $10,000 and still be 50% less than black market figures.

  3. Ensure accountability and transparency of the companies such as Serco that are running these centres. If the government was running them and people were being abused, the government would be held accountable; when private companies do this the government wipes its hands and doesn't do a thing.
And on a more conversational note, I'd be interested in your views on this as an economist. There is obviously an economy of people smuggling - do we understand it? Is there any economic justification for offshore detention? All markets must work with a certain amount of illegal activity - can we work _with_ the black market rather than trying to work against it?

I do appreciate your updates and information and I look forward to more of your podcasts.

All the best,


Last updated: | path: society | permanent link to this entry

Tue 23rd Sep, 2014

That time that I registered an electric vehicle

So, tell us a story, Uncle Paul.

Sure. One time when I was in Rovers, ...

No, tell us the story of how you got your electric motorbike registered!

Oh, okay then.

It was the 20th of February - a Friday. I'd taken the day off to get the bike registered. I'd tried to do this a couple of weeks before then, but I found out that, despite being told a month beforehand that the workload on new registrations was only a couple of days long, when I came to book it I found out that the earliest they could do was the 20th, two weeks away. So the 20th it was.

That morning I had to get the bike inspected by the engineer, get his sign-off, and take it down to the motor registry to get it inspected at 8:30AM. I also had to meet the plumber at our house, which meant I left a bit late, and by the time I was leaving the engineer it was already 8:15AM and I was in traffic. Say what you like about Canberra being a small town, but people like driving in and the traffic was a crawl. I rang the motor registry and begged for them to understand that I'd be there as soon as possible and that I might be a couple of minutes late. I squeaked into the entrance just as they were giving up hope, and they let me in because of the novelty of the bike and because I wasn't wasting their time.

The roadworthy inspection went fairly harmlessly - I didn't have a certificate from a weighbridge saying how heavy it was, but I knew it was only about eight kilos over the original bike's weight, so probably about 240 kilos? "OK, no worries," they said, scribbling that down on the form. The headlights weren't too high, the indicators worked, and there was no problem with my exhaust being too loud.

(Aside: at the inspection station there they have a wall full of pictures of particularly egregious attempts to get dodgy car builds past an inspection. Exhaust stuffed full of easily-removable steel wool? Exhausts with bit burnt patches where they've been oxy'd open and welded shut again? Panels attached with zip ties? Bolts missing? Plastic housings melted over ill-fitted turbos? These people have seen it all. Don't try to fool them.)

Then we came up to the really weird part of my dream. You know, the part where I know how to tap dance, but I can only do it while wearing golf shoes?

Er, sorry. That was something else. Then we came to the weird part of the process.

Modified vehicles have to get a compliance plate, to show that they comply with the National Code of Practice on vehicle conversions. The old process was that the engineer that inspected the vehicle to make sure it complied had blank compliance plates; when you brought the vehicle in and it passed their inspection, they then filled out all the fields on the plate, attached the plate to the vehicle, and then you transported it down to Main Roads. But that was a bit too open to people stealing compliance plates, so now they have a "better" system. What I had to do was:

  1. Get the bike inspected for road worthiness.
  2. They hand me a blank compliance plate.
  3. I then had to take it to the engineer, who told me the fields to fill in.
  4. He then told me to go to a trophy making place, where they have laser etchers that can write compliance plates beautifully.
  5. I arrive there at 11AM. They say it'll be done by about 2PM.
  6. Go and have lunch with friends. Nothing else to do.
  7. Pick etched compliance plate up.
  8. Take compliance plate back to engineer. Because he's busy, borrow a drill and a rivet gun and attach the plate to the bike myself.
  9. Take it back to Main Roads, who check that the plate is attached to the bike correctly and stamp the road worthiness form. Now I can get the bike registered.
Yeah, it's roundabout. Why not keep engrave the plates at Main Roads with the details the Engineer gives to them? But that's the system, so that's what I did.

And so I entered the waiting department. It only probably took about fifteen minutes to come up next in the queue, but it was fifteen minutes I was impatient to see go. We went through the usual hilarious dance with values:

Many months ago I had enquired about custom number plates, and it turns out that motorbikes can indeed have them. Indeed, I could buy "3FAZE" if I wanted. For a mere $2,600 or so. It was very tempting, but when I weighed it up against getting new parts for the bike (which it turned out I would need sooner rather than later, but that's a story for another day) I thought I'd save up for another year.

So I finally picked up my new set of plates, thanked her for her time, and said "Excuse me, but I have to do this:" and then yelled:


Well, maybe I kept my voice down a little. But I had finally done it - after years of work, several problems, one accident, a few design changes, and lots of frustration and gradual improvement, I had an actual, registered electric motorbike I had built nearly all myself.

I still get that feeling now - I'll be riding along and I'll think, "wow, I'm actually being propelled along by a device I built myself. Look at it, all working, holding together, acting just like a real motorbike!" It feels almost like I've got away with something - a neat hack that turns out to work just as well as all those beautifully engineered mega-budget productions. I'm sure a lot of people don't notice it - it does look a bit bulky, but it's similar enough to a regular motorbike that it probably just gets overlooked as another two-wheeled terror on the roads.

Well, I'll just have to enjoy it myself then :-)

Last updated: | path: personal / ebike | permanent link to this entry

Fri 19th Sep, 2014

Learning as we go

Today I broke my record for distance on the electric motorbike: 70km on a charge.

The battery at the end of that was at 121.8V, which is about 3.2V per cell, and the lowest cell was 3.16V. Watching the voltage as I was riding (taking quick glances) showed that even under load it wasn't dipping below 110V, so there's a good chance that most cells were still running just fine. It's possible that there's a cell with lower capacity, but I think as I ride it and the battery gets more chance to level out I'm actually improving its range.

Unfortunately the battery meter on the bike thinks that power is leaking out when it isn't, so that doesn't tell me much. The meter on the BMS read "0%" at 38Km (the last time I read it) and "0%" before I started charging it, so I have no idea whether that's being caused by a low cell or some other random error. Either way, it's still trial and error to see how much distance I can actually get out of the battery.

According to the meter on the wall socket, that was half a kilowatt-hour. According to my calculations, that was about 11 amps at 120 volts over four hours, so about five kilowatt-hours. Either the decimal point is wrong on the meter and it's an order of magnitude too low, I'm reading it wrong, or it's just plain incorrect. Five kilowatt-hours is in the right ball park. At $0.17 per kilowatt-hour I just paid 89 cents to fill up the bike. So it's about 1.2 cents per kilometre, at this rough guess.


The other amusing thing is that I'm having to get used to coming back to the bike to find people peering intently at it. Fluoro-clad workers, motorbike enthusiasts, general passers by - I get all types. There are still lots of people who walk on by, so I don't think I've really changed the planet. But it's still fun to explain it and to see people's different reactions, all without exception positive. That's pretty cool.

Last updated: | path: personal / ebike | permanent link to this entry

Mon 4th Aug, 2014

Stop the unemployed

I think there's quite a simple answer to why the Coalition is putting so much effort into punishing the unemployed: it's exactly the same as their "stop the boats" policy. If they make it so unattractive to actually register as unemployed (in order to get Centrelink benefits) then a lot of people will just not register. They'll look for work, and they'll depend on the charities and on other forms of income (begging, stealing, etc), but they won't show up in the books as unemployed. Instantly, unemployment goes down, the Coalition looks like it's "tough on dole bludgers", and someone else foots the bill.

I've been there. I was unemployed for six months in late 2000. I tried to get benefits, and after spending every day trying to find work so that I could tick off my twenty-five job searches a month I was told that since I had shares I didn't qualify for assistance. In other words, you can't save, you can't have any reserves, you have to be scraping the bottom of the barrel yourself before you get any money from the Government.

So I just didn't bother. I didn't show up to Centrelink again, and they (presumably) removed my name from the list of unemployed. Howard at that time was trumpeting the work he was doing to reduce unemployment and kept pointing to the unemployment figures. No-one looked at the number of jobs at the same time. I think a few people pointed out at the time that the worse of his policies - which match those of the Coalition today - were just designed to punish the unemployed and make them into a cheap work force rather than actually get more jobs.

It's especially poignant since the Coalition also wants to sign into law trade partnerships that send jobs overseas (by making it uneconomical to manufacture things here in Australia), denying climate change and cutting spending on renewable energy (stopping the entire renewable energy sector from investing and thus losing jobs), cutting funding for education and health (as if States are suddenly going to find that money somewhere - more jobs lost), and telling Toyota and Holden that they get nothing to keep people employed here in Australia.

It's been well pointed out that there only about 146,100 jobs for 741,700 unemployed people. But the government's own way of calculating this only thinks a person is unemployed if they actively looked for work in the week of being surveyed - they identify an entire group called "currently inactive (not in the labour force)" and then coyly ignore that group entirely, not even mentioning it on the page.

That group comprises a lot of people, some of who need help. It's the long term unemployed who have given up on the mindless form-filling, tracking and justification required for government payments. It includes mothers who stay at home full time to care for their children (which is a full time job in my opinion). It includes lots of disabled people. It includes people on pensions. And it's that group that seems to be conveniently ignored by the Coalition in their War On Bludgers.

Last updated: | path: society | permanent link to this entry

Wed 9th Jul, 2014

New web server, same old content.

Over the last couple of years I've been implementing bits of my website in Django. Those initially started on my home server, and recently I moved them to a test domain on the new server my web host provided. Then they advised me that their old hardware was failing and they'd really like to move my domain off it onto the old one.

So I took backups, and copied files, and wrote new code, and converted old Django 1.2 code which worked in Django 1.4 up to the new standards of Django 1.6. Much of the site has been 404'ing for the last couple of days as I fix problems here and there. It's still work in progress, especially fixing the issues with URL compatibility - trying to make sure URLs that worked in the old site, in one Perl-based CGI system, work in the new site implemented in Django with a changed database structure.

Still, so far so good. My thanks once again to Daniel and Neill at Ace Hosting for their help and support.

Last updated: | path: tech / web | permanent link to this entry

Fri 11th Apr, 2014

Sitting at the feet of the Miller

Today I woke nearly an hour earlier than I'm used to, and got on a plane at a barely dignified hour, to travel for over three hours to visit a good friend of mine, Peter Miller, in Gosford.

Peter may be known to my readers, so I won't be otiose in describing him as a programmer with great experience who's worked in the Open Source community for decades. For the last couple of years he's been battling Leukaemia, a fight which has taken its toll - not only on him physically and on his work but also on his coding output. It's a telling point for all good coders to consider that he wrote tests on his good days - so that when he was feeling barely up to it but still wanted to do some coding he could write something that could be verified as correct.

I arrived while he was getting a blood transfusion at a local hospital, and we had spent a pleasurable hour talking about good coding practices, why people don't care about how things work any more, how fascinating things that work are (ever seen inside a triple lay-shaft synchronous mesh gearbox?), how to deal with frustration and bad times, how inventions often build on one another and analogies to the open source movement, and many other topics. Once done, we went back to his place where I cooked him some toasted sandwiches and we talked about fiction, the elements of a good mystery, what we do to plan for the future, how to fix the health care system (even though it's nowhere near as broken as, say, the USA), dealing with road accidents and fear, why you can never have too much bacon, what makes a good Linux Conference, and many other things.

Finally, we got around to talking about code. I wanted to ask him about a project I've talked about before - a new library for working with files that allows the application to insert, overwrite, and delete any amount of data anywhere in the file without having to read the entire file into memory, massage it, and write it back out again. Happily for me this turned out to be something that Peter had also given thought to, apropos of talking with Andrew Cowie about text editors (which was one of my many applications for such a system). He'd also independently worked out that such a system would also allow a fairly neat and comprehensive undo and versioning system, which was something I thought would be possible - although we differed on the implementation details, I felt like I was on the right track.

We discussed how such a system would minimise on-disk reads and writes, how it could offer transparent, randomly seekable, per-block compression, how to recover from partial file corruption, and what kind of API it should offer. Then Peter's son arrived and we talked a bit about his recently completed psychology degree, why psychologists are treated the same way that scientists and programmers are at parties (i.e. like a form of social death), and how useful it is to consider human beings as individual when trying to help them. Then it was time for my train back to Sydney and on to Canberra and home.

Computing is famous, or denigrated, as an industry full of introverts, who would rather hack on code than interact with humans. Yet many of us are extroverts who don't really enjoy this mould we are forced into. We want to talk with other people - especially about code! For an extrovert like myself, having a chance to spend time with someone knowledgeable, funny, human, and sympathetic is to see sun again after long days of rain. I'm fired up to continue work on something that I thought was only an idle, personal fantasy unwanted by others.

I can only hope it means as much to Peter as it does to me.

Last updated: | path: tech | permanent link to this entry

Tue 11th Feb, 2014

The Day We Fight Back?

I've blacked out and my partially complete Django replacement for it as part of:

The Day We Fight Back.

It's a token gesture, and I'd prefer something that actually causes a real change in the state of affairs. But hopefully the few people that visit my site will ask why its blacked out, and I'll tell them. Or they'll find out why for themselves. Or they'll know already.

Ultimately, the thing that worries me in all of this is that all the data collection, all the wire tapping and interception, all the bad cryptography and bastardised standards, all the spying and all the secrecy doesn't really improve our actual security. It hasn't found anything that normal detective work and normal policing and existing laws couldn't already deal with. It hasn't prevented any crimes, either against real people or against 'the state' or anything.

The 'baddies' are already adapting their methods and covering their tracks. There are far too many false positives, and much too much confirmation bias, to make the resulting 'intelligence' anything but a joke. The FBI already spends more money on covering up its mistakes - like its total waste of resources watching Brandon Mayfield - than it would if it had just asked him for an interview. Mean time they're missing the Boston Marathon bombers despite lots of evidence pointing to them. Then follows a lot of chest puffing and excuses and "we can't tell you the details, they're classified".

(Meanwhile, we have banks that are laundering money to supply to exactly the same terrorist organisations that get a slap-on-the-wrist fine and no jail time for anyone because they're "too big to fail". So not only did the NSA and all the security TLAs not find a massive source of funding for these organisations - something that's causing far more damage to USAdian society than 'terrorism' - but the entire rest of the government quietly brushed it under the carpet and pretended it didn't happen. Yeah, good one.)

Ultimately, all it's really about is perpetuating the existince of the security complex - mainly in the USA, but everywhere really. Its first imperative is to preserve itself, and it has all the means to do so. It has the secret courts and the secret laws to prevent legal challenge, and the arms and the blackmail material to prevent other attacks. And its paranoid level of secrecy and security makes it automatically treat any rein, any check on it, as a threat to its own existence - because, well, it would be.

So what REALLY scares me is that nothing we do will actually stop them at all. At this stage, it's basically impossible to even rein in the NSA's powers - and that'd be like taking a rabid tiger and smacking it on the nose to tell it to go away. To put in the high-level open oversight that lets the public see whether these agencies are actually doing anything useful with the vast quantities of money they control is a task that's beyond the realistic abilities of any government (to say nothing of the blackmail and subversive influence that any security agency can bring against anyone that wants to downsize them). Tackling the companies who run the prisons and supply the equipment and make a profit from all the unrest - that's just bordering on insane.

We've made the tiger, and we've fed the tiger because it said it would protect us, and we're on its backs because it's better than being in its jaws, and we've fed it more because we're afraid it might eat us, and it's only grown larger and hungrier. To be honest, I think the security complex will kill the world before climate change does.

Last updated: | path: society | permanent link to this entry

Wed 15th Jan, 2014

Ignorable compression

On the way home from LCA, and on a whim, in Perth I started adding support for LZO compression to Cfile.

This turned out to have unexpected complications: while liblzo supports the wide variety of compression methods all grouped together as "LZO", it does not actually created '.lzo' files. This is because '.lzo' files also have a special header, added checksums, and file contents lists a bit like a tar file. All of this is added within the 'lzop' program - there is no external library for reading or writing lzo files in the same way that zlib handles gz files.

Now, I see three options here:

Yeah, I'm going for option one there.

LZO is a special case: it does a reasonable job of compression - not quite as much as standard gzip - but its memory requirements for compression can be miniscule and its decompression speed is very fast. It might work well for compression inside the file system, and is commonly used in consoles and embedded computers when reading compressed data. But for most common situations, even on mobile phones, I imagine gzip is still reasonably quick and produces smaller compressed output.

Now to put all the LZO work in a separate git branch and leave it as a warning to others.

Last updated: | path: tech / c | permanent link to this entry

All posts licensed under the CC-BY-NC license. Author Paul Wayper.

Main index / tbfw/ - © 2004-2010 Paul Wayper

Valid HTML5 Valid CSS!