Yestarday was fun. Making that face was also fun. :-)

Today I cleaned the apartment.

I'm about to bike to Noatun to buy groceries, launch subscriptions on pagekite.net and then cook dinner for my girlfriend.

I am ever so slightly hung over today, for some unfathomable reason...

My photo, taken at 00:00, April 30, 2012.

Oops! It appears I haven't blogged for just under 2 months. Sorry about that.

Things that have happened in my life, in no particular order:

I'm sure I am forgetting something.

It's odd how sometimes I feel like I'm getting nothing done, until I write down a list like that. Jeez I've been busy. I've also been a bit stressed out, but Malta helped with that.

Happy birthday, me! My plans:

Since Ewelina's not around, I am going to postpone any proper partying until next weekend. There may be a party on Friday or Saturday night. Or maybe just a game of pool at Bar 46. I haven't decided. :-)

I have now been growing my sexy 'stache for almost a week! I am sure my team-mates will be proud.

It is easy to see. You just have to squint a bit.

How awesome is that?

Very.

In March, Icelandic men grow moustaches!

Well, some do - to support the Mottumars cancer awareness and fund-raising campaign.

I will be sacrificing my dignity for this excellent cause and will let my sparse, blonde upper lip growth flourish for the next few weeks.

My 'stache will belong to the FSFI team.

I just had a minor epiphany.

I was scanning Twitter and someone mentioned Apache Hadoop. Suddenly I realized:

Apache Hadoop is not a Free Software project.

Hadoop may, in some technical/legal sense be Free Software, but I will now stick my neck out and repeat:

Apache Hadoop is not a Free Software project.

Wait, hold your fire, let me explain...

Sometimes it's easiest to identify a social movement by what it is not. Atheists are not religious. Pacifists are against violence. These explanations are clear and far easier to understand than a hypothetical list of all the things atheists or pacifists are.

Hadoop is Open Source. It is a cluster computing system which companies and scientists love, but individuals and consumers couldn't care less about. It is based on ideas made popular (invented?) by Google, but Google kept their code to themselves, so the Hadoop project was created to advance the state of the art and bring MapReduce to the rest of the industry.

To me, Hadoop is the perfect poster child for Open Source, but most of the Free Software movement won't care about it one bit. Thinking about why shines a strong light on the differences between the two movements.

Why does this matter?

Well, although I'm sure most people don't care, us geeks can get really emotional about the difference between Free Software and Open Source. Violent, even.

Every geek has an opinion. But when asked to explain, we falter and wave our hands, "admitting" that there's really no difference, or spouting vague nonsense about pragmatism vs. idealism or even communism vs. capitalism.

I've always found this deeply unsatisfying. Because although I respect Open Source, I love Free Software.

So what is the difference? I propose the following definitions:

The Free Software Community writes and shares software with an explicit intent to safeguard the rights and freedoms of its users by eliminating antifeatures and natural monopolies.

The Open Source Community writes and shares software with an explicit intent to advance the state of the art of computing through open collaboration and publication.

Both are noble goals, members of both camps can stand proud. But when stated this way, they are also obviously different.

Hadoop fits perfectly into the latter category, but it does nothing at all for the former. If anything, it brings more antifeatures to the world and strengthens cloud-based monopolies by making deep data mining easier for all those people with their own personal datacenters.

Enough about Hadoop. Hadoop's cool.

What I want to talk about, is why I feel this is a useful way to explain these closely related movements. I think it helps me understand my community a tiny bit better, and I want to explain why.

Both the Open Source and Free Software movements use similar tools and licenses and there is a lot of overlap between them. Together they have built GNU/Linux, Firefox and many other wonderful things.

Most of the time they are allies - but not always.

I feel the definitions above shed light on why:

Free Software proponents often prefer licenses which restrict the freedom of the developer, in order to protect the end users from antifeatures and natural monopolies.

Open Source folks are focused on technical excellence and getting a job done. Talking about ideals and rights becomes a distraction at best.

Depending on the field, "advancing the state of the art" and "protecting users rights" may be fundamentally incompatible.

Free Software folks benefit greatly from Open Source, but their ideals and restrictive licenses make them ill equipped to reciprocate.

This last bullet in particular goes a long way towards explaining the animosity many Open Source advocates have shown the Free Software movement (Richard Stallman has borne the brunt of this). From their point of view, Free Software takes and takes, giving little back and adds insult to injury by passing moral judgement over those who don't live up to their lofty ideals.

Zealots (like me) are just so annoying! ;-)

But seriously, knowing where we differ can make it easier to "agree to disagree" and get on with all the things we do agree on. Which is probably most of them...

Everybody loves labeling things. Being able to pidgeonhole things into little boxes helps us feel like we understand them. This way of defining the two movements makes it really easy to do that sort of thing to various unsuspecting FLOSS projects.

For example:

The first category includes mostly tools which are written for direct use by individuals or with an explicit "social intent".

The second includes software written to solve particular hard technical problems or "platforms" which provide the foundations upon which end user software is built.

This of course is just my opinion. You're welcome to invent your own classification scheme and pidgeonhole my software any time.

As a techie, it's easy to read my definition and notice whether one or the other resonates more strongly. This in turn may help you choose your friends and decide which projects to contribute to.

This may also have implications for new software projects - it's easy for a developer to look at those two categories and decide which they want their project to focus on. That in turn will inform all manner of decisions, from choice of licenses to "branding" and advocacy.

It's also not a great stretch to expect projects to be more successful if they align themselves with the movement more interested in the problem they aim to solve.

As an extreme example, is entirely possible to imagine a successful Open Source project which is actively hostile towards the goals of the Free Software movement - an open DRM system would be a good example. Trying to create "GNU libdrm" would make no sense.

Conversely, the Open Source movement might not be very interested in "me too" projects which are merely providing an alternate solution to a well understood problem. But the Free Software folks would probably be very welcoming and supportive if there was no Freedom-focused project existing in that space.

Even though there are many things which unite us, and even though many of us belong to both categories, I think there are real differences between the Open Source and Free Software communities.

By highlighting them I don't mean to be divisive - quite the opposite. My hope is that by understanding our motivations better, we'll find it easier to get along, respect each others differences and work together.

Thanks for reading!

Please feel free to tell me what you think - I may end up converting this post into a talk or try to spread it more widely, so any and all feedback is very welcome.

Also, since you actually read this far, you might want to follow me on Twitter or subscribe to my RSS feed. Also, this is what I do for a living.

Comments and follow-ups:

It's funny how little, neglected things are sometimes the most successful.

One of the most widely distributed, popular tools I've ever written is a trivial little perl script named colormake. All it does is make the output of "make", when compiling a C program, more colorful and a bit easier (or at least more fun) to read.

According to the copyright on the file, I wrote it back in 1999! At the time, I was working on porting a gigantic C++ code-base to Linux. That may have been the most boring job I've ever had - colormake was a little distraction that supposedly helped me scan the builds for problems. It's pretty cool that people are still using it, I wonder if that's the case for the C++ monster I was porting...

Colormake is available as a Debian package:

As it turns out, the Debian version is better than mine - it fixes some bugs and is just a better tool.

I just went through my e-mail and integrated all the fixes people had sent me over the years, and now I'm looking at the Debian versions and integrating changes they've made...

I may just end up copying the Debian code and calling that Colormake 1.0. :-)

As I mentioned last week, Google has a bit of a bad smell these days...

It's funny, looking back at that post and the list of smells I complained about, that I somehow managed to forget the one smell that actually bothers me every single day:

I can't believe I left that one out!

This week, I decided to experiment with some Free Software alternatives to Google Reader. The one I am testing at the moment, is Tiny Tiny RSS.

Combined with PageKite and an Android app, Tiny Tiny RSS has the potential to do everything Google Reader did and more - using only Free Software and running on a computer that is fully under my control.

Here are my thoughts so far on how the two compare.

I have been using Google Reader for many years. It solved two problems for me, and it solved them really, really well:

One, was to skim large volumes of news. I really like their on-line user interface and I really like the Android app. This still works really well, and especially the Android app has been getting better over time.

Two, I used Google Reader to manage feeds, aggregate multiple feeds into one and for bookmarking and sharing interesting sites. Google Reader would let me re-export any category as an RSS feed and it also had a "Shared in Reader" feed which I could add to at any time by either pressing a button in Reader or the Android app, or by using a bookmarklet on any site on the web. This was by far the most convenient way I have found to curate my own collection of interesting links.

This second use case, bookmarking and sharing, is what Google broke when they decided to push Google Plus. They removed the ability to "share" articles within Reader and they broke the "Note in Reader" bookmarklet. The idea was to push people to sharing over Google Plus instead.

(I don't like being pushed around, so this was when I decided it was time to find a free-as-in-freedom alternative.)

But even before the scent of Google Plus started to permeate everything, Reader had two other flaws which I tolerated, but was never quite happy with.

The first is common to all cloud solutions: Reading news while off-line was impossible. This matters mostly when traveling - but I really like to read when traveling, and at times in my life I have traveled a great deal!

The other problem with Reader was I never felt I could trust it with password-protected, private feeds. I would actually like to keep track of quite a few such feeds, both work-related and personal. As an example, for a long time I had a password-protected section on this blog where I wrote about my personal life. I never felt comfortable enough to use Google Reader to keep track of peoples' comments in the private section, I had to just keeping reloading the page...

This mistrust was actually justified just last month, when I noticed that the GoogleBot was spidering works-in-progress on the PageKite website. The only explanation I have found for this, is that I used Google Reader and some semi-private RSS feeds to monitor activity on the site, so I wouldn't miss a new forum topic or comment on some random wiki page. It seems that once a feed is in Reader, the Google Bot knows about it too, even if there are no links to the content on the public web.

(This is a rather significant privacy issue, which anyone who thinks a secret or obfuscated URL is "private enough" should keep in mind. If the link ends up in any Google product, the Google Bot may find out about it and then the whole world will know...)

Tiny Tiny RSS is in active development and there are Debian packages available (from their own repo), which are two of the things I always look for when choosing new software. The software is written in PHP (ick) and can use either PostgreSQL or MySQL as a database.

I decided to install TT-RSS on my laptop, so the database will stay private and I will be able to read news even when I am offline. This solves both of the problems I had with the pre-Google-Plus Reader.

Installation went without a hitch. I had to edit the config file in /etc/tt-rss-mysql/ to enable multi-user support and on-line account creation in order to get a password-protected account. The Debian packages integrated TT-RSS very cleanly with my Apache config, which in turn made TT-RSS instantly visible on my laptop's public PageKite address. Awesome!

Tiny Tiny RSS was able to import Google Reader's OPML dump (the fact that Google offer this export is one of the reasons I don't consider them evil yet, just a bit smelly) without a hitch, meaning I didn't have to waste any time re-subscribing and re-organizing my feeds.

So far, so good. Next, I went looking for an Android client...

Tiny Tiny RSS actually has two different Android clients available in the Android Market. One is created by Andrew Dolgov, the author of TT-RSS. The other, (TTRSS-Reader), is currently maintained by Nils Braden. Both are open source.

Even though I run TT-RSS on my laptop, the Android apps were both able to communicate with it thanks to PageKite. TTRSS-Reader did a slightly better job handling my PageKite's SSL certificate, I had to tell Andrew's app to "accept any certificate", which isn't really OK from a security point of view.

So far I also prefer TTRSS-Reader's user interface. It's not as slick as Google's offering, but it's still quite nice. All that is missing is widgets on the home screen to let me know when there is new news to be read...

The worst news so far, is that Tiny Tiny RSS doesn't seem to do as good a job as Google Reader did managing feeds. It does allow me to "star" articles I like and then republish that list as an RSS feed, but that's all I could get it to do. There are some note-taking and archiving features built-in, but they didn't seem immediately useful.

I miss the ability to:

The first would probably be best solved by hacking on the TT-RSS source code, which might be fun but I probably don't have time for any time soon.

The second may be best solved by a minimal external RSS-authoring tool. I could probably hack one together, along with a Javascript bookmarklet, in an evening or two. We'll see!

Finally, the TT-RSS interface is definitely not as nice as Google Reader's and I find that TT-RSS is frustratingly slow, even though I'm just talking to my own laptop. But it is usable and I think I'll get used to it. Since most of the time I read news using my Android phone, the slowness of the web UI probably won't matter so much in practice.

Of course, since TT-RSS is an active proejct (or is it?), any or all of these issues may get better with time. I may also figure out solutions to some of them as I get more familiar with the software.

(If I had a bathtub full of money, I'd pay a designer to spruce up the TT-RSS user-interface and Android apps, a hacker to fix the sharing limitations and a tech-writer to document Tiny Tiny RSS properly. These problems are all very fixable.)

Overall, I'm happy.

It looks like combining Tiny Tiny RSS, TTRSS-Reader and PageKite will be good enough to break my Google Reader habit and bring my news consumption out of the cloud and onto my laptop.

I will have to live with a somewhat clunkier user-interface, but I've gained some real benefits instead. The setup needs some work and will take some getting used to, but overall, this is a huge step in the right direction. Hooray!

Today I got to wondering what it takes to create a directory structure with 500 million files, on Linux.

First, you need 500 million inodes and 500 million blocks, plus whatever is required for the directories themselves. Since a block on ext2 is at least 1k, that implies it will take at least 0.5TB of disk space. I don't have that much space free on my laptop for such a silly experiment, so I decided to set myself a more interesting goal, and see how much I could cram into 4GB instead.

By tweaking the inode-to-block ratio, I was able to create a 4GB file system with 2 million inodes and 3.6 million 1k blocks, like so:

This I mounted using the loopback driver:

Then I started creating directories and just experimenting.

One of the things I discovered, which I had forgotten, is that a hard link takes very little space because it is contained entirely inside the directory entry - it has neither a block nor an inode of its own.

However, I also discovered that a single file can only have 32000 hard links to it, most likely due to an upper limit on the reference counter.

The up-shot of this is, that I can actually create what looks like millions of files in this 4GB filesystem with relative ease. Assuming a 36 byte file-name, hard-links are costing about 42 bytes per "file". After counting the overhead of creating half a million directories, this leaves enough space for... 71 million files. Give or take.

So, assuming a similar structure, I should be able to create 500 million files in roughly 30 GB of space.

Of course, they will all have the same unexciting contents, but hey, at least the filenames can be entertaining or insightful! What this might be useful for, if anything, is left as an exercise for the reader...

Note: The mke2fs options above are not optimal for this exercise, as it turns out most of my directory entries ended up being relatively large, and I didn't need nearly that many inodes. Better values would would have been more efficient, lowered overhead and ultimately reduced the number of disk seeks.

I used to work for Google.

I was an odd one, because I didn't like it much. I did like my coworkers and I learned a lot - which was enough to keep me there for 3 long years - but I mostly felt useless, insignificant, isolated and guilty about earning too much money. I hated living in the Google bubble of elitism, TGIF and secrecy. I proved to myself that I could do the job, and then I left.

Since leaving, I have been critical but usually defended them. My impression while I worked there, was that Google genuinely meant well and was doing good work. I left the company less cynical than I joined it. I may not have enjoyed the job, but the company earned my respect.

Now I find that's wearing off.

The company under Larry Page, with it's focus on social and Google+, has, for lack of a better term, started to smell bad.

I'm not going to dwell on details, but these are the smells that bother me:

It smells like desperation. Loss of focus on the user. Weakness.

Don't get me wrong, Google are still leaps and bound better than Facebook, Apple, Microsoft or any major telco or media company. No other big name in mainstream tech has even come close to Google on civil liberties and giving back to the community.

But that is exactly what makes it so sad and worrying. Google was our champion, a beacon of hope that open source, a commitment to the user and a policy of "not being evil" could succeed and even lead the tech world. For a while, they did. Now it smells like they are losing ground on all those fronts.

I wonder how my old friends at Google feel about all this. Of course, since they are still trapped in the bubble, I don't really expect to find out any time soon. But I do wonder.