Relay v1.5.1 is released!

Relay v.1.5.1 is released and is available now at http://code.google.com/p/relay!

New features in v1.5.1:

  • database is now used only for authentication and virtual directory definitions
  • thumbnail images moved out to the filesystem (greatly reducing db bloat)
  • many, many scalability issues are fixed – Relay now gracefully handles directory trees containing thousands of files
  • lots of minor UI improvements
  • Relay now gracefully handles files and directories in its filestore which were NOT uploaded with Relay itself

AJAX large file transfer – Relay

Relay screenshot
Screenshot - Relay (in Chromium browser)

I have been looking for a user-friendly, AJAX-y web application to allow easy sharing of arbitrarily large files for YEARS. Finally, FINALLY, I found one – Relay.

Relay was a “dead project” that hadn’t been updated in several years, but my testing finds it perfectly functional.  It was very difficult to actually find the code for it, because the web’s full of links to the original page, which is long since defunct.  However, if you are persistent enough in your Googling, you eventually find that a Google Project was created for it several years ago – which was uploaded by someone with a now-defunct Google Code account.  Luckily, I managed to get ownership of the project at Google Code transferred to me; so Relay is now available for download – and actively seeking interested developers to address a couple of issues I’ve addressed on the project page.

You can find Relay (and help develop it further!) at http://code.google.com/p/relay.

More virtualization: multiple Win7 guests on a single Debian host

As a proof-of-concept for USC computer science labs, I set up eight Windows 7 VMs on the same physical host in the Windows Server demonstration below, and recorded firing them up simultaneously and doing some light web browsing, etc. on several of them. Performance is pretty solid; you could probably cram double this many guests on that host and still have as good or better performance than the typical physical lab workstation.


update: replaced video with somewhat more watchable version, with all eight guests tiled on one screen.

Aside from good performance and a single box to maintain, this setup offers some fairly compelling advantages over the traditional computer lab: the host also has a 2TB conventional drive in it, which is where a “gold” image of the Win7 guests is maintained. It only takes about 10 minutes total to reset all of the guests to the “gold” standard; and it would be just as easy to keep multiple gold images on the conventional drive for different classes – Linux images for one class, Windows images with Office for a basic class, Windows images with Visual Studio for another, Solaris for yet another… you get the idea.

Also, the time to “reset” the guests could be substantially faster than that, even, with a little tweaking – using .qcow files instead of whole LVM volumes would allow you to use rsync with the –inplace argument and only have to write over the (relatively few) changed blocks, for example; or in a more advanced layout a separate FreeBSD machine with a large RAIDZ array and iSCSI exports could be used to store the images. There’s still plenty of room for improvement and innovation, but even the simple proof-of-concept (which I put together in roughly half an hour) looks pretty compelling to me.

Virtualizing Windows Server with KVM

I’ve been surprised and pleased at just how well Windows Server 2008 runs virtualized under Debian Squeeze. I first started running virtual Windows Servers purely for the disaster recovery and portability aspects, expecting to pay with a drop in performance… but what I found was that in a lot of cases, Windows 2008’s performance is actually somewhat better when running virtually. In particular, the ever-annoying reboot cycle gets cut to a tiny, tiny fraction of what it would be if running on “the bare metal.”

It’s also pretty nice never, ever having to play “hunt-the-driver” – the virtual “hardware” is all natively supported by Windows, so a virtual install “just works” the moment it’s done, no fuss no muss. But what about that performance?

Smokin’! Which exposes yet another reason to think about virtualization: being able to take advantage of Linux’s highly superior kernel RAID capabilities. The box shown above is running four Crucial C300 128GB solid state drives connected to SATA-3 6Gbps ports on an ASUS board; the Debian Squeeze host has them set up in a kernel RAID10. The resulting 250GB or so of storage is on a performance level that just has to be seen to be believed.

Note that while this IS a really “hot” machine, it’s still just one machine, running on commodity hardware – there’s no $50,000 SAN lurking in the background somewhere; that performance is ALL coming from a single machine with a price tag of WELL under $10K.

Ready to upgrade yet? =)

Graphic Equalizer (Treble/Bass) under Linux/Gnome

One of the things that I’ve missed on the Linux desktop is simple audio tone control in the sound volume applet. It particularly annoys me that Gnome allows you to set cruddy little reverb profiles (wow, all my audio sounds like a dog barking now… uh… thanks…), but if your speakers need a little help in the bass or treble department, you’re out of luck. Well, now you’re not!

PulseAudio Multiband Equalizer
the PulseAudio Multiband Equalizer

The PulseAudio System-Wide Equalizer is available from its own Ubuntu PPA, and it is a thing of absolute beauty. I particularly like the fact that the bottom slider is centered at 50Hz – where you want it to add a crisp punch to capable speakers – rather than at the more common 80Hz or even 100Hz, which is more immediately audible but also muddies up the sound rapidly.

Thank you psyke83 for this excellent tool!

Cross-platform Windows Event Log viewer

Another consultant emailed me a .evt file recently for review. Which is great, except I frequently go days now without sitting in front of a Windows workstation – or at least, not one that isn’t broken and in need of fixing. So, I needed to find a Windows Event Log viewer.

There isn’t currently one in the Debian or Ubuntu repositories, but I did find a free-as-in-beer tool at TZWorks, LLC which did the trick nicely. It’s currently available for download in Windows, Linux (i386), and Mac versions – I haven’t tested the Mac version, but the Windows and Linux versions both run fine and do the job well, both for the older .evt and the newer .evtx (Vista and up) formats.

Note: the Linux binary provided is currently 32-bit only, so if you’re running a 64-bit system you’ll either need to install ia32-libs (apt-get install ia32-libs on Debian or Ubuntu), or just run the Windows version under WINE.

EDIT, September 2014: you can’t tell from looking at the download page, but this app now costs $228 for a single copy of it. So, uh, keep moving if you want a reasonable tool to look at Event Viewer logs with, sorry. >=\

B&N Nook Color

So, I finally got an e-reader today. After getting my wife a Nook Color for her birthday, I found it intriguing enough to take the plunge and get my own. I still wasn’t sure I would really be into it, but the only way to find out for sure was to go ahead and take the plunge.

So far, so … well, OK. Some things I really like, others annoy me a lot. The color touchscreen is WORLDS better, for me, than the “e-ink” more typically found in e-readers. The “PC application” is Windows-only… but it does run fine, so far, under WINE in Linux, so there’s that. Battery life seems pretty sweet so far.

One thing that bothers me – the “lending” feature, which was something I heartily approved of, so far seems to require that you link the Nook to your Facebook account… and give it permission to post on your wall. NOT COOL, B&N. I am really, really not okay with applications which can pretend to be you by posting things as though they were you, ever, from pretty much anybody. And to be honest – I am looking at you, Mark Zuckerberg – the fact that this is even an option with Facebook apps drives me insane. There should never be a legitimate case for an application making a post as a human being without that human’s express consent, expressed beforehand, for that particular post. Anyway. Back to the actual device:

The feel of the device in my hands – which was a really big concern for me – is pretty nice so far. Part of how nice it is to hold is the leather “book” cover I got for it, which I am frankly kind of in love with – it’s glossy, nice-smelling black leather, with reverse-embossed classical authors’ names in big all caps serif text all over. I wasn’t sure when I went into B&N today whether I would get the Nook or not – I was really leaning more towards a Samsung Galaxy android tablet. I’m still not sure if I would have actually taken the plunge, without that cover sitting there all seductive-like. Having seen it though… had to have it.

My biggest gripe so far is the interface of the shop. The Nook store is frankly AWFUL – it’s almost impossible to navigate effectively. If you just want to buy whatever is selling well, you’re in luck, and you’ll be very happy. If you have more specific tastes… prepare for some pain. You can search for author name or book title, which is great if you know EXACTLY what you want – and by “great” I mean “OK”, because all you have is a simple, single-level search with no sorting or grouping. Better hope your favorite author has an unusual name, because you can’t limit searches by genre; for example, searching for “David Drake” got me both the military sci-fi author and some young gay dude who wrote a tell-all book. The lack of sorting or grouping is even worse; should you actually find the author you’re looking for, you can expect to find a complete mish-mash of crap: in a series of novels you’ll likely see #5 first, followed by three unrelated books, followed by #7, followed by more unrelateds, followed by #2… you get the idea.

You are also ridiculously likely to see the SAME book multiple times, with a different cover image. It’s even worse in the “free books” section – some dude wrote his own Star Wars book and it’s listed, I kid you not, AT LEAST ten different times. Which wouldn’t be so bad if it was SORTED or grouped in any way, but… did I mention that you can’t sort, or group, and your searches are single-level simple searches only?

Still, so far I’m enjoying the experience of actually *reading* on the device, and with any luck eventually B&N will sort out their godawful navigation issues on the store.

Diagramming

http://www.yworks.com/en/products_yed_about.html

That’s pretty much all you need to know. yED lets you stick polygons graphically into blank space, label them, and draw connections between them haphazardly, then click a single menu to automatically reorganize the jumble you made in any order you like – hierarchical, tree, organic, circle, UML, etc. It also supports import of existing data. Cross-platform, open-source, awesome.

Open Source Billing Software

Like it says on the tin, I’m an IT consultant. One of the things I need to do my job is a decent invoicing system – and, importantly, one that’s internet-friendly – I need to be able to access it from anywhere via the internet, and I need to be able to email invoices. Because of the very agile nature of a small IT consulting business, I absolutely need the ability to write one-off invoices “on the fly” as well, without depending on inventory or long look-up tables of specific parts or services.

That said, I do also offer monthly and annual service and/or hosting agreements to customers, which I have been (actually, my lovely bookkeeper has been) banging out manually each month/year/whatever until now. And some parts and services change infrequently enough that it would be nice to just pick those from a list – particularly if I can override the default pricing set if I need to. So it would be a giant plus for my invoicing application to gracefully handle recurring billing and pre-configured line items as well as one-offs.

There are quite a lot of subscription services which do this kind of thing as a hosted service, but being who I am, I much prefer to host it myself – and if at all possible, I prefer an open source solution as well. Unfortunately, while there are quite a lot of them to choose from, they’re all pretty flawed in one way or another. Listed below are the results of my search, with the one I eventually settled on listed last.

Argentum – unsuitable for me, because of how much labor you need to go through in order to create an invoice – you need to create a client, create a project for that client, then create a ticket for that project for that client… which you can then add to an “invoice” which has no details within it; it’s just a collection of “tickets” which have been marked complete. I’m sure this works for some business models, but it doesn’t work for mine.

Billwerx – unsuitable for me, because it has no support whatsoever for on-the-fly invoice items. Any item invoiced must be selected from a table of possible invoice items with preset prices; neither prices nor descriptions can be set or overridden invividually on particular invoices.

Agilebill – this is a formerly commercial and rather large product which has been open sourced. Support remaining is dubious at best, and it was too large and complicated to suit what I needed. If a large community (including developers!) ever coalesces around this as an open source project, it might look better… though still probably not so great for me, as it’s not really “agile” enough for what I do.

BambooInvoice – Usable; in fact I used it for two years. If what you want is the ability to write an invoice on-the-fly, with whatever price you feel like setting for each line item, then generate (and email) PDF invoices, and enter payments later, this will work. However, it’s got problems – it’s open source, but sort of grudgingly (the author has griped numerous times about people forking the code), it uses a “framework” that I’ve only seen one other place (CodeIgniter), the release schedule is glacial at best, and it’s missing a lot of features. The problems that finally made me get up and leave were lack of support for reports based on payments rather than invoice amounts (I actually shelled into my server to run raw SQL queries from the mysql client each year to give totals to my CPA!) and the fact that the only view you have of the invoices only shows the last [x] invoices (I forget the number – 60?) with no way to skip back further. So if an invoice disappears off the screen, you have no way of getting to it – it’s still in the database, but you can’t find it in the program. I manually changed that number to 9999, which worked well enough for me NOW with only a few hundred invoices in the db, but obviously that “solution” won’t last forever. Also, Bamboo is missing an option for recurring billing, which sucks hard if you do a lot of that. Bamboo is usable, and it looks fantastic, but there are just too many itches like this.

MyClientBase – See BambooInvoice. This program basically IS BambooInvoice, complete with the dependency on CodeIgniter, and complete with the long laundry list of essential missing features – in fact, the author is a former BambooInvoice user, who decided to roll his own competitor; the dependence on CodeIgniter makes me think it’s probably a fork to some degree or another (redacted – see comments). The missing features list is somewhat different than BambooInvoice, but it’s of a similar length. I skipped over this one pretty quickly because it was just more of the same – looks quite usable, and it’s more attractive than most of the others on this list, but it’s just missing too many basic features. It’s also missing an option for recurring billing. If you’re willing to consider BambooInvoice, you should probably also consider MyClientBase. Otherwise, keep looking.

P-Books – demo is radically broken, which made me not look for long. At all.

CitrusDB – unsuitable for me, because it has no real provision for off-the-cuff invoices. CitrusDB is a popular product, and if all you do is offer hosting or other monthly services, it will probably work well for you. But if you need the ability to sit down and bang out a single invoice for an arbitrary thing you just did or sold, it’s not going to work for you.

Black Sheep Invoices – line items are too ludicrously simple. If you ever need several lines to describe one item in an invoice (for example, running down the parts in a computer, or the exact services performed in a 10 hour block of service), this just isn’t going to work for you, because the input form does not support carriage returns and only shows 30 characters or so at a time. Show stopper for me; I didn’t look any further.

SimpleInvoices – This is what I am migrating to from BambooInvoice.
The pros: supports both off-the-cuff line items AND out-of-a-table line items, as well as allowing you to override the price on any given line item. Has reporting based on both invoice amount AND payment amount, as well as reports grouping on invoice age, individual clients, etc etc. Plain-jane PHP: no weird “framework” that it depends on that you 1. need to learn if you want to modify it and 2. have to worry about as a dependency, should that “framework” fall off the face of the earth. Supports recurring payments!

The cons: the author clearly doesn’t know much about databases; all tables are MyISAM with *no indexes at all*, resulting in very slow performance unless you fix that yourself. All the reports, invoices, etc look EXTREMELY basic to the point of “did somebody make this on a typewriter?” and the web interface is quirky enough that you will have to hunt for things fairly hard for your first hour or so using it. The author’s site is also pretty quirky and hard-to-navigate, making it more difficult than necessary to find support for bugs / report bugs / etc.

The mitigating factors: Since the code is so very simple, it’s MUCH easier to find and fix bugs yourself than would be the case for any of the above applications (it took me a ridiculously long time just to find where the variable defining the number of invoices shown on the front page was in BambooInvoice, for example). The performance issues can be solved VERY easily simply by adding a few indexes to the database tables (and, optionally, converting them to InnoDB). For that matter, the code is simple and straightforward-enough that future updates to PHP itself are much less likely to break things, making you that much less reliant on support from the author in the future.

setting locale to UTF-8 in Debian

If you have to deal with foreign languages and character sets (Cyrillic, Katakana, Hiragana, Kanji, etc) you need to have UTF support on your server. If you don’t already have it, here’s how you get it:

1. nano /etc/default/locale.gen and uncomment the line with en_US.UTF-8 (assuming your default language should be English)
2. locale-gen
3. edit /etc/profile and /etc/bash.bashrc and add the following: export LANG=en_US.UTF-8

When you next start a shell (exit, call bash manually, run sudo -s, whatever) you should then see UTF support available:

me@box:~$ locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

And you’re done.