Contact frederick.j.harpootlian@jrs-s.net at our Honeypot department if you are desperate to get blacklisted.

Open Source Billing Software

Like it says on the tin, I’m an IT consultant. One of the things I need to do my job is a decent invoicing system – and, importantly, one that’s internet-friendly – I need to be able to access it from anywhere via the internet, and I need to be able to email invoices. Because of the very agile nature of a small IT consulting business, I absolutely need the ability to write one-off invoices “on the fly” as well, without depending on inventory or long look-up tables of specific parts or services.

That said, I do also offer monthly and annual service and/or hosting agreements to customers, which I have been (actually, my lovely bookkeeper has been) banging out manually each month/year/whatever until now. And some parts and services change infrequently enough that it would be nice to just pick those from a list – particularly if I can override the default pricing set if I need to. So it would be a giant plus for my invoicing application to gracefully handle recurring billing and pre-configured line items as well as one-offs.

There are quite a lot of subscription services which do this kind of thing as a hosted service, but being who I am, I much prefer to host it myself – and if at all possible, I prefer an open source solution as well. Unfortunately, while there are quite a lot of them to choose from, they’re all pretty flawed in one way or another. Listed below are the results of my search, with the one I eventually settled on listed last.

Argentum – unsuitable for me, because of how much labor you need to go through in order to create an invoice – you need to create a client, create a project for that client, then create a ticket for that project for that client… which you can then add to an “invoice” which has no details within it; it’s just a collection of “tickets” which have been marked complete. I’m sure this works for some business models, but it doesn’t work for mine.

Billwerx – unsuitable for me, because it has no support whatsoever for on-the-fly invoice items. Any item invoiced must be selected from a table of possible invoice items with preset prices; neither prices nor descriptions can be set or overridden invividually on particular invoices.

Agilebill – this is a formerly commercial and rather large product which has been open sourced. Support remaining is dubious at best, and it was too large and complicated to suit what I needed. If a large community (including developers!) ever coalesces around this as an open source project, it might look better… though still probably not so great for me, as it’s not really “agile” enough for what I do.

BambooInvoice – Usable; in fact I used it for two years. If what you want is the ability to write an invoice on-the-fly, with whatever price you feel like setting for each line item, then generate (and email) PDF invoices, and enter payments later, this will work. However, it’s got problems – it’s open source, but sort of grudgingly (the author has griped numerous times about people forking the code), it uses a “framework” that I’ve only seen one other place (CodeIgniter), the release schedule is glacial at best, and it’s missing a lot of features. The problems that finally made me get up and leave were lack of support for reports based on payments rather than invoice amounts (I actually shelled into my server to run raw SQL queries from the mysql client each year to give totals to my CPA!) and the fact that the only view you have of the invoices only shows the last [x] invoices (I forget the number – 60?) with no way to skip back further. So if an invoice disappears off the screen, you have no way of getting to it – it’s still in the database, but you can’t find it in the program. I manually changed that number to 9999, which worked well enough for me NOW with only a few hundred invoices in the db, but obviously that “solution” won’t last forever. Also, Bamboo is missing an option for recurring billing, which sucks hard if you do a lot of that. Bamboo is usable, and it looks fantastic, but there are just too many itches like this.

MyClientBase – See BambooInvoice. This program basically IS BambooInvoice, complete with the dependency on CodeIgniter, and complete with the long laundry list of essential missing features – in fact, the author is a former BambooInvoice user, who decided to roll his own competitor; the dependence on CodeIgniter makes me think it’s probably a fork to some degree or another (redacted – see comments). The missing features list is somewhat different than BambooInvoice, but it’s of a similar length. I skipped over this one pretty quickly because it was just more of the same – looks quite usable, and it’s more attractive than most of the others on this list, but it’s just missing too many basic features. It’s also missing an option for recurring billing. If you’re willing to consider BambooInvoice, you should probably also consider MyClientBase. Otherwise, keep looking.

P-Books – demo is radically broken, which made me not look for long. At all.

CitrusDB – unsuitable for me, because it has no real provision for off-the-cuff invoices. CitrusDB is a popular product, and if all you do is offer hosting or other monthly services, it will probably work well for you. But if you need the ability to sit down and bang out a single invoice for an arbitrary thing you just did or sold, it’s not going to work for you.

Black Sheep Invoices – line items are too ludicrously simple. If you ever need several lines to describe one item in an invoice (for example, running down the parts in a computer, or the exact services performed in a 10 hour block of service), this just isn’t going to work for you, because the input form does not support carriage returns and only shows 30 characters or so at a time. Show stopper for me; I didn’t look any further.

SimpleInvoices – This is what I am migrating to from BambooInvoice.
The pros: supports both off-the-cuff line items AND out-of-a-table line items, as well as allowing you to override the price on any given line item. Has reporting based on both invoice amount AND payment amount, as well as reports grouping on invoice age, individual clients, etc etc. Plain-jane PHP: no weird “framework” that it depends on that you 1. need to learn if you want to modify it and 2. have to worry about as a dependency, should that “framework” fall off the face of the earth. Supports recurring payments!

The cons: the author clearly doesn’t know much about databases; all tables are MyISAM with *no indexes at all*, resulting in very slow performance unless you fix that yourself. All the reports, invoices, etc look EXTREMELY basic to the point of “did somebody make this on a typewriter?” and the web interface is quirky enough that you will have to hunt for things fairly hard for your first hour or so using it. The author’s site is also pretty quirky and hard-to-navigate, making it more difficult than necessary to find support for bugs / report bugs / etc.

The mitigating factors: Since the code is so very simple, it’s MUCH easier to find and fix bugs yourself than would be the case for any of the above applications (it took me a ridiculously long time just to find where the variable defining the number of invoices shown on the front page was in BambooInvoice, for example). The performance issues can be solved VERY easily simply by adding a few indexes to the database tables (and, optionally, converting them to InnoDB). For that matter, the code is simple and straightforward-enough that future updates to PHP itself are much less likely to break things, making you that much less reliant on support from the author in the future.

how malware prevents programs from running

In today’s battle with malware, I learned a couple of interesting new places in the registry to check:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options

Place a key in here named after the file you want to prevent running, then place a STRING value under the key named “debugger”. Now, set the value of “debugger” to cmd, or some other relatively harmless executable that ignores its standard input – and presto, the application matching the keyname won’t run. BAD MALWARE. NO COOKIE.

Ironically, this is also quite useful for the GOOD guys keeping relatively clueless but persistent users from running things they really shouldn’t, like notorious P2P clients. For extra points, create a file C:\null.cmd or similar that simply exits, and use that as the “debugger” – they don’t even see anything happening at all, it just “doesn’t work”. This will probably frustrate them enough to desist, at least for a while… particularly given how used they probably are to the machine not working, if they’re that persistently malwaring it up in the first place.

HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Policies\Explorer\DisallowRun

Place a STRING value in here, and ditto above. (This is where GPO disallowing particular mutexes (I think it’s by mutex, not filename) to run takes effect.)

The More You Know…

setting locale to UTF-8 in Debian

If you have to deal with foreign languages and character sets (Cyrillic, Katakana, Hiragana, Kanji, etc) you need to have UTF support on your server. If you don’t already have it, here’s how you get it:

1. nano /etc/default/locale.gen and uncomment the line with en_US.UTF-8 (assuming your default language should be English)
2. locale-gen
3. edit /etc/profile and /etc/bash.bashrc and add the following: export LANG=en_US.UTF-8

When you next start a shell (exit, call bash manually, run sudo -s, whatever) you should then see UTF support available:

me@box:~$ locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

And you’re done.

RDP Client under Ubuntu

The performance of the built-in RDP clients in Ubuntu as of 10.04-LTS Lucid Lynx (and previous) is plagued with difficulties – Windows machines, among other things, frequently have events which refresh the local desktop every second or so which Windows’ native RDP client doesn’t trigger on, but for some reason the TS Client and Remote Desktop Viewer in Ubuntu does, making using same to control a Windows box a real hassle.

Fortunately, there’s a solution – as of 11.04 (Natty Narwhal?), Ubuntu will be migrating to Remmina as its RDP and VNC client. Better yet, Remmina is already available in the repos for 10.04 Lucid Lynx!

me@box:~$ sudo apt-get update && sudo apt-get install remmina remmina-gnome

After installation, you’ll find Remmina in Applications->Internet. Be sure to turn on the performance features “enable bitmap caching” and “enable compression”, they make a big difference. Also, you’ll find that Remmina supports transferring sound (leave it off if you don’t need it!), sharing local printers, and even sharing local drives! Pretty sweet. I’ve been very impressed with it so far. I haven’t experimented with the sound / folder / printer share options, but the performance difference is night and day.

setting a mail smarthost in exim on a Cpanel box

add a Router block to the end of /etc/exim.conf.local:

@ROUTERSTART@
            smart_route:
                driver = manualroute
                domains = !+local_domains
                transport = remote_smtp
                route_list = * host.name.of.smart.host.server

then run the scripts to update configuration and restart exim:

root@box:~$ /scripts/buildeximconf  
root@box:~$ /scripts/restartsrv_exim  

You’re done.

Using APC cache on Magento 1.4

First, of course make sure APC is installed. On Debian or a Debian-derived system (like Ubuntu), you can do this with sudo apt-get install php-apc. Once you’ve got APC installed and working, add the following snippet to app/etc/local.xml under your Magento site, in between the <global> and </global> tags:

<global>
        ...
        <cache>
            <backend>apc</backend>
            <prefix>SOMETHINGUNIQUE_</prefix>
        </cache>
        ...
</global>

“SOMETHINGUNIQUE” should be just what it says – something unique to the site you’re caching. This is a prefix that lets the webserver figure out what bits of cache go with what sites; so if you’ve got two different sites running, both with the same APC prefix set, you will have serious problems later. I recommend using your domain name, without the periods – so if your site is mystore.com, your prefix would be MYSTORE_ in the snippet above.

Make sure cache is enabled under System/Cache Management, and refresh it. You’re done.

When Outlook stops getting new IMAP mail

The problem is almost certainly that its local cache is corrupt… which happens disturbingly frequently. The easiest way to fix it is to simply close Outlook, delete the local cache, then start Outlook again – the good news being that it works and your new mail starts showing up; the bad news being, of course, that it starts synchronizing /everything/ again.

Stick this command in a batch file, and you’ll have something that users can simply double-click to fix the issue. Just remember to tell them to CLOSE OUTLOOK FIRST! =)

del "%UserProfile%\Local Settings\Application Data\Microsoft\Outlook\*IMAP*-0*.pst"

Slow domain login on Windows 7

Had a client whose login to the domain took upwards of 40 seconds on his Windows 7 machine.  Oddly, completely deleting the profile from his workstation didn’t fix the issue – even after setting up the profile anew after first domain login, it still took upwards of 40 seconds.  Just as oddly, other profiles on the domain didn’t have the same issue on his workstation – they logged in in less than 10 seconds.

Never did figure out what caused it, but I did find a cure –

  • Run gpedit.msc.
  • Go to computer configuration.
  • Go to Administrative templates.
  • Go to System.
  • Go to User profiles.
  • Enable “Set maximum wait time for the network if a user has a roaming user profile or remote home directory” and set to 0 seconds

Problem solved; the user could now logon in less than ten seconds.

Troubleshooting Exchange 2007/2010: a quick guide

This is mostly intended for myself… but if it helps you, you’re welcome.

Exchange 2007/2010 with Outlook 2007 clients is a hellkitten to get right, and I do not say this affectionately.  You need to get RPC over HTTP working, or the Out-Of-Office Assistant will not work, and neither will the offline Address Book (or, very likely, the GAL).

In order to get RPC over HTTP working, you must have several virtual directories running right in IIS, you must have client certificates ignored on those virtual directories, you must have both Basic AND Integrated authentication on those directories, and you must have a proper SSL certificate on the site.  On a standard Exchange setup, this will be the (Default Web Site).  On an SBS setup, this will be the (SBS Web Applications) site.

Definition of “proper” SSL certificate: you must have both the internal domain name AND any external domain names ON THE SAME CERT.  If your internal domain is “domain.local” or something like that, this probably means you’re going to have to use self-signed certs (and deal with security warnings on clients outside the local domain).  If you have an FQDN, you ought to be able to get everything on one UCC certificate… you will need, at a minimum, internaldomain.com, mail.internaldomain.com, externaldomain.com, and mail.externaldomain.com.  If possible, you also want autodiscover.externaldomain.com and autodiscover.internal.com, but they aren’t strictly necessary.

Here are some incredibly brief tips toward finagling the virtual directories and the certificates.  Except where specified otherwise, these are all commandlets run from the Exchange Management Shell – there is very little you can or should be doing from the Exchange Management Console for working with these issues.

Testing from Outlook:
control-right-click the Outlook icon in the system tray, and you will have options for “Connection Status…” and “Test E-Mail Autoconfiguration…” available.  Your ultimate goal here is to get the “Test E-Mail Autoconfiguration…” option working.  If you DON’T get this working, you’re not going to have a fully functional Exchange setup, regardless of what anything in the “Connection Status…” tells you.  To get this working, you will need to have either mail.yourdommainname.com or autodiscover.yourdomainname.com both in DNS and on the SSL certificate bound to the site in IIS which hosts the virtual directories for Available Services, the OAB, UM, and OWA.  If you specified both internalurls and externalurls in your virtual directory setup, both
of them need to work properly from inside the domain or local clients will not work; you can’t really control whether they decide to use the internalurl or the externalurl, and in my experience, they will frequently choose to use the externalurl, even if they’re plugged into the same switch and sitting physically right next to the Exchange server.

If your “Testing Autoconfiguration…” comes up with failures, you’ve got problems with your certificates, your virtual directories, your settings for URLs to your virtual directories, or all three… head to the tips below to examine and troubleshoot.

A word of warning about the Exchange Management Shell:
The EMS commandlets sometimes use Uri and sometimes use Url for their argument names… so be careful; even though they both mean the same thing, you have to get the right arbitrary spelling for the right arbitrary commandlets.  (Thanks for that, Microsoft…)

Another word of warning about the EMS:
you can get away with using all lower case for the commandlets themselves, but argument names for the commandlets require CamelCase as shown in the examples below.

A third and final word of warning about the EMS:
The examples I’ve shown below are extremely terse, and assume that, once pointed to examples of working usage, you can figure out the gist of what they mean, what they do, and likely useful ways to do related things just from seeing the syntax shown.  If you don’t feel comfortably that this is the case, then for the love of working systems stop right now and hire a (more experienced) professional!

And now, on to the actual EMS usages:


test basic RPC proxy connectivity:
rpcping -t ncacn_http -s servername -o RpcProxy=proxyservername -P "user,domain,pass" -I "user,domain,pass" -H 2 -u 10 -a connect -F 3 -v 3 -E -R none
test RPC proxy through to Information Store default port on back-end:
rpcping -t ncacn_http -s servername -o RpcProxy=proxyservername -P "user,domain,pass" -I "user,domain,pass" -H 1 -F 3 -a connect -u 10 -v 3 -e 6001
test RPC proxy through to IS backend default port using Mutual auth:
RpcPing –t ncacn_http –s ExchangeMBXServer  -o RpcProxy=RpcProxyServer -P "user,domain,password" -I "user,domain,password" -H 1 –F 3 –a connect –u 10 –v 3 –e 6001 –B msstd:server_certificate_subject
test all web services:
Test-OutlookWebServices
setting the Exchange cert: (note that not all services may be installed)
enable-ExchangeCertificate -thumbprint "thumbprintfromcert" -services "IIS,IMAP,POP,SMTP,UM"
if private key is missing: get serial number from cert and…
certutil -repairstore my "serialnumberfromcert"
Autodiscover:
Get-ClientAccessServer | Select Name, *Internal* | fl
Set-ClientAccessServer -Identity servername -AutoDiscoverServiceInternalUri: https://mydomain.com/Autodiscover/Autodiscover.xml
OAB:

in EMS, Server Configuration -> Client Access -> select server in top window -> click Offline Address Book Distribution tab in bottom window -> click OAB properties in right window, under Actions; set internal and external URLs from there

Web Services:
Get-WebServicesVirtualDirectory | Select name, *url* | fl
Set-WebServicesVirtualDirectory –Identity “<EWS Name>” –InternalUrl: https://url.domain.local/EWS/Exchange.asmx
Unified Messaging:
Get-UMVirtualDirectory | Select Name, *url* | fl
Set-UMVirtualDirectory –Identity: “<UM Virtual Directory>” –InternalURL: <URL/UnifiedMessaging/Service.asmx>

Perspectives on Open Source: The Three P’s

Yesterday and today I attended POSSCON, the Palmetto Open Source Software Conference.  They’ve got a pretty great speaker lineup this year – Chris Wanstrath, one of the co-founders of GitHub, was particularly inspiring.  It’s also pretty interesting to me, as a consultant, to see just who shows up for this kind of thing – an open source conference in a town not particularly known for being a giant mecca for open source.  (In fact, the POSSCON speakers and promoters went out of their way to praise Columbia for opening itself up to the conference – but that’s not the same thing as being someplace like SF or NY, a lodestone that accumulates OSS devs and culture whether it likes it or not.)

So who does show up for an OSS conference in a mid-sized Southern town?  A pretty randomized mix of “suits”, hobbyists, and developers.

The thing that these three basic types of attendant have in common, of course, is that one way or another, they’re interested in open source software – and for the most part, they’re “for it”.  But the reasons vary pretty wildly, and they vary in ways that don’t necessarily match up evenly with the three “obvious” divisions that you’re most likely to see at first glance.

So, if you’re “for” open source software, and you’re interested in actively promoting it, it helps to understand not only why you like it yourself, but why others might – and how their perspectives and yours can dovetail, even if they aren’t the same.  I like to think of these perspectives as “The Three P’s”:

  • Philosophy
  • Pragmatism
  • Paranoia

First, let’s talk about philosophy.  There are a lot of folks – yours truly included – who can get pretty excited about the basic philosophy of open source.  The idea that we’re all contributing to a permanent increase in the sum of human knowledge and capability is pretty heady, and ultimately, that’s what the OSS philosophy is all about.  Proprietary software and knowledge can very easily go away and be lost forever (until somebody reinvents it all over again), but OSS is a lot more likely to survive changes in underlying technologies, organizations, and motivations to remain available for whoever might need it.  Additionally, the reduction of the barrier-to-entry to effectively nil means that a lot of people get empowered further than their monetary income or social circumstances would normally allow.  When somebody talks disparagingly (or affectionately) about “open source hippies”, this is what they’re talking about!

But maybe you don’t care about that.  Maybe you’re a hard-headed realist – and that’s where pragmatism comes into play.  There may be things that you simply can’t do with closed source software, but you’ve found open source software projects that let you do them, or let you do them more easily and cheaply.  If what you want to do is create a collaborative documentation project, then you probably can’t find anything better than Mediawiki on an open source software stack to do it.  Or perhaps you’re a developer, and you want easy access to the sheer volume of peer review, in-the-field testing, and free QA and contributions that open sourcing your project can provide.  Or maybe you’re a small business – or a small cog in a very large business – and it’s easier to get the motivation to put a project together than the approval for budget to pay for software licensing for it.  Ultimately, though, this P is about a hardheaded, realistic intention to get a job done, and OSS just happens to be the tool that makes it possible for you… or not.  People who fall into this category are the most likely to have “mixed source” infrastructure, where OSS tools sit side-by-side with closed-source, proprietary tools; whatever gives the best ROI is what gets used, period.

Finally, we have paranoia.  This one’s a little misleading; the word has negative connotations, but as the old saying goes “you’re not paranoid if they really are out to get you.”  Someone primarily motivated by the third P is worried not just about the current situation, but about what can happen tomorrow.  They might be worried about what’s “hidden in the code” in proprietary applications – what if they left a backdoor in the code?  Can they get at my private data?  Might they disable functionality and potentially shut down my business because some automated check “thinks I’m a pirate?” – or they might be worried about the changing motivations and viability of other organizations – anybody who started creating documents in the 80s has probably been through at least one horrified realization “I still have the data from that old app, but I don’t have anything that will open it!”  Corporate mergers can also create some pretty nasty situations for the end-user; big orgs frequently swallow small orgs with the express purpose of getting access to the smaller org’s customer base… and putting them in a “forced switch” situation where the app the end-user originally installed is no longer available or supported, so now the end-user has to migrate to something that may cost more money, may not have the desired feature set, or may for whatever reason “just not fit”.

Conversely, all three P’s can be viewed the other way: someone might think “it’s my work, I don’t want to give it away!” or “things are only as good as what you pay for them” or “how can I control it if I don’t have to budget for it?” and be philosophically in opposition.  They might believe that the documentation isn’t sufficient, or that the support structure isn’t rigid enough, etc. and be pragmatically opposed.  Or, they might cling fiercely to the idea “it’s not safe if there isn’t somebody I can sue” or “I don’t want the whole world to know intimate details of how my systems work!” and be opposed on grounds of paranoia.

It’s important to think about these “three P’s”, and how they apply to you, to others around you, and to each other.  If you’re advocating OSS and want to see it more widely used in your community, understand your own motivations for it, and understand the motivations of the folks who you’d like to spread it to.  If you’re curious about OSS and trying to figure out how or why you should use it (or care), understand your own motivations, and go from there.  And if you, or someone you’re discussing OSS with, are primarily motivated by only one or another of the three P’s, be sure to address how the other two P’s inform the one that’s the primary concern, rather than wasting your time flogging philosophy to a pragmatist, or pragmatism to a paranoiac.