Virtualizing Windows Server with KVM

I’ve been surprised and pleased at just how well Windows Server 2008 runs virtualized under Debian Squeeze. I first started running virtual Windows Servers purely for the disaster recovery and portability aspects, expecting to pay with a drop in performance… but what I found was that in a lot of cases, Windows 2008’s performance is actually somewhat better when running virtually. In particular, the ever-annoying reboot cycle gets cut to a tiny, tiny fraction of what it would be if running on “the bare metal.”

It’s also pretty nice never, ever having to play “hunt-the-driver” – the virtual “hardware” is all natively supported by Windows, so a virtual install “just works” the moment it’s done, no fuss no muss. But what about that performance?

Smokin’! Which exposes yet another reason to think about virtualization: being able to take advantage of Linux’s highly superior kernel RAID capabilities. The box shown above is running four Crucial C300 128GB solid state drives connected to SATA-3 6Gbps ports on an ASUS board; the Debian Squeeze host has them set up in a kernel RAID10. The resulting 250GB or so of storage is on a performance level that just has to be seen to be believed.

Note that while this IS a really “hot” machine, it’s still just one machine, running on commodity hardware – there’s no $50,000 SAN lurking in the background somewhere; that performance is ALL coming from a single machine with a price tag of WELL under $10K.

Ready to upgrade yet? =)

Cross-platform Windows Event Log viewer

Another consultant emailed me a .evt file recently for review. Which is great, except I frequently go days now without sitting in front of a Windows workstation – or at least, not one that isn’t broken and in need of fixing. So, I needed to find a Windows Event Log viewer.

There isn’t currently one in the Debian or Ubuntu repositories, but I did find a free-as-in-beer tool at TZWorks, LLC which did the trick nicely. It’s currently available for download in Windows, Linux (i386), and Mac versions – I haven’t tested the Mac version, but the Windows and Linux versions both run fine and do the job well, both for the older .evt and the newer .evtx (Vista and up) formats.

Note: the Linux binary provided is currently 32-bit only, so if you’re running a 64-bit system you’ll either need to install ia32-libs (apt-get install ia32-libs on Debian or Ubuntu), or just run the Windows version under WINE.

EDIT, September 2014: you can’t tell from looking at the download page, but this app now costs $228 for a single copy of it. So, uh, keep moving if you want a reasonable tool to look at Event Viewer logs with, sorry. >=\

Troubleshooting Exchange 2007/2010: a quick guide

This is mostly intended for myself… but if it helps you, you’re welcome.

Exchange 2007/2010 with Outlook 2007 clients is a hellkitten to get right, and I do not say this affectionately.  You need to get RPC over HTTP working, or the Out-Of-Office Assistant will not work, and neither will the offline Address Book (or, very likely, the GAL).

In order to get RPC over HTTP working, you must have several virtual directories running right in IIS, you must have client certificates ignored on those virtual directories, you must have both Basic AND Integrated authentication on those directories, and you must have a proper SSL certificate on the site.  On a standard Exchange setup, this will be the (Default Web Site).  On an SBS setup, this will be the (SBS Web Applications) site.

Definition of “proper” SSL certificate: you must have both the internal domain name AND any external domain names ON THE SAME CERT.  If your internal domain is “domain.local” or something like that, this probably means you’re going to have to use self-signed certs (and deal with security warnings on clients outside the local domain).  If you have an FQDN, you ought to be able to get everything on one UCC certificate… you will need, at a minimum,,,, and  If possible, you also want and, but they aren’t strictly necessary.

Here are some incredibly brief tips toward finagling the virtual directories and the certificates.  Except where specified otherwise, these are all commandlets run from the Exchange Management Shell – there is very little you can or should be doing from the Exchange Management Console for working with these issues.

Testing from Outlook:
control-right-click the Outlook icon in the system tray, and you will have options for “Connection Status…” and “Test E-Mail Autoconfiguration…” available.  Your ultimate goal here is to get the “Test E-Mail Autoconfiguration…” option working.  If you DON’T get this working, you’re not going to have a fully functional Exchange setup, regardless of what anything in the “Connection Status…” tells you.  To get this working, you will need to have either or both in DNS and on the SSL certificate bound to the site in IIS which hosts the virtual directories for Available Services, the OAB, UM, and OWA.  If you specified both internalurls and externalurls in your virtual directory setup, both
of them need to work properly from inside the domain or local clients will not work; you can’t really control whether they decide to use the internalurl or the externalurl, and in my experience, they will frequently choose to use the externalurl, even if they’re plugged into the same switch and sitting physically right next to the Exchange server.

If your “Testing Autoconfiguration…” comes up with failures, you’ve got problems with your certificates, your virtual directories, your settings for URLs to your virtual directories, or all three… head to the tips below to examine and troubleshoot.

A word of warning about the Exchange Management Shell:
The EMS commandlets sometimes use Uri and sometimes use Url for their argument names… so be careful; even though they both mean the same thing, you have to get the right arbitrary spelling for the right arbitrary commandlets.  (Thanks for that, Microsoft…)

Another word of warning about the EMS:
you can get away with using all lower case for the commandlets themselves, but argument names for the commandlets require CamelCase as shown in the examples below.

A third and final word of warning about the EMS:
The examples I’ve shown below are extremely terse, and assume that, once pointed to examples of working usage, you can figure out the gist of what they mean, what they do, and likely useful ways to do related things just from seeing the syntax shown.  If you don’t feel comfortably that this is the case, then for the love of working systems stop right now and hire a (more experienced) professional!

And now, on to the actual EMS usages:

test basic RPC proxy connectivity:
rpcping -t ncacn_http -s servername -o RpcProxy=proxyservername -P "user,domain,pass" -I "user,domain,pass" -H 2 -u 10 -a connect -F 3 -v 3 -E -R none
test RPC proxy through to Information Store default port on back-end:
rpcping -t ncacn_http -s servername -o RpcProxy=proxyservername -P "user,domain,pass" -I "user,domain,pass" -H 1 -F 3 -a connect -u 10 -v 3 -e 6001
test RPC proxy through to IS backend default port using Mutual auth:
RpcPing –t ncacn_http –s ExchangeMBXServer  -o RpcProxy=RpcProxyServer -P "user,domain,password" -I "user,domain,password" -H 1 –F 3 –a connect –u 10 –v 3 –e 6001 –B msstd:server_certificate_subject
test all web services:
setting the Exchange cert: (note that not all services may be installed)
enable-ExchangeCertificate -thumbprint "thumbprintfromcert" -services "IIS,IMAP,POP,SMTP,UM"
if private key is missing: get serial number from cert and…
certutil -repairstore my "serialnumberfromcert"
Get-ClientAccessServer | Select Name, *Internal* | fl
Set-ClientAccessServer -Identity servername -AutoDiscoverServiceInternalUri:

in EMS, Server Configuration -> Client Access -> select server in top window -> click Offline Address Book Distribution tab in bottom window -> click OAB properties in right window, under Actions; set internal and external URLs from there

Web Services:
Get-WebServicesVirtualDirectory | Select name, *url* | fl
Set-WebServicesVirtualDirectory –Identity “<EWS Name>” –InternalUrl: https://url.domain.local/EWS/Exchange.asmx
Unified Messaging:
Get-UMVirtualDirectory | Select Name, *url* | fl
Set-UMVirtualDirectory –Identity: “<UM Virtual Directory>” –InternalURL: <URL/UnifiedMessaging/Service.asmx>

Re-targeting existing .NET apps to a specific runtime version

File under “wow, I had no idea you could even do that”… also, file under “you really shouldn’t have to do that,” but that’s another story.

CPA clients are always an interesting challenge. By the standards of the IT world, their applications are really pretty simple and they don’t have a lot of data to deal with… but unfortunately, the tax codes they have to model were designed for human clerks physically stamping and filing papers in actual filing cabinets, not for modern data storage. Worse, those same tax codes vary from year to year, in arbitrary ways that seem “simple” from a human perspective but are difficult to fit into a logical framework. The end result is, your poor accountant has to deal with software that’s actually re-written once a year, every year, rather than a sane, logical framework that just gets a few variables tweaked from time to time.

Needless to say, quality control on this kind of software is frequently not all that it could be. In one particular case, a client was using an application called TaxWorks. The client has a nice, modern firm with a Windows Terminal Server for all of his accountants to do their work in. Doing it this way lets him (and me!) maintain the bewildering array of niche applications a CPA needs on a single machine, rather than having to keep them installed and updated across an entire network of workstations. For the most part, this is a great thing… but unfortunately, the folks who write TaxWorks apparently never did too much testing on any Windows Server platforms; the application works fine on Windows XP, and at first it worked fine on Server 2003 R2… until one day, my client called me and said TaxWorks just quit working. Which it did; the app would start but then would hang in several places. Many many hours of troubleshooting and many many calls to technical support later, we finally got to the root of the problem: on XP, TaxWorks works fine. But on Server 2003, TaxWorks stops working if any version of the .NET framework later than 2.0 gets installed.

For a while, we got by just avoiding installing .NET 3.0 and 3.5 from Windows Updates; but eventually, of course, one of the other applications on the server actually required .NET 3.5. The good news is, it turns out that even if you’re not the developer of a particular .NET app, you can target it to a specific version of the .NET runtime after-the-fact. It’s not particularly well documented – unless you’re a .NET developer yourself, in which case you should have done this BEFORE your app ever got in front of a customer – but the capability is there. .NET applications have an XML “application configuration file”, which is usually named “(yourapp).exe.config”, in the same directory as (yourapp).exe. Inside that configuration file, there should already be a supportedRuntime directive, which targets a specific version of the .NET framework… but if there isn’t one, you can insert it like this:

 <!-- JRS : attempting to force use of .NET v2 SP2 -->
    <supportedRuntime version="v2.0.50727" />

Of course, one of the challenges is figuring out what the heck runtime version corresponds to a human-readable “version” of .NET – it is extremely specific, and if you don’t get the string right, your app will then refuse to run.  After some trial and error navigating MSDN, I finally found this kb article showing the correct version strings for the various versions of the framework that are out there.  Long story short, if you drop to a cmd prompt and dir %systemroot%\Microsoft.NET\Framework, you’ll get a list of the versions of the .NET framework installed on the machine in question, in the formats you need to fill in the supportedRuntime directive in an application configuration file like the example above. With all that figured out, the code snippet above did the trick – the offending application no longer tries to reference the latest version of .NET installed; now it sticks to 2.0 (and works!) even with 3.5 also installed on the server.