The reply turned out to be longer than I’d originally intended, so here it is as its own post.

I’ll try to be as objective as possible in this reply:

The most important thing for frustrated end users to keep in mind is that Mozilla/Firefox cannot be held responsible for cases where incorrectly written plugins and/or extensions cause Firefox to abuse system memory – that’s the trade-off between empowering developers and keeping the code squeaky clean.

Most of the cases reported are indeed caused by one or more extensions or plugins gone awry, doing something they shouldn’t be doing, or something they don’t know how to do properly. Some of the most popular plugins for Firefox are notorious for their memory leaks; but few users realize just how dangerous they can be, and that the Firefox devs cannot really do anything about it.

At the same time, there can be no doubt that Firefox has some memory leaks in the codebase itself. They’re clearly not easily reproducible and they don’t happen very readily nor often enough because the developers have clearly spared no effort in their attempts to address this problem for once and for all. But they’re there, nevertheless.

No matter how you look at it, the fact remains that under certain circumstances, doing certain stuff on certain machines in certain ways for certain people, Firefox still leaks memory. A lot. On Mac, Windows, and Linux. Yes, on clean installs too.

Now as a systems developer (Mac, Windows, and Linux w/ their respective native APIs; embedded systems; .NET and more with years of experience), I must say that of all the bugs and problems I’ve ever encountered, there is nothing that “cannot be fixed.” To say that this behavior is out of Firefox’s hands because it’s not their code that’s causing the problem is simply not true.

I’ve experienced memory leaks like this (and worse) in my own code in the past, largely due to stupid mistakes and silly oversights. It takes extreme persistency to make memory leaks go away – a willingness to spend 24 hours on-end & non-stop crawling through code, memory dumps, and stack traces to try and find out where things are going wrong. It requires remote debugging on allegedly-affected machines. It requires reading through dozens to hundreds of sometimes clueless users describing in the most general of terms what they were doing when things went wrong. In short, it requires a lot of effort and very little recognition and a hell of a lot of hair-pulling.

But it can be done.

C++ is an incredibly powerful language. If you know the code you’re developing and the systems you’re writing it for, there’s nothing you cannot fix. Dynamic memory allocation is the biggest gift/curse in the world, but in C and C++ if you can allocate something that means you can free it. Even if you don’t have a mechanism to find out where it is and how it got there. But you just have to be cunning enough to figure out how to track them down and set them free, taking care to know when and where to do so safely…. and you have to be familiar with every single routine and how they work; which is obviously extremely difficult with codebases as large and complicated as Firefox’s.

There are even workarounds for the memory leaks (assuming they can be isolated) if the developers aren’t willing or capable of doing the aforementioned. If you’re dealing with leaky libraries that you can’t fix, in the very worst-case scenarios you can hook into them at runtime, access the functions you need, reserve the memory required, get the job done, copy only what you need, then free it right back. All of it. You can have helper threads or processes handle this stuff then wipe them and their memory spaces clean when they’re done to complete the memory insulation.

There’s a lot of stuff that can be done, and none of them are easy. But the journey of a thousand miles begins with a single step, and developers and evangelists denying a problem exists isn’t the way to go about addressing the matter at hand.

At the end of the day, Firefox is a great browser and any complaints about its performance and its shortcomings are only out of a sense that it can do better – that it has to in order to remain at the top of its game in a cutthroat market of only the most intense of competition.

For a device that’s supposed to do Firefox, Skype and not much more, an underpowered PC with a touchscreen isn’t going to accomplish much. For one thing, Firefox is a huge performance drain and a memory hog to boot that underpowered hardware (even on-par with an Eee) simply won’t support and for another, there’s no way to get PC hardware down to the sub-$200 price range.

What TechCrunch wants – whether they know it or not – is an oversized PDA, not an underpowered PC. And it’s not just a question of semantics, it’s a question of foundations and principles – and it makes a huge difference in terms of end-user experience and the bottom line.

For the functionality that TechCrunch is trying to pack into this opensource, mass-market web gadget, there’s nothing that wouldn’t work better, faster, and cheaper on specialized hardware rather than on generic PC components.

While the world is now in the midst of a touch-screen craze, it’s important to keep in mind when and where that works. For a web browser and a VoIP client, a touchscreen doesn’t provide much added value, but it does add quite a hefty amount to the bottom line. A couple of buttons at the top/side of the device that provide basic functionality (Go/Dial, Stop/End) would certainly suffice for most purposes. A thin slide-out keyboard is far-cheaper and more user-friendly than an onscreen keyboard, and would make things like entering site addresses and using email clients and Google Docs quite enjoyable.

A PDA-style ARM processor, running software compiled for the ARM platform could provide a more satisfactory end-user experience with regards to performance and can come in smaller form-factors and/or as embedded systems.

It’s important to bear in mind the difference between consumer electronics and a computer. Whereas Asus had to keep their Eee x86 so that it can run whatever a a PC user could demand from it, a web browsing tablet only needs to run what the manufacturer intends it to. In hardware design, there’s a constant compromise between flexibility and complexity which is directly tied to price, size, and ease-of-use.

A tablet designed to surf the web and run Skype doesn’t need to do anything else; but it shouldn’t do anything else if price and size are of any concern. It’s easy to get caught up imagining a device that can do anything and everything; but you can only go so far before things begin to spiral out of control.

With this update the installation process has been simplified somewhat, in particular the need modify HTTPD.INI to set the server variables has been eliminated – you just need to install ISAPI_Rewrite 3, configure php.ini to load up request_uri.inc, and you’re set.

Request_URI for Windows 1.1 retains backwards compatibility with ISAPI_Rewrite 2.x for those of you who’d rather not switch to the new (and much-improved) version 3.x.

Download Request_URI for IIS 1.1

The full instructions for installing and configuring Request_URI for Windows can be found at the original posting. Please post any support questions in the forums.

NeoSmart Technologies is not affiliated with Helicon software in any way.

Facebook recently conducted a poll which showed up on the homepage newsfeed, and asked Facebook members just how exactly did they think Facebook’s “friend finder” worked when it prompted them for their email address & password in order to get a list of contacts. The numbers pretty much speak for themselves, here’s what they looked like near the end of the campaign:

Now ignore the dark blue bar: it’s a red herring and doesn’t contain any interesting info. The real juicy bit is the “Yes” option, and its 20% response.

20% of Facebook’s 80 Million active users (give or take) believe that the passwords for their email addresses are being stored when they use the Friend Finder…. and that doesn’t bother them in the least. That’s sixteen million people who don’t give a damn about their privacy, the contents of their email, or who has control of their entire online personas.

This is a subject that’s been chewed half to death already countless times by people far more in the know than myself; Jeff Atwood’s excellent article on the topic covers the dangers of sites asking for users’ email addresses & passwords, and – far more importantly – presents several more secure alternatives for web application developers looking to expand their social networks.

To put things in perspective, take a look at this downright horrifying tale on ReadWriteWeb about software that prompted users for their email addresses & passwords, then proceeded to save them for malicious use… then realize that 16 million Facebook users out there don’t care if this happens to them. Think about all the private, sensitive, confidential information available on your email account and just how truly terrible it would be for that info to fall in the wrong hands.

Of course all this begs the question: who’s to blame for this bout of end-user stupidity (for lack of a more politically-correct term)? Is it naïveté/trust in the goodwill of others that gets users to give out such sensitive data to people (Facebook has 500 employees!) they don’t know from Adam? Or is it that they just don’t get how dangerous it can be (see the ReadWriteWeb article for proof)? Or is it, maybe, that they’ve simply gotten accustomed to being asked for their email address and corresponding password by “trusted” sites they love to visit, too caught up in the “gather as many friends as you can” game to give a second thought to identity theft and fraud?

Personally, I can recall a time when most “normal people” I know would refuse flat-out to share such sensitive data with a site (phishing, tech support, etc. obviously excluded); but in the wake of “Web 2.0” it’s become so normal to ask for email addresses and passwords that no one ever gives it a second thought.

And it’s not just Facebook. To be totally frank, even if Facebook were to store end users’ passwords in their database, the access to that info would probably be very highly guarded… but when every new social network on the block is suddenly doing the same thing – you can get a good picture of just how easy it would be to steal users’ passwords.

MQ’s 3 Steps for World Domination

1. Send out an email purporting to be from “the hottest new social network around” informing the recipient that their “friends” want them to join: “Click here to show Peter you’re a real friend!”
2. Get the user to register a new account – make the procedure as pain-free and simple as possible… and right then and there on the registration page ask for the user’s email address and password so as to “make it easy to tell all your friends you care and get popular really fast…”
3. Profit.

As soon as it’s OK for one person to do it, it’ll be OK for everyone to… and then we’ll be in too deep to do anything about it.

So why does Facebook – after polling their end users and seeing just how dire the situation is – continue to use the same flawed mechanism of harvesting email addresses… especially when better, safer alternatives exist?

Memory usage: Several new technologies work together to reduce the amount of memory used by Firefox 3 over a web browsing session. Memory cycles are broken and collected by an automated cycle collector, a new memory allocator reduces fragmentation, hundreds of leaks have been fixed, and caching strategies have been tuned.

We’re sorry to have to break it to you, but if you thought it was too good to be true you were right. Firefox still uses a lot of memory – way too much memory for a web browser.

We haven’t seen it reach 1GiB+ like we have with previous versions, but it’s quite normal for Firefox 3 to be sucking up ~300MiB of memory right off the bat, without a memory leak (the difference between memory leaks and normal memory abusage is that in a memory leak you’ll see the memory usage keep increasing the longer the browser is open/in-use).

This is a screenshot of Firefox’s memory usage after just a half hour or so with only a couple of HTML-only tabs open. This particular screenshot was taken on Linux where Firefox is using the shared GTK libraries – on our Windows PCs, it’s normal to find Firefox 3 taking up ~350MiB or so on both XP and Vista.

The sad thing is that isn’t caused by one of the memory leaks that plagued previous versions of Firefox. It’s Firefox 3 is supposed to take up that much memory – at least, that’s our assumption given how we’ve never seen it take up less.

Firefox 3 has a number of memory-hogging features added to the mix that are probably at least partially responsible for the absolutely gargantuan memory footprint. For example, Firefox now uses an SQL engine to keep track of your history and bookmarks, amongst other things. While that particular feature is powered by SQL-lite, which should – in theory – not take up too much memory, we’re at a loss to explain what else is wasting memory left, right, and center in the world’s most-popular open source web browser.

Things like full-text on-the-fly searching of the web cache for when you type text in the address bar certainly have an impact as well – that’s a lot of stuff to keep in memory at one time. But Opera 9.5 does the same with a lot less memory, so obviously Firefox 3 is doing something wrong.

It’s a shame that Firefox 3 is on the verge of a release and is so terribly unfit to run on any machine – Windows, Linux, or OS X – with less than at least a couple of gigabytes of memory.

Whether or not MinWin is the very same kernel that went into Vista or not is officially unknown at the moment; but what we do know is that Shipping Seven is either one huge fake, or else that the Windows core programmers at Microsoft are so stupid that they don’t know the first thing about coding, kernels, operating systems and compilers.

The post at Shipping Seven is littered from beginning to end with fallacies, lies, and incorrect deductions that anyone with even the most basic coding skills would know better than to ever post, especially not when attempting to pass it off as the work of some of the more talented coders out there.

Here are some of the more-glaring factual errors in the post that completely strip Shipping Seven of any authenticity or authority it may have on the topic of Windows 7:

How many times has the Ubuntu or Mac OS X kernel been rewritten?

Correction: OS X is powered by a rewrite of the XNU kernel which is a modified version of the Mach kernel which, in turn, is a complete rewrite of the original BSD kernel. And, of course, Ubuntu isn’t an OS in and of itself, rather it’s just a distribution of Linux.

While it can be argued that not every developer at Microsoft is expected to have intimate knowledge of the inner-workings of other operating systems, no one in their right mind would believe that the Windows kernel programmers don’t even know what kernels their strongest competitors are currently using.

We spent a boatload of time during Windows Vista making everything ‘componentizable’ – So that we could (by creating some xml files that our build process uses) create a boatload of different versions of Vista (and Server 2008).

….

You already have MinWin – It is the core system components that Windows Vista needs to function; everything else on the system depends directly or indirectly on it. It is the last thing you could (theoretically) uninstall.

So, if you really really want it, you can get it, I suppose – you probably could (using the command line) uninstall almost every single Windows Vista system component, including the user interface. I don’t know what the hell you’d do with just a kernel and a kernel loader on your machine, though.

Assuming you can get past the way that the post was written (with references like “using the command line” which indicate a general lack of knowledge about computers in general; treating the command line as if it were a “god mode” that can be used to do just about anything), there’s still the matter of factual inaccuracies – and inconsistencies in the article itself.

You can’t change/modify/revert pre-build settings by running commands in the command line. Components that are integrated at compile time simply cannot be removed by running a bunch of commands afterwards – especially not from within the resulting OS itself.

Anyone that’s ever manually compiled a Linux kernel knows this. You can’t strip ext3 support from the kernel after it’s already built any more than you can add Reiser4 support to the kernel without re-building it. As a matter of fact, anyone who’s built anything at all should know this – the same rules apply to any other program as well. For example, you can’t remove PHP support from Apache if you’ve compiled mod_php directly into the binaries.

Shipping Seven is a big, fat fraud. It’s written by someone with only the most basic knowledge of computers, zero knowledge of coding concepts, and absolutely no experience with kernels and operating systems. Shipping Seven is most likely written by the equivalent of script kiddy, eagerly awaiting the first leaked builds of Windows 7 to appease an inner itch – most likely all the while lamenting his lack of involvement in the Longhorn beta. It isn’t worth the time it takes to read, and definitely doesn’t deserve even the questionable authority it now has on the topic.

One of the first thing Computer Science teachers drill into the heads of their students is that it’s important to map everything out beforehand. Design the algorithm. Draw the UML diagrams. Decide the entire flow of data and the relationships between everything before you even touch the IDE. While this is integral advice for anything above a small-complexity project, there is an exception: if you have a gut feeling, follow it.

For instance, the other day I sat down to write a simulator for a MIPS datacache, with different replacement policies. “Ideally,” the planning procedure would have involved designing the sequence diagram, a flowchart detailing the method used by the cache to determine expired entries, and generally-speaking a lot of time down the hole just visualizing what happens beforehand.

While mulling things over for a minute or two in my head to gather my thoughts, I found myself scribbling three phrases on the PostIt note in front of me:

• Access times -> FIFO
• Access times defined by unique tag
• Reading memory == dequeue item

These were spur-of-the-moment analyses of the caching procedure. Just thinking about the caching methodology brought these ideas to mind, with no particular “thinking through” of type. Looking back at them, I couldn’t (instantaneously) recall why I felt that access times should be removed FIFO from the data structure on read, but I figured it must have made sense on some subconscious level and plowed ahead right into the code.

In the end, the code worked, the project was done far quicker than originally imagined, and I had time to kill post this article.

Any good programmer you talk to will have a similar experience to share. The only reason I trusted my instinct in the first place on this matter (keeping in mind that the implementation of “random” algorithms can be a huge timesink if they don’t actually work) was because trusting my coding instincts has generally proved to be the right thing in the past – this is simply the most recent example.

Of course, every once in a while an impromptu idea may not be perfect; indeed, it’s possible that they’re fatally flawed. Whether it’s because the problem itself wasn’t initially understood or simply a matter of arriving to the wrong (hasty) conclusion, mistakes can (and have) occur. But it’s important to bear in mind the ratio of brilliant ideas to failures – most good programmers I’ve spoken to reaffirm that it’s a very workable hit-miss ratio.

Certainly programming by instinct has its drawbacks. For instance, these “strokes of brilliance” make up the majority of what we refer to – in retrospect – as quick and dirty solutions. There’s a time and a place for using them; and being unable to figure that part out can end up hurting you and your program. Typically speaking, using the first thought that occurs to you when structuring your program (what classes, how they interact with one another, etc.) isn’t such a good idea. Things like this need to be refined non-stop (until you get that warm, fuzzy feeling when you look at your code and quick solutions won’t help you there.

It’s usually within individual functions when you need to get something done based on the state of one or more other objects that instincts prove handy. To put it another way: when the code you’re working needs some measure of creativity and out-of-the-box thinking, don’t ever pass up on a gut feeling. Programming gives experience and insight, much of it subconscious. Patterns and ideas may leap out at you in ways that aren’t immediately obvious, but even if we don’t fully comprehend how the ideas came into existence it doesn’t mean they don’t have a substantial amount of intricate logic and supporting foundation to back them.

The next time an idea flashes into existence, like the proverbial lightbulb flickering on above your head, take a minute to appreciate its glow. Realize that you’re lucky to have such inexplicable moments, and trust them to take you and your code to the next level.

iReboot isn’t by any means a major application, but it’s gathered a pretty strong following over the months, mostly by people interested in boosting productivity (or increasing laziness) to the max. But there was one flaw in iReboot that made all the hard work we put into making it as unobtrusive and minimalistic as possible almost meaningless: if you had UAC enabled, iReboot will not run automatically at startup, no matter what you do.

This behavior comes as a result of the architecture that Microsoft used to secure Windows Vista, which doesn’t allow for applications requiring admin approval to run at startup. It doesn’t matter what your application does or if you absolutely trust it beyond the shadow of the doubt, Windows Vista simply won’t let an application that runs in elevated privileges mode to launch at startup – end of story.

Users of iReboot were quick to point out that this is a major drawback that made it almost useless – after all, it’s far less productive to have to manually run an application when you want to reboot than it is to wait for that startup screen to appear and select the OS you want. So we set about finding a solution.

We’ve just released iReboot 1.1, a UAC-free implementation that doesn’t require admin approval, elevation, etc. past the initial installation. And, yes, it does run automatically at startup too!

The Gory Details (feel free to skip below to the download links!)

In order for iReboot to be of any use, we had to get around Microsoft’s UAC limitations. For iReboot, it was of the absolute importance that it run at startup, and that it be allowed system access from normal user accounts. On Windows XP – where everyone runs as an Administrator and there are no annoying UAC prompts – it was a non-issue. But on Windows Vista, the new architectural requirements for running applications in elevated privilege modes made it near impossible.

While digging around for possible solutions, it became clear that the only possible fix would be to split iReboot into two parts. One would run in the background as a service, running under the SYSTEM or LOCAL SERVICE accounts and having privileged access to the OS without requiring admin approval or UAC elevation, and with the second half running as an unprivileged userspace client program which interacts with the service backend to get stuff done.

The resulting application has an installer – which requires admin privileges, of course – which installs and launches the background service. The background service has full permission to do what we need to get operating system XXXX to be the default option for the next boot, but – in line with the Windows Service Model – cannot be interacted with by end users.

The installer also adds a normal UI application which sits in the taskbar (from where end-users may interact with and use iReboot) and communicates with the backend service via a custom API which must not require the execution of any privileged code. The service can do whatever it wants (well, whatever we want it to do, but lets not get picky here!), but the client program must only perform actions which normal, unprivileged users have permission to execute.

By using a standard inter-process communication API we avoided the need for any special actions on behalf of the client application, effectively separating logic (residing and executing on the backend service, free from the many limitations of UAC) and presentation/design (the client application, bound to obey UAC’s every wish).

The Bottom Line

Anyone running Windows XP or Windows Vista – with or without UAC and/or admin approval mode enabled – can now run iReboot at startup and use it to boot into whatever OS they like (in conjunction with EasyBCD, of course!).

But getting this far wasn’t easy. With Windows Vista, what should have been 100 lines of code maximum ended up being a dozen times longer, split across two different processes, and requiring way too much man-hours to write the most minimalist and to-the-point piece of software we’ve released to date.

Perhaps most importantly though, is the fact that Windows Vista’s newly-implemented security limitations are artificial at best, easy to code around, and only there to give the impression of security. Any program that UAC blocks from starting up "for good security reasons" can be coded to work around these limitations with (relative) ease. The "architectural redesign" of Vista’s security framework isn’t so much a rebuilt system as much as it is a makeover, intended to give the false impression of a more secure OS.

With the current Windows Vista security models, Microsoft can claim that Vista blocks system-modification tools from running at startup; but the truth is, there are still many ways to get them to run. At the end of day, our experience with iReboot and Vista’s security implementations brings us to the sad conclusion that with Windows Vista, Microsoft has made ISVs’ jobs more complicated without actually providing any any further protection for end users from malware authors – which certainly isn’t the best way of going about this task.

Anyway, the fruits of our efforts:

Download iReboot 1.1 (248 KiB)

[support] [donate] [changelog]

The "good news" is, if you’re on Windows Vista SP1 x86, DEP doesn’t get in the way as often. And for when it does, Windows Vista x86 lets you disable DEP and continue along on your merry way. But Windows Vista x64 isn’t as forgiving – even after you use a program like EasyBCD to disable DEP entirely, you can’t stop hardware-based DEP or exempt software from the protection list on 64-bit operating systems.

Adobe has yet to provide an official (or even an unofficial) response on the matter; but seeing as Adobe hasn’t properly touched the Audition code-base since buying out Cool Edit Pro, it’s probably safe to assume we won’t be seeing an update anytime too soon. (for instance, Adobe Audition 3.0, released in Sep. of 2007, still doesn’t have that omnipresent 3.0.1 patch out yet).

A program’s executable code is split into multiple sections depending on what’s stored in it and what it’s used for. Some of these sections are marked as "no-execute" in the extended x86 instruction set. However, an incorrectly-written program will either incorrectly try to run code that shouldn’t be executed or else mistakenly mark executable code as "no-execute" – both of which will cause DEP to interfere and get Windows to kill the process.

The interesting thing here is – why didn’t Windows Vista RTM cause DEP to fire as well? After all, if Adobe Audition is attempting to execute portions of its code that shouldn’t be touched, Windows should block the attempt SP1 or not.

One possible explanation is that Adobe Audition is attempting to run code from a core Windows library, but it’s accessing the code in a non-standard way that breaks when that library has been updated or modified. Or perhaps its determining the entrypoint for a library file dynamically, and something about Windows Vista SP1 makes it start at the wrong place.

Either way, it seems you shouldn’t be using Windows Vista x64 SP1 if you depend on audio or video encoding to make a living. The sad thing is, 64-bit operating systems were often touted as "the answer" to encoders’ needs, between the slightly-optimized processing of encoder instructions and the ability to support 4GiB+ of memory; yet here we are at square zero once more.

Update (April 21st, 2008)

Reader Mike B. wrote in letting us know that x86 versions of Windows Vista SP1 do not trigger a DEP alert at all, so it seems this is a problem exclusive to Vista x64 machines running SP1. If you are using Windows Vista SP1 x86, you shouldn’t have to worry about this issue. Thanks, Mike!

NeoSmart DevNet is a new effort on behalf of NST to reach out and lend a helping hand to other software developers by providing a number of tools, libraries, and frameworks that we’ve developed over the last several years to address certain commonly-encountered issues in a generic-yet-customizable manner.

Basically, DevNet is an attempt at getting software developers to spend more time on developing their applications verses worrying about the stuff that they need to get there. Unlike other developer resources on the internet, the goal of DevNet is to provide complete working subsystems and frameworks to developers, almost entirely doing away with the need for supplemental coding and code-monkey work.

For instance, the first of our DevNet projects to be released is a scriptable graphical HTTP/FTP downloader. The NST Downloader is intended to be drop-in solution for anyone looking to add the ability to download and run components from the internet to their software projects. All interfacing with the NST Downloader is done via command-line arguments, making it language-agnostic and dead-simple to use.

Downloader components have become an integral part of most desktop applications. Whether its to download upgrades, plugins, extensions, documentation, or other remotely-located data, most large projects have come across the need for such a module at one point of time or the other. They’re straight-forward in functionality, but a huge pain to write, debug, and test. The bottom-line is, there’s no reason anyone should be writing and re-writing a downloader when that’s not the purpose of their project.

Developers shouldn’t have to re-invent the wheel over and over again, especially not when other free software developers have already written and tested the needed components and tuned them as necessary. Just as end-users have come to expect quality software that gets the job done from their favorite ISVs, developers too should have the comfort of using drop-in subsystems to accomplish their various needs; especially when it comes to such esoteric and straight-forward needs that simply take time and resources to develop and maintain.

Other DevNet projects currently undergoing the last phases of testing include a Regionalization Toolkit intended to make creating, deploying, and implementing regionalizations of .NET projects dead-simple and an Upgrade Framework that features one-liners to easily check for updates to your applications and even automatically download and install any available upgrades.

Still not clear on what the point of all this is? Just think of DevNet as extending the NeoSmart experience to include our beloved developer brethren.

I really don’t need to go into details about this much, since Lloyd Budd has done such a good job explaining what it is and what WordPress hopes to achieve in this program. This year, WordPress has an even-larger and more-exciting list of possible projects than before, along with a list of the mentors available for each idea. This Google Summer of Code, I’ll be mentoring for the WordPress projects in the one area that is closest to my heart: improving performance.

It would be unfair to say that WordPress is slow or an inadequately-performing blogging engine, because that’s not really true. "Performance," more than any other software characteristic or trait, is a very relative and subjective index. It depends on thousands of different factors, it has dozens of different baselines, and most confusingly of all, sometimes the perception of performance matters more than the performance itself.

But no matter where WordPress is now in terms of high-performance, there’s always room for improvement. Late last year, disappointed in this very blog’s less-than-stellar showing during the many Slashdot and Digg appearances (even on a damn-decent dedicated server), we decided to re-write the WordPress front-end from scratch.

You’re looking at the result now: the homepage and individual posts are powered by PerformancePress, our in-house re-write of the WordPress page-displaying code. The NeoSmart Files is still running WordPress, just behind the covers. The procedure we followed in developing PerformancePress was simple: only load what you need, don’t waste CPU cycles if you don’t have to, minimize SQL queries wherever possible, don’t loop when you can avoid it, and a number of other load-reducing principles.

The thing is, there’s a huge difference between in-house code and software that’s been "generically-developed" for use by the masses. While we could make assumptions about our own use of the blogging system, the WordPress project doesn’t have that privilege. There is no cutting-corners or jumping to conclusions on a for-redistribution project. Flexibility usually comes at the cost of optimization; and the net result can be more than a little tricky to understand.

These are some of the same reasons we cannot just "release" the current PerformancePress code we’re using here – it wouldn’t last 10 minutes in the real world. But hopefully this Google Summer of Code we will have a chance to bring out in WordPress the best of both worlds. PerformancePress is most likely a dead-end; but it’s been an invaluable experience in realizing how to create the most resource-efficient, well-scaling webapps; and hopefully we can take that to the next level this summer.

Don’t get this wrong: we’ve got our own reservations about SP1 (between performance and usability – or, more accurately, the lack thereof). But Microsoft is not to blame because certain system tools and utilities won’t run on Windows Vista SP1 when it’s released in March.

According to The Register, "Vista SP1 kills and maims security apps, utilities" and that it’s somehow Microsoft’s fault that antivirus/firewall software by BitDefender, Jiangmin, Trend Micro, and Zone Alarm no longer works on SP1 – but it seems they forgot to mention two facts:

1. It’s bad coding habits that breaks these utilities.
2. Thanks to pre-release builds of Vista SP1, all 5 malware-protection programs have updated versions available that are Vista SP1 compatible and shouldn’t give their users any problems come mid-March and SP1.

Microsoft may have many faults and may have made many mistakes starting with Windows Vista’s initial release, but one thing they have done right is give developers (software and hardware alike) plenty of time to test their products against SP1 builds and plenty of opportunity to find out why they no longer work.

We’ve studied (and indeed, ran into) some of the issues that these programs experience under SP1 in our own code. In our testing, it seems that these incompatibilities only occur when developers attempt to use undocumented/unofficial workarounds to gain access to system operations, work around UAC limitations, and interface with the desktop & other running applications.

Following the guidelines published on MSDN (at least those marked as "Vista-compatible"), software can be written that correctly provides an interface between services, desktop applications, UAC, the internet, and low-level subsystems including drivers and raw access to disks and files – albeit with a lot of extra work and with some monetary expenses (namely, signed drivers).

At the end of the day, Vista SP1 may be many things but one thing it’s not is a compatibility sinkhole. Especially when compared to the problems experienced back in the days of Windows XP SP2’s release, the list of broken applications on Windows Vista is appreciably short – and moot, seeing as new versions of the software have already addressed the issues satisfactorily.

In a nutshell, round-robin algorithms pair an incoming request to a specific machine by cycling (or, more specifically, circling) through a list of servers capable of handling the request. It’s a common solution to many network load balancing needs, even though it does not result in a perfectly-balanced load distribution, strictly speaking. In the non-ideal world we our servers live in, there are many reasons why the stock round-robin algorithm just isn’t good enough when it comes to properly balancing server loads.

The first and most important thing to keep in mind is that not all servers are created equal. One should be able to take advantage of all available resources, and it’s impossible to guarantee that all the servers available to process incoming requests are capable of dealing with the same load quantities, take as long to carry out each command, and deal with larger/longer queues as elegantly. Nor can all requests be treated the same, either. Some take longer to process than others, involve more work, and are generally more-demanding than the rest – just as others are finished relatively fast and with far-fewer resources.

Weighted round-robin is one way addressing these shortcomings. In particular, it provides a clean and effective way of solving the first-half of the problem by focusing on fairly distributing the load amongst available resources, verses attempting to equally distribute the requests.

In the server environment, a difference in server capacities and processing capabilities can result in a more-marked difference in the performance of the different servers (read: less balanced) when compared to the effect of a difference in the requests themselves.

In a weighted round-robin algorithm, each destination (in this case, server) is assigned a value that signifies, relative to the other servers in the list, how that server performs. This “weight” determines how many more (or fewer) requests are sent that server’s way; compared to the other servers on the list.

Take, for example, the case of web servers. Assume you have three servers that have been individually benchmarked and configured that are to be deployed in a weighted round-robin environment. The first can handle 100 req/sec, the second can do 300 req/sec, and the last can only do 25 req/sec (all on average, tested with an automated benchmark serving the same data). Normally, with such a discrepancy in the servers’ performance, the third would be excluded from the setup entirely. But in a weighted round-robin, each server can be assigned as much as it can handle in the round-robin configuration script/file:

Resource			Weight
--------			------
server1.fqdn	     4
server2.fqdn	    12
server3.fqdn	     1

It’s pretty clear what happens next: for every 12 requests sent to server two, 4 will be sent to server one, and just 1 will be sent to server 3. The result is a more even, if less equal, load distribution.

The only problem with weighted round-robin algorithms is that they haven’t been adopted by the majority of the load-distribution implementations on the market…

…Which brings us to the point of this article. You may have noticed the question mark in the title – it’s there for a reason. Do you know of any load-distribution implementations that operate based on the weighted round-robin principle? On Linux it wouldn’t take more than a couple of fairly-trivial changes to the code for your favorite DNS server to switch from round-robin load distribution to a weighted round-robin solution… But the same cannot be said for Windows where Microsoft’s DNS server is the standard. So the question stands: what kind of load distribution solutions can you recommend with support for weighted round-robin load balancing, if any?

That’s not true. Don’t fall for it. Don’t attempt to use PHP in a multi-threaded environment (mpm_worker on Apache, ISAPI on IIS, etc.), because PHP thread-safety is a myth.. nothing more than a bunch of lies, if you will.

If you look at the PHP download page, you’ll see that the pre-built binaries (in this case, Windows) are split into two: thread-safe and non-thread-safe:

The problem is, no matter which you choose, PHP isn’t thread safe. You’ll still get the same, old, dreaded “PHP has encountered an access violation at memory_address” error.

It’s not a question of server configuration so much as it is one of PHP writing bad code and pretending that’s not the case. PHP isn’t multi-threading ready and most everyone knows it… but it seems they still feel the need to pass it off as if it were, never mind the complaints and bug reports that come.

There isn’t a known solution to this problem. The source code for the official PHP distribution is simply not re-entrant and isn’t capable of being used in multi-threaded environments. It’s filled with race conditions and lacks the proper monitors/locks on shared memory spaces (which needn’t be shared in the first place). Unfortunately, there’s not much more to be said; making this post more of a rant than a guiding light for those of you experiencing this issue.

At NeoSmart Technologies we still recommend the only other production-ready, decently-performing *sapi alternative; which is the deployment of PHP by means of the FastCGI protocol. Whether on Linux or Windows, FastCGI scripting engines are considerably faster than their age-old CGI implementations (though the outcome is more-clearly felt on Windows where the CGI model is hideously slow). But we’ve already discussed this before, countless times.

The moral of the story is that PHP isn’t multi-threaded – yet if you go by what they say, you’d come to think it was… until you try it out for yourself and realize that not a darn thing has changed in the 5.2.x releases when it comes to thread-safety and the deployment of PHP in multi-threaded environments.

If you still insist on using ISAPI implementations, you should probably use the latest versions of the PHP4 distribution instead. The older PHP line is less-apt to cause an access-violation error on your machine, and will more than outclass the PHP5 “thread-safe” distributions when it comes to reliability.

PHP needs to clean up their act and either produce real (and verifiably so) thread-safe, re-entrant code or stop the false propaganda that would lead you to believe it’s thread-safe in its current incarnation – especially when most other dynamic scripting languages (such as Perl) have proper, thread-safe engines available.

Microsoft isn’t new to the whole “virtual” monopoly business (where a single company holds the entire market thanks to “superior technology” and “better business sense”) – it’s just not too often that they’re on the wrong side of this particular proverbial fence.

When Silverlight was first announced and PopFly, Microsoft’s social network built to demonstrate and hopefully kickoff Silverlight, were simultaneously launched; we were quick to appreciate the technical aspects of .NET and WPF taken online, but were careful to make it clear that we didn’t think it stood much of a chance.

But things might be on the verge of a big change. Large portions Microsoft’s website are in the middle of a redesign that will feature a fully Silverlight-powered interface – doing away with HTML and everything else. We’ve had a chance to test the new interface (currently in beta), and here’s what we think:

1. According to Compete, Microsoft.com is the 8th most popular site on the internet, with around 60 million unique visitors a month. Put another way, if Microsoft successfully pulls this off, that’s 60 million new Silverlight users in the first month alone!
2. The new, Silverlight-powered interface is a pretty big step up from the old design, making it easy to access information about individual downloads and view overall info and lists.
3. The Silverlight part of the interface is almost wholly unnecessary. It’s really nice to use, it’s smooth, it’s easy, and it’s beautiful – but it’s nothing that requires a RIA in the first place. Microsoft could have easily implemented the same user experience (give or take) with HTML + JavaScript/AJAX; with a lot less effort and greater compatibility.
4. At the moment, very few non-Microsoft-owned sites are using Silverlight at all; let alone for the entire UI. And of those that do, none have anywhere the amount of exposure that Microsoft.com gets.

Keeping these facts in mind, there’s only one logical conclusion to be drawn: Microsoft realizes (as has the rest of the geek community) that Silverlight is on the verge of being forgotten. Claims of superiority aside (true or otherwise), Microsoft has realized that if Silverlight is to stand a chance, it’s going to take more than a failed attempt at making a Silverlight-powered social community to get developers and consumers alike to adopt Silverlight.

It’s a desperate move, there’s no doubt about it. While Microsoft will no doubt be making an alternative HTML interface available for a mixture of legal and practical purposes, switching Microsoft.com over to Silverlight is a sure-fire way to get that attention…. and depending on how it’s both marketed and carried out, it could be what it takes to make developers start taking Silverlight seriously.

Whether this’ll work out or not, only time can tell. We don’t know when the new Silverlight-interface will be going mainstream, but it’s probably not for a couple more months at the very least. The current interface still links to many as-of-yet not updated pages, and portions of the Silverlight section still appear to be missing some features here and there. Overall, the new interface is very user-friendly and well-developed, though.

You can either view NeoSmart Technologies screenshot gallery of the upcoming Microsoft.com re-design or attempt to access the beta link directly (working as of 01/03/08); but still pictures don’t really do the interface justice. Where the new UI really shines is the overall grace and fluidity of the interface, with gentle hover effects and fade in/out transitions that are done just right.

Editor’s Note: This article has been updated to clear up some references to a full-site redesign of Microsoft.com. We do not have any evidence that all of Microsoft.com is being redesigned to take advantage of Silverlight, just large portions of it. Sorry for any confusion.