MySQL on the Fritz

Earlier today (according to UTC at any rate), NeoSmart Technologies had a severe MySQL problem. After some very heavy load on our resource-intensive NST Gallery it seems that MySQL just started coughing up bytes, nuts, and bolts. It’s all fixed now though, and we’re sorry for the downtime.

The reason it took so long is that we decided to “seize the moment” to upgrade MySQL to the new 5.1.x series, and unfortuantely, we ran into a bug that has been in the 5.1.x series for a while that stops InnoDB tables from loading up. We tried to go back to the 5.0.x line, but that didn’t work either. It took a long time to get our binary recovery data from MySQL to another server to get a SQL dump of the data, then completely scrub the main NST server clean of all MySQL residue, then restore the SQL dumps.

Hopefully everything is well now, and we’ve put in some safeguards that should prevent or at least minimize this kind of problem in the future. If you notice anything weird still happening, please do let us know. Some of the symptoms included an inability to view Forum posts, crashes when commenting here on The NeoSmart Files, and some problems in the Gallery. Again, our sincerest apologies for the downtime.

Leave a Reply

Your email address will not be published.