Today there was a bit of down time where when trying to connect first you received a database error message and then a Site not Available message. This outage lasted until ~11:23 AM CT The cause of the database error was the swap files being at 100%, data could not be processed. To fix this I had to reboot. This took about 7 min all told and was the cause of the Site not Available message. I'm in the process of learning how to identify what is causing the swap dirs to be filled, but I suspect it is the Top Gear files. I'm also learning how to create a script that will clean these up before the file gets to the same point again. In the mean time I'll probably have to reboot the server now and again. I'll try to schedule these for time window when the least amount of members will be affected. Thanks for supporting M/A and bearing with my learning curve on running a web server. Also if you are UNIX/Linux guru and know this stuff...lets talk please.
Mechanisms are in place and have been tested that will prevent another meltdown like the one we experienced before. Backups are run every hour and stored off site. The most we would lose is an hours worth of material.
Wierd, when it happened I got a message that said Google was unavailable - I use Google for my homepage - yet I could go directly to other sites, then a little while later all was right with the world again! Strange goings-on out there on the interwebs! Good job on finding the culprit and getting the site up again Nathan, well done! Have a cold one on me..... :beer
Working with the host we have expanded the temp space to 5GB from 512MB. This will prevent the issue in the future. Video uploads will not be counted in this tmp space. This was done in relation to a MySQL repair maintenance operation.