I've got a site that has been running fine for almost 5 years. On Saturday, the site crashed. Called support to find that the drive on the dedicated server went south. They replaced the drive, but upgraded the OS to latest CentOS 5.x (closest match to what we had). I copied the code files and restored the database (thank god for backups!) and turned the site on.
All was fine... for a while. Suddenly the site started slowing. Up and down, up and down. Very odd because same machine (all we replaced was drive and upgraded OS) had no problems for over 4 years. Called support to find that apache was running out of memory. They said it was likely a memory leak in our code. OK, fine, but then why has it not been an issue for 4 years? Of course, they don't know, they can't examine our code (liability). The support guy reduced the max number of apache connections to 2000 (from 4000), but it is just going to prolong the problem, I'd like to try to resolve it.
So I started investigating. Some of the things I've come up with:
1. The current OS is installed with PHP 5.3.3. I installed the previous system in Jan 2009, before 5.3 was released. I am "guessing" it was on 5.2, but not certain. I went through change log and migration documentation
and couldn't find anything. I don't do anything really "cutting edge" so I'm wondering if this is a non-issue.
2. One of the things we do is use an .htaccess file to run a script to make sure someone is logged in before streaming them an audio file. Seems benign, but I notice when a new audio goes up, we see more of the problem.
3. I uploaded some new files and noticed they were owned by <user>:psacln. All the files I copied over (command line) were owned by root:root (what I was logged in as). I changed the permssions (recursively) on the web root (/var/www/httpdocs) to <user>:psacln and the site stopped working (unauthorized access). So I changed the folder only (not recursive) to <user>:psaserv and it seems to be fine (all files/folders under that root are now owned by <user>:psacln as if I had uploaded them by FTP).
So, any ideas if I am moving in the right direction? So far, so good, but it's only been a couple hours. Should this not work, is there some way to tune the code so I can find said memory leak? Any help appreciated.
>NOTE: Just way too much code to upload... I'm having trouble pinpointing where said leak might be... <