Keeping string contents in memory across requests?
let me first tell you a bit of the background to my recent problem.
Some months ago, I wrote a series of PHP scripts, that are based around an SQLite database. One part of the scripts is used to generate new or fetch existing external data, which is then saved in the db-file. The other half of the scripts later extracts this data and removes it from the database after it has been used.
The problem stems from the new level of use the scripts have seen and the way these scripts are designed: if data is fetched from an external source, it always happens in smaller pieces (of 5-100 kilobyte) per single request. Data-input tasks are split up in most cases, which means, that a single task can generate a few dozen HTTP-requests and disk-writes or more. The read/delete part of the equation have proven to not be an issue.
For technical reasons - I'm not root on the server this runs on and have no physical access to the hardware - I can not use a different SQL database with a server process and load the whole database into the RAM, put in an SSD for better performance or maybe even create a RAM-disk and move the db-file there. Since the process of fetching external data will always be fragmented, this creates a problem. When too many scripts are both writing and reading/deleting from the SQLite database file, the CPU load spikes due to the high number of disk-writes and the hard-disk fails to keep up with the demand created by the many disk-seeks and -writes.
Now my question is: can I somehow keep a few strings (to the total amount of a few megabytes at a time) in the memory for a short while and later access them from a different script instance? At first, I thought about using sessions, but as far as I know, they will also be saved to disk immediately.
If an in-memory solution exists, I would be able to cache everything until the last fragmented database entry and write them in one go at the end. This would reduce the number of disk-writes by a few orders of magnitude and solve my problem, because the current level of activity will not rise again.
If it's not possible, a new server is needed and I will have free choice of the different solutions I already outlined earlier anyway.
Thanks for your help,
The only way to keep stuff somewhere to remember it is in a DB or in a file such as the session file (which CAN be access from other scripts but you need the session id).
As you're not wanting to use the disk, neither of those are particularly great ideas... so there is one more option..
You'd need another script that cyces in a loop with a listening TCP socket. It then keeps its own variables in memory like a normal script but can accept socket connections and read/write to the connected client. It's probably not the most efficient way of doing what you want but given that your disk is already frying with activity its probably the only realistic way of achieving the desired effect. Be warned though, you can't multithread PHP scripts on windows so as far as I know, you can only handle one connection at a time. On linux you can fork so you can effectively handle more clients (as I understand it). I did once find a script that claimed to do this on windows but I didn't entirely understand how it worked after several days of disecting it.
..there is the pecl extension memcache but you wont be able to install that if you are not root.
@ firepages : I might be able to swing the addition of a php-extension with the server admin. But this won't be the route to go, since memcache/d is only the connecting piece to a separate memchached server. No go, unfortunately.
@ tangoforce : your idea sounds very intriguing. I have never done anything like that, but I will use the might of the Internet to look for some examples and convert it to my use.
Thank you :)
This is the multi-client code I was talking of.. it's not multithreaded but supposedly deals with multiple connections via a loop. I have no idea quite what its doing (I could have studied it a but more but had other more pressing things) or whether its safe to use without muddling TCP connections but it is supposed to do what the author reckons (not that I can even remember whereI got this - probably PHP classes).
Additionally, you'd probably want to trigger it via an ajax call instead of via url otherwise your browser will wait for data indefinitely whereas an ajax call can disconnect after a timeout.
|All times are GMT +1. The time now is 04:52 AM.|
Powered by vBulletin®
Copyright ©2000 - 2013, Jelsoft Enterprises Ltd.