I'm having a problem with one of my servers locking up, and I think what is happening is that it is getting stuck in an endless loop. CPU load is close to 100% and there are no error logs generated, and obviously ServerLoader doesn't detect the problem and restart the server.
This led me to think about possible ways that ServerLoader might detect this and restart the server if this occurs. Would it be possible for ServerLoader to monitor CPU load? If it could do this then what it could do is if CPU load stays at X% for Y number of queries, it would shut the server down and restart it. It would have to use the shutdown methods you talked about in the other thread about locked up servers, because shutting down a looping program can be difficult.
X and Y could be configurable, so, for example, I could set it that if CPU load stays above 96% for 10 queries in a row, ServerLoader will shut the server down and restart it. Chances are if the server is experiencing that kind of load for that long, there is some kind of problem, so a restart is probably definitely in order, even if the server isn't technically caught in a loop.
This kind of protection could be extended to detecting problems with memory leaks. If memory usage climbs above XMB for Y queries in a row, the server will shut down and restart. So I could configure it that if memory usage climbs above 256MB for 10 queries in a row, the server will shut down and restart.
Don't know how difficult these things would be to code, but they would certainly be useful I think. (Easy for me to make suggestions when someone else is doing the work.
)