[Onno Kreuzinger]

Hi,

about 30 hours ago the server went into normal background manintanace, unfortunately some one also did start a cache cleanup on his "Gallery" DB [table g2_CacheMap].

Short info:since Debian 4.0 converts all DB's to innodb it thinks that are fne with that (in our case the larger one of two installed gallery instances...), the DB was unexpectetly innodb type. The "delete from" transaction was canceled by the backup tool after 4600 seconds. The backup failed due to mysqlhotcopy reporting (correctly) a improper shut down DB. This happend around 20h GMT.

This was a deadlock case, since db was unusable and backups got stuck having no realy current copy to restore from.

Unfortunately I could not get hold of a mysql crack (~23:30 local time)
so it took me some time to get the DB back up and running. At aprox 0:30 GMT (thats 2h off local time) the db was accessible again and with in 15 minutes the webservices had been back online. Do not ask me what I did wrong, I assume my only brave deed was to have dumped the data before messing aroung with ibd and log files... I am not used to a DB rolling back a 65K rows transaction totaly silent (45 minutes), the percent display gets updated after the 100% has been finished, and having no to chance to stop it.

At the moment the DB is running without innodb engine, all innodb databases have been converted to MYISAM (which at least has myisamchk and one file per table, not one file for everything but the log).

The backupcycle was continued, a current backup exists now, BUT we are facing minor problems in some web-applications. So expect another two or three short outages for some/all websites in the next days. As the problem is not yet investigated in depth, we (Me and Nova) can not tell if and how long maintance will be.

In case anyone knows how to reinitialize the innodb storage engine, please send me a PM ith good hints. And no, just deleting the log and db file and restarting mysql does not work at all.

Thanks for staying tuned.

Cheers Onno