Discussion in 'Server/Website Chat' started by Spykodemon, 19 Dec 2009.
A huge amount of things are currently broken, probably won't have time to fix until tomorrow
@Dark, shits hitting the fan yo
- Forums (updated to XF 1.5 also)
- imgkk uploading
- Logging into sourcebans
- Pretty much all my old dodgy PHP scripts which used shit that is removed in PHP 7 - so sprays, homepage widgets, dsock, etc.
- Anything managed by DSM, ie the gameservers
- The Kirino bot
- My custom changes to Murmur ([[limit:x]] tag on channels)
On the bright side PHP 7 is a lot faster so I guess all this effort is worth it
Wait, you upgraded to PHP7? dayum, forums super speedy though.
Also, dat PHP 7 spaceship operator.
- DSock core
- Servers sidebar widget
- Mumble sidebar widget (will need rewrite or wait for PHP 7 ICE extension)
- HLstatsX web panel (horrible old code, needs further investigation for PHP 7)
- Murmur custom shit (don't really see the need)
- Kirino bot
The issue that initially crashed the system has happened another 3 times and appears to be connected to the gameservers, so they will be down for at least another day
- Gameservers, I think
- Remaining parts of DSock
- Kirino bot
- Most of MGE stats
- Mumble channel limit
- Mumble sidebar widget (holy fucking shit the amount of hoops I had to jump through)
annnnd another kernel crash and the tf2 servers are all running twice each and constantly restarting and sapping all the cpu
Apart from a hard disk that's been slowly failing for months (need to get off my ass and go to london to replace it), things actually seem ok now
Stare into the servers soul that'll fix it
Enterprise spec drives when?
It's the made in china 3tb
It's exhibiting the same symptoms as the Samsung drives that were in there ages ago - the intense vibration makes it slightly miss a write now and again, rendering the sector unreadable until it is overwritten with zeroes. According to smart it is not actually reallocating the sector, so its not a bad sector in the traditional sense
To be fair to it, it lasted 3.5 years 24/7 until any corruption started
Conclusion: only use ssds or enterprise hdds in anything with 5 digit rpm fans
Get on the phone wit the CEO of ITunes. Give them the ok to add the most powerful servers to handle this
Seagate (shivers) enterprise drives are meant to be one of the best according to LTT and backblaze
Literally running on fumes at this point
Definitely will have to replace the ssd too when I get round to doing the hard disk
Thank fuck SSDs don't just fail at random, still 10% rated life left, enough for a few months
"enough for a few months" famous last words.
Was down for 30 mins there for some preventative maintenance for the ssd, which consisted of turning the entire server off for a few mins - seems to have helped, no longer clocking up wear levelling count every 10 mins
1% ssd lifetime left
mfw 'a few months' turns out to be < 3 weeks
Separate names with a comma.