Server failure

tl;dr this is a backup server, main server has exploded

And anyone looking to play on our TF2 servers: you'll have to wait until next week Mon 4th Tuesday at the earliest a while.

Not a clue what is wrong with the dedi, but something is very much on the way out. It's behaving as if a RAM stick was failing, but it passed 2 hours of memtest with no errors. Basically it's fucked and will not boot anything without kernel panics, won't respond to the BIOS setup key (but does respond to other keys at bios level such as boot menu) and even randomly freezes on the GRUB menu.

I've restored a backup of the forum and mumble that hopefully isn't too far out of date on to a secondary server. That's what you are seeing now. Of course this will be the 'master' copy, and when the main server is eventually fixed, it will be transferred over.


The following services are available:

• Mumble
• DNS
• Email
• gamingmasters.org forum
• imgkk.com existing images only
• Various internal dependencies such as mysql, memcached, etc

The following services are unavailable (non-exhaustive list):

• Every other website hosted on the server
  
• FTP
• DSock network (incl Kirino etc)
• All game servers
• Prophunt Stats (not enough horsepower available to handle the database)
• HLstatsX (again lack of power)
• Sourcebans (pointless)

Please note that the forum will likely be pretty slow, as it's running on (and I shit you not) a 1.2ghz single core celeron.

 

NOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO!!!

 

I haven't decided what to do yet, and any advice/help would be appreciated.

As far as I can see, these are the options:

1. Pay for courier to ship server back to me, chalk it up to experience, give any working parts to the dog, rent a server elsewhere and upload all data. ETA: 2-5 days + another IP change. Cost: ~£50/month indefinitely + £40 shipping
   
2. Pay for courier to ship server back to me, diagnose problem by gutting blue's pc for parts, replace faulty part(s) with server grade equivalent, pay for courier to ship server to datacenter. ETA: ~14 days. Cost: £60-100 shipping, £50-200 parts, £300 optional KVM
   
3. Buy new CPU, RAM, motherboard and fit them somehow. ETA: 3-14 days. Cost: £500 parts (incl IPMI so no need for KVM) + travel costs if applicable
4. Order a motherfucking prebuilt server directly to the datacenter and get the disks moved over. ETA: 7-14 days. Cost: £1000-1500.
   
5. Convince Justin to let me put my disks in one of the old spare Supermicro servers. ETA: as soon as I finish using it for other purposes (aka 1-2 weeks). Cost: £0. Likely would have to sacrifice a large number of gameservers and I'd be amazed if anything more than 24 slots per server ran smoothly.

Annoyingly there is a fair bit of data that is not remotely backed up, not really sure how to handle that.

 

Well there is an option, but it's a rough one.

Some servers do offer Premium Accounts for a small, monthly fee.

Something like £5/monthly for access to the Propmenu. You could offer a 6 to 12 month discount.

Just a thought.

 

Added a couple more options above

My brain is completely full of fuck, this is a nightmare

 

Well, like Arrk just said to me in Mumble...the most expensive option looks to be the most reliable.

We could have a Support GM Fund Drive to pay the £1000-1500.

 

mhm these are exactly my thoughts

i'm just not sure if it would be better to travel to london myself or get a trustworthy GM member to perform the upgrades/swaps, rather than shipping it via courier.


courier pros:

- More time to work at it and diagnose the problem
- Potentially less cost as only the faulty part need be replaced

courier cons:

- Ridiculously expensive to fully insure the shipping. Was about £50 originally for one-way shipping with only £500 insurance
- Not sure what to do about the hard disks. If they are lost or damaged, shit will seriously hit the fan
- It might be an incredibly simple issue to fix. I've seen some evidence of a similar issue (where RAM is ruled out) being resolved by simply resetting the CMOS, something which I'm still trying to manage remotely. Running absolutely 24/7 for >2 years then shutting down for 12 hours may have confused things.
- Have to provide packaging materials

 

Would be inclined to agree with Dan. Option 1 seems silly, given the huge opportunity of free co-lo. Option 3 appears to be option 2 but you fly to London instead of getting the dedi shipped over. Option 4 while convenient, is crazy expensive - you would end up paying the majority of it yourself. Option 5 I would consider a last resort if option 2 doesn't work out.

As for courier vs flight, I would work out the costs associated with each.

 

Option 5 isn't all that bad, especially if I get both of them.

Specs are Xeon E5506 (2.13GHz Nehalem series) and 24GB RAM, each

 

if only server grade parts were that easy to buy in the uk

have to go to wanker trade sellers like lambda-tek

 

Priced out a few components:


Option A:

Decent server-grade motherboard: £160-200
IPMI/IP-KVM addon board: £30-50
CPU: £210+ (210 for same ghz as current, probably faster though because haswell)
RAM: £150 for 16GB, £260+ for 32GB

Option B:

New shite motherboard: £60-100
optional Lantronix SpiderDuo: £230
optional Lantronix single PCU: £45-80



Option A would be a nice upgrade and is overall the more reliable option
Option B is the pussy way out

 

So is there anyone local to London that knows how to build a PC?

 

Flights are pretty fucking reasonable actually