While it's a really nice feature to have on a day to day basis, having remote access to your servers and network gear during a crisis / emergency is vital. Especially considering that you or your staff may not be able to even get into the data center. These remote access cards or devices are typically not that expensive, and can really save your skin if it comes to it. The more things you can do remotely, the better off you are.
Of course all of this depends on the Internet connection at your data center being up and running, so you should start there. You clearly can't fix line problems (tree took out your fiber line from your RBOC), or problems at your ISP (rogue bread truck drove into the cross-connect room), but you can and must deal with your own problems. To make sure you have remote access to your routers, the best way is to go "old school": setup your console server to have a dial-up modem connected to it (with an unpublished number of course, and NOT a VoIP trunk that runs over your Internet connection). If you can't get into the modem, then you know that either the power is completely down or the phone lines are physically down. In either case, there's little you can do about it (unless your power being down is caused by your generator not working). Once you have access into your routers, you can then verify if you have a link to your ISP or not and start to isolate the problem.
I just mentioned it, but it's worth mentioning again. Get a console server for all of your devices that are manageable via serial connections. Just like a KVM gives you access to your servers keyboards and monitors, the serial console gives you the needed access to all your serial devices. Digi makes lots of them, and they are widely available on eBay if you don't want to buy a new one.
KVMs are great, but you need to make it remotely accessible so that you can fix any problems you have with servers. If you already have a KVM that you're happy with, you can easily make it an "IP KVM" by adding a simple "IP console" to it. For example, Aten sells a device that goes inline between your KVM console and your KVM, and provides access to the console via a web browser (look at the Aten KN1000).
One thing a KVM doesn't let you do is power cycle a server. The problem with the solution is it makes things a lot more complicated and unreliable. You can control the power to the servers remotely with a managed & switched power strip. The problem with these, which I've experienced first hand, is that they fail and servers turn off...typically at 2am as you're comfortably sleeping. These managed power strips have relays in them to control the power outlets, and those relays fail. The management buss in the device can fail too, causing nothing to turn on. It's not a great solution, but if you want to be able to remotely power cycle a server, this is unfortunately what you have to do...
...unless you have servers with remote management cards. These are cards that are typically add-ons to most servers that allow the server to be managed via a web browser. With these cards you can control the power of the server, check it's health, open a console, and lots more depending on the brand of server. An example of one of these cards is the Dell DRAC (Dell Remote Access Controller), which is an option on most Dell servers you can find. These cards are great even in non-emergencies, and they are worth their weight in gold!
None of this is any good if you don't have Internet access yourself. The best way to "guarantee" Internet access is to have a backup 4G connection either through a phone with a mobile hotspot feature or through a dedicated mobile hotspot / jetpack. This is much cheaper then you'd expect. For example Verizon Wireless' new "share everything" plan gives you free hotspot access on your phone, if it supports it. If not, you can add a dedicated hotspot to your "share everything" account for $10/month.
You also want to make sure that your VPN server/concentrator/connection into your data center is tested regularly, and that it too has redundant power supplies, etc.
Testing your emergency connections is fairly easy. Just go home (or somewhere remote, even your mobile hotspot in your office is "remote") and for all of your equipment, try to get to their consoles and verify that you can get to the remote power controls. (It's handy to have your complete inventory available while you are doing this.)
Nothing about remote access is particularly challenging, except possibly working out the different pinouts of serial-based devices, and nothing here is prohibitively expensive. If you make sure everything is controllable from everywhere, not only will your emergencies flow smoother, but your day-to-day operations will go nicer as well.
Part 3 is up next. We'll cover documenting your network and servers. This is something we all should do but none of us actually do.
As a side note, sorry this post was late. I've been behind writing lately, and Thanksgiving didn't help.
No comments:
Post a Comment