[quote]Thousands of air passengers around the world have been left stranded after a power cut forced the US airline Delta to suspend flights.
The incident caused delays across the US and in Japan, Italy and the UK.
Airport check-in systems, passenger advisory screens, the airline's website and smartphone apps were affected by the systems failure on Monday.
Flights resumed six hours later but Delta warned of continuing delays as a backlog of passengers was cleared.
The airline suspended or cancelled dozens of departures early on Monday, with airport agents forced to write out boarding passes by hand.
By Monday evening, Delta said more than 740 flights had been cancelled but its computer systems were working again.
Delta offered passengers refunds and $200 (£153) vouchers while CEO Ed Bastian apologised for the inconvenience in a video message.
"The Delta team is working very hard to restore [services] and get the system back [operating] as quickly as possible," he said.
The overnight power failure took place in Atlanta, near Delta's headquarters, the company said, causing computer systems to crash.[/quote]
[URL="http://www.bbc.com/news/world-us-canada-37007908"]Source[/URL]
Apparently they had a power outage with the building that houses their servers and their UPS' all fried. A server room for a billion dollar company that relies on that infrastructure only has one redundancy? Yea ok.
[QUOTE=Cyke Lon bee;50851965][URL="http://www.bbc.com/news/world-us-canada-37007908"]Source[/URL]
Apparently they had a power outage with the building that houses their servers and their UPS' all fried. A server room for a billion dollar company that relies on that infrastructure only has one redundancy? Yea ok.[/QUOTE]
There is a lot of technology that goes behind transferring power during an outage in a data-center and it's never going to be risk free when moving from grid to UPS and generator. For all we know, an employee performing a weekly/monthly test followed the procedure wrong.
With that said, being a billion dollar company they should have offsite backup processing but in some scenarios you might consider that overkill.
[QUOTE=Smoot;50851987]There is a lot of technology that goes behind transferring power during an outage in a data-center and it's never going to be risk free when moving from grid to UPS and generator. For all we know, an employee performing a weekly/monthly test followed the procedure wrong.
With that said, being a billion dollar company they should have offsite backup processing but in some scenarios you might consider that overkill.[/QUOTE]
Considering this is a billion dollar company and their whole company relies on that infrastructure, it wouldn't be overkill. The company movies tens of thousands of people on flying pieces of aluminum every day, and if those people get stranded because of a power outage, then thats a loss for the company in the millions. When an entire company is propped up by digital logistics data, no amount of redundancies is enough when it comes to their data centers and servers.
Not to mention this has pretty much ruined Delta's reputation. A lot of the people who were stranded because of this probably won't ever fly Delta again.
I'd hate to be the CTO who didn't okay a request for offsite redundancy costs.
I'll be working for these guys in the next year or two as a system admin.
There always behind in the technology department and rather slow, last time I visited one of there hangers they were still using XP for most of there workstations. They actually downgraded most of Northwests systems when they merged and took over here. The employees joke about their stupidity when they call Atlanta, they wear a tin foil hat (which is kept next to the direct line to them) because talking to them makes it feel like your brain is being drained.
I got stranded with my girlfriend for another 4 days. Can't complain.
[QUOTE=Smoot;50851987]There is a lot of technology that goes behind transferring power during an outage in a data-center and it's never going to be risk free when moving from grid to UPS and generator. For all we know, an employee performing a weekly/monthly test followed the procedure wrong.
With that said, being a billion dollar company they should have offsite backup processing but in some scenarios you might consider that overkill.[/QUOTE]
Companies whose software and servers aren't the product think of these things as cost centers. They skimp out whenever they can
[QUOTE=Cyke Lon bee;50852027]Considering this is a billion dollar company and their whole company relies on that infrastructure, it wouldn't be overkill. The company movies tens of thousands of people on flying pieces of aluminum every day, and if those people get stranded because of a power outage, then thats a loss for the company in the millions. When an entire company is propped up by digital logistics data, no amount of redundancies is enough when it comes to their data centers and servers.
Not to mention this has pretty much ruined Delta's reputation. A lot of the people who were stranded because of this probably won't ever fly Delta again.[/QUOTE]
This seems like common sense to someone in IT, but to the bean counters and execs that actually have the final say on those decisions, its not immediately obvious. If it doesn't help their bottom line this year, they will probably decline. After all, what's the likelihood of that [i]ever[/i] happening? Why spend millions on that incredibly unlikely scenario?
There's a negative perception of IT in many companies. Its a catch 22. "Everything works fine, what do we pay you for?" -or- "Everything is broken, what do we pay you for?"
You try your best to stay in the former situation. You just hope that you made it clear that this was a possibility and CYA'd so your not at fault when they try to pin the blame on you.
Sorry, you need to Log In to post a reply to this thread.
