Malderi
11-03-2006, 09:44 PM
The title's pretty descriptive... I just spent the day today at a shift in the American Mission Control Center (MCC) for the International Space Station (ISS). I won't go into the details for how I got to shadow a flight controller for the day, but suffice it to say, it's pretty damn cool. Since I know there's some hardcore space geeks here, and some others that may just be interested in day-to-day operations, I figured I'd post a thread about it. Note: Because NASA is acronym-heavy, so will my post. If the portion of your brain that deals with acronyms is missing, reading the rest of my post may cause your brain to asplode. Sorry in advance.
First of all, I shadowed a controller at the ODIN console. I don't remember exactly what it stands for (who cares what they stand for as long as you know what they mean). ODIN deals with Command & Control Software, as well as the computer systems and data networks onboard ISS, including the PCS (Portable Computer System) laptops on board. ODIN is one of maybe 12-15 controller stations in MCC. Note that this is just the "front-room" controllers... all major controllers (i.e., those monitoring major systems, like ODIN, and not specialty controllers like ROBO which deals with robotics) have "back-rooms" specifically for them. The back-rooms are typically another 1-3 people per controller. ODIN's back-room person is called RAVEN. Somebody in the ISS program loves Norse mythology, because another position is THOR, there's an ATLAS, etc.
On to my day...
I picked a pretty cool day to be shadowing ODIN. (All controllers are called by the name of their station while on duty, so I'll refer to the controller I shadowed by the name "ODIN" appropriately.) One of the computer systems on board the station was being switched out between a primary & a backup, I think because the primary (at the time) was acting up slightly. For a little background, there are MANY computer systems onboard ISS, ODIN monitors maybe 30-40 systems separated into three tiers. We were just swapping out one (electronically, not physically) in the second tier. This was about two hours worth of specific procedures that had to be triple-checked by RAVEN and THOR, who also was part of the changeout procedure. I wish I could explain exactly what happened, but since I barely understood any of it, I really couldn't. Anyway, there was a lot of chatter on the various comm loops (I had my own headset plugged in!). The changeout went well, and it was all transparent to the crew.
Unlike Shuttle, where MCC basically commands the crew to flip switches when they need something done, ISS is too big and complicated for that. A lot of ISS operations are controlled totally from the ground, although the crew onboard can do just about everything themselves (although there's three of them and 15 down here, so MCC ends up doing most of it.) This also causes an interesting paradigm shift: In Shuttle operations, all of MCC is basically on the same page: supporting whatever the Shuttle crew is doing at the time. However, on ISS, there's twenty different things going on at once, of which maybe five the crew is even aware of, much less involved in. Michael Lopez-Alegria, the current ISS commander, was swapping out and cleaning smoke detector units while we did the computer changeout. His only clue that we were doing something was our communication "hey by the way, we're switching out the computer systems now, so we've suppressed a whole bunch of Caution & Warning lights for it... but we may have missed one so if you see anything in the next 30 seconds, don't worry about". Several flight controllers remarked about how different the operations are.
Anyway, the second major thing that happened with ODIN was only partially scheduled. One of the PCS laptops in the U.S. lab was scheduled to be rebooted, this happens once a month per laptops. So, one of the crew shuts it down... and, voila, it doesn't turn back on! Yes, even astronauts have six-year-old IBM thinkpads running a modified Red Hat Linux on orbit that don't turn on when you tell them to. He reported seeing several lights flash on, but the monitor didn't show anything. Since PCS is ODIN's responsibility, we got to do a little troubleshooting. We booted up a PCS we have in MCC for reference and saw which lights came on. One of the ones that flickered on a lot during boot up was the hard drive access light, but the crew didn't report that light turning on... so, we didn't get far in trouble-shooting it, but they did have the power lights on, but no hard drive light. So, that sucks. They pulled a spare (they have plenty) out of the Quest airlock, which functions as ISS's storage closet when they're not doing EVA's. Put it in the same place, turned it on, voila, all's fine up in space.
There were all sorts of VIP tours going through MCC all the time, plus there's a bunch of cameras. If you happened to be watching NASA T.V. between 10 and 11 or 2 to 3, you would've seen me... I was the guy with the blue shirt sitting on a console near the bottom of the screen in the wide-view shots. On the off-chance that anyone recorded that (haha yeah I know, nobody would), would you mind sending me a copy? :-)
Anyway, if any of ya'll have questions about what ISS MCC is like, feel free to ask. For reference, here's a good photo of it, courtesy of NASA:
http://www.nasa.gov/images/content/160446main_jsc2006e43860_high.jpg
You can read "ODIN" on the full-res picture. It's the console second from the left, in the second row from the front. Yes, all consoles have at least five flat-screen monitors (and it's still not enough, they're still juggling windows a lot), and some have 7 or more. It's pretty awesome.
First of all, I shadowed a controller at the ODIN console. I don't remember exactly what it stands for (who cares what they stand for as long as you know what they mean). ODIN deals with Command & Control Software, as well as the computer systems and data networks onboard ISS, including the PCS (Portable Computer System) laptops on board. ODIN is one of maybe 12-15 controller stations in MCC. Note that this is just the "front-room" controllers... all major controllers (i.e., those monitoring major systems, like ODIN, and not specialty controllers like ROBO which deals with robotics) have "back-rooms" specifically for them. The back-rooms are typically another 1-3 people per controller. ODIN's back-room person is called RAVEN. Somebody in the ISS program loves Norse mythology, because another position is THOR, there's an ATLAS, etc.
On to my day...
I picked a pretty cool day to be shadowing ODIN. (All controllers are called by the name of their station while on duty, so I'll refer to the controller I shadowed by the name "ODIN" appropriately.) One of the computer systems on board the station was being switched out between a primary & a backup, I think because the primary (at the time) was acting up slightly. For a little background, there are MANY computer systems onboard ISS, ODIN monitors maybe 30-40 systems separated into three tiers. We were just swapping out one (electronically, not physically) in the second tier. This was about two hours worth of specific procedures that had to be triple-checked by RAVEN and THOR, who also was part of the changeout procedure. I wish I could explain exactly what happened, but since I barely understood any of it, I really couldn't. Anyway, there was a lot of chatter on the various comm loops (I had my own headset plugged in!). The changeout went well, and it was all transparent to the crew.
Unlike Shuttle, where MCC basically commands the crew to flip switches when they need something done, ISS is too big and complicated for that. A lot of ISS operations are controlled totally from the ground, although the crew onboard can do just about everything themselves (although there's three of them and 15 down here, so MCC ends up doing most of it.) This also causes an interesting paradigm shift: In Shuttle operations, all of MCC is basically on the same page: supporting whatever the Shuttle crew is doing at the time. However, on ISS, there's twenty different things going on at once, of which maybe five the crew is even aware of, much less involved in. Michael Lopez-Alegria, the current ISS commander, was swapping out and cleaning smoke detector units while we did the computer changeout. His only clue that we were doing something was our communication "hey by the way, we're switching out the computer systems now, so we've suppressed a whole bunch of Caution & Warning lights for it... but we may have missed one so if you see anything in the next 30 seconds, don't worry about". Several flight controllers remarked about how different the operations are.
Anyway, the second major thing that happened with ODIN was only partially scheduled. One of the PCS laptops in the U.S. lab was scheduled to be rebooted, this happens once a month per laptops. So, one of the crew shuts it down... and, voila, it doesn't turn back on! Yes, even astronauts have six-year-old IBM thinkpads running a modified Red Hat Linux on orbit that don't turn on when you tell them to. He reported seeing several lights flash on, but the monitor didn't show anything. Since PCS is ODIN's responsibility, we got to do a little troubleshooting. We booted up a PCS we have in MCC for reference and saw which lights came on. One of the ones that flickered on a lot during boot up was the hard drive access light, but the crew didn't report that light turning on... so, we didn't get far in trouble-shooting it, but they did have the power lights on, but no hard drive light. So, that sucks. They pulled a spare (they have plenty) out of the Quest airlock, which functions as ISS's storage closet when they're not doing EVA's. Put it in the same place, turned it on, voila, all's fine up in space.
There were all sorts of VIP tours going through MCC all the time, plus there's a bunch of cameras. If you happened to be watching NASA T.V. between 10 and 11 or 2 to 3, you would've seen me... I was the guy with the blue shirt sitting on a console near the bottom of the screen in the wide-view shots. On the off-chance that anyone recorded that (haha yeah I know, nobody would), would you mind sending me a copy? :-)
Anyway, if any of ya'll have questions about what ISS MCC is like, feel free to ask. For reference, here's a good photo of it, courtesy of NASA:
http://www.nasa.gov/images/content/160446main_jsc2006e43860_high.jpg
You can read "ODIN" on the full-res picture. It's the console second from the left, in the second row from the front. Yes, all consoles have at least five flat-screen monitors (and it's still not enough, they're still juggling windows a lot), and some have 7 or more. It's pretty awesome.