World Series Tickets Fiasco
October 23rd, 2007The Colorado Rockies made it to the World Series for the first time in their short franchise history. This meant that the World Series was coming to Coors Field and fans in Colorado had a chance at tickets to see the game right here in Denver. But, getting tickets wasn’t going to be easy — all sales were going to take place online and there would certainly be a huge demand for the limited supply. Fortunately, we had a plan.
Tickets went on sale at 10:00A on October 22, 2007. A group of us banded together to try and get tickets and many of us were using multiple browsers to try and get in. Unfortunately, nobody was getting in. The company that sells tickets for MLB online experienced huge demand and their systems seemed unable to keep up with it — shortly after tickets first went on sale, the Rockies suspended sales. After several delays, the Rockies held a press conference to indicate that “malicious activity” brought the servers down — it seemed possible, but unlikely in my mind. My feeling is that they just didn’t prepare for the demand. Anyway, they announced that ticket sales would resume on Tuesday, October 23, 2007 at noon.
We restarted our ticket operation on Tuesday. As things got underway, it appeared to many that it was just a repeat of Monday’s system issues. We plugged along slowly, then all of the sudden my connection to the site seemed to be blocked. Several other people in our group also reported, what appeared to be, blocked connections. I tried connecting in through a different provider and was able to get in without a problem — it looks like we were indeed being blocked. Rockies were back on announcing tickets were selling at the rate of 1,500 per minute (which means they should have sold out within 34 minutes, but it was 2:15P when they made the announcement). In the end, when the tickets did sell out (at 2:45P), we ended up with nothing — if we were going to go to the game, it would take jumping on StubHub to pay 3 - 5 times face value for tickets.
So what happened? Well, in my opinion, the ticket vendor failed to anticipate the load and their systems didn’t seem to do a good job throttling visitors into the checkout process. The same servers that seemed to act as “bouncers” were also handling tickets sales. The “bouncers” were essentially pages that refreshed every 120 seconds (it was 60 seconds on Monday, but they changed it to 120 seconds on Tuesday) and tried to connect in. It wasn’t immediately clear if there was a queue, or if it just happened to be when you hit the server and if there was room. However, later it appears there was no queue, it was randomly letting visitors into the ticket process without regard for how long you were waiting. The “malicious activity” was apparently just automated processes used by ticket brokers to grab up tickets. I’m not sure why that wasn’t anticipated, but it should have been easy enough to block at the firewall. On Tuesday, they resorted to blocking anything that looked remotely suspicious, which is why we were blocked and several companies I know of where they share a single Internet connection. My feeling is that they saw a repeat of Monday’s events and wanted to prevent that from happening, so they overreacted with their IP blocking to keep things manageable.
The ticket vendor has some work to do. This process should be much smoother and for a company the deals with ticket sales all the time, there is not excuse as to why they didn’t anticipate this demand. They need to re-think their system architecture and flow and be better prepared for next time — including understanding how to respond appropriately to automated processes or genuine “malicious activity”.