hotlineage
09-18-2007, 03:57 AM
We all make predictions - but some people get paid for it. Just because of that we put the microscope on the world of videogame market research and analysis to see if their bar charts are worth the graph paper they're written on.
In many ways an analyst is in an unenviable position: it's a damned if they do, damned if they don't situation. While we might baulk at the Gordon Gecko salaries they get paid (though by all accounts these are exaggerated) everyone always remembers the things they get wrong, not the things they get right.
Let's take the industry's most outspoken crystal-ball gazer, Michael Pachter (analyst at Wedbush Morgan) as an example. Back in 2005 in the pages of New York Times he made the extraordinary claim that World of Warcraft was going through a bit of a fad. Over the previous ten months Blizzard had notched up over 3 million subscribers for its game but this didn't impress Pachter.
"I don't think there are four million people in the world who really want to play online games every month," he said while grinding some salt and pepper over his hat. "World of Warcraft is such an exception. I frankly think it's the buzz factor, and eventually it will come back to the mean, maybe a million subscribers." World of Warcraft now commands over nine million subscribers worldwide.
While it's true to say that WoW caught most industry pundits by surprise few would have predicted a reverse in subscriber numbers given its ability to woo and absorb. But we'll be coming back to Pachter a little later and revealing some of his other whoppers.
At this point we should note an important distinction between analysts and market research groups. While both attempt to predict future events analysts usually report directly to one company and generally use their knowledge of a key area to inform financial institutions. Market Research groups systematically gather, record and analyse info in an attempt to report on, and predict future trends.
Market research groups sell their information, graphs and predictions for a premium, but are they any more accurate? Well... in 1995 Forrester Research projected that the Internet would have a worldwide user base of 34.9 million people by 1998. Notice the precision. Not 35 million, but 34.9 million.
The real number ended up being in excess of 100 million users. Forrester also grossly underestimated the size of the electronic commerce marketplace. In 1995 Forrester predicted $2.3 billion. The real number turned out to be more than $13 billion.
Yes, it's easy to rake over old ground and nit-pick but what's startling is how much companies pay for this kind of information. We contacted one independent market research group in the US for a quote.
They said they'd be able to process a 15 question survey to their database of 500 clients (videogame enthusiasts in this case) 'tabulate' the information and get it back to us for a measly $15,000. Makes you want to start up your own company doesn't it?
One worrying feature of videogame analysis is how out of touch some of the commentators seem. Recently Richard Newboult, analyst at Panmure Gordon, made this extraordinary statement: "Last year 70-75 percent of all SCi's profits were generated by two hit games - Lara Croft's Tomb Raider Anniversary and Hitmen Bloody Buddy." Whoops!
And if that wasn't bad enough listen to this brilliant piece of advice from Nick Williams of GamerMetrics: "The Wii Balance Board, for example, could provide a completely fresh experience for all kinds of gamers. Could you imagine a Grand Theft Auto game where you actually run out of breath while trying to outrun the cops?" Probably not the kind of GTA game most fans would want to imagine.
Truth is, market research groups and analysts clearly do get things right and if publishers and financial institutions are willing to pay them money for their stone-casting then who are we to complain. But it's when they get things instinctively wrong that it really rankles. You get the impression a publisher would be better simply posting a message on a game forum if they want heartfelt feedback. And some do.
Most gamers we know saw the huge popularity of Nintendo's Wii coming over the horizon along time before the suits did. E3 2006 was a bit of a giveaway given that queues to play Nintendo's console were over four hours long compared to 30 minutes to experience Sony's titles.
Michael Pachter, however, called it a little differently. "In 2010 Sony's going to have 55 percent, Microsoft's going to have 35 percent and Nintendo 10 per cent," he said smearing mayonnaise on his shoe before adding: "With all three of those having plus or minus five per cent."
To be fair Pachter's prediction was in line with other forecasts. Indeed, Yankee Group had the split at 44 percent to Sony, 40 percent to Microsoft and a lowly 11 percent to Nintendo by 2011, and that was just a year ago. While there's obviously some time to go these forecasts look a little silly when you consider Nintendo's Wii currently commands 41.5 percent of the market share and shows few signs of easing up. Indeed this figure is 68 percent in the UK alone.
But it's easy to sit on the sidelines and scoff, putting your reputation on the line and prophesising the future requires guts - as baseball legend Yogi Berra once said: "making predictions is dangerous, particularly ones about the future." We'll leave the forecasting to the highly paid graph-botherers of the world and leave you with a few of our favourite ever IT prediction gaffes:
"Get your feet off my desk, get out of here, you stink, and we're not going to buy your product."
- Alleged comment said by the president of Atari when Steve Jobs was trying to drum up interest in his concept of the Mac.
"Dead, finished, gone."
- Alan Sugar's verdict on the iPod, Daily Telegraph 2005
"I think there's a world market for about five computers."
- Thomas J. Watson, chairman of the board of IBM
"There is no reason anyone would want a computer in their home."
- Ken Olson, president of Digital Equipment Corp. 1977
"640K ought to be enough for anybody."
- Bill Gates, 1981
In many ways an analyst is in an unenviable position: it's a damned if they do, damned if they don't situation. While we might baulk at the Gordon Gecko salaries they get paid (though by all accounts these are exaggerated) everyone always remembers the things they get wrong, not the things they get right.
Let's take the industry's most outspoken crystal-ball gazer, Michael Pachter (analyst at Wedbush Morgan) as an example. Back in 2005 in the pages of New York Times he made the extraordinary claim that World of Warcraft was going through a bit of a fad. Over the previous ten months Blizzard had notched up over 3 million subscribers for its game but this didn't impress Pachter.
"I don't think there are four million people in the world who really want to play online games every month," he said while grinding some salt and pepper over his hat. "World of Warcraft is such an exception. I frankly think it's the buzz factor, and eventually it will come back to the mean, maybe a million subscribers." World of Warcraft now commands over nine million subscribers worldwide.
While it's true to say that WoW caught most industry pundits by surprise few would have predicted a reverse in subscriber numbers given its ability to woo and absorb. But we'll be coming back to Pachter a little later and revealing some of his other whoppers.
At this point we should note an important distinction between analysts and market research groups. While both attempt to predict future events analysts usually report directly to one company and generally use their knowledge of a key area to inform financial institutions. Market Research groups systematically gather, record and analyse info in an attempt to report on, and predict future trends.
Market research groups sell their information, graphs and predictions for a premium, but are they any more accurate? Well... in 1995 Forrester Research projected that the Internet would have a worldwide user base of 34.9 million people by 1998. Notice the precision. Not 35 million, but 34.9 million.
The real number ended up being in excess of 100 million users. Forrester also grossly underestimated the size of the electronic commerce marketplace. In 1995 Forrester predicted $2.3 billion. The real number turned out to be more than $13 billion.
Yes, it's easy to rake over old ground and nit-pick but what's startling is how much companies pay for this kind of information. We contacted one independent market research group in the US for a quote.
They said they'd be able to process a 15 question survey to their database of 500 clients (videogame enthusiasts in this case) 'tabulate' the information and get it back to us for a measly $15,000. Makes you want to start up your own company doesn't it?
One worrying feature of videogame analysis is how out of touch some of the commentators seem. Recently Richard Newboult, analyst at Panmure Gordon, made this extraordinary statement: "Last year 70-75 percent of all SCi's profits were generated by two hit games - Lara Croft's Tomb Raider Anniversary and Hitmen Bloody Buddy." Whoops!
And if that wasn't bad enough listen to this brilliant piece of advice from Nick Williams of GamerMetrics: "The Wii Balance Board, for example, could provide a completely fresh experience for all kinds of gamers. Could you imagine a Grand Theft Auto game where you actually run out of breath while trying to outrun the cops?" Probably not the kind of GTA game most fans would want to imagine.
Truth is, market research groups and analysts clearly do get things right and if publishers and financial institutions are willing to pay them money for their stone-casting then who are we to complain. But it's when they get things instinctively wrong that it really rankles. You get the impression a publisher would be better simply posting a message on a game forum if they want heartfelt feedback. And some do.
Most gamers we know saw the huge popularity of Nintendo's Wii coming over the horizon along time before the suits did. E3 2006 was a bit of a giveaway given that queues to play Nintendo's console were over four hours long compared to 30 minutes to experience Sony's titles.
Michael Pachter, however, called it a little differently. "In 2010 Sony's going to have 55 percent, Microsoft's going to have 35 percent and Nintendo 10 per cent," he said smearing mayonnaise on his shoe before adding: "With all three of those having plus or minus five per cent."
To be fair Pachter's prediction was in line with other forecasts. Indeed, Yankee Group had the split at 44 percent to Sony, 40 percent to Microsoft and a lowly 11 percent to Nintendo by 2011, and that was just a year ago. While there's obviously some time to go these forecasts look a little silly when you consider Nintendo's Wii currently commands 41.5 percent of the market share and shows few signs of easing up. Indeed this figure is 68 percent in the UK alone.
But it's easy to sit on the sidelines and scoff, putting your reputation on the line and prophesising the future requires guts - as baseball legend Yogi Berra once said: "making predictions is dangerous, particularly ones about the future." We'll leave the forecasting to the highly paid graph-botherers of the world and leave you with a few of our favourite ever IT prediction gaffes:
"Get your feet off my desk, get out of here, you stink, and we're not going to buy your product."
- Alleged comment said by the president of Atari when Steve Jobs was trying to drum up interest in his concept of the Mac.
"Dead, finished, gone."
- Alan Sugar's verdict on the iPod, Daily Telegraph 2005
"I think there's a world market for about five computers."
- Thomas J. Watson, chairman of the board of IBM
"There is no reason anyone would want a computer in their home."
- Ken Olson, president of Digital Equipment Corp. 1977
"640K ought to be enough for anybody."
- Bill Gates, 1981