Edward Hunter, comScore
Summary
Some useful stats, and some interesting issues raised in terms of privacy and practicalities of gathering stats.
A lot of good advice on how to select a target market for an online game that’s better than “any hardcore RPG players” – both better as in more precise and usable, and also better as in bigger and worth more money.
LOTS of questions afterwards; read to the bottom to see them all.
Accuracy / Predictability
The same month that Blizzard announced 10 million players, comScore reported 9.8 million players.
You can’t categorize online gamers as any particular single genre.
March 2007: 771 million people browsing the web
216 million play online games
march 2007 – web is 840 million users
US online gamers outnumber all other individual countries
– 70 million US players; china 15 m, canada 12 m
– but 206 million non-US
(although europe: 82 million)
global online gamer audience has increased 27% 2007 – 2008
online gamers age: 25% 25-34 ,35-44, 45-54
10%: 18-24, 55-64
under-18? immeasurable
(Q: what do you mean by online games?)
A: I’ll get to that later. It’s an important point, thank you for spoiling it)
Defining Online Gamers: “Heavy”, “Medium”, and “Light”
heavy online gamers: online 57 hours a motn, 24% is online gaming
light – 36 hr / month – 1% of that is online gaming
heavy: leans towards social networking, gaming info sites
light – consumes more traditional entertainment and retail content
heavy favs – mmo, fps, rts
light favs – puzzle, card, strategy
lights rank £1 for visitors to retail computer hardware sits
heavy gamers rank it number 4, placing books and department stores above that
(this was obvious because he’d said that the light gamers spent all their time visiting websites)
light come in first for online music consumption – both paid and free
heavy come second
and medium come third
medium online gamers spend 7% of their time in XXX adult, whereas heavy dont have any.
medium – 8% genealogy
heavy – 9 percent fmaily
light spends 23% on portals, compared to 10% for heavy / medium (similar)
“online gamers” is far too broad, generic and meaningless a target to go after
The new way to target
- start by looking at the intensity of usage
- then narrow in on target by age demographic, region, and finally other oline and offline behaviours that apply to that market (like … what else do they do when online? what do they do with their free time? etc)
Players’ behaviours change a lot over time, so it’s necessary to track that way so you can change it during dev years in line with the changing themselves.
Try to match up the underlying behaviours of your players with the core fundamental elements of your game
Compare:
- search terms + volume
- purchasing patterns
- preferred online content types
- genre/platform/group-vs-solo preferences from games in general
Once you’ve established their behaviours, you now need to work out who they actually are – age/gender/location/net worth/etc. [ADAM: all the standard traditional marketing demographic stuff]
Then, you can work out where they hang out, and then market/advertise in those locations.
Build your games to be measured
Signal user actions:
- as exposure
- session start/session stop
- significant actions
Why?
Because the market needs to know independently measured numbers about who is using what – internal data that you keep private for business reasons are not valid as evidence when negotiating with advertisers etc
[ADAM: he recommends put in fine-grained metrics and making them PUBLIC to be used by advertisers. Of course, advertisers are currently refusing to use this data, and are being pushed through lowest-common-denominator services by the middlemen agencies. So … I’m not sure it’s as simple as he implies]
Analytics are also essential to be looked at side-by-side with reading the philosophical / essay post-mortems.
[ADAM: this is interesting. I’ve not seen such a subtly disguised yet selfish sales pitch for a while ;). Not that I mind, it was a good talk, but really it’s comScore et al who “need” this more than anyone else. Ironically, I think a lot of this stuff SHOULD be given away for less altruistic reasons than he tries to map out]
Link to download the slides
http://www.comscore.com/request/gamers.asp
Q: we have nothing like the Nielsens, no independent sources of how many people are playing concurrently. This has plagued me throughout my MMO dev career. Do you think that that’s on the way in the future? I’d argue its essential to make good investment decisions
People operate under the misconception that this can’t be measured. For most of the popular MMOs this is already being done (e.g. our own WoW analyst record).
We do it by looking at calls made out to the internet by the game, and use that to infer how many people etc – e.g. people clicking links from the game client and going to a webpage.
[ADAM: or, IMHO better, go the Thottbot / Curse route: write a custom client that sits there and scrapes]
Q: do I just make artificial calls to pages / sites – even ones that don’t exist – just to cause them to be tracked?
Yes, that’s how ISP’s do it, we aggregate it all etc.
Caveat: this is an emerging technology. Some phenomena to be aware of
How many people have deleted the cookies in their browser this month? (about 60% of audience)
Many people ONLY use cookies to track unique users.
ISP routers are dynamically changing the IP address even in-between successive requests while browing the same site
Think twice before believing the “uniques” that are reported by software or even by service providers (like Hitwise)
Comscore is based on observed behaviour, so we’re able to tie it all together even across cookie deletions, dynamic IP changes etc.
Q: what’s the memory footprint on your app?
We have some people who are desperate to be measured. For others, they opt in for a bunch of various incentives like free firewall software.
small numbers of megabytes of RAM
Q: how do you make sure that’s a representative sample of “all online gamers”
We have rigid rules over how many people we require to constitute a quorum for any given segment etc.
e.g. considered specifically advertising to second life users, but knew that couldn’t be scaled up because SL is too small [ADAM…not sure about that, I may have misunderstood]
Q: do you do it differently by country?
We do it based on a factor of their percentage of the population of the place where they live.
Our lowest threshold to consider something statistically relevant is lower in countries with lower populations.
[ADAM: essentially … no, they don’t change their methodology for different countries]
Q: does the app start and stop when you start and stop accessing the internet
We see everything they do offline as well as everything they do online.
The details of how are proprietary; we hold several patents on how this works.
We see:
- Every app install
- Every digital download
- Every time a media device is accessed on the system
I’m a huge privacy fan, I spent my first 5 of 6 years here working on carefully making sure we weren’t storing private info.
We’ve been audited to prove that we automatically destroy info and unrecognized info is locked up and excluded into a separate inaccessible store until we know what it is and know whether its safe to use or whether we have to destroy it.
Q: it’s a PC client. is this PC specific or does it include console?
It’s PC specific. We regularly survey 4200 people a month. We’re about 8 months away from measuring online console play and iPhone play.
For now, a combination of connections to console-focussed websites, in-basket purchases on Amazon and ebay (did you buy a console-only game?, etc).
[ADAM: I think this scared quite a few people. Yes, this is what happens when you allow someone else to read your internet URL traffic, let alone actually inspect the HTTP stream…]
Q: heavy gamers had a core thing of social networking, and yet it didn’t appear on the use. Why was it not?
It’s because it’s categorization vs actual visits to social networking sites. It’s the difference between a “stated behaviour” (asked) and an observed behaviour.
[ADAM maybe: I didnt really understand that]
When we define heavy/light/medium online gamers, we use a variety of measures, not just a single one like sheer hours. [ADAM: a multi-value metric]
Q: can you drill down to intensities within a given game? Within that game, can you say what a heavy gamer is doing as opposed to a light gamer?
Short answer is: we can, but I’m trying to focus more on what the industry can achieve here. Sure, we can do it as ComScore, but others may not be able to.
Q: what a developer knows is what genre they’re going for, and they want to know the stats for that genre. Why is that info not put first?
I think that’s chicken before the egg. I dont know anyone who starts by saying “what genre shall we develop for?”
Bottom line is you can do it anyway you want, but look at gamers as a level of engagement within a set of characteristics instead of looking at one individual characteristic at a time.
Q: how quickly can new stats be added to a single game and rolled out into the system
doesn’t require any change to comscore stuff. We don’t have definitions for all games, we add stuff to our rack of things to track when the amount of data coming from that game exceeds a threshold.
Q: how long do you keep the data?
6 months, or up to 15 months for some stuff [ADAM: I think, not sure again. It’s too hot in here. I’m dieing!]
Q: so you keep metadata about the games themselves so that we can do our own analysis about subtypes of genre etc, we might want to subdivide it a different way?
[ADAM: essentially: “not for you to use”. But he used a lot more words]
We collect 100 terabytes of info a month. We have the single largest commercial data warehouse that anyone will openly talk about having or using.
3 replies on “ION 08: Understanding the Online Gamer”
I’m really glad to see that folks out there were listening. Sure, as with many talks given at ION (how about that Sony keynote eh?), comScore is obviously in the marketplace to position products and services. However, it should be noted that without helping align the online gaming sector to the ways of the traditional online marketer, we stand little chance to succeed. In short, we win nothing if the industry doesn’t win.
So we dedicate a lot of time and passion to the business. We don’t have people in suits looking at gaming and going ‘harumph, that looks like a bit of good business there wouldnt you say old chap?”
No, rather, we get involved, and we start by involving people who eat, sleep and breathe online games.
When it is said that our methodology doesn’t change based on geography, thats true and it isn’t true. Our methodology for sample across geographies is consistent, but its scalable. It’s designed to accommodate geographies by population. So while we don’t have to change our approach for measuring across the globe, we do have to account for variables.
I want to clarify the social networking subject. Heavy Gamers don’t spend a lot of time ‘visiting’ sites like Facebook compared to the rest of the segments or theinternet in general. However, they do spend a lot of time engaged in social networking activities such as guilds, forums, etc.
With regards to the ‘metadata’ question, I think I answered it the best way we can; we collect what we can see, and this varies greatly from game to game. Some games consist of an executable that, once launched, simply becomes a ‘program’ we see running. We can categorize that program only so much, genre, name, categories, etc.
Conversely, some games create a lot of visible traffic that we can see and measure, but often times these data streams are so particular to the game that originates them that there is really no bucket in which to consolidate them. If there was, we would report on it and share it.
Regarding privacy, I spoke with a lot of people who attended, both before and after the lecture. I really didn’t get a sense of fear from them, but regardless, lets compare it to other measurements. We see what we observe at the source of the users requests. This is not wholly dissimilar to services that tap the ISP data, nor is it that different from social networking sites that utilize user information to create complex methods to reach them with advertising.
If there was a difference in how we go about what we do versus what many other measurement methods entail, it would be that we’re significantly more sensitive to privacy. We do capture a lot, but we’ve spent a lot of time and money ensuring the privacy of our panelists. We’re in business to provide aggregate measurement, its our bread & butter. The moment we go poking around in someones personal affairs is the moment that business ends. So, we’re not gonna do it.
In terms of the effectiveness of our measurement, sure, we could write plugins for games to measure and monitor various behaviors, but doing this online instead of observing it would make us prone to the same follies that all non-observed measurement have; people delete cookies, ISP’s recycle IP addresses.
I’ll comment more soon, right now I need coffee. Thanks so much for posting this!
Thanks for the extra info. I think it’s a good point you make that organizations like comScore are a lot more likely to behave well w.r.t. privacy issues, because their own revenues / business models are so sensitive to getting it right (or, perhaps more accurately “not getting it wrong” :)). Personally, I see nothing new to worry about here – but then most people really have no idea how much they should have been worried for the past 10 years already :).
This will be a big help to RPG players, maybe they could find players who are more in tune with the game they are producing