Metrics for a Brave New Whirled

This afternoon I delivered a data-packed 30-minute lecture at the GDC Worlds in Motion Summit: Metrics for a Brave New Whirled. In many ways this was an update on the talk Andrew Chen and I did last year at the Virtual Goods Summit (see Andrew’s cool spreadsheet, more info on the talk in my last post). Here’s the talk description:

    Join Daniel James, CEO of Three Rings, on a journey into the mystic realm of MMO metrics: a whirlwind summary of the numbers behind an online games business. The in-flight meal; a delicious serving of fresh data from Whirled, Three Rings’ web-based virtual world of rooms, avatars, games and more, created and sold by players. In addition, a side of metrics from Puzzle Pirates, the hit indie MMO, will be served. The focus throughout will be on the key numbers that define the success of an online game: the user acquisition funnel, retention, referral activity, and revenue generation. By understanding these parameters future pilots will be better equipped to lead successful flights into new online game territory, and hopefully avoid ending their journey in a smouldering crater.

Without further ado, here’s the deck, some of which might actually make sense without me babbling along over the top of it:

You will notice that there’s a metric crapload of Whirled and Puzzle Pirates metrics in there. People often ask me, with a wary look such as you’d give a lunatic, ‘Why do you dish out your numbers like this?’ It’s a good question. I don’t think there’s a direct business benefit to me or Three Rings. I am secure in my infamy. There are possible downsides, but they are limited; if a competitor looks at my numbers and then goes on to execute better than us, I don’t think that has much to do with our numbers. They executed better, that’s the hard bit. Well done to them. If an investor is at all serious about investing in Three Rings, they are going to dig through all these numbers and more. Sure, many of our numbers are not that pretty, right now, but many have improved by an order of magnitude in the last four months. If we can improve them another order of magnitude, then I think investors would respond positively. If we can’t then, well, we’re not executing very well.

The upside is that more information circulates the startup and games community, and more people will share their data. This rising tide will raise all boats. If I can shame my fellows into parting with their data, we’ll all benefit. I don’t mind leading the charge, and if nobody follows, well… I’m used to having arrows in my back!

Besides, everyone wants to know the numbers, everybody loves data. I get thanked every time I dish out some real, concrete information at a conference. ‘Wow, you gave numbers!’ they say, eyes slightly bugging. I’ve paid for enough useless talking-head ‘waffle waffle oh noes i couldn’t tell you our real numbers blah blah but they’re awesome yadda yes i agree ur awesome too waffle’ panels. I’m sick of ’em. If you’re going to stand up in front of a couple of hundred people who’ve paid their hard-won dollars and time to listen to you, the least you can do is deliver value. Think of the children! Or at least, the other entrepreneurs, because we are all pioneers in the wild west of wacky MMO-land, and we need to circle our wagons, share buffalo meat and watch out for rattlesnakes.

As I acknowledged at the beginning of the talk, I am not an expert on metrics. I think of myself as firmly in the ‘Product’ side of some arbitrary spectrum between ‘Product’ (as in wacky, creative and often big ideas) and ‘Iteration’ (small incremental improvements, usually driven by analytics). However, Three Rings and I are trying hard to learn the skills of the analytic, metric and iteration-driven company, in the hope that we can make our crazy big ideas more successful. A lot of my learning has been driven by three excellent chaps whose blogs you are much better off reading than this one: Andrew Chen, Eric Ries and Dave McClure. Specifically this deck is strongly influenced by hilarious ‘AARRR’ rant (Updated). I am indebted to these guys, hats off to ’em.

I’ve also been reading Steve Blank‘s book ‘Four Steps to the Epiphany‘ which I wish I’d read a few years ago. Eric introduced me to this and the related concepts around his ‘Lean Startup’ ideas, and hats off for IMVU for pioneering execution in this style and now sharing with the rest of us.

I was asked in the Q&A about tools and tech for these kinds of analytics. I meant to mention these in the talk, but forgot — that’s what you get for putting your deck together at 1am on a Sunday! Over the last year or so we built a very comprehensive and scalable Metrics platform cunningly named ‘Panopticon’. It’s yet another example our nutty (see ‘Product person above’) proclivity for building ‘yet another startup’ inside of a project (Whirled) that exists inside a startup with four projects (Puzzle Pirates, Bang! Howdy, Whirled, and a sekrit one — and that’s not counting all the Whirled games!) Panopticon is very clever; it writes log data out from our servers to Amazon S3 and then EC2 instances grind the data and then we generate web reports from it. It’s very scalable to large datasets. It’s very complicated. It’s still not quite finished.

Don’t do this. Hack together something as simple as possible — use the database, or start a new logging database if you’re worried about load, run the reports on a slave, etc. Glue in some graphing software. Do the simplest thing possible to get at numbers, just like you should make the minimum viable product. Then iterate. If you run into scaling problems, hurrah! You have scaling problems. Solve them. In the meantime, someone is working on a startup to solve your problems, like my friend Hiten Shah at Kissmetrics . He wants to solve your problem, and promises me that he will be shipping ‘real soon now’. Someone called out Orbus Gameworks, too. Like Billing integration with payment gateways, which I mention briefly, you really don’t want to build this stuff. That’s another one of our startups-in-a-startup, and we’d be happier letting someone else like Twofish, Gambit or FatFooGoo (or many others some of whom I listed in the deck) concentrate on it.

Whilst I am here I may as well list my remaining GDC sessions: tomorrow, Tuesday, at the Indie Games Summit at 11.15am I’m on a panel called ‘The Indie Businessman’ where we will definitely not waffle or nod sagely at platitudes. On Wednesday, Thursday and Friday I’m reprising the legendary ‘Free to Play, Pay for Stuff’ roundtables with Matt Mihaly. These are awesomely good fun; Matt and I coach a group of very well informed people to dish out numbers, argue violently and pontificate on the future. Worth every penny! This is a pretty light year for me, which is nice.

For the engineering-oriented amongst you, do not miss my partner in crime Michael Bayne deliver his Whirled Tour of Whirled’s technical innards on Thursday at 1.30pm. It will be awesome.

Lastly, many folks have asked if we’re having our usual ultra-legendary GDC party this year. Sadly not; we’re taking a year off as a nod to the collapse of Western Civilisation and our need to preserve our stock of whiskey and baked beans (we’ll need the ammunition to fight off the drunken gatecrashers who just think they didn’t get an invite). Fear not! We’ll be back next year, partying in the rubble and building the New New Whirled Order.

Exile observations, China: Operators Selling Bots?

Recently I had to leave the country for the month of April to get a new visa. Fortunately I received said visa and all is well. During the month I visited Canada and then China and Seoul, Korea. I was hoping to post a long-winded travelogue, but didn’t finish it in a timely fashion and decided to trim things down for individual posts.

One tale I was told that boggled me was this; apparently a very large Chinese MMO operator (begins with S) has a very successful MMORPG that is overrun with botting (the practice of running third-party software robots to kill monsters and thereby gather up more goodies). Nothing new there, bots are common in many games. The Bot for this game is sold for ~$20. Fair enough, you say, underground Bot-hackers need to eat, too.

The incredible thing is that you can pay for this Bot using the official game-time cards sold by the operator. You can only pay for two things with the card; game-time (and/or in-game items, I believe their are switching models) and the Bot. Yes, the operator is essentially selling the robot. The implications for concurrent user numbers and revenue (where $20 is a lot more than the ~$2 monthly charge) are profound.

Somebody please tell me I’ve got all this wrong. It’ll help my poor eyes get a rest from boggling.

E3 MMORPG Lunacy

Each year I go to E3 and each year I am dismayed in various ways. I used to harbour resentment at the vast expense in creating noise and spectacle for essentially two important clients; the Walmart buyers, and the press, but I’ve relaxed on that front. If you need to throw $5M away to impress these important folks, so be it. In past years I would roll my eyes at yet more beat-em-ups or platformers or other expensive console games that all looked the same, of which only a few would have a snowball’s chance of success. Whatebbs. I confidently expect the console and retail business to be completely disrupted over the next ten years, to the point of anihilation or complete transformation for the dinosaurs walking the halls of E3.

This year I had a somewhat new experience; that of seeing 10+ brand-new MMORPGs that were all pretty much indistinguishable from one another. Each one of these projects has a budget of $10M+. Each one has incredible graphics with a swanky engine. The characters are beautifully modelled, high-fantasy and often anthropomorphic. Yes, Tiger-Man, you look very cool with that battleaxe. Each one has the same user interface of incomprehensible panels of tiny text and flaming icons. Although I’m sure they are all designed by smart folks to have their own intricate and unique details, each has apparently the same basic gameplay, down to the same ‘-20’ bling bling hit points that pop up over the poor hapless goblin/frogluk/lionwing/krog/alien lizard thing as you pummel it into glittering transluscent fade-away.

This is madness. For all their high production values, I feel that none of these new games looked as good as WoW. All the fanboys were still crowded around Burning Crusade. Maybe the overspill and churn from a 2.5M subscriber business (discounting Chinese subscribers, that are worth ~10% of US/EU/Korea) is worth scrapping over, but is it auspicious and wise to all bring exactly the same weapon to the fight?

I went through my requisite couple of years addiction to goblin-whacking on Essex MUD in 1982, and I’ve never managed to get excited about levelling much in the modern age of MMOs. Perhaps I am missing out. Perhaps I don’t understand the beauty and distinctiveness of each of these games — I will readily admit that I’m not as immersed in these games as I could be. But I maintain that this level of investment in what appears to be me-too design is foolish. Even Webzen’s WIKI, which looks kawaii and fun, is all about wacking clowns with swords.

Please, if you can get the check for $XX Million to make an gigantic MMO, do something different. Perhaps it’s not possible; often it seems that only fantasy games succeed, but at least you’ll be able to say that you tried. You might just hit it big and reach outside of the existing, heavily-exposed and saturated demographic to find a nice juicy new audience just ready to experience MMO fun.

There’s a thousand flowers still to bloom in the MMO space. Let us please start planting something other than whack-a-goblin RPGs.