Cross-circuiting to "B"

Anyone who follows EVE is by now aware of CCP Zulu's Iterative development and what's happening in 2011 devblog, and the controversial threadnaught it spawned (1742 posts at the time of writing).

In the devblog, Zulu revealed how many people were working on EVE, and what they were working on. Unfortunately for him, EVE players are spreadsheet-whores, and after a few taps on their pocket-calculators, they realized that after all the people who were being dedicated to Incarna, the DUST linkup, EVE-Gate, Planetary Interaction, and Infrastructure (including the LagBusters) there was almost nobody left who could work on making changes to EVE that the players have asked for -- such as the 168 items in the current CSM backlog.

Now, the CSM is supposed to be a Stakeholder, and Stakeholders are supposed to be able to compete to get resources for backlog items they think are important. However, if there are few resources to be competed for, that doesn't really amount to much, does it?

Needless to say, the eruption of what I will diplomatically refer to as "concern about the future of EVE" was somewhat energetic, even by the volcanic standards Icelanders are inured to. CCP responses in the thread consisted of very limited, technical answers to questions, which predictably did nothing to stem the tide of player discontent.

Faced with a tsunami of extremely coherent rage, CCP Hellmar posted a note that basically said "we are listening and working very hard," to which the player response was "no you aren't, and no you aren't".

Oh my oh my, what is a mild-mannered CSM like your humble correspondent to do? If the dev-blog is taken at face value, then not many of the CSM's backlog items are going to get implemented in the near term, and possibly not for 18 months, or even longer.

Well, since the hordes of players telling them they were wrong clearly wasn't having any effect, I decided to take a different tack. I posted a two-part open letter to CCP Hellmar making a business case for why devoting resources to the concerns of existing players was smart business -- it was in effect an insurance policy against the possibility of delays and other business disruptions.

A couple of days later, I tried another tack. CCP Explorer had been making some very technical answers to questions about CCP's development process (which, of course, was totally avoiding the point of the whole thread). So I asked him, in what I freely admit was a somewhat teasing manner, for advice on how CSM could compete for the apparently limited developer resources available.

All in good fun, of course, and I hope I made a positive contribution to the discussion. But until I hear otherwise, I have to proceed on the assumption that the CSM won't be able to get significant attention to its issues.

So, how can CSM be effective given the circumstances we find ourselves in? Well, obviously we'll keep on lobbying CCP for more resources to be devoted to player concerns. But it has also occurred to me that if CSM can't get big things done, maybe we can slip some small things into the cracks here and there.

One of the most overlooked, but most promising things in the Summit minutes is the fact that CCP now has 3 people devoted to maintaining and improving the client UI, and improvements to the UI will be noticed by everyone. So I've started up a User Interface - Big Wins, Fan Favorites and Low Hanging Fruit proposal, with an associated forum thread to collect and crowdsource UI improvement suggestions.

My hope is that we'll be able to give the CCP UI team a prioritized list of UI improvements, big and small, that they can pick from and implement when they have some extra time.

Come and help us try to make some lemonade.

The CCP Metric System

It's clear that CCP (like many of their customers!) loves numbers, and bases a lot of their decision-making on various "metrics" of performance. You can see this in several places in the minutes; here are my favorites:

Speaking on behalf of CCP, Nathan disagreed strongly with the claim that CCP isn’t committed to excellence. He pointed out that CCP probably spends a bigger part of its income on development than most other large, established game companies. He stated that this is a clear sign of this commitment.

Memory usage on the server per user has decreased in the last 12 months.

CPU Per User on the server has remained constant for the last 6 months except for a short period in late January where it went up just a bit.

Server log lines have been decreasing significantly since February, meaning fewer errors are being generated by the server

Only 0.3–0.5% of sessions end in a dirty shutdown (client crash).

But the most interesting example of all came not in the minutes, but just yesterday, in a comment by Nathan that was quickly reposted on Scrapheap Challenge:

That's funny, considering Apocrypha, the example for quality, created the most abysmal technical debt we've had.

What Nathan is saying here is that Apocrypha, an expansion the players loved, created a lot of issues behind the scenes at CCP that required them to do a lot of code cleanup afterwards.

This statement illustrates why Nathan was so upset by the CSM's complaints about lack of excellence. He apparently defines excellence as "excellence of the code providing the EVE experience". Players, on the other hand, define excellence as "excellence of the EVE experience".

Nathan is a very smart guy, and no doubt he has very good metrics on EVE code excellence. But experience excellence metrics are inherently fuzzy and hard to quantify, so when CSM presented some to him (including the huge Assembly Hall threadnaught), his response was to dismiss them as invalid or irrelevant.

And indeed, quantifying experience excellence is difficult, not the least because excellence is in the eye of the beholder, and there are over 100,000 different pairs of eyes playing EVE. I am reminded of US Supreme Court Justice Stewart's famous observation about porn: "I can't define porn but I know it when I see it."

Nevertheless, attention to experience excellence is very important, for the simple reason that CCP is not selling software. They are not selling a game. They are selling an experience, the experience of living in New Eden. EVE is not a commodity like potato chips (or World of Warcraft, for that matter); it is an emotional, immersive experience -- in fact, it is the only emotional, immersive MMO available today. EVE's crucial innovation can be summed up by the oft-repeated comment that "EVE is not a game; it's a job." You play World of Warcraft to escape; you live in EVE. The brilliance of EVE's original design can be summed up very simply: EVE is a job so good that people are willing to pay to work there.

This is why difficult to quantify metrics like experience excellence are so vitally important, because they encapsulate the very essence of the difference between EVE and other games; it's unique selling proposition.

Alas, history is replete with examples where decision-makers have learned the hard lesson that focusing on the "easy" metrics, ignoring the broader context, and dismissing contradictory information, was the road to ruin.

The whole point of the CSM is to give CCP a broader context, an extra dimension to their metrics. So when CCP staff go into a meeting with the CSM and their reaction is "how can these people be so clueless?", they ought to consider the possibility that they might also need to buy a vowel.

Or at least an umlaut or two.

Shuttle Diplomacy

During the preparations for the Iceland summit, I took on the task of building a CSM presentation on lag.

I decided to approach the subject as a game-player. I expected that CCP would expect that CSM would come in with a bunch of half-baked lag-fix ideas (you know, like the ones I put in my election manifesto...), which they would then proceed to shoot down using their information advantage.

So I decided to try another approach. After consulting with the other CSMs, I built our presentation around several key points:

* Lag is one example of a class of problems for which game-design solutions are appropriate.

* Player expert focus groups can help CCP develop solutions that are both implementable (by them) and acceptable (to us).

* The CSM is in a unique position to mediate the interaction between player experts and CCP.

To my great regret, events conspired to brutally time-crunch the presentation; I was hoping to get 15-20 minutes, but CCP's presentation and the associated discussion ran long. In the end, I had only 5 minutes, which meant a rushed presentation and little discussion; CCP's response was basically "meh..."

But the next day, in the Low Sec session, CCP basically asked CSM to do what I wanted them to do for Lag:

CCP suggested that CSM work more with players to draft a well-supported vision for Low Sec, then submit a "foundation story" to CCP as a proposal that reflects that vision.

While what CCP asked for does not go to the level of detail I would want, it's a step along the right path -- as long as CCP is really serious about using, or at least strongly considering, the results.

IMHO, without a real commitment from CCP in this regard, it will be hard to convince the player experts to put in the time and hard work of coming up with a proposal.

Furthermore, for this process to actually work, CCP must provide a method for rapid iteration of player proposals, otherwise the players are just groping in the dark. This is where CSM, with our unique perspective -- we are players, but we are also under NDA -- can perform a vital role.

An example of this came in the 29 Issues session. Take a look at the Show damaged drones in drone bay item. Players want this functionality for several reasons, but one of the most important is that when fighting with drones, you want to be able to recall damaged drones and send out fresh ones -- but once drones are back in the drone bay, their state is unknown to the players.

In the space of 5 minutes, CCP and CSM iterated the proposal into something that could be implemented 100% on the client, with no added server or communications overhead, yet would allow players to launch undamaged drones in almost all circumstances (and, by the way, do the opposite -- launch the damaged drones for repair). It wasn't a perfect or elegant solution, but it provided extra function at very low cost.

Hmm... maybe (nudge nudge) someone should raise it as a proposal in the Assembly Hall (wink wink).

Now consider iteration between CCP and a player expert focus group. There is a trust problem; the players might say "you could do X", and CCP might come back and say "We can't do X", but not be willing to explain why. At which point, the players have no choice to take CCP at their word, and their ability to iterate is compromised; they're in the dark -- and darkness breeds mistrust.

But if CSM, with their magical NDA power, is in the middle as a trusted intermediary, shuttling ideas back and forth, then everything changes. Now CCP can tell CSM that "We can't do X because of NDA issues A, B and C", and CSM can tell the players "X has some problems, but your ideas Y and Z don't suffer from those problems; please explore them further."

Is CCP willing to make a real commitment to this kind of process? At the present time, probably not, for the simple reason that even if the focus groups are purely advisory, interacting with them will be perceived as a loss of control on their part.

But in a social sandbox game like EVE, as in life itself, control is ever an illusion -- even for the devs. Let's hope CCP has the vision to take a leap of faith.

The Reality Distortion Tank

One of the biggest eye-openers for me at the Iceland Summit was the Commitment to Excellence session.

The Commit to Excellence thread is probably the most supported thread in EVE-O Assembly Hall history. CSM put together a sober and respectful presentation explaining the concerns and perceptions of the player base.

The response from CCP was, to put it charitably, a wee bit defensive. You could almost see the FX of their X-Large C5-L Emergency Shield Overload Boosters being overheated as the CSM tried to break the tanks of their reality distortion fields.

I think one paragraph from the minutes speaks volumes:

Speaking on behalf of CCP, Nathan disagreed strongly with the claim that CCP isn’t committed to excellence. He pointed out that CCP probably spends a bigger part of its income on development than most other large, established game companies. He stated that this is a clear sign of this commitment.

That may be so, but it misses the point. The size of CCP's development budget is irrelevant because most of it is being spent on things that the current EVE player base does not want and does not care about. What counts to the current players is how much is being spent to improve the quality of their game experience.

Note also this line:

It was mentioned by CCP that the data does not seem to support that polished quality sells better than new features. This led to a discussion on the balance of customer acquisition through new features versus customer retention through quality and polish.

Several people have told me that EVE's long-term retention rate is quite high, which is in line with my personal impression. To put it another way, long-time EVE players are crack-whores, hopelessly addicted to the game we love.

So clearly, the key to long-term success is to generate more crack-whores, an insight that should surprise absolutely nobody.

However, the above line seems to indicate that CCP views the crack-whore production process as looking like this:

New Expansion -> Marketing Blitz -> Pulse of Noobs -> Some % become addicted -> long-term customer (high lifetime customer value)

However, there is an alternate path that can be significantly more effective:

Polish the Game -> Less Noobs -> Larger % become addicted -> more long term customers (with higher lifetime customer value)

The reason the lifetime customer value is higher is because the game is better, which means that the noobs stay longer, and the bitter old vets don't give up in despair.

As some of you may know, I run a website that helps people with search engine promotion. One of my favorite sayings is, "It's not the number of hits you get, but the number of sales you make".

In the context of EVE, doubling the noob retention rate is as good as doubling the number of incoming noobs -- and a lot cheaper. So focusing resources on polishing the game, and in particular those parts of the game noobs inhabit, is a big win.

For a final example of the reality distortion field, consider the Lag Issues and Resolution Status meeting, in which we are told:

Lag will always be an issue however EVE, but EVE is now, from a technical standpoint, in a better state than it has ever been.

Now read the technical justifications for this statement, which appear immediately thereafter.

All of them are true.

And all of them are meaningless from the player perspective.

Players don't care that "memory usage on the server per user has decreased in the last 12 months". That is as relevant to them as "the brand of coffee in CCP Coffeemachine has been improved".

All that matters to them is the end result -- does the game play better now than it did a year ago? If it turns out that higher-quality coffee resulted in more of the CSM's backlog getting attended to, then, and only then, would the upgrade of CCP Coffemachine be relevant.

So that there is no confusion, this posting is not an argument for "no new expansions". It's an argument for balance, and for the recognition of the fact that addressing the concerns of the current players is a profitable business model.

I contend that CCP is in the business of farming gamers, and sustainable agriculture is a better long-term strategy than slash-and-burn.

Bandwidth is a bitch

Now that the CSM summit minutes have been released, there is much to discuss. I predict an orgy of bloggery over the coming days, a vice in which I will happily indulge.

Consider the first two meetings, "CSM as Stakeholder" and "EVE Production". These illustrate the problems the CSM faces in having any real influence over CCP decisions regarding EVE.

If you look at how CCP development works, you quickly realize that it has a cognitive blind-spot. The problem is that after subtracting the resource demands for the big vision projects like DUST and Incarna, the marketing demands for big expansion features every 6 months, and the constant maintenance battle to keep the game itself running, there are few if any resources available for going back and fixing the mistakes of the past -- which is a big part of what CSM is supposed to be about.

Furthermore, in the great game of EVE Development, the noobs (that would be CSM) have an incredible disadvantage. We're not in Iceland on a day-to-day basis, and our voice is channeled through a representative.

So it's like playing EVE over a 110-baud modem via a proxy server on the Cassini space probe. We suffer from low bandwidth and huge ping-times. And you thought you have lag problems!

At present, the only way to compensate for that huge disadvantage is something that CCP is not willing to give -- dedicated developer resources for CSM. Given CSM's mission, this would be equivalent to explicitly giving renovations similar status to vision, marketing and maintenance.

Personally, however, I would suggest doing something more subtle, and with broader benefits for CCP. If I could wave my magic wand and make one change at the company, it would be this:

Implement the Google 80-20 rule.

Massive buff for dev morale - check.

Lots of resources for micro-improvements in EVE = happy players - check.

Big PR win - check.

Huge Human Resources win (eg: "Yeah, you have to move to Iceland, but on the other hand...") - check.

I wonder, does the company with the most innovative game in the history of MMOs have the vision to do this?

I could tell you, but then I'd have to kill myself

One of the things that makes EVE such an interesting game is that the social stew is always at a constant boil. The latest spicy ingredient to be added to the mix is the expulsion of Ankhesentapemkah from the CSM for an alleged NDA violation.

For someone in the games business, as Ankh is, such an allegation has significant career consequences. So it's no surprise that everyone involved is saying as little as possible; nobody wants to get hot-dropped by lawyers. Unfortunately, the lack of details has not stopped the conspiracy-crafters on the forums from spinning theories both sublime and ridiculous, and the "this just proves the CSM is worthless"-subcategory is annoyingly popular.

One frustrating aspect of being a CSM (and not just in this particular instance) is that the NDA often prevents you from posting a direct reply to a player question, and then your lack of specificity is interpreted in the worst possible way by one of the fine specimens of Homo Trollus that have found such verdant hunting grounds on the forums.

I'm not complaining about this too much, it's an occupational hazard of the job, but it's something to keep in mind.

Occasionally, however, you can have some fun with it. In one forum thread, the question arose:

BTW, is the NDA agreement you signed at CSM, under NDA too?

Which allowed me to indulge in a little whimsy:

As CSMs, we can neither confirm nor deny that the NDA is under NDA, nor can we confirm or deny that above confirmation or denial either confirms or denies any policy of the CSM or CCP.

Further, we neither confirm nor deny any rumors about what happened in Iceland, nor do we confirm or deny that we actually had any meetings in Iceland. In fact, we're unwilling to confirm that Iceland even EXISTS. And as for this internet thing you're using, it might just be a figment of your own demented imagination.

We do confirm that we do listen to what players want, however bizarre these desires might be. We regret doing this, however, since some of you are even stranger than we are!

In more serious news, the coming week should see the release of the much-awaited CSM Summit minutes, and I'll have a thing or two to say about them, you can be sure.

The Night of the Long Spoons

The first CSM5 summit was 3 days of intense effort, 8-10 hours a day of meetings, followed by discussions until late in the evening. As a noob, while I definitely had points I wanted to make, a key goal was to get some sense of the landscape in which I was operating. The returning CSMs all have developed relationships with people at CCP, whereas I was coming in more or less blind.

Several other CSMs have already posted accounts of the meeting that I broadly agree with, so rather than re-invent the wheel, I will point you to Ankhesentapemkah (Eva) and Mynxee (Carole)'s reports. When the official minutes come out, I'll make some specific comments, but for now, I'll indulge in some color commentary.

One of the more amusing developments that occurred after the summit was the hysterical overreaction by forum trolls about a trivial incident that occurred during one of the final sessions. How trivial? Well, I was sitting right next to one of the people involved and I barely noticed it at the time.

For me, the significance of this incident was that it was a classic example of what happens when partial information and preconceived notions collide on the Internet. Before the summit, I was bombarded with people telling me what to expect from my fellow CSMs, and I was very interested to see how these opinions reflected the ground-truth.

And the answer, of course, is not much. Which leads me to coin Woodhead's Second Law:

"The more controversial your forum personality, the less likely it is to reflect your real personality."

The fact is, when you're dealing face to face, you're not talking to Mynxee, you're talking to Carole. Similarly, Sokratesz is not Tim, Ankhesentapemkah is not Eva, Dierdra Vaal is not Valentijn, Korvin is not Andrey, TeaDaze is not Jason, Meissa Anunthiel is not Stephan, I am not Trebor, and Vuk Lau is not Vuk Lau.

Okay, Vuk Lau is Vuk Lau, but that's because he's from the Balkans and is the exception that proves the rule.

Forum trolls, of course, are the extreme example of this law -- if they acted in real life the way they do online, they'd be getting the crap kicked out of them on an hourly basis.

What I found on the ground in Iceland was that while I disagreed (sometimes strongly) with some of the opinions held by other CSMs, I had no problems whatsoever in working with them on matters of common interest, and they all behaved like reasonable people -- or at least, like reasonable Internet Spaceship Nerds. No doubt I annoyed them more than they annoyed me.

Just something to keep in mind when you read a forum post and go "WTF?! He's insane!" We are, after all, playing a roleplaying game.

PS: Here are some pictures of the summit and Iceland, and high-speed videos of Gullfoss Waterfall and Strokkur Geyser.

PPS: Woodhead's First Law is "The further you are from your server, the more likely it is to crash."