Complex multivariate analysis

...using Dota

No, wait, don't go away i'm being serious.

Lately, i've been freaking out (and listening to that Jaydiohead mashup thing, which is really quite good) about evaluative procedures and judgments. The DoTA aspect of this is just another notch in the wall, so to speak, though it relates more specifically to issues of measurability and baseline testing.

First: A little (humourous) context. The spark that lit the tender was the one and only, the incomparable DFW. And more specifically, the 2005 Kenyon Commencement Address, which you should read, if you haven't already. Really, he says it better than I can:

It is about the real value of a real education, which has almost nothing to do with knowledge, and everything to do with simple awareness; awareness of what is so real and essential, so hidden in plain sight all around us, all the time, that we have to keep reminding ourselves over and over:

"This is water."

"This is water."

(if you're confused, there's are a cute parable involved, which he sets out in the beginning.) See, was it wrong of me to take the interpretation of this as, 'Yes of course, that's right. Consider all the variables, double-check your evidence, engage in some hardcore Bayesian inference and examination...right? That's what he meant when he said those things right?'*

Maybe. I doubt it, but I wouldn't put it past him. Either way, it inspired what it did, and I started to thinking about more general methods of evaluation. More grist for the mill: The article in the NYT that got press from one of the more populated areas of the sane interwebs, including a writeup by the Situationist.

What I was thinking with the NYT article was this: it's true, we do self-handicap all the time! But that was obvious. What I'm thinking of is how to obtain an honest (loaded I know) or atleast, empirically valid method of determining intelligence, aptitude or whatever else. If self-handicapping pushes your scores down, and self-affirmation drives scores up, is it possible to design an evaluative procedure that will give you an honest indication of your scores, one that isn't 'tainted' by self-handicapping or self-affirmation?

I don't know. It could well be that it doesn't matter.

Anyway, back to the point(ish). As i've said before many times, one of the reasons I find DoTA to be such a compelling game to play is that it's such a complex edifice. Even within the relatively narrow goals of winning a game, there's so many factors to consider! Hero choice, item choice, item builds, the skills of the various players, hero synergy, game modes and on and on and on. And lest you think that that seems like a short list, when there are 93 heroes (all of which possess a minimum of 4 unique abilities), with literally over a thousand items, with (usually) 5 players a side, 15+ game modes...you can see where i'm going here. This thing has an absolutely MASSIVE number of variables.

Let's try to answer that example question which I hinted at; What are the factors most responsible for winning the game? You can quickly intuit some responses, but the more interesting, more worthwhile, more correct thing to do would be to measure what factors are crucial in determining who wins, because this is how science works.

But, as i've mentioned, how the fuck do you measure such nebulous factors such as player ability? What is your control? What are your baseline measurements? Simply put, how the fuck do you maintain ceteris paribus?

This issue gets even murkier if you consider questions of game balance. Say you decide to increase one specific heroes damage dealing spell by 100. How do you find out how this affects the overall gameplay? How do you figure out the the untold number of synergistic and antagonistic effects with items and other heroes and game modes and so on?

Obviously, this matter is not entirely new. There exist a whole field of problems like these within the social sciences, known as "wicked problems". To quote teh wiki:

"Wicked problem" is a phrase used in social planning to describe a problem that is difficult or impossible to solve because of incomplete, contradictory, and changing requirements that are often difficult to recognize. Moreover, because of complex interdependencies, the effort to solve one aspect of a wicked problem may reveal or create other problems.

See the similarities? Continuing:

Rittel and Webber's (1973) formulation of wicked problems^[2] specifies ten characteristics, perhaps best considered in the context of social policy planning. According to Ritchey (2007)^[3], the ten characteristics are:

1. There is no definitive formulation of a wicked problem.
2. Wicked problems have no stopping rule.
3. Solutions to wicked problems are not true-or-false, but better or worse.
4. There is no immediate and no ultimate test of a solution to a wicked problem.
5. Every solution to a wicked problem is a "one-shot operation"; because there is no opportunity to learn by trial-and-error, every attempt counts significantly.
6. Wicked problems do not have an enumerable (or an exhaustively describable) set of potential solutions, nor is there a well-described set of permissible operations that may be incorporated into the plan.
7. Every wicked problem is essentially unique.
8. Every wicked problem can be considered to be a symptom of another problem.
9. The existence of a discrepancy representing a wicked problem can be explained in numerous ways. The choice of explanation determines the nature of the problem's resolution.
10. The planner has no right to be wrong (planners are liable for the consequences of the actions they generate).

Look me in the eye and say DoTA doesn't satisfy that.

Classically, a whole range of societal issues are traditionally considered wicked problems. Crime, poverty, healthcare, taxes, (un)employment**, climate change, abortion***, you name a societal hot-button issue, it's probably a wicked problem.

Why this is interesting in the realm of gaming is that wicked problems are often contrasted with 'tame', or 'simple problems'. Simple problems are those often found in mathematices and puzzle solving; a classic example would be a sudoku puzzle. It's a closed, definitionally-complete system; it has clear, explicit rules, which can be applied in an algorithmic manner to complete the problem. Simple problems are such that you can increase the scale of the system without necessarily increasing the complexity of the system; fundamentally, a 81 x 81 sudoku is no different in solution methods than a 9 x 9 one. You can have even simple problems that have enormous algorithmic complexity that are still closed; for all its complexity, chess is still a 'simple' problem.

The parallels with contemporary gaming should be obvious. Gaming, for all its lush, multi-faceted verisimilitude, is supposed to be a closed system. Despite the complexity of the algorithms used today, they're still (supposedly) algorithms, and those things have to end somewhere...right? One of the reasons I love gaming so much is that effectively, in the end they're 'just' complex puzzles, albeit with nicer graphics and more interesting gameplay. In the end, every game can be gamed; that is to say, you can obtain and follow a series of rules and steps that will allow you to achieve a win condition.

DoTA seems to defy that categorisation. To try to figure out why you won or lost, or even how to win or lose, seems to be, at best, intractable; at worst, impossible. In summary: DoTA is the game that defies being gamed. DoTA is the wicked problem, borne out of simple dynamics.

I don't actually know whether I adequately elucidated the questions I was having with evaluative procedures.**** I hope one of the major points got across: that a traditionally solvable, procedurally-evaluative field has now been transformed. It starts making me question a whole lotta other issues, I guess.

There. That was my rambly, multi-disciplinary answer to why I play DoTA. Now leave me alone, I gotta go do more...'research' DoTA.

*And you people wonder why I am no longer romantic.

**Tangential sidenote: Employment is an interesting subject to study within the context of psychology of schools of economic thought. It seems to be that to idealogues and bad economists, employment is basically seen a simple problem, solvable within (idealised) market conditions. The application of the supply-and-demand formula is apparently all that is needed; lower wages, and employers hire more. Raise wages, and employment falls. The Austrian School is probably the one most susceptible to this idea; I was wondering if the Chicago School would do it, but they seem to have too much sense to make such a mistake. It'd be interesting to see if there are other traditional economic problems that suffer from this perception issue.

***In fact, it was my research into the issue of abortion that I first ran into the idea of wicked problems.

****Actually, I think this is standard procedure for me. Whenever I don't know the answer or find it difficult to answer the question, I just ramble off into all these interesting tangents and sideshows and hope you get lost in the damn house of mirrors. And if you're a marker, hopefully the house that gives me high marks.

I have no idea why I just did this post. I think part of the motivation comes from some kind of latent guilt i'm feeling, now that people are going into honours or going overseas to learn or going to Melbourne for a friggin intensive Chemistry Olympiad training session. This is my one productive thing over the summer.

Actually, that's probably another post I should do sometime: How is it that nowadays, I have no idea what motivates me to do anything anymore. I used to know, or atleast I used to think I knew; and I even used to think I knew quite well what my motivations were. But now, these days, I barely have any idea why I do what I do. Crazy. But that'll be for another post, for another day...