Business, Featured - Written by on Tuesday, August 4, 2009 22:12 - 2 Comments

Naumi Haque
The collaboration box score

A few weeks ago I asked what the NBA could teach us about measuring collaboration.  As a follow-up, I thought it might be neat to think about the elements that would make up someone’s collaboration box score.  The box score is telling because it’s an aggregate of performance, so it accounts for tradeoffs made by players during the game (e.g. shoot the ball for a point, or pass it for an assist) and demonstrates how they use the limited time they have on the court.  Brainstorming with others in the office, I came up with this initial list of box score categories, but I’d love to hear what other Wikinomics reader think:

Signal-to-noise ratio: I think the most visibly important metric – analogous to points on a basketball stat sheet – should be one that is focused on the value and quality of content you broadcast.  Using Twitter as an example, measure tweets versus re-tweets.  If your content is getting re-tweeted it’s safe to assume that it’s valuable (signal) and not noise. For blogs, the metric might be comments per post, indicating a compelling or timely argument worth discussion. Using online sentiment analysis tools, companies could add an additional layer of complexity to this stat by also measuring positive versus negative comments.  As a basic example of signal-to-noise, my ratio based on re-tweets/tweets is: 10/91 = 0.11.  For comments/blog posts it would be: 289/79 = 3.7.  Of course in a multi-channel world, the metrics get muddled.  If re-tweet is the new blog comment, how do you calculate the metric?

In/out ratio: How good of a curator are you of information?  We interviewed the company Cataphora a few weeks ago (recently profiled in BusinessWeek) – their software uses social network analysis to identify good content by how much it is shared and passed around and track document originators and curators to assess individual productivity.

Document “originator” stats: Building on the previous point, how much good content do you generate, where “good” is defined by the number of times your content is reused (this is similar to signal-to-noise, but attempts to quantify the strength of the signals). At a more granular level, the originator stat might also highlight certain areas of expertise – e.g. all good ideas related to robots originate from Alan.  A twist on this metric could be “conversation initiator stats,” which would track who kicks off popular conversations on ones that lead to valuable ideas that are implemented.

Responsiveness: This would be a fairly basic stat that looks at how quickly you respond to other people’s requests.  As part of this, you would also have to account for the number of requests an individual gets (which might actually be another stat around collaborative demand or reputation).

Feedback assessment: To temper responsiveness stats, you’d want to have something like the “assist” stat in basketball, where you only get credited for responses that lead to a positive outcome.  As an example, you could base it how much change occurs to documents as they flow through you.  This would have to use software that analyzes document content and tracks versioned documents (independent of filenames, which often change as they pass through different users – e.g. docs that come through me usually leave as filename_nh).

Sociometric factors: These are the intangible aspects of collaboration that don’t necessarily leave a digital trail that is easily measured.  I liken this to measuring an individual’s plus/minus in basketball – it’s based on how other people perform on their stats when they are around you.  Since there’s no “court time,” in enterprise collaboration, you could measure face time through badges, digital connections, or even video.  Using this type of reality mining, a company could also analyze things like the tone of conversations as well as emotional response in order to gauge the impact you have on the morale of those around you (without necessarily measuring content, which leads to privacy issues).  This stat might also highlight diminishing collaborative returns – if too many minuses start showing up, maybe you’re collaborating too much (or are not very good at collaborating).  A company could also develop a “starting line-up” for projects based on sociometric factors that show positive results from certain combinations of employees.



2 Comments

You can follow any responses to this entry through the RSS 2.0 feed. Responses are currently closed, but you can trackback from your own site.

Naumi Haque
Aug 10, 2009 10:58

Just a heads up – Laura C. just made me aware of a cool tool that enables some of the Twitter metrics I was talking about. Check out: http://www.twitalyzer.com

“Twitalyzer provides activities analysis of any Twitter user, based on social media success yardsticks. Its metrics include (a) Influence score, which is basically your popularity score on Twitter (b) signal-to-noise ratio (c) one’s propensity to ‘retweet’ or pass along others’ tweets (d) velocity – the rate one’s updates on Twitter and (e) clout – based on how many times one is cited in tweets. Its Time-based Analysis of Twitter Usage produces graphical representation of progression on various measures. Using Twitalyzer is a easy; just enter your Twitter ID and that’s it! It doesn’t require any password to use its service. Speed of analysis is depending on the size of your Followed and Followers lists.”

Wikinomics – Complexity and Wikinomics
Dec 18, 2009 10:12

[...] venues for consensus-building; and enterprise monitoring tools that map the informal networks and measure productivity within organizations. In fact, I’m sure there are many more connections to be made here, and [...]

Now available in paperback!
Don Tapscott and Anthony D. William's latest collaboration, Macrowikinomics: New Solutions for a Connected Planet. Learn more.

Business - Oct 5, 2010 12:00 - 0 Comments

DRM and us

More In Business


Entertainment - Aug 3, 2010 13:14 - 2 Comments

Want to see the future? Look to the games

More In Entertainment


Society - Aug 6, 2010 8:19 - 4 Comments

The Empire strikes a light

More In Society