Home > Facebook, Fan-Generated Content, Social Media, Social Networking > Google Sidewiki Experiment Results

Google Sidewiki Experiment Results

When Google’s Sidewiki was announced, I began a small experiment – posting factual but mildly-controversial entries on two websites – Facebook.com and Foxnews.com.   On Facebook, I talked about the ease of use of the site but also about the site’s repeated controversies over protecting the privacy of its users.  On Fox News, I deliberately avoided the value judgments about whether or not the organization is biased (like most of the other entries) and instead cited two studies that demonstrate that Fox News viewers tend to be the most poorly-informed of all of the major news networks.

After waiting for a week, these are the results thus far:

Facebook.com:  Currently my entry is the first entry listed for Facebook, likely a result of having garnered six “useful” votes.  Interesting about Facebook’s Sidewiki entries is that many of them (13 total) are from non-English speakers (Spanish, Italian, French).  Facebook Sidewiki entries have many fewer “yes” or “no” votes on entries when compared to Foxnews.com (likely as a result of the fact that it’s less controversial than an entity such as Fox News).

  • Total Entries on Site:  62
  • Positive / Negative / Neutral Ratio:  18 / 6 / 16
  • Ranking of my Entry: Useful? Yes (6) No (0)
  • Spam Entries: 9 (14.5%)

Fox News:  My entry appears first in the listing Fox News, likely a result of being one of the only entries that has more “yes” votes for being useful (the other entry with more positive votes is an entry that is overtly negative, as are the majority of the other entries).  Interestingly, some discussions are taking place as users comment on each other’s entries; one user named “Joe G” even responded to my post (incorrectly attempting to debunk the 2003 PIPA study I cited, not realizing that the study was conducted back in 2003 before any conventional WMDs were discovered in Iraq).

  • Total Entries on Site: 25
  • Positive / Negative / Neutral Ratio:   9 / 10 / 6
  • Ranking of my Entry:  Useful? Yes (25) No (19)
  • Spam Entries: 1 (4%)

Conclusions:

It’s likely too early to tell how this will play out given that there are only a very limited number of users (and thusly a very skewed sample frame) of users who have jumped through the hoops necessary to start using Sidewiki.  The dynamic will likely change if it catches on and the adoption rate increases.  Thus far it does not appear that censorship is necessarily a problem – however this could be a result of the hosting sites’ inability to influence Facebook or their lack of awareness about their Sidewiki entries.

There are some interesting observations to be made:

  • Spam: For high-traffic sites like Facebook, spam is already proving to be an issue (14 percent of the Facebook sidewiki entries were spam; and fairly obviously spam – there were many other entries I classified as “neutral” because they weren’t overtly related to Facebook but also were not overtly advertisements).
  • Reconciling Content: One of the problems of Sidewiki is that one domain may have multiple, disparate Sidewiki pages full of entries.  For example, the discussion in the Sidewiki entry for Facebook.com while one is logged out is different from the Sidewiki entry for Facebook.com when one is logged in (and the “logged-in” page has vastly more entries as a result of more people spending time there).  I’ve posted the same entry to the “logged in” Sidewiki page on Facebook so we’ll see where that goes.
  • Language Barrier: Google may need to come up with some way to separate Sidewiki pages by language; it may be that legitimate entries posted in another language could be marginalized (and voted down) by the predominantly English-speaking audience of sites like Facebook.
  • Restrictions on Dialogue: The interface of Sidewiki makes it difficult to have discussions and back-and-forth about entry content.  (For example, if I were to respond to “Joe G,” I would either have to clutter up my existing entry, or create a separate entry which would exist apart from my original entry and arguably be less relevant – and likely voted down – as it does not directly refer to Fox News).
  • Recency Bias for Larger Sites: When one creates an entry for an established page with many entries, that entry does not appear on the first page, rather it is placed deeper within the site.  The presumption is that through merit, the entry will rise in the ranks if it is deemed worthy by Sidewiki users – however this introduces the problem on the more highly-trafficked sites that good content may have trouble unless people are willing to slog through all of the content about those sites in order to “mine” them for good content. Sidewiki will work for smaller sites with fewer comments, but as the number of comments scales with the traffic to the site – the current merit system breaks down.
Advertisements
  1. Arthur
    October 6, 2009 at 9:20 pm

    Glad you did this, since I haven’t seen very many real examples of spam in the product. I reported one, and it was gone in a few minutes. For your experiment, I’ll have to take your word for it, since if the spam did exist during your experiment – I don’t see it now. What is your criteria for spam BTW?

    86% of comments are legit for a fairly active web property? That’s not bad. If I can get those ratios into my yahoo mail account, I’d have no need to switch!

    What are you considering fair game for spam, btw? The comments below the fold (the “These entries may be less useful” area)? If so, I’m thinking Google intended for that playground to exist. For a another analysis, consider just tracking what’s on the first panel. The majority of users will only see that first panel and it is the only one that sidewiki seems to vett as acceptable quality. When I click on “Next” I know that what I see on subsequent pages may not be useful.

    Like

    • derekdevries
      October 7, 2009 at 3:45 am

      Thanks for the kind words.

      I just went back and checked and the spam I saw is mostly still intact (though it’s only been a few hours). I totally agree; the ratio of spam is very reasonable – I would hypothesize that if people find Sidewiki valuable it will stay that way.

      My criteria for spam was primarily that it be a message that was 1) unrelated to the website being annotated, 2) primarily commercial/promotional in nature, and 3) repeated elsewhere. To that end, I went into the profile of each suspected spammer to be sure that they were posting the same message on other unrelated Sidewiki entries and I reported the profile (rather than each individual spam post – which could explain why it’s taking longer to be removed).

      I definitely agree that people should be free to determine the relative value of a particular entry.

      There were some entries that were totally unrelated, or very opaque/poorly-written – but I didn’t count those as spam because it looked as though some were accidentally posted in the wrong place (they may have referred to particular stories that were at one time featured on, say, Fox News’ website). Those I left for people to vote up or down based on their merits.

      Like

  2. October 7, 2009 at 3:19 am

    Very interesting. I’m just writing about #sidewiki myself and am pretty concerned about how it will play out. Your experiments were interesting though and I look forward to hearing more. I’ll subscribe to your rss feed so I don’t miss your posts.
    Thanks,
    Radiance

    Like

    • derekdevries
      October 7, 2009 at 3:48 am

      Thanks very much; glad you found it useful. I’ll definitely keep cataloging what I find as Sidewiki continues to evolve; I’m anxious to see if these ratios hold as its user base scales larger and incorporates a broader cross-section of the population.

      Like

  1. October 7, 2009 at 1:36 pm
  2. October 21, 2009 at 10:20 pm
  3. December 18, 2009 at 2:07 pm

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: