On this blog, I’ve already blogged about crowdsourcing across cultures and crowdsourcing on Chinese platforms. To be honest, it’s a topic that I would like to explore further on my PhD work, because there seems to be few research about this. This post relates some findings of a design expermiment called Red Mat, which is a personal project of Jan Chipchase, who is also Executive Creative Director of Global Insights at Frog Design. I found out about the Red Mat project on PSFK, but at first I didn’t really understand what it was about… Now I see that it’s an exploratory project that heavily involved the use of Chinese crowdsourcing services. Here are some of the learnings…

Chipchase’s objective was to better understand China’s digital culture. In order to do that, he designed an experiment that had a simple objective:

Make a giant Chinese flag using new red mats bought from across China

This seems surprising and even a little bit far-fetched. As I read the booklet, I understood that the task itself was less important than the process. As Chipchase states in his document, “Red Mat is very much a process driven experiment with over ~20 discrete tasks and many more sub-tasks required to complete the design“. Some of these tasks were outsourced to the crowd via crowdsourcing websites like Zhubajie, Sandaha or Renwumatou (we’re not talking about creative crowdsourcing here, we’re talking about micro-tasks by which individuals perform easy tasks for direct payment).

We hoped to tease out the optimal (crowdsourcing) platforms, number and type of participant for different types of task by the end of the experiment

During the experiment, crowdsourcing was used to…

• Control whether the design of the Chinese flag (that was used for the experiment) was correct
• Identify shops that sold red mats online and the town/city of the seller
• Verify if Chinese internet users are trustworthy in transferring money to a third party
• Propose methods of choosing random shops that sold customized maps

These are fairly simple tasks. For the last task i.e. ask the crowd to come up with ideas to randomize the choice of shops, it worked very well. As Chipchase writes, this task ignited “considerable discussion and creativity“. But the field researcher and his team also faced some difficulties in crowdsourcing. Why? Mainly because of two reasons : (1) crowdsourcing platforms might refuse to host a task because it’s considered too “sensitive” or (2) you might end up with poor quality contributions:

1. Chinese crowdsourcing websites might refuse a task because it’s considered “sensitive”. This actually happened twice during Chipchase’s experiment: for the flag design verification task (“ZhuBaJie rejected the task because the task description included words considered sensitive. The task was rewritten and posted to SanDaHa and the task was finally accepted“) and for the money transfer task (“the task was initially rejected by ZhuBaJie for being too sensitive, we assume because it deals with anonymous money transfers, something that is an issue the world over“). Hence, it is important to design the task according to a site’s guidelines or avoiding “sensitive” topics.
2. The Chinese crowd of contributors might come up with deceiving results. This occured when Chipcase asked people to identify shops that sold red mats. As they say, “most responses were cut and pasted from Taobao (online shopping portal) rather than people looking in their neighbourhood as we had hoped” which means that “there was a significant volume of poor quality answers that needed to be filtered out“. This is a common issue of crowdsourcing, Jeff Howe himself once said that 90% of the content generated by the crowd was crap (see here). A related issue is that Chinese crowdsourcing sites are increasingly used to produce fake content, as a recent paper from Cornell University underlines.

Finally, after a long and insightful sequence of experiments, Chipchase and his team managed to gether all the mats. Check it out in the following video:

But the output is not very important, the process of getting there is! The flag or the mats went back to a warehouse in Shanghai and are now being sold on Taobao, a popular online marketplace. Chipchase says that the money will be donated to artists “to run similar experiments exploring national identity, taboos and trust in countries of their choosing“. Noble cause. I hope that we’ll see more such projects in the future. I would have liked to have a clearer overview of Chipchase’s learnings – a kind of do’s & don’ts. Anyway, I learned a lot!

If you want to see a visual presentation of the experiment, you might want to check Chipchase’s slideshare deck here.