Shih Wen Huang

Show me the money!: an analysis of project updates during crowdfunding campaigns

Hundreds of thousands of crowdfunding campaigns have been launched, but more than half of them have failed. To better understand the factors affecting campaign outcomes, this paper targets the content and usage patterns of project updates -- communications intended to keep potential funders aware of a campaigns progress. We analyzed the content and usage patterns of a large corpus of project updates on Kickstarter, one of the largest crowdfunding platforms. Using semantic analysis techniques, we derived a taxonomy of the types of project updates created during campaigns, and found discrepancies between the design intent of a project update and the various uses in practice (e.g. social promotion). The analysis also showed that specific uses of updates had stronger associations with campaign success than the projects description. Design implications were formulated from the results to help designers better support various uses of updates in crowdfunding campaigns.

Voyant: generating structured feedback on visual designs using a crowd of non-experts

Feedback on designs is critical for helping users iterate toward effective solutions. This paper presents Voyant, a novel system giving users access to a non-expert crowd to receive perception-oriented feedback on their designs from a selected audience. Based on a formative study, the system generates the elements seen in a design, the order in which elements are noticed, impressions formed when the design is first viewed, and interpretation of the design relative to guidelines in the domain and the users stated goals. An evaluation of the system was conducted with users and their designs. Users reported the feedback about impressions and interpretation of their goals was most helpful, though the other feedback types were also valued. Users found the coordinated views in Voyant useful for analyzing relations between the crowds perception of a design and the visual elements within it. The cost of generating the feedback was considered a reasonable tradeoff for not having to organize critiques or interrupt peers.

Don't hide in the crowd!: increasing social transparency between peer workers improves crowdsourcing outcomes

This paper studied how social transparency and different peer-dependent reward schemes (i.e., individual, teamwork, and competition) affect the outcomes of crowdsourcing. The results showed that when social transparency was increased by asking otherwise anonymous workers to share their demographic information (e.g., name, nationality) to the paired worker, they performed significantly better. A more detailed analysis showed that in a teamwork reward scheme, in which the reward of the paired workers depended only on the collective outcomes, increasing social transparency could offset effects of social loafing by making them more accountable to their teammates. In a competition reward scheme, in which workers competed against each other and the reward depended on how much they outperformed their opponent, increasing social transparency could augment effects of social facilitation by providing more incentives for them to outperform their opponent. The results suggested that a careful combination of methods that increase social transparency and different reward schemes can significantly improve crowdsourcing outcomes.

Leveraging the crowd to improve feature-sentiment analysis of user reviews

Crowdsourcing and machine learning are both useful techniques for solving difficult problems (e.g., computer vision and natural language processing). In this paper, we propose a novel method that harnesses and combines the strength of these two techniques to better analyze the features and the sentiments toward them in user reviews. To strike a good balance between reducing information overload and providing the original context expressed by review writers, the proposed system (1) allows users to interactively rank the entities based on feature-rating, (2) automatically highlights sentences that are related to relevant features, and (3) utilizes implicit crowdsourcing by encouraging users to provide correct labels of their own reviews to improve the feature-sentiment classifier. The proposed system not only helps users to save time and effort to digest the often massive amount of user reviews, but also provides real-time suggestions on relevant features and ratings as users generate their own reviews. Results from a simulation experiment show that leveraging on the crowd can significantly improve the feature-sentiment analysis of user reviews. Furthermore, results from a user study show that the proposed interface was preferred by more participants than interfaces that use traditional noun-adjective pair summarization, as the current interface allows users to view feature-related information in the original context.

Motivating crowds using social facilitation and social transparency

We reported results from an experiment using an image labeling task, when workers were able to compare their own labels with the labels generated by another worker, they were motivated to generate more labels. In addition, when the workers shared their demographic information with their colleagues, the number of labels generated by them became even higher. This indicates that we can utilize the power of social facilitation and social transparency to motivate workers in crowdsourcing to reduce operation costs and even enhance outcome quality.

Leveraging Human Computations to Improve Schematization of Spatial Relations from Imagery

The process of generating schematic maps of salient objects from a set of pictures of an indoor environment is challenging. It has been an active area of research as it is crucial to a wide range of context- and location-aware services, as well as for general scene understanding. Although many automated systems have been developed to solve the problem, most of them either require predefining labels or expensive equipment, such as RGBD sensors or lasers, to scan the environment. In this article, we introduce a prototype system to show how human computations can be utilized to generate schematic maps from a set of pictures, without making strong assumptions or demanding extra devices. The system requires humans (crowd workers from Amazon Mechanical Turks) to do simple spatial mapping tasks in various conditions, and their data are aggregated by filtering and clustering techniques that allow salient cues to be identified in the pictures and their spatial relations to be inferred and projected on a two-dimensional map. In particular, we tested and demonstrated the effectiveness of two methods that improved the quality of the generated schematic map: (1) We encouraged humans to adopt an allocentric representations of salient objects by guiding them to perform mental rotations of these objects and (2) we sensitized human perception by guided arrows superimposed on the imagery to improve the accuracy of depth and width estimation. We demonstrated the feasibility of our system by evaluating the results of schematic maps generated from indoor pictures taken from an office building. By calculating Riemannian shape distances between the generated maps to the ground truth, we found that the generated schematic maps captured the spatial relations well. Our results showed that the combination of human computations and machine clustering could lead to more-accurate schematized maps from imagery. We also discuss how our approach may have important insights on methods that leverage human computations in other areas.

Review explorer: an innovative interface for displaying and collecting categorized review information

Review Explorer is an interface that utilizes categorized information to help users to explore a huge amount of online reviews more easily. It allows users to sort entities (e.g. restaurants, products) based on their ratings of different aspects (e.g. food for restaurants) and highlight sentences that are related to the selected aspect. Existing interfaces that summarize the aspect information in reviews suffer from the erroneous predictions made by the systems. To solve this problem, Review Explorer performs a real-time aspect sentiment analysis when a reviewer is composing a review and provides an interface for the reviewer to easily correct the errors. This novel design motivates reviewers to provide corrected aspect sentiment labels, which enables our system to provide more accurate information than existing interfaces.