Reaction or Speculation: Building Computational Support for Users in Catching-Up Series Based on an Emerging Media Consumption Phenomenon
aa r X i v : . [ c s . H C ] F e b RIKU ARAKAWA ∗ , The University of Tokyo, Japan
HIROMU YAKURA ∗† , University of Tsukuba, JapanA growing number of people are using catch-up TV services rather than watching simultaneously with otheraudience members at the time of broadcast. However, computational support for such catching-up users hasnot been well explored. In particular, we are observing an emerging phenomenon in online media consump-tion experiences in which speculation plays a vital role. As the phenomenon of speculation implicitly assumessimultaneity in media consumption, there is a gap for catching-up users, who cannot directly appreciatethe consumption experiences. This conversely suggests that there is potential for computational supportto enhance the consumption experiences of catching-up users. Accordingly, we conducted a series of stud-ies to pave the way for developing computational support for catching-up users. First, we conducted semi-structured interviews to understand how people are engaging with speculation during media consumption.As a result, we discovered the distinctive aspects of speculation-based consumption experiences in contrastto social viewing experiences sharing immediate reactions that have been discussed in previous studies. Wethen designed two prototypes for supporting catching-up users based on our quantitative analysis of Twitterdata in regard to reaction- and speculation-based media consumption. Lastly, we evaluated the prototypes ina user experiment and, based on its results, discussed ways to empower catching-up users with computationalsupports in response to recent transformations in media consumption.CCS Concepts: •
Human-centered computing → Empirical studies in collaborative and social com-puting ; Social content sharing ; Social media .Additional Key Words and Phrases: Media consumption, Speculation, Spoiler, Catch-up TV
ACM Reference Format:
Riku Arakawa and Hiromu Yakura. 2021. Reaction or Speculation: Building Computational Support for Usersin Catching-Up Series Based on an Emerging Media Consumption Phenomenon.
Proc. ACM Hum.-Comput.Interact.
5, CSCW1, Article 151 (April 2021), 28 pages. https://doi.org/10.1145/3449225
As McLuhan [43] predicted, the Internet has enabled us to communicate across physical distancesin ways that have resulted in new forms of media consumption. One of the first ways that the Inter-net was leveraged to create new consumption experiences was presented in the context of SocialTV [17]: computer-mediated TV viewing experiences shared by distanced users. This concept isnow commonplace due to the rise of social networking services, especially Twitter [65]. Moreover, ∗ These authors contributed equally and are ordered alphabetically. † Also with Teambox Inc., Japan.Authors’ addresses: Riku Arakawa, The University of Tokyo, Tokyo, Japan, [email protected]; HiromuYakura, University of Tsukuba, Tsukuba, Japan, [email protected] to make digital or hard copies of all or part of this work for personal or classroom use is granted without feeprovided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice andthe full citation on the first page. Copyrights for components of this work owned by others than the author(s) must behonored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists,requires prior specific permission and/or a fee. Request permissions from [email protected].© 2021 Copyright held by the owner/author(s). Publication rights licensed to ACM.2573-0142/2021/4-ART151 $15.00https://doi.org/10.1145/3449225Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CSCW1, Article 151. Publication date: April 2021. it has been empirically shown that users feel a sense of connectedness to a broader audience inthe social viewing experiences by sharing their immediate reactions on Twitter [52]. Vatavu [60]took advantage of this finding to enhance users’ viewing experiences by proposing an interactiontechnique to convey the presence of remote users through their silhouettes.In addition, we are observing an emerging phenomenon in online media consumption experi-ences in which not just emotional reactions but also speculation plays a vital role. One exampleof this phenomenon can be found in
Your Turn to Kill ( あなたの 番 です ), a Japanese mysteryTV series broadcast by Nippon Television from April to September 2019. This TV series sparkedspeculations about what might happen next on social networking services [68], where its hashtagrepeatedly ranked first in a global ranking of Twitter trends [58]. Notably, the series’ productionteam intentionally encouraged this movement through marketing efforts, such as holding a promo-tional event on Twitter using the hashtag “ 最 初 の 被 害 者 を 推 理 せよ [deduce the first victim]”during the broadcast of the first episode and posting a summary video about the unsolved myster-ies directly prior to the broadcast of the final episode [34]. In light of events like these, speculationabout upcoming episodes of serial media content can be considered to be an important factor inoffering better media consumption experiences for users on the Internet.These reaction- and speculation-based consumption experiences can both attract users, assum-ing that they appreciate media content synchronously at the same time that it is delivered. On theother hand, the Internet has also changed how media content is delivered and consumed. Specifi-cally, many TV networks now provide catch-up TV services that allows users to watch previousepisodes online [1]. This transition suggests that we can no longer assume that media consumptionis simultaneous, in contrast to the above discussions for offering better consumption experiences,which rely heavily on synchronous consumption.We hypothesized that it would be possible to leverage computers to enhance the consumptionexperiences of such catching-up users . One intuitive approach is to exploit a pseudo-synchronouseffect [29] in the manner Danmaku interaction does [66]. That is, when a user watches a previ-ously aired episode using a catch-up TV service, tweets posted during the initial broadcast arepresented in relation to playback time within the episode. Considering the effect of Danmaku[39] and Twitter-based social viewing experience [52], it would make users feel as if they werewatching simultaneously with other users and enhance their sense of social presence. However,we argue that this approach does not sufficiently incorporate media consumption experiences in-volving speculation, as we mentioned above. This is because, besides such social aspects, onlinespeculative discussions may have informative aspects, which have not been focused on by thisapproach. Moreover, online speculative discussions often contain spoilers [21], which should betaken into consideration when developing computational supports for catching-up users.With these points in mind, we investigated the current status of online media consumptionexperiences, especially for serial media content; evaluated possible approaches to providing betterexperiences for catching-up users; and discussed possibilities for future computational support.The steps of our research were as follows:(1) We first conducted semi-structured interviews to identify how users involve with and areaffected by online media consumption experiences centering on speculation.(2) We then performed quantitative data analysis of tweets about two TV series to provide back-ground for developing computational supports for catching-up users, illustrating the uniqueaspects of reaction- and speculation-based media consumption.(3) Based on the results of (1) and (2), we carried out a user experiment and evaluated the effectsof two different approaches to enhancing the consumption experiences of catching-up users. Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CSCW1, Article 151. Publication date: April 2021. eaction or Speculation: Building Computational Support for Users in Catching-Up Series 151:3
Our findings and subsequent discussions illustrate further possibilities for computationally sup-porting catching-up users, especially with regard to speculation, which has received little researchattention to date despite its effectiveness in enhancing users’ consumption experiences.
To situate our work, we begin by examining the previous literature on catch-up TV, which hashighlighted the increase in the number of catching-up users. We also review the literature on socialexperiences in the context of media consumption and discuss the need for interaction techniquesto support catching-up users.
TV from the so-called “network era” (1952 to the mid-1980s) was long considered a one-way com-munication medium with limited programming choices, where viewers had to base their dailyduties around the schedule that a few TV networks had mandated [40]. This relationship betweenTV programs and the audience was drastically altered by the advent of digital video recorders. Stud-ies were conducted in many countries [31, 36] to investigate the impact of digital video recorders,which ultimately found their increased usage [64]. Following the success of digital video recorders,technologies for enabling other viewing practices—such as downloading, streaming, and mobileviewing—were developed and became widespread [6].In recent years, it is becoming difficult to assume that many people enjoy media content at thetime that it is distributed. Instead, a growing number of users are choosing to catch up after thefirst air using a catch-up TV website hosted by the networks or on other video-distribution ser-vices such as Amazon, Netflix, and Hulu [41, 63]. According to Vanettenhoven and Geerts [59],users of video-on-demand services prefer movies and drama series to news and comedies. The au-thors conducted in-home interviews to investigate the use of on-demand platforms and reportedusers’ preferences. They also suggested that users might appreciate replays of episodes in a pre-vious season before the beginning of a new season. Considering the demand for catching up onthe contents of series, we suspect that there is an excellent opportunity to address the needs ofcatching-up users through computer-supported approaches.
On the other hand, we argue that the existing computer-supported approaches which highlightsocial experiences are insufficient to address the current experiences of media consumption in thelight of the increase in catching-up users. To situate our research and develop ideas for improve-ment, we review the articles about social experiences in media consumption from two aspects:sharing immediate reactions and sharing speculations.
One popular approach to exploiting computers for media con-sumption is invoking the feeling of watching TV with peers by sharing reactions in real time.For instance, Ducheneaut et al. [17] investigated social interactions among TV viewers and devel-oped the concept of “Social TV” that would enable geographically distributed viewers to commu-nicate with one another. Though their experimental design involved only audio relays betweentwo rooms or pre-recorded videos of previous viewers to simulate the idea, many Internet-basedinteraction techniques have since been proposed for social TV [8, 10]. As Cesar and Geerts [9]listed, this approach has also been implemented in commercial products and services. To furtherenhance viewer experiences in this manner, Vatavu et al. [60] introduced audience silhouettes forTV by overlaying viewers’ body movements on the content in real time.
Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CSCW1, Article 151. Publication date: April 2021.
In addition, the advent of social networking services has promoted social viewing experienceson the Internet [12, 65]. Kim et al. [33] analyzed the dynamics of fans on Twitter during the 2014FIFA World Cup and suggested that fans’ social communication on Twitter induced feelings ofsocial presence on a global scale. Schirra et al. [52] reported the existence of Twitter users whoactively posted during TV series broadcasts and conducted in-depth interviews with them to un-derstand their motivations for posting. They concluded that these so-called “live-tweeters” seeksocial viewing experiences that evoke feelings of being connected with others. These results arealso supported by Kim et al. [32], who conducted an online survey which found that social presenceplays a mediator role in the process whereby social viewing experiences lead to viewer enjoyment.These interaction techniques, however, are not directly applicable to enhancing the consumptionexperiences of catching-up users, as we can no longer assume that peers are watching the samecontent simultaneously.
Although speculation during media consumption has recently be-come popular, as we mentioned in Section 1, little research has focused on it to date [21, 27, 44].Jenkins [27] first reported the existence of an online fan community in which speculations wereactively posted in the context of the narrative hook of the TV show
Survivor . Based on his account,Gray and Mittell [21] conducted an in-depth investigation of speculative discussions on a fan fo-rum of
Lost , which they described as the most elaborated and narratively complex TV program ofthe 2000s. Their online qualitative survey designed to understand fans’ motivations for participat-ing in such discussions offered several explanations, including that fans regard the speculationsthemselves as enjoyable texts. Mittell [44] reported that the
Lost fan wiki had served as a place tostore such speculation.However, the existing literature has not explored whether catching-up users can appreciate suchstored discussions or how to design interactions that leverage speculation for them. In addition, itis possible that social networking services, which emerged after the aforementioned studies, havesignificantly transformed the ways of media consumption related to speculation. Accordingly, anup-to-date investigation is warranted.Here, we note that our focus is inconsistent with those of previous studies regarding user com-ments and reviews in media consumption [14, 25]. As Gray and Mittell [21] stated, speculationis not a retrospective experience (for example, posting a review on the Internet after watchinga movie); rather, it is an ongoing experience based on a form of seriality that is predicated onanticipating the next episode.
Although a lot of previous studies have explored computer-supported media consumption, fewhave been proposed and designed explicitly for catching-up users. One such interaction techniqueis Danmaku [66], a new commenting system popularized in Asian countries that offers social inter-actions to viewers of online videos by leveraging pseudo-synchronicity. In Danmaku, commentsare recorded in connection to a specific playback time within a video, indicating when the com-ments are typed. The comments are then overlaid on the video in synchronization with futureplaybacks. While this is not exactly a synchronous co-viewing experience, Danmaku commentingis known to create a pseudo-synchronized viewing experience [29], which leads to a sense of beingsocially connected [11].However, Danmaku was not originally designed for catch-up TV, and there is room to dis-cuss how Danmaku-like interactions can contribute to offering better consumption experiencesto catching-up users. In particular, considering the positive effect of the Twitter-based social view-ing experiences [32, 52], we can suppose that presenting live-tweets in a Danmaku interface would
Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CSCW1, Article 151. Publication date: April 2021. eaction or Speculation: Building Computational Support for Users in Catching-Up Series 151:5 enhance their consumption experiences, which, however, is not experimentally confirmed. More-over, considering that Danmaku is mainly used to share immediate reactions to scenes in videos[11], we anticipate that simply applying a Danmaku interface to sharing comments on videoswould not address the unique particularities of sharing speculations.Another possible solution is a companion application to help viewers understand complex nar-ratives, such as that proposed by Silva et al. [55]. Similarly, Nandakumar and Murray [47] con-firmed that their second-screen application displaying a story map improved the first-time view-ers’ comprehension of the latest episode of a TV series that they had never watched. Yet whilethis technique can effectively summarize information contained in missed episodes without theneed to watch those episodes, it is not directly applicable to catching-up users who want to enjoywatching the missed episodes from the beginning.Therefore, in this paper, we aim to develop computational support for catching-up users byfocusing on the effects of sharing immediate reactions and speculations. We discuss the charac-teristics of each through semi-structured interviews and analyzing public data and propose newapproaches, whose effectiveness is evaluated via a user experiment.
Our literature review revealed that previous studies have paid little attention to media consump-tion related to speculation, especially with reference to interaction techniques. On the other hand,as mentioned in Section 1, we are observing that speculation plays a vital role in recent mediaconsumption and anticipating that providing opportunities for catching-up users to relate to spec-ulation would enhance their consumption experiences. For this purpose, we first conducted semi-structured interviews to understand how people are engaging with such speculation on the In-ternet during media consumption experiences. If they are not appreciating speculation, providingopportunities for engaging with speculation would not be beneficial for catching-up users. In ad-dition, if they are not having trouble relating to speculation in catch-up situations, developing anew computational approach would not be necessary. In this section, we describe our procedureand findings of our interviews.
Our interviews included 10 participants without compensation aged between 23 to 34, of whomthree were female. The nationalities of all participants consisted of East Asian countries (Japanese,Korean, and Chinese). They were recruited via word of mouth and online communication in alocal community where over 100 university students gather. At the time of the recruitment, allparticipants self-reported they had experience in engaging with online speculative discussionsduring media consumption.
The interviews were conducted face-to-face, except for four participants who were interviewedvia video call. The interviews were conducted in Japanese, as all participants were fluent Japanesespeakers, and were audio-recorded. Each interview lasted approximately 30 minutes. Interviewquestions were designed to explore participants’ behaviors in online speculative discussions, in-cluding motivation, timing, and feelings and thoughts. For example, we asked, “How do you engagewith speculation, e.g., reading or posting?”; “How often do you engage with speculation?”; “Whatis your motivation or aim of engaging with speculation?”; “What is on your mind when you engagewith speculation?”; “How do you feel when you are engaging with speculation?”; “What do you
Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CSCW1, Article 151. Publication date: April 2021. care about when you engage with speculation?”; and “What is your usual feeling after engagingwith speculation?”Our analysis was guided by previous research on media consumption-related social interactionson the Internet [26, 52]. Using open coding [56], the transcriptions of the interviews were analyzedto document the behaviors and opinions of participants with regard to speculation on the Internet.First, one author read through the transcriptions carefully and extracted emergent themes. Allauthors then reviewed and organized the themes found in the interviews and discussed apparentdiscrepancies until a consensus was reached. Through this iterative refinement process, three keythemes were identified, which are outlined below.
Our analysis of the interviews revealed several aspects in regards to how the participants areaffected by engaging with speculation on the Internet. In this section, we describe each aspect andpresent relevant quotes from them.
In our interviews, most participants mentionedthat deepening their understandings of media content is one of the motivations or values of en-gaging with online speculative discussions:When I want to know how other people interpret the storyline or scenes, I use theInternet to look for speculation. As they remind me of the creators’ intentions andhooks that I didn’t notice, the value of the content increases much more. (P1)For media content which has connotations behind it, I am sometimes confused abouthow to interpret their descriptions. In such cases, I use Google to look for fans’ spec-ulation on them. (P2)It is difficult to be confident that I’m enjoying 100% of the creator’s messages by justreading or watching the content once. So, I wonder if there is a way to enjoy [thecontent] that I haven’t noticed, and then, I go looking for speculation. I google thetitle with the word “ 考 察 ” [meaning “speculation” in Japanese] and jump to them.(P10)These comments suggest that users’ sense that they lacked understanding of media content ledthem to actively engaging with speculation—for example, searching with the title on the Internet.They then deepened their understandings or interpretations of the story or connotations of thecontent, which in turn yielded an enhanced consumption experience.In addition, participants mentioned various platforms they used to deepen their understandingsof media content, such as:I sometimes refer to personal blogs on the Internet on which long articles involvingdeep speculation about various content are posted. (P4)I like reading long speculation posts on Twitter sharing screenshots of note-takingapps. I have a private list of Twitter users whom I trust in the quality of their specula-tions and often look for their comments when I finish the latest episode. (P7)I usually visit Twitter to search with hashtags for other users’ thoughts. But, in somecases, such as when I couldn’t find it interesting though its reputation was good orwhen I got confused in its interpretations, I googled the title and read some speculatingarticles. (P10)I often use Instagram to access the speculations of other users. Searching the title ofthe TV series provides me a stream of many posts. I think the photo accompanyingeach post reflects the sense of its author well. (P8) Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CSCW1, Article 151. Publication date: April 2021. eaction or Speculation: Building Computational Support for Users in Catching-Up Series 151:7
These comments imply that speculative discussions available in online platforms can be leveragedfor constructing computational support for catching-up users.
At the same time, we observed that partic-ipants felt a sense of connectedness with other users through engaging with online speculativediscussions, in the same manner as live-tweets [52]:Seeing reactions to my post of a link to speculation articles or joining online discus-sions lets me feel an atmosphere of excitement. (P3)It is simply fun to discuss online with people who watched the same content. Some-times I’m convinced by seeing different opinions from mine, but rather, I feel happywhen I confirm that someone shares the same feelings or interpretations. (P9)Just after watching videos, I often search the titles on Twitter. It’s not only aboutpursuing a novel insight or interpretation. But just diving into other users’ thoughtsor speculations makes me bask in the afterglow. (P4)On Twitter, I can see speculations posted just after the broadcast. I like this sense ofliveness with other users. (P7)These comments revealed an effect that resembled observations about
Lost fan forums [21]: Thesespeculative discussions not only provide opportunities to reread content with a more in-depthanalysis and enhance narrative pleasure, but also offer a communal relationship that circulatessuch discussions.In addition, we found that user behaviors could be divided into “posters” and “lurkers,” as pre-vious studies of online communities [18, 53] have suggested. That is, posters are actively involvedin speculation and discussion with other users, such as the first two quotes from P3 and P9 aboveindicate, whereas lurkers merely consume speculations posted by other users, as indicated by thelast two quotes from P4 and P7. Lurkers noted that they also felt a sense of connectedness withothers through reading speculative discussions.In summary, our interviews demonstrated commonalities with the behavioral types and effectsdescribed in previous studies of social experiences in media consumption. In addition, we eluci-dated an unrevealed aspect that would not achieved by sharing immediate reactions, that is, deep-ening understanding through online speculative discussions. These positive aspects delivered byengaging with online speculative discussions suggest that providing opportunities for catching-up users to relate to speculation would enhance their consumption experience. On the other hand,our interviews also revealed a side effect regarding this phenomenon, which in turn require con-sideration when constructing computational support.
Participants also mentioned a certain side effect ofengaging with online speculative discussions:While I’m not caught up with the latest episode, even though I didn’t search the title,someone on my timeline tweeted about the speculation. Such a spoiler causes me tolose motivation to continue watching. (P4)I felt quite a bit of regret when I saw spoilers for content that has an elaborate storyline.(P5)Sometimes when I searched the names of characters, I happened to see images con-taining spoiling speculation. (P2)According to Gray and Mittell [21], some users prefer to be spoiled because spoilers enable themto take control of their emotional responses. However, the participants in this study expressedcomplaints about spoilers or at least wished to control their exposure to such information.
Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CSCW1, Article 151. Publication date: April 2021.
In particular, some participants mentioned their hesitation to see speculations due to the risk ofbeing spoiled when they were catching up on previously published stories:In the case that I start following long-running content after many episodes have al-ready been published, I think it would be exciting if I were able to refer to other peo-ple’s speculations or discussions. However, considering the possibility of being spoiledby searching them on the Internet, it is not possible. (P4)I would like to take a look at speculations for each episode in order to savor it. Butgiven the risk of spoilers, it’s hard to do. (P3)These comments suggest that it would be important to avoid spoilers in developing support forcatching-up users by leveraging online speculative discussions.
In Section 1 and Section 2.3, we discussed the possibility of providing computational support forcatching-up users by presenting tweets that share immediate reactions. At the same time, oursemi-structured interviews confirmed the positive aspects of engaging with the online speculativediscussions, which implies another emerging approach for supporting catching-up users. In par-ticular, given that the effect of deepening understandings of media content would be specific tospeculation (Section 3.3.1), it is worth to explore both approaches.To develop computational support for catching-up users via these approaches, we need to con-struct a deeper understanding based on actual data regarding both of immediate reactions andspeculations. For this purpose, we conducted tweets analysis in regard to reaction- and speculation-based media consumption since Schirra et al. [52] and the participants in Section 3.3 suggested theuse of Twitter as a platform to relate to those activities. This analysis is crucial to develop com-putational support because the approaches would not be feasible without the tweet data to bepresented. In addition, the characteristics of such tweets can provide implications for designingcomputational support, which we discuss later in Section 5. We thus reviewed two examples of me-dia content and analyzed users’ behaviors quantitatively and qualitatively using publicly availabledata.
As Twitter has over 330 million monthly active users [54], previous research has used Twitter datato understand social behaviors in diverse areas, including politics [4], crises [5], and TV programs[15]. For example, as mentioned in Section 2.2, some studies examined live-tweeting during TVprograms by classifying types of content posted [65] or analyzing user motivations [52]. Morespecifically, Schirra et al. [52] presented a time plot of tweeting activities during scheduled airtimesand suggested the existence of “have-to-tweet-about” moments, represented by triggers in thestory that tempt users to tweet. Our approach is similar in that we collected tweets and analyzedthem both quantitatively and qualitatively. However, we targeted not only on reaction-relatedactivities that these studies dealt with but also on speculation-related activities.For the purpose of comparing the two activities, we used hashtag searches to gather tweetsaround the airtimes of the content. We first determined hashtags corresponding to two categories:(1) tweets involving speculation about a target media content ( speculation-tweets ), and (2) othertweets about the same content ( non-speculation-tweets ). Here, we reviewed tweets containing thetitle of the target media content and manually extracted popular hashtags for each category, whoseprocess is detailed in the following sections. We note that this is based on our anticipation that speculation-tweets would contain specific words or hashtags, such as “ 考 察 ”, as suggested in Section 3.3.1. Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CSCW1, Article 151. Publication date: April 2021. eaction or Speculation: Building Computational Support for Users in Catching-Up Series 151:9
In contrast, most non-speculation-tweets would reflect the reaction-related activities while therewould be a few tweets used for other purposes such as pre-broadcast advertising by the officialmarketing account or the actors of the TV show.For each hashtag found, we collected corresponding tweets that had been posted during a spec-ified period around the broadcast date of certain episodes. We also took into account that tweetscould contain multiple hashtags, leading search results for different hashtags to include the sametweets. To ensure an accurate count, we thus removed duplicated tweets using the tweet post IDafter aggregating the results of each search. In addition, if a tweet contained hashtags for both speculation-tweets and non-speculation-tweets , then we assumed the tweet was intended to sharespeculation from our observation and classified it as a speculation-tweet . We used the collecteddata to analyze the characteristics of tweeting behavior related to speculation, including the tim-ing, length, and content of tweets.
Your Turn to Kill ( あなたの 番 です ) We first examined tweets about the Japanese TV series,
Your Turn to Kill ( あなたの 番 です ),a fictional narrative set in an apartment building where serial murders occur. As described inSection 1, this series was designed to encourage speculation on social networking services, whereviewers often posted their opinions and discussed their predictions about the next victim or thetrue culprit.To determine hashtags for data collection, we searched tweets using the title of the TV show asa hashtag. Our initial results indicated that, in addition to the title, its simple abbreviation, “ あな 番 ,” was frequently used in the tweets. Since we observed these two hashtags were widely usedfor sharing immediate reactions, we selected them for non-speculation-tweets . We next found that speculation-tweets incorporated hashtags that appended the word “ 考 察 ” [meaning “speculation”in Japanese] to the two hashtags (“ あなたの 番 です 考 察 ” or “ あな 番 考 察 ”). Interestingly, wealso discovered that a specific hashtag had been coined for posting speculation about this TV series:“ オラウータンタイム ” [meaning “time of orangutan” in Japanese]. The use of this hashtag isattributable to the main character’s frequent use of this term when speculating about solutions tothe mysteries he confronts. It is notable that the viewers also reused this term to present theirown speculations.The TV series was broadcast weekly between April 14, 2019, and September 8, 2019. We selectedone episode from the middle of the series and a second episode from the later in the series, whichwere broadcast on June 9 from 22:30–23:30 JST and September 1 from 22:30–23:30 JST, respectively.To observe viewers’ behavior around these airtimes, we collected tweets approximately one daybefore and after each airtime, i.e., from June 9 at 0:00 JST to June 10 at 23:59 JST and from September1 from 0:00 JST to September 2 at 23:59 JST. Table 1 summarizes the selected hashtags and the totalnumber of tweets for both speculation-tweets and non-speculation-tweets during each period.In the middle episode period, there were 1,253 and 18,804 total speculation-tweets and non-speculation-tweets , respectively. During the late episode period, these numbers increased to 9,903 and97,154, respectively, as the program became more popular with a greater number of fans involvedin online communication. The ratio of speculation-tweets to non-speculation-tweets also signifi-cantly increased, from 6.7% in the middle episode period to 10.2% in the late episode period—thatis, more speculation-tweets posted toward the end of the TV series as the climax of the story ap-proached. According to online speculative discussions, this term seemed to come from “The Murders in the Rue Morgue” by EdgarAllan Poe. Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CSCW1, Article 151. Publication date: April 2021.
Table 1. Hashtags selected and the number of tweets collected for speculation-tweets and non-speculation-tweets about
Your Turn to Kill ( あなたの 番 です ). Tweet category Hashtags Number of total tweets duringJun. 9 – Jun. 10 Sep. 1 – Sep. 2 speculation-tweets あなたの 番 です 考 察 [title + “speculation”] 1,253 9,903 あな 番 考 察 [abbr. title + “speculation”] オラウータンタイム [“time of orangutan”] non-speculation-tweets あなたの 番 です [title] 18,804 97,154 あな 番 [abbr. title] Fig. 1. Time plots of the number of speculation-tweets and non-speculation-tweets within the two observedperiods (top: the middle episode, bottom: the late episode) for
Your Turn to Kill ( あなたの 番 です ). Graybackgrounds denote each episode’s airtime. A time plot of the volume of tweets from this dataset is presented in Figure 1. Tweets werecounted on a minute basis, with units smaller than minutes are rounded down. Similar to theresult of the previous research on live-tweeters [52], peak volume was aligned with the episode’sairtime for both speculation-tweets and non-speculation-tweets . Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CSCW1, Article 151. Publication date: April 2021. eaction or Speculation: Building Computational Support for Users in Catching-Up Series 151:11
Fig. 2. Time plots of the relative volume of speculation-tweets and non-speculation-tweets during the twoobserved periods (top: the middle episode, bottom: the late episode) for
Your Turn to Kill ( あなたの 番 です ).Gray backgrounds denote the airtime of each episode. To further analyze the timing of users’ involvement in the speculation, we created a time plot ofthe normalized volume of tweets around the episode’s airtime (18:30–27:30), calculated by divid-ing by the total number of tweets in each category during the period. The difference in behaviorbetween speculation-tweets and non-speculation-tweets is illustrated in Figure 2. While the peaksof both speculation-tweets and non-speculation-tweets were concentrated near the end of each air-time, we observed lingering segments of speculation-tweets within a few hours after the end ofthe broadcast. This suggests that speculation-tweets are likely to appear around and after the endof the program, which can be attributed to viewers’ motivation to post forecasts about what willhappen in the next episode, rather than sharing immediate reactions. We infer this trend wouldbe a unique property of media consumption related to speculation, in comparison with previouslyobserved consumption experiences centered on immediate reactions [52].Next, we investigated the contents of the tweets in the dataset to characterize qualitative per-spectives. In addition to the observation that text information in speculation-tweets was related topredicting future story developments compared to that in non-speculation-tweets , we found that speculation-tweets often contained hyperlinks to external content—such as images, other tweets,blogs and other articles, and YouTube videos. To confirm this, we classified the tweets based onpattern-matching of the contained URL and calculated the ratio of tweets in each hyperlink type to
Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CSCW1, Article 151. Publication date: April 2021.
Table 2. Ratio of tweets containing hyperlinks by the type of its destination for
Your Turn to Kill ( あなたの 番 です ). Period Tweet category Hyperlink types TotalImages Other tweets YouTube videos Blogs & other articlesThe middle episode speculation-tweets non-speculation-tweets speculation-tweets non-speculation-tweets speculation-tweets non-speculation-tweets
Fig. 3. A tweet which includes a screenshot of a note-taking app for the purpose of sharing long sentencesof speculation. the total number of tweets. As shown in Table 2, the ratio of containing hyperlinks in speculation-tweets is nearly double that of non-speculation-tweets for each type.The images in non-speculation-tweets were observed to be often pictures of the actors or ac-tresses on the show, accompanied by comments such as “Her acting is amazing” or “He is cool!”In contrast, when sorting tweets by number of likes, we observed a wide variety of speculation-tweets incorporated different types of images. In particular, we found a number of screenshotsof note-taking apps containing extensive speculations, which correspond to the comments by P7in Section 3.3.1. An example of one such tweet is shown in Figure 3. This suggests a potentialemerging usage of Twitter as a tool to deliver lengthy speculations.Similar behaviors were observed for blogs and other articles and YouTube videos. The linkedcontent in speculation-tweets often presented opinions and predictions about the next victim or the This is taken from https://twitter.com/WXkKXnSZVF0zObE/status/1169210679619813376 (Accessed: January 1, 2020).Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CSCW1, Article 151. Publication date: April 2021. eaction or Speculation: Building Computational Support for Users in Catching-Up Series 151:13
Fig. 4. Number of characters in speculation-tweets and non-speculation-tweets during the two observed peri-ods (top: the middle episode, bottom: the late episode) for
Your Turn to Kill ( あなたの 番 です ). Speculation-tweets contained significantly more characters than non-speculation-tweets ( 𝑝 < . ). true culprit, while links in non-speculation-tweets typically included content with which fans werepleased with, such as costume design of the show’s actors and actresses. Moreover, speculation-tweets were often quoted in other tweets with comments such as “I’m just amazed by this reasoning.Certainly, this idea can explain that scene,” “I don’t agree with this because this contradicts hisalibi,” or “I totally didn’t notice this point. I need to watch the episode again.” Another unique usageof Twitter was the creation of collection threads of speculation-tweets using Twitter’s threadingfunction, with additional meta-speculation based on a comparison of the listed tweets.Finally, to explore the quantitative difference in the information that users shared in speculation-tweets versus non-speculation-tweets , we compared the number of characters contained in eachtype of tweets. Here, we excluded the tweets with hyperlinks in order to count the number ofcharacters in text without URLs. As a result, the number of characters in speculation-tweets wassignificantly larger than that in non-speculation-tweets (Figure 4), which suggests a distinct userbehavior of sharing lengthier, speculative thoughts and opinions that differ from immediate re-sponses to the program.These results suggest the unique characteristics of speculation-tweets in comparison with thoseof non-speculation-tweets , such as their lingering occurrence after the airtime, longer text length,and tendency to contain hyperlinks, which were not covered in previous studies. Therefore, wewould be required to adopt different strategies for providing opportunities for catching-up users torelate to immediate reactions and speculations. To ensure that this distinction is not specific to YourTurn to Kill ( あなたの 番 です ), we further investigated speculation-based media consumptionwith different media content. Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CSCW1, Article 151. Publication date: April 2021.
Table 3. Hashtags selected and the number of tweets collected for speculation-tweets and non-speculation-tweets about
Ship of Theseus ( テセウスの 船 ). Tweet category Hashtags Number of total tweets duringMar. 15 – Mar. 16 speculation-tweets テセウスの 船 考 察 [title + “speculation”] 1,345 テセウス 考 察 [abbr. title + “speculation”] non-speculation-tweets テセウスの 船 [title] 22,416 テセウス [abbr. title] Table 4. Ratio of tweets containing hyperlinks by the types of destination for
Ship of Theseus ( テセウスの 船 ). Tweet category Hyperlink types TotalImages Other tweets YouTube videos Blogs & other articles speculation-tweets non-speculation-tweets
Ship of Theseus ( テセウスの 船 ) Next, we analyzed another recent TV series,
Ship of Theseus ( テセウスの 船 ), in the same manner.This Japanese TV series is set in a science-fiction world where the lead character travels intothe past to find clues about unresolved mysteries related to his father’s dishonor. The series wasbroadcast weekly from January 19, 2020, to March 22, 2020. As a tightly plotted story with manyepisodes foreshadowing upcoming events, we anticipated that a large number of viewers wouldparticipate in speculation about upcoming content on the Internet.Similar to the case of Your Turn to Kill ( あなたの 番 です ), we first searched tweets containingthe title of the TV show to determine the hashtags for data collection. As a result, we found asimple abbreviation of the title, “ テセウス ” [Theseus], was also frequently used in the tweets.Since we observed these hashtags were widely used for sharing immediate reactions, we selectedthem for non-speculation-tweets . Also, we found that speculation-tweets incorporated hashtagsthat appended the word “ 考 察 ” [meaning “speculation” in Japanese] to the two hashtags usedfor non-speculation-tweets . Note that these observations were consistent with the previous case(Section 4.2).Table 3 presents the hashtags used for data collection, the data collection period, and the numberof speculation-tweets and non-speculation-tweets . We selected one late episode, which was broad-cast on March 15 from 21:00 to 22:00 JST, and collected tweets from approximately one day beforeand after the episode aired.Figure 5 presents a time plot of the volume and ratio of the collected tweets on a minute basis.Here, we observed a lingering segment of speculation-tweets after the airtime, similar to thoseidentified in Figure 1 and Figure 2.A similar commonality was also presented in the ratio of tweets containing hyperlinks (Table 4).The ratio for Ship of Theseus ( テセウスの 船 ) was similar to that for Your Turn to Kill ( あなたの 番 です ). In particular, tweets containing hyperlinks to blogs and other articles were much morefrequent in speculation-tweets .Furthermore, we compared the number of characters in a tweet in the same manner as Figure 4,which yielded a result that met our expectations—namely, significantly more characters were ob-served in speculation-tweets than in non-speculation-tweets (Figure 6). Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CSCW1, Article 151. Publication date: April 2021. eaction or Speculation: Building Computational Support for Users in Catching-Up Series 151:15
Fig. 5. Time plots of the volumes of speculation-tweets and non-speculation-tweets (top: the quantity, bottom:the relative volume) for
Ship of Theseus ( テセウスの 船 ). Gray backgrounds denote the airtime. These results support the difference in the characteristics of speculation-tweets versus non-speculation-tweets described in Section 4.2 and, in particular, illustrate the unique aspects of sharing specula-tions about serial content. In other words, in the context of developing computational support forcatching-up users, different strategies should be adopted for providing opportunities to relate toimmediate reactions or speculations.
Until this point, our semi-structured interviews illustrated that engaging with speculative discus-sions not only gives a sense of being connected with others but also deepens understandings ofmedia content. Furthermore, our tweets analysis revealed distinctive characteristics in the tweetdata of reaction- and speculation-based media consumption, which suggested the need for adopt-ing different approaches to present each data to catching-up users.To lay the groundwork for discussing the possibility of computational support for catching-upusers, we prepared prototypes based on the two different approaches. We compared their effec-tiveness by replicating the situation of catching-up users in a user experiment. The first prototypeprovides immediate reactions while watching, and the second offers speculative discussions aftereach episode. In this section, we describe our procedure and the results of the user experiment.
Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CSCW1, Article 151. Publication date: April 2021.
Fig. 6. Number of characters in speculation-tweets and non-speculation-tweets about
Ship of Theseus ( テセウスの 船 ). Speculation-tweets contained significantly more characters than non-speculation-tweets ( 𝑝 < . ). To replicate the situation of catching-up users, we needed to select a TV series to present to par-ticipants. We chose
Your Turn to Kill ( あなたの 番 です ) because its suitability for speculation wasconfirmed by the number of speculation-tweets (see Section 4.2). In addition, since we had collectedtweets related to this TV show in Section 4.1, we were able to use our previously acquired data toconstruct the prototypes. Our experiment involved 18 participants aged between 20 and 52, of whom five were female. Theywere recruited via word of mouth and online communication in the same manner as described inSection 3.1. In order to be eligible, participants had to speak Japanese, since a Japanese-languageTV series was used in this experiment. At the time of recruitment, all participants self-reportedthat they had never watched
Your Turn to Kill ( あなたの 番 です ) and were not familiar with thestory. To explore possibilities of computational support for catching-up users, we prepared two differ-ent prototypes: live-tweets and speculation display. The first displayed tweets posted during theoriginal broadcast in relation to playback time within the episode, similar to Danmaku [66], aswe mentioned in Section 1 and Section 2.3. Since we confirmed that tweets associating with suchimmediate reactions were concentrated around the airtime (Section 4), this design would be rea-sonable. In addition, given the effect of Danmaku [39] and Twitter-based social viewing experience
Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CSCW1, Article 151. Publication date: April 2021. eaction or Speculation: Building Computational Support for Users in Catching-Up Series 151:17 [52], we could expect that participants would appreciate the immediate reactions when using thisprototype while watching the episode.The second prototype was intended to induce the positive effects of engaging with speculativediscussions described in Section 3.3. Here, since speculative discussions often have long sentencesor hyperlinks to external content (Section 4), a Danmaku-based interface would not be optimal.We thus constructed a unique web page for each episode that displayed speculation-tweets postedbetween the broadcast of that episode and the following episode. This post-time-based data col-lection is intended to eliminate the risk of spoilers, as discussed in Section 3.3.3. The tweets wererandomly arranged on the web page. We considered that participants interact with this page afterthey finished watching each episode, expecting that they would engage with speculation. This de-sign is analogous to our findings in Section 4; speculative discussions on Twitter become active atthe end of and after the airtime.The interfaces of the two prototypes are presented in Figure 7. All tweets were displayed usingthe embedding function of Twitter so that participants can preview attached images or videos andfollow hyperlinks in the tweets regarding both prototypes, though non-speculation-tweets were lesslikely to contain such attachments according to our tweets analysis in Section 4. Here, differentfrom some previous proposals of second-screen applications, which we mentioned in Section 2.3,we designed these prototypes to be used in the main screen. This is because some participantstook part in this experiment remotely, as we mention later, and we could not guarantee that allof them can prepare similar second-screen devices. Instead, all participants, including them, usedthese prototypes with a laptop.
To evaluate the effect of the prototypes, we set up measures corresponding to our design strategies.Our measures were guided by Fang et al. [19], who conducted a questionnaire-based user study toreveal how Danmaku interactions contributed to the enhancement of viewing experiences.We adopted their questionnaire sets to measure social presence (based on [24, 48, 70]) and utili-tarian value (based on [28, 62, 69]). Because social presence is elicited by the feeling of being con-nected with others [19, 70], we expected that it can be enhanced not only by the live-tweets displaybut also by the speculation display, especially considering the findings described in Section 3.3.2.On the other hand, utilitarian value refers to participants’ perceived benefits from some systems[19, 69]. We thus expected that it is more related to the speculation display because, as we men-tioned in Section 3.3.1, the effect of deepening understandings of media content would be specificto speculation-related activities.In addition, we followed Fang et al. [19] to evaluating the quality of such experiences by mea-suring users’ e-loyalty intention, i.e., their revisit intention and positive word-of-mouth referrals(based on [45, 69]). If our prototypes contribute to enhancing participants’ consumption expe-riences, the e-loyalty intention would be increased, as they confirmed in the case of Danmakuinteractions. Here, the reason why we used this measure is that its prospective aspect has an affin-ity with speculation, as we mentioned that speculation is an ongoing experience based on a formof seriality in Section 2.2.2. In other words, other measures that require retrospective evaluation,such as immersion [51], would not fully reflect the effect of the speculation display and not optimalfor discussing the pros and cons of the two prototypes.In summary, from Fang et al.’s questionnaire [19], we adopted the same five items to measuresocial presence, four to measure utilitarian value, and four to measure e-loyalty intention, all ofwhich were rated on a 7-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree). The usernames and icons are grayed out for anonymity.Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CSCW1, Article 151. Publication date: April 2021. (cid:127)€(cid:129)‚(cid:129)ƒ„…†‡ˆ(cid:129)‰…‰ƒ…„Š‡‚…‹€(cid:129)‹ ‹!…"‡†(cid:129)
Fig. 7. Two prototypes presented to participants in our experiment (a: displays tweets in relation to aplayback time within the episode; b: displays tweets containing speculative discussions after watching theepisode). We note that these items were validated in their original studies, and thus, we analyzed the col-lected scores using the standard procedure involving ANOVA, as we describe later in Section 5.6.1.
The experiment was conducted in a quiet room with a laptop we prepared, except for nine partici-pants who took part in remotely. For the remote participants, we sent URLs to the prototypes andinstructed them to access the URLs using their laptops in accordance with the experimental steps.Our experiment followed a within-participant design with three conditions: two prototypes andno intervention. Participants watched the first through the third episode of
Your Turn to Kill ( あなたの 番 です ) in order with each condition. The order of conditions was balanced across par-ticipants to avoid the influence of order effects or the content of each episode. After participantsfinished watched each episode, they completed the questionnaire described above, except in thecase of the speculation display (Figure 7b). In this case, we asked participants to freely explorethe speculative discussions on the page to their satisfaction before filling out the questionnaire. Inaddition, in the case of no intervention, we omitted the measure for utilitarian value in the samemanner as Fang et al. [19] because it was not applicable. After participants watched the three Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CSCW1, Article 151. Publication date: April 2021. eaction or Speculation: Building Computational Support for Users in Catching-Up Series 151:19
Table 5. Result of a two-way ANOVA in participants’ responses to measures for social presence.
Effect F df 𝑝 -valueEpisode 0.623 2 0.541Condition 11.413 2 < . × Condition 0.385 4 0.818
Table 6. Result of a two-way ANOVA in participants’ responses to measures for utilitarian value.
Effect F df 𝑝 -valueEpisode 1.886 2 0.310Condition 18.778 1 0.001Episode × Condition 3.679 2 0.108
Fig. 8. Participants’ responses to measures for social presence and utilitarian value. episodes and filled out the questionnaire, we asked them for their comments about their consump-tion experiences and the overall experimental process, which lasted on average approximately 3.5hours.
Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CSCW1, Article 151. Publication date: April 2021.
Table 7. Result of a two-way ANOVA in participants’ responses to measures for e-loyalty intention.
Effect F df 𝑝 -valueEpisode 7.337 2 0.002Condition 1.463 2 0.242Episode × Condition 3.976 4 0.008
For social presence, we confirmedthat Levene’s test did not found a significant difference in its variance ( 𝑝 = . 𝑝 < . 𝑝 = . 𝑝 < . 𝑝 = . 𝑝 = . 𝑝 = . 𝑝 = . 𝑝 = . 𝑝 = . 𝑝 = . 𝑝 = . Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CSCW1, Article 151. Publication date: April 2021. eaction or Speculation: Building Computational Support for Users in Catching-Up Series 151:21
Fig. 9. Participants’ responses to measures for e-loyalty intention. computational support can enhance the consumption experiences of catching-up users at leastwhen the content has not fully attracted them yet.
As shown above, our quantitative results demonstrated the distinctive ef-fectiveness of our two prototypes. In this section, we review the comments obtained from partici-pants during the experiment to further explore our results.All participants approved our prototypes and described their benefits, e.g., enhanced feelingsof watching together with peers through the live-tweets display and a deepened understanding ofthe show’s content through the speculation display:It was nice to be able to see how other people were watching at it and feel as if wewere watching it together.Watching the speculation for the next episode enabled me to appreciate the contentfrom various perspectives. What’s more, it helped me to deeply understand each ofthe many characters that appeared in the story.These comments align with the existing literature on live-tweets (Section 2.2.1) and our findingson speculation (Section 3.3). Therefore, in combination with the results presented in Section 5.6.1,we conclude that our prototypes enhanced participants’ consumption experience, which suggeststhe potential of computational support for catching-up users.In addition, some participants offered suggestions for improving the prototypes. Although thefeeling of being connected with others enabled by the live-tweets display was favored by many
Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CSCW1, Article 151. Publication date: April 2021. participants, five participants mentioned a desire to control the content presented on the displayto some extent:Sometimes I felt it a bit annoying that most of the tweets were about the coolnessof the actor of the protagonist. I wanted to focus on the story by suppressing thosetweets.The live-tweets interface was not bad, but I would like to not see lots of reaction-liketweets such as “oops” and “lol,” but rather some meaningful tweets like “I’m sure thisperson is lying because. . . ” which can help me to think about the story more.These comments suggest a demand for the ability to customize the live-tweets display based oncontent. Such add-on functionality could be enabled by analyzing tweet data and applying cluster-ing techniques.Interestingly, two participants mentioned experiencing a dilemma between their desires to watchthe next episode and to enjoy the speculation content:I really enjoyed the diverse speculations, some of which I had never imagined. How-ever, the more articles I read, the more I was tempted to stop reading them and towatch the next episode to see which one would be correct.These comments are unique to catching-up users, who can binge watch an entire series [41, 50]without exposing themselves to speculations after each episode. Therefore, it is desirable to con-sider this unique predicament when we design applications for such users. We discuss severalpossible options for addressing this point in the next section.Another interesting comment was provided by one participant, who said:It is fun to go through so many different speculations. It would be even better if I couldtell my opinion to others.It would be challenging to meet this demand in catch-up situations, as our prototypes cannot ad-dress it. In other words, while our prototypes enhanced the consumption experiences of catching-up users, further technological developments would be desirable to close the gap in the experienceof watching simultaneously with other users at the time of broadcast.
So far, we have revealed that engaging with online speculative discussions can contribute to deep-ening understandings of media content through both the semi-structured interviews (Section 3.3.1)and the user experiment (Section 5.6). In particular, the comments in the experiment (Section 5.6.2)suggested that the participants obtained perspectives beyond the individual by appreciating spec-ulations posted by various Internet users. We would like to say that this structure can be alignedwith collective intelligence [38].In fact, Jenkins’s early observation [27] of the online speculative discussions of TV series pointedout the role of a fan community as collective intelligence. We note that our observation was dif-ferent from his discussion in terms of the lack of close-knit communities (e.g., a fan wiki), whichcan be attributed to the advent of social networking services. However, the structure that peoplewho relate to speculation exploit the Internet to search or join discussions about a sequel is com-mon. Also, in the same manner as Jenkins [27] depicted, we observed that the well-consideredspeculations are shared by many users on the Internet and trigger further discussions on Twitter.In this context, our prototype leveraging speculations can be understood to be served as an in-terface for harnessing collective intelligence. That is, collecting related opinions from the Internetand displaying them is one of the basic approaches to leverage collective intelligence [22]. Given
Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CSCW1, Article 151. Publication date: April 2021. eaction or Speculation: Building Computational Support for Users in Catching-Up Series 151:23 that, as many researchers have proposed sophisticated interfaces for harnessing collective intelli-gence [2, 61], we anticipate that our prototype for supporting catching-up users could have beenimproved based on their designs. Yet, as speculation is not only for deepening understandings ofmedia content (Section 3.3), there is room for exploring how we can incorporate existing tech-niques for collective intelligence in terms of enhancing the consumption experiences of catching-up users based on speculation.
Through the user experiment with our two prototypes, we confirmed that engaging with imme-diate reactions and speculative discussions are both beneficial to catching-up users, albeit withdifferent effects. This result suggests potential application scenarios for supporting catching-upusers. For example, catch-up TV services might leverage the combined effect of both approachesby offering users an interface to see live-tweets while they watch content and, after watching theepisode, guiding them to explore speculative content before starting to watch the next episode. Inaddition, sending a notification to a user who has stopped watching a series on an episode in themiddle of a series with a link to a discussion article speculating about the consequent episodes canstimulate the user’s desire to watch the next episode.In addition to enabling catching-up users to see live-tweets and speculations posted by otherusers who watched the episode in sync with the broadcast, we can design and develop interac-tions that only catching-up users can appreciate. Inspired by the recent proposals for story-basedretrievals from TV shows [57, 71] and comics [37, 42] or annotations [16, 49], we suggest that itmay be helpful to guide catching-up users to be attentive by showing them a subtle alert duringscenes about which others have frequently speculated. This can be identified through an analysisof speculation-tweets . We acknowledge that such guidance can simply be prepared by the pro-ducer of the content without online speculative discussions. However, as Gray and Mittell [21]noted, speculation is often regarded as a “game” between viewers and producers. As such, viewersmay refuse spoilers that are officially released by producers. In this way, such interactions couldeffectively enhance the consumption experiences of catching-up users without spoiling them.In addition, this type of interaction could remedy the dilemma described in Section 5.6.2. Throughleveraging the automatically retrieved relationships between scenes and speculations, it is possibleto present speculations in the same manner as live-tweets. Catching-up users can then enjoy bothwatching the content and speculating about it simultaneously and thus would not need to spendtime to explore speculations between the intervals of the episodes. Further studies are required todesign and evaluate such interactions so that users can appreciate speculations without significantcognitive load while watching.
Despite the promising applications of leveraging speculation to support catching-up users, oursemi-structured interviews identified a potential problem: the risk of encountering spoilers (de-scribed in Section 3.3.3). In this respect, many studies have examined spoiler detection and relatedinteraction techniques [3, 20, 23, 46, 67]. For example, Guo et al. [23] proposed a method to detectspoiling movie reviews using latent Dirichlet allocation, while Golbeck [20] demonstrated howsimple keyword-based filtering using the name of actors or sports players could effectively blocklive-tweets containing spoilers, although the method’s precision was poor. Yang et al. [67] pro-posed a spoiler detection method specifically designed for Danmaku comments that relied on acomment’s similarity to comments made during the climax period, i.e., peak volume.However, we argue that these methods are inadequate for supporting catching-up users. Asdiscussed in Section 3.3, the motivations behind this phenomenon of speculation are related both
Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CSCW1, Article 151. Publication date: April 2021. to social presence and to gain a deeper understanding of media content. As such, any design thatignores false positives (i.e., that filters non-spoiler information) would not desirable, as it wouldrun the risk of missing a large amount of information important for deepening the understanding.In addition, a criterion for filtering spoilers should be made dynamic: Depending on which episodethe user has finished watching, the criteria should change so that as much speculation as possibleis provided about the story prior to the episode, while eliminating any information to be revealedin later episodes.We note that Jones et al. [30] proposed a method to avoid spoilers on a fan wiki that alloweda user to specify an episode and regarded information disclosed before that episode as known bythe user in advance. However, this method relies on the Memento extension [13] for the HTTPprotocol, which is not yet globally available. In addition, fan wikis are not always available forvarious media.Therefore, we conclude that further developments to address the problem of spoilers are re-quired to offer better consumption experiences to catching-up users. For example, if we can as-sume that our data have reliable timestamps, the simple approach of filtering based on airtimewould suffice, as in Section 5.3. Alternatively, it may be possible to determine topics that appearedaround the airtime of an episode by employing methods for detecting emerging topics [7, 35]. Fil-tering such topics for catching-up users who have not yet watched the episode would help themavoid spoilers.
Although our results shed light on the unique characteristics of speculation in contemporary me-dia consumption and confirmed the positive effects of our prototypes for catching-up users, theparticipants in this study were mainly from East Asian cultures. In addition, the fact that our ex-periment involved participants who took part remotely, as mentioned in Section 5.5, would raisesome concern on the ecological validity, as it eliminated the use of second-screen devices. To gen-eralize our results, further investigations involving participants in other cultures and consideringvarious use cases are desirable.At the same time, seeking ways to support speculations for diverse media content, not onlyTV series from outside of East Asian countries but also other media types, would be interesting.For example, considering the importance of books as a form of media, future studies might de-velop a prototype to enhance readers’ experiences of catching up on series of comics or novels bydesigning how catching-up readers engage with speculations.Another limitation is related to data collection—namely, that our proposed two prototypes reliedon the assumption that there were enough live-tweets and speculations on Twitter to entertainusers. This assumption may not always hold true, given that there is a wide range of media content,including ones not so popular. However, especially for speculation, its source is not limited toTwitter but also includes other social networks, such as YouTube, blogs, and Instagram, as observedin Section 3.3 and Section 4. This would allow us to gather speculations to be presented in otherways. Therefore, future work is demanded to establish such an approach while taking into accountthe spoiler problem discussed in Section 6.3.
In this paper, we focused on the increase in the number of catching-up users and the emergence ofa new media consumption experience centering on speculation, elucidating new approaches to en-hancing the experiences of the catching-up users. We first conducted semi-structured interviews tofigure out how people are engaging with speculation during media consumption, which revealedthat two positive effects (i.e., deepening understanding and sense of connectedness with others) as
Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CSCW1, Article 151. Publication date: April 2021. eaction or Speculation: Building Computational Support for Users in Catching-Up Series 151:25 well as the possible risk of encountering spoilers. We then conducted a quantitative analysis usingpublic tweet data to provide background for discussing computational supports for catching-upusers, which illustrated the unique aspects of speculation-tweets , including their timing, length,and content. Finally, we performed a user experiment to evaluate the effect of two different ap-proaches based on immediate reactions and speculations, which demonstrated their effectivenessin enhancing consumption experiences. Our results and discussions lay the groundwork for provid-ing catching-up users with computational support, particularly by leveraging speculation, whichhas not been well explored to date despite its effectiveness.
ACKNOWLEDGMENTS
This work was partially supported by JST ACT-X, Grant Number JPMJAX200R, Japan.
REFERENCES [1] Jorge Abreu, João Nogueira, Valdecir Becker, and Bernardo Cardoso. 2016. Survey of Catch-up TV and other time-shift services: a comprehensive analysis and taxonomy of linear and nonlinear television.
Telecommunication Systems
64, 1 (2016), 57–74. https://doi.org/10.1007/s11235-016-0157-3[2] Antonietta Grasso and Gregorio Convertino. 2012. Collective Intelligence in Organizations: Tools and Studies –Introduction.
Computer Supported Cooperative Work
21 (2012), 357–369. https://doi.org/10.1007/s10606-012-9165-3[3] Jordan Boyd-Graber, Kimberly Glasgow, and Jackie Sauter Zajac. 2013. Spoiler Alert: Machine Learning Approaches toDetect Social Media Posts with Revelatory Information. In
Proceedings of the 76th ASIS&T Annual Meeting . Associationfor Information Science and Technology, Silver Spring, MD, 1–9. https://doi.org/10.1002/meet.14505001073[4] Axel Bruns and Jean Burgess. 2011.
Commu-nication, Politics and Culture
44, 2 (2011), 37–56.[5] Axel Bruns, Jean Burgess, Kate Crawford, and Frances Shaw. 2012. . Technical Report. ARC Centre of Excellence for Creative Industriesand Innovation, Queensland University of Technology.[6] Rhiannon Bury and Johnson Li. 2013. Is it live or is it timeshifted, streamed or downloaded? Watching television inthe era of multiple screens.
New Media & Society
17, 4 (2013), 592–610. https://doi.org/10.1177/1461444813508368[7] Mario Cataldi, Luigi Di Caro, and Claudio Schifanella. 2010. Emerging Topic Detection on Twitter Based on Temporaland Social Terms Evaluation. In
Proceedings of the 10th International Workshop on Multimedia Data Mining . ACM, NewYork, NY, Article 4, 1–10 pages. https://doi.org/10.1145/1814245.1814249[8] Pablo Cesar and David Geerts. 2011. Past, Present, and Future of Social TV: A Categorization. In
Proceed-ings of the 2011 IEEE Consumer Communications and Networking Conference . IEEE, New York, NY, 347–351.https://doi.org/10.1109/CCNC.2011.5766487[9] Pablo Cesar and David Geerts. 2011. Understanding Social TV: A Survey. In
Proceedings of the 2011 Networked andElectronic Media Summit . NEM Initiative, Heidelberg, Germany, 27–29.[10] Pablo Cesar, David Geerts, and Konstantinos Chorianopoulos (Eds.). 2009.
Social Interactive Television . IGI Global,Hershey, PA. https://doi.org/10.4018/978-1-60566-656-3[11] Yue Chen, Qin Gao, and Pei-Luen Patrick Rau. 2017. Watching a Movie Alone yet Together: Understanding Rea-sons for Watching Danmaku Videos.
International Journal on Human–Computer Interaction
33, 9 (2017), 731–743.https://doi.org/10.1080/10447318.2017.1282187[12] Buschow Christopher, Schneider Beate, and Ueberheide Simon. 2014. Tweeting Television: Explor-ing Communication Activities on Twitter while Watching TV.
Communications
39 (2014), 129–149.https://doi.org/10.1515/commun-2014-0009[13] Herbert Van de Sompel, Michael Lloyd Nelson, and Robert Sanderson. 2013. RFC 7089: HTTP Framework for Time-Based Access to Resource States – Memento. Internet Engineering Task Force, http://tools.ietf.org/rfc/rfc7089.txt.Accessed: January 2, 2020.[14] Chrysanthos Dellarocas and Ritu Narayan. 2006. What Motivates Consumers to Review a Product Online? A Studyof the Product-Specific Antecedents of Online Movie Reviews. In
Proceedings of the 17th Workshop on InformationSystems and Economics . WISE, Evanston, IL, 1–6.[15] Ruth Deller. 2011. Twittering on: Audience Research and Participation Using Twitter.
Participations
8, 1 (2011), 216–245.[16] Mike Dowman, Valentin Tablan, Hamish Cunningham, and Borislav Popov. 2005. Web-Assisted Annotation, SemanticIndexing and Search of Television and Radio News. In
Proceedings of the 14th International Conference on World WideWeb . ACM, New York, NY, 225–234. https://doi.org/10.1145/1060745.1060781Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CSCW1, Article 151. Publication date: April 2021. [17] Nicolas Ducheneaut, Robert Moore, Lora Oehlberg, James Thornton, and Eric Nickell. 2008. Social TV: Designing forDistributed, Sociable Television Viewing.
International Journal of Human–Computer Interaction
24, 2 (2008), 136–154.https://doi.org/10.1080/10447310701821426[18] Brianna Dym and Casey Fiesler. 2018. Vulnerable and Online: Fandom’s Case for Stronger Privacy Norms and Tools.In
Companion of the 2018 ACM Conference on Computer Supported Cooperative Work and Social Computing . ACM, NewYork, NY, 329–332. https://doi.org/10.1145/3272973.3274089[19] Jiaming Fang, Lei Chen, Chao Wen, and Victor Ronald Prybutok. 2018. Co-viewing Experience in Video Web-sites: The Effect of Social Presence on E-Loyalty.
International Journal of Electronic Commerce
22, 3 (2018), 446–476.https://doi.org/10.1080/10864415.2018.1462929[20] Jennifer Golbeck. 2012. The Twitter Mute Button: A Web Filtering Challenge. In
Proceedings of the2012 ACM CHI Conference on Human Factors in Computing Systems . ACM, New York, NY, 2755–2758.https://doi.org/10.1145/2207676.2208673[21] Jonathan Gray and Jason Mittell. 2007. Speculation on Spoilers: Lost Fandom, Narrative Consumption and RethinkingTextuality.
Participations
4, 1 (2007), 1–35.[22] Dawn Greayer. Gregg. 2010. Designing for collective intelligence.
Commun. ACM
53, 4 (2010), 134–138.https://doi.org/10.1145/1721654.1721691[23] Sheng Guo and Naren Ramakrishnan. 2010. Finding the Storyteller: Automatic Spoiler Tagging Using LinguisticCues. In
Proceedings of the 23rd International Conference on Computational Linguistics . International Committee onComputational Linguistics, Beijing, China, 412–420.[24] Khaled Hassanein and Milena Head. 2007. Manipulating perceived social presence through the web interface and itsimpact on attitude towards online shopping.
International Journal of Human–Computer Studies
65, 8 (2007), 689–708.https://doi.org/10.1016/j.ijhcs.2006.11.018[25] Libby Hemphill and Jahna Otterbacher. 2012. Learning the Lingo?: Gender, Prestige and Linguistic Adaptation inReview Communities. In
Proceedings of the 2012 ACM Conference on Computer Supported Cooperative Work . ACM,New York, NY, 305–314. https://doi.org/10.1145/2145204.2145254[26] Serena Hillman, Jason Procyk, and Carman Neustaedter. 2014. ’alksjdf;Lksfd’: Tumblr and the Fandom User Expe-rience. In
Proceedings of the 2014 ACM conference on Designing Interactive Systems . ACM, New York, NY, 775–784.https://doi.org/10.1145/2598510.2600887[27] Henry Jenkins. 2006.
Convergence Culture: Where Old and New Media Collide . NYU Press, New York, NY.[28] Arun Rai JJ Po-An Hsieh and Mark Keil. 2008. Understanding Digital Inequality: Comparing Continued Use BehavioralModels of the Socio-Economically Advantaged and Disadvantaged.
Management Information Systems Quarterly
32, 1(2008), 97–126. https://doi.org/10.2307/25148830[29] Daniel Johnson. 2013. Polyphonic/Pseudo-synchronic: Animated Writing in the Comment Feed of Nicovideo.
Japan-ese Studies
33, 3 (2013), 297–313. https://doi.org/10.1080/10371397.2013.859982[30] Shawn Morgan Jones, Michael Lloyd Nelson, and Herbert Van de Sompel. 2018. Avoiding Spoilers:Wiki Time Travel with Sheldon Cooper.
International Journal on Digital Libraries
19, 1 (2018), 77–93.https://doi.org/10.1007/s00799-016-0200-8[31] Shalini Kalia. 2014. DVR and Its Impact on Indian Market.
SAGE Open
4, 4 (2014), 215824401456055.https://doi.org/10.1177/2158244014560551[32] Jihyun Kim, Kelly Merrill Jr., and Hocheol Yang. 2019. Why We Make the Choices We Do: Social TV View-ing Experiences and the Mediating Role of Social Presence.
Telematics and Informatics
45, 101281 (2019), 11.https://doi.org/10.1016/j.tele.2019.101281[33] Jae Won Kim, Dongwoo Kim, Brian Keegan, Joon Hee Kim, Suin Kim, and Alice Oh. 2015. Social Media Dynamicsof Global Co-presence During the 2014 FIFA World Cup. In
Proceedings of the 33rd ACM CHI Conference on HumanFactors in Computing Systems . ACM, New York, NY, 2623–2632. https://doi.org/10.1145/2702123.2702317[34] Takashi Kimura. 2019. The Hit of
Your Turn To Kill
Shows the Future of TV Series (in Japanese). Toyo Keizai,https://toyokeizai.net/articles/-/301578. Accessed: December 28, 2019.[35] April Kontostathis, Leon Galitsky, William Morton Pottenger, Soma Roy, and Daniel Phelps. 2004. A Survey of Emerg-ing Trend Detection in Textual Data Mining. In
Survey of Text Mining: Clustering, Classification, and Retrieval , MichaelBerry (Ed.). Springer, New York, NY, 185–224. https://doi.org/10.1007/978-1-4757-4305-0_9[36] Mercedes Medina Laverón, Mónica Herrero Subías, and Cristina Etayo-Pérez. 2015. The impact of digital-ization on the strategies of pay TV in Spain.
Revista Latina de Comunicación Social
70 (2015), 252–269.https://doi.org/10.4185/rlcs-2015-1045en[37] Thanh-Nam Le, Muhammad Muzzamil Luqman, Jean-Christophe Burie, and Jean-Marc Ogier. 2015. Content-Based Comic Retrieval Using Multilayer Graph Representation and Frequent Graph Mining. In
Proceedingsof the 13th International Conference on Document Analysis and Recognition . IEEE, New York, NY, 761–765.https://doi.org/10.1109/ICDAR.2015.7333864Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CSCW1, Article 151. Publication date: April 2021. eaction or Speculation: Building Computational Support for Users in Catching-Up Series 151:27 [38] Pierre Levy and Robert Bononno. 1997.
Collective Intelligence: Mankind’s Emerging World in Cyberspace . PerseusBooks, Cambridge, MA.[39] Joon Soo Lim, YoungChan Hwang, Seyun Kim, and Frank Biocca. 2015. How Social Media Engagement Leads to SportsChannel Loyalty: Mediating Roles of Social Presence and Channel Commitment.
Computers in Human Behavior
The Annals of the American Academy of Political and Social Science
Jeunesse:Young People, Texts, Cultures
6, 1 (2014), 119–138. https://doi.org/10.1353/jeu.2014.0002[42] Yusuke Matsui, Kota Ito, Yuji Aramaki, Azuma Fujimoto, Toru Ogawa, Toshihiko Yamasaki, and Kiyoharu Aizawa.2017. Sketch-Based Manga Retrieval Using Manga109 Dataset.
Multimedia Tools and Applications
76, 20 (2017), 21811–21838. https://doi.org/10.1007/s11042-016-4020-z[43] Marshall McLuhan. 1964.
Understanding Media: The Extensions of Man . McGraw Hill, New York, NY.[44] Jason Mittell. 2009. Sites of Participation: Wiki Fandom and the Case of Lostpedia.
Transformative Works and Cultures
International Journalof Electronic Commerce
17, 4 (2013), 7–38. https://doi.org/10.2753/jec1086-4415170401[46] Satoshi Nakamura and Takanori Komatsu. 2012. Study of Information Clouding Methods to Prevent Spoilers of SportsMatch. In
Proceedings of the 2012 International Working Conference on Advanced Visual Interfaces . ACM, New York, NY,661–664. https://doi.org/10.1145/2254556.2254679[47] Abhishek Nandakumar and Janet Horowitz Murray. 2014. Companion Apps for Long Arc TV Series: SupportingNew Viewers in Complex Storyworlds with Tightly Synchronized Context-Sensitive Annotations. In
Proceedings ofthe 2014 ACM International Conference on Interactive Experiences for TV and Online Video . ACM, New York, NY, 3–10.https://doi.org/10.1145/2602299.2602317[48] Carol Xiaojuan Ou, Paul Pavlou, and Robert Davison. 2014. Swift Guanxi in Online Marketplaces: The Role ofComputer-Mediated Communication Technologies.
Management Information Systems Quarterly
38, 1 (2014), 209–230. https://doi.org/10.25300/misq/2014/38.1.10[49] Moisés Henrique Ramos Pereira, Celso Luiz de Souza, Flávio Luis Cardeal Pádua, Giani David Silva, Guilherme Tavaresde Assis, and Adriano César Machado Pereira. 2015. SAPTE: A Multimedia Information System to Support the Dis-course Analysis and Information Retrieval of Television Programs.
Multimedia Tools and Applications
74, 23 (2015),10923–10963. https://doi.org/10.1007/s11042-014-2311-9[50] Matthew Pittman and Kim Sheehan. 2015. Sprinting a Media Marathon: Uses and Gratifications of Binge-WatchingTelevision Through Netflix.
First Monday
20, 10 (2015). https://doi.org/10.5210/fm.v20i10.6138[51] Jacob Mark Rigby, Duncan Brumby, Sandy Gould, and Anna Louise Cox. 2019. Development of a Questionnaireto Measure Immersion in Video Media: The Film IEQ. In
Proceedings of the 2019 ACM International Conference onInteractive Experiences for TV and Online Video . ACM, New York, NY, 35–46. https://doi.org/10.1145/3317697.3323361[52] Steven Schirra, Huan Sun, and Frank Bentley. 2014. Together Alone: Motivations for Live-Tweeting a TelevisionSeries. In
Proceedings of the 32nd ACM CHI Conference on Human Factors in Computing Systems . ACM, New York, NY,2441–2450. https://doi.org/10.1145/2556288.2557070[53] Andreas Schneider, Georg von Krogh, and Peter Jäger. 2013. “What’s coming next?” Epistemic Curios-ity and Lurking Behavior in Online Communities.
Computers in Human Behavior
Pro-ceedings of the 2015 ACM International Conference on Interactive Experiences for TV and Online Video . ACM, New York,NY, 167–172. https://doi.org/10.1145/2745197.2755519[56] Anselm Strauss and Juliet Corbin. 1990.
Basics of Qualitative Research: Grounded Theory Procedures and Techniques .Sage Publications, Newbury Park, CA.[57] Makarand Tapaswi, Martin Bäuml, and Rainer Stiefelhagen. 2015. Aligning Plot Synopses to Videosfor Story-Based Retrieval.
International Journal of Multimedia Information Retrieval
4, 1 (2015), 3–16.https://doi.org/10.1007/s13735-014-0065-9[58] Reiko Uehara. 2019. A Music Video of the Theme Song of
Your Turn To Kill
December 28, 2019.[59] Jeroen Vanattenhoven and David Geerts. 2015. Broadcast, Video-on-Demand, and Other Ways to Watch TelevisionContent. In
Proceedings of the 2015 ACM International Conference on Interactive Experiences for TV and Online . ACM,New York, NY, 73–82. https://doi.org/10.1145/2745197.2745208[60] Radu-Daniel Vatavu. 2015. Audience Silhouettes: Peripheral Awareness of Synchronous Audience Kinesics for SocialTelevision. In
Proceedings of the 2015 ACM International Conference on Interactive Experiences for TV and Online Video .ACM, New York, NY, 13–22. https://doi.org/10.1145/2745197.2745207[61] Xuan-Truong Vu, Pierre Morizet-Mahoudeaux, and Marie-Hélène Abel. 2013. Empowering Collaborative Intelligenceby the Use of User-Centered Social Network Aggregation. In
Proceedings of the 2013 IEEE/WIC/ACM InternationalConferences on Web Intelligence . IEEE, New York, NY, 425–430. https://doi.org/10.1109/WI-IAT.2013.60[62] Yi-Shun Wang, Ching-Hsuan Yeh, and Yi-Wen Liao. 2013. What drives purchase intention in the context of onlinecontent services? The moderating role of ethical self-efficacy for online piracy.
International Journal of InformationManagement
33, 1 (2013), 199–208. https://doi.org/10.1016/j.ijinfomgt.2012.09.004[63] Michael Wayne. 2018. Netflix, Amazon, and branded television content in subscription video on-demand portals.
Media, Culture & Society
40, 5 (2018), 725–741. https://doi.org/10.1177/0163443717736118[64] Kenneth Wilbur. 2008. How the Digital Video Recorder (DVR) Changes Traditional Television Advertising.
Journalof Advertising
37, 1 (2008), 143–149. https://doi.org/10.2753/joa0091-3367370111[65] Donghee Yvette Wohn and Eun-Kyung Na. 2011. Tweeting about TV: Sharing Television Viewing Experiences viaSocial Media Message Streams.
First Monday
16, 3 (2011). https://doi.org/10.5210/fm.v16i3.3368[66] Qunfang Wu, Yisi Sang, and Yun Huang. 2019. Danmaku: A New Paradigm of Social Interaction via Online Videos.
ACM Transactions on Social Computing
2, 2 (2019), 7:1–7:24. https://doi.org/10.1145/3329485[67] Wenmian Yang, Weijia Jia, Wenyuan Gao, Xiaojie Zhou, and Yutao Luo. 2019. Interactive Variance Attention basedOnline Spoiler Detection for Time-Sync Comments. In
Proceedings of the 28th ACM International Conference on Infor-mation and Knowledge Management . ACM, New York, NY, 1241–1250. https://doi.org/10.1145/3357384.3357872[68] Kei Yokoyama. 2019. The hottest TV series
Your Turn To Kill create a new speculation
Journal of Management Information Systems
29, 1(2012), 273–306. https://doi.org/10.2753/mis0742-1222290108[70] Zhimin Zhou, Chenting Su, Nan Zhou, and Ning Zhang. 2016. Becoming Friends in Online BrandCommunities: Evidence From China.
Journal of Computer-Mediated Communication
21, 1 (2016), 69–86.https://doi.org/10.1111/jcc4.12147[71] Yukun Zhu, Ryan Kiros, Richard Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fi-dler. 2015. Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and ReadingBooks. In
Proceedings of thee 2015 IEEE International Conference on Computer Vision . IEEE, New York, NY, 19–27.https://doi.org/10.1109/ICCV.2015.11