Election 2016: Tracking Emotions with R and Python

Temperament has been a key issue in the 2016 presidential election between Hillary Clinton and Donald Trump, and an issue highlighted in the series of three debates that concluded this week. Quantifying “temperament” isn’t an easy task, but The Economist used the Microsoft Emotion API to chart the anger, contempt, sadness and surprised expressed in the faces of the candidates during key sequences of the debates, like this from the third debate: < embed src="https://c.brightcove.com/services/viewer/federated_f9?isVid=1" bgcolor="#FFFFFF" flashvars="videoId=5178598371001&linkBaseURL=http%3A%2F%2Fecon.st%2F2e4LAfm&playerID=1425961410001&playerKey=AQ~~,AAABDH-R__E~,dB4S9tmhdOo20g03jDsDgNBGDcclfHEU&domain=embed&dynamicStreaming=true" base="https://admin.brightcove.com" name="flashObj" width="450" seamlesstabbing="false" type="application/x-shockwave-flash" allowfullscreen="allowfullscreen" swliveconnect="true" allowscriptaccess="always" pluginspage="https://www.macromedia.com/shockwave/download/index.cgi?P1_Prod_Version=ShockwaveFlash" /> Economist Data Journalist Ben Heubl explains how you can analyze emotions in a video file using Python and R. The Emotion API provides scores for eight attributes of emotion as expressed by a face in a still image or video clip. For example, this expression by Donald Trump expresses mostly anger, with a touch of disgust and a soupçon of contempt. Ben provides Python code for passing a video clip into the Emotion API and retriving frame-by-frame emotion scores. He then uses R to analyze and chart the scores: mostly happiness for Clinton; mostly sadness for Trump. Ben points out that emotion and sentiment analysis can be useful tools for data journalists, noting for example:

Facial recognition data, as the one that we collected via the Microsoft emotion API, suggests that when Trump is challenged with arguments which he doesn’t like, his facial attitude changes. In the second debate, Clinton provoked him with mentioning his involvement in degrading a former Miss Universe. His expression promptly changed to a mixture of an angry and sad expression.

For more analysis of emotion in the debates, including a text-based sentiment analysis of the debate transcript, check out Ben’s complete blog post linked before. And be sure to check out the original article about the debates from The Economist. Ben Heubl: How to apply face recognition API technology to data journalism with R and python