BEER EXPERTS AND SENSORY EXPERIENCES: THE CASE OF REVIEW WRITING

Transcription

1 BEER EXPERTS AND SENSORY EXPERIENCES: THE CASE OF REVIEW WRITING LEXICAL ANALYSIS OF AROMA AND FLAVOUR DESCRIPTIONS AND EVALUATION OF CONSISTENCY AND INFORMATION USABILITY OF EXPERT BEER REVIEWS Aantal woorden: 14,037 Liesbeth Allein Studentennummer: Promotor: Prof. dr. Els Lefever Masterproef voorgelegd voor het behalen van de graad master in de richting meertalige communicatie: talencombinatie Nederlands, Engels, Duits Academiejaar:

2 Verklaring i.v.m. auteursrecht De auteur en de promotor(en) geven de toelating deze studie als geheel voor consultatie beschikbaar te stellen voor persoonlijk gebruik. Elk ander gebruik valt onder de beperkingen van het auteursrecht, in het bijzonder met betrekking tot de verplichting de bron uitdrukkelijk te vermelden bij het aanhalen van gegevens uit deze studie. Het auteursrecht betreffende de gegevens vermeld in deze studie berust bij de promotor(en). Het auteursrecht beperkt zich tot de wijze waarop de auteur de problematiek van het onderwerp heeft benaderd en neergeschreven. De auteur respecteert daarbij het oorspronkelijke auteursrecht van de individueel geciteerde studies en eventueel bijhorende documentatie, zoals tabellen en figuren. De auteur en de promotor(en) zijn niet verantwoordelijk voor de behandelingen en eventuele doseringen die in deze studie geciteerd en beschreven zijn.

3 Acknowledgements I would like to thank my promotor, Els Lefever, for advising and helping me with the construction of the automatic colour prediction system and for giving me the opportunity to cowrite and submit an accepted paper on the subject of this thesis. Furthermore, I would like to thank my significant other, Stijn, for listening to me when I talked for hours about the interesting things I encountered in my search for related research and the results of the four experiments I conducted. It will be of no surprise if he already knows the entire content of this thesis. Two other people that deserve a spot in this preface are my parents. They have given me the opportunity to read at university whatever lies in my field of interest and have supported me both emotionally and financially during my four years at Ghent University. 1

4 2

5 Table of Content 1. INTRODUCTION RELATED RESEARCH SENSES AND LANGUAGE DESCRIPTION OF BEER PROPERTIES Colour and aroma description: correlation Flavour and aroma description: correlation Colour and flavour description: correlation EXPERTS VERSUS NOVICES: BIOLOGICAL DIFFERENCE AND DIFFERENCE IN BEER PROPERTY DESCRIPTION THE AUTOMATIC PREDICTION SYSTEM RESEARCH QUESTIONS AND HYPOTHESES CORPUS EXPERIMENTS EXPERIMENT 1: LEXICAL ANALYSIS FLAVOUR AND AROMA DESCRIPTIONS EXPERIMENT 2: AUTOMATIC COLOUR PREDICTION SYSTEM BASED ON FLAVOUR AND AROMA DESCRIPTIONS EXPERIMENT 3: COMPARATIVE ANALYSIS OF LEXICAL DESCRIPTORS IN BEER AND WINE REVIEWS AND AUTOMATIC COLOUR PREDICTION SYSTEMS EXPERIMENT 4: POSSIBLE FURTHER BEER PROPERTY PREDICTIONS BASED ON EXPERT REVIEWS Colour and bitterness Bitterness and category Country of Origin DISCUSSION CONCLUSION BIBLIOGRAPHY APPENDIX A. FULL LIST AROMA AND FLAVOUR TERMS RANKED BY FREQUENCY APPENDIX B. BEER CATEGORY AND BITTERNESS LABEL LIST word count: 14,037 3

6 4

7 Abstract Over the years, beverages experts have been giving more attention to beer review writing. This thesis examines the lexical descriptors of sensory experiences, especially aroma and flavour perception, in expert beer reviews. For the evaluation of lexical consistency and information usability of those reviews, an automatic colour prediction system based on flavour and aroma descriptions is constructed. A corpus of written beer reviews is compiled from data collected from a North American beverages expert website. Consequently, it is analysed on lexical descriptors and serves as training and testing data for the prediction system. The prediction system is similarly constructed as the system for wine colour prediction of Hendrickx et al. (2016). The results of this thesis indicate that beer experts primarily use source-based terms and metaphors and that these experts are subject to odour-taste synaesthesia. Moreover, the roughly 60% accuracy of the prediction system implies a certain level of consistency, structure and information usability of beer reviews. Lastly, beer experts bear similarities to wine experts in wording their sensory experiences. 5

8 6

9 1. Introduction In Belgium, one of the world s top beer brewing countries, beer is gaining more and more attention from food and beverages experts. This is not only the case in this country, but also in several other countries all over the world. In addition, restaurants worldwide are providing their customers with both a wine and beer list on a more regular basis, which provides beer with more prestige. This gained prestige is reflected in newspapers, journals and on the internet, whereas beer experts regularly describe their perceptions and opinions about beer in multiple reviews. Firstly, this master s thesis examines the content and wording of these beer reviews. In other words, it explores the different phrases and vocabulary that is used for the description and evaluation of sensory experiences during beer tasting. Therefore, a lexical analysis of these descriptions is conducted. Previous research has mainly focused on the content and wording of wine reviews (Croijmans & Majid, 2016; Hendrickx et al., 2016; Paradis & Eeg-Olofsson, 2013; Hopfer & Heymann, 2014), but less attention has been given to beer reviews. Given the rising popularity of beer among experts and restaurant customers, it is highly interesting to bring beer reviews into the spotlight and compare beer reviews to wine reviews. Secondly, this master s thesis examines whether expert beer reviews are consistently phrased and informative. For the evaluation of the usability of beer reviews for information retrieval, an automatic prediction system is built to predict a missing beer property based on beer properties described in expert beer reviews. While this thesis focuses on descriptions of sensory experiences, the automatic prediction system is built to predict sight perceptions based on taste and smell perceptions. In other words, the prediction system is built to automatically predict colour properties based on flavour and aroma properties in the expert beer reviews. If the accuracy of this prediction system is sufficient, the experts reviews will probably contain enough useful information and these experts will assumedly describe beer properties in a consistent manner. This automatic prediction system is trained on a large corpus of beer reviews, for which data is automatically extracted from the beverages expert website For the construction of the prediction system, this thesis builds further upon previous research by Hendrickx et al. (2016). They applied natural language processing (NLP) to a corpus of wine reviews to build a system that can automatically predict several wine 7

10 properties based on multiple descriptions in expert wine reviews. This thesis, however, examines whether an automatic prediction system can be built for beer properties instead of wine properties and whether that system can base its colour predictions on both aroma and flavour descriptions from experts reviews. As a result, the outcomes of this thesis will be compared to the results of the study of Hendrickx et al. (2016). In general, this thesis provides more insight in the lexical properties of reviews in the beverages industry, the vocabulary of sensory experience description, the possibilities of automating beer property prediction based on sensory descriptions, the similarities and differences in sensory descriptions between beer and wine experts and the lexical variance in both beer and wine expert reviews. The remainder of this thesis has the following structure: chapter two explores related research and chapter three contains the research questions and hypotheses. Chapter four elaborates on the data set and on the construction of the corpus. Chapter five describes the four experiments that are conducted. The first experiment analyses the lexical properties of flavour and aroma descriptions. For the second experiment, the automatic colour prediction system is constructed, which bases its colour predictions on flavour and aroma descriptions in the reviews. Furthermore, experiment three contains a comparative analysis on lexical descriptors and automatic colour prediction systems between beer and wine reviews. Lastly, beer properties bitterness, beer category and country of origin are analysed in experiment four. Chapter six contains the discussion and is followed by the conclusion in chapter seven. 8

11 2. Related research 2.1 Senses and language Beer experts write about their sensory experiences. Therefore, the correlation between language and senses is explored first. Olofsson and Gottfried (2015) listed two neurocognitive limitations of language on aroma and flavour description: naming failure and configural perception. Naming failure implies the inability of naming every single flavour and aroma correctly, which is the result of the limited vocabulary for olfactory description. Configural perception is the ability of identifying odours in a mixture. Both experts and novices, however, are incapable of identifying all odours in a mixture. Furthermore, people tend to blend perceptual qualities so that even pure odours can be perceived as a mixture of odours due to almost constant exposure to binary odour mixtures (Olofsson & Gottfried, 2015). These two neurocognitive limitations of language suggest that the vocabulary for flavour and aroma description in the beer reviews will be limited and that some aroma and flavour descriptions will be present and others will be absent in reviews of the same beer due to different configural perceptions. In other words, one expert will describe the aroma of a beer as hoppy and rich, whereas another expert uses rich and fruity. The same applies to flavour description. Each expert perceives a mixture differently and consequently describe it differently. In this thesis, however, only one review of each beer will be adopted in the training corpus. Therefore, the number of words describing the aroma and flavour of a single beer will probably be more limited. Regarding the vocabulary for aroma and flavour description, the study of Croijmans and Majid (2016) about the vocabulary used by novices, wine experts and coffee experts shows that, when analysing the words and terms these three groups use in their sensory experience descriptions, both wine and coffee experts tend to use more specific source-based terms (metaphors or it smells like + source) and novices more evaluative terms (e.g. nice, bad, good ). Consequently, the reviews in the corpus of this thesis are expected to contain more sourcebased descriptions and fewer evaluative terms for the description of aroma and flavour properties. This is because the data for the corpus is extracted from a beverages expert website. This thesis examines whether that claim is applicable to beer reviews written by beer experts as well. 9

12 Majid and Levinson (2011) suggest that there are intrinsic limits of language (p. 7): senses cannot be fully coded by language, but senses can be more richly coded in one language than another. This suggests that the limits of language for describing sensory experiences are not universal. In addition, sensory experiences of individuals of the same language and culture can differ, but they are associated with the same terms that are learned in the culture. For example, two individuals can experience the colour blue differently, but they both name the colour with the same term, namely blue. Taste and smell are perceptual categories that are generally neglected in western cultures and that are described by using a more concise vocabulary. Sight, on the other hand, is a perceptual category that is more valued by western cultures and is, therefore, described by using a more elaborate vocabulary (Majid & Levinson, 2011). Following that study, the corpus of this thesis is expected to contain more words describing colour properties than words describing aroma and flavour properties. Consequently, this thesis examines whether this is the case for beer reviews written by beer experts. The following subsection discusses the correlation between colour, aroma and flavour descriptions. While the automatic colour prediction system for beer will base its colour predictions on both flavour and aroma descriptions, the three descriptions need to be related in a way. 2.2 Description of beer properties Colour and aroma description: correlation Research has shown that perceptions of food and beverages depend on both visual and orthonasal sensory input, especially before the tasting (Spence & Piqueras-Fiszman, 2014). The colour and look of a drink influence both the perception and description of aroma (Morrot, Brochet & Dubourdieu, 2001). Furthermore, Dematte, Sanabria and Spence (2006) state that people link certain aromas with certain colours. It is therefore assumed that sight and colour perceptions are closely related to aroma perceptions. Consequently, certain colour descriptions in the reviews are possibly accompanied by the same aroma descriptions and certain aroma descriptions by the same colour descriptions. Therefore, an automatic prediction system could predict missing colour properties by looking at the terms in the reviews for aroma descriptions it is often accompanied by, and vice versa. For example, gold refers to the colour of a beer and in the corpus, reviews about golden coloured beer contain often the words hoppy, rich 10

13 and herbs to describe the aroma of that golden beer. If a new review lacks a colour description, but the review contains the words hoppy, rich and herbs, the automatic prediction system would be able to define the colour of the beer as gold, even though the reviewer does not clearly mention any colour or sight property. This thesis examines whether beer colour can be predicted based on aroma and flavour descriptions. Therefore, an automatic colour prediction system is built. However, there are studies that oppose claims of linked sight and aroma perceptions. Calvert, Spence and Stein (2004), for example, state that aromas only display flavour properties, but not auditory or visual sensations. Due to opposite views, this thesis examines whether colour and aroma descriptions in beer reviews are closely linked. Furthermore, an automatic prediction system is built to test whether colour can be predicted based on aroma descriptions in a new beer review Flavour and aroma description: correlation Not only colour and aroma descriptions, but also flavour and aroma descriptions are claimed to be closely connected. According to Spence (2015), food flavour is described by both gustatory and olfactory stimuli. There are two olfactory stimuli: orthonasal smell and retronasal smell, which are respectively the smell we sniff before the food or drink is tasted and the smell that is pulsed out after the food or drink had been swallowed. It is the combination of retronasal aromas and gustatory cues that defines a flavour and lead to descriptions such as fruity and malty (Spence, 2015). Calvert, Spence and Stein (2004) consider odour-taste synaesthesia (smelling tastes and tasting smells) a factor of the link between smell and taste. When people smell an aroma, they recognise the aroma as a flavour and describe it as such, e.g. something smells sweet. In addition, people recognise a taste-like quality more easily than a more specific quality and describe it metaphorically. This is also due to the co-occurrence of retronasal odour stimulation and oral stimulation and the result of a unitary perception. (Calvert, Spence & Stein, 2004). Odour-taste synaesthesia, however, is not a universal given. Howes (2006) has evidence of audio-odour synaesthesia in many Melanesian languages and Young (2005) has evidence of colour-odour synaesthesia among the Anangu of Australia s Western Desert (Howes, 2006). According to Stevenson and Boakes (2004), not everyone describes aromas the same way. For example, aromas of vanilla, strawberry and caramel are described as smelling sweet in western cultures, whereas it can be perceived and described differently elsewhere (Stevenson & Boakes, 2004). The website on which the corpus is based, is North American and therefore, the reviewers working for this website will assumedly be subject to odour-taste synaesthesia 11

14 and describe flavours and aromas similarly. Similar to colour and aroma descriptions, it can be suggested that the flavour and aroma descriptions in the reviews of the corpus are possibly closely connected. Consequently, certain aroma descriptions in the reviews are claimed to be accompanied by the same flavour descriptions and certain flavour descriptions will assumedly be accompanied by the same aroma descriptions. Therefore, an automatic prediction system could possibly predict missing aroma properties by looking at the flavour descriptions in the reviews it is often accompanied by, and vice versa. This thesis, however, focuses on automatically predicting beer colour. As a consequence, the automatic prediction of aroma or flavour properties is not tested in this thesis. Unlike the correlation between aroma and colour descriptions, flavour and aroma descriptions are probably more likely to be phrased similarly and to consist of the same vocabulary due to the assumed odour-taste synaesthesia, which claims that people of western cultures perceive flavour and aroma unitarily and describe these properties by using similar terms. This thesis, therefore, examines whether beer experts use unique or identical words for flavour and aroma description and whether odour-taste synaesthesia can be found in beer reviews Colour and flavour description: correlation Considering the abovementioned hypotheses of linked colour and aroma descriptions and linked flavour and aroma descriptions, it can be suggested that colour and flavour descriptions are closely linked as well and that colour properties can be predicted based on flavour descriptions, and vice versa. This may lead to a triangular relationship between colour, aroma and flavour descriptions. If one of the descriptions lacks in a new beer review, that property could possibly be predicted based on the other two descriptions that are present in the review. Consequently, this thesis examines not only the assumed correlation colour/aroma and aroma/flavour, but also the correlation colour/flavour. Therefore, this thesis confines itself to building an automatic prediction system that predicts lacking colour properties based on both flavour and aroma properties. If the automatic colour prediction system shows a high accuracy level, it can be suggested that colour and flavour might be closely connected as well. 12

15 Figure 1. Triangular relation between colour, aroma and flavour descriptions COLOUR DESCRIPTIONS AROMA DESCRIPTIONS FLAVOUR DESCRIPTIONS 2.3 Experts versus novices: biological difference and difference in beer property description For this master s thesis, a corpus of online beer reviews written by experts is composed. Experts are widely considered to be more accurate and detailed in their aroma, flavour and colour descriptions, which is important for the construction of an automatic prediction system. This is because too general descriptions lead to incorrect, less accurate and general predictions of beer properties. For an automatic prediction system, it is important to have unique descriptions for each beer property so that the system can classify the descriptions in specific categories and produce more precise and accurate predictions. The following two studies suggest that experts are biologically different from novices. Bartoshuk (2000) claims that experts or supertasters have 16 times more buds on their tongues than non-tasters. Garneau et al. (2014), however, claim that people s sensitivity to bitter-tasting foods and drinks influences the tasting ability and depends more on their sensitivity to PROP 1 and on the variation in the TAS2R38-gene 2 than on the density of taste buds. In sum, both researches claim that experts are biologically superior to novices when it comes to distinguishing flavours. Croijmans and Majid (2016) conducted research on the capability of novices, wine experts and coffee experts to describe and name aromas and flavours of wine and coffee. Despite the fact that the wine experts aroma and flavour descriptions for wine proved to be more consistent 1 PROP or 6-n-propylthiouracil is a bitter tastant chemical (Garneau et al., 2014; Hayes, Barthoshuk, Kidd & Duffy, 2008) 2 TAS2R38 or taste receptor 2 member 38 is a bitter taste receptor gene (Tepper et al., 2008) 13

16 than the wine descriptions of the coffee experts and the novices, the coffee experts in the study were not more consistent in describing aromas and flavours of coffee, their own field of expertise, than the wine experts and the novices were. This suggests that not all experts are always more consistent in describing aromas and flavours in their own field of expertise than experts of a different field of expertise. Furthermore, no expert group identified everyday aromas or flavours more accurately than the other. Hence, experts appear to have an advantage when describing aromas and flavours in their own field of expertise, but not necessarily in every field of expertise. This thesis, however, does not examine the difference between novices and experts because the corpus only contains expert reviews. Therefore, a comparison between novice and expert descriptions cannot be made with the data collection in the corpus. 2.4 The automatic prediction system For the examination and evaluation of description consistency and information reliability in the experts reviews, an automatic prediction system for beer properties is built. If the reviews are well-structured and consistently phrased, an automated system will be able to automatically extract, label and predict useful information. This thesis examines whether an automatic prediction system can be built to predict beer colour based on flavour and aroma descriptions in expert beer reviews. The approach that is used for the automatic prediction system is machine learning (Witten, Frank, Hall & Pal, 2016): the system detects patterns in data, in this case in flavour and aroma descriptions in the reviews of the annotated corpus. These patterns enable the automatic prediction system to make fast and accurate predictions of beer properties. The system trains it predictions on the annotated data of the corpus. It is important to have a large corpus, because the higher the number of annotated data, the higher the accuracy of the system will probably be. The machine learning technique used in this thesis is supervised machine learning, which means that every instance in the beer review corpus is labelled (Kotsiantis, Zaharakis & Pintelas, 2007). There are several supervised machine learning techniques such as logic based algorithms, perceptron-based techniques, and statistical learning algorithms. This thesis uses Support Vector Machines (SVM). SVMs learn a linear hyperplane that separates two data classes, in this case colour classes, where the margin is maximal. The margin is the distance 14

17 from the hyperplane to the nearest data point. The data is mapped onto a higher-dimensional space, which is called the feature space. Kernel functions such as linear or RBF kernels are used to map new data points into the feature space. Limitations of SVMs are the low training speed and the impracticality for multi-class problems (Kotsiantis, Zaharakis & Pintelas, 2007). Figure 2. A simple linear support vector machines (Tong & Koller, 2001) The punctuation of the data retrieved from a specialist website, in this case the website is stripped off in the Python script. Subsequently, the reviews are considered as bag-of-words (BoW) representations. The automatic prediction system is then trained and tested with a ten-fold cross validation. The automatic prediction system is based on the system of Hendrickx et al. (2016). They have built an automatic colour prediction system based on various sensory experience descriptions in expert wine reviews. They pre-processed the data with the Stanford toolkit (Manning et al., 2014) to perform tokenisation, POS-tagging and lemmatisation. Subsequently, they randomised and split the randomised data set into an 80% training and 20% test set. The information sources consist of both lexical and semantic features. Each wine review is converted into a bag-ofwords (BoW) feature vector for the first experimental setup. Only content words such as nouns, verbs and adjectives that occurred more than once in the training set are selected for the construction of these BoW features. BoW, however, ignores word order (Wallach, 2006). If the feature vector, both unigrams and bigrams, of a new review shows sufficiently similarities with a feature vector in the training corpus, the automatic prediction system considers the wine colour in the new review and the one in the trained review identical. The BoW features are 15

18 combined with 100 LDA topics and 100 Word2Vec clusters for the second experimental setup. The colour classification consists of three classes: red, white and rose. For the experiments, 14,213 reviews are used as test instances. The f-scores for these colour classes are respectively 98.4, 97.4 and They have also built prediction systems for country of origin, grape variety and price (Hendrickx et al., 2016). After the construction of the automatic colour prediction system for beer in the second experiment of this thesis, both automatic prediction systems are compared in the third experiment. 16

19 3. Research questions and hypotheses This thesis is based on previous research conducted by Hendrickx et al. (2016) and therefore, adopts similar research questions and hypotheses. In addition, new research questions are added to this thesis in order to guarantee a more elaborate account. The following four research questions will be addressed: 1. Are expert reviews meaningful providers of information considering the limited vocabulary and ways of describing sensory perceptions such as aroma, flavour and colour? Therefore, sensory experiences should be worded in a consistent manner. Considering the results of the research of Hendrickx et al. (2016), which states that wine experts are capable of describing wine properties in such a consistent manner, beer experts are probably capable of producing beer descriptions in a consistent manner and provide useful information as well. It can therefore be hypothesised that expert beer reviews are meaningful providers of information considering the limited vocabulary and ways of describing sensory perceptions such as aroma, flavour and colour. 2. Can lacking colour properties be automatically predicted based on expert reviews considering a link between aroma, flavour and colour descriptions? The automatic prediction system is expected to be able to predict colour properties on the sole basis of aroma and flavour properties. This means that the system can assign colour properties to the beer in the review even though colour descriptions are lacking. The automatic prediction system bases its colour predictions on the aroma and flavour descriptions present in the review. Automatic predictions should also be possible for lacking aroma and flavour properties. The automatic prediction system then bases its aroma property predictions on colour and flavour descriptions and its flavour property predictions on aroma and colour descriptions. These possibilities, however, are not tested in this thesis. For the construction of an accurate automatic colour prediction system, flavour and aroma descriptions need to be closely related to colour descriptions. Based on abovementioned studies, this claim is assumed to be true. 17

20 3. How are the descriptions worded? What are the stylistic and lexical patterns? Based on the research mentioned in the second chapter of this thesis, beer experts will probably use more specific source-based terms than evaluative terms for describing flavour and aroma. Furthermore, aroma and flavour properties are possibly described by the following constructions and phrases: it tastes like + source construction, it smells like + source construction, metaphors and adjectives such as sweet, malty and fruity. The experts of the collected beer reviews are assumed to be subject to odour-taste synaesthesia. Consequently, flavour and aroma descriptions will probably contain similar terms and metaphors. The difference between aroma and flavour descriptions will therefore be less clear. Lastly, the vocabulary for flavour and aroma descriptions is claimed to be less varied than that for colour descriptions. A fourth research question is added, while this thesis can also examine the differences and similarities between the automatic prediction of beer properties and the automatic prediction of wine properties. 4. In what way do the automatic prediction of beer properties and lexical descriptors of beer reviews differ from the automatic prediction of wine properties and lexical descriptors of wine reviews? And what are the similarities between the two tasks? The general hypothesis is that beer experts, just like wine experts, are capable of describing beer properties in a sufficiently consistent manner, which allows beer properties to be automatically predicted on the basis of experts reviews. This thesis has a shared hypothesis with the study of Hendrickx et al. (2016), while the beer and wine industry can be considered similar: beer and wine are both alcoholic drinks, with different subgroups and differentiated characteristics. There are beer and wine experts that are specialised in distinguishing different brands and subgroups and in describing taste, smell and sight properties. They are respectively named zythologists and oenophiles. Furthermore, the lexical descriptors in both beer and wine reviews and the automatic colour prediction systems will be compared to each other as well. It is hypothesised that the lexical descriptors and the results of the automatic prediction systems will be similar. 18

21 4. Corpus The data for the corpus is retrieved from expert website This website has been powered by the Beverage Testing Institute since It hosts over 3,723 spirit reviews, 15,876 wine reviews and 3,065 beer reviews. In total, 2,205 beer reviews can be downloaded and contain information on beer properties that can be automatically extracted. Due to the two columns in which the beer properties are clearly detailed and described in a structured manner, the beer properties in the beer reviews can also be automatically labelled. The first column, of which an example of Clatham Brewing Imperial Porter can be found below, displays the following beer properties: name, category, date of tasting, country of origin, alcohol percentage, points awarded by the reviewer (80-100) and the medal and rating adjective that accompany this score. For the corpus, the beer properties name, category, country of origin and alcohol percentage are automatically extracted 3 and labelled from this column. Image 1. First column of a beer review on The first column is followed by a written review. The review itself contains an average number of 43 words and can therefore be called short and concise. The reviews are written by fifteen different reviewers and are similarly structured: the first sentence contains a colour description, the second sentence an aroma description followed by a flavour description and the third sentence a bottom line. Due to this reoccuring structure, an accurate automatic prediction system can be built more easily. Not only the information in the first column, but also in the 3 All automatic data processing was performed by means of a Python script. 19

22 written review can be automatically extracted and labelled. Three examples of written reviews are provided below. 1. Dark brown color. Bright, attractive aromas and flavors of chocolate cherry bon bons, chestnut brittle, brown sugar, and sarsaparilla candy with a supple, tangy, finely carbonated, dry-yet-fruity medium-to-full body and a tingling, complex, medium-long finish that shows impressions of salty roasted nuts, vanilla salt, grassy earth, and peppery radish. An exceptionally zesty, well balanced and attractive imperial porter. (Chatham Brewing Imperial Porter, USA) 2. Brilliant old gold color. Rich green apple, golden raisin pound cake and nougat aromas and flavors with a fruity, tangy medium-full body and a long, mouthwatering Meyer lemon and tangerine accented finish. Absolutely delicious. (Brouwerij Lindemans Pomme Lambic, Belgium) 3. Pale light brilliant light gold color. Delicate, yeasty, grainy herbal curious aromas and flavors of egg sandwich, salty roasted corn, and pear skin with a supple, crisp, fizzy, fruity light-to-medium body and a smooth, interesting, medium-length melon, apple, sweet cream, and grass finish. A very tasty, nicely fruity lager that is super sessionable. (Innis & Gunn Brewing Company Lager Beer, Scotland) In the second column, the reviewer provides a structured overview of style, aroma, flavour, bitterness, pairing possibilities and repeats the bottom line of the written review. The use of these columns makes it more accessible to automatically extract and label the different beer properties. The second column for the Clatham Brewing Imperial Porter can be found below. Image 2. Second column of a beer review on 20

23 In total, nine beer properties are automatically extracted, labelled and put in a tab-separated text file that will be used for the examination of lexical properties and the construction of the automatic prediction system: name of the beer, country of origin, aroma, flavour, bitterness, category, style, colour and written review. Style is the only beer property that will not be discussed in this thesis. Unlike the other beer properties, colour descriptions could not be automatically extracted from the two columns, because they are only present in the written review itself. Therefore, the word that is prior to the word colour in the review was labelled as the colour description. This master s thesis elaborates on the following six properties: country of origin, aroma, flavour, bitterness, category and colour. 5. Experiments This chapter contains four experiments. In the first experiment, the lexical properties of both flavour and aroma descriptions are analysed. In addition, the lexical differences and similarities between both descriptions are examined. For the second experiment, an automatic colour prediction system based on flavour and aroma descriptions is built for the evaluation of the informativeness and lexical consistency of expert beer reviews. The third experiment explores the differences and similarities between the lexical analysis of beer reviews and the lexical analysis of wine reviews (Hendrickx et al., 2016). The automatic colour predicition systems for beer reviews and wine reviews are compared as well. Lastly, the fourth experiment analyses possible further beer property predictions based on the expert beer reviews in the corpus. 5.1 Experiment 1: lexical analysis flavour and aroma descriptions As mentioned earlier, this thesis aims at developing an automatic colour prediction system based on aroma and flavour descriptions to evaluate the structure, consistency and usability of expert beer reviews. Before constructing that system, the lexical properties of both flavour and aroma descriptions are examined. In chapter two, several studies claim that the perception and description of both flavour and aroma are closely connected in western cultures. This experiment analyses the lexical components of aroma and flavour descriptions and examines whether the experts use unique terms or mainly identical terms for the description of either aroma or flavour. This leads to the examination of odour-taste synaesthesia (Calvert, Spence 21

24 and Stein, 2004). Furthermore, the differences and similarities between aroma descriptions and flavour descriptions are explored. Firstly, the presence of the it tastes like + source construction for flavour description and the it smells like + source construction for aroma description is examined. According to Croijmans and Majid (2016), these constructions, as well as metaphors, are used for aroma and flavour description by wine and coffee experts. All the beer reviews in the corpus are similarly structured: the first sentence contains the colour description, the second sentence aroma description followed by flavour description and the third sentence the bottom line. Therefore, the three beer review examples that were given in the previous chapter are repeated to illustrate the analysis of the present or absent it tastes/smells like + source constructions. 1. Dark brown color. Bright, attractive aromas and flavors of chocolate cherry bon bons, chestnut brittle, brown sugar, and sarsaparilla candy with a supple, tangy, finely carbonated, dry-yet-fruity medium-to-full body and a tingling, complex, medium-long finish that shows impressions of salty roasted nuts, vanilla salt, grassy earth, and peppery radish. An exceptionally zesty, well balanced and attractive imperial porter. (Chatham Brewing Imperial Porter, USA) 2. Brilliant old gold color. Rich green apple, golden raisin pound cake and nougat aromas and flavors with a fruity, tangy medium-full body and a long, mouthwatering Meyer lemon and tangerine accented finish. Absolutely delicious. (Brouwerij Lindemans Pomme Lambic, Belgium) 3. Pale light brilliant light gold color. Delicate, yeasty, grainy herbal curious aromas and flavors of egg sandwich, salty roasted corn, and pear skin with a supple, crisp, fizzy, fruity light-to-medium body and a smooth, interesting, medium-length melon, apple, sweet cream, and grass finish. A very tasty, nicely fruity lager that is super sessionable. (Innis & Gunn Brewing Company Lager Beer, Scotland) The it smells/taste like + source constructions are absent in the written reviews of the corpus. The beer experts, however, adopt the following constructions: - metaphors - adjective/noun + aromas - adjective/noun + flavours - aromas of + adjective/noun - flavours of + adjective/noun 22

25 These constructions reoccur in the bulk of the written beer reviews in the corpus. Therefore, it can be suggested that beer experts consistently structure and phrase aroma and flavour descriptions. For completeness sake, the constructions for colour descriptions in the written reviews are analysed as well. As a result, it can be stated that these experts use the adjective/noun + colour construction for colour description. Concerning the adjectives and nouns that are used for aroma and flavour description, the top 20 most frequently used terms in the corpus for aroma and flavour description are ranked in table 1. These terms are automatically extracted from the second columns of the reviews, but not from the written reviews. For aroma description, 19 out of the 20 most frequently used terms are nouns and source-based and one term is an adjective ( dark ). The latter is rather surprising, because that adjective is considered to be mostly used for sight/colour description. This could perhaps contradict the hypothesis of odour-taste synaesthesia. In the corpus, however, dark always precedes a noun and forms terms such as dark chocolate, dark toast and dark roasted nuts. It does not occur on itself for the description of aromas and therefore, only describes an aroma describing noun. Consequently, it does not imply that the experts are subject to colour-odour synaesthesia, which is mentioned by Howes (2006). For flavour description, all 20 most frequently used terms are nouns and source-based as well. Table 1. Top 20 most frequently used terms for aroma and flavour description Aroma Freq Aroma Freq Flavour Freq Flavour Freq nuts 360 banana 143 pepper 518 bread 150 chocolate 360 fruits 136 nuts 279 radish 138 bread 324 peach 133 citrus 254 peppery 137 toast 255 pepper 133 lemon 234 arugula 132 apple 231 honey 131 apple 223 root 122 lemon 211 muffin 129 greens 216 melon 117 citrus 176 dark 127 grass 158 toast 116 cake 158 orange 123 nut 154 fruit 115 fruit 157 herb 115 chocolate 153 orange 111 nut 146 corn 114 tangy 151 earth 99 Concerning the uniqueness of the most frequently used terms for aroma and flavour description, it can be deduced from table 1 that many terms are used for both descriptions. Approximately half of the top 20 most frequently terms are unique when compared to the top 20 most frequently used terms of the other property. In order to investigate the uniqueness of the vocabulary for aroma and flavour description in the entire corpus, the terms used for aroma description and the 23

26 terms used for flavour description are ranked by frequency and compared with each other. Both full rankings can be found in appendix A (p 53). Table 2 shows the number of unique terms for respectively aroma and flavour description in the entire corpus. When a term is uniquely used for the description of either aroma or flavour, it is only present in one ranking. The full aroma ranking consists of 3,121 terms and the full flavour ranking of 2,466 terms. In total, there are 1,993 unique terms for aroma description and 1,456 unique terms for flavour description. In other words, respectively 63.86% and 59.04% of the rankings are unique. Table 2. Unique terms compared with the entire corpus Terms for aroma description Terms for flavour description Number of unique terms Percentage in entire corpus (%) Number of unique terms Percentage in entire corpus (%) Top 10 Top Top 25 Top Top 50 Top Top 100 Top Top 250 Top Top 500 Top Top 1,000 Top 1, Top 1,500 Top 1, Top 2,000 Top 2,000 1, , Entire corpus Entire corpus 1, , The analysis of table 2 shows that the ten most frequently used terms for aroma description are also used for flavour description, which means that none of these terms are unique for aroma description. Only one term ( roasted nuts ) is unique in the top 25 and even in the top 50 most frequently used terms in the aroma ranking. In its top 100 most frequently used terms, only two terms ( roasted nuts and danish ) are unique for aroma description. For the description of flavour properties i.e. flavour, only one term ( tangy ) in the top 10 and top 25 most frequently used terms is unique. Two terms ( tangy and grassy ) are unique in the top 50 of the flavour 24

27 ranking and in its top 100, five terms ( tangy, grassy, driven, radish sprouts and bitter greens ) are solely used for flavour description. Graph 1. Unique terms for aroma and flavour description in the entire corpus Entire Top 2,000 Top 1,500 Top 1,000 Top 500 Top 250 Top 100 Top 50 Top 25 Top 10 Unique terms aroma description Unique terms flavour description Graph 1 displays the similarly, rapidly stagnating number of unique terms for aroma and flavour description used in the entire corpus and in the top 500. This can be explained by the frequency of the terms in the corpus: in the aroma ranking, 1,832 terms are only used once, 389 twice and 189 three times in the entire training corpus. In the flavour ranking, 1,416 terms are used once, 310 twice and 160 three times. These are probably rather uncommon terms used for both aroma and flavour description. The probability of a less frequent unique aroma term being used in a new review is lower than the probability of a more frequent unique aroma term being used in a new review. The same applies for less and more frequent unique flavour terms. Therefore, the top 500, 250, 100, 50, 25 and 10 most frequently used terms in the corpus contain more reliable descriptions of aroma and flavour than their top 1,000, 1,500, 2,000 and the entire corpus. As mentioned above, more than half of the terms used for the description of aroma and flavour are only used once or twice in the training corpus. When, for example, a term in the top 50 for aroma description is only used once for flavour description, it is labelled as not unique. However, the probability of that term being used for the description of an aroma property is higher than the probability of that term being used for describing flavour. For a more accurate labelling of unique terms, the top 10, 25, 50 and 100 of both rankings are this time not compared to the entire corpus, but to the top 100 most frequently used terms for aroma and flavour 25

28 description. Therefore, the labelling of frequently used aroma terms as not unique due to the occurrence of those terms in the bottom of the flavour ranking is prevented, and vice versa. Table 3 shows the unique terms for aroma and flavour description compared with the top 100 of each ranking. Table 3. Unique terms compared with the top 100 most frequently used terms for aroma description and the top 100 most frequently used terms for flavour description Terms for aroma description Terms for flavour description Number of unique terms Percentage in top 100 (%) Number of unique terms Percentage in top 100 (%) Top 10 Top Top 25 Top Top 50 Top Top 100 Top In comparison to the figures in table 2, the results in table 3 show that both rankings have a significant higher number of unique terms when compared to the top 100 ranking than to the entire corpus. This was expected while table 3 excludes less frequent terms of both rankings. The number of unique terms in the top 100 most frequently used terms for aroma description and flavour description has respectively increased from 2 and 5 to 47 terms. This means that more than half of the terms in the top 100 are still used for both aroma and flavour description. Regarding the top 10 most frequently used terms for aroma and flavour description, the number of unique terms has not increased substantially. There are still no unique terms for aroma description, but the number of unique terms for flavour description has increased from one to two terms. The introduction of this thesis mentions the hypothesis of Calvert, Spence and Stein (2004) about odour-taste synaesthesia, which means that people recognise and describe aromas and flavours, respectively smell and taste properties, similarly. The low number of unique terms for both aroma description and flavour description and the high number of identical terms confirm this hypothesis. Table 4 and table 5 display the 47 uniquely used terms for both categories. Table unique terms for aroma description 26

29 Unique terms for aroma description Term Freq. Term Freq. Term Freq. butter roasted nuts apples toffee pastry mango yogurt sourdough pineapple herbs pie baguette egg spicy chutney clay soufflé brittle scone raisin toast nougat praline dark chocolate honeycomb cheese nut brittle beans herb muffin lavender buttery aromas Chocolate nuts ginger raisin bread nut bread fig peaches coconut tea sugar jerky soy berry danish cookie color omelet Table unique terms for flavour description Unique terms for flavour description Term Freq. Term Freq. Term Freq. grass tangy radish peppery arugula earth sprouts hop lettuce watercress grassy kiwi skin turnip peppercorn vegetable jicama root vegetable minerals mocha peppered greens crisp lemon pepper potato mineral rapini wafer radicchio driven kale chestnut green apple sorbet nutskin water radish sprouts starfruit peel sweet potato bitter greens pepper bread rind salty creamy bark carrot yams When looking at the lexical and stylistic patterns of the most frequently uniquely used terms displayed in table 4 and 5, the claim of Croijmans and Majid (2016) that experts reviews contain more source-based descriptions and fewer evaluative terms, can be confirmed. In the corpus of this thesis, almost all descriptions are exclusively source-based and terms are used as methaphors or in the constructions aromas/flavours of + term and term + aromas/flavours. 27

30 Most terms are noun phrases or adjectives derived from nouns. Therefore, not only wine experts reviews and coffee experts reviews, but also beer experts reviews contain more source-based terms and fewer evaluative terms. Regarding the constructions it tastes/smells like + source, they are almost absent from all reviews in the corpus. Not only the hypothesis of Croijmans and Majid (2016), but also the hypothesis of Calvert, Spence and Stein (2004) about odour-taste synaesthesia can be confirmed. Many terms in the corpus are used for both aroma and flavour description, especially the top 10 most frequently used terms for the description of both beer properties. 5.2 Experiment 2: automatic colour prediction system based on flavour and aroma descriptions For the evaluation of the lexical consistency and usability of the information in the expert reviews, an automatic prediction system is built. If the accuracy and precision of this prediction system is fairly high, the reviews probably contain similar descriptors for aroma and flavour properties and the provided information is useful. For this thesis, the focus lies on an automatic prediction system for colour properties based on flavour and aroma descriptions in the review, while this thesis is based on the study of Hendrickx et al. (2016). They constructed an automatic colour prediction system based on flavour and aroma descriptions in wine reviews. This thesis, therefore, examines whether this is possible for beer colour prediction. Unlike the other beer properties, colour descriptions could not be automatically extracted from the website, because they were only present in the written review itself and not in a seperate column. Therefore, the word that is prior to the word colour in the review was labelled as colour description. Altogether, 49 different colour terms were extracted. This number, however, is too high and makes the automatic colour prediction for new reviews more difficult and less accurate. Therefore, the terms referring to the same colour class are taken together and eleven new colour classes are formed: very light/white, yellow, amber, brown, black, red/rose, green, cloudy, hazy, oak and deep. This classification groups adjectives and nouns that refer to the same colour and assembles them under one class. Terms that have no link to colour in any way, such as prepositions and articles, and words that only occur once in the corpus (indigo, brilliant, violet, platinum, gray, nickel, wood) are not considered for the colour classification. In the classification, however, four classes do not directly refer to a 28

31 specific beer colour, but they are artefacts introduced by the automatic extraction of the label: cloudy, hazy, oak and deep. Cloudy, hazy and deep add an extra dimension to a specific colour and describe the turbidity of a beer. Oak, on the other hand, can refer to multiple colours. Its definition depends on the adjective that preceeds the word, for example light or dark. Despite this, these four terms are considered as four colour classes, because they occur a sufficient number of times in the corpus. The colour class and the colour labels that are assembled under each class can be found below. Two colour labels, however, can be subsumed in two colour classes: orange and sienna. The word sienna can refer to either amber or brown and the word orange to either gold or amber. For the colour mapping, they are each subsumed in one categorie that reflects their meaning best: sienna in brown and orange in amber. There are sufficiently enough unique terms in these categories for a feasible accurate and less biased automatic colour prediction system. Colour class Very light/white Gold Amber Brown Black Red/rose Green Cloudy Hazy Oak Deep Colour labels silver gold, yellow, golden, sunburst, straw, light, bright, brassy, sunset, white, sunrise amber, copper, bronze, orange, maroon, penny brown, mahogany, medium-brown, walnut, dark-brown, sienna black, ebony, cola ruby, garnet, pink, red, salmon green, emerald cloudy hazy oak deep The classification algorithm that is used for this experiment, is Support Vector Machines as implemented in the LIBSVM toolkit (Chang and Lin, 2011). The experiment is evaluated with a 10-fold cross validation: 90% of the corpus function as training data and 10% of the corpus as testing data. This process is repeated until every fold (representing 10% of the corpus) is used as testing data and the remaining nine folds (90% of the corpus) as training data. Beer reviews that do not contain any specific colour description are excluded for this experiment, which brings the total number of tested and trained reviews from 2,205 to 2,121 instances. The task of predicting the colour of the beers was conceived as a supervised classification task. Two sets of bag-of-words features were extracted as information sources from the expert review texts: unigrams (single words) and bigrams (sequences of two words). Sentences containing a 29

32 colour description were automatically removed from the review for the examination of the viability of automatically predicting the colour of the beer based on the sole review text. The reviews were further preprocessed by removing all punctuation marks and by lower-casing all words contained in the review. After programming and testing the automatic prediction system, the results of this system are validated with precision, recall and! " -score. Firstly, recall is the number of times that a predicted label corresponds with the correct class in relation to the times that label occurs in the corpus. To put it differently, recall is the number of correctly predicted labels divided by the number of predicted labels for that class. Secondly, precision is the number of times that a predicted label corresponds with the correct class in relation to the times that label is predicted. In other words, precision is the number of correctly predicted labels divided by the total number of gold standard labels. Thirdly, f-score reflects the weighted average of both precision and recall scores. Finally, the accuracy is calculated by dividing the total number of correctly predicted labels by the overall number of test instances, without differentiating between the different class labels. #$%&'(')* =,-./%$ )1 &)$$%&234 5$%6'&2%6 37/%3( 8)273 *-./%$ )1 5$%6'&2%6 37/%3( 9%&733 =,-./%$ )1 &)$$%&234 5$%6'&2%6 37/%3( 8)273 *-./%$ )1 :)36 (27*67$6 37/%3(! (&)$% = 2 #$%&'(')* 9%&733 #$%&'(')* + 9%&733?&&-$7&4 = # &)$$%&234 5$%6'&2%6 2%$.( # 2)273 *-./%$ )1 2%$.( A first experiment is run with the linear kernel of LIBSVM (Hsu & Lin, 2002). These are the results of this first experiment: 30

33 Classes Frequency in corpus Recall Precision F-score Very Light/white Deep Hazy Cloudy Gold Amber Brown Black Red/rose Green Oak The average cross-validation accuracy is 56.05%. Classes very light, deep, hazy, cloudy, red/rose, green and oak have no recall, precision and F-score results. This is due to their low frequency in the corpus. It is nearly impossible for an automatic prediction system to make unbiased and accurate predictions when there is an unsufficient amount of training data. This low frequency has various reasons. Firstly, the number of gold, amber, brown and black coloured beers are significantly higher than the number of silver coloured, red/rose coloured and green coloured beers in reality. This is also reflected in the corpus: there are more golden, amber, brown and black beers than red/rose and green beers. Secondly, oak has the same frequency as red/rose, which is too low to build an accurate automatic prediction system. According to the results above, deep, hazy and cloudy only occur twice or three times in this corpus. These terms, however, are much more frequently used in the corpus than is stated in the results above. This is due to automatic extraction of the colour label. On the website the colour property is not explicitely mentioned in one of the two columns listing the other beer properties such as flavour, aroma and bitterness. In order to automatically extract the colour description, the term that comes before the word colour in the written review is labelled as the colour class of that particular beer. The terms hazy, cloudy and deep, however, generally add a hue to the colour description that follows. For example: hazy brown, deep red and cloudy amber. These three terms scarcely precede the word colour directly and are therefore rarely labelled as colour description. The frequency of the colour terms used for gold, amber, brown and black in the corpus is high enough to train an automatic prediction system on. The results show that not only the recall scores, but also the precision scores and the f-scores for gold, amber and brown are 31

34 significantly higher than for black : the recall, precision and f-score for black are respectively 41.2%, 31.8% and 35.9% compared to gold (59.8/62.7/61.2%), amber (55.7/62.8/59.0%) and brown (52.4/49.7/51.1%). Furthermore, the results for gold and amber are in general better than for brown, especially the precision scores. This can be easily explained by the frequency of black, gold, amber and brown terms in the corpus. Gold and amber are more represented in the corpus than brown and black. As a consequence, the automatic prediction system can be trained on more gold and amber data than brown and black data. The automatic colour prediction system is, therefore, more biased towards predicting gold and amber over brown or black. Regarding the precision scores for these four classes, the precision score for gold (62.7%) and amber (62.8%) are significantly better than for brown (49.7%) and black (31.8%). Regarding the recall scores for these four classes, the recall score for gold (59.8%), amber (55.7%) and brown (52.4%) are significantly better than the recall score for black (41.2%). Table 6. Results for gold, amber, brown and black Labelled class Predicted class Gold Amber Brown Black Gold Amber Brown Black Table 6 displays the results of the automatic colour prediction system. After analysing the results, it appears that the automatic prediction system often predicts a labelled class as a neighbouring class. The labelled class gold is 237 times predicted as amber, whereas amber is 221 times predicted as gold and 75 times as brown, and brown is 111 times predicted as amber and 56 times as black. It is, however, remarkable that the automatic prediction system predicts the labelled black more as brown (70) than as black (47). This could mean that for the prediction of black, the automatic prediction system is biased towards brown. It is, however, rather predictable that the automatic prediction system predicts a class that is closely related to the labelled class. In reality, the difference between the beer colour classes gold, amber, brown and black is not as clear as the difference between wine colours white, rosé and red, for example. On the following page, Image 3 shows that there is a gradual transition from gold to amber, amber to brown and brown to black : deep gold is very similar to pale amber, amber brown to brown and deep brown to black. Consequently, 32

35 experts might be unable to accurately define the right colour and probably use the same terms to describe these closely related colours. This leads to an automatic colour prediction system that predicts a labelled class as a neighbouring class. Image 3. Gradual transition of beer colour (Mosher, 2015) A second experiment is run with the other settings of the classifier. A grid search was performed on one training fold in order to optimise the parameter settings for the default (RBF) kernel. This resulted in an optimised c parameter value of 2.0 and a g parameter value of These are the results for the second experiment: Category Frequency in corpus Recall Precision F-score Very Light/white Deep Hazy Cloudy Gold Amber Brown Black Red/rose Green Oak The average accuracy is 59.88%, which is higher than the accuracy of the first experiment (56.05%). The recall scores for gold, amber, brown and black are raised with respectively 4.7%, 2.9%, 3,1% and even 14.7%. The change in settings had an especially postive effect on the recall score of black : it has risen from 41.2% to 55.9%. Opposed to this, the precision and f-score of black are negatively affected: the precision score considerably dropped with 19% from 31.8% to 12.8% and the f-score with 15% from 35.9% to 20.9%. Considering the precision and f-scores of gold, amber and brown, the performance of the automatic colour prediction 33