Multiple Factor Analysis

Transcription

1 Multiple Factor Analysis François Husson Applied Mathematics Department - Rennes Agrocampus husson@agrocampus-ouest.fr 1 / 39

2 1 Data - Introduction 2 Equilibrium and global PCA 3 Studying groups Group representation Partial points representation Separate analyses 4 Further topics Qualitative data Contingency tables Interpretation aids Outline 2 / 39

3 Sensory description of Loire wines 10 white wines from the Loire valley : 5 Vouvray - 5 Sauvignon sensory descriptors : acidity, bitterness, citrus odor, etc. 3 / 39

4 Sensory description of Loire wines 10 white wines from the Loire valley : 5 Vouvray - 5 Sauvignon sensory descriptors : acidity, bitterness, citrus odor, etc. O.fruity O.passion O.citrus S Michaud Sauvignon S Renaudie Sauvignon S Trotignon Sauvignon S Buisse Domaine Sauvignon S Buisse Cristal Sauvignon V Aub Silex Vouvray V Aub Marigny Vouvray V Font Domaine Vouvray V Font Brûlés Vouvray V Font Coteaux Vouvray Sweetness Acidity Bitterness Astringency Aroma.intensity Aroma.persistency Visual.intensity Grape variety 3 / 39

5 Sensory description of wines : comparing juries 10 white wines from the Loire valley : 5 Vouvray - 5 Sauvignon sensory descriptions from 3 juries : experts, consumers, students tasting note of 60 consumers : overall appreciation Expert (27) Student (15) Consumer (15) Appreciation (60) Grape variety (1) Wine 1 Wine 2 Wine 10 How to characterize the wines? Are wines described in the same way by the different juries? Are there specific responses from certain juries? 4 / 39

6 Multi-tables Groups 1 j J Variables 1 1 k K j Individuals i x ik I X 1 X j X J Examples with quantitative and/or qualitative variables : genomics : DNA, expression, proteins questionnaires : student health (product consumption, psychological state, sleep, age, sex, etc.) Economics : annual economic indices 5 / 39

7 Aims Study the similarity between individuals with respect to the whole set of variables AND the relationships between variables Take the group structure into account Study the overall similarities and differences between groups (and the specific features of each group) Study the similarities and differences between groups from an individual s point of view Compare the characteristics of individuals from the separate analyses Balance the influence of all of the groups in the analysis 6 / 39

8 1 Data - Introduction 2 Equilibrium and global PCA 3 Studying groups Group representation Partial points representation Separate analyses 4 Further topics Qualitative data Contingency tables Interpretation aids Outline 7 / 39

9 Balancing the influence of each group of variables In PCA : normalizing balances each variable s influence (when calculating distances between individuals i and i ) In MFA, we balance in terms of groups 1st idea : divide each variable by the total inertia of the group it belongs to Group 1 Group 2 Group 3 8 highly correlated variables 3 orthogonal variables 3 orthogonal variables 2nd idea : divide each variable by the (square root of the) 1st eigenvalue of the group it belongs to 8 / 39

10 Balancing the influence of each group of variables Doing data analysis, in good mathematics, is simply searching for eigenvectors ; all the science of it (the art) is to find the right matrix to diagonalize Benzécri MFA is a weighted PCA : calculate the 1st eigenvalue λ j 1 of the jth group of variables (j = 1,..., J) do an overall PCA on the weighted table : [ ] X 1 X 2 X ; ;...; J λ 1 1 λ 2 1 λ J 1 X j corresponds to the jth normalized or standardized table 9 / 39

11 Balancing the influence of each group of variables Before weighting After weighting Expert Student Consumer Expert Student Consumer λ λ λ / 39 Same weights for all variables from the same group : group structure is preserved For each group, the variance of the principal dimension (first eigenvalue) is equal to 1 No group can generate the first axis on its own A multi-dimensional group will contribute to more axes than a one-dimensional group

12 MFA - a weighted PCA Same plots as in PCA Study similarities between individuals in terms of the set of variables Study relationships between variables Characterize individuals in terms of variables Same outputs (coordinates, cosine, contributions) Add individuals and variables (quantitative, qualitative) as supplementary information 11 / 39

13 Individuals plot Individual factor map Dim 2 (24.42%) Sauvignon Vouvray S Michaud S Buisse Domaine V Aub Silex S Trotignon S Renaudie S Buisse Cristal V Font Coteaux V Aub Marigny V Font Domaine V Font Brules The 2 grape varieties are well-separated The Vouvray are more varied in terms of sensory perception Several groups of wines Dim 1 (42.52%) 12 / 39

14 Variables plot Correlation circle Dim 2 (24.42%) Expert Student Consumer O.Intensity.before.shaking_S Expression Attack.intensity O.Intensity.before.shaking Oxidation Acidity O.Intensity.after.shaking O.passion O.Intensity.before.shaking_C O.Intensity.after.shaking_S O.Intensity.after.shaking_C O.plante_C O.passion_S O.passion_C O.flower Astringency_S O.citrus A.persistency Bitterness Balance_S Astringency_C Bitterness_C O.plante A.intensity Acidity_S Finesse Acidity_C A.intensity_C O.mushroom_S O.Typicity_S O.fruity A.alcohol_C A.intensity_S Bitterness_S O.alcohol A.alcohol_S O.wooded O.vanilla A.Typicity_S O.mushroom_C Balance_C Astringency A.Typicity_C O.Typicity_C Sweetness_C Sweetness_S Sweetness O.plante_S O.candied.fruit Smoothness O.mushroom O.alcohol_C Typicity O.alcohol_S Visual.intensity Grade.colour Surface.feeling Dim 1 (42.52%) 13 / 39

15 Variables plot Correlation circle Dim 2 (24.42%) Expert Student Consumer O.passion_S O.passion_C Sweetness_C Sweetness_S O.passion Sweetness Acidity Acidity_C Acidity_S / 39 Dim 1 (42.52%)

16 1 Data - Introduction 2 Equilibrium and global PCA 3 Studying groups Group representation Partial points representation Separate analyses 4 Further topics Qualitative data Contingency tables Interpretation aids Outline 15 / 39

17 In PCA (reminder) : arg max v 1 R I First MFA component K cov 2 (x.k, v 1 ) k=1 In MFA : J arg max cov 2 x.k, v 1 = arg max v 1 R I j=1 k K j λ j v 1 1 R I J 1 j=1 λ j 1 cov 2 (x.k, v 1 ) k K j }{{} L g (K j,v 1 ) L g (K j, v 1 ) = projected inertia of all the variables of K j on v 1 The first principal component of the MFA is the variable which maximizes the link with all groups, in the L g sense. 0 L g (K j, v 1 ) 1 L g = 0 : all variables in the jth group are uncorrelated with v 1 L g = 1 : v 1 the same as the 1st principal component of K j 16 / 39

18 Group plot Using L g to plot groups The jth groupgroups has representation coordinates L g (K j, v 1 ) and L g (K j, v 2 ) Dim 2 (24.42%) Expert Student Consumer 1st axis is the same for all groups 2nd axis is due to the Experts group 2 groups are close to each other when they induce the same structure Dim 1 (42.52%) This plot provides a synthetic comparison of the groups Are the relative positions of individuals similar from one group to the next? 17 / 39

19 Measuring how similar groups are The L g coefficient measures the connection between groups of variables : L g (K j, K m ) = cov 2 x.k x.l, k K j l K m λ j λ m 1 1 The L g coefficient as an indicator of a group s dimensionality L g (K j, K j ) = Kj Kj k=1 (λj k )2 k=2 (λ j = 1 + (λj k )2 1 )2 (λ j 1 )2 L g (K j, K m ) RV (K j, K m ) = 0 RV 1 L g (K j, K j ) L g (K m, K m ) RV = 0 : all variable in K j and K m are uncorrelated RV = 1 : the two point clouds are homothetic 18 / 39

20 Measuring how similar groups are > res$group$lg Expert Student Consumer MFA Expert 1.45 Student Consumer MFA > res$group$rv Expert Student Consumer MFA Expert 1.00 Student Consumer MFA The experts give more sophisticated descriptions (larger L g ) The students and experts are quite related : RV = 0.85 The students are closest to the shared configuration : RV = / 39

21 Partial points representation Comparing groups in terms of individuals Comparing descriptions provided by each group in a shared space Are there specific individuals related to certain groups of variables? 20 / 39

22 Data i Projections of partial points G1 G2 G3 xxx xxx xxx xxx xxx xxx xxx Mean configuration of MFA i 21 / 39

23 Data i Projections of partial points G1 G2 G3 xxx xxx xxx xxx xxx xxx xxx Mean configuration of MFA i Projection of group 1 xxx xxx xxx xxx Projection of group 2 Projection of group 3 i 1 i 2 xxx xxx xxx Partial point 1 Partial point 2 Mean point Partial point 3 21 / 39 i

24 Partial points Tutorial participants What you expected for the tutorial What you have learned during the tutorial F 2 What you expected for the tutorial What you have learned during the tutorial What you expected for the tutorial Happy learner Disappointed or pleasantly surprised What you have learned during the tutorial F 1 22 / 39

25 Transition formulas The transition formulas apply for the mean points F s (i) = 1 J 1 K j λs j=1 λ j x ik G s (k) 1 and the partial points F s (i j ) = J 1 λs 1 λ j 1 k=1 K j x ik G s (k) k=1 The superimposed plot with mean points and partial points can be analyzed in the same space 23 / 39

26 Partial points plot Graph with the partial points Dim 2 (24.42%) Expert Student Consumer S Renaudie S Buisse Domaine S Trotignon V Aub Silex S Michaud S Buisse Cristal V Font Brules V Font Domaine V Aub Marigny V Font Coteaux Dim 1 (42.52%) 24 / 39 Partial point = representing an individual as seen by a group An individual is at the barycenter of its partial points

27 Inertia ratios I J I J I J (F i j s) 2 = (F is ) 2 + (F i j s F is ) 2 i=1 j=1 i=1 j=1 i=1 j=1 total inertia = between-individual inertia + within-individual inertia Between inertia on axis s Total inertia on axis s = J I i=1 (F is ) 2 Ii=1 Jj=1 (F i j s) 2 > res$inertia.ratio Dim.1 Dim.2 Dim.3 Dim.4 Dim On the first axis, the coordinates of the partial points are close to each other (0.93 close to 1) The within-inertia on an axis can be broken down by individual 25 / 39

28 Connection with components obtained from separate PCA Do separate analyses give comparable results to the global MFA? Groups 1 j J Variables 1 1 k K j 1 s S Individuals i x ik I X 1 X j X J PCA 1 s S 1 i I 26 / 39

29 Connection with components obtained from separate PCA Principal components of separate PCA are projected as supplementary information Partial axes Dim 2 (24.42%) Expert Student Consumer Dim2.Consumer Dim2.Student Dim2.Expert Dim1.Consumer Dim1.Student Dim1.Expert The PCA dimensions for the students are like those of the MFA The first two dimensions of each group are well-projected Dim 1 (42.52%) 27 / 39

30 1 Data - Introduction 2 Equilibrium and global PCA 3 Studying groups Group representation Partial points representation Separate analyses 4 Further topics Qualitative data Contingency tables Interpretation aids Outline 28 / 39

31 Qualitative data Balance the effect of each group of variables in the global analysis The usual plots for treating qualitative data (individuals and categories) Specific plots (groups plot, superimposed plot, partial axes plots, separate analyses plots) Same methodological approach, just replacing PCA with MCA 29 / 39

32 Groups representation Qualitative data Individual factor map Dim 2 (24.42%) Grade Expert Student Consumer Dim 2 (24.42%) Sauvignon Vouvray S Renaudie S Michaud Sauvignon S Buisse Domaine V Aub Silex S Trotignon S Buisse Cristal V Font Domaine Vouvray V Font Brûlés V Aub Marigny V Font Coteaux Partial axes Dim 1 (42.52%) Individual factor map Dim 1 (42.52%) Dim 2 (24.42%) Grade Expert Student Consumer Dim2.Student Dim2.Consumer Dim2.Expert Dim1.Consumer Dim1.Student Dim1.Grade Dim1.Expert Dim 2 (24.42%) Expert Student Consumer Sauvignon Vouvray 30 / Dim 1 (42.52%) Dim 1 (42.52%)

33 Mixed data Some groups with quantitative variables and others with qualitative variables Locally, MFA behaves like : a PCA for the quantitative variables an MCA for the qualitative variables The MFA weighting allows us to analyze the two variable types together Special case : if each group has just one variable = Factor Analysis of Mixed Data (FAMD) 31 / 39

34 MFA for contingency tables MFA can be extended to contingency tables : MFACT The tables must have the same rows (or the same columns) Examples survey in several countries (Profession Questions / country) ecology : Sites Species / Year Year 1 Year 2 Year K Spe 1... Spe J 1 Spe 1... Spe J 2 Spe 1... Spe J K Site 32 / 39 Frequency of species j in the site i during the year 2

35 Plotting supplementary information Expert (27) Student (15) Consumer (15) Appreciation (60) Grape variety (1) Wine 1 Wine 2 Wine 10 Questions : Are preferences linked with sensory characteristics? Does the grape variety explain the sensory characteristics? 33 / 39

36 Visualizing quantitative supplementary groups Individual factor map Groups representation Dim 2 (24.42%) S Trotignon S Renaudie S Michaud S Buisse Domaine V Aub Silex S Buisse Cristal V Font Domaine V Font Brûlés V Aub Marigny V Font Coteaux Dim 2 (24.42%) Grade Expert Appreciation Student Consumer Correlation circle Dim 1 (42.52%) Correlation circle Dim 1 (42.52%) Dim 2 (24.42%) Expert Student Consumer Acidity O.passion O.passion_S O.passion_C Sweetness_C Sweetness_S Sweetness Acidity_C Acidity_S Dim 2 (24.42%) Appreciation 34 / Dim 1 (42.52%) Dim 1 (42.52%)

37 Indices : contributions and representation quality Individuals and variables : same as the PCA calculations Contribution of the kth group to construction of the sth axis : Ctr s (k) = F ks Kk=1 F ks ( 100) 35 / 39 > res$group$contrib Dim.1 Dim.2 Dim.3 Dim.4 Dim.5 Expert Student Consumer Representation quality of the kth group in a subspace : cos 2 between the kth point and its projection > res$group$cos2 Dim.1 Dim.2 Dim.3 Dim.4 Dim.5 Expert Student Consumer

38 Using quantitative variables : Characterizing the axes correlation between each variable and the sth principal component is calculated the correlation coefficients are sorted and the significant ones retained > dimdesc(res) $Dim.1$quanti $Dim.2$quanti corr p.value corr p.value O.vanilla e-04 O.Int.bef.shaking_S Bitterness_S e-04 Attack.intensity O.wooded e-03 Expression A.intensity_C e-03 O.Int.bef.shaking Grade.colour e-03 Acidity Acidity_S e-03 O.Int.after.shaking Balance_S e-03 Typicity O.Typicity_S e-03 O.alcohol_S A.Typicity_S e-06 O.plante_S / 39

39 Using qualitative variables : Characterizing the axes do analysis of variance with an individual s coordinates (F.s ) described in terms of the given qualitative variable one F -test per variable for each category, a Student s t-test > dimdesc(res) $Dim.1$quali $Dim.2$quali R2 p.value R2 p.value grape variety grape variety $Dim.1$category $Dim.2$category Estimate p.value Estimate p.value Vouvray Sauvignon Sauvignon Vouvray / 39

40 Putting MFA into practice 1 Define the structure of the dataset (group composition) 2 Define the active groups and supplementary elements 3 Standardize the variables or not? 4 Run the MFA 5 Choose the number of dimensions to interpret 6 Simultaneous analysis of the individuals and variables plots 7 Group study 8 Partial analyses 9 Use indices to enrich the interpretation The MFA function of the FactoMineR package 38 / 39

41 Conclusion MFA : a multi-table method for quantitative variables, qualitative variables, and frequency tables MFA balances the influence of each table Represents the information brought by each table in a shared setting Classical outputs (individuals, variables) Specific outputs (groups, separate analyses, partial points) Bibliography Pagès, J. (2014). Multiple Factor Analysis by Example Using R. CRC Press. 39 / 39