These sequences were then quantified according to the three criteria developed. For each of the 166 performances, Pauline Larrouy-Maestri used an objective analysis of three possible types of errors and therefore three values. Following this, the researcher selected 18 experts who had studied classical music for several years or who had some expertise relating to the voice. They evaluated the 166 songs on a scale of 1 to 9 ranging from “not in tune at all” to “absolutely in tune”. “We therefore had three objective criteria for judging accuracy and 18 subjective evaluations from expert judges. We were able to compare objective and subjective data, apply statistical analyses to observe the relationship between the opinions of the different judges and the value for the three criteria measured”. The result showed a strong correlation between the objective criterion and the judgement of the different experts. For example, if the computer identified a large number of interval or tonality errors, the judges gave this same performance a low mark (for more details see below: “layman listeners take part in the study”). This results , published in 2013 in the Journal of Voice, were also used for the present study. If the researcher had just found a model that was capable of quantifying and explaining the perception of accuracy by the human ear, her experiment had only just begun.

A large number of studies show the benefits of learning music. Musicians have better attention spans, better concentration and sound perception, are better at learning languages and develop faculties which they can transfer to other activities. “I am not saying that these studies are unfounded”, says the logopedics expert. “But from their point of view, they draw a picture of a musician as a person with particular cognitive faculties. And yet, I have friends who are not musicians and sing better than me or who have a keener sense of hearing and who can recognise and reproduce sounds, or find a theme on a piano without ever having learned to play this instrument. All our lives we sing or listen to music, and assimilate the musical system in which we live. At school we make music together, we experiment with writing songs… So to some extent, we are all musicians. In the same way we are all inclined to evaluate or judge the performances of others”.

Layman listeners take part in the study

“We are all musicians”. This is the intuition that the speech-language expert followed in order to scientifically verify that non-musicians had an implicit musical training. To complete her experiment, she had the 166 versions of “happy birthday” evaluated by 18 new judges who were laymen this time. The judges were required to have had no musical training but needed to be able to hear correctly (normal audiometry and not low-scoring on the Montreal Battery of Evaluation of Amusia). They were also chosen in such a way as to be able to suggest similar profiles to those of the experts. The same proportion of men and women, young people and elderly people from the same socio-cultural background… The only major difference was the absence of formal music training. In contrast to the experts, the non-experts had to take the test two times at intervals of eight to fifteen days. This strategy ensured that they did not change their judgement criteria in the interim which would have suggested, for instance, that the experiment was itself a learning process that had had influence on the participants.  


The main thrust of the research can be summarised in the figure below which lists the relationship between the opinions of the different judges and the results at the three objective criteria. The first column represents the data for the group of experts (mentioned in the 2013 article), the second and third columns represent the data gathered for the two non-expert groups. On the Y axis, the gradation ranging from 0 to 1 represents the correlation coefficient between the objective values and the opinions of the judges. The closer the coefficient is to 1, the more the objective criteria analysed by the author of the experiment correlate with the judgement of the judges. For example, the first box on the top left represents the opinions of the experts and the criterion “deviation of intervals”. Before being submitted to the panel, the 166 versions of “happy birthday” were objectively analysed according to this criterion. In other words, the approximations around the intervals between two notes were quantified. The correlation coefficient for this box is around 0.8. “This result signifies that the two operations are connected”, explains the researcher. “The more the computer records a high level of precision in the intervals, the higher the score given by the judges. This is reassuring because it was to be expected”.

The red line present on each of the boxes corresponds to the level of significance of the results. Everything below this line is considered as statistically significant. “The relationship between the judgment and the objective criteria do not occur by chance. If we reconducted the same experiment, we would have a 95% chance or greater of obtaining the same results, or, put another way, that the results observed have a less than 5% chance of being obtained by chance”. Another general point to be understood from this figure is the gradation of the X axis ranging from 1 to 18. “The black box above the number 18 represents the average opinion from among the 18 judges. To the left of the box is the average of 17 of the 18 judges taken randomly etc. On the far left, the opinion of only one judge is represented, also taken randomly. In all of these cases, we observed the relationship, the correlation between these groups of judges ranging from 1 to 18 and the objective analyses obtained by means of our computer programs. It is to be noted, for example, that in the first box, we only need 3 judges to obtain a strong correlation between measurement of objectivity and evaluation”.