The Judgment of Lenox – Box Wines Taste Best
In recent wine postings, I have reviewed the findings of blind wine tastings back to the 1973 “Judgment of Paris”. In Paris, Californian wines did “as well as” French wines for the first time. Since then, Robin Goldstein and others analyzed data from 6,000 blind tastings. I quote from their findings:
Individuals who are unaware of the price do not derive more enjoyment from more expensive wine. …we find that the correlation between price and overall rating is small and negative, suggesting that individuals on average enjoy more expensive wines slightly less….
Lecocq and Visser analyzed data from three data sets totaling 1,387 observations on French Bordeaux’s and Burgundies. They report similar findings:
When non-experts blind-taste cheap and expensive wines they typically tend to prefer the cheaper ones.
Only the Chinese remain willing to pay exorbitant prices for French Bordeaux and burgundies. But I hasten to add: other studies have shown that like perfume, higher prices and brand names do “add value” for wine buyers.
With this as backdrop, I turn to the results of the latest tasting of the Lenox Wine Club.
The Lenox Wine Club – Sixth Tasting
The Lenox Wine Club (LWC). LWC was formed last fall. Consisting of 14 “experienced” wine drinkers, it has conducted 6 blind tastings:
- “Heavy Whites”;
- “Heavy Reds”;
- “Heavy Red Blends”;
- “Light Whites”;
- “Light Reds”, and
- “Latin American Wines”.
At the sixth tasting, the featured wines all came from Latin America: Argentina, Chile, and Uruguay. The wine varietals included: Malbec, Carmenère, Tannat, and a blend including Cabernet Sauvignon, Carmenère, Merlot, and Syrah.
We tasted two Malbecs:
- A 2007 Alta Vista Alto Mendoza from Mendoza, Argentina. This wine received a 94 rating from Wine Spectator (WS), the highest WS rating LWC has ever tasted, and its price reflects it – $56.66.
- A Bota Box Malbec from Argentina. WS does not rate box wines, even though they have done very well in our tastings. And the price is right: you can buy the 3-liter box at supermarkets for $15 which works out to $3.75 equivalent for a normal-sized wine bottle (750 ML).
We tasted a 2011 Casa Lapostolle Carmenère Cuvée Alexandre from the Apalta Vineyard in the Colchagua Valley of Chile. This wine was also highly rated by WS (92) and cost a reasonable $17.73.
Uruguay is has been producing wine for many years, but very little gets to the US. The Tannat grape is grown widely in Uruguay and is seen rarely elsewhere. We tasted a 2011 Pueblo del Sol Tannat that has not been rated by WS. Its price is very reasonable – $8.73.
A Blend: Cabernet Sauvignon, Carmenère, Merlot and Syrah
And finally, we tasted a red blend from Chile – a 2010 Veramonte Primus “The Blend” from the Colchagua Valley. WS did not rate the 2010 vintage but gave it ratings of 90 for the two previous years. RP rated it 87 in 2010. The price is a modest $16.66.
Testing Tasters’ Competency
While there were 5 wines, we actually tasted one wine in two separate glasses. What is this all about? As I have described in an earlier posting, Robert Hodgson has his own winery and has been troubled by erratic ratings his wines had been receiving from judges at tastings. So he came up with a way to rate potential judges. The key to his method? Have the candidates do blind tastings that include more than one glass of the same wine. If the candidates do not score glasses of the same wine nearly the same, they are not competent to judge wines. Hodgson’s suggested overall scheme is quite rigorous: candidates must do four blind tastings of ten glasses each. We used his methodology in a less rigorous way: two glasses poured from the same bottle: we can’t be as sure this will single out incompetent tasters but the results are “indicative”.
The results are presented in Table 1. In the tasting, 2 glasses of the Bota Box were tasted: Bota Box (1) and Bota Box (2). The wines were reverse rated meaning the wine with the highest total was considered the best. Bota Box (1) was the winner followed by the Blend and then the Alta, the highest priced wine.
Table 1. – Lenox Wine Club Tasting Results – Latin American Wines
The “Bota Box Spread” is simply the absolute difference between the ratings of the two Bota Boxes. The spreads ranged between 0 and 4. Recognize that with a 6 to 1 ranking system, the maximum spread is 5. Suppose we take these spreads seriously as a meaningful indicator of taster competence and look only at the rankings given by tasters with a spread of 1 or less. If this is done, Bota Box (1) wins by a larger margin and Bota Box (2) comes in second. Beyond that, the Casa Lapostolle stays in last place with the other three registering the same scores.
Looking Only at the “Tails”
I have recently argued that most wines taste the same. I went on to suggest that consequently, the only really interesting wines are those we either really like or dislike. One way to see how the wines did on this basis is to count the number of times each wine got the top or bottom score (6 or 1) with ties (5.5 and 1.5). This is done in Table 2. The Bota Boxes still did okay, but the really interesting wine measured by tails was the Alta Vista, the most expensive wine. 8 people really liked it while 6 really did not like it.
Table 2. – Number of “Tail” Scores
Reflections on All LWC Tastings
Looking back on the six LWC tastings to date, what really stands out? Take a look at Table 3. We find that a box wine either won or came in second in all six tastings and that the most expensive wine in the tastings came in last or next to last in five of the six tastings.
Table 3. – Box and Most Expensive Wine Ratings, LWC – 2012-13
What do we make of these findings and how definitive are they? Findings are more definitive if all tasters agreed. Perhaps the best statistic for measuring taster’s agreement is the Kendall’s Tau: a higher number indicates greater uniformity among tasters. The Tau for our six tastings ranged from a low of .013 for the “heavy whites” tasting to .222 for the Latin American tasting. These numbers suggest very little uniformity among tasters.
That having been said, two findings from the tastings are nevertheless quite striking:
- The consistently excellent performance of the box wines, and
- The consistently poor performance of the most expensive wines.
How can the consistently poor performance of the expensive wines be explained? Were they low-rated and with inflated prices? There are certainly plenty of these around. But the high-priced wines we chose got high ratings! Let us consider each of them.
- At our first tasting, the high-priced wine was the Joseph Drouhin, Clos des Mouches. It had just been rated the best white wine at the Judgment of Princeton. In addition, it received a 92 rating from Wine Spectator (WS) and a 93 rating from Robert Parker (RP). Our tasters rated it next to last.
- At our second tasting, the Neyers Cabernet was the expensive wine. WS gave the 2009 vintage a 92 rating but the MA distributors I had to buy from (another story) only had the 2010 so I took a chance on it. Dead last.
- At our third tasting, the Remoissenet was chosen. RP gave it a 90 rating. Dead last.
- At the fourth tasting, the Bordeaux Chateau Dominique was rated 93 by both WS and RP. Next to last.
- The high-priced wine at the fifth tasting was the Greywacke. It got a 92 from WS. Dead last.
- And finally, at the last tasting, the Alta was rated 94 by WS and 92 by RP. Middle of pack rating.
In short, the pedigrees of all the expensive wines were in order. For me, this accumulated evidence is strong enough to change my wine purchasing patterns: while I am happy to accept and drink more expensive wines as gifts, I will never again pay more than $15 for a bottle of wine.
Consider next the performance of the box wines: five top scores/ties and two 2nd place scores/ties. The consequences for me: most of the wines I now purchase are box wines. To give wine drinking experience from a box a degree of elegance, I normally pout the wine from the box into a decanter before dining. And I have started giving box wines as house gifts when visiting others.
But before moving on, I want to be clear. Nearly all wines are good enough today so that “Taste” is not alone in adding value to a wine. For some, a more expensive wine is always better. And who can forget that romantic night in that little bistro on the Left Bank when we found the greatest wine ever…. Fine. We all have these memories and they will always be great. And they add value. But I submit that the wonder of these moments has very little to do with how the wine actually tasted….