Let’s face it, there are a lot of reviewers out there writing for websites, magazines and books. There are several commonalities: They’ll tell you a bit about the history or backstory, they’ll talk about mashbills, they’ll describe aromas, tasting notes and finishes. Some will even mention prices and where you can pick up bottles (we do that at Bourbon & Banter, but we’re cool like that). The crux, however, is the recommendation and rating.
There can be great misunderstanding and confusion as it pertains to ratings. When I write reviews, occasionally folks will comment and ask why I don’t do a rating in their preferred manner or using a more “classical” method. Some want a scale of one to five (along with cute icons like Glencairn glasses, thumbs, etc.). Others request a scale of one to ten. And, for some, they want a score of one to 100. Then, there’s the 50 to 100, such as those done by Whisky Advocate.
Of course, none begin to touch the bronze, silver, gold and double-gold ratings that you’ll find at tasting events.
But, what do they all mean? In my opinion, not a lot, and all of them leave plenty of room for interpretation. One of the great misconceptions is the award method. At most shows, everyone gets at least a bronze. That’s right! If you’ve paid your entry fee, you’re almost guaranteed to get a participation trophy (bronze). That bronze also doesn’t say much about a whiskey, either. It could be drinkable to decent, but not decent enough to get to the next level, silver. And, silver could be anywhere from the same scale (drinkable to decent). Gold is where it is at, and even then, not all golds are created equal. Then there’s that double-gold. Double-gold can be great, but not as great as Best in Category or Best in Show, which are also equally meaningless because if everything at the show is mediocre, then Best in Show is the best of the mediocre.
Next are the one-in-five or one-in-ten rankings. You’d think these would be simple to understand, but they’re not. Sure, one is pretty much horrible and, depending on the scale, five or ten is stupendous. If, on a five-point scale, a three is average, is that average decent, is that average good, or is that average, well, average? The ten-point scale leaves more room for interpretation but still doesn’t say a whole lot in the end. Do I want to buy a seven? Is there a huge difference between a seven and eight, or an eight and nine? How often is a ten given out?
Many folks enjoy the x to 100 scale ratings. Personally, I get the scale if it starts with one. But, why start it at 50, 60, or some other arbitrary number of points? To me, this is like taking the SAT: you automatically get 400 points just for writing your name. Unless the point is to not hurt anyone’s feelings by saying You suck, it just doesn’t make sense. If I use the Whisky Advocate scale, their ratings aren’t evenly measured. There’s a five-point difference between each of the categories, until, of course, you hit the bottom. The “Not Recommended” category has a 25-point spread. Is a 94-point whiskey really worse than a 95 (Outstanding versus Classic)?
When it comes to ratings, I pefer simplicity. I won’t pretend I formulated the Bourbon & Banter review/rating model, but I adopted it immediately once I saw it. I write reviews that people can both understand and relate. I don’t use fancy, esoteric descriptors that no one but the reviewer comprehends, such as lilac from grandma’s attic or simply beautiful marzipan. The Bourbon & Banter scale is very easy to understand: Bottle (buy it), Bar (try it) or Bust (leave it). It doesn’t compare apples to oranges or even whiskeys to whiskeys. It is a simple recommendation that makes perfect sense.
Known throughout Wisconsin (and now the world) as Whiskeyfellow, Jeff was a late-bloomer to the Wonderful World of Whiskey. At the suggestion of his wife, he started with Scotch and was hooked. He was under the impression that he was happy. A friend asked him several times to try Bourbon, and he eventually gave in, only to fall completely in love with it. Those first steps started him on his #DrinkCurious adventure that led him to #RespectTheBottomShelf. Jeff now relishes many types of whiskeys, ranging from the super-affordable to the super-premium and everything in between. Aside from simply sipping and writing about it, Jeff now enjoys spreading the whiskey gospel by hosting educational tasting events.
Read Jeff's full profile.
About the Author
Completely agree! Thanks Jeff I think that sums things up as the great thing about bourbon is it is different for us all.
Thank you, Ken, cheers!
Thank you Mr. Schwartz for keeping it simple.Cheers!!
That’s always my goal, cheers Jeffrey!
I’m with you. I don’t like rating scales that start at arbitrary numbers. Nothing like reading a review that sounds 100% negative the entire time, then it gets a 72/100 at the end. I think pure moonshine would get a 60 on some of those scales.
I review a few bottles here and there that nobody sees, but my scale goes 1.00 to 5.00 and I actually rate stuff from 1 up to 5. If the juice is terrible, it gets a damn 1! No participation points here.
I’m sure the scales make sense to them, but for me, I want to know if I need to buy it or not. Cheers!
While our ratings were not mentioned by name, the ratings for Sips, Suds, & Smokes are rating of 1-5.
All of our SIPS episodes contain these ratings on the products we discuss.
We wanted something simple and easy to navigate for our audience and consumers.
SIPS Ratings:
1- Give me a glass of water to wash out my mouth!
2- Nice, but what else do you have?
3- Hmm interesting, what was this again?
4- Let’s keep this secret to ourselves-pour me another
5- Oh my, I was unaware anything could be this good
These ratings span a wide range of products and are not specific to whiskey.
We’ve done some experiments with shelf tags in retail locations and may do those in the future.
Most people can remember what we really liked (5) .
There is one common problem, it’s media ratings on products nobody can find. We get lots of products and many advance products to discuss. Just because you can rate something does not mean you should. We like to discuss things that are widely available and people can find at retail. While we know that your cousin’s launch of corn whiskey in Thornburg, Iowa is real exciting, if you can only get it within a 50 mi radius it most likely will never get rated on our global radio show.
We do have some exceptions like our recent Unicorn Don’t Fart episode covering the BTAC 2018 lineup.
Hey Mike… first of all, congratulations on getting picked up by CBS, that’s huge. I know I congratulated Made Man Bob. 🙂 I don’t mind reviewing your cousin’s whiskey from Thornburg. With the gray market, nearly everything is accessible. But, there’s also a big difference between a radio show and a web-based review. You have to be more selective and adhere to a schedule versus dropping reviews whenever it is convenient.
Saying all that, when the day comes where you let me be a guest panelist (please do not use Brother Brent’s nickname for me), I’ll be happy to adhere to your rating system. Cheers!
Jeff, I like your comment about esoteric descriptors like “grandma’s attic and lilac and marzipan,” because not everyone’s been in someone’s grandmother’s attic, smelled a lilac or tasted marzipan. The use of such is risky because they’re so subjective.
So what to do with a “grandma’s attic” memory when you smell it in a whiskey? Simplify it and say, “dusky or musty,” aromas everyone recognizes. But if that person really smells lilac and it’s a prominent aroma, I think it’s now on the table. Paul Pacault, the brilliant reviewer behind the Spirits Journal, once used lilac in a review. I thought I knew what it smelled like, but wasn’t sure. I knew, however, that it was in bloom in Kentucky right at that time and went to a neighbor’s yard and smelled it. I have since picked up lilac in some whiskeys (ryes, I think). So that stretched me to learn more. It’s also true that Pacault uses such descriptors sparingly, i.e. only when it’s the only choice.
Marzipan’s another interesting one because, as a former chef, most of my reference point centers on food flavors. Parker Beam, I was told, couldn’t understand all the food-related descriptors he read reviewers mention because, “All I taste is bourbon. It’s either good or bad.” Whiskey alone was his reference point. A broad range of food and drink, a real melange that’s often muddy and vague, is my reference point. But as much as I do find marzipan notes in some whiskeys, as a reviewer, I should drill down and simplify it to “almond paste,” which will connect with most people.
Reviewing need never be prissy or showy, or so painfully subjective as to show the reviewer’s true colors: a try-hard who thinks he impresses everyone with his sensitive palate. That nonsense is simply irritating. It needs to be balanced between personal taste (one’s unique palate), experience (real knowledge of what one’s tasting) and a little bit of whimsy/humor/edge to make the review an interesting read. In other words, it’s a real writing discipline that too many think they’ve mastered.
Bourbon & Banter has done a wonderful job of working to make reading spirits reviews simple, educational and entertaining.
Jeff, The last time I commented was back in April regarding Elijah Craig. Being retired and living in the Madison area I’ve spent time reviewing the various sources online made available to learn more about bourbon. As I mentioned back in April, my son and I are fairly new to the world of bourbon, and now rye. I’ve accumulated 24 bourbons and rye, opening only 4 so far.
Maybe I’m crazy, but having read the information available from many sources, and also trying the products, your article really makes sense to me more than ever. I’ve read the various reviews of the different brands and how the flavors, aromas, and tastes remind the taster of a candy store, coffee shop, fruit stand, or garden. Maybe they can breakdown those intricate, subtle things, but I can’t. Lets be real here. For a taster to have the great ability of being able to pull a pomegranate flavor from a bourbon, really!!! They should be working at making some serious money as a professional taster.
Back on track now….I’ve come to appreciate the time I get to spend with a pour of a bourbon or rye, sipping it neat. I look for the smooth feel of the liquid, with some sweetness and some spice. I do appreciate a good single barrel Four Roses with its warming spice. I also like the flavor profile of rye as I get more experience with it. However, I do admit the I have picked up some of the flavors associated with a particular bourbon/rye. I’m also trying to learn about what a distiller does to make their product unique and stand out from the rest.
Knowing what little I do, your description of “numbers” really makes some sense. It has allowed me to maybe consider trying a bourbon or rye, but ultimately in the end its the taste of the product that will help me settle on what I keep on hand. I’ve spent a few bucks initially, which I guess is part of learning, but numbers gave me a starting point to work with.
Jeff,
Great article. However, using some of the numbers and awards you speak of, has helped my son and I to focus on some bourbons and ryes that we thought the experts felt were “remarkable.”
You and I had communicated last April regarding your thoughts on Elijah Craig, which has helped me build my collection to 24 bourbons and ryes, four of which I have opened so far. Crazy, right?
Living in and around the Madison area has allowed me to taste a number of these products, moving me to start my collection.
I’ve found some bourbons that really don’t move me. What I mean by that is when I find a sweet profile that also gives me some spice, I try to stick with it. I really like the “O” single barrel Four Roses, but also Eagle Rare, Buffalo Trace, and Michter’s rye.
Tasting notes puzzle me because I can’t seem to get more than one or two, maybe three distinct flavors from these products. When I read a review and the taster is describing hints of pomegranate, or cardamon, or juniper, this drives me crazy!! I’m sorry, but I’m sticking with my simple flavor profiles, a little sweet, a little spicy. The “numbers” and “awards” just give me thought on whether I should buy and try that bourbon or rye. Maybe one of these days we can get together and try some of these products, I’m buying, so that I can find out what charred oak tastes like.
Great article! Last year I analyzed the results from a few different awards competitions and came to the same conclusion.
This lack of clarity, not just in awards but also arbitrary numerical ratings, led to me rank the whiskys I’ve tried in two different ways: the Colley Matrix, used to rank college sports teams; and a simple Page Rank, one of the early methods Google used to rank webpages. These methods connect each whisky I’ve ever tried together in one list from 1 – 302 (once I update the list).
But even after doing all this, it still doesn’t communicate how much I actually like a whisky. People often see how low Buffalo Trace (265) and Jack Daniel’s (281) are and ask why I don’t like them. It’s not that I don’t like them, I just like other whiskies more. They’d be a 2 in your system and I’ll happily by a handle of each over the next year!
I’m new to this game.
Welcome to the game!