Just as surely as the swallows make their annual return to Mission San Juan Capistrano, U.S. News and World Report (USNWR) embarrasses not only itself, but the scientific and nutritional communities via its annual (January) subjective ratings of various popular, not so popular, and virtually unknown diets. As I and others have previously pointed out in 2014’s Rebuttal and 2013’s Rebuttal, these listings have no basis in objective science, but simply represent tallied subjective ratings by a group of experts, cherry picked by an unknown process, presumably by non-scientists at USNWR. I recognize a number of colleagues and good scientist in the expert list and am dismayed that they would participate in such a baseless and non-scientific process. Surely they did not participate in analyzing the data, because it is statistically inappropriate.
Sound nutritional science should not be about people’s opinions, but rather should employ the scientific method in which hypotheses are tested and conclusions drawn using replicable data that is statistically analyzed. Clearly, these procedures were not even remotely followed for this report, consequently the data USNWR has compiled is virtually meaningless. A better use of resources would be for USNWR to actually experimentally test one or more diets against any other diet for the seven end points of concern in this report. Or, at the very least, the panel of experts should be required to read, tabulate and analyze peer review publications of the diets rated in this report. This objective undertaking would be virtually impossible as more than half of the diets have never been written up in peer review scientific journals, much less experimentally tested against one another.
Evidently, a non-scientist at USNWR, not versed in statistics, compiled these results. Let me explain. The panel of 22 experts were asked to give each diet a 1, 2, 3, 4 or 5 ranking for each of the seven categories of interest (short term weight loss, long term weight loss, easy to follow, nutrition, safety, diabetes, heart health). When data is correctly analyzed, one of the first decisions is to determine the level (nominal, ordinal, interval or ratio) of the data. The expert panel’s 1 to 5 rankings represents ordinal data and not higher level interval or ratio data. Accordingly, it is statistically incorrect to calculate a mean score from ordinal data for each of the seven categories as USNWR has done. Rather a frequency distribution displaying the totals, for each of the 5 rankings, is required for all seven categories to more accurately display this data. But why even bother, as this subjective information, even if analyzed correctly, has zero objective merit when making any logical decisions about diet.
Loren Cordain, Ph.D., Professor Emeritus