ALL BLOG POSTS AND COMMENTS COPYRIGHT (C) 2003-2018 VOX DAY. ALL RIGHTS RESERVED. REPRODUCTION WITHOUT WRITTEN PERMISSION IS EXPRESSLY PROHIBITED.

Friday, February 09, 2018

Trusting science

Are you placing your faith in scientistry or scientody? Because if you believe in the reliability of the latter, you need to understand that the former, on average, no longer practices it:
More than 70% of researchers have tried and failed to reproduce another scientist's experiments, and more than half have failed to reproduce their own experiments. Those are some of the telling figures that emerged from Nature's survey of 1,576 researchers who took a brief online questionnaire on reproducibility in research. The data reveal sometimes-contradictory attitudes towards reproducibility. Although 52% of those surveyed agree that there is a significant 'crisis' of reproducibility, less than 31% think that failure to reproduce published results means that the result is probably wrong, and most say that they still trust the published literature.
This is absolutely incredible. Even Hollywood accounting is not this slipshod! In how many other fields does the failure of the numbers to add up correctly not mean that the result is wrong?

What this means is that nearly 7 in 10 so-called scientists are not utilizing the scientific method at all. What now passes for "science" is now little more than a modern spin on the logical fallacy of argumentum ad verecundiam, the appeal to credentialed authority.

Labels: ,

67 Comments:

Blogger Clint February 09, 2018 11:07 AM  

The twisted pseudo-logic that I see on a daily basis on my college campus is indicative of this. It is not just in the sciences, either. It pretty much permeates the whole of academia. Since these scientists learned the foundation of their craft at their various universities, I am not surprised at all.

Blogger Doug Cranmer February 09, 2018 11:12 AM  

Science is now just a racket like every other racket out there.

Blogger pyrrhus February 09, 2018 11:12 AM  

And it's worse in the medical sciences....A study several years ago found that 85% of NIH papers were not reproducible...your tax dollars at work.

Anonymous Anonymous February 09, 2018 11:15 AM  

"Science is now just a racket like every other racket out there."

Sadly, that has become the truth.

My father was a research scientist with the DOD back in the 60's. Back then they were OBSESSED with accuracy, both in the process and in the results.
Now science is politicized, and they produce the results that the people who pay them want.

Blogger The Cooler February 09, 2018 11:31 AM  

This comment has been removed by the author.

Blogger Nate February 09, 2018 11:34 AM  

"less than 31% think that failure to reproduce published results means that the result is probably wrong, and most say that they still trust the published literature"


Women Ruin Everything

Blogger The Cooler February 09, 2018 11:38 AM  

At some point in the early 90's, universities stop turning out students that had a firm grasp of what a "fact" was. By the late 90's, it had gotten unbearable.

^ My Dad on why he retired early as President of Research from... a medical technology company.

True story.

Blogger pdwalker February 09, 2018 11:42 AM  

This is also a side effect of the dumbing down of universities.

It's amazing that we discover anything new at all these days with the amount of unreproducible crap that is published as gospel.

Can you imagine the risks of taking a new medicine? No thanks.

Blogger DonReynolds February 09, 2018 11:44 AM  

If it cannot be reproduced with the same results, it is not science....it is magic.
Even if the outcome is successful 10,000 independent times, that does not make our understanding a "scientific fact". There can still be a cause that escapes us. And it is that doubt that keeps science open and curious and subject to debate and controversy. Once we begin to insist that scientific understanding is a fact, then it no longer remains curious. Come into science with doubt.

Blogger Rick February 09, 2018 11:46 AM  

I much prefer this scientific method:
By their fruit you will know them.

Reliable: 100%

Blogger Cataline Sergius February 09, 2018 11:48 AM  

Yes but what is your definition of science?

You dont' seem to understand that science is now a religion that makes you feel smug about yourself for believing that there are 102 genders and that Neil deGrasse Tyson is Science Pope. I mean he had a TV show so it was either him or Bill Nye and Nye's White, so he can't be a good science choice.

Pity, he had a bowtie and everything.

Blogger Noah B The Savage Gardener February 09, 2018 11:49 AM  

I had a chem course where several lab teams attempted to reproduce the synthesis of a platinum compound by a recently reported procedure; we all failed. The assistant professor thought we were all idiots, so he tried it for himself, and his attempt also failed. The next week we tried again with all new reagents - same result.

At the time it never occurred to me to suspect that the underlying paper was fraudulent; now I think that's almost certainly the case.

Blogger Stilicho February 09, 2018 11:50 AM  

Wave two feathers over the ouchie and wear this chicken bone necklace for a week.

What, you doubt diverse-science? Raciss!

Blogger Solaire Of Astora February 09, 2018 11:51 AM  

Sounds like scientists are the modern astrologers of Nebuchadnezzar's court. It's no wonder they rely on brow beating to maintain proper group think.

Blogger Jeff Weimer February 09, 2018 11:53 AM  

I beg to differ, Hollywood Accounting is decidedly *not* slipshod; it is creatively and precisely calibrated to accomplish exactly what it's designed to do.

Blogger Carl Philipp February 09, 2018 11:53 AM  

@Noah B
"At the time it never occurred to me to suspect that the underlying paper was fraudulent; now I think that's almost certainly the case."

Results like that SHOULD warrant a paper, and in fact, it SHOULD be possible these days to build an entire scientific career around publishing things like that.

Blogger Cataline Sergius February 09, 2018 11:54 AM  

At some point in the early 90's, universities stop turning out students that had a firm grasp of what a "fact" was. By the late 90's, it had gotten unbearable.

That coincides pretty much exactly with the point in time that Bill Clinton changed the rules on student loans to favor children who had both an over inflated opinion of their own intellect and a bad grasp of lower math.

One of Reagan's best moves was restoring the old standard for student loans. Unless you were genuinely desperate or could prove that you had lived on your own for four years, you were automatically unqualified for a student loan.

Clinton changed that rule to read, anyidiot can get a student loan.

This was the point where universities created programs that they could funnel illiterates into.

Bill and Hillary did a lot of damage to this country but that was easily the worst thing they ever did to America.

Blogger Mocheirge February 09, 2018 11:57 AM  

If non-reproducibility makes acceptable science, does that make miracles scientific?

Blogger Ken Prescott February 09, 2018 11:59 AM  

One issue that I am aware of is that reproducing the experimental setup is getting more and more difficult because what's in the journal is incomplete, whether from fraud, the legal shop demanding withholding of pending IP claims, etc.

Blogger Cataline Sergius February 09, 2018 12:00 PM  

Stilicho wrote:Wave two feathers over the ouchie and wear this chicken bone necklace for a week.

What, you doubt diverse-science? Raciss!


May Science bless you. Oh, child of the True Faith.

Blogger DonReynolds February 09, 2018 12:01 PM  

We really hope that one field in particular is based on science and that is Medicine.
Fortunately or unfortunately, human beings could not wait for the science to be perfected and there were "physicians" throughout human history, many centuries before they could provide any scientific basis for the practice of medicine.
For that reason, I nominate Louis Pasteur as the most important person in the modern era. He was despised and rejected by the physicians of the day, since he was a chemist, who was meddling in their practice of medicine.
Pasteur is an example of a courageous scientist and a great human being.

Blogger wrf3 February 09, 2018 12:07 PM  

Reality is what we say it is.

Blogger Cataline Sergius February 09, 2018 12:08 PM  

DonReynolds wrote:We really hope that one field in particular is based on science and that is Medicine.

Fortunately or unfortunately, human beings could not wait for the science to be perfected and there were "physicians" throughout human history, many centuries before they could provide any scientific basis for the practice of medicine.

For that reason, I nominate Louis Pasteur as the most important person in the modern era. He was despised and rejected by the physicians of the day, since he was a chemist, who was meddling in their practice of medicine.

Pasteur is an example of a courageous scientist and a great human being.


If Nate is reading, I think I will let him answer the question; what do #RealScientists think of medical doctors?

Unless we have another MD that would like to make his angry voice heard.

Blogger Noah B The Savage Gardener February 09, 2018 12:17 PM  

We really hope that one field in particular is based on science and that is Medicine.

LOL

Blogger Lance E February 09, 2018 12:20 PM  

I wonder what the aggregate irreproducibility rate would look like if you eliminated social psychology and its related branches - all likely lumped into that "other" bucket.

It's not looking great anywhere outside physics or chemistry, but at least medicine and biology still resemble actual scientific disciplines with fat middles (50% repro), as opposed to fat tails at ~20-30% repro. And there's an immense, ridiculous amount of "research" being done in that "other" bucket which probably skews the aggregate significantly.

The nod to publication bias is also nice. An experiment is more than twice as likely to publish if it repros vs. doesn't repro. The peer review process that scientists herald as the pinnacle of truth-filtering is actually partly responsible for the massive distortions, even when the research scientists themselves are doing their best to follow the scientific method.

Blogger The Cooler February 09, 2018 12:20 PM  

Pasteur is an example of a courageous scientist and a great human being.

That raw milk, tho.

Blogger Hammerli280 February 09, 2018 12:23 PM  

There's Engineering, Speculation, and Superstition. You can trust your life to only one of these.

Blogger Rory February 09, 2018 12:25 PM  

This is where I fall a little short on buying into the science-based side of alt right politics ("genetic studies show that..."). Scientists, even when acting their best, are shy to make very certain statements. And those p-value of whatever statements then have to be coached in: "oh yeah, and maybe the data isn't even all that reliable, and the turtles can't be reproduced all the way down".

Philosophical arguments and broad common-sense assertions I can buy. When it comes down to "the science/studies/research prove this or that about what happens as a result of genes/races/immigration/etc", I'm a lot less certain. In those cases I do just trust the people who've done more reading than me, and assume they know what they're talking about. ¯\_(ツ)_/¯

Blogger Stg58/Animal Mother February 09, 2018 12:25 PM  

Quantity is not quality.

Oil and gas dominated by white men=you'd better know what the fuck you're doing.

Blogger Stickwick Stapers February 09, 2018 12:31 PM  

Some thoughts from a resident scientist.

Most of the respondents on the survey were in biology and medicine, both of which are dominated by flawed scientific paradigms. Medical research is also heavily influenced by its ties to a billion-dollar corporate-government complex, which practically guarantees poor science.

Physics and chemistry are in better shape, as reflected by the responses from scientists in these fields.

Having freshly emerged from two decades in academia, I can tell you that the problems that do exist in the physical sciences are largely driven by the current hyper-competitive nature of funding, resources, and institutional positions.

Breaking it down:

- In my field (astrophysics), 30-40 years ago you could apply for a grant with a 40% expectation of having it funded. Today it’s 7%.

- There are hundreds of applicants vying for dozens of institutional jobs.

- Many subfields require access to expensive and limited resources, like labs, telescopes, supercomputers, etc.

- The pressure to publish is extreme.

Somebody modeled the effects of all this, and it was pretty disturbing. Even if there is zero corruption and the majority of research is of high quality, the combination of hyper-competitiveness and pressure to publish results in an alarming rate of error propagation.

If we're to minimize unreproducible work in the physical sciences, we have to address these two issues, i.e. we need far fewer scientists than we currently have and we need to return to a much slower rate of publication.

Blogger Allen L. February 09, 2018 12:31 PM  

I noted the fields where the problems were more inherent, biology, medicine, social sciences, etc. I can certainly understand some inability to reproduce results in that the underlying data could probably best be represented by a distribution. But that's not how they sell it is it?

That's where the true dishonesty lies. It is absolutely unacceptable to posit a universal meaning to scientific results that are at best only representative of a portion of the possibilities. That's not just a mistake, it's outright fraud.

Blogger Noah B The Savage Gardener February 09, 2018 12:34 PM  

@28 The elephant in the room is that many of these "extra scientists" come from cultures where intellectual honesty is not highly valued. You've just made a strong argument for eliminating H1B visas.

Blogger Ninny Mouse February 09, 2018 12:35 PM  

The definition of reproducibility according to the questionnaire: "For the purposes of this survey, we consider a study to be reproduced when its findings are confirmed in similar experimental systems (these may include slight variations in methods or materials.)
By contrast, a study is replicated when it is repeated exactly, using the same reagents."

"Different experimental systems" could mean a lot depending on the field and the individual. For example, was it done in different cell lines? If so, there could be valid biological reasons why the results aren't "reproducible". Now, to me that isn't a slight change, but would someone else see it as a slight change. Who knows?

And don't get me started on the documentation issues in science. As in sometimes very small changes can make big differences, but no one thinks to add that change to the protocol.

Not saying that science doesn't have reproducibility issues. It does.

Blogger James Dixon February 09, 2018 12:44 PM  

> When it comes down to "the science/studies/research prove this or that about what happens as a result of genes/races/immigration/etc", I'm a lot less certain.

You've heard the phrase "fight fire with fire", haven't you?

Blogger kurt9 February 09, 2018 1:01 PM  

I think it should be clear at this point that the only reality test of science is if it results in real engineering and technology.

Anonymous Anonymous February 09, 2018 1:02 PM  

This has happened before: the Protestant Reformation.

The University system grew out of the monastic system. The Professoriate is the modern Priesthood, they even wear the same black robes (on formal occassions) and make somber pronouncements about the Heavens ("Look those two black holes collided and we measured a gravitational wave over a distance 1/10000th the size of an atom. Really. Believe us").

Just as the Catholic Church's monopoly was disrupted by the printing press, the Ivory Tower is now being disrupted by the internet.

We will demand reforms, we will destroy their centralized authority (and it's link to the authority of centralized governments) and distribute it.

History doesn't repeat, but it sure does rhyme.

Blogger Steampunk Koala February 09, 2018 1:02 PM  

> When it comes down to "the science/studies/research prove this or that about what happens as a result of genes/races/immigration/etc", I'm a lot less certain.

There are two strong factors in favor of them being accurate though. One, these discoveries are often accidental rather than being sought (IE the study wasn't specifically setting out to prove that certain race/gender groups favor the thing in question). Two, the result has predictive power. If it were merely biased research to prop up a narrative it wouldn't show up unbidden in unrelated studies and cause alarm to equalitarians.

Blogger kurt9 February 09, 2018 1:45 PM  

Some random thoughts I had some years ago. My view on this has not changed one bit.

Our Future

I identified with what is now called the transhumanist worldview starting in 1986, while living in Southern California. This was the confluence of the old L-5 Society, Alcor cryonics, and generalized life extension over the back drop of libertarianism.

Many of you point to numerous negative national and global trends. These trends range from the decline of fertility, the pernicious effects of feminism and PC, and on and on. It is often believed that these trends will somehow make U.S. Society more vulnerable to the “invader” or other competition. Some of you have correctly identified that these trends are not unique to the U.S. But are, in fact, occurring in the East Asian countries to an even greater extent than in the U.S. A few of you have the perception to see that the entire “southern” developing world (Latin America, Middle-east, South-east Asia, and India) are not immune to these trends and are, in fact, only 20 years behind us and China in this.

None of this is relevant. A very good friend of mine pointed out that to a large extent, humanity is not an intelligent species, but rather is a species of termites with a few intelligent individuals, who are disrespected most of the time. The only thing that matters is that those few intelligent individuals are able to create their own future that is INDEPENDENT of the rest of humanity. This “decoupling” from the rest of humanity, if necessary, is fundamental for realizing the future that we aspire to.

This requires the development of “self-replicating” or “exponential” manufacturing technology. A technology that will allow a small group of individuals to accomplish tasks that can only be accomplished by giant corporations or nation-states. These tasks are usually considered to be the development of biological immortality and space colonization (sea steading is short term version). At one point, some of us believed that nano-mechanical nanotechnology (AKA “dry” nanotechnology) would be the magic bullet. As most of us believed, this is likely impossible. Instead, we hope to use both synthetic biology as well as advances in robotic automation and 3-D “fabber” additive manufacturing to accomplish this goal.

The development of these technologies will be driven by commercial competition through out the world, primarily in East Asia, North America, and Europe. I believe the technical capability to “close to loop” in manufacturing automation will be possible by 2040, perhaps as early as 2030. This development, in my opinion, will make possible for a small group of individuals, say, a few thousand, to create an entire industrial infrastructure on their own. I consider the development of this capability, along with radical life extension, by 2040 to be absolutely essential for us to have an open, limitless future for those of us who seek it.

The demographic “crises” will actually aid us in this objective. The Japanese and Koreans, for various reasons, have decided to overcome their declining work forces with automation and robots rather than though immigration. This, of course, will help us immeasurably as they will do the expensive R&D work necessary to realize this, that we do not have the personal resources to do ourselves. The same can be said for the U.S. Military's development of robotics (and additive manufacturing) for warfare purposes. Aging can be cure for around $1 billion today. Much of the R&D necessary to realize the SENS cure for aging is being done around the world for other medical applications.

We can “piggy-back” off of all of these work to realize our capabilities by 2040.

Blogger Fifty Seven February 09, 2018 2:23 PM  

"Offer two oxen to Jupiter. Come back next week if it still hurts."

"Is there a generic option?"

"It's supposed to be two bulls, oxen are the generic."

Blogger Mr. Bee February 09, 2018 2:38 PM  

So basically, they're all "Journals of Irreproducible Results".

Blogger NeoNietzsche: February 09, 2018 3:18 PM  

If you have not yet read this article, consider doing so before your next MD visit!

"Lies, Damned Lies, and Medical Science"

https://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/308269/

Much of what medical researchers conclude in their studies is misleading, exaggerated, or flat-out wrong. So why are doctors—to a striking extent—still drawing upon misinformation in their everyday practice? Dr. John Ioannidis has spent his career challenging his peers by exposing their bad science.

Also:

"When Evidence Says No, but Doctors Say Yes"

https://www.theatlantic.com/health/archive/2017/02/when-evidence-says-no-but-doctors-say-yes/517368/

Long after research contradicts common medical practices, patients continue to demand them and physicians continue to deliver. The result is an epidemic of unnecessary and unhelpful treatments.


And some good discussion of it here:

"John Ioannidis and the Little White Lies"

http://barefootfts.com/john-ioannidis-little-white-lies/

(from which:)
He eventually established predictable error rates in published research which vary based on the type of study. A full 80% of non-randomized studies, 25% of randomized studies and 10% of the most respected variety, large-scale randomized studies are flawed.

This observation wasn’t merely applied to random, poorly popularized studies pulled out of obscure journals. Ioannidis assessed 49 of the most highly regarded medical studies in the past 13 years.


and

The USDA food pyramid is heavily substantiated by a mountain of scientists, government agencies and a multi-million dollar ad campaign. But have you ever noticed that the children raised on those principles are the fattest, most medication dependant ones on the planet?

Blogger Troushers February 09, 2018 3:47 PM  

I'm a Post doc, got about 6 years experience, and I think Vox's scepticism about science is pretty much spot on. Couple of comments - I remember reading a paper explicating how fields matured, and how risk taking innovators would found a new field of endeavour, and be replaced by managers, and then drones as the field matured. Biology is a highly variable study, and so a ripe environment for charlatans, and much of the current reputation science and scientists enjoy in the general public was earned by the pioneers with genuine skill, drive and morals, not the current crop publishing any old crap to get ahead. This is long so I'll make my second point brief: the scientific and publishing process relies to a huge extent on personal integrity, and guess what happens when you stuff it with students raised in low trust, clannish cultures with societal mores that openly shit on such "outmoded Western values"

Anonymous Anonymous February 09, 2018 4:10 PM  

@39:

'"Lies, Damned Lies, and Medical Science"

I've learned this because of my grandmother. She has been sick for 40 years and tried basically every medical treatment under the sun (alternative, standard, and otherwise), and the only thing that relieves her pain is a osteopathic adjustment (and that holds for about only half a day at the most).

Blogger Matthew February 09, 2018 4:28 PM  

Magic Labcoats

Blogger dvdivx February 09, 2018 5:03 PM  

The problem is modern "scientists" come up with a conclusion then look for facts to support it. Global warming is a classic example of non-science.

Blogger The Pepe report February 09, 2018 5:05 PM  

I love how when democrats want some kind of science to be widely accepted and not questioned they use a scientist spokesperson with glasses, because glasses obviously means the person is very smart

Anonymous Anonymous February 09, 2018 5:44 PM  

It comes down to money. Hardly anyone is willing to pay for reproducability studies, especially in the soft sciences where studies are very expensive and difficult to do.

Blogger The Observer February 09, 2018 7:23 PM  

Science was better when it was the hobby of aristocratic gentlemen, and you had to prove your findings in public before a heckling audience.

Peer review was the final nail in the coffin, deciding what was true and what not by committee behind closed doors.

Blogger Wynn Lloyd February 09, 2018 8:31 PM  

Thanks for the informative summary. It's definitely interesting.
Would you believe that at one of the faculty meetings in my department (History), we were told that we should, "apply to grants with the same fervor of scholars in the sciences," which is setting people up for disappointment. You might be able to get a small travel grant here and there to go do research in a foreign library, or to produce a book in a particular area, but no way are grants ever going to be provided in the humanities like in the sciences. It's absurd on the face of it, but just like humanists now have to have pseudo-scientific jargon in every article, I guess we also now have to mimic the scientists' pursuit of funding.

Blogger Stickwick Stapers February 10, 2018 12:37 AM  

Peer review was the final nail in the coffin, deciding what was true and what not by committee behind closed doors.

That's not what peer review is.

In the physical sciences, peer review is just a quality filter for publication.

Here's how it works:

- You finish your experiment, and you write up a paper.

- You submit your paper to XYZ Journal.

- The editor of XYZ Journal sends your paper to one or more scientists in your field -- your peers -- who become the paper's referees.

- The referees read through your paper and try to determine whether it meets the standards of the journal. The referees check to make sure:

a) your experiment addresses an important scientific question
b) your methodology is sound
c) your analysis is sound
d) your conclusions are warranted based on the results you presented

On this basis, they will decide whether to:

A. recommend the paper for publication as is (rare)

B. recommend the paper for publication with the proviso that you make anywhere from minor to major changes

C. reject the paper for publication

The editor makes the final call on whether the paper is accepted or rejected for publication.

If it’s B, there’s usually a lengthy back and forth process between you and referee to ensure that the changes are properly implemented. This process can take up to a year.

If C, you can petition the editor to have other referees assess the paper. If it’s still rejected, you can submit to another journal or just post it on a free preprint server like arXiv.

The only thing peer review is intended to guarantee is that the paper meets publication standards. It’s not intended as an arbiter of truth.

Whether or not your paper represents scientific truth is hopefully sussed out over time as the rest of your peers subject it to scrutiny.

Blogger Troushers February 10, 2018 5:13 AM  

You're making a motte and bailey style argument, Stickwick. We are talking about reality here, not theoretical notions of a perfect peer review process

Blogger The Lab Manager February 10, 2018 7:03 AM  

Most of the stuff in academia is worthless junk. Even as graduate student in engineering in the 90's I used to peruse some articles for my research and think who has time to really read all this and then sort through figure out which is not garbage.

IEEE alone has (3) dozen journals in various areas. It's just as bad for the other fields.

Blogger The Observer February 10, 2018 7:11 AM  

@48 and @49:

Exactly. "Referees". "Editor". Instead of demonstrating your findings in front of everyone, decided by committee behind closed doors.

Peer review is not part of the scientific method. Peer review keeps out cranks, but it also keeps out the impious. It makes it safe for a science to become a religion, and for scientists to become priests of a state sponsored theocracy. Cranks have never been a threat to science, state sponsored theocracy has always been a threat to science, thus peer review has always been a failure. It is caused by science becoming theology, and causes science to become theology.

Back in the days when physics made dramatic progress, there was little peer review, perhaps none. For example, “Electrodynamics of Moving bodies” was not peer reviewed, and my guess is that today it would never have passed peer review – because it was written by a patent clerk, and was in large part a novel way of looking at results that were a hundred years old. Outsiders, and novel ways of looking at things are pretty much guaranteed to fail peer review, so you have to wait for the gatekeepers to die and hope the new ones are more amenable.

Blogger NeoNietzsche: February 10, 2018 7:52 AM  

@41 "She has been sick for 40 years ... and the only thing that relieves her pain is a osteopathic adjustment (and that holds for about only half a day at the most)."

When medicine "fails" -- look to diet! I quit eating wheat (mostly) a few years ago -- and yeah, it was REALLY hard! Wheat, apparently, lights up the same brain pleasure centers as morphine! (I think it is -- some endogenous *lovely* drug...) Now, when I do have wheat (oh I LOVE pasta!), with a half hour my carpal tunnel comes ROARING back; and by the next day my arthritis in my hips comes to visit for 4-5
days! Then, the inflammation dies down, and no more pain. She/you might try any of these:

"Wheat Belly" Dr Wm Davis
"Grain Brain" Dr Daniel Perlmutter

If you like reading the science, this is a BRILLIANT book by the world's best science writer! (Yes, I'm absolutely prejudiced! I'm a tech editor by profession!) "Good Calories, Bad Calories" by Gary Taubes (publ. in Britain as "The Diet Delusion"). If you DON'T like science, his popularizer book (and without the 170 pages of refs!): "Why We Get Fat and What to DO About It."

If you "like to watch": Tom Naughton's very funny speech, "Science for Smart People" https://www.youtube.com/watch?v=y1RXvBveht0 and documentary: "Fat Head" https://www.youtube.com/watch?v=evcNPfZlrZs (both free on YouTube). (Tom also was invited to give 'testimony' to a Congressional panel on "Diet, Health, and the Wisdom of Crowds" https://www.youtube.com/watch?v=KzPnnDDCIjo . (The panel was looking into WHY so many Americans have quit believing in the "orthodoxy" and gone looking for what people are actually succeeding with, not what Big Pharma /Big Ag / Big Medicine is directing/selling us to do.)
Avalanche

Blogger NeoNietzsche: February 10, 2018 7:54 AM  

@43 "Global warming is a classic example of non-science. "

Helluva money-maker though...

Blogger NeoNietzsche: February 10, 2018 7:59 AM  

@48 "peer review is just a quality filter for publication."

Uh, don't you mean that's what it's SUPPOSED to be! We've got lots of examples of: "this paper denies global warming; this 'scientist' is a heretic and must be publicly burned at the stake! Then denied future funding and sentenced to a dank room in the basement if tenured, and turfed out if not!"

And the peer reviewers ride their own hobby horses about the topic, and some don't even check the math, and some don't have or spend the time to actually consider what's in the paper...

It MAY have been a useful system at one time. Now? It's as riddled with .... problems ... as the FBI!

(Sure, you may know "reviewers" who still try -- but WAY too many don't.)

Blogger VD February 10, 2018 9:36 AM  

The only thing peer review is intended to guarantee is that the paper meets publication standards. It’s not intended as an arbiter of truth.

Exactly. If used correctly, it's a fancy name for a crowdsourced editorial process. At worst, it's institutionalized blacklisting.

Blogger Stickwick Stapers February 10, 2018 9:42 AM  

You're making a motte and bailey style argument, Stickwick. We are talking about reality here, not theoretical notions of a perfect peer review process

No, dopey. I was responding to the patently false claim that the peer review process has anything to do with “deciding what was true and what not by committee behind closed doors.”

It’s supposed to be a quality filter, nothing more. Whether or not it’s effective in that capacity is another issue entirely, and you will never hear me argue that it’s free of the usual human limitations.

Blogger Stickwick Stapers February 10, 2018 9:52 AM  

Exactly. "Referees". "Editor". Instead of demonstrating your findings in front of everyone, decided by committee behind closed doors.

I’m not sure what you mean by “demonstrating your findings in front of everyone.” How do you propose scientists demonstrated their findings in front of everyone back in olden times — in front of audiences in some kind of traveling scientific roadshow?

They published in books and journals, just like they do now, all of which were subject to some sort of review process. Newton’s groundbreaking paper on his new theory of light was published in a peer-reviewed journal in the 17th century. Newton was famously reticent about publishing his results and withheld his most important work, The Principia, for years, and guess what the result was? Nobody knew about his work.

Scientists have always "demonstrated their findings in front of everyone" by publishing, and if you think it was without some kind of review filter, you’re delusional. An editorial board or publisher always decided whether the work merited publication. Today, you have much greater freedom to publish with the internet and publish-on-demand than scientists ever had in Newton’s or Einstein’s time. If you're an outsider, you may not have access to the built-in audiences of journals, but you have far more than anyone had for 99% of scientific history.

Blogger Stickwick Stapers February 10, 2018 9:58 AM  

Back in the days when physics made dramatic progress, there was little peer review, perhaps none. For example, “Electrodynamics of Moving bodies” was not peer reviewed…

The editors for Annals of Physics reviewed Einstein’s papers and decided whether they were fit for publication. Just like the editors for any journal today decide whether a publication is fit for publication. The difference, as Vox pointed out, is that editors now outsource much of the review process. In 1905, it was still a committee of human beings with their own biases and limitations deciding what got published. In fact, that was much of the impetus behind making peer review the standard.

Reasons for establishing the peer review process:

- To mitigate bias: There was concern that editors and editorial boards were imposing their own narrow views and biases on papers submitted to journals. It was intended that review from amongst the population of peers would minimize that bias.

- To deal with the sheer volume of submissions: There are orders of magnitude more submissions to journals today than there were a hundred or three hundred years ago. One editorial board alone cannot possibly filter them.

- To expand expertise: Members of an editorial board have limited expertise even in their own fields. An expert in condensed matter physics likely doesn’t know what’s going on in cosmology. The idea was that condensed matter papers should be reviewed by condensed matter experts, etc.

Note that this does not constitute an argument for why peer review is good, trustworthy, etc. Intentions are not outcomes, and I have my own grave concerns about the peer review process. But disinformation about what the peer review process is and fantasies about some golden age before peer review are not helpful.

Outsiders, and novel ways of looking at things are pretty much guaranteed to fail peer review, so you have to wait for the gatekeepers to die and hope the new ones are more amenable.

Thus it has always been. Read Kuhn’s The Structure of Scientific Revolutions. It’s not the peer review process holding science back, it’s the entire culture of science at present. You're concerned, I'm concerned, we're all concerned, and nothing has changed since the inception of scientific thinking. There has never been an era that was free of philosophical and practical roadblocks to the advancement of science. It's just human nature.

Blogger Danby February 10, 2018 12:01 PM  

The problem, @Stickwick, that you don't much see as you're on the inside, is that peer review is now used as the measure of scientific truth in the popular discussions of science. This was promulgated by the global warmists.
If it's not published in a peer reviewed journal, it's false, and if it is so published it must be true. And since we've maneuvered to control the peer review process, anyone who disagrees with us in any detail is just a crank.
Whether that is what it's supposed to do, or is designed to do, is irrelevant. That is how it's being used, as a secret committee that determines the truth.

Blogger Michael Maier February 10, 2018 12:31 PM  

@Stickwick: Thank you for the your input on this thread.

Be well.

Blogger Allen L. February 10, 2018 12:39 PM  

The purpose of a refereed journal, which by definition includes peer review, is to provide a forum for people to publish new scientific findings. That's it, nothing more. Not truth, not something given from on high. The fact that other people ascribe more to it than that is their problem, not the journal, nor the process. Is it misused? Of course it is, there are people involved aren't there?

Stickwick, has it exactly right, though I never really put it that way, it's a filter. Not every finding is worth publishing, there are finite resources and finite time limitations.

The whole system has become unwieldy though. When I first started out I could spend a few hours per week reading the journal articles related to my field. Now? Forget it, it's a veritable flood of information. Which creates it's own problems.

Blogger James Dixon February 10, 2018 12:53 PM  

> ..because glasses obviously means the person is very smart

Adjusts glasses... Well, that goes without saying.

I thought I even saw Vox in a pair of reading glasses in one of the darkstreams a while back.

> ...peer review is now used as the measure of scientific truth in the popular discussions of science.

That's not the fault of scientists or the peer review process though.

Blogger Stickwick Stapers February 10, 2018 3:10 PM  

The problem, @Stickwick, that you don't much see as you're on the inside, is that peer review is now used as the measure of scientific truth in the popular discussions of science.

I see that all the time, and it’s frustrating. But, as Allen and James pointed out, whose fault is that really? All you have to do is actually read peer reviewed scientific papers to be disabused of the notion that they’re some godlike measure of truth.

Even good quality papers that follow proper research protocols are filled with academic weasel words like “preliminary,” “possible,” “suggest,” “imply,” “further improvement,” “constrain,” and “indicate.” Just look at this week’s abstracts on the astrophysics preprint server. Excerpting from the first three abstracts:

However, further improvement of automated techniques is needed, which is especially important having in mind the very large numbers of new discoveries expected from all-sky surveys.

Finally, we demonstrate that the Sun has a subtle deficiency in refractory material relative to ~95% of solar twins, suggesting a possible signpost for planetary systems like our own.

Combining the 21cm 2PCF and 3PCF can further break the degeneracy between the linear growth function of matter fluctuations and the reionization bias parameters, and hence constrain cosmological models during the epoch of reionization.

Now, I’m not saying “weasel” in derogatory way, but rather to show the humility that’s implicit in good research. These words are acknowledgement that none of this information has been handed down from on high, and that it’s all subject to the inherent limitations of any human endeavor.

Any person who can read through stuff like that and still claim that it represents “truth” has much bigger problems to worry about, like how to function in life, because he wouldn’t know truth if it walked up and kneed him in the groin.

Blogger Stickwick Stapers February 10, 2018 3:30 PM  

Ha, I just noticed the fourth paper in that abstract list. It's right there in the title:

The Planes of Satellite Galaxies Problem, Suggested Solutions, and Open Questions

It's a review paper, which means it's summarizing the current state of the problem, and right there, the title is an open admission that there's still a lot scientists don't know.

Seriously, if anyone claims scientific literature represents "truth," they're either lying, stupid, or they haven't even so much as glanced at the research.

Blogger James Dixon February 10, 2018 7:57 PM  

> ...because he wouldn’t know truth if it walked up and kneed him in the groin.

Unfortunately, that seems to describe far to many people nowadays.

Post a Comment

Rules of the blog
Please do not comment as "Anonymous". Comments by "Anonymous" will be spammed.

<< Home

Newer Posts Older Posts