ALL BLOG POSTS AND COMMENTS COPYRIGHT (C) 2003-2020 VOX DAY. ALL RIGHTS RESERVED. REPRODUCTION WITHOUT WRITTEN PERMISSION IS EXPRESSLY PROHIBITED.

Monday, August 27, 2018

The gold standard for truth

Is now flipping a coin. If recent studies in reproducibility are to be believed, modern science is now less than 50 percent reproducible.
The same could be said about big projects in which psychologists work together to replicate past studies. Six such projects, including the SSRP, have now been completed. Between them, they’ve successfully replicated just 87 out of 190 studies, for an overall rate of 46 percent. “This is not acceptable,” says Simine Vazire from UC Davis.

The 62-percent success rate from the SSRP, though higher, is still galling to Vazire, since the project specifically looked at the two most prestigious journals in the world. “We should not treat publication in Science or Nature to be a mark of a particularly robust finding or a particularly skilled researcher,” she says. These journals “are not especially good at picking out really robust findings or excellent research practices. And the prediction market adds to my frustration because it shows that there are clues to the strength of the evidence in the papers themselves.”
It's really rather appalling to contemplate that the foolish likes of Richard Dawkins and Sam Harris were dumb enough to seriously believe that what has - at best - a reliability rate of less than two-thirds should be our primary metric of reality.

Labels: ,

59 Comments:

Blogger Dirk Manly August 27, 2018 3:42 PM  

Well, first, you have to consider what sorts of people go into Psych and Sociology in the first place. Most of them aren't particularly interested in truth, as much as finding a convenient story to support their favorite manifestations of psychosis.

Blogger Daniel August 27, 2018 3:57 PM  

See, self-correction in science is one of those reliable processes that just takes billions and billions of years to complete. And if that doesn't work, there are always more illions we can throw at it.

Blogger Daniel August 27, 2018 4:05 PM  

Dirk Manly...Nature and Science are not psych and sociology only journals. Are you really willing to bet that their bio and cold fusion experiments are more reliably reproducable?

Blogger Maorio August 27, 2018 4:09 PM  

as it is now there is no down side to dumping out a paper that no one can replicate. at worst you get a smack on the fingers and that's that. now engineering on the other hand, money is involved and lots of it and possibly your own hide so you make very sure never to hit the brick wall of reality or at least know where it is at the moment.

Blogger Johnny August 27, 2018 4:10 PM  

A long time ago I was involved in research in the area of Economics. The methods used made heavy use of multi-variate statistical analysis. What became apparent to me in time was how extraordinarily easy it was to fall into a pattern of confirmation bias. And how common it was. When the data seemed to support conventional thinking, the tendency was to regard it as correct. If it didn't fit current models then it was easily thought of as spurious. I forget who said it, but one wry comment was that if you torture the data long enough, eventually it will tell you what you want to hear.

Blogger tz August 27, 2018 4:11 PM  

Scientific truth is now a fiat currency. And going Whisky Zulu Victor.

Blogger Homesteader August 27, 2018 4:13 PM  

Incentives matter.

The Royal Society/ Noble Amateur/ love of-discovery-of-God's-creation approach was replaced bythe peer review/ publish or perish/ patentable-profitable approach.

Alignment of incentives is everything.

Blogger Unknown August 27, 2018 4:19 PM  

*Adjusts fedora* But...but...muh science should be a guide for our morality.

As an actual scientist, I can say the general culture within most scientific circles, especially within academia, is rife with sloppy science that has little interest in finding the non-biased truth. Good luck if you don't constantly include "muh climate change" or "muh evolution" in studies where it doesn't have any relevance whatsoever. There is a lot of money out there right now to "study" "muh climate change" and if you want to get grants, publish, or get promotions you are basically forced to continue the "consensus" narrative.

Blogger Daniel August 27, 2018 4:24 PM  

Jan Hendrik Schön won the Braunschweig. Medical fraud is common and lucrative. Computer science is pre-virtualized. Peer review is unscientific.

Blogger Spud August 27, 2018 4:36 PM  

This comment has been removed by the author.

Blogger pyrrhus August 27, 2018 4:38 PM  

O/T slightly...China also suffering dysgenic breeding effects, similar to the West, and Asian researchers confirm NE Asians less creative or able to think out of the box....http://www.unz.com/article/dysgenics-and-low-creativity-why-china-cant-save-civilization/

Blogger Uindo August 27, 2018 4:41 PM  

New Alt-Hero villain Science-Face! Half Bill Nye half Neil Degrasse, he fips a coin to make his assertions.

"What gender am I today? Let's see what science has to say about it!" He flips a coin

Blogger Nick Siekierski August 27, 2018 4:42 PM  

I’d like to see media companies sued for promoting the latest “studies” that conclusively show that X,Y,Z will kill/cure/poison/save you. How many millions of people have died after making huge lifestyle changes based on the latest junk science fad. The latest is that 6-8 hours of sleep is ideal. And more or less might kill you. Really?!

Blogger Unknown August 27, 2018 4:43 PM  

pyrrhus wrote:O/T slightly...China also suffering dysgenic breeding effects, similar to the West, and Asian researchers confirm NE Asians less creative or able to think out of the box....http://www.unz.com/article/dysgenics-and-low-creativity-why-china-cant-save-civilization/

I can confirm this with my first-hand experience in China. Virtually every aspect of their science or technology was stolen from the West. Creativity and individuality are strongly discouraged over there. I really do not expect them to be a serious global leader any time soon due to those deep cultural characteristics.

Blogger Gen. Kong August 27, 2018 4:44 PM  

There is no truth but warming, and Algore is its profit!

The summation of 'science' in a single statement. Have (((they))) elevated Dick-Dawk to the intellectual dank web yet along with the other luminarias like Harris and Crazy-Christ?

Blogger Fuzzums Wuzzums August 27, 2018 4:45 PM  

People who blindly believe a science paper because it was published in some science journal are always people with no experience in science., such as Dawkins or Harris.

A study was made on scientific papers years ago and it discovered that the more references and pictures a paper had of brains, neuroanatomy, or flat out artistic interpretation the more likely it was to be published and accepted as true by peer review.

Then there's science journalism which is abysmal. Even now people believe light has a wave/particle duality or that we only use 10% of our brains. It's not misinformation, it's people who are too stupid to understand the actual paper trying to explain a concept they read about from a second source.

In the world of academia, books and papers aren't written because people want to propagate knowledge or do science. They are written because they have to be written in order for some to keep that "Professor" title in front of their name.

Blogger Trid August 27, 2018 4:47 PM  

But who will reproduce the reproducibility studies? :^)

Blogger Fuzzums Wuzzums August 27, 2018 4:50 PM  

Aracuru wrote:Virtually every aspect of their science or technology was stolen from the West.

This is true for all countries and in all fields.

Blogger fiendeJ August 27, 2018 5:13 PM  

The faith of Scientism clearly needs an inquisition to purge these heretics abusing Scientody.

Blogger The Kurgan August 27, 2018 5:16 PM  

As far as I know the “cold fusion” experiment has been reproduced in something like 90 different labs to some extent or other. Assuming that data point itself wasn’t a complete fabrication.

Anonymous Anonymous August 27, 2018 5:19 PM  

Well we know empiricism in economics is false, and I think it’s false across the board, but even Stefan will defend it from time to time on his show.

Blogger Greg Hunt August 27, 2018 5:29 PM  

Looked around and can't find a definition; what does that acronym stand for?

Blogger Ostar August 27, 2018 5:30 PM  

What is often not even said is that college students are the main source of subjects in "scientific" research. Certainly in the social sciences, where they have to participate for class credits. So when you read "men and women studied/asked/participated/etc.", substitute a self-selected, very limited-in-range group of (barely) adults.

Blogger Unknown August 27, 2018 5:36 PM  

Wuzzums Fuzzums wrote:This is true for all countries and in all fields.

That is factually incorrect. If you are interested in the facts I would suggest you look into the on-going mass theft of patented science/technology as well as theft of intellectual property.

Blogger tublecane August 27, 2018 5:40 PM  

OT, but I have to repeat a quote from John McCain's farewell letter: "I had the privilege to concede defeat in the election for president."

Is that the loser right, or what? Not even the privilege to be his party's nominee, but the privilege to admit defeat. Who even thinks like tgat, let alone writes it down for history to record? (I know, probably some PR guy.)

At long kast, did he throw that election?

Blogger Paul, Dammit! August 27, 2018 5:50 PM  

My very first hard science publication as primary investigator was a salvage job made from a failed attempt to model chemical diffusion in chaotic turbulent environments. At the time I was working for my department head, a well-known neuroscientist. We were getting paid at the time to make search algorithms for autonomous mine-sniffing robots.
My approach bombed, which, as an undergrad sophomore was no big deal, but since my department head hated E.O. Wilson and I was forced to perform animal behavioral trials (which is about as robust and objective a science as tarot card reading or reiki), I spent 12 hours a day for 6 months, measuring lobster mechano- and chemosensory nerve conduction signals during behavioral decisions. Along the way I realized I wasn't modeling animal data to make the robots more effectivt. I got shoehorned into a search for something, anything, that would refute anything that E.O. Wilson published. My department head hated E.O. Wilson because he was rude once at a dinner.
My crappy data got cherrypicked, turned over, arcsine-transformed and forced into relevance until two things became visible- a weak association contrary to one of E.O. Wilson's ideas, and an accidental actual discovery-by-association- a simple meat-and-potatoes useful thing that I had to wait to publish until I was finished being a pawn to make E.O. Wilson look bad. By the time I was finished touring around academic conferences, I had been pigeonholed and no one gave a damn about the actual cool thing I also found. The BS sold because nerds are awful people.

I ran away to sea the same day of my last day of my last final in grad school. Far more honest work and honest people.

Blogger weka August 27, 2018 6:06 PM  

Well, since I do human research... And meta analyse such for a living, some problems.

1, Most non clinical psychology trials are either based on abstract models or receiving my on equipment that sldoea not consider how subjects change with experience. This includes most if not all neuroanatomy trials, as behaviour changes neural networks. (As does mood, stage in menstrual cycle, and most antidepressants).

2, We are still not sure how the two most effective treatments for serious mental illness -- Lithium and Clozapine -- work. We do have replication of these trials.

3. The FDA want two trials showing a new pharmaceutical work. The EU also want two trials, but they have different standards. That means four trials, powers for a small effect difference, or about 1500 -- 2000 participants in total. It costs between 2-10k per participant to get this done. Do the math, and then you will realise that pharma is like Hollywood --you need big winners (Quetiapine, Sidefinil) to pay for the failures.

4. Clinical psychs use wait list as a control. Pill placebo is much more effective.

5. Finally, a meta analysis is only as good as the papers that make it up. You have not lived until you are back translating 40 yr old papers from Italian and getting help with Russian and Chinese.

In short, non clinical psychologists are lousy philosophers and clinical psychologists impractical team members you work around. Sociologists are far, far worse.

Blogger tublecane August 27, 2018 6:15 PM  

I used to want to think the non-social sciences were better, but after the God Particle junk I figured even High Physics has been dubious since quantum mechanics.

Blogger Crunchy Cachalot August 27, 2018 6:24 PM  

we only use 10% of our brains

The truth or falsity of this statement depends almost exclusively on who is defined as "we".


If it includes the cohort that "believes in SCIENCE!" and views it as a quasi-mystical alchemical process that transforms ordinary humans into nearly godlike yet impartial preceptors of the only truth, then, yes, 10% is pretty accurate.

Blogger Will August 27, 2018 6:32 PM  

To be fair, neither "Science" or "Nature" select submissions based on reproducability. Everything comes down to "Impact Factor"

Blogger pyrrhus August 27, 2018 6:41 PM  

@14 Aracuru wrote--I can confirm this with my first-hand experience in China. Virtually every aspect of their science or technology was stolen from the West. Creativity and individuality are strongly discouraged over there. I really do not expect them to be a serious global leader any time soon due to those deep cultural characteristics.

Indeed, my opinion has always been that it is a genetic inclination to conformity, selected for over millennia...I think Greg Cochran once wrote that in China, the grass blade that stuck out was pulled out and thrown away...probably for the last 3-5000 years.

Blogger Al K. Annossow August 27, 2018 6:50 PM  

Now the whole world had one language and a common speech of mathematics. They used theories instead of truth, and speculation for mortar. Then they said, “Come, let us build ourselves a comprehensive system, with an understanding that reaches to the heavens, so that we may make a name for ourselves; otherwise we will be scattered over the face of the whole earth."

Blogger Chris Mallory August 27, 2018 6:56 PM  

weka wrote:3. The FDA want two trials showing a new pharmaceutical work. The EU also want two trials, but they have different standards. That means four trials, powers for a small effect difference, or about 1500 -- 2000 participants in total. It costs between 2-10k per participant to get this done.

Even that 10k number can be low. I spent two years in a medical study testing a double dose of an approved medication against the standard prescribed dose.

I would be given 3 months of meds at a time. Every three months I had exams by two different neurologists. Every 6 months I spent close to two hours in what was then the top of the line MRI machine

Blogger Dirk Manly August 27, 2018 7:12 PM  

@20

> As far as I know the “cold fusion” experiment has been reproduced in something like 90 different labs to some extent or other. Assuming that data point itself wasn’t a complete fabrication.

Yes, but every one of those replications of the Fleischman and Pons "Cold Fusion" experiment has failed to reproduce the much bally-hooed "we ran straight to the newspapers without even showing any of our colleagues" result.

And they never will.

Fleischman and Pons FORGOT to deduct the amount of power coming out of their power supply in their power output calculations.

Once THAT power was accounted for, it turns out that their experiment proved that they had set up a really simple but expensive resistive circuit.

Blogger Johnny August 27, 2018 7:27 PM  

I have noticed two things about psychology. One is that their description of diseases is more often a list of symptoms than causes. If regular medicine were like that there would be diseases called things like headache or upset stomach.

The other is a tendency to call symptom relief a cure. Regular medicine does that some but not so much. Insulin for example does not actually cure diabetes, it treats the symptoms. You retain the infirmity. Conversely, penicillin is an effort to cure, not to repress symptoms. Meanwhile most or all the psychotropic drugs are symptom relief, not cures.

Blogger cmbaileytstc August 27, 2018 8:34 PM  

Oddly enough I quit being a libertarian when I started using strategy and ACTUAL rational self-interest (which includes acknowledging that I have collective interests). I like to explain it by saying that thinking about what people should be like is irrelevant, you have to base policy on what they are like. Can that rightly be called a scientific approach?

Blogger Playto August 27, 2018 10:07 PM  

That's what happens when you turn science into a religion and give scientists the power and status of priests. If science were open, it wouldn't be so prone to fraud, and hence the push.

Blogger Johnny August 27, 2018 10:17 PM  

@36
Science usually stresses experiment as the gold standard, but there is also a tendency to call it science when it is observe and report. Thus I suppose taking people as they are could be called a scientific approach.

Blogger kurt9 August 27, 2018 10:19 PM  

50%? Are you kidding? The scientific claims that are actually reproducible is more like 25%, if that.

Blogger Timmy3 August 27, 2018 10:54 PM  

Neither are scientists.

Blogger Fuzzums Wuzzums August 27, 2018 11:38 PM  

Aracuru wrote:That is factually incorrect. If you are interested in the facts I would suggest you look into the on-going mass theft of patented science/technology as well as theft of intellectual property.

Wait, what's incorrect?
I was agreeing with you. Virtually every scientific progress or basis for a field of study can be traced back to the West no matter what country we are talking about.

Blogger widlast washere August 27, 2018 11:49 PM  

My father was a research scientist, first for big pharma in hormone biology, then in physiology for the DOD. Back then all that mattered was detailed and accurate analysis of results, hoping for an in depth understanding of the underlying reality. Then during the sixties, politics and "movements" started to muddy the waters. Nothing has been the same since. He was rather disgusted with the whole mess, and tailored his career to avoid it.

Blogger tublecane August 28, 2018 12:14 AM  

@35- Psychology is not medicine, simply put. The concept of mental disease is at best metaphorical. They don't bother with etiology.

Theoretically we could develop mental medicine, but psychology will never do it.

Blogger Robert Coble August 28, 2018 12:26 AM  

It appears the "reproduction" problem is affecting more and more aspects of life around the globe, and not just making babies. It is hardly surprising, given that lower IQ peoples are not only reproducing like bacteria but also trying so hard to fill the professional vacancies for which sufficient geniuses are no longer available. Or is it just more profitable to publish what the PWTB want to pay for?

Isn't Crazy Christ one of those "distinguished, published" social science researchers?!? I wonder what his "reproduction" rate would be if anyone ever bothered to examine his oeuvre.

Inquiring intelligent minds are a terrible thing to waste!

Anonymous Anonymous August 28, 2018 12:31 AM  

As I once told my older brother, a devout scientism acolyte, the tyranny of men in black dresses has been supplanted by a tyranny of men in white dresses.

The only thing we ever agreed on is that you cannot ever trust men who wear dresses.

Blogger SciVo August 28, 2018 2:17 AM  

"Science's Credibility Damaged, God-Haters Hardest Hit"

Blogger justaguy August 28, 2018 3:04 AM  

Of note: one of the standards for scientific testimony in court, where tens of millions of $ can be on the line based on dubious theories of science, is whether or not the dubious theory has been published in a peer-reviewed journal?

Blogger Nostromo August 28, 2018 5:19 AM  

Somewhere in a quantum wave state, Shroedinger's cat chuckles softly, while also decaying quietly.

Blogger Brian Dean August 28, 2018 7:15 AM  

Given the many different religions that are out there, I wonder how reliable religion is at determining what is true.

Blogger Unknown August 28, 2018 7:43 AM  

Best Android Apps for PC

sound cloud for pc

spotify premium apk

Blogger Blunt Force August 28, 2018 9:26 AM  

Evolution is real.

“But what sets us apart from the rest of the animal kingdom is the capacity and the output of our brain. That makes us human. So it turns out humanity is very difficult to model in an animal system.”

https://www.geekwire.com/2018/rosehip-neuron-discovered/

Blogger Tars Tarkas August 28, 2018 2:10 PM  

This is what happens when Diversity is forced into a field and when zealots are in charge and they are doing it with other people's money.

So much of academia is now loaded with advocates and not sober judicious people looking for the truth.
Then they wonder why public trust in experts is falling.

Blogger SciVo August 28, 2018 2:34 PM  

Tars Tarkusz wrote:Then they wonder why public trust in experts is falling.

If public trust came to depend on BBQ ability, then the very next day all of the established institutions for judging BBQ would be co-opted by a Mos Eisleyan menagerie of talented narcissists and charming sociopaths, and repurposed to a variety of competing bad ends from the banal to the diabolical.

Blogger SciVo August 28, 2018 3:15 PM  

This, incidentally, explains the Lavender Mafia of the Vatican: something like it was always literally inevitable, because of our fallen human nature. The only sustainable solution that I can see is to teach public skepticism of all authority, and you can tell a fake if he doesn't also teach skepticism of himself.

You will know a tree by its fruit, and for all that atheists cry "Skepticism, skepticism!" they only show skepticism for claims they don't like. In practice, secular humanist schools inculcate the most despicable passive credulity in their victims, only substituting in the anti-Christian authorities of the Social Justice cult.

At its essence, socialism is neither more nor less than the grifter class's quest to replace the resource-extracting mechanisms of the Church with one backed up by guns. It takes the heart of either a conman or an easy mark to think that's okay. In that sense, secularism is like poker: if you look around the table and can't tell who the chump is, it's you.

Blogger Zander Stander August 28, 2018 3:52 PM  

You prepared to swear that the picture of Crazy-eyes-Cortez was not photoshopped?

Blogger SciVo August 28, 2018 8:51 PM  

Well the caption wasn't just hovering in mid-air when the picture was taken, but otherwise I think that rainy-streets photo is legit. (The one where she's holding a whiteboard is of course Exhibit Umpteen-million for why you don't do that in the Photoshop age.)

Blogger Expendable Faceless Minion August 29, 2018 3:02 AM  

I hear thr Russians say
"The nail that sticks up gets the hammer."

Blogger Zeroh Tollrants August 29, 2018 4:43 PM  

Does he control his minions by posting crappy dad-tier Tweets, that he constantly recycles, year after year?

Blogger Th3 J3st3r September 15, 2018 2:20 PM  

Probably more pragmatic.
Either way, it's more sane. I had the same conundrum as you.

Post a Comment

Rules of the blog

<< Home

Newer Posts Older Posts