ALL BLOG POSTS AND COMMENTS COPYRIGHT (C) 2003-2014 VOX DAY. ALL RIGHTS RESERVED. REPRODUCTION WITHOUT WRITTEN PERMISSION IS EXPRESSLY PROHIBITED.

Tuesday, April 17, 2012

That touching faith in science

I thought it was interesting to see that one of Wängsty's commenters, Cornucopia, still erroneously clings to a blind faith in the "self-correcting" nature of science:
In the operational sense, does it really matter whether science is intrinsically or extrinsically self-correcting? The study you alluded to previously was done by confirming the results of scientists by scientific means. It’s not as if somebody sat down with a Ouija board and confirmed or refuted scientific findings or had them fed to them by revelation. If you happen to be basing your claim that science is not intrinsically self-correcting on something as superficial as who happens to be funding the effort to confirm it, I think you’ll just engaged in a cheap slander against the process of science.
He missed the point. Science isn’t self-correcting by any sense that doesn’t apply equally well to any number of other non-scientific fields. Peer review is nothing more than editing. Experimental replication, in the very rare instances it is actually performed and is successful, is nothing more than auditing. There is no substantial difference between one scientist re-running another scientist’s experiment and one accountant re-calculating another accountant’s books. In other words, science isn’t self-correcting in any meaningful sense even in its ideal form.

And, of course, as was demonstrated in the paper I cited, most “science” is not performed according to the ideal form, and even when it is, it often turns out to be unreliable. Even the best, "gold standard" science has been reported to be 89 percent unreliable, as a matter of fact. It must also be pointed out that if scientific error is identified by non-scientists who aren’t engaged in science, then the correction cannot be considered extrinsic self-correction because it is not self-correcting in any sense.

One might as reasonably claim that crime is self-correcting because the police sometimes arrest criminals.

And while I find it strange to have to point this out, my argument about the unreliability of science is absolutely not based upon an appeal to a genetic fallacy of who happens to be funding the science, although it is worth noting that the intrinsic unreliability of modern science does create the opportunity for a significant amount of undetected corruption.

Labels: ,

51 Comments:

Anonymous Rantor April 17, 2012 10:49 AM  

Nature and other science magazines should demonstrate their self correctingness by dedicating a third of their content to retractions and corrections. A short, half page article admitting wrongness, explaining it briefly, and perhaps recommending areas for further research would be a good start. You could certainly self-correct 15 to 20 findings per issue. Things like Darwinian Evolution and AGW may need several pages just to list all the scientests that were wrong.

Anonymous Rantor April 17, 2012 10:50 AM  

Scientists... must learn to proof read

Anonymous stg58 April 17, 2012 10:51 AM  

Extrinsic self correction is an oxymoron.

Blogger Markku April 17, 2012 10:55 AM  

Windows' vulnerabilities are self-correcting. Hackers exploit them, eventually Microsoft has to fix them when the hackers do enough damage, and voilà! The vulnerability has self-corrected.

What this means for the layperson is that you should trust Windows.

Anonymous civilServant April 17, 2012 11:14 AM  

Science isn’t self-correcting by any sense that doesn’t apply equally well to any number of other non-scientific fields. Peer review is nothing more than editing. Experimental replication, in the very rare instances it is actually performed and is successful, is nothing more than auditing.

So. There is no such thing as science. Yes?

Blogger Astrosmith April 17, 2012 11:19 AM  

So what sort of business model could one create where you ran a lab that did nothing but attempt to replicate all experimental results that are in the published literature? Vox's point about auditing in the accounting realm is valid; there are firms which have that as a big part of their business.

Anyone want to start this? We could call it "Self Correction Laboratories" or something like that.

Anonymous kh123 April 17, 2012 11:21 AM  

I figured they would have taken to spelling it as Ş'cïênčę™ over at his page by this point.

Anonymous civilServant April 17, 2012 11:41 AM  

So what sort of business model could one create where you ran a lab that did nothing but attempt to replicate all experimental results that are in the published literature?

Free-market capitalism cannot support such an activity.

Anonymous James Dixon April 17, 2012 11:47 AM  

> Free-market capitalism cannot support such an activity.

Are you familiar with Underwriters Laboratories? It's pretty mcuh exactly such an activity; though in hardware, not research.

Anonymous Rantor April 17, 2012 11:50 AM  

So what sort of business model could one create where you ran a lab that did nothing but attempt to replicate all experimental results that are in the published literature? Vox's point about auditing in the accounting realm is valid; there are firms which have that as a big part of their business.

You would have to get the government to make it a requirement for a patent, copyright or some such. Of course then it becomes an evil, indirect tax on science. But if they are only 11% accurate, maybe they should be taxed. Of course government funding for science will probably pay for much of the research and the testing! No tax on science, just another tax on the masses.

Anonymous BillB April 17, 2012 11:55 AM  

The point the commenter missed is that science is 1) developed by humans and 2) driven by humans.

Dialing for dollars to perform research has brought us to this point. No one gets refunded if the results of their work conflict with the self-interest of the funding agency. AND humans have an agenda to obtain wealth, fame, and power regardless of the number of lies they must tell. Scientists do not have a corner on honor.

Anonymous Durandel April 17, 2012 11:55 AM  

Isn't their a Greek medical researcher, Dr Ionidas or something like that, who's made a business auditing medical research? In an interview he estimated that at least 60 of the medical research out there is false. Vox, did you not post an article about him a year or two ago?

Anonymous VD April 17, 2012 12:04 PM  

There is no such thing as science. Yes?

No, the fact that science doesn't operate as it is ideally envisioned doesn't make it nonexistent. It merely makes it less reliable than claimed.

Blogger JDC April 17, 2012 12:05 PM  

I think you’ll just engaged in a cheap slander against the process of science.

Careful...science may hire Jeffrey Fieger and sue you for libel. How can you so crassly defame such a pure and benevolent institution like science. I mean...science gave us CGI - and allowed Anakin's young spectral image to be cast alongside Yoda and Sir Alec Guinness.

Anonymous Jimmy April 17, 2012 12:07 PM  

Much published science goes unread. I think the greatest controversies continue to be our hot button issues of Evolution and Global Warming. Why they are considered science is beyond me? They are actually philosophy disguised as science.

Blogger Markku April 17, 2012 12:10 PM  

"slander against the process of science"

In other words:

Blasphemer! Blasphemer!

Anonymous civilServant April 17, 2012 12:12 PM  

Free-market capitalism cannot support such an activity.

Are you familiar with Underwriters Laboratories? It's pretty mcuh exactly such an activity; though in hardware, not research.


I have heard of it but am unfamiliar with its basis for existence. If it focuses on hardware then I will guess that it exists to satisfy consumer protection laws.

No one gets refunded if the results of their work conflict with the self-interest of the funding agency.

Precisely. Though they may if the self-interest of the funding agency includes avoidance of legal penalties and consumer protection lawsuits. But this would not seem to qualify as free-market capitalism.

Blogger Jehu April 17, 2012 12:13 PM  

Were reactionary Christians ever to establish a profoundly reactionary University, the science departments could specialize in this sort of thing (replication and debunking). It would be a healthy thing for society.

Anonymous Roundtine April 17, 2012 12:14 PM  

It could become a business model if they government gave out grants in pieces. For instance, you get 50% of the grant to conduct your research, the other 50% sits in escrow. If an experimental bounty hunter can show your results are false, they take the other 50%. If they can replicate, they take 10% and you receive the remaining 40%.

Anonymous civilServant April 17, 2012 12:16 PM  

There is no such thing as science. Yes?

No, the fact that science doesn't operate as it is ideally envisioned doesn't make it nonexistent. It merely makes it less reliable than claimed.


So. To make science more reliable it must apply the scientific method?

Blogger JDC April 17, 2012 12:16 PM  

Markku nailed it. I think this blatant desecration of science calls for that which nobody expects

Anonymous Roundtine April 17, 2012 12:16 PM  

UL is funded by Underwriters. Insurance companies need to know the odds of a product failing and causing an accident. The trick here is to make scientists, and their deep pocketed employers (universities, foundations) financially responsible for bad science.

Anonymous civilServant April 17, 2012 12:21 PM  

Were reactionary Christians ever to establish a profoundly reactionary University, the science departments could specialize in this sort of thing (replication and debunking). It would be a healthy thing for society.

Much modern research is expensive. Such a university would be difficult to fund.

If an experimental bounty hunter can show your results are false, they take the other 50%. If they can replicate, they take 10% and you receive the remaining 40%.

This sounds like a lawyers' playground.

Blogger Ilíon April 17, 2012 12:23 PM  

".. just to list all the scientests that were wrong."
"Scientists... must learn to proof read."

I call "Science!" groupies 'scientistes', in the manner of Miss Piggy, the Artiste.

Anonymous The Gray Man April 17, 2012 12:34 PM  

Great post. The police being analogous to the non-scientists is a great point.

Anonymous Roundtine April 17, 2012 12:34 PM  

This sounds like a lawyers' playground.

I'm not advocating for it, butaAnything that assigns property rights will allow a market to develop, if that's what you're after. And any market will attract lawyers, Anglo-Saxons have been sue happy for 1000 years.

Anonymous the abe April 17, 2012 1:15 PM  

"A cheap slander against the process of science"

In nearly 10 years(!) of spouting nine kinds of BS on this blog, some of it funny, most of it forgettable, I don't think I could possibly hope to broach in satire what Horn of Plenty or whatever says in seriousness.

Blogger Doom April 17, 2012 1:20 PM  

I've heard a saying, used by French, Swiss, Danes, by themselves, and about women (though never by a woman)...

You can tell (fill in the blank), you just can't tell them much.

I think you are at that point with this "debater". And this debater seems more like a woman than a Dane or myself, for example. Some of us know, on some things, we just aren't going to change. Some don't. All good, though. It's... entertaining to watch.

Anonymous Cheddarman April 17, 2012 1:24 PM  

wangsty does not seem to understand human nature. People will advance and protect their own interests. To believe that science is somehow immune from this readily observable fact of human nature suggests that Wangsty has the same child like faith in science as he would accuse others of having in religion.

Anonymous Azimus April 17, 2012 1:32 PM  

VD:

Experimental replication, in the very rare instances it is actually performed and is successful, is nothing more than auditing. There is no substantial difference between one scientist re-running another scientist’s experiment and one accountant re-calculating another accountant’s books. In other words, science isn’t self-correcting in any meaningful sense even in its ideal form.


Interesting thought. Tilting a little in the direction of a "let's have a definition war" argument, but an intersting thought. By that yardstick would you call the market, or engineering self-correcting?

Anonymous FUBAR Nation (Ben) April 17, 2012 2:11 PM  

The problem isn't science. The problem is dark science. Don't you know the difference?

It's all FUBAR

Anonymous Jimmy April 17, 2012 2:34 PM  

Just in. Retractions. The new "self-correcting" nature of science.

http://www.nytimes.com/2012/04/17/science/rise-in-scientific-journal-retractions-prompts-calls-for-reform.html?_r=1

Blogger Nate April 17, 2012 4:07 PM  

In this case... Science did not correct science. Business did.

Blogger Nate April 17, 2012 4:07 PM  

This comment has been removed by the author.

Anonymous civilServant April 17, 2012 4:28 PM  

By that yardstick would you call ... engineering self-correcting?

Perhaps "self-corrected" would be more appropriate? Engineering seldom trods new ground.

Anonymous Jimmy April 17, 2012 4:32 PM  

Science corrupted itself especially with increased influence of government, academia, and career prospects. Science is open to the high bidder, the horror.

Anonymous civilServant April 17, 2012 4:37 PM  

Anything that assigns property rights will allow a market to develop ...

Yes. And then investments. And then corporations. And then a de-facto monopoly. And then profit-driven business decisions. And then cover-ups. And so on. No-one hates the Free Market more than investors. They run from it as soon as it costs them anything at all.

Blogger JohnG April 17, 2012 4:57 PM  

Unfortunately I don't think many people pay any attention. "Science says..." almost seems to carry the same weight as a fatwa to the Muzzies. But outside of a few narrow fields, it seems like it's mostly garbage anymore - pretty well documented that the "science" is simply produced to keep the grant money rolling in, I suspect most teams that debunk the latest holy cow will soon be looking for work elsewhere.

Blogger Markku April 17, 2012 5:01 PM  

Unfortunately I don't think many people pay any attention. "Science says..." almost seems to carry the same weight as a fatwa to the Muzzies.

But the best of all is to let him read no science but to give him a grand general idea that he knows it all and that everything he happens to have picked up in casual talk and reading is "the results of modern investigation". Do remember you are there to fuddle him. From the way some of you young fiends talk, anyone would suppose it was our job to teach!

Your affectionate uncle
SCREWTAPE

Anonymous kh123 April 17, 2012 5:21 PM  

"And then a de-facto monopoly."

Barring gov't crony intervention, what makes it de-facto in a competitive marketplace.

Anonymous Azimus April 17, 2012 5:25 PM  

civilServant April 17, 2012 4:28 PM

Perhaps "self-corrected" would be more appropriate? Engineering seldom trods new ground


Speaking on the topic I know the most about, civil engineering, increased knowledge on the performance and long-term behavior of materials has driven cost of projects down (in real terms): where once an engineer would've used a 72" I-beam incorporating a real factor of safety of say 20, with the additional knowledge (and improved manufacturing quality controls of steel) gained through time, a civil engineer designing the same project today might select a 48" I-beam incorporating a factor of safety of 3. I would consider that increase in material efficiency self-correcting, or maybe self-improving. Same with the development of ultra-high strength concrete, laminated beams and other more efficient structural products and methods.

Blogger Markku April 17, 2012 5:32 PM  

where once an engineer would've used a 72" I-beam incorporating a real factor of safety of say 20, with the additional knowledge (and improved manufacturing quality controls of steel) gained through time, a civil engineer designing the same project today might select a 48" I-beam incorporating a factor of safety of 3. I would consider that increase in material efficiency self-correcting, or maybe self-improving.

"Design load being the maximum load the part should ever see in service."

"Should" is a scary word when the cost of being wrong is measured in human lives and of being right is measured in dollars. I would use the word "improvement" only with SERIOUS qualification.

Anonymous stg58 April 17, 2012 5:44 PM  

Speaking on the topic I know the most about, civil engineering, increased knowledge on the performance and long-term behavior of materials has driven cost of projects down (in real terms): where once an engineer would've used a 72" I-beam incorporating a real factor of safety of say 20, with the additional knowledge (and improved manufacturing quality controls of steel) gained through time, a civil engineer designing the same project today might select a 48" I-beam incorporating a factor of safety of 3. I would consider that increase in material efficiency self-correcting, or maybe self-improving. Same with the development of ultra-high strength concrete, laminated beams and other more efficient structural products and methods.

The increase in material efficiency isn't self correcting, because civil engineers do not generally design higher strength materials that allow them to reduce their safety factors. Outside companies take information from engineers, apply their own processes, and produce better materials. The only way you can classify this as self correcting is if the engineers take note of which building methods hold up, and which don't, then use that information to design new buildings.

Anonymous WaterBoy April 17, 2012 5:53 PM  

Roundtine: If an experimental bounty hunter can show your results are false, they take the other 50%. If they can replicate, they take 10% and you receive the remaining 40%.

Then you have the unfortunate circumstance of giving the auditors a financical incentive to falsify their results...which would then have to be audited again...and so on...and so on...

It's scientists all the way down.

Anonymous civilServant April 17, 2012 6:14 PM  

And then a de-facto monopoly.

Barring gov't crony intervention, what makes it de-facto in a competitive marketplace.


What makes it de-facto is that the real marketplace competition is not between business and business for customers but rather between business and customer for money. Recall - what is the primary purpose of any business? Businesses have no inherent interest in competing with each other and are in fact natural allies. It is much more cost efficient for them to fleece customers in concert than to wage price wars with each other.

Of course if they can knock off each other efficiently they will do so. But that is not their primary function.

Anonymous Cornucopia April 17, 2012 7:36 PM  

Vox, to a large extent I do find your argument convincing, and as pointed out above, extrinsic self-correction is an oxymoron, but I claim there is still a grain of truth to my side. Your analogy is not complete since the police, ideally, do not use criminal means to correct criminals. However, crime is in fact self-correcting to a degree by criminal means when one criminal shoots another. The paper you cited came as the result of a profit seeking venture verifying scientific findings by scientific means. Whether you want to include this in the purview of science really depends on where you draw the line between the enterprise of science and something else, for instance the legal or ethical oversight of science. Since these days privately and publicly funded science are in bed with each other, there a strong case to be made that your example was self-correction, at least partially.

Anonymous Azimus April 17, 2012 10:57 PM  

stg58 April 17, 2012 5:44 PM

The increase in material efficiency isn't self correcting, because civil engineers do not generally design higher strength materials that allow them to reduce their safety factors. Outside companies take information from engineers, apply their own processes, and produce better materials. The only way you can classify this as self correcting is if the engineers take note of which building methods hold up, and which don't, then use that information to design new buildings.


You're hitting on what I was implying when I mentioned "increased knowledge on the performance and long-term behavior of materials has driven cost of projects down". Time was a civil engineer erred heavily on the side of F.O.S. because precise calculations were time-consuming and unflattering assumptions were made about the quality of the building materials, both initially and for the long term. The Eiffel Tower was designed with this philosophy in mind. When it was still standing 50yrs after it's design life with no sign of collapse they observed that they could lower the F.O.S. on similar structures and rely on the material's strength more than they had. So yes I think you're right it is exactly a case of noting what sort of buildings hold up.

Anonymous Cornucopia April 17, 2012 11:52 PM  

There is no substantial difference between one scientist re-running another scientist’s experiment and one accountant re-calculating another accountant’s books. In other words, science isn’t self-correcting in any meaningful sense even in its ideal form.

Let me just add another thing, because the comparison between the scientific body of knowledge and that of accounting systems is an interesting one, and science is self-correcting in way similar to that of accounting audits, regardless of whether you want to call both or either trivial. The difference is in domain and scope, both physical and temporal, whereas an accounting error in Australia might not have effect on one in America, or one in 1745 will have either no or minuscule effect on our finance today, reality always persists through space and time. A false scientific finding remains relevant exactly to the degree that impinges on other findings, so that not only is science self-correcting, but it also auto-focuses on those errors that most merit correction.

Anonymous kh123 April 18, 2012 12:30 AM  

"Of course if they can knock off each other efficiently they will do so."

Interesting. And again, barring gov't intervention, how exactly do they knock eachother off; the standard method.

Anonymous map April 18, 2012 1:00 AM  

I think the "self-correcting" concept is filtered through a layman's understanding. Self-correcting is supposed to mean that the easy replication of any scientific experiment automatically makes a scientist honest and unimpeachable. Technically speaking, unreplicable science would never get published because no one should lie about something that could be easily checked. Yet, what we are seeing unreplicable science actually getting published, contrary to what is actually happening.

So, because no one has any incentive to audit other people's work, no one is afraid of getting caught in a lie.

Anonymous VD April 18, 2012 7:20 AM  

Vox, to a large extent I do find your argument convincing, and as pointed out above, extrinsic self-correction is an oxymoron, but I claim there is still a grain of truth to my side. Your analogy is not complete since the police, ideally, do not use criminal means to correct criminals.

I agree - ideally. In reality, I fear that my example is well-founded in material fact. You might be interested in listening to my interview with Nick Novello, the Dallas policeman and whistleblower in this regard. But nevertheless, there were likely many other examples I should have used instead.

Post a Comment

NO ANONYMOUS COMMENTS. Anonymous comments will be deleted.

Links to this post:

Create a Link

<< Home

Newer Posts Older Posts