ALL BLOG POSTS AND COMMENTS COPYRIGHT (C) 2003-2019 VOX DAY. ALL RIGHTS RESERVED. REPRODUCTION WITHOUT WRITTEN PERMISSION IS EXPRESSLY PROHIBITED.

Sunday, October 13, 2019

The collapse of science

Illustrating once more that science is dependent upon technology rather than the other way around, a petty Python script bug may force the retraction of more than 100 published scientific studies:
Scientists in Hawaiʻi have uncovered a glitch in a piece of code that could have yielded incorrect results in over 100 published studies that cited the original paper.

The glitch caused results of a common chemistry computation to vary depending on the operating system used, causing discrepancies among Mac, Windows, and Linux systems. The researchers published the revelation and a debugged version of the script, which amounts to roughly 1,000 lines of code, on Tuesday in the journal Organic Letters.

“This simple glitch in the original script calls into question the conclusions of a significant number of papers on a wide range of topics in a way that cannot be easily resolved from published information because the operating system is rarely mentioned,” the new paper reads. “Authors who used these scripts should certainly double-check their results and any relevant conclusions using the modified scripts in the [supplementary information].”

Yuheng Luo, a graduate student at the University of Hawaiʻi at Mānoa, discovered the glitch this summer when he was verifying the results of research conducted by chemistry professor Philip Williams on cyanobacteria. The aim of the project was to "try to find compounds that are effective against cancer,” Williams said.
To help understand how devastating this sort of thing could be for the profession and practice of science, consider the very frightening possibility that modern science increasingly relies upon the sort of people responsible for enhancing your user experience of Skype and manning Twitter "customer support".

Labels: ,

142 Comments:

Blogger spinoza October 13, 2019 9:56 AM  

How likely is it that what is called a glitch was more like implied institutional subjectivity rationally justified and supported by accepted theory? The glitch was coming from their own minds!

Blogger Newscaper312 October 13, 2019 9:59 AM  

So took some Chinese sperg to see if the results actually made sense? I actually taught C++ to Engineering students from 2002 to 2008. One of the things we beat on them was just because the computer performs calculations virtually perfectly in no way means they are the right calculations for your problem. Also GIGO for data quality.

Blogger cecilhenry October 13, 2019 10:01 AM  

This does call into question the effectiveness of peer review in (at least quickly) finding and removing errors.

Aren't these papers repeated for verification?

Too often not it appears.

Blogger spinoza October 13, 2019 10:07 AM  

If the entirety of intelligence is compromised, one day all intelligence will be declared artificial or have they already? Why write admissions if there is no problem?

Blogger [Redacted] October 13, 2019 10:13 AM  

I suspected that the abstraction layers on programming languages would result in nondeterministic results the further you got from hardware, and this sort of thing was inevitable as the ecosystem got more complex... But I have to say it.

They trusted a programming language named after a type of snake, and it came back to byte them.

Blogger JG October 13, 2019 10:16 AM  

Using python to do any important calculation was the first mistake. Who knows if the math libraries have been thoroughly tested and verified across platforms? A real scientist or engineer would use Matlab or something similar that has verified libraries.

Blogger Ominous Cowherd October 13, 2019 10:18 AM  

For calculations and messing about with data, it's hard to beat R.

Blogger VFM #7634 October 13, 2019 10:18 AM  

I've always thought it stupid to rely on computers when humans (even the ones around now) do a better job.

Blogger spinoza October 13, 2019 10:20 AM  

The errors came from how files were classified so what were the erroneous sorting metrics that have changed, I’m sure it’s hilarious.

Blogger Karhu October 13, 2019 10:28 AM  

The scary thing is that this is an example of the system working. The code was published, so 100 studies by multiple groups used it. A data set at this university was available for another scientist at it to double check it and stumble upon the bug, a difference in how different operating systems sorted files. That there was such a bug speaks to how people who barely understand how to program, and use other people's programs in script kiddie fashion.

But most of the time the code is never published, code, data, and how to use them aren't even preserved inside a single institution. In the rare cases like Climategate we see how this results in new programmers not being able to get old programs and old data sets to reproduce previously published results.

And we see it doesn't matter for anything really important, since politics trumps science. See also how the scientific fraud of Ancel Keys in 1955 resulted in the crusade against fats which is still being fought more than six decades later.

Blogger VFM Bear October 13, 2019 10:42 AM  

I'm going to miss indoor plumbing the most.

Blogger Felix Bellator October 13, 2019 10:44 AM  

"Enough is left besides to search and know.
But knowledge is as food, and needs no less
Her temperance over appetite, to know
In measure what the mind may well contain,
Oppresses else with surfeit, and soon turns
Wisdom to folly, as nourishment to wind." Paradise Lost, Milton

Blogger LP9 October 13, 2019 10:45 AM  

Tech ruined skype.

Science ruined science, code excuses will not be accepted.

Blogger Nathan Hornok October 13, 2019 10:51 AM  

So much science "research" now happens "in silico" as they say. The various fields have narrowed down into such fractal categories, and they all rely on software that most researchers probably don't understand. Combine this with the institutionally entrenched false assumptions, and it is clear to see that the whole cathedral is collapsing. It will take awhile, but eventually more people will start to realize that no net progress is being made. The small areas of advancement are being out-paced by the areas where things simply don't work anymore.

Every time a study fails to be duplicated, is an example where science simply doesn't work anymore. This failure may be in part due to lack of integrity, but it might also be due to various researchers not fully understanding all the variables involved in their study. They have narrowed down into such focused realms of study that they are simply blind to all the other factors involved that affect their data (not least of which is their own false assumptions that accept the prevailing scientific dogma, i.e. evolutionary theory). The study recreation rate is already hovering at around 50%, and IMO it's going to keep getting worse.

If the study recreation rate was around %75 that would be alarming, it would mean that false assumptions and mistakes are at work 25% of the time. However, this would be a manageable burden of errors that the self correcting mechanisms of peer review could overcome. When the error rate is %50, how do you even begin to sort the mistakes from the truth and make any progress at all? Science is dead.

Blogger Jack Ward October 13, 2019 10:53 AM  

The horror of it all. Is the human race really doomed? With an ice age [glacial] nearly upon us and all the stray asteroids lining up for target practice, maybe we should just get drunk and eat popcorn.
Reality, though, is that decent Engineers are worth much more than most scientists.

Blogger Stilicho October 13, 2019 10:53 AM  

Forget science. We should be worried that engineering infrastructure-power grid, traffic lights, etc.- is increasingly dependent on those code monkeys. Don't even get me started on the attempts to push self-driving cars on us.

Blogger Keith October 13, 2019 11:00 AM  

@JG @Ominous Cowherd

The underlying issue isn't with the math library implementation:

> Sun then adjusted the code to fix the glitch, which had to do
> with how different operating systems sort files.

The fact that these guys appear to be relying on an algorithm that gets different results depending on the order that the OS globs "*.myfile" (my best guess at what went on, based on the article) is flabbergasting.

Blogger Rakshasa October 13, 2019 11:05 AM  

Using python for scientific purposes.

What's next, BASIC for nuclear science?

There's a reason why some languages are annoyingly strict about data types and are a pain to make portable.

Blogger spinoza October 13, 2019 11:11 AM  

“Science ruined science, code excuses will not be accepted“

00001010 01110100 01101000 01100101 00100000 01101110 01100101 01100111 01100001 01110100 01101001 01110110 01100101 00100000 01110011 01110000 01100001 01100011 01100101 00100000 01101111 01100110 00100000 01100010 01101111 01110100 01101000 00100000 01110011 01110111 01100001 01110011 01110100 01101001 01101011 01100001 01110011 00100000 01110011 01110101 01101101 01110011 00100000 01110100 01101111 00100000 01110011 01100101 01110110 01100101 01101110 00100000 01110011 01100101 01110110 01100101 01101110 00100000 01101111 01101110 01100101 00100000 01110011 01101001 01111000

Blogger Karhu October 13, 2019 11:18 AM  

@14 Nathan Hornok:

It isn't quite that bad, because there's no single thing that's "science". Psychology is facing a 90% reproducibility failure crisis. Biology and medicine are something worse than 50%, although some of that involves hard to transfer skills and reagents. Chemistry seems to be better, many fewer variables. "Condensed matter physics", solid-state physics with liquids added, is often very practical and gets quickly tested in the real world of engineering. Moore's Law is still hanging in there, if you have Chinese or Koreans running things instead of Pajeets, that's Taiwan Semiconductor Manufacturing Company (TSMC) and Samsung vs. Intel.

One other factor is that a lot of bad science is handled sub rosa, word gets out that you can't trust the work of a particular research lab. The PI isn't punished, he's just ignored. This can happen to entire countries, like the CCP created a "number of publications" metric to judge their scientists, there are multiple subfields where no one trying to do real science trusts anything coming out the PRC.

That self-correction mechanism is corrupt, but to repeat, politics trumps science.

Blogger VFM #7634 October 13, 2019 11:30 AM  

Forget science. We should be worried that engineering infrastructure-power grid, traffic lights, etc.- is increasingly dependent on those code monkeys. Don't even get me started on the attempts to push self-driving cars on us.

And, what the bleep is up with the full-fledged push into replacing absolutely everything analog with digital? Especially since the analog is in many if not most cases superior?

Blogger Chase October 13, 2019 11:35 AM  

Science has been dead for some time. More alarming is that we are unable to maintain any semblance of engineering. See the collapse of the still-under-construction Hard Rock Hotel yesterday. Vox was right: there are people alive today who will have to deal with unreliable indoor plumbing in First World countries.

Blogger rognuald October 13, 2019 11:36 AM  

It will be interesting to see what caused the collapse of the Hard Rock hotel in New Orleans. Pictures of the construction workers showed them to be fine "Americans."

Blogger Esmar Tuek October 13, 2019 11:45 AM  

Climate trend prediction models have absolutely no fudge factor though, 100% reliable

Blogger Karhu October 13, 2019 11:47 AM  

@17 Keith:

"The fact that these guys appear to be relying on an algorithm that gets different results depending on the order that the OS globs "*.myfile" (my best guess at what went on, based on the article) is flabbergasting."

You're guess is correct, see the README file in the published supplementary files: https://pubs.acs.org/doi/suppl/10.1021/acs.orglett.9b03216/suppl_file/ol9b03216_si_002.zip

"The key correction to the original scripts is the inclusion of the list_of_files.sort() that sorts the files prior to calculating the Boltzmann averages.

def read_gaussian_outputfiles():
list_of_files = []
for file in glob.glob('*.out'):
list_of_files.append(file)
list_of_files.sort()
return list_of_files"

He also added a sanity checking function. The whole field is close to cargo cult programming, akin to Feynman warning us about cargo cult science starting in 1974.

Blogger Gregory the Great October 13, 2019 11:53 AM  

George Soros to Huawei:

Build me a cheap mass market calculator on the basis that 2+2=5

Blogger Brett baker October 13, 2019 12:05 PM  

A big part of the problem is "I believe in evolution" until it comes up against PC believers.

Blogger Ska_Boss October 13, 2019 12:07 PM  

Perhaps these soon to be unemployed scientists should learn to code.

Blogger DonReynolds October 13, 2019 12:08 PM  

For a short while, I was a tax auditor and revenue agent for the state in the liquor-by-the-drink division. I was surrounded by attorneys and CPAs, without being either. When I first read the audit procedures, I noticed a glaring mathematical bias (repeated arithmetic averages of arithmetic averages) and pointed it out as a "matter of fact". And that was how I spent nearly all of my short tenure at the Department of Revenue...convincing attorneys and CPAs that it was not just a theory or an opinion, and they could verify it themselves by setting up a dummy set of books and doing the state audit procedures. My immediate supervisor was Tom Gore, cousin of Al Gore, and he was the only one willing to believe it. So, every week, there was another meeting to discuss my "theory" and "opinion" that there might be something wrong with audit procedures that had been tested in the courts, hundreds of times, by "really smart people"....so they cannot be wrong. Personally, I did not care either way, except when I thought of the many people who had been taken to court and lost thousands of dollars over taxes they did not owe, and penalty and interest that should have never been charged. Some of the businesses had actually been forced to close, because of the error in the audit procedures. Revenue people, especially the higher ups, were surprisingly sensitive and aggressive about defending the audit procedures. Had Tom Gore not backed me up, I am sure I would have been labeled "crazy" and discarded.

After four months, I was asked to re-write the audit procedures to remove the mathematical bias, which I did, along with my normal duties. At the end of my six month probation, I was not recommended for permanent hire and discharged. As for the taxpayers, all over the state, that were ripped off and cheated for years by faulty audit procedures, I suggested that the department go back and refund the amounts they overcharged, and interest and penalty. As for the obvious torts leading to civil suits against the state, that would be for general counsel and the state supreme court to figure out. The department decided to do nothing, make no public announcement, no refund to taxpayers.....nothing other than correct the error (and get rid of the person who corrected the error).

Blogger Clayton Barnett October 13, 2019 12:16 PM  

I'm reminded of Dr Pournelle's Co-Dominium Series, where the US & Russia deliberately corrupt databases to retard tech innovation (and its related political instability).

Blogger Snidely Whiplash October 13, 2019 12:22 PM  

a difference in how different operating systems sorted files.
I'm trying to think of a valid mathematical operation in which the sort order of the files should be important. I'm not coming up with anything.
@Don Reynolds,
You could have publicized the issue yourself, and earned a hefty sum as a consultant/expert witness in the numerous civil suits that would ensue.

Blogger RandyB October 13, 2019 12:26 PM  

Dentistry. I'm going to miss dentistry.

Blogger ASH October 13, 2019 12:28 PM  

I had low tire pressure on a new vehicle not too long ago. Figured it was time to get a tire pressure gauge for the glove box. Stopped at a cycling store thinking they would have the old school pen shapped gauges that fit anywhere. Nope. Everything was big, clunky, digital and ran on batteries... also they sold for no less than 20 dollars. My dad still has his pen shapped gauge from 30 years ago working just fine.

Blogger The Lab Manager October 13, 2019 12:32 PM  

I read the rest of the article. But this was a 'victory for science'. Yeah, okay, whatever. And it's only five years after the first study.

Blogger Akuma October 13, 2019 12:34 PM  

What scientist runs Microslut Winderps? My first instinct when someone builds a scientific app on the aformentioned is to hunt them down and beat them to a bloody pulp.

Blogger Akuma October 13, 2019 12:44 PM  

"Im trying to think of a valid mathematical operation in which the sort order of the files should be important. I'm not coming up with anything."

Maybe one of the files determines a force to change the momentum of an object. The timing of the file running would determine when the force is applied do change direction. Hence However, sort order (name of file). If coded properly the naming and sort order doesnt matter. Just have to tell the script to grab a specific file. Essentianlly, they built it to look good in a GUI, not EMACs.

Blogger Avalanche October 13, 2019 12:45 PM  

Algo-decay. It's real and it's here!

By happenstance, I'm just now re-reading Castalia House's Corrosion. Wonder how many reactor-mgmt programs and direct-the-medical-robot in surgery programs and direct-the-flows at waste treatment plants, and and and.... ALSO sort files differently.

I don't think even popcorn will do.

Blogger Crew October 13, 2019 12:50 PM  

I get the feeling that Vox is very busy these days. Perhaps with a certain venture.

Blogger Dire Badger October 13, 2019 12:58 PM  

ASH- check on wish.com. They still have the old style gauges. I hate buying chinese, but when Americans stop making what you want...

Blogger Keith October 13, 2019 12:59 PM  

@JG @Akuma

>re: ew python, windows is gross, what's next genocide^H^H^H^H^HBASIC?

People doing scientific computing with python aren't using python itself for the arithmetic. They use NumPy, which is a wrapper for the commonly used stable and performant LAPACK linear algebra library (originally written in FORTRAN) (plus bindings to FFT and others).

People like using Python because it makes things like interfacing with test harnesses, automation, generating reports, and *dealing with files* easier. Except that's what seems to have bit them in the ass this time :-)

What it really did is expose a potentially fatal flaw in their whole approach, and underscore OP's main point: everything everywhere seems to be staffed with an ersatz version of the caliber of people we had doing things a couple of generations ago.

Blogger English Tom October 13, 2019 1:02 PM  

@Esmer Tuek

I've heard they are creating a triple verified process to prove the veracity of climate change. It's going to be called the Thunberg technique.
Absolutely 100% proven, trusted, reliable!

Blogger Snidely Whiplash October 13, 2019 1:06 PM  

Akuma wrote:Maybe one of the files determines a force to change the momentum of an object.
I understand perfectly how you could write such an abomination. I've seen it hundreds of times. I don't see how an intelligent person would think it was okay to write it that way. File name sort makes perfect sense if you're rotating files, or deploying application builds or the like. It makes no sense at all for a mathematical operation. Even if you're storing interim results into files for another process to handle, DO IT DETERMINISTCALLY, pass the f'ing filename.
Of course I've seen very intelligent and experienced people make this same mistake, because they assume that what the OS is doing is deterministic, or they fail to consider failure modes. But still...

Blogger kurt9 October 13, 2019 1:08 PM  

I think much of research is based on an over reliance on computer modelling in place of actual experiments. This is a well-known problem in scientific circles these days. Its easier and cheaper to make and run computer simulations instead of conducting real experiments, when the latter is the only thing that is real. I think this is a combination of laziness, trendiness, and the prevalence of computer geekdom these days.

I also do not understand the use of Python programming language rather than Matlab in this case. Matlab is intended for scientific calculation and has proven itself in the field for a couple of decades. Python is new and is intended more for business and social media data processing.

Blogger Karhu October 13, 2019 1:12 PM  

@35 Akuma:

"What scientist runs Microslut Winderps?"

There are no great choices for power users who don't have systems administrator skills, and assuming he had a choice in what his university will allow to be connected to their net, he might have picked it because it's the only system that can control some of the instruments he uses.

Apple's software development processes are even worse than Microsoft's after Satya Nadella fired the people who tested Windows, and Linux on the desktop is still a complete mess, with Gnome and systemd steadily making it worse. Linux doesn't run Word for Windows, which depending on the science subfield might be required for submitting papers. I would be running a BSD there wasn't domain specific software I need that runs on Ubuntu LTS distributions.

Blogger Keith October 13, 2019 1:13 PM  

Matlab uses the exact same linear algebra library that the Python people use:

https://www.mathworks.com/help/matlab/math/lapack-in-matlab.html

"Linear algebra functions and matrix operations in MATLAB® are built on LAPACK, and they continue to benefit from the performance and accuracy of its routines."

Blogger kurt9 October 13, 2019 1:13 PM  

Matlab is big in this area. There are specialized software tools like COMSOL (used in physics) that are useful for certain areas of research. However, you have to plan and do your initial experiments first, then you can use the software tools to model your (already performed experiments) to come up with ideas for the next experiments. But the actual experiments must take precedence over the computer modeling. The computer modeling is only a guide for future experimentation, nothing more.

Blogger Jim the Curmudgeon October 13, 2019 1:38 PM  

I can't stand Python, but it is super popular with millennials and data scientists these days. Along with others here, I would have thought that Matlab or Fortran would be better choices in this area of science.

However, even errors in Python would have been caught with a decent test suite. Did the authors of this 'script' not put a bunch of tests in a git repository so that other people could evaluate it on their own platforms? I might forgive the original authors for not catching the bugs, since they are probably not trained in software engineering, but to provide a script without test cases is a big mistake.

I don't think this is as shocking as the case where a grad student discovered errors in a spreadsheet that was used in a major economics paper. Said paper was relied upon for actual policy implementation.

Blogger RedJack October 13, 2019 1:40 PM  

Jack,

Most newly minted engineers can't do the math. So expect massive bridge failures soon.

Blogger The Depolrable Podunk Ken Ramsey October 13, 2019 1:46 PM  

Computer guy working in a research lab here. This sort of thing with Python is problem. The problem has become managing Python. It's gotten to be a real pain.

Previously before the ascent of Python you'd find scientists and engineer using MATLAB to work out proof and algorithms and so forth. And then they'd hand the results over to computer guys like myself to hammer out stable and efficient implementations. Well now we've had the rise of Python and R, and MATLAB is receding. That's not necessarily a bad things -- working with MATLAB can be a tremendous pain for a large number of reasons.

The numeric facilities in Python are top notch, but as mentioned above this is true because Python is sitting there on top of a large body of proven C and FORTRAN code that's doing all the heavy lifting. Python acts as a "glue" language. You want to most if not all of the actual calculations outside of it. God forbid you have to do a loop in Python.

The problem is in the "gluers" who are using Python, the users. (Same problem applied when we had MATLAB). And chasing them around has resulted in a Wild West in managing Python on machines.

On my main development machine on day I decided to count the Python installations. I got up to 16 different Python installation before I quit in disgust. I was by no means done. It's out of control.

I had my "main" Python installations, two versions, Python 2 family and Python 3 family (they are not compatible). And, each one comes in 32-bit and 64-bit flavors, so that's 4 Python installations just at the main level. But then, each tool and each new program might require a certain installation of Python and various dependencies, so that brings on version hell. One project may require Python 3.5 only, not Python 3.6. The other might require 3.6 but not 3.5. You've got competing requirements for various versions of Python that require specific versions of all their dependencies. Tensorflow and OpenCV with Python in a specific configuration here, but another completely different one there, and so forth. And, throw in something like MingW or Visual Studio, which comes in 32-bit or 64-bit compilations flavors each of which might not only require Python 2 and Python 3 and each in 32-bit and 64-bit flavors, so you might be looking at a package installer laying down EIGHT Pythons.

People have taken to using Anaconda and VirtualEnv to manage all these growing numbers of Python installations. And that might work until you install stuff for the next project and it breaks the pathing and botches everything.

I've seen groups beginning now to abandon the whole idea of shipping packages for installation. They're sending virtual machines or containers. "Here's how we have it configured. Just run this VM. DON'T update anything on the VM!"

It's chaos and fragility.

Problems like this, where an environment change might introduce a slew of downstream bad effects, are laying out there in minefield. In this case it was operating system differences -- the CLASSIC case where have to be sure you verify inputs and outputs -- and nobody bothered to notice for FOUR years.

What worse, nobody has done what's the standard-issue practice in my lab at least whenever we take in an outside algorithm. "Here's this algorithm, make sure when we run it over the sample datasets that our results match theirs. To precision." That is step one. One!

Blogger Azure Amaranthine October 13, 2019 1:47 PM  

"And, what the bleep is up with the full-fledged push into replacing absolutely everything analog with digital? Especially since the analog is in many if not most cases superior?"

If you don't want people to see reality through instruments capable of continuum color spectrum, you ensure all their instruments are replaced with those only capable of recording in black and white binary digital.

Blogger Unknownsailor October 13, 2019 1:47 PM  

DonReynolds wrote:For a short while, I was a tax auditor and revenue agent for the state in the liquor-by-the-drink division. I was surrounded by attorneys and CPAs, without being either. When I first read the audit procedures, I noticed a glaring mathematical bias (repeated arithmetic averages of arithmetic averages) and pointed it out as a "matter of fact". And that was how I spent nearly all of my short tenure at the Department of Revenue...convincing attorneys and CPAs that it was not just a theory or an opinion, and they could verify it themselves by setting up a dummy set of books and doing the state audit procedures. My immediate supervisor was Tom Gore, cousin of Al Gore, and he was the only one willing to believe it. So, every week, there was another meeting to discuss my "theory" and "opinion" that there might be something wrong with audit procedures that had been tested in the courts, hundreds of times, by "really smart people"....so they cannot be wrong. Personally, I did not care either way, except when I thought of the many people who had been taken to court and lost thousands of dollars over taxes they did not owe, and penalty and interest that should have never been charged. Some of the businesses had actually been forced to close, because of the error in the audit procedures. Revenue people, especially the higher ups, were surprisingly sensitive and aggressive about defending the audit procedures. Had Tom Gore not backed me up, I am sure I would have been labeled "crazy" and discarded.

After four months, I was asked to re-write the audit procedures to remove the mathematical bias, which I did, along with my normal duties. At the end of my six month probation, I was not recommended for permanent hire and discharged. As for the taxpayers, all over the state, that were ripped off and cheated for years by faulty audit procedures, I suggested that the department go back and refund the amounts they overcharged, and interest and penalty. As for the obvious torts leading to civil suits against the state, that would be for general counsel and the state supreme court to figure out. The department decided to do nothing, make no public announcement, no refund to taxpayers.....nothing other than correct the error (and get rid of the person who corrected the error).

It should be noted that Accounting is not a STEM field, but is, instead, in Arts.

Math is not their strong suit. Basic single variable algebra is about all they have to know.

Blogger Azure Amaranthine October 13, 2019 1:48 PM  

That way when you stab them in the back you can assure them that they aren't bleeding since there's no red anywhere to be seen.

Blogger Azure Amaranthine October 13, 2019 1:54 PM  

"I'm trying to think of a valid mathematical operation in which the sort order of the files should be important. I'm not coming up with anything."

Imagine this, you label your files in a way that your code doesn't discriminate between different ones. Perhaps the differences between "Seed_01" and "seed 01". You put different things in the different files, and one OS orders the first one at the top because it starts with a capital letter, while another orders the second first because it's a shorter entity and thus strictly higher in alphabetical order.

Whichever pops first gets called.

Blogger Azure Amaranthine October 13, 2019 2:03 PM  

"I don't see how an intelligent person would think"

A what now? Diversity hires are just as intelligent. Even better credentials (from India)!

Blogger liberranter October 13, 2019 2:04 PM  

To help understand how devastating this sort of thing could be for the profession and practice of science, consider the very frightening possibility that modern science increasingly relies upon the sort of people responsible for enhancing your user experience of Skype and manning Twitter "customer support".

An even more horrifying thought is to ask the question of how much pharmacological research has been conducted using similarly flawed algorithms and applications, and how much of the flawed results thereof have gone into the production of the Big Pharma poison currently being pawned off on us as "medicine?"

Blogger pyrrhus October 13, 2019 2:15 PM  

O/T The Extinction of Reason....https://www.melaniephillips.com/extinction-of-reason/

Blogger pyrrhus October 13, 2019 2:17 PM  

@55 One of my kids who's a software developer says some of the same code is used in other applications, including statistics...

Blogger Cyril October 13, 2019 2:20 PM  

Very little scientific research is repeated in the laboratory. Peer review means other scientists have read the paper and 'agree' it is 'reasonable.'

Blogger liberranter October 13, 2019 2:23 PM  

Linux doesn't run Word for Windows, which depending on the science subfield might be required for submitting papers.

Simple workaround for that. Install LibreOffice on the Linux platform. It creates and converts .doc/.docx files.

Blogger Esmar Tuek October 13, 2019 2:38 PM  

Big pharma just ignore anything that doesn't equal their desired outcome. That's been going on forever. They don't need crappy code.

If antidepressants were actually tested against placebo over a decent length of time....

Blogger Crew October 13, 2019 2:47 PM  

It is very clear that all those journalists who are out of work need to learn to code!

NOT!

Blogger Doktor Jeep October 13, 2019 3:02 PM  

This is why I don't follow the cargo cults of scripted languages.
Is it so hard to use the standard compiled and linked languages? Is FORTRAN so damned hard? My first programming gig was maintaining old FORTRAN code.
Or is there no IDE for FORTRAN full of intellisense and ready made modules with all the answers baked into them?
Native binaries. Then use a calculator to double check. Oh but that's too hard for the H1B wonders and snowflakes who get these jobs now.

Blogger luisonmcbiel October 13, 2019 3:02 PM  

@38 Yeah he is not posting much, maybe something is coming. Social hierarchy book? ASOS completed?

Blogger Karhu October 13, 2019 3:03 PM  

@59 liberranter:

"Linux doesn't run Word for Windows, which depending on the science subfield might be required for submitting papers.

Simple workaround for that. Install LibreOffice on the Linux platform. It creates and converts .doc/.docx files."

LibreOffice in my experience isn't hardly as good as Word, and can save out files in native ods that it can't read back. If you live and die by publication acceptance, do you really want to take a chance that conversion to and from really messed up Microsoft formats will work 100%?

What about the chance of a forced retraction because that messed something up? At the very least, you'd still need a machine that can run Word to do final proofreading of the converted document, and that's a dangerous approach, especially since you're generally in a bit of a hurry in case someone might be poised to scoop you.

Blogger Karhu October 13, 2019 3:11 PM  

@62. Doktor Jeep:

"This is why I don't follow the cargo cults of scripted languages. Is it so hard to use the standard compiled and linked languages?"

Yeah, for most people. And someone using a language implementation with an interpreter is going to finish the job a lot faster than someone bound to the edit-compile-link-load-run-debug loop, everything else being equal.

That's also a call to remove lower skilled people from programming, and I'm not sure overall that would be for the best. Yes, they make more errors than us "pros", but they also get a lot of good done that just doesn't have the time or budget to engage a pro, and finding a real pro is *HARD*. Amateurs could easily have better outcomes overall doing it themselves than the common pattern of hiring a faker who he can't judge until the job should be finished and it's not.

Blogger God Emperor Memes October 13, 2019 3:11 PM  

"Accounting is not a STEM field, but is, instead, in Arts.
Math is not their strong suit."

That is 100% accurate. My wife is an accountant and has frequently said, "You don't have to be good at maths to be an accountant. You don't even have to be very smart..."

Blogger cyrus83 October 13, 2019 3:17 PM  

The problem of faulty scripts is not new, I recall the bad random number generator RANDU from IBM causing lots of studies done in the 70s to be suspect because the script was anything but random.

For a crowd that tends to be anti-religion, scientists have an unusual amount of faith in their silicon deity to furnish them accurate results.

Blogger sammibandit October 13, 2019 3:18 PM  

@DonReynolds, just to check in, if I understand what you're saying ... a mathematical error normally taught around age 13 is what not only got you sacked but tax revenue wrongly collected and not disbursed?

If I did weighted averages of a set of averages wrong for sales reports I'd get my superior mad at me for wasting his time. But he was from a naval family of ethnic Scots. Wow.

Figuring out how to compute these things isn't hard if you're trained in Arts like I am, coming into business pretty ignorant ... you just search how to do it and use free online calculators to set up your own spreadsheet. No excuse not to do it correctly.

Blogger Hammerli 280 October 13, 2019 3:24 PM  

I'm getting ready to retire as an engineer (large UAV flight test, to be specific) in about three years. Just shy of 60. Sounds like I might be able to find part-time employment.

Blogger Jack Ward October 13, 2019 3:25 PM  

48. RedJackOctober 13, 2019 1:40 PM
Jack,

Most newly minted engineers can't do the math. So expect massive bridge failures soon.

So, maybe they should do some reading at Miles Mathis site; specifically, his unpack and his 'corrections' and 'simplifications' of Calculus. See section 3 [and other locations].
Perhaps, things like that would help them out a bit. Or, not.
Good luck to those precious snowflakes [or, just flakes]

Blogger Jack Aubrey October 13, 2019 3:30 PM  

Akuma wrote:What scientist runs Microslut Winderps? My first instinct when someone builds a scientific app on the aformentioned is to hunt them down and beat them to a bloody pulp.

It's a poor craftsman who blames his tools.

Blogger sammibandit October 13, 2019 3:55 PM  

@Jack Ward. Great recommendation. I just finished Calculus Simplified. Really good article for helping visual or tactile learners. I was thinking how the curve flattening was a lot like adding ease when sewing around a collar. I sew each stitch on that zoomed in curve, making minute angle changes each stitch, to get a bigger curve like how Mathis explained the Greeks did. Just totally great. Makes me think abstract thought is foundational to Greco-Roman civilization.

Blogger Crew October 13, 2019 4:02 PM  

@62: Scripting languages have their place. You just have to know their limits and be prepared to put the effort into a compiled language when a scripting language is not appropriate.

Blogger Crew October 13, 2019 4:05 PM  

@72: Seems Miles does not know the correct words:

Several years ago I wrote a long paper on the foundation of the calculus. The paper was not really dense or difficult—as these things usually go—since I made a concentrated effort to keep both the language and the math fairly simple.

Perhaps someone who does not know the difference between concerted and concentrated should not write.

Blogger weka October 13, 2019 4:05 PM  

This comment has been removed by the author.

Blogger Doktor Jeep October 13, 2019 4:44 PM  

There would be more pros if they actually hired pros.
It's like replacing real nurses with medical techs from third world shitholes.
Always to save money, they say. it's really all about profit. But...muh maximizing profit and decreasing costs! Yeah and this is where it leads.
If this errant code bug issue has cost anybody any grief, then lawsuits need to happen now, if not for compensation then for a duty to bring integrity back. Make it more expensive to hire Pajeets from certification Mills and snowflakes too muddled by their mental illness.

Blogger Doktor Jeep October 13, 2019 4:50 PM  

Air conditioning. Losing that will suck.

Blogger eldiabloloco October 13, 2019 4:50 PM  

It may be more likely to be seen with interpreted scripting languages, but the same sort of differing results can be found with compilers as well. It happens on compiler version upgrades, even on the same platform. Been there, with multiple vendors and platforms.

Blogger Doktor Jeep October 13, 2019 4:56 PM  

You would be amazed to see how they are putting microcontrollers in places were a simple logic circuit and some Darlington pairs would be sufficient. I think they do it to allow for easier changes later on. Microcontrollers are super cheap now.

Blogger Doktor Jeep October 13, 2019 4:59 PM  

I have yet to have a SPICE simulation tell me exactly what the real circuit is going to do. I mainly use it to avoid cratering the prototype board.

Blogger tublecane October 13, 2019 5:01 PM  

Is shoving an apostrophe into "Hawaii" part of some style manual, or is the author of this piece looking to get punched?.

@8- Obviously you're not lazy/innumerate.

Blogger kurt9 October 13, 2019 5:35 PM  

Doktor Jeep wrote:I have yet to have a SPICE simulation tell me exactly what the real circuit is going to do. I mainly use it to avoid cratering the prototype board.

We used an early version of SPICE in college in the 80's. You could design an amplifier with an infinite bandwidth! These tools have their limits.

Blogger Ominous Cowherd October 13, 2019 5:47 PM  

Linux doesn't run Word for Windows, which depending on the science subfield might be required for submitting papers.

If you aren't using LaTeX for publishing, it's not science.

Blogger bodenlose Schweinerei October 13, 2019 5:48 PM  

a common chemistry computation

One involving a "C" and an "O" and a "2"?

I guess is does pay to Learn To Code!, although I think the idea is to learn before you start coding, not after.

Blogger Azure Amaranthine October 13, 2019 6:09 PM  

"Perhaps someone who does not know the difference between concerted and concentrated should not write."

Either word works in context, though one would normally be considered closer to redundant. Can you discern the meaning and is the meaning accurate? If so, nit picking....

"Is shoving an apostrophe into "Hawaii" part of some style manual"

It's the local spelling, and indicates the correct pronunciation. So, technically it's correct. Not including the apostrophe is the lazy/foreign way to do it.

Blogger John Bradley October 13, 2019 6:18 PM  

It's a poor craftsman who blames his tools.

Indeed. A pro can do excellent work with nearly anything; an idiot can find a way to produce garbage no matter what tools he has at his disposal.

Blogger N October 13, 2019 6:32 PM  

> I'm trying to think of a valid mathematical operation in which the sort order of the files should be important. I'm not coming up with anything.

When you have multiple floating point operations the order of the operations could radically change the result by virtue of rounding errors.

See:
https://en.wikipedia.org/wiki/Kahan_summation_algorithm

This is what happen when you have scientist using tools that they don't fully understand.
You can't just put whatever the fuck you want into a program and get a nice result.

Blogger lions paw October 13, 2019 6:41 PM  

I'm SURE some members of this group understand that replicable and predictive research is demanded by the top of the pyramid.

Muh masses get table scraps and in fact heavy doses of dis-science.

It's a major way the top tier stays ahead of muh masses. The more the masses leave the cave and learn to discern the less the top tier will have an advantage.

Keep fighting

Blogger Damelon Brinn October 13, 2019 6:41 PM  

@25 Karhu, that still looks like someone's first week in programming class. glob returns a list, so they loop through it adding each item to a new list, which they then sort (in-place, thankfully), and then return the list, and then whatever called it is almost certainly going to loop over it again. There should be no loop at all here, just one line, assuming you want to use glob: return sorted(glob.glob('*'))

That's the kind of crap you get when people imitate what they've seen in tutorials without understanding the concepts, which is how most programming is done today. Scripting languages are fine. I use shell and perl scripts all the time on Unix systems to get work done. But I'd never send them to a Windows user and expect them to work as-is, because operating systems vary and every shell script that does anything useful makes operating system calls. Python is a scripting language, but most of those using it don't know that or even what it means. It's the one hammer they were given, and everything looks like a nail.

Blogger N October 13, 2019 6:46 PM  

I will add that my best guess is that the baked solution of the library was pushed beyond its scope by scientist of other disciplines that didn't take into account that operating over their datasets would carry big floating point errors and they based their results on it.
And because in different operation systems the datasets were ordered differently, then the accumulated error was different and it generated inconsistencies with the results.
When you are doing experimentations with a p value of 0.05 then an innocent accumulated error of 0.03 after millons of operations can completely fuck your hypothesis.
But this is not fault of the tool in itself.

Most scientis of other disciplines that heavily rely on computers don't have an education in computer science.
Most people don't know for example that a perfectly baked solution could potentially explode and give faulty results given certain conditions.

You can for example have a perfectly fine algorithm for resolving linear algebra problems that will explode if the elements of the matrixes are too small.
Some tools can fight this by using high precision implementations, but of course this is a tradeoff and now the whole computations time will tank.

Blogger sammibandit October 13, 2019 6:46 PM  

Crew wrote:@72: Seems Miles does not know the correct words:

Several years ago I wrote a long paper on the foundation of the calculus. The paper was not really dense or difficult—as these things usually go—since I made a concentrated effort to keep both the language and the math fairly simple.

Perhaps someone who does not know the difference between concerted and concentrated should not write.


Maybe so. It's a silly error. But I got a C- in uni intro calc so you've got to understand I already don't learn well in math. Even though I studied like I never studied before for that class putting in 3 hours a night I earned the lowest passing grade. That essay Mathis wrote helped me understand what was previously a mystery.

Blogger OneWingedShark October 13, 2019 6:54 PM  

Keith wrote:@JG @Ominous Cowherd

The underlying issue isn't with the math library implementation:

> Sun then adjusted the code to fix the glitch, which had to do

> with how different operating systems sort files.

The fact that these guys appear to be relying on an algorithm that gets different results depending on the order that the OS globs "*.myfile" (my best guess at what went on, based on the article) is flabbergasting.

About the only thing I can think of is if the files involved were either (a) containing instructions themselves, or (b) contained information that was [essentially] operated on with "rounding" at each step.

Rakshasa wrote:Using python for scientific purposes.
What's next, BASIC for nuclear science?
There's a reason why some languages are annoyingly strict about data types and are a pain to make portable.

That strictness is the reason that I really like Ada; I've personally compiled 25-30 year old programs written for different architectures with a compiler that literally didn't exist at the time only having to (a) search and replace one identifier [which had become a keyword in a subsequent standard], and (b) split a single file into two because the implementation I was using has the limitation of not having multiple compilation units in a single file.

Repeated rounding will screw up your calculations; which is why you only do it once at the end of the process.

VFM #7634 wrote:Forget science. We should be worried that engineering infrastructure-power grid, traffic lights, etc.- is increasingly dependent on those code monkeys. Don't even get me started on the attempts to push self-driving cars on us.

And, what the bleep is up with the full-fledged push into replacing absolutely everything analog with digital? Especially since the analog is in many if not most cases superior?

The propaganda that "digital is better" in a lot of cases, in my more cynical thoughts it's because they want to be able to real-time edit human knowledge: imagine if all references to, say, Plato suddenly disappeared from electronic media from text to audio to video, just gone.

Blogger Primus Pilus October 13, 2019 7:22 PM  

rognuald wrote:It will be interesting to see what caused the collapse of the Hard Rock hotel in New Orleans. Pictures of the construction workers showed them to be fine "Americans."

https://twitter.com/vitriolicgoy/status/1183480073321615360

"So it turns out the guy behind the collapsed Hard Rock hotel, Praveen Kailas was previously convicted of ripping off the state, and was allegedly cutting corners by employing unqualified labor."

Blogger buzzardist October 13, 2019 7:34 PM  

This matches very closely with my experiences with academics in the sciences (both social and hard). I've edited numerous papers (dissertations, articles for publication, etc.) for grad students and professors. On many occasions, they would be using statistical calculations that they completely did not understand or drawing conclusions from data whose initial research question was not at all designed to reach that conclusion. I remember one occasion in particular where I asked a grad student, "I'm not sure if I understand your use of this statistical model here. You want the model to say this, but the model seems designed to say something else. Can you explain this better?" She couldn't. In fact, she didn't understand the statistical model herself whatsoever. She couldn't explain what it was doing. She couldn't explain why she was using it, except that her professor told her to use it. And it plainly wasn't relevant to what she was trying to prove in this article she was preparing for publication.

Needless to say, she didn't edit that part of the paper. It was sent along for publication, where it received considerable less scrutiny in peer review than I gave it.

Mind you, this was not at some low-level university. It was a top-ten public research university in the U.S. That grad student is likely in the position now of being the professor telling grad students what formulas and models to use. She still doesn't understand the models herself, just as her professor probably never understood them. Their entire discipline is reliant upon people in another discipline recommending to them formulas and models, and upon other people in technology writing for them software to run those calculations and models...and they take at faith that all of this is useful, accurate, and relevant for their field and for their particular research questions. As often as not, in my experience, they are wrong, and they are all (professors, grad students, peer reviewers, journal editors) oblivious to the fact that they are wrong.

Blogger SciVo October 13, 2019 7:39 PM  

I think that in a lot of cases, the point is for insiders to profit off of mere churn, without actually producing a better product (and in many cases actually making a worse one). I see a lot of post-WWII dietary advice that way, as a cynical ploy to profit from disrupting traditional American diets with something more easily mass-produced.

Blogger James Lovebirch October 13, 2019 7:40 PM  

@74

Unless Mathis had people he made his effort in concert with, it wasn't a concerted effort. Keep your sperging to yourself.

Blogger justaguy October 13, 2019 8:04 PM  

Why would one expect any science to come from academia? The university system was corrupted with the influx of large amounts of government funding from the National Science Foundation at the same time enrollments skyrocketed. The result of grants controlling what is researched and quickly what is found ( results not expected/in the narrative means no follow-up grants). Research and publication is more often than not for tenure/employments and signalling-- not discovering something. Besides-- just about every mid-wit academic knows that if they depart from the established narrative with something new-- they are not heralded but cast out. Look at how many discoveries had some poor guy having to endure decades of scorn before enough giants in his field died and someone new looked at his data?

The mid-wits who stayed in the university system knew it was in trouble but the lure of high salary with tenure as long as you kept grants coming in was too much for too many non-christians. The results are publication not for "science" but for tenure and signalling. Hell, the Trial lawyers have gotten smart and find ways to fund research for their benefit to get $ from companies.

Science stopped in way too many fields a long time ago and we only get minimalist discoveries that slip between the cracks.

Engineering still goes on and is improving-- as long as the trial lawyers can be kept away. Someone blows up a building and the trial lawyers will blame the engineers who designed it, and the construction company that build it.

Blogger Hieroglyph October 13, 2019 8:09 PM  

Back in the day, when I was failing my physics higher, I once got gamma-annoyed (I was 16) at my teacher. He was telling us that light is both a particle and a wave. This sounded like bullshit, so I said so. He responded, quite correctly, that I didn't know much about physics, and needed to learn more.

So, I was recently interested in the Big Bear's streams. BB is much more familiar with physics, and he too expressed some scepticism around the wave\particle duality. The lesson, of course, is that the teacher was correct, I really needed to know more about physics before opening my stupid mouth. However, I may have unwittingly stumbled upon a larger subject, merely by observing that an explanation sounded a little daft. George Monbiot once wrote a book where in one chapter he climatesplained that harnessing the power of cow farts could be a useful tool against climate change. And I'm not kidding. Sounds a little, dare I say it, daft?

I'd home-school if I had children.

Blogger Snidely Whiplash October 13, 2019 8:19 PM  

The thing is, light is not both a particle and a wave, light is neither a particle nor a wave.

Neither model applies across the full domain. The closer you are to the high end of the spectrum, the more it acts like a particle, the closer to the low end of the spectrum, the more it acts like a wave.

Which implies that an entirely different model must apply, and we none of us know all that much about physics.

On the other hand, we know exactly how each part of the spectrum acts, and can accurately predict the behavior when we know the pertinent parameters are known. We just don't have a single model that can explain it.

Blogger Nikolai Collushnikov October 13, 2019 8:20 PM  

I've often wondered how many scientific papers where blown by doing calculations in Microsoft Excel.

Modern Python does have a opt-in static type system.

Blogger James Lovebirch October 13, 2019 8:41 PM  

I've enjoyed Ken Wheeler's videos on his YT channel Theoria Aphophasis resurrecting the theory of the ether. I can't be sure he's totally right right now, but before watching those videos I had no clue what a magnetic field could be.

He is correct that all the men who were instrumental in the invention of our electrical grid subscribed to views similar to his and rejected the atomistic photon/wave-particle-duality paradigm.

Blogger map October 13, 2019 9:01 PM  

Unknownsailor wrote:It should be noted that Accounting is not a STEM field, but is, instead, in Arts.

Math is not their strong suit. Basic single variable algebra is about all they have to know.


Whatever one wants to say about accounting, they have a whole field dedicated to checking the work of other accountants: It's called Audit. You can actually make a living policing other's' work.

Nothing like that exists in any of the other sciences. That's why Amgen found 90% of their seminal papers were unable to be duplicated.

https://voxday.blogspot.com/2013/10/science-has-lost-its-way.html

Blogger James Lovebirch October 13, 2019 9:04 PM  

Here's an example of a set up that produces an effect that is reminiscent of Tesla's claims of free energy

https://www.youtube.com/watch?v=O75gkskkkZ8

Blogger HouellebecqGurl October 13, 2019 9:07 PM  

Slightly OT since this thread is very OT-we have a serious pharmacological issue in this country.
5 drugs that I have been prescribed to take have for very long periods of time, have now been pulled off the market.
The 1st drug, taken years ago & pulled while I was still on it, caused a large ulcerated hole in my stomach lining which still flares up to this day

I've reached the point where I'm afraid to take any meds or have any surgical procedure done, because I've watched them incompetently open & close my husband so many times he looks like a human skin quilt.
I agree the future of unreliable AC & toilets are a frightening thought, but I'm afraid of these people RIGHT NOW.
I guess the best advice is try not to get sick or need surgery. Some Pajeet from Mumbai might kill you.
My hubby's cardiologist, neurologist, phrenologist, vein specialist and general practitioner- all street shutters & still prescribing oxycodone in high doses and amounts routinely because f*ck these white Americans on their way out, who cares about the opioid epidemic.

Blogger DonReynolds October 13, 2019 9:08 PM  

Unknownsailor wrote:
It should be noted that Accounting is not a STEM field, but is, instead, in Arts.

Math is not their strong suit. Basic single variable algebra is about all they have to know.


It is probably worse than you might think. For accountants and attorneys, a simple arithmetic average has magical properties. Add, subtract, multiply and divide is pretty much the end of their arithmetic. Maff is a totally foreign language. Statistics, the philosophy of mathematics, is worse than Greek to them. Finite math? Non-linear algebra? All you will get is blank stares, even if you can put it in the form of a graph. My ex-wife was an attorney SHE SAID because she could not do math. Yet, these are the people who run the revenue departments and make the big decisions on tax collections.

sammibandit wrote:@DonReynolds, just to check in, if I understand what you're saying ... a mathematical error normally taught around age 13 is what not only got you sacked but tax revenue wrongly collected and not disbursed?

If I did weighted averages of a set of averages wrong for sales reports I'd get my superior mad at me for wasting his time. But he was from a naval family of ethnic Scots. Wow.

Figuring out how to compute these things isn't hard if you're trained in Arts like I am, coming into business pretty ignorant ... you just search how to do it and use free online calculators to set up your own spreadsheet. No excuse not to do it correctly.


There were no on-line calculators in 1978, in fact, I got no end of grief from other staffers at Revenue for using a TI-58, instead of the standard issue ten-key calculator. Everything was still pencil and working papers, as it had always been.

The basic idea of audit procedures was to determine if the tax owed had been paid and the method should return the correct tax owed, as if an honest person had kept the records and paid the tax when due. When the procedures do not result in the same tax amount as an honest set of books, then people get refunds that are not due or they get charged for taxes that are not owed...and they might resist to the point of collection and court action. That was the worst part of the experience for me, people who had honestly paid but the audit procedures overstated the tax in EVERY case. Most people just paid it.

Blogger HouellebecqGurl October 13, 2019 9:19 PM  

Ok, I feel better about my OT post above. As someone with serious lifelong health issues that can only be maintained and a 20 yrs older husband with life-threatening health issues, I have decades of experience with the medical field & prescription and OTC meds. I have tons and tons of stories of their incompetency from the life threatening to just the sheer incompetence of hospital aides and home health nurses and rehabbers.
Home health nurse comes to home to inspect, clean and bandage surgical incisions.
Each (non white) one they send does the procedure incorrectly and the task falls to be to become the home health nurse because when they insisted he go to an after surgery rehab clinic for a few days in order to observe him closely, the place nearly killed him and I had to call an ambulance to go get him from there and take him to the ER where he nearly died.
Sorry for the rambling, this is a topic very close to me & the things I've seen are impossible to believe if you haven't had Indian doctor-blavk nurse-Dominican aide care here in the good ol USA.

Blogger furor kek tonicus ( 'A Pillowed Dawn', the Boomer Horror movie ) October 13, 2019 9:56 PM  

33. ASH October 13, 2019 12:28 PM
Everything was big, clunky, digital and ran on batteries... also they sold for no less than 20 dollars.



check a NAPA store.

the pen type gauge they have online also doubles for checking tread depth:
https://www.napaonline.com/en/p/OTC4751?partTypeName=Tire+Pressure+and+Tread+Depth+Gauge&keywordInput=tire+pressure+gauge

but they have a round analog unit for pressures only.

or you could keep an eye out for garage sales around you and grab an old one for cheap.

Blogger Kirk Parker October 13, 2019 10:01 PM  

Azure @ 85,

NOT if we're writing English, which does NOT use apostrophe to indicate a phoneme.

Blogger VFM #7634 October 13, 2019 10:05 PM  

Is shoving an apostrophe into "Hawaii" part of some style manual, or is the author of this piece looking to get punched?.

The original Hawaiian has a glottal stop between the two I's, represented with an okina.

But putting it in while writing English strikes me as PC douchery along the lines of pronouncing any Spanish name or word with a thick Spanish accent while speaking English.

Blogger cyrus83 October 13, 2019 10:12 PM  

I work in accounting and it is much more art than any sort of science. My strength is on the audit side of the business though I have had my hand in probably every function at some point in time learning through experience and/or necessity.

The worst part of accounting nowadays is that accounting software tends to be badly written, particularly when the accounting function is bundled with the operational software. I have dealt with 5 such creations in my professional life, and in each case I would have preferred having a clerk hunched over a paper ledger book recording all the transactions manually as opposed to the unholy and virtually unreconcilable mess the software produced. I know in at least one case debits and credits did not balance.

Blogger furor kek tonicus ( 'A Pillowed Dawn', the Boomer Horror movie ) October 13, 2019 10:31 PM  

42. Snidely Whiplash October 13, 2019 1:06 PM
or they fail to consider failure modes.



cf. Boeing


43. kurt9 October 13, 2019 1:08 PM
I think this is a combination of laziness, trendiness, and the prevalence of computer geekdom these days.



you're forgetting the most important consideration: failed experiments don't boost funding.

and Real World Data and Experiments have a distressing tendency to invalidate your pet hypothesis.

besides, when you're going to fake the data anyways why waste time getting Real Facts?


81. tublecane October 13, 2019 5:01 PM
Is shoving an apostrophe into "Hawaii" part of some style manual, or is the author of this piece looking to get punched?.


that's actually the proper way to spell the native pronunciation.

it's only Amerifags who have a problem with it.

Blogger James Dixon October 13, 2019 10:51 PM  

> LibreOffice in my experience isn't hardly as good as Word, and can save out files in native ods that it can't read back.

You do realize that the various versions of Word over the past few decades are equally incompatible, don't you? For simple composing and saving, LibreOffice should work fine. I wouldn't want to use it for collaborative work though.

Blogger pdwalker October 13, 2019 11:00 PM  

wouldn’t it be nice if there was another book in the corrosion series. it seems timely.

Blogger themusicgod1 October 13, 2019 11:04 PM  

Step #1 : Any science dependent upon any code that is not
a) reproducibly built
b) free software
should be considered as not acceptable and any certainty of their conclusions should be considered exponentially less likely per bit of unverified code.

Blogger Amethyst Dominica October 14, 2019 12:26 AM  

Aydin Paladin just dropped an essay showing how Academia is full of biased tribalists,and why ordinary people defer to them as an unquestioned, priestly class:

https://www.youtube.com/watch?v=PeIw91H97Fc

Blogger Keith October 14, 2019 12:34 AM  

@themusicgod1
Yes yes a thousand times yes! I’ve been advocating this for years. In an era when anyone can inspect the code of an entire OS, and an entire ecosystem has evolved around “social coding” where people fork and add function to thousands of powerful projects all the time, there is simply no excuse for important software like climate modeling codes that influence the expenditure of trillions of dollars to remain secretive.

When people push back and tell me that it’s infeasible, I ask them why, then, if I want to open up a factory that is expected to release airborne pollution, not only does the EPA have agreed-upon software air dispersion models to help industry and the government manage this part of the economy THEY EVEN HOST THE OPEN SOURCE CODE ON GOVERNMENT SERVERS. My favorites are the components of the AERMOD toolkit:

https://www.epa.gov/scram/meteorological-processors-and-accessory-programs#aermet

There is absolutely no excuse not to adopt a rule that requires open source code and data for government climate research $$.

Blogger The Depolrable Podunk Ken Ramsey October 14, 2019 2:57 AM  

I've grown skeptical of the benefits of Open Source code. Yes it has its advantages and these are often vaunted, but it has downsides, too, which are rarely talked about. Many very significant downsides exist with Open Source. (Not that proprietary software is some kind of Panacea.)

In any event, look at this problem, and here we had a community actively using a flawed, open source script for 4-5 years and nobody noticed.

Why? I wonder. I'm curious to find out what exactly went wrong. They say something was peculiar about the file name sorting with Python. I wonder how this got screwed up because Python doesn't exhibit this kind of behavior in normal usage. I started using Python in my lab with Python 1.5 on an IRIX machine decades ago, and I've used it since on many versions of Solaris, Windows, Linux, BSD, MacOS, Android, iPhone, and also a wide number of embedded systems with all kinds of nuttiness about them. Never had a problem with file globs, what went wrong here? How did this fly under everybody's noses for so long?

This is a punch in the gut to the Open Source ethos. Here you have an obvious problem sitting right out there in the open. Nobody noticed that the guys on the Macs and the guys on Windows and the guys on Linux were all getting different results.

I think the answer is to be found in the user base. They aren't checking their work. Is this a surprise? We know they aren't really peer reviewing each other or trying to reproduce each others work to check it. And they obviously aren't doing this because they're too busy with their sleeves rolled up digging through pitfalls in each other's published algorithms or code.

The benefits of open source do not exist just because the source is open. They only accrue when a significant number of people get active in scouring the code base. If that doesn't happen, you've got nothing.

Open source software absent active review is as useless as the scientific method without active review.

Blogger Paul M October 14, 2019 3:34 AM  

Nathan Hornok wrote:So much science "research" now happens "in silico" as they say. The various fields have narrowed down into such fractal categories, and they all rely on software that most researchers probably don't understand.
Scientists tend to be bad at coding. Not their fault: it's not their profession, they're expected to know about molecules, not about the ins and outs of versioning Python libraries.

Problem is, they actually are pretty smart. Smarter than me, anyway. When smart people hit a wall, they will find ways around it.

Last job, I learned something important via dealing with a biologist: never give a scientist access to DEV or TEST environments. Actually, never give any end user access to those environments.

Blogger Paul M October 14, 2019 3:39 AM  

The Depolrable Podunk Ken Ramsey wrote:I'm curious to find out what exactly went wrong. They say something was peculiar about the file name sorting with Python.

A sensibly written system would not have a dependency like this. You don't use file names for that sort of thing - you have a table or a document or something goddammit mapping the sort keys to the corresponding file name. This isn't a problem with Python, it's a problem with the people writing the Python.

Blogger tublecane October 14, 2019 4:13 AM  

@85- Foreign? Pretty sure we colonized that island and replaced its monarchy with banana barons, or something like that.

Ain't no apostrophe on the Hawaii state seal (1959).

Blogger tublecane October 14, 2019 4:19 AM  

@109- The original Hawaiian. Okay. But what are we talking about here? The Hawaii that is currently a U.S. state, I think. I've seen images of their constitution, and I'm pretty sure there's no apostrophe there.

Blogger tublecane October 14, 2019 4:20 AM  

@111- Amerifags...you do realize Hawaii is a part of the U.S. They *are* Amerifags.

Blogger Dr. T October 14, 2019 5:51 AM  

I looked up the original paper describing the problem and it is indeed quite trivial: The Python script reads output from another software, a well-established program for quantum-mechanical calculations called Gaussian. For some reason that probably makes sense Gaussian writes its output into many different files, one for each atom and parameter, with both the indicator number of the atom an the parameter included in the file name. When the he Python script reads these files, it extracts these information from the file name. However, in the faulty version it did this only for the list of files containing the first parameter. For subsequent parameters it was assumed that the list of files was sorted identical to the first one. However, some operating systems, in particular Linux, have no default sorting of file names because this depends on the local settings. In this case parameters can be matched wrong. The reason this went undetected for so long seems to be that the Python script only calculates corrections, the effect of thermal movements, to the data calculated by Gaussian. These corrections are small and thus faulty output was not obviously wrong, only of lower quality than it could have been. So it took a few years before somebody noticed it and did a proper cross-platform validation of the script.

Blogger Ominous Cowherd October 14, 2019 8:27 AM  

tublecane wrote:@111- Amerifags...you do realize Hawaii is a part of the U.S. They *are* Amerifags.


Citizens, but it's increasingly obvious they aren't Amerifags.

Blogger Ominous Cowherd October 14, 2019 8:32 AM  

James Dixon wrote:You do realize that the various versions of Word over the past few decades are equally incompatible, don't you? For simple composing and saving, LibreOffice should work fine. I wouldn't want to use it for collaborative work though.

I have used Libreoffice for collaborative work, because it is more compatible with MSWord than is MSWord.

Blogger Nikolai Collushnikov October 14, 2019 9:18 AM  

117. The Depolrable Podunk Ken Ramsey

A counter example that I already mentioned is closed source Microsoft Excel. It had known flawed statistical functions for years. Microsoft didn't care, and took years to do fixes. With open source, at the very least, you could patch your own copy.

Blogger Azure Amaranthine October 14, 2019 9:18 AM  

"NOT if we're writing English, which does NOT use apostrophe to indicate a phoneme."

Hawaii isn't originally an English word. We use French pronunciation when using French loan words, for example.

"But putting it in while writing English strikes me as PC douchery"

Maybe for some people. Having lived there I can tell you a lot of the locals would do it with the apostrophe, PC or not. More of a loyalty to tradition thing for them.

Blogger Azure Amaranthine October 14, 2019 9:27 AM  

"But what are we talking about here? The Hawaii that is currently a U.S. state, I think."

Hawaii is an odd one.

On the one hand, it's the result of us being imperialistic. So, the natives aren't us even if we forced them to behave like us to an extent and called them us.

On the other hand very few of the people who live there -- or even consider themselves cthonic at this point -- are actually the original natives, being instead Samoans and other pacific islanders.

Right of conquest valid and permanent? Hawaii. Invalid and will eventually fade? Hawai'i is only a matter of time.

Blogger OneWingedShark October 14, 2019 9:30 AM  

Jack Aubrey wrote:Akuma wrote:What scientist runs Microslut Winderps? My first instinct when someone builds a scientific app on the aformentioned is to hunt them down and beat them to a bloody pulp.
It's a poor craftsman who blames his tools.

There is some truth to that, but it's also true that it's a poor craftsman that uses wrong or inferior tools.
Take, for example, C: it's used everywhere, despite being wholly unsuitable for anything that requires high-reliability or that is medium- or large-sized. — For the better part of 40 years people have been saying "a good programmer can write safe C" and yet we have Heartbleed-style errors and buffer-overflow errors even now — at some point we-as-an-industry need to take a look at ourselves and realize that, no, we really do need tools that assist us in producing quality software, and that includes language: it should help, not hinder.

John Bradley wrote:It's a poor craftsman who blames his tools.

Indeed. A pro can do excellent work with nearly anything; an idiot can find a way to produce garbage no matter what tools he has at his disposal.

And yet, what pro carpenter would regularly use a rock instead of a hammer? Or a shard of glass instead of a saw?

Crew wrote:@72: Seems Miles does not know the correct words:

Several years ago I wrote a long paper on the foundation of the calculus. The paper was not really dense or difficult—as these things usually go—since I made a concentrated effort to keep both the language and the math fairly simple.

Perhaps someone who does not know the difference between concerted and concentrated should not write.

Except it's not incorrect; if you are trying to keep both the math and language simple, 'talking' about the same thing, then it is [keeping them in] concert, and thus a "concerted effort". That said, it does flow more naturally with 'concerted' and is likely a spell-checker 'correction'.

Ominous Cowherd wrote:Linux doesn't run Word for Windows, which depending on the science subfield might be required for submitting papers.
If you aren't using LaTeX for publishing, it's not science.

Or PostScript.

Blogger Keith October 14, 2019 11:59 AM  

The Depolrable Podunk Ken Ramsey wrote:I've grown skeptical of the benefits of Open Source code. They say something was peculiar about the file name sorting with Python.

There is nothing peculiar about the filename order with Python. The very first sentence of the glob specification lays it right out: https://docs.python.org/3/library/glob.html : The glob module finds all the pathnames matching a specified pattern according to the rules used by the Unix shell, although results are returned in arbitrary order.

As discussed above, what happened here is the team's algorithm had a previously unknown behavior--their code gets different results based on the ordering of their input files. Apparently it's a large enough difference to potentially call into question final conclusions drawn from the simulations as well.

I've grown skeptical of the benefits of Open Source code.

The major benefit of forcing climate hoax software to be open source isn't any expectation that it will become better software (even though some think it would). It's that major policy decisions are being made based on code that you literally have no idea if it's real, or some bullshit like this:

time.sleep(1e6) # pretend to be calculating some STEM stuff
print("It will be 3 degrees warmer by next thursday")

That is an over simplification of the potential fraud, of course, but opening the code to scrutiny would let us finally see what types of assumptions they are feeding their models, whether their model code appears to be written by people who know what they are doing, etc.

Blogger SPQRCincinnatus October 14, 2019 12:25 PM  

As a faculty member at a Top 10 University, I can confirm "muh science" has never been in a worse state. Academia is largely a nonproductive circle-jerk that demands conformity to the ever-changing insanity of Leftist virtue signaling. Somehow, someway practically all "research" needs to integrate climate change and "diversity".

Blogger Ominous Cowherd October 14, 2019 1:02 PM  

OneWingedShark wrote:Or PostScript.

LaTeX is how you make Postscript. Or Plain TeX, but most journals expect to see LaTeX.

Blogger Ominous Cowherd October 14, 2019 1:06 PM  

OneWingedShark wrote:Take, for example, C: it's used everywhere, despite being wholly unsuitable for anything that requires high-reliability or that is medium- or large-sized. — For the better part of 40 years people have been saying "a good programmer can write safe C" and yet we have Heartbleed-style errors and buffer-overflow errors even now —

A good programmer. Do you see the flaw in that plan?

Blogger Jack Ward October 14, 2019 2:53 PM  

@74. CrewOctober 13, 2019 4:05 PM

Back to this posting from yesterday.
Concerted/concentrated. OK. I'm not sure you have a valid complaint with Mathis on this. I read concentrated and knew exactly what he was talking about.
Maybe your thought or comprehension needs some attitude adjustment. Just saying....

Blogger OneWingedShark October 14, 2019 3:22 PM  

Ominous Cowherd wrote:OneWingedShark wrote:Or PostScript.
LaTeX is how you make Postscript. Or Plain TeX, but most journals expect to see LaTeX.

PostScript is a fully capable programming language and was designed to be so [see the PostScript Language Reference Manual], while LaTex is a markup/macro system for the TeX typesetting language, which IIUC originally wasn't turing complete.

Ominous Cowherd wrote:OneWingedShark wrote:Take, for example, C: it's used everywhere, despite being wholly unsuitable for anything that requires high-reliability or that is medium- or large-sized. — For the better part of 40 years people have been saying "a good programmer can write safe C" and yet we have Heartbleed-style errors and buffer-overflow errors even now —
A good programmer. Do you see the flaw in that plan?

What plan?
I am of the opinion that C is terrible due to the sheer amount of detectable and avoidable pitfalls it incorporates, and suspect it has set Computer Science back decades. — Indeed, I would argue that a lot of the things that preclude "a good programmer" from the standard CS graduate, aside from the "it's popular in industry" BS, is in fact due to the laziness C (and its arguable decedents) encourage… doubly so if that descendent is PHP.

Blogger Ominous Cowherd October 14, 2019 3:50 PM  

Shark, a good programmer can write good code in assembler. C will help him do it faster and more portably. He will probably choose to use some dialect of Lisp.

Good programmers will avoid the avoidable mistakes, but good programmers are rare. It goes beyond mere competence in some language. That's the flaw in any plan that requires good programmers.

Blogger sammibandit October 14, 2019 3:52 PM  

@DonReynolds

Funny you bring up showing your work. As in not funny. Obviously I wasn't trained to do that--ever. Another sign of decay. One gal I worked with at that job i spoke about did all her pricing by hand. Since I filed her papers I was able to teach myself the calculations for doing pricing. She's Gen X.

Hawaii isn't originally an English word. We use French pronunciation when using French loan words, for example.

I still got to laugh at the difference between how Anglo Canadians and Anglo Americans pronounce French words. Tabernacle is one that always makes me laugh. Clique just makes me die inside. In French it's "kleek" but English it's "click". I used to get so confused when popular girls talked about their clicks. So I'd asked them if they bought their lotion from Clinik and not Clinique. In general it's true we retain pronunciation unless it's a word used by whammen.

Blogger OneWingedShark October 14, 2019 4:26 PM  

Ominous Cowherd wrote:Shark, a good programmer can write good code in assembler. C will help him do it faster and more portably.
I'm going to call BS on this assertion: C is horribly non-portable.
The reason is because there's so much that's implicitly defined and there's [often] no way to explicitly ensure required parameters/ranges (that are often reliant on the implicit) are adhered to.

As an example, consider "int" — what would the impact be compiling code on a machine where "int" is 16-bits? 64? How about 48? What about "char" is it 8-bits? 16? the same size as int? — Now, what do all these fundamental interactions do to your compilation's integrity?

One reason I like Ada is because I can say something like "Type Domain is range 0..(2**63)-1;" when the domain is known to be a Natural number representable in 63 bits, I could further say "Subtype Constrained_Domain is Domain range 1..17_179_869_184;" if, after some analysis, I know the results will be in that range.

He will probably choose to use some dialect of Lisp.
I rather like the idea behind Lisp, but much prefer static type-systems; there also may be considerations for real-time and/or embedded that would preclude using Lisp.

Good programmers will avoid the avoidable mistakes, but good programmers are rare.
I agree; but far too often they are taught in a manner contrary to the "Software Engineer" mindset.

It goes beyond mere competence in some language. That's the flaw in any plan that requires good programmers.
True, but you're not thinking about the other side: the management often uses the absolute wrong tool for the job. If C is, as billed, "a portable assembler" than why the hell is it being used as the main implementation in large, long-lived, complex programs (e.g. modern desktop OSes)?

…the answer is really quite simple, it's because the [management of] corporations want coders, not programmers, and certainly not Software Engineers and so don't value the skillsets of Software Engineers and "good programmers". Which is why they've got such an addiction to the H1B "programmers" — the cheap labor is the only thing they care about — because quality actually costs.

Blogger RedJack October 14, 2019 7:59 PM  

@70
Jack.
I have done that. But my issue is that I can not afford to teach an engineer remedial math and get the job done. If I do that, the Lean guy will walk over and walk both of us out.
And that is another issue. We run so lean, it becomes impossible to train or even mentor. Instead of a senior engineer with a team of juniors to grow and lead, you get one doing it all. If you don't know how, you get bounced or replaced by three H1B's.

Blogger Snidely Whiplash October 14, 2019 9:08 PM  

"Lean" operation is the formalization of short term thinking as a business strategy.

Blogger John Kim October 15, 2019 12:37 PM  

I bet those scientist won't be able to reproduce their results. Science is bogus, engineering is where it's at.

Blogger Ominous Cowherd October 15, 2019 8:36 PM  

OneWingedShark wrote:: C is horribly non-portable.

The reason is because there's so much that's implicitly defined and there's [often] no way to explicitly ensure required parameters/ranges (that are often reliant on the implicit) are adhered to.


C is indeed non-portable. So is assembler, to which I was comparing it.

Post a Comment

Rules of the blog
Please do not comment as "Anonymous". Comments by "Anonymous" will be spammed.

<< Home

Newer Posts Older Posts