2 Comments

My comment is a bit tough - please don't take that as - I totally disagree with you - as I do think you are on to something. If you'd like to spend some more time in discussion where we could jointly come up with a stronger concept together - let's do so.

Expand full comment
Nov 1·edited Nov 1

This article has some merit, but it does rather sound like it comes from someone who hasn't spent enough time in academia to even earn a PhD and doesn't really understand how academia does and should work.

On the plus side

I rather agree with the notion that researchers should go in to industry, understand real problems in industry and then come back with solutions in a format that industry can understand. I actually think that standards today severely lack proper research in to them - for example MISRA standards that lots of people follow, include some real howlers if you look at (https://repository.tudelft.nl/record/uuid:646de5ba-eee8-4ec8-8bbc-2c188e1847ea)

" From the data obtained, we can make the following key observations. First, there are 9 out of 72 rules for which violations were observed that perform significantly better (α = 0.05) than a random predictor at locating fault-related lines. The true positive rates for these rules range from 24-100%. Second, we observed a negative correlation between MISRA rule violations and observed faults. In addition, 29 out of 72 rules had a zero true positive rate. Taken together with Adams' observation that all modifications have a non-zero probability of introducing a fault, this makes it possible that adherence to the MISRA standard as a whole would have made the software less reliable."

Generally when I look at standards in robotics I get the feeling that they were created when a group of people came in to a room and dreamed up some ideas of how to do things. Except in a few instances like communications standards which have to work otherwise the standard breaks they are usually poorly validated - and often contain nonsensical ideas in them yet they force thousands of people to spend endless hours following them.

A lot of people don't even grasp why we have standards in the first place? In a progressive business system standards are the hypothesis for the best way of doing things, which implies further experiments and refinements. Practically no-one views them that way. Some very successful businesses do (e.g. Toyota) but for most people they are viewed merely as a set of rules that come down from on high.

If we can get people to understand the scientific process better and be more involved in it, with standards being viewed as the best known hypothesis of current knowledge then that would, of course be great! Everyone should be a scientist as everyone should be executing the hypothesis.

On the negative side

Publish or perish comes from the fact that people stopped trying to assess whether the research that someone was doing was brilliant or not, they just looked at the number of papers they wrote, which is easy. This lazy approach led to the fairly obvious manipulation that people go out an publish lots of meaningless papers. The right solution is stop doing the lazy thing, and figure out how you are going to incentivize people for a small number of highly brilliant contributions to knowledge not a large number of hopeless ones. There are pretty obvious ways to do this, such as award research funds to deans of universities that know what they are doing, give them a mandate, hold them accountable and so forth, rather giving cash to faceless committees of people to hand out cash through anonymous research awards which consume enormous amounts of time for everyone and yield questionable results.

If you think about "publish or perish" its silly - because you want a small number of insightful papers from researchers not a large number of meaningless ones. We'll get in to the fact that our most brilliant researchers come up with a small number of principles and that's what makes them great in a minute.

Perhaps more importantly why are "standards" as they are currently promulgated the right format. The way standards are written today, with dozens, hundreds or thousands of pages of impenetrable text the right format? Research should simplify. Einstein is known for basically 4 equations that could be written on the back of a postage stamp. Maxwell is known for 4 equations, 3 of which he stole from other people, Feynman only wrote only 30 papers and really is known for quantum electrodynamics which is just so elegantly simple . I could go on, but the greatest researchers of our time basically took something very complicated and condensed it down to a simple principle that enlightened all of us. THAT - is what researchers should be doing - and some do.

If your idea of a "standard" is a set of well validated principles that everyone can understand, are simple to follow and make things better, something that takes no more than half a page to write down, then I agree with you. But that's not what we call "standards" today - so I would say research to principle - is probably a better way of describing it.

But really that's what academia should be doing all along anyway. Coming up with a hypothesis, doing experiments to prove or disprove that hypothesis, articulating their results and coming up with a conclusion. That 4 step process (lots of people misarticulate it as a 3 step process like you have) has actually been validated as leading to enlightenment. That's what academic publishing should be about - it should be the messy process of coming up with a small number of crucial ideas / principals that are well understood to be right that help all of us - to the extent that it doesn't well shame on us.

Expand full comment