Some time ago I read an article about safety regulations in some profession, I forget now which. A trainee sat in on a class with an instructor who would go over OSHA law as it applied to them. He began his class by telling the trainees that every regulation he was about to teach them was written in blood.

I spent just over three decades of my life as a software engineer. You could say that my job didn’t involve any hazards to my own health. True enough, I sat at a desk staring into a computer screen trying to mentally picture the algorithms I was creating in program code. Not very dangerous stuff. To me. Unless you count all the alcohol and nicotine I consumed to manage stress (I worked at an IT firm back when I was a youngster as the mailroom clerk, and a programmer there called his inevitable pack of cigarettes his “programmer’s candy”). But what I did was potentially extremely dangerous to others if my program didn’t do what it was supposed to.

One of the contracts I worked on ages ago, before I came to Space Telescope, was for a large medical diagnostic company. I had a piece of their new diagnostic machine’s reporting system. The machine it was hooked up to would identify specimens that had whatever disease they were testing for just then, and the report terminal would keep a database of results and generate reports for individual patients and disease rate of spread in a population for the CDC. The federal government imposed requirements on the software process for medical equipment, including allowable software tools, multiple code reviews, and independent verification of requirements as the software matured. Because lives were at stake.

I wasn’t subjected to that level of oversight at Space Telescope because I was not involved in creating any of the actual flight software. I did business applications mostly, science needs business applications too, although I could also build you a PC if you point me at a source of parts for one. I attended meetings, issued progress reports on my work, discussed requirements with managers. Those I gave my output to, both during development and when it was in production, were able to verify that my software was correct or not from their own experience with their numbers.

I would occasionally get a call from one about something missing from one of their spreadsheets. I was almost always able to trace that to the data not yet being entered in the “problem report” or “discrepancy report” databases. I could re-run the program for them when that was fixed. But now and then I found a bug in my software that I had to fix that was making the numbers wrong. That was unlikely to get anyone killed, but it could have given management a false idea of progress being made. So it was still very important to get it right.

At the major public utility I once contracted for, I worked with an accountant who Knew His Numbers. He could look at my output and go…”ah…you know…I don’t think you’re picking up…” and he would refer to some number bucket, one of many such, that I needed the report to digest. And I would go to my code and look and sure enough I wasn’t doing that. So I’d fix it and run the output by him again. I loved it. He didn’t know software but he knew his numbers, and I didn’t know his numbers but I knew my software and it was a perfect working relationship. He was my independent verification.

I can’t stress enough how important independent verification of your software is. The more mission critical it is, the more thorough that must be. It gets to a point, and I lived this while working on that medical diagnostic machine report, where periodically a group of other programmers get together with you and beat up on your code. It can be brutal. Ask me how I know. But when it’s that important, like with medical diagnostic software, it Must happen. Lives actually are at stake.

But beyond bugs, there’s also making sure both you and your users understand what the software is, and is not doing. Most of the business software I wrote wasn’t about life and death situations, or so you’d think. But that’s not always cut and dried. Like the one at my first big contract at a major public utility where I was tasked with designing and coding a report that would tell management how much revenue each of their field technicians were generating and at what cost. My report uncovered such a massive subsidy of one department by another such that, so I was told later, jaws dropped in the boardroom. Annapolis was Not going to be happy.

Then came the layoffs. I heard later that one of those laid off had a sudden heart attack at his dinner table in front of his wife and kids and died. You could say it was management policies that did that…they knew they were putting money where it did not belong, that they were subsidizing something they weren’t supposed to, but didn’t realize how badly it had become until my report waved it in their faces. So they did what management often does, they just started laying people off to make the money stop going in that direction.

They did that. But…my software did that. Management wasn’t just using my report to tell them where the money was coming from and going to, they were using it to tell them who their best workers were. And I still deeply regret it. I wonder if I couldn’t have done more in meetings to remind them a computer program can’t substitute for human judgment. There was a union that was supposed to be protecting the workers from that sort of thing. You could say none of that was on my plate. But my hands were in it too. He was a nice guy. Always had a friendly smile for me.

I never doubted after that, how dangerous my tools could be.

The procedures and best practices of software development have evolved over the decades from big iron to little silicone for a reason. Maybe they aren’t all written in blood, but they are all at least written in sweat and tears. They say mistakes are human but to really foul things up takes a computer. But no. What it takes, is thinking you can get away with bending the rules. Maybe just this once. Because we have a deadline. Because we have to get it out the door. Because there isn’t enough money for best practices. Just this once. And then the next once. And the next once. And the next. And the next. Take off your engineer’s hat and put on your management hat.

There are reasons for all those practices and procedures. And especially, there are reasons developers do not test their own software before putting it into production. Yes, do your unit testing, but then you hand it off the testers. Testers are your friends. They keep crap from going out the door that worked fine for you but that’s because you were running it on a workstation with all your software tools on it, and the test data you cobbled together that you always use. Out in the wild it will be different.

The physicist Richard Feynman once said (paraphrasing) that science is a way of trying not to fool yourself, and you are the easiest person to fool. I think that’s a good general rule for software testing too. Testing is a way of trying not to fool yourself. And since you are the easiest person to fool, because you wrote it, you hand it off to an independent tester. Or put another way: beware the result you wanted because you had a deadline. It needs looking at by someone with no stake in the outcome, other than honestly reporting did it pass or fail.

There are reasons for the procedures and best practices of software engineering. They need to be respected. Not for the sake of tradition, but for the sake of not screwing things up badly. Because we are human and it takes a human to screw things up badly. But we can be aware of this, and build our guardrails accordingly. We do that, and we are capable of wonderful, marvelous things.

Why am I telling you all this? Just venting. I’m old and cranky is all.