The Black Liszt

Making Things Worse with AI

AI is on a hot streak. Businesses claim they’re applying the latest AI and/or ML to their operations. There are articles and books about the wonders of AI and how it deeply threatens to replace humans doing various things. Awwwwk! What can we do??

Dilbert to the rescue

When in doubt about any important technology issue, for example about whether a programmer should become a manager, it often helps to turn to the wisdom of Dilbert for help. Here he illustrates how AI will transform HR and management in technology firms.

AI has built a perfect data-driven case on the uselessness of perpetually-lazy employee Wally. How can Wally possibly defend himself?

As you see in the cartoon, Wally pulls off the feat with aplomb. First he points out a typical flaw is logic (how different is Wally from the rest), and second he identifies what in statistics is called a confounding factor to account for his perfect record of failure. The boss’s method of job assignment was not, of course, included in the data processed by the AI algorithm, nor of course the classic error of correlation vs. causation.

The main point of the Dilbert cartoon is that AI made things worse in this case. Without Wally being given the chance to defend himself to a human being, he would have been out the door.

A little history

AI has been on the verge of ousting human beings from important jobs since the 1950’s. First it was checkers and chess. By the mid-1960’s, ELIZA and SHRDLU were having conversations with people in English. ELIZA impressed many with its conversational, interactive abilities, while SHRDLU could not only talk but it could answer questions and perform actions in its world of blocks. By the early 1970’s many experts were talking about how AI would soon rule the world.

I won’t go through all the history, but all the talk about what would happen “soon” faded away and was forgotten. Some event like Big Blue beating human chess masters would capture the news, which would again fill with experts opining about what would happen “soon.”

We’re in a hot cycle again. People in traditional roles are again being cajoled into “thinking outside the box” in order to change what they do to incorporate the wonders of AI. If they do it right, maybe they can dramatically improve things and put boatloads of people out of work! Thinking outside the box is easy enough to say, but making practical results happen with it is another things altogether. It happens less often than you might think.

Funerals for misbegotten AI projects are extremely rare, in spite of the astounding death rate.

There is real value in AI

My undergraduate thesis at Harvard was written about AI. It was an early attempt to design the data structures that would represent real-world knowledge in an intelligent robot. I focused on “simple” things like walking and catching a ball. In contrast, the vast majority of AI efforts then and since focus on the hardest and most uniquely human things people do.

Where AI and ML have made the most headway is performing tasks that most humans think are repetitive. And before you get to that, there is the perpetually challenging job of getting all the required data. Even then it is challenging to get a real-world system working. These posts illustrate the main strategies for getting good results.

Meanwhile, the current furor about the imminent taking over the world by AI is likely to continue making heavy use of the future tense, as it gradually fades as it has in the past, along with the broken promises and failed predictions.

Who is Censoring US Intelligence Agency Criticism on Amazon?

A critical review I wrote of a book on Amazon was censored/suppressed. Was it profane or otherwise deserving deletion? No. It simply gave one star and pointed out errors and omissions, while acknowledging that fans of US Intelligence Agencies would admire the book. The review appeared briefly, then the link to it went bad. I interacted with customer service, who said they would submit a trouble report. I wrote another review, which appeared briefly then vanished. I tried once more. The last review was up for months. I got a warning from Amazon warning me that an unspecified review I had written failed to meet community standards, and if I did it again I would be banned. The review stayed up for many more months. And then it disappeared!

Spies, Lies and Algorithms was published by the Princeton University Press. It was written by Amy Zegart, a prominent, well-connected Professor at Stanford. I'm writing up this little experience because I suspect it's not an isolated incident; It's a step towards a society run like Orwell's 1984. Here's the story.

US Intelligence Agencies and Cyber-security

The grossly misnamed US "Intelligence" Agencies continue on the path of proclaiming their wonderfulness while US cybersecurity continues its free fall into unmitigated disaster. I describe the details of the sad state of cybersecurity in both government and private sectors in these posts.  For government cybersecurity, see this. Here is a detailed analysis of the DNC server hack declared by Intelligence agencies to be a Russian hack. They failed utterly to prove it, and ignored the clear evidence that the most prominent emails never resided on the DNC server!

It should come as no surprise that they are doing what all bureaucracies do, i.e., protect themselves and their reputations by hiding the facts and suppressing criticism. The author of the book clearly and explicitly takes the side of the Agencies on this issue and similar ones. While she has given courses to congressional staffers on cyber issues, she clearly does little but echo the propaganda of Agency Experts.

The Book

Amazon, always serving up books they think I might buy, showed me Spies, Lies and Algorithms: The History and Future of American Intelligence by Amy Zegart.

What a cleverly designed cover! I checked it out. The author seemed to be well connected and educated about the Agencies and recent events. Take a gander at her background:

Amy Zegart is the Morris Arnold and Nona Jean Cox Senior Fellow at the Hoover Institution and Professor of Political Science (by courtesy) at Stanford University. She is also a Senior Fellow at Stanford’s Freeman Spogli Institute for International Studies, Chair of Stanford’s Artificial Intelligence and International Security Steering Committee, and a contributing writer at The Atlantic. She specializes in U.S. intelligence, emerging technologies and national security, grand strategy, and global political risk management.

Zegart has been featured by the National Journal as one of the ten most influential experts in intelligence reform. She served as a commissioner on the 2020 CSIS Technology and Intelligence Task Force and has advised the National Security Commission on Artificial Intelligence. She served on the Clinton administration’s National Security Council staff and as a foreign policy adviser to the Bush 2000 presidential campaign. She has also testified before the Senate Select Committee on Intelligence and advised senior officials on intelligence, homeland security, and cybersecurity matters.

Skipping my usual practice of carefully scanning the reviews, I bought the book.

I then read the book. The author was nearly everything you could want: highly educated, knowledgeable, articulate, connected. There was just one little problem: she has completely bought in to the deep state, to the extent of denying that there is such a thing. She faithfully ignores what they ignore, reflects their lies and omissions and shows near-total ignorance of computers and the "algorithms" of her title. That's all! In fact, her book scared me, because it was such a compelling read to anyone who was ignorant of cybersecurity (i.e., practically everyone) or hadn't dug into the big lies.

The Reviews

I've written a number of reviews of all sorts of products since joining Amazon in early 1998. Most have been positive, all of them constructive. I thought this book needed a review from me; while my immersion in the Intelligence community was trivial compared to the author's, my knowledge of cybersecurity and related subjects clearly eclipsed hers.

I went to the review section and found a handful of authoritative-person reviews, all highly complementary, and about a dozen reviews by readers, all but one very positive.

The sole negative review, 1 star, was by a self-described retired Intelligence officer who titled his review "bias detracts too much." 57 people found it helpful. He didn't go into detail but said:

She tries to deny it but the Deep State is real and is our biggest challenge. ... She tipped her hand with selective alarm and omission of key events rampant in the first chapter.

The next most popular review, granting 5 stars and titled "There is some bias: here's why it isn't important" mostly demonstrates the power of being in the bubble:

I am a politically conservative individual studying intelligence and the IC for a number of years. Amy’s summary is accurate, thorough, and well-written. ...
I couldn’t find any factual inaccuracies that would render the book a decorative piece or kindling.

These reviews demonstrate the problems of the Intelligence community and its image. It isn't about bias so much as it is about ignorance and denial.

Writing a Review

I dove into writing a review. My goal was to praise the author for her evident virtues, but to give a few facts to illustrate the basis for my giving it 1 star. I typed it into the box Amazon provided without keeping a copy of what I gave -- silly me. I posted it and waited. Three days later I got a notice that my review was posted.

Hooray! I clicked to see the full review. This is what I got:

Cute dog, but what's going on here?

I went back once a day for many days and clicked again. The review never showed up.

Next I got online with Amazon customer service in a chat. After a bit of effort, the person agreed that the error about the Zegart review was a problem. I asked him to submit the issue for resolution, and he said he would. I waited until he said it was submitted.

Writing a Review Again

After a number of days with no change, I decided to simply post another review, this time saving what I had written. My submission of the second try was confirmed:

I never received the confirming email from Amazon saying that my second try at a review had been posted. But I kept checking, and to my surprise, my second review was posted. The bad link to the original review (Feb 12) now led to the new review (Feb 19). Hooray, it's posted! Here's a screen shot of the first part of it:

There's nothing bad going on here -- I'm just being paranoid! And the review has already gotten several votes for being Helpful.

The Second Review Disappears

I went to the page again, and noticed that the total number of reviews had gone down. I look for my review. I was thorough. It's not there. Not there.

I had a thought. I had saved the original confirming email I got from Amazon pointing to my original review (which never showed up), which mysteriously DID point to my second try at a review. I click, and here's the result:

At least they're different dogs...

The Second Review Magically Reappears

I went to the page again and found, to my great surprise, that the elusive review was back!!

Just as a test, I wrote a positive 5-star review of socks I had bought on Amazon.

The Warning

A day or two later after the reappearance of the review, this showed up in my inbox:

Notice the careful wording. Notice that the exact review was not referenced, and that no specific violation of the guidelines was identified. I carefully read the community guidelines. Nothing that I wrote could be construed as violating them. The only offense of the review was to point out the deep issues of the Intelligence Community and to give a couple specific examples that illustrate it -- one of them the supposed Russian hacking of the DNC server.

The second review stays up for many months, and then...

I checked the review occasionally over the following months. It stayed up and received dozens of "helpful" votes. It was second or third on the list of "critical" reviews.

Then, more than 6 months later, I got this.

Nothing about which content I posted that violates the Guidelines. No links. Nothing about removing the awful content. I'm just supposed to know, I guess. Naturally, I suspected that the Zedaris book review was the issue, so I went to the book's page. The book is doing well -- it's now an editor's pick in nonfiction. I went down to the reviews. The "Bias detracts too much" 1-star review is still up and went from 57 up votes to 102. Sure enough, mine is gone.

Boy, those misinformation police at Amazon are really on the job, giving proper notice and telling you what law you violated. And prompt! And careful to only take down evidence-based reviews that might change some minds. Or something.

The last chapter

I wrote all of the above. Before posting it, I checked once more and found to my surprise that the review was up again! It wasn't on the top reviews list, even though it had over 50 "helpful" votes, but it was there. Oh well, much ado about nothing. The review is visible, my post is irrelevant.

Many months later, I had an impulse to check again. The review is gone again!! I carefully checked, looking at all the 1-star reviews, which takes some clicking just to get to them. The "bias detracts too much" review is still there, still with exactly 102 up votes, unchanged from at least 6 months ago. I bet that somehow the review isn't shown to many people. The handful of other critical reviews each had just a couple votes at most.

My review -- gone again!

Conclusion

Somewhere operating deep within Amazon there is an operation to suppress criticism of the CIA/FBI/NSA/whatever. I guess it could be Amazon employees who feel the same way, without official blessing. Or maybe, as we now know was the case with Twitter, there was an explicit back-channel connection to one of the 3LA's (3-letter agencies). After being allowed to be viewed for many months, my review garnered dozens of "helpful" votes. During that time I thought that the "image protecting" operation was walking a fine line between total suppression and letting low-attention items like my review alone to avoid attracting notice. But then it was visible again, just hard to find. And now, roughly a year after it was originally posted, it's gone.

FWIW and to be explicit about it, my review was NOT politically motivated. It was driven by facts and competence, consistent with these posts. Specifically see this. The job of the agencies is to enforce the law and protect us all from criminal and foreign attacks, a job for which, sadly, they have a track record of failure. And the job of prestigious professors who teach people about cybersecurity is to objectively assess the quality of the work, not ignore failures and incompetence.

Be Healthy and Don't Schedule Your Heart Health Visit

As I write this, it's Valentine's Day and my doctor is after me! Valentine's Day is all about hearts getting together, so it makes sense that my heart should be healthy and ready for whatever comes along, right? I found this email in my inbox:

Awww, it's American Heart month too...

They even took the trouble to make a pretty graphic, with lovers getting together, hopefully with healthy hearts:

What are these numbers they're talking about?

Ah, so it's those nasty numbers you hear about so often: blood pressure, glucose and cholesterol. Oh well, who wants to die of a heart attack? Not me! Let's see what I'm in for.

They talk about the three numbers I've got to know.

All the major authorities agree that high blood pressure is a horrible thing that must be measured regularly and controlled. They really want us to be healthy -- that number of 120 was just lowered to that about 6 years ago from the 140 number it had been for many years. A huge number of people take blood pressure medications to control the pressure and get the number down.

But there's a problem. Not only is hypertension NOT a disease, taking the pills more than doubles your chances of going blind. The more "routine" side effects of the pills can be awful, even harming the heart. And they don't even help you live longer!

Let's hope the next number is worth checking.

With rare exceptions, measuring this is a way of telling how far you are to having diabetes. Which is mostly caused by bad diet and life habits a.k.a. being fat. So what does the doctor do? Maybe they'll talk about that. But the usual practice is to put you on a drug.The data shows that almost no one is helped and that a large number taking drugs are harmed.

Now we've gotten to the last number, the most mysterious of all, the one about "bad" cholesterol.

Like blood pressure, this is one where ALL the medical authorities are united: it's bad and you've got to do something about it -- basically take pills.

There's just one little problem: taking the pills and lowering your cholesterol does NOT help you. It does help pharma -- to the tune of billions of dollars. But you? Nope, sorry. And let's not talk about the fact that serious side effects are experienced by something like 10% of the people who take the pills, and that it demonstrably (by the numbers), shortens the life of women. See this for the numbers, facts and studies that demonstrate this, studies that have never been refuted.

Is it just my provider? No, they're all into it. I got this random email from ZocDoc, a place that tries to "help" you find a doctor and make appointments:

They're all in on it.

Conclusion

I admit it's hard to think that all these prestigious doctors and major institutions could be so wrong. But this isn't an exception. A massive study on hundreds of thousands of people has shown that checks of this kind -- including these specifically -- do not help people live longer, healthier lives. See this for context and summary, and this for the study itself.

There are many wonderful, health-making things doctors can do to help patients. The items mentioned in these Valentine's Day emails don't happen to be among them.

Should a Software Engineer aspire to become a manager?

Software is a strange field. Its rules and patterns for career growth and advancement are unlike most others. It's thought of as a professional occupation, requiring a college degree and offering advanced degrees. At the same time, it's treated much like a manufacturing job, with no one showing any interest in how it's done other than wanting assurance that modern standard methods are used, and wanting quality results on budget and on time.

Being a software manager is better in many ways than being a software engineer. Dilbert, as usual, pretty much sums it up:

Surely there are other ways to advance a career in software than becoming a manager like pointy-hair-boss above. There are indeed. Here is a detailed description of how to climb up the ladder of status in software. Bottom line: the farther away you are from customers and actually producing working code, the higher your status. It's kind of like being a field hand on a farm. Necessary work, but everyone would rather someone else did it.

To explain what you do to people who know nothing about software, it's hard to avoid using physical metaphors to explain things -- metaphors that are widely accepted but deeply misleading. And then it's easy to slip into  lying about the software, because it's invisible to anyone who doesn't look at the code -- in other words, practically everyone.

Partly because of the invisibility, the compensation system is totally nuts. The star players in sports get the best salaries -- everyone watches them do their thing and win! Makes sense. But in software, it's as though no one can see what happens on the field. And worse, they've never picked up a ball or bat themselves, and have no idea, visual or otherwise, how it all works. So the pay is worse.

That's part of why software is so awful. It's no secret. There have been over 50 years of attempts to make the process of building software work the way that managers imagine it should. The attempts continue to this day. They fail. Here's a bit of history on the subject.

Why keep on keeping on as a programmer? It's invisible. You are pressured to lie to get ahead. You're constrained by awful methods that are supposed to make things better but in fact make them worse. Far easier to slog through the ten seconds of guilt described by pointy-hair-boss above, and make the move.

Be Healthy And Don't Schedule Your Annual Wellness Visit

It came again. Of course it did. It always does. What came?

The polite command to get into my doctor for what used to be called a "physical" and now most often is called something like an "annual wellness visit." Why do I have to visit a doctor to "make sure you're up-to-date on all health checks and screenings?" Simple: we patients don't care about our own health and fail to track elementary things. So we need to be told and then have our arms twisted to submit to various forms of testing. It's important -- early detection saves lives!!

What's included?

Here's what was in my annual thingy, which is probably similar to the one you got:

Draw some blood. Ask how depressed you are. Check whether you're had all the covid and other shots you're supposed to get. Colon cancer is terrible -- make sure you've had the colonoscopy you're supposed to have!

It's interesting that I also sometimes get something like this from my medical insurance company. It costs them money -- why would they send me to the doctor? I guess they really care -- or maybe they think prevention will be cheaper than cure. That seems to be what everyone thinks.

Houston, we've got a problem

Let's step back. Why this emphasis on health checking? It wasn't a big deal decades ago. Has something happened?

You bet something has happened. In spite of all the things pushed by public medicine, it's getting worse instead of better. Here are some highlights.

U.S. life expectancy fell to 76.4 years in 2021, the lowest since 1996, erasing a quarter-century of progress. By the end of 2021, the life-expectancy gap between the U.S. and Germany had widened to 4.3 years, and between the U.S. and France to six years.

...

The U.S. 2020 death rate from overdoses of 277 per million compared unfavorably with ... Germany’s 19 per million, and France’s 7 per million.

...

Between 2000 and 2020, the rate of obesity in the U.S. has risen from 30.5% to 41.9%. It is now the highest of any developed country and a stunning 10 times the rate in the lowest, Japan.

This is awful! No wonder we're getting harangued about getting to the doctor's office -- we're getting sicker and dying younger! Better do something about it!

What good does this checking do?

Has anyone run a trial to see if this massive testing and prevention actually makes anyone live a longer, healthier life? You'd think it would -- after all, doing sensible things like preventing getting too cold i

n the winter seems like it's better than trying to fix frost-bitten fingers and toes. It turns that many such trials have been run! And there's even a meta-analysis of all the trials, involving over 182,000 patients! Let's see what was discovered:

(NNT = Number Needed to Treat, i.e., the number of people who need to be treated in order for a single person to benefit. For more see this and this.)

Here are the highlights:

This Cochrane Collaboration review compared health checks, defined as “screening general populations for more than one disease or risk factor in more than one organ system,” to no health checks in a general, non-geriatric adult population. The authors included 14 trials of 182,880 subjects, testing three outcomes primarily: overall mortality, cardiovascular mortality, and cancer-related mortality.

Nine trials found no difference in overall mortality (7.4% in the intervention group and 7.5% in the control groups) with no observed heterogeneity across studies. Subgroups included less than or greater than five year follow up, old versus recent trials, USA versus European cohorts, and others, with no differences noted.

Health checks also failed to improve cardiovascular-specific mortality in eight trials, at 3.8% in the intervention group and 3.7% in the control group, though heterogeneity for this comparison was substantial. This was attributed to varying definitions of cardiovascular disease (myocardial infarction, stroke, etc).

For cancer-specific mortality rates were 2.1% in both intervention and control groups with moderate heterogeneity.

For secondary outcomes no difference was found in morbidity, hospitalizations, disability, physician visits, specialist referrals, absence from work, or patient worry, however there was a slight increased sense of self-health among those assigned to checkups.

You would think that an ounce of prevention is worth a pound of cure. There's no doubt that preventing getting cold is better than recovering from frostbite, but apparently it's not the same for medical testing.

What's being checked?

Everyone's list is probably a bit different, but let's check out the one I got as a typical wellness list as shown above.

First on the list was cancer screenings.

• Breast cancer. Getting a mammography seems to make sense, since catching something early sounds like a good idea. Bottom line: no one has a longer life because of the tests and lots of people are hurt by them. See this.
  
• Colon cancer. It's a terrible disease. you don't want to get it. Colonoscopies are supposed to not only detect it early and even help prevent it. Good idea! Sorry, not a good idea. In spite of tens of billions a year spent on prevention, the only controlled trial ever conducted, a big multi-country one, clearly shows that colonoscopies do NOT help you live longer. See this for the facts.
  
• Cervical cancer. According to the National Cancer Institute, cervical cancer was only 0.7% of all new cancer cases and deaths in 2022. Most women who get it survive 5 years or more. It's a tiny fraction, for example of liver cancer. And the recommended test, for HPV, has huge numbers of false positives. Why is this even on the list?
  
• Lung cancer if you've been a smoker. There have been lots of trials, careful tests. People have tried safer methods of testing than X-rays.  They don't work. See this and this for the details.
  

Second was vaccinations.

• The one they mention is the "yearly flu shot." This may be the mostly highly promoted shot on the planet. Sadly, there is conclusive evidence from scientific studies, including the CDC, that flu shots do not help. The studies show that not only are flu shots useless, they are a bad idea for old people. See this for my review of the details.
  
• Here is a discussion of "efficacy" in vaccines and what it really means, with specific illustration of covid vaccines. Unless you go into NNT for benefits and harms, you won't understand vaccines.

Third amounts to a blood test.

• Blood pressure. Everybody seems to accept that so-called hypertension (a.k.a. high blood pressure) is a disease. A huge fraction of the population take pills to "control" it. Not only is hypertension NOT a disease, taking the pills more than doubles your chances of going blind. The more "routine" side effects of the pills can be awful.
  
• Diabetes. Type 2 diabetes is not a good thing. Often it can be controlled by losing weight. Drugs are often prescribed. The data shows that almost no one is helped and that a large number taking drugs are harmed.
  
• Cholesterol. High cholesterol in the blood is supposed to lead to heart attacks. Why not take pills to keep it down and live longer? Simple: random controlled trials conclusively show that not only doesn't reducing cholesterol help, the pills are actively harmful in serious ways. See this.
  

Bottom line: these things shouldn't be tested for or, worse, "cured."

Conclusion

If you've got a medical problem and need help, by all means contact your doctor! There may be preventative or screening things I haven't covered here that are valuable, particularly with inheritable conditions. But screening overall doesn't help as the study I quoted above shows, and the screening things I've covered here as examples are the most common, and the data shows they do more harm than good. Your health is ... owned by you -- own it! And get help when you need it.

Software Management and Business School

While I was student at Harvard College, I worked at several jobs writing production software. I also audited a course at Harvard Business School. By wild coincidence, one of the jobs I had was with a small company that was the subject of a new case study used in the HBS course I took. What I learned from the contrast between the reality of the software company and the fantasy of the case study was the seed from which grew my current understanding of the application of standard management techniques to software. Bottom line: HBS-taught management techniques result in software that is expensive and bad.

How HBS case studies are written

Early in my four years at Harvard College I worked part time at a software company in Central Square in Cambridge called CPSI, Computer Programming and Systems, Inc. It was a programming services company, doing jobs for clients. I knew the languages they needed. I remember little about it or what I did there now; I remember not being impressed or excited by the work; with the campus chaos in the spring of 1969 I dropped it. Nonetheless the experience led to things that strongly shaped my career.

Not long after working at the Cambridge software company, I walked across the river to audit a course at the Harvard Business School, HBS.

The course was pretty new, called “Management of Small Enterprises: Operating Problems and Strategies.” It was a typical giant HBS course run on case studies taken from real life. The students competed to “nail the case,” by finding what HBS said was the best answer. The course was pioneering in concentrating on small business, since mostly HBS cranked out students to fill the big consulting companies and public companies. I attended to learn about starting and running a company.

Doing some research now, I found that the engaging professor, Arch R. Dooley, was a Yale and HBS grad. He never actually worked for a company, but nonetheless became an esteemed professor of manufacturing management, and started the programs for entrepreneurs, which have remained a small part of what HBS offers.

Part way through the course, which was interesting because of the type-A students fighting and clawing each other, a case study about a small software company became the focus. By the end of the case, I realized that the case was about CPSI, the company I had worked for during the prior semester. While a couple things had been changed to anonymize it, the company was easy for me to recognize. Even so, the case got important things about the company, its software and the market for software wrong. The HBS students were triumphant when they arrived at Prof Dooley’s approved approach to managing the company, which had absolutely nothing to do with what the software actually did. This blew me away.

After I graduated and stayed around Cambridge, along with a fair number of my classmates, I discovered that some of the super-bright, verbal people I knew were hired by HBS to … write case studies! They were given a bunch of existing material and case studies, with descriptions of generic “right answers” and thrown at a company. They would quickly figure out what they could and write up a cogent, compelling study. Maybe this worked for companies that did physical things they could easily understand, but none of them had the slightest exposure to software, and of course their managers were equally clueless. The result was the case study of the company I had worked for, which everyone involved thought was good material, but was in fact deeply flawed.

This HBS case study was my first exposure to the massive moat separating software reality and entrepreneurship from management and academia. Dooley was convinced that, although he’d never written a line of code and couldn’t even see it or read it, he could teach about software. He was equally convinced that, even though he’d never worked at a company, much less an entrepreneurial one, he could teach others how to do it. To this day, his colleagues think the same, and all the students and those who hire them are equally convinced. And much of it is based on case studies written by talented Harvard English majors fresh out of school.

This is stupidity, but it’s so common there has to be an explanation. I believe it’s one of the widespread manifestations of our class system, which nearly everyone implicitly understands but few people discuss. While the classes evolve, there’s always a clear hierarchy. One of the long-standing aspects of class is that people who do stuff in the real world generally are lower class than those who direct, observe or think. Attending HBS is clearly a way to enter the directing class. Engineers are highly technical, but usually other people build what they design. Programmers, by contrast, are people who use tools to build things, the way a craftsman might build a house. This puts them levels below the people who command what is to be built, just like the people who are served a meal in a grand hall are levels above the people who serve them and the cooks in the kitchen who make the food. All the grand professors at HBS and their students are far above the minions who write code in terms of class. Grand as they are, it’s clearly beneath them to stoop to dirty themselves with details of how things are built. They hire people for that. The fact that this attitude results in inefficiency, chaos and often disaster somehow isn’t their problem.

There are good reasons to teach management

The whole idea of a school teaching "management" is that there is knowledge about managing a group effort that is independent of the details of the effort and the products and services being created. This is a reasonable thought and generally valid. Think about finance, for example. Nearly every organization -- even non-profits -- have revenue and expenses, assets and liabilities. Expenses naturally fall into multiple categories, including payroll and capital. Different industries have different patterns of these things and the details matter, but the common elements are large and important.

Other things that are taught in business school have substantial elements that are similar across different industries. All industries have people who need to be hired, directed, and organized. Even software fits into the pattern. Suppose you're a construction company and want to build a bridge. The overall requirements of the bridge first need to be determined (number of lanes, etc.), followed by overall and detailed architectural and engineering design, and finally building. Just like software, right? It makes common sense and is widely accepted.

Why standard management fails for software

The widely accepted common sense notion that building software is pretty much like building a bridge now has a track record of many decades of failure.

Software development, security and management is an ongoing disaster, with all attempts to fix it failing. If bridges failed anywhere close to the rate of software failures, there would be revolution. See this. See this also.

One big reason is that software is different from all those things that management techniques work with. Food is cooked quickly, and you can taste the results. You can see buildings in progress and most other things. Software, by contrast, is invisible. How can you manage something you can't see?

Not being able to see it is bad, and not being able to do it yourself compounds the problem. At least with cooking and building you have a little idea of how it's done. In many fields, the best managers were good at doing what they're managing. This excellent principle is rarely followed in software.

Because of the invisibility, managers demand huge piles of documentation about the stuff they can't see. Makes things worse.

Everyone involved makes big efforts to track software as it's being built. It's often the case that glowing reports are made of software up until days prior to its being done. And then disaster ensues.

Because of its invisibility, managers tend to understand how software works in terms of metaphors -- metaphors that are inappropriate and make things worse. They want to create a software factory. They think of software as being like a building, with an architect and the rest. Even the notion of "building" software, as though it were a passive thing, is wrong. It's actually extremely rare that software is "built" -- mostly software is changed.

Because they're trained in business management, they think their methods apply to everything, including software. They use standard project management and variants created specifically for software like Agile to keep things under control. Without realizing the damage they cause, they insist on optimizing for predictability, while nimble startups beat them by optimizing for speed.The difference is like the difference between building bridges in peacetime or war.

Conclusion

Things have changed little since my bright college friends were churning out case studies to teach generations of future MBA's at HBS. Software is still invisible to the vast majority of managers who are nonetheless confident in the certified management methods they use, endorsed by all the Experts and high-paid people at the top of our class system. The methods that work to enable small teams to produce top-quality software quickly and well aren't secret; they are simply beneath the dignity of the people who dominate the upper ranks of all large organizations, including Big Tech.

 

The Value of Colon Cancer Screening

The health experts are united in proclaiming the importance of preventative health in general, and regular colonoscopies in particular. Get one so you can avoid dying of colon cancer! As is sadly all-too-usual, the colon cancer early detection campaign is propaganda to cajole people into putting lots of money into the pockets of doctors and health systems -- with no provable benefit to the patients who do their best to patiently put up with the pointless nonsense.

Do people who get colonoscopies avoid getting colon cancer? Yes, the vast majority of tested people avoid it; but then the vast majority of untested people also avoid getting it. Does getting colonoscopies as recommended help you live longer? All the authorities say it does, but the recently published RCT (random controlled trial) with over 80,000 subjects -- the ONLY such gold-standard trial EVER conducted -- shows that colonoscopies do NOT help the people who get them live longer.

It's no wonder that the medical-industrial complex has united to discredit this trial that threatens their revenue stream. Experts are speaking out, waving their arms wildly and pronouncing with deep-voiced authority that the trial is misleading. I guess all the people and organizations drinking from the fire hose of tens of billions of dollars a year in testing fees have been too busy to conduct a trial of their own to demonstrate that what they do actually helps people.

Colon cancer is a big deal

There is no doubt that colon cancer is something to be avoided, if at all possible. It's the fourth leading cause of cancer deaths, behind breast, prostate and lung cancer. Lots of people are diagnosed with it and die from it, according to the National Cancer Institute:

Getting colon cancer isn't a death sentence -- look at the survival rate above -- but we would all like to avoid getting it.

An ever-increasing number of people are screened for this terrible disease. According to the National Cancer Institute:

This adds up to over 16 million colonoscopies in 2019! The cost? Good numbers are hard to find, but it's probably in the range of $30 to $60 Billion dollars a year for screening.

The Voice of the Experts

Here is the summary recommendation of the U.S. Preventative Services Task Force:

The Grade A is primarily because the screening has "substantial net benefit." (Bold in the original.)

This organization has plenty of prestigious company in making this recommendation. For example, they say:

There is a general consensus that average-risk adults aged 50 to 75 years should be screened. The American Academy of Family Physicians (AAFP),⁴² American College of Physicians (ACP),⁴³ American Cancer Society (ACS),⁴⁴ and the US Multi-Society Task Force (which includes the American College of Gastroenterology, American Gastroenterological Association, and American Society for Gastrointestinal Endoscopy)³¹ all recommend routine colorectal cancer screening in this age group, although specific recommended tests and frequency of screening may vary.

It's not just the big organizations. Doctors and health systems on the front lines pitch the same message. Mount Sinai hospital in New York City recently put up a strong pitch on the subject. They lead their long article with this:

Colonoscopy is one of those important, routine medical procedures that most people would rather avoid. But experts say the test is a highly effective tool for both preventing colorectal cancer and diagnosing it at an early stage.

Along with the American Cancer Society, they recommend that screening start at age 45.

The trouble is, when you read all this carefully, particularly the extensive review that led to the USPST report above, you find a complete lack of RCT's for the effectiveness of colonoscopies. Nothing but modeling and authoritative-sounding guessing.

Experts attack the skeptics

Every once in a while, someone pops up who says that colonoscopies don't do any good. Fortunately for concerned patients, the experts tend to jump on such baseless assertions and tear them apart with facts and sound reasoning. Most of the Mount Sinai article just mentioned is devoted to this. It says:

Those looking for an excuse to put off a colonoscopy might now point to a large study conducted in Europe and published in September 2022 in The New England Journal of Medicine (NEJM) that appeared to question the benefits of colonoscopies.

For some reason, the Mt Sinai article fails to give a link to the study in question, which is here. I guess they feel that readers don't need to waste their time, since Dr. Greenwald's  take-down is authoritative. It continues with this:

In this Q&A, Dr. Greenwald, Immediate Past-President of the American College of Gastroenterology, and Co-Chair of New York’s Citywide Colorectal Cancer Control Coalition (C5), discusses the recent study and why the value of colonoscopies remains unchanged.

Here's the man himself. He is a seriously authoritative-looking guy:

The NordiCC Study and the response of the Experts

The NordiCC study is the one described in the recent New England Journal of Medicine article that Dr. Greenwald tore apart in the Mt. Sinai article promoting colonoscopies. The study followed over 84,000 "presumptively healthy men and women 55 to 64 years of age drawn from population registries in Poland, Norway, Sweden, and the Netherlands between 2009 and 2014."

The article seems to support performing colonoscopies. Here's the conclusion of the Abstract at the beginning of the paper:

In this randomized trial, the risk of colorectal cancer at 10 years was lower among participants who were invited to undergo screening colonoscopy than among those who were assigned to no screening.

I guess Dr. Greenwald and others were concerned about the fact that the numbers weren't strong. The Mt. Sinai article said the study "appeared to question the value of colonoscopies." Dr. Greenwald is quoted by Mt Sinai as saying

This study, along with prior studies, shows that colonoscopy decreases your chances of getting and dying from colorectal cancer. Getting sick and dying from colorectal cancer—especially due to delayed screening—is real. Screening with colonoscopy saves lives.

Done!

The NordiCC Study

Why would multiple European national governments go the trouble and expense of such a massive trial if the value of colonoscopies had been conclusively proven? Hmmm. The answer is simple: colonoscopies are expensive and highly unpleasant, and there have been NO RCT's that show a causal relationship between getting them and avoiding getting and dying from colon cancer. So before taking on all the cost and trouble, those groups figured they'd better get some real evidence on the subject.

First, let's see what the expert from Mt Sinai drew from the study:

Most importantly, in the section of the study that analyzed people who actually had a colonoscopy, the risk of developing colorectal cancer decreased by 31 percent and the risk of dying from colorectal cancer decreased by 50 percent, which is huge.

Wow. Why would anyone be worried about the NordiCC study scaring people from getting a colonoscopy, when it has such a huge benefit -- and the doctor claims that in the US it is performed better, yielding even stronger preventative results!

When you read the NordiCC study itself, aided by understanding of the statistical tricks that are used to distort the results, a whole different message emerges. This is why the medical-industrial complex goes to great lengths to hide the truth.

In this case, the core trickery is a biased sub-group created from the group invited  to have a colonoscopy that they call an "adjusted per-protocol analysis." The other main tricks are widely used: reporting "efficacy" (relative risk); ignoring NNT (absolute risk); and endpoint selection (dying of cancer vs. dying of any cause).

Here is a brief summary of the real results of the trial by Dr's John Mandrola and Vinay Prasad. I have verified that this summary accurately reflects data in the NEJM paper (see Table 2 of the paper):

• Over 10 years of follow-up, an invitation to screening colonoscopy modestly reduced the risk of being diagnosed with colorectal cancer, but it did not significantly reduce the risk of dying from colorectal cancer. Survival from cancer was nearly identical in both groups.  And all-cause mortality was the same.

The specific numbers of the primary outcome:

• The chance of getting (diagnosed with) colorectal cancer in the invited group was 0.98% vs 1.2% in the usual care group. This represents an 18% reduction in relative terms, and an absolute risk reduction of 0.22% or 22 per 10,000.

• The chance of dying from colorectal cancer in the invited group was 0.28% vs 0.31% in the usual care group. This 10% reduction in relative terms amounted to a difference in 3 in 10,000 and did not reach statistical significance.

• In the invited group, 11.03% of patients died; in the usual care group, 11.04% of patients died.

Here is a key chart from the paper, showing the risk of dying from colon cancer. Note that the real data is a barely noticeable squiggle along the X axis at the bottom; most of the chart is a big blow-up of the bottom 1% of the Y axis.

The tiny difference (3 in 10,000) in dying of colon cancer was not statistically significant. And the most-important-of-all measure, all-cause death, was identical.

Those are the indisputable facts from the study itself.

The authors of the study did their best to fiddle with the results -- they declared themselves to be "disappointed" in the study results. They put their fiddling in the paper.

The most significant game they played involved the fact that 42% of the subjects who were invited to have a colonoscopy actually had one. They recalculated the results for just the people in the invited group who actually had the procedure, trying to get better results; in the study, they call this a "per-protocol analysis." This shows the desperation of the authors, since doing this violates all the randomness and invites a host of what statisticians call "confounding factors," exactly the thing that a RCT avoids by being random. Naturally, this improved the results. By a small amount.

The authors and all the establishment defenders of colonoscopies do the classic thing that people who want to promote a drug or procedure do to mislead patients: they focus on relative risk instead of absolute risk. Relative risk, often called "efficacy," makes you think that the procedure is terrifically effective. What do you think when you read that your risk of death is "decreased by 50 percent?" It's huge, right? See this post on vaccine efficacy for a detailed explanation. What most people care about is absolute risk, which is how likely it is that the feared thing will happen to you. This is sometimes called NNT (Number Needed to Treat), which is the number of people who have to undergo the procedure in order for just one of them to benefit.

The per-protocol analysis (the best case) showed the risk of dying from colon cancer to be 0.15% in the invited group and 0.30% in the control group. This is a 50% improvement in relative terms which sounds great!  But in real-life absolute terms, it's a different of 0.15%, which is 15 for each 10,000 people And again, the difference of dying from any cause between these two groups was effectively zero. Here is a detailed discussion of per-protocol fiddling and the results of a statistically sound approach, which reduces the statistical significance of the colonoscopy benefit to zero ... as shown in a table of the supplement of the NEJM study.

And then there are the harms. What can go wrong with a colonoscopy? The USPST, the government group quoted above that recently recommended that colonoscopies start earlier than before, at age 45, discusses them deep in the supporting material. They report: "Harms from screening colonoscopy have been reported in 67 observational studies (n = 27,746,669)." You have to read carefully, Here's what they report:

14.6 major bleeding events per 10,000 colonoscopies (95% CI, 9.4-19.9; 20 studies; n = 5,172,508) and 3.1 perforations per 10,000 colonoscopies (95% CI, 2.3-4.0; 26 studies; n = 5,272,600)

That is a high confidence result of harm based on millions of patients, vs. the nearly identical low-confidence results of benefit from the NordiCC study.

Conclusion

The authors of the NordiCC study wanted to find that colonoscopies are effective -- they say so! They did their best to slant the results and obscure the real results. Experts reporting on the study in the US cherry pick and criticize the slanted results and confidently proclaim that colonoscopies are essential to health and longevity, when the clear numbers in the published study of over 80,000 subjects show no such thing. This is yet another example of self-serving advice from a medical establishment that depends on trusting, gullible patients to keep the money rolling in. For more, read these posts about flu, diet, saturated fat, cholesterol and blood pressure.

Why do insurance companies jack up premiums to shell out big bucks for drugs and procedures that don't help?

 

Summary: Cryptocurrency and Blockchain

The first Bitcoin was mined early in 2009, practically yesterday! A myriad of other cryptocurrencies have followed in its wake, and the underlying Blockchain technology has been applied to many non-currency domains. It can seem as though Bitcoin came out of nowhere, particularly given the anonymity of its creator.

Looking back, it now seems obvious that the world-wide culture and infrastructure for video gaming had a great deal to do with the rapid acceptance and spread of the digital currency:

https://www.blackliszt.com/2022/05/cartoons-and-video-games-evolved-into-bitcoin-nfts.html

In the gaming world, there's always someone in charge. Bitcoin is a digital currency that operates without any one person or group in charge. How it accomplishes this through the use of miners that are incented to maintain the integrity of the system is a real innovation. Here is what is novel and technically new about Bitcoin:

https://www.blackliszt.com/2019/02/the-novel-idea-at-the-heart-of-bitcoin.html

I think most people would be surprised to realize that Bitcoin didn't invent the idea of a digital currency. Currencies in the video gaming world are digital, but perhaps more surprising to many is the fact that the vast majority of US dollars are completely digital, no less so than Bitcoin! This is explained in my most popular post on Forbes, over 80K views, explaining how the US dollar is already digital and how a central bank digital currency would add nothing:

https://www.blackliszt.com/2020/12/we-dont-need-fedcoin-we-already-have-a-national-digital-currency.html

The core technology of Bitcoin and other digital currencies is called blockchain. Here are the basics of blockchain.

https://www.blackliszt.com/2018/02/blockchain-101.html

The common impression is that blockchain is a wonderful new technology that can solve all sorts of problems involving distributed data. Sadly this is not the case. Blockchain is an excellent solution for the purpose for which it was invented, and not much else:

https://www.blackliszt.com/2017/11/blockchain-a-sailboat-without-sails.html

Here's an example of a real-world problem of exactly the kind blockchain is supposed to solve -- that is already solved well with existing technologies.:

https://www.blackliszt.com/2020/12/could-blockchain-help-fix-my-car-that-was-destroyed-by-a-tree-branch.html

Here’s how the WSJ says that blockchain will solve the problem of stock transfers and why it can’t, mostly because blockchain has nothing to do with the hard part of the problem:

https://www.blackliszt.com/2015/11/the-magic-of-block-chain.html

There has been an explosion of investment in blockchain. Here is an example of how hyped but irrelevant technology can become a big financial winner:

https://www.blackliszt.com/2016/06/how-blockchain-will-deliver-value.html

Microsoft and Intel spell out the problems of blockchain:

https://www.blackliszt.com/2019/11/microsoft-and-intel-detail-the-deep-seated-problems-with-blockchain.html

Here is the pattern of ignoring the hacks of cryptocurrency that “everyone says” can’t happen because of blockchain’s “immutable ledger”

https://www.blackliszt.com/2019/02/crypto-currency-hacks-and-losses-mount-while-supporters-remain-silent.html

Here is an explanation of how those hacks of the crypto-protected and immutable ledger happen:

https://www.blackliszt.com/2019/03/how-can-an-immutable-distributed-ledger-have-assets-lost-or-stolen.html

Ethereum’s “immutable ledger” was re-written by a hack that “couldn’t” happen:

https://www.blackliszt.com/2019/05/blockchain-has-been-unchained-and-unblocked-and-its-broken.html

Here are the basics of Smart Contracts, the ability to write programs that run inside the blockchain, made famous by Ethereum:

https://www.blackliszt.com/2019/03/are-blockchain-smart-contracts-smart-are-they-contracts.html

Facebook created a new Smart Contract language, which is now maintained and extended by Aptos Labs. There are serious issues with it.

https://www.blackliszt.com/2019/07/facebooks-libra-crypto-currency-introduces-a-brand-new-smart-contract-language.html

Blockchains are increasingly run on cloud computing services, a “private blockchain.” If you understand why Bitcoin was novel by running with no one in charge, you can see why having someone in charge makes no sense for blockchain, and why conventional DBMS systems are vastly superior:

https://www.blackliszt.com/2019/03/if-you-like-private-blockchain-you-should-also-like-living-in-a-tent-instead-of-a-house.html

How is crypto-currency better than apps like Venmo, CashApp and Zelle (hint: except for criminals, it's not):

https://www.blackliszt.com/2019/09/facebooks-libra-cryptocurrency-and-the-p2p-apps-venmo-and-cashapp.html

A well-publicized event at the time was the missing emails of an official at the IRS. Everyone thinks they know how email works. Email is indeed simple compared to crypto. But nearly everyone, including all relevant officials, clearly were clueless about how email actually works! So how is it going to work out for something that is much more technically deep like crypto/blockchain?

https://www.blackliszt.com/2015/11/what-e-mail-teaches-us-about-bitcoin-blockchain.html

This general ignorance of how crypto actually works was certainly behind the fact that so many "sophisticated" people fell for the scam of FTX run by SBF. They actually thought their "wallets" were theirs and under their control|

https://www.blackliszt.com/2022/12/the-phony-wallet-of-ftx-and-how-it-enabled-the-corruption-of-sbf.html

There has been a live roll-out in a group of countries of government versions of crypto-currency (CBDC, Central Bank Digital Currency). It hasn't worked out too well, and the people who tried to use it were hurt.

https://www.blackliszt.com/2022/03/dcash-government-cryptocurrency-shows-why-fedcoin-would-be-a-disaster.html

https://www.blackliszt.com/2022/03/what-is-behind-the-dcash-central-bank-digital-currency-disaster.html

https://www.blackliszt.com/2020/12/we-dont-need-fedcoin-we-already-have-a-national-digital-currency.html

Ransomware has exploded. In general, the criminal use of crypto has grown greatly. Crypto groupies go to great lengths to deny it:

https://www.blackliszt.com/2021/06/the-colonial-pipeline-cyber-security-disaster-in-context.html

https://www.blackliszt.com/2021/06/bitcoin-enables-criminals-to-thrive-while-promoters-deny-the-facts.html

https://www.blackliszt.com/2022/10/cryptocurrency-and-crime.html

There have been quite a number of crypto crashes, including some in which large numbers of people lost lots of money. Perhaps the most famous of these was the demise of crypto exchange FTX, with criminal charges being brought against its founder, SBF. Here is an explanation of how the "wallet" provided to FTX customers that held all their funds was constructed, and how that meant that crypto that customers thought they owned was actually owned by FTX. This deception had a major role in FTX's crash.

https://www.blackliszt.com/2022/12/the-phony-wallet-of-ftx-and-how-it-enabled-the-corruption-of-sbf.html

The wonderful "distributed immutable ledger" couldn't protect people from the fact that they didn't understand that the crypto held in their FTX "wallets" wasn't theirs, in any sense of the word!

For more detail on cryptocurrency, see the book authored by Oonagh McDonald on the subject. I was her technical advisor:

https://www.blackliszt.com/2022/01/cryptocurrency-money-trust-and-regulation-book.html

 

How to reduce the cost of medical imaging and PACS

Medical imaging devices like MRI's, CT and X-Ray machines are extremely valuable. They're also extremely expensive. So expensive, in fact, that health insurance companies typically require a pre-authorization for an MRI scan to make sure that it's "medically justified." The market is currently estimated at about $40 Billion a year, with more spent on proprietary PACS (Picture Archiving and Control Systems) for storing and managing the systems.

The medical imaging market is highly regulated, with the design and construction of the devices subject to detailed requirements for how the hardware and software should be designed and built. The result of the regulations is that a small number of large companies control the market, effectively preventing innovation and new companies from entering the market.

There is a proven path towards opening the market to innovation and dramatic cost reductions, while improving quality. We should break the iron grip of monopolistic companies and harmful government control to enable a medical imaging revolution.

The Software industry case

Something similar happened in the software industry. IBM mainframe computers and software once owned the world. Everyone bought from IBM, and were then required to buy IBM software and applications. They worked, but were incredibly expensive. A government anti-trust suit broke some of their monopolistic power, and new mini-computers changed the game. Then with computers built on multiple microprocessors, low cost, high quality and performance with intense competition ruled the roost in the computer industry. Separate companies built each part of the new world; each competed to be the best.

The crowning touch was that for important parts of the software such as operating systems, open source software emerged and became the norm. Even IBM acknowledged this by porting the Linux open source operating system to its IBM mainframe computers.

What should happen in medical imaging

Medical imaging machines are like specialized mainframe computers. In addition to the physical hardware that does the scanning, there are processors with operating systems and application software. Software controls each step of the scanning process, collects the data, stores it and displays it. Today, every bit of that computer hardware and software is built by the hardware supplier. Just like it was for IBM mainframes before the anti-trust suit.

The big difference is that no government agency exercised control over the details of how the IBM software was designed and built. Sadly, ignorant bureaucrats at the FDA exercise total control over this process, as I detail here. They require the use of methods that are so old and bad that even giant corporations have long-since moved on for their unregulated software.

The argument is that this is about your health. Do you want imaging devices that don't work or give bad results? The FDA performs the essential function of guaranteeing quality and safety, they say.

What they actually do is the equivalent of demanding that only hand saws be used for turning trees into lumber and refusing to allow nails or hammers to be used in house construction. Of course it can be done. But people using modern tools get far better results faster at lower cost. There is a simple way the FDA can assure quality, by shifting from lengthy HOW style regulations to simple WHAT style regulations as I explain here.

The Result

The result of this change will probably resemble what happened to IBM once their monopoly power was broken. IBM continues to this day to manufacture the successors of mainframe computers, now called the Z series. They support both their own operating system and a leading open source one. Applications that run on their systems are available from a wide variety of companies.

Similarly, major vendors such as GE and Siemens will continue to do what they do, but all of the hardware and software will be open to competition by both new and existing vendors, and possibly also by open source efforts. It's likely that Linux would be ported.

Image storage systems for imaging continue to cost many billions of dollars a year. They don't do much more than what you could with Dropbox or AWS S3 storage, for example. Each patient would have cloud folder that would hold all their records and images. The system would store each new file in the cloud, which would securely store it with full multi-site protection and backup. Sharing can be accomplished simply by creating and sending a link, something that can be done with a few lines of code or manually in seconds. The huge problem of medical imaging records storage and sharing that I demonstrated here would go away! Yes, you'd put some UI on top of the cloud storage to make it super-easy and not dependent on any one cloud storage vendor.

Conclusion

The essential and growing world of medical imaging and supporting systems form an indispensable part of modern medicine. It's long since time for them to catch up to the transformation of the computer industry four decades ago, dispense with harmful regulation and allow healthy competition to flourish. We would all benefit by the resulting increase in availability and dramatically lower costs. And yes, with better quality.

 

The Phony Wallet of FTX and How it Enabled the Corruption by SBF

As the world now knows, FTX was a corrupt mess. The misuse of customer funds by SBF and his cronies has been widely reported. What isn’t reported is the way that what are called customer “wallets” inside FTX and possibly other crypto exchanges typically work and how it facilitated the crime. Using the metaphor “wallet” to describe how customer funds are held gives a feeling of security that is completely unjustified by the technical implementation of customer accounts inside FTX. This technology practice enabled the mis-use of customer funds and made it effectively invisible to the outside world.

Was FTX an Exchange?

FTX described itself as an “Exchange,” specifically a “cryptocurrency exchange.” However, it was sadly lacking as an “exchange” as the term is used outside the world of cryptocurrency. Real exchanges like NYSE and NASDAQ do NOT support direct customer accounts – go to them and see if you can sign up – you can’t! Each stock you can buy and sell is listed on one exchange, and all the traders who interact with the stock must be members of that exchange. Among other things, this guarantees an open and transparent market; by listing shares for sale on an exchange, ALL the people who may want to buy or sell your shares go to that exchange. This guarantees the best price to both buyer and seller.

SBF, the FTX founder, knew all about how crypto exchanges break all the ethical rules of real-world exchanges. One of his early efforts was leveraging the price differences of crypto assets listed on multiple exchanges. He would buy a given crypto coin on one exchange, for example, and sell it for 30% more on a different exchange! Worked well for him! For the ignorant but trusting people buying and selling on those exchanges, not so much…

Who owns your cryptocurrency?

At the core of cryptocurrency is a unique data and transaction structure that has fueled its rapid rise in utilization and value. I’ll describe how BitcoinBTC works, since it’s the first and by far the largest cryptocurrency.

As we all know, banks don’t publish lists of their customers, and keep private the account values and transactions of those customers. What’s amazing about Bitcoin is that every single Bitcoin owner is publicly known by its “public key,” a string of numbers unique to the owner. Nothing else about the owner is public knowledge in the Bitcoin world. It’s as though everyone were invisible but owned a house with a normal address. You can visit the house, see it’s there, but only the owner has the key, called a “private key,” to go into the house. The house is built in such a way that it can’t be broken into – only the key affords entry.

Another fascinating thing about Bitcoin is that you can see automated vehicles  leaving the house with gold bars completely visible in the open trunk. There can be multiple groups of bars, each wrapped with a cord that gives the address of where it’s going. Each vehicle, instead of a license, has the address of the sender emblazoned on it. The vehicle may be used often or not appear for months. When it finishes making its one or multiple deliveries, it returns to its house.

Each house has a special place to receive deliveries from vehicles. When a vehicle shows up, it backs up the driveway to the garage and a special little door opens. The package of gold bars addressed to that house goes into the little door, which then closes and the vehicle drives off.

There is lots of room to build houses in Bitcoin world, but the roads don’t have much capacity and the speed limit is low. In addition, there is a network of vehicle schedulers, kind of like air traffic control towers at airports, who are in constant communication with each other. Just like when an airplane wants to take off it has to get permission from its control tower, a vehicle can’t be assigned to a house that wants to make a gold shipment without permission from traffic control. Traffic control takes account of congestion and makes sure there isn’t too much crowding on the roads.

The same way airlines pay fees that pay for the controllers, Bitcoin owners pay fees to the vehicle controllers that manage all the deliveries. The fees are the same whether there is just one or multiple packages in the delivery, and you can pay more money to get preferential treatment. In Bitcoin world the controllers who manage the deliveries and collect the fees are called “miners.” In addition to their central work, the miners collaborate and assure accuracy of a public record of all the vehicle deliveries that have been made. The record includes the address of the sender and the amounts of gold (Bitcoin) dropped off at each address. This record is usually called a blockchain. Literally anyone in the world can see it, but no one can change it. For this reason, it’s called an “immutable ledger” of all the transactions. Because all the transactions are public, it’s not hard to run through all the transactions of what’s been sent and received to figure out the exact total of Bitcoin that resides at each address. In fact, the miners do this all the time, to make sure that no one is being tricked by trying to send more Bitcoin than they actually own!

All this should make it clear that each owner of a house address (public key) in Bitcoin world is the owner of all the Bitcoin that has been sent to that house – and that only the owner who possesses the private key for that house can access the Bitcoin in it to send it out.

Technically, a Bitcoin owner can download and run software on their PC that manages their “house.” No one else need be involved. Even if the PC is lost or damaged, the owner can download the software on a new computer and, with the private key that’s been written on paper, access everything without loss – because the public blockchain holds the record of what you own!

The FTX exchange “wallet”

When customers signed up with FTX, they were assigned a “wallet,” owned by them but managed by FTX. Much easier than downloading and running software on your computer! The wallet was your account and securely held all your crypto assets, including Bitcoin. Supposedly. It seemed to be true: you could login to your FTX account and see your current assets and transactions. They’re yours! Safe and secure in your “wallet.”

Now let’s turn to the way Bitcoin (and the others) actually work. As we know, in order to truly own Bitcoin, you must have an address (public key) used for receiving Bitcoin. In order to send Bitcoin, you MUST have the private key that matches the public key address – otherwise, the controllers (miners) will refuse to process your request. But people with wallets/accounts at FTX didn’t have such private keys! All they had was a way to access their accounts. So where was the Bitcoin that the FTX software told you was in your wallet? The answer is simple and terrifying: FTX held all its Bitcoin in the FTX address (public key) with the private key that enabled its use under its exclusive and secret control!

Where did the amount that you supposedly owned and held at FTX come from? Simple: FTX kept two set of books: one for itself and one for its customers. When FTX showed you the amount of Bitcoin you “owned” in your “wallet,” it was just showing you the number from a second set of books it maintained internally for its customers.

In Bitcoin world, it’s as though there were a giant house that held huge amounts of Bitcoin. As FTX grew, many more of its customers would buy Bitcoin than would sell it. So FTX would go outside its walls, make big Bitcoin purchases for its corporate account; it would then update its customer-facing books to make it seem as though they really owned the Bitcoin. As the Bitcoin in FTX’s own account grew, it became increasingly able to satisfy customer trades just by making changes to its customer-facing books without anyone in the broader world of Bitcoin being involved. Unless someone got suspicious and carefully examined the blockchain for the transactions, no one would know that no such transactions in the real world of Bitcoin ever took place! In FTX world, FTX actually owned ALL OF its customers’ Bitcoin. It could do whatever it wanted with them with no one being the wiser!

Keeping Bitcoin transactions off the completely public Bitcoin blockchain was a nice additional source of revenue for FTX. No one knew that most trades did NOT get the best price, because FTX was highly incented to only do buys and sells among its customers. And FTX didn’t have to bother with miners, paying fees, waiting when there’s congestion or any of those pesky and expensive things.

None of this was truly secret! There have been multiple crashes of exchanges that have taken place, with huge losses. The losses were always when the customers didn’t actually own what they thought they owned – the exchange owned it. Without a personal miner and private key that only you know, the Bitcoin isn’t yours!

Conclusion

The FTX “wallet” was a mirage, a sales tool to make customers feel comfortable that they actually owned what FTX told them they owned. Even though there is a public record of all Bitcoin ownership and transactions on the public blockchain, FTX customers went with the flow, lulled by all the publicity and famous people. The two sets of books were there from the start, and the absence of transactions on the public blockchain was visible for anyone who cared to see. For all the supposed computer literacy and general savvy in the crypto world, this scam wasn’t secret like Bernie Madoff’s was, who kept the real books a big secret. What’s amazing with FTX is that the scam was operating in the full light of day – but in a large bunch of computer data that is completely accessible to the public but ignored by “savvy” investors and public regulators alike. What’s needed to prevent this kind of thing from happening is not new laws; it’s savvy, paranoid, nerdy public snoops who care about details and know what they’re doing. Not to mention more people heeding the advice: “not your keys, not your coins.”

Note: this was originally published at Forbes.

Flu Shots Propaganda and Reality

The drumbeat happens every year, echoing far and wide: It's flu season! Protect yourself and others -- get a flu shot right away! Roughly half of the population heeds the message and gets the shot.

The reality of the flu and flu vaccines can be found, but it's elusive. Once you find out what's really going on, you can't help but wonder why public authorities and provider groups keep pounding everyone to take these shots.

What we're told about the flu and flu shots

I got an email from my primary care provider. Here was the attention-grabber:

And here was the lead paragraph:

They go on to give lots of detail in an FAQ.

It's pretty clear: get a flu shot and you won't get the flu. It's not just my medical group; they all seem to say it, along with major pharmacies like CVS. The FDA is equally clear: "A flu vaccine can be given to anyone who wants to avoid the flu (persons over 6 months of age."  The CDC pushes it hard:

The message is similar to "shovel the snow off your snowy walk, and you won't slip and fall." 

The flu is awful!

The CDC makes repeated strong claims about how awful flu is and how flu shots protect you from getting the flu. Surely those claims are backed by solid data, gathered by some of the more than 10,000 employees of that organization.

First, the CDC collects and presents highly detailed data about what they call the "burden" of flu, by which they mean the number of people made ill, hospitalized or killed by it. While the numbers vary from year to year, the totals are massive:

For example, for the last flu season entirely unaffected by Covid, here are the annual numbers from the CDC:

The overall burden of influenza (flu) for the 2018-2019 season was an estimated 29 million flu illnesses, 13 million flu-related medical visits, 380,000 flu-related hospitalizations, and 28,000 flu deaths (Table 1)

Huge, right? It makes sense that everyone should be protected against these consequences of the terrible disease of flu.

Let's dig a little deeper into those catastrophically bad outcomes. Again, we'll use the CDC's own data.

First, let's look at the flu disease burden by age. The CDC, quite sensibly, presents these next numbers as a rate per 100,000, so that you can get how likely the outcome is to actually affect you.

For little kids, age 4 and under, the rate of catching the flu is a whopping 15,238 per 100,000 -- about 15% of little kids got the flu! For people aged 5 to 64, it was in the range 7- 12,000 for most older kids and adults; less for those 65 and older. Lots of people get the flu, around 10%.

With so many people catching the flu, it makes sense that the flu shot is pushed by health authorities. It's not rare, it's widespread! The numbers back it up.

What about all those hospitalizations and deaths?

While getting the flu, going to the doctor and even getting hospitalized aren't great, the thing you really want to avoid is death.

So what happens after you get the flu? For the vast, vast majority, not much. For young kids and older adults, only 0.1% end up in the hospital, and for most it's less than half of that. Death? The death rate for adults 18 to 49 is 1.2 per 100,000. That's a rate of 0.001%. The rate for younger people is even lower.

How the the flu rank among the other causes of death? The CDC has the numbers.

Flu ranks number nine out of the top ten, about the same as kidney disease. Things like accidents, strokes and the ever-present heart disease and cancer are way more dangerous. And the chart says "flu and pneumonia," without getting specific. Chances are it's mostly pneumonia. From another CDC data table, I find that the rate of death due to suicide in 2019 was 14.5 per 100,000. So the chances of dying from the flu or pneumonia were about the same as killing yourself!

But wait! Something's wrong! The deaths per 100,000 are listed as about 13, while we learned from the CDC (table 2 above) that the flu deaths for kids and adults per 100,000 were only about 1 -- truly tiny!

Putting the death rate for kids and most adults into the chart above, the rate of dying of the flu is less than one tenth that of the least likely of the top ten, way under 1% of dying from one of the top two causes. For kids and most adults, dying of the flu is not even close to being a leading cause of death.

Old people and the flu

Let's read the stats about flu burden by age (table 2 above) more carefully. Go down to the bottom and look at the numbers for older people, 65 and over. Only 4% of older people get the flu (way less than other ages), but for those who do, the rates of hospitalization and death are dramatically higher. Roughly 10% of those who get the flu end up in the hospital and 10% of the hospitalizations result in death! That's a mortality rate of 40 per 100,000. So it's the old people dying of flu (and pneumonia) that puts flu into the top ten causes of death! Without the old people, not only would the flu death rate not make the top ten causes of death, it would be less than 10% of the bottom cause, kidney disease! And way lower than other things such as accidents and suicide.

Let's understand the death rate for old people. Yes, it's forty times larger than the rate for most younger people. Scary, right? But running out the numbers, 0.4% of old people are hospitalized due to flu, and 0.04% of old people die of the flu. That's according to the CDC. Here's a chart from the CDC's annual report (a PDF file) of the leading causes of death for older people:

See flu and pneumonia there on the left? Pretty low on the list of things for older people to be worried about.

The charts for younger people are also interesting. For ages 10 to 24, accidents, suicide and homicide cause more than two thirds of deaths, while flu is less than a percent. For adults aged 25 to 44, flu is one percent, while for ages 45 to 64, flu doesn't even make the chart.

What about the flu shots?

The CDC and the rest of the health establishment may flash warning lights with loud warning sounds about the flu and how we need protection from it, but we now know that the numbers aren't compelling. But getting a flu shot isn't a big deal. shouldn't we get it anyway so that we're protected against the flu?

Flu shots: the CDC

First, let's look at the CDC's own numbers. They went to the trouble of assembling patient panels so they could get good numbers. The numbers have increased over the years; the recent 2018-19 panels are over 10,000 people. Here is the chart from the 2018-19 panel for effectiveness of flu shot against by age:

This is tough to make sense of at a glance. First, look at the top line, for all ages. 2763 people got the flu -- and almost half of them (48%) were vaxed! Meanwhile, 7249 people were flu-free, with a bit over half (56%) vaxed. Clearly, the vax didn't make much difference.

The numbers for older people are interesting. As we saw above, people 65 and older are the least likely to get the flu but the most likely to be hospitalized or die from it. The table shows that the older were far more likely than the rest to be vaxed, over 80%. But the number vaxed was nearly identical in the groups with and without the flu. Let's dig in.

The last column, CI (confidence interval) will be mysterious to most people. The CDC has an explanation in its section on vaccine effectiveness  It's standard statistics. They give the effectiveness number based on a tiny fraction of the number of people who could get the flu. How likely is it to be the true value? The CI is the range of values it would be, with 95% statistical confidence. See the negative numbers in the far right for teens and adults age 50 and older? According to the CDC (and statistics in general) that means:

...if a confidence interval crosses zero, for example, (-20% to 60%), then the point value estimate of VE provided is considered “not statistically significant.”

Not only was the VE tiny, the number is effectively meaningless -- the flu vax could increase your chances of getting it, or decrease your chances. The CDC's main flu vax effectiveness ongoing study can't tell the difference! Which means ... getting the flu shot has no impact on whether you get the flu.

Flu shots: published controlled trials

There have been dozens of published controlled trials of flu vaccines in multiple countries. Reviews of the many trials have been conducted and published. This is an updated version published in 2018 of the prior review.

We included 52 clinical trials of over 80,000 people assessing the safety and effectiveness of influenza vaccines.

The effectiveness they found:

Inactivated influenza vaccines probably reduce influenza in healthy adults from 2.3% without vacination to 0.9% (risk ratio (RR) 0.41

"Risk ratio" is what is called the "efficacy" of the vaccine, normally reported as a percent; in this case it would be 41%. Most people when they read a number like that think it means they've reduced their chances of getting the flu by about 40%. Not so. Read the quote again: "2.3% of the unvaccinated got the flu, while 0.9% did get it." This means that most people didn't get the flu, whether vaxed or not. The study translates this into NNV (number to treat NNT, which is NNV for vaccines -- see this for an explanation)

71 healthy adults need to be vaccinated to prevent one of them experiencing influenza,

What about other consequences?

Vaccination may lead to a small reduction in the risk of hospitalization in healthy adults, from 14.7% to 14.1%

What do they conclude?

Healthy adults who receive inactivated parenteral influenza vaccine rather than no vaccine probably experience less influenza, from just over 2% to just under 1% (moderate-certainty evidence). They also probably experience less ILI following vaccination, but the degree of benefit when expressed in absolute terms varied across different settings. Variation in protection against ILI may be due in part to inconsistent symptom classification. Certainty of evidence for the small reductions in hospitalizations and time off work is low. Protection against influenza and ILI in mothers and newborns was smaller than the effects seen in other populations considered in this review. Vaccines increase the risk of a number of adverse events, including a small increase in fever, but rates of nausea and vomiting are uncertain. The protective effect of vaccination in pregnant women and newborns is also very modest. We did not find any evidence of an association between influenza vaccination and serious adverse events in the comparative studies considered in this review. 

Makes you want to run right out and get the shot, doesn't it?

Flu shots: Huge UK study on older people

There was a massive study in the UK on the extent to which getting flu shots helps older people. This is the background they give:

Observational studies using traditional research designs suggest that influenza vaccination reduces hospitalizations and mortality among elderly persons. Accordingly, health authorities in some countries prioritize vaccination of this population. Nevertheless, questions remain about this policy's effectiveness given the potential for bias and confounding in observational data.

They studied adults aged 55 to 75 in England and Wales during 2000 to 2014. Here is what they found:

The data included 170 million episodes of care and 7.6 million deaths. Turning 65 was associated with a statistically and clinically significant increase in rate of seasonal influenza vaccination. However, no evidence indicated that vaccination reduced hospitalizations or mortality among elderly persons. The estimates were precise enough to rule out results from many previous studies.

Flu shots: are they harmless?

The data says clearly that flu shots don't do much good. But it's no big deal to get one, it might help, so why not? It turns out there are good reasons to avoid getting the flu vaccine, based on how vaccines work and supported by data. The CDC even explains some it, buried away.

Let's start with a little-discussed fact: vaccines do not in themselves prevent anything. Everyone has an immune system. Immune systems are amazing things. They fight against a huge variety of invaders to the body, defeating the vast majority of them most of the time. What vaccines do is "trick" the immune system into strengthening its defenses against a particular invader by sending things that look like the real attackers, except they've been neutered. Here's the issue: immune systems don't have endless defensive "weapons," so increasing defenses against one thing means decreasing them for others. Also the defenders aren't endlessly energetic. If they've been mobilized against an attack that doesn't occur, they don't have as much energy for the next mobilization.

This means that if you trick the immune system into shifting weapons for a threat and a different one occurs, the defense against the real attackers isn't as strong. This happens every year with flu vaccines as the variants change and particularly older peoples' immune strength fades. It also happens with covid.

Here is a summary of the most important studies.

The CDC has claimed that influenza vaccines do confer this benefit to older people. However, to support that claim, the CDC relies on observational studies that have been discredited by the scientific community.

In fact, as observed in a 2005 study in Archives of Internal Medicine (which is now JAMA Internal Medicine), despite a considerable increase in vaccination coverage among the elderly between 1980 and 2001, pneumonia and influenza mortality rates actually rose substantially. A 2008 review in Virology Journal similarly observed that “influenza mortality and hospitalization rates for older Americans significantly increased in the 80’s and 90’s, during the same time that influenza vaccination rates for elderly Americans dramatically increased.”

The effect of influenza vaccines on influenza mortality—much less their effect on all-cause mortality—was never studied in clinical trials, and later studies showed that the studies relied upon by the CDC to support its claim were fatally flawed due to a selection bias known as “healthy user bias”. In short, it wasn’t that elderly people who got a flu shot were less likely to die that flu season but that elderly people who were so frail that they were likely to soon die were less likely to get a flu shot. As a 2006 article in the International Journal of Epidemiology observed, the magnitude of this demonstrated selection bias “was sufficient to account entirely for the associations observed”.

In other words, there is no good evidence to support the claim of a mortality benefit of influenza vaccination for elderly people.

Unlike some vaccines, flu vaccines don't seem to do much active harm to those who get them. However, they do little to no good, and getting them repeatedly weakens your immune system, making you more likely to get the disease you are supposedly protected from. The fact that as more elderly people get the flu shot the more of them die of the flu is unsettling, to say the least. Doesn't sound like a boat I'd like to jump on to.

Conclusion

Why does the entire health establishment keep banging away at promoting flu shots? What is it exactly that causes the medical establishment to heavily promote treatments that not only don't help but can seriously hurt you? Things like diet, cholesterol, blood pressure and flu? If your goal is to promote trust in government and medical health advice, this is the last thing you should be doing.

New York's Met Opera is Crippled by a Cyber Attack

New York City's Metropolitan Opera has been the victim of a cyber attack that has shut down its website, box office and call center. As of this writing, performances continue while restoration work continues.

The Met Opera

The Met Opera is an incredible organization. Here it is on its Twitter page:

As they describe themselves on Twitter, they are "The world's greatest singers on the world's greatest stage."

I live a short walk away from this wonderful place and have enjoyed many amazing performances there over the years.

I've got tickets to see the Saturday matinee of their new production, The Hours. Something came up on Thursday that led me to go online to see if I could change to another date. This is what I saw:

Wow. Now it's Friday, and they've been down since Tuesday. It's a big deal:

No small loss, as the company handles about $200,000 worth of tickets each day.

Aida and The Hours, two major operas that are always popular with audiences, went on stage with an unusually empty theater, not even last-minute sales at deeply discounted prices helped save the day.

They are really scrambling. The Met Opera made an important announcement on Twitter:

And then said:

Just $50 tickets -- cheap for the Met -- and they're doing it for Dec 14, while this tweet was made on Dec 9. Their systems went down on Dec 6. So they're not planning on a resolution soon. And they have to use the symphony's box office instead of their own.

Computer Security and Cyber Attacks

The Met has "called in" the FBI, but that's a move that has yet to help any cyber attack victims. Here's why.

The Met has said nothing about the nature of the cyber attack. However, the fact that it's been days makes me think that it's likely to be a ransomware attack or something like it, because normally you'd just have to restore all your computers from backup or employ your disaster recovery protocol, which would normally take hours, not many days.

Ransomware is a relatively new kind of attack in which the hackers encrypt all your computers and won't give you the key to unencrypt them unless you pay a ransom, usually in terms of untraceable Bitcoin. Ransomware is amazingly widespread, hitting dozens of hospitals, corporations and government agencies many times a day. It's gone way beyond a crime wave -- it's a crime hurricane, and shows no signs of stopping.

It's likely that the Met followed all the regulations and expert advice for running their computers and keeping them secure. The trouble is, the regulations are ineffective and the experts don't know what they're doing -- they can't even keep their own systems secure, as has been famously illustrated many times over. Here is an understandable explanation of the issues. There are effective methods of cybersecurity, It's just that all the experts ignore them. In the middle of this post is a description of how smart people can recover from a successful attack in under an hour, if they've set things up correctly -- not unlike the way smart people have electric power backup generators for their homes.

Conclusion

I really hope the Met Opera recovers soon. Meanwhile, the cyber attack hurricane continues to intensify, largely staying out of the news, with the regulators and experts deepening their cluelessness as they press for larger budgets.

My question is simple: When will customers scream loud enough to make the complacent, computer-ignorant grandees in charge of the attacked organizations stop their support of computer executives who say impressive-sounding things while they continue to march to the suicidal orders of experts and regulators whose commands have been overwhelmingly proven to amount to holding up giant "welcome criminals, this is a defense-free zone" signs while spending ever-growing money to harm the organizations that pay them? WHEN??

Revolutionize health by making medical data and studies open source

Medical studies are essential to knowing what works and what doesn't work in medicine. There are a few problems, though. There aren't nearly enough studies, they are expensive and cumbersome, the funding is often by groups seeking an outcome, there isn't enough follow-up, most of the data is secret and they are rarely crafted for personalization. Among other things. What can we do?

Often the cure for a problem isn't isolated genius, but finding a field that had a similar problem that got solved and adapting the solution. I propose that the problem of building software (expensive, cumbersome, takes too long, etc.) is similar to that of medical studies, and the solution of making software open source can be adapted to the problem of medical studies. If medical research and data were open source, most of the problems I listed could be solved.

Open Source Software

The open source software movement has revolutionized the industry. Operating system software, for example, was the proprietary crown jewel of computer manufacturers. IBM's 360 mainframe operating system software, for example, took over 1,000 people years to build. A well-known book by one of its leaders, Fred Brooks' "The Mythical Man-Month" went into detail explaining the nightmare.

There's been a revolution since then. The Linux operating system completely dominates the operating system market; for example, it runs on over 95% of the top million web servers. This isn't new news -- Linux was started over 30 years ago! Since then, even major profit-making software companies such as Google (Android, Chrome, Kubernetes) and Facebook (React) sometimes open-source valuable software they've built internally.

Much (not all) open source software is built by volunteers and the resulting software is freely available. Sometimes company employees work on open source that is valuable to their employers. There are hybrid models such as Red Hat, which charge for services they offer to companies that want to use the open source software. After the early years of resistance and skepticism by traditional programmers and managers, open source software is broadly accepted as a fact of life -- and a good fact! -- in the software world.

Open Source Medical Data

The data from a research study is incredibly important to the people whose disease or condition is studied, to the medical professionals who treat it and to the device or pharma company that creates the new device or drug. The results of the study cause the patients (guided by medical providers) to take drugs, change their behavior or undergo procedures that can have a major impact on their lives. Shouldn't that data be freely available to anyone who cares to study it? Just as open source software hugely benefits by having large numbers of volunteers pore over the code looking for errors, limitations and omissions, so would open source medical test data benefit by having large numbers of people who are even more motivated than software contributors comb through the data -- in software, we're talking about annoying bugs, while in medical data we're talking about life and death.

Anyone with software experience knows that no amount of software testing in a lab environment can match what happens to the software when it's widely distributed. When things go wrong with open source software in the field, open source contributors have a real-life test case of error and have a reasonable shot of finding and fixing the problem, contributing their fix to the central source code. With thousands upon thousands of copies of the software working all over the world and motivated engineers responding to issues and pooling their solutions, open source software achieves a quality that can't be matched by dedicated groups of employees working for a company. Much less a government agency.

The equivalent of this for medical testing is to start with opening all the test data to volunteer analyzers, withholding nothing. Releasing all the data that is now kept secret would be a big step forward.

But that's the equivalent of lab testing.The huge value in open source data will come from extending it to more people than were included in the study, and to include much more data about them, both before the formal start of the study to continuous aggregation of data over time. Among other things, this will enable surfacing factors that weren't considered by the original study designers, both from patient history and from medical events that take place after the formal end of the study. For example, this kind of extended data could surface the facts about the relationship between blood pressure pills and going blind, as I describe here and here.

Open Source Medical Studies

There is no reason why paid medical researchers couldn't continue to define and run medical studies in much the same way as they do today, much the same way as for-profit tech companies create software that they then open source. However, they would have to make 100% of their data open source and fully available anyone to investigate.

The "open source" version would be first to expand the selected participants in the study far beyond what would normally be done with volunteers, and second to extend the data collected to everything that is knowable about the participants, both before the start of the study and continuing long after what would normally be its conclusion. I don't claim to know how best to accomplish this, but I know that today the cost of running study sites, qualifying participants and so on is high. A way would have to be found to enable participants to volunteer remotely, and to enable local volunteers to perform whatever actions like drug injection that have to be performed locally and physically.

This process really kicks in when the new drug or procedure gets past the test environment and becomes more widely deployed. It would be good to emulate the open source software practice of having a careful staged roll-out of a new release instead of the current medical practice of unlimited distribution after approval. This would enable reports from the field, enhancing the open source data, to surface problems that weren't clear in the earlier, more limited testing of the new drug or procedure.

Once the distribution gets very broad, there still needs to be a way to surface and report issues. For example, here is a message from Google to enable broad data reporting about one of their products:

Why shouldn't such permission be added to patient medical records, so that as those records are updated for any reason, the updates are added to any relevant open source data collections? This would make longitudinal tracking automatic and painless to everyone involved.

Conclusion

Medical studies and associated data strongly resemble the proprietary operating systems of computer vendors in the 1960's and 70's. Each body of code was created at great expense by employees of the companies. The code (like medical data) was considered a trade secret, never to be revealed to an outsider. Problems usually surfaced after the code was shipped, just as many problems with approved drugs only surface after they are distributed. Manufacturers kept spending more time and money to make their software bug-free in the lab before shipment, but never got it right -- just as drug makers jump through endless FDA hoops prior to approval, and there are still problems. Makers of proprietary software have huge quality problems to this day, as I have documented, which the "free" open source software largely avoids.

Applying open source software concepts to medical drug and procedure testing and tracking could greatly enhance the safety and effectiveness of augmenting the toolkit available to patients who have medical issues. As it became understood and widely used, patients would have reason to have confidence and trust in the medical profession far beyond what many of them have today. Instead of being constantly hammered about how some drug is "safe and effective," which kinda tells many patients that it probably isn't, the open source method would create a level of transparency and openness that would let people draw their own conclusions.

I have been thinking of this issue for a long time; a discussion with Jonathan Bush at the recent HLTH conference inspired me to write it up.

 

Twitter can improve software quality by losing most of its engineers

Twitter has fired boatloads of software engineers and more are jumping ship. Most people predict that losing all those engineers will lead to software disaster. But then, most people don’t know much about software and don’t realize what a disaster Twitter software has been for years. With some intelligent leadership, Twitter's software could dramatically improve while dispensing with 90% of their engineers.

Twitter Customer Losses

First, let's note that the situation has users really worried. Customers are bailing the ship:

Here's some up-to-date analysis:

While some say the massive customer losses are due to the new CEO, others are sure that the loss of so many excellent Twitter software engineers will seal the deal.

Twitter Software Quality

Those cool internet software companies – their top people wear hoodies or whatever they feel like. They must be fabulous programmers, right?

Nope. Facebook, for example, produces amazingly bad software. See this and the included links for details on how bad it is – and how they hide as much as possible from their users.

Facebook, Twitter and the rest aren’t better than “normal” software companies, except in how rich they made their founders. Internet software is a horror show, as I document here.

I dove into Twitter in particular years ago to see just how bad it was. I found and documented inexcusable failures, which they went to great lengths to disguise.

I tested doing searches for tweets with Twitter’s own search engine, and documented random patterns of tweets being dropped out or included from searches just days apart. See this and this.

What this means, among other things, that the basis of “trending on Twitter” is based on bad Twitter data – the kind produced by those wonderful, oh-so-cool Twitter engineers.

Twitter hides the awfulness of its terrible software from you

Note that the errors I documented were NOT associated with an error message. They were cases where the right answer (for example search results) could be determined and compared to what Twitter provided. That’s what I did, and found that Twitter blithely would state “here’s your answer,” and then provide an answer that was demonstrably false. It would have been more honest had they given an error message instead. But no -- that would have been honest; Twitter engineers, while terrible at building software that actually works, have become masters of the simpler, nefarious job of masking bad software.

Even with such errors, Twitter would still admit failure from time to time, showing this image:

Since then Twitter has gotten better at hiding their errors. Following the pattern of bad search results, they give you your feed of tweets and you merrily scroll down, reading, retweeting, liking, and whatever. They have learned to almost never admit failure. The fail whale shown above is history. How do you know if you’re getting all the tweets from the people you follow, and in the right order? You don’t! In fact, there is loads of anecdotal evidence that part of how their engineers spend their time is figuring out how to manipulate your feed for a variety of reasons -- including masking the results of their inability to do it the right way -- instead of just giving you a common-sense, complete ordered feed.

Fail whale? I haven’t seen it in years. They can have dozens of servers crash and burn and no one will know the difference. What Twitter does is just pull together your search results or feed based on whatever servers are still limping along with whatever the pathetic Twitter software will give them, and give you whatever crap it has. How will you ever know it’s incomplete, missing important things, etc.? Unless you’re an obsessive crazy person like me and run tests, you’ll never know.

So what do all those thousands of software engineers do all day long except attend meetings while relaxing on comfy bean bags and digesting all the free food the company provides? Obviously not much in the way of useful programming.

The alternative

Small teams of smart, motivated programmers typically out-perform teams of hundreds and thousands of people with titles of “software” employed in big bureaucracies – including in “cool” companies. The concept is simple: most programming organizations are like people who build bridges in times of peace. They take thousands of people years to build. When you build bridges in war, like the bridge over the Rhine in World War 2, you have to build it in a DAY, while under enemy fire. And it has to work. Small, under-resourced groups of programmers operate in wartime mode, getting things DONE while groups of thousands continue plodding away at requirements planning. See this. For more, see this.

I see this all the time in my work of evaluating small, innovative start-ups. The small companies don’t have enough time or money to do things the “right” way. They have to get things done. Fast! The ones that do succeed.

A great example from another field is the author of the first important dictionary of the English language, Samuel Johnson. He produced excellent work, arguably 200 times more productively than the big committee trying to do the same for the French language. See this for the story.

What Twitter does isn’t hard! And what it does hasn’t fundamentally changed for at least a decade! Those thousands of engineers couldn’t make it work well nine years ago when I ran tests, and all they’ve managed since then is do a more polished job of hiding the errors and bad answers.

Whether Elon Musk or someone like him is in charge, the best thing for Twitter and its customers would be for most of the programmers to be shown the door, for the remainder to get with the program of making things actually work and work well, or join their former colleagues as ex-employees. If they really get it going, some programmers who are actually good and, get ready, WANT to write code, GOOD code, may seriously consider joining a re-invented Twitter engineering group.

It might actually happen, since Mr. Musk is violating the most sacrosanct rule of management: getting involved with the people on the front lines of actually doing the work! There's even evidence that he seems proud of his disgusting behavior.

Who does he think he is??!!

How to Cure AMD Macular Degeneration

AMD is the leading cause of visual impairment in the US. It hits many people as they age and causes vision to worsen to the point of blindness.

First the bad news: all I have is some evidence and common-sense logic that there MAY be a cure. Now the good news: there is a never-refuted major study that shows that people who take pills to reduce their blood pressure get AMD at more than twice the rate of those who don't, and there is anecdotal evidence (needs more study to confirm) that stopping taking the pills causes the AMD to reverse (not stop or slow down -- reverse) its progression. Do you think that's worth digging into? And at least letting everyone who takes BP medications that there is a good chance that at least 10% of them will needlessly go blind with AMD?

Blood Pressure Pills

All the medical authorities are united in the importance of fighting the “silent killer” of blood pressure that’s too high, i.e., hypertension. I’ve described in detail that what doctors call ‘essential hypertension” is NOT a disease.

Pills to lower blood pressure are the mostly widely prescribed pills in the US, with over 100 million people supposedly cursed by the “disease” of hypertension.

I started taking them eight years ago as a small part of my fight against a cancer that I had.  Last year I started experiencing symptoms that could have been evidence for heart problems. I had extensive testing and did research on my own. My heart was good but I discovered that my symptoms were side effects of BP pills, widely reported by people but ignored by doctors. I stopped taking the pills and my symptoms faded away to nothing. I tell the story here.

Blood Pressure Pills and AMD

Then I wondered if the pills could possibly have something to do with my AMD that was first diagnosed about three years ago. I studied hard and came up with nothing. Big authorities said that having "uncontrolled" hypertension could cause it along with "bad" diet. In a story I tell here, I discovered here’s a never-refuted medical study published by the American Academy of Ophthalmology and sponsored by the National Eye Institute (part of NIH) showing that taking those pills greatly increases the risk of going blind.

The study remains on-line but is shockingly difficult to find. I found only one eye doctor group that mentions it. You would kinda think that they would recommend getting off of BP pills, don't you think?

Blood Pressure Pills and AMD

I had detailed pictures of my macula clearly showing the drusen that are the things in the eye that hurt vision in AMD. I just went to the same doctor, who took another set of careful pictures so we could compare them to the ones from a year ago, while I was still taking BP meds. I stopped taking the pills a bit more than six months ago.

Below are pictures my doctor took of a section of the left eye. (You may have to click the picture to see it all, particularly the wavey drusen on the right.) The top picture is from a year ago and the bottom a couple days ago. The drusen are the wavey parts of the two white curves on the right.

As she said, the central drusen got larger but the ones in the periphery diminished significantly.

Generally speaking, drusen either stay the same size or they grow. Mostly grow. That's why AMD is progressive and no one has found a way to make it stop or slow down, much less reverse course. Here's an example of drusen shrinking. Why?

If you smoke, your chances of getting lung cancer are high. If you stop smoking, the chance of getting cancer goes down. If you're an alcoholic, your chances of terrible liver damage go up the longer you drink. If you stop drinking, your liver usually stops getting worse and often gets better. Is it reasonable to think that if taking BP pills doubles your chances of getting AMD as demonstrated by the Beaver Dam study, that stopping taking the pills would result in good things happening with AMD? We need a study to prove it, but it's a reasonable assumption. It's particularly reasonable given the proven fact that taking BP pills more than doubles the chance of getting AMD as you age.

Conclusion

I thought blood pressure pills were benign, something you probably had to take when you got older to lengthen your stay on earth. It's what the whole medical establishment and nearly every doctor says. What's a visit to the doctor without taking your BP?

They're wrong. The evidence that they're wrong is available, but they're no more willing to change than they were in the cases of antiseptic surgery or blood-letting.

That the horrible side-effects of BP pills are universally denied by doctors and bad enough. But making you blind? If there were any conscience in the medical establishment, they would defy the pharma companies and immediately create studies to validate this. Serious mining of the RWE in centralized medical charts that should show the relevant data points would be a good start.

 

How to Improve Software Productivity and Quality: The Common Sense Approach

I talked with a frustrated executive in a computer software company. I was about to visit their central development location for the first time, and he wanted to make sure I asked sufficiently penetrating questions so that I would find out what was “really” going on.

He explained that while he had written software, it was only for a few years in the distant past, and things had changed a great deal since his day. His current job in product marketing didn’t really require him to get into any details of the development shop, and in fact he preferred to stay out of the details for several reasons: (1) he was completely out of date with current technology and methods; (2) he didn’t want his thinking constrained by what the programmers declared was possible; (3) it was none of his business.

He had developed a keen interest in what was going on in the software group, however, because he realized that it had a dramatic effect on his ability to successfully market the product. His complaints were personal and based on his own experience, but they were fairly typical, which is why I’m recounting his tale of woe here.

The Lament

The layman’s lament was an interesting mish-mash of two basic themes:

• I’m not getting the results I need. There are certain results that I really need for my business. My competitors seem to be able to get those results, and I can’t. Basically, I want more features in each release, more frequent releases, more control and visibility on new features, fewer bugs in new releases, and the ability to make simple-sounding changes quickly. Our larger competitors seem to be able to move more quickly than we do.
• I think the way the developers’ work is old-fashioned, and if it were brought up-to-date, I would get the results I need. What they do seems to be “waterfall,” with lots of documentation that doesn’t say a lot. There must be something better, along the lines of what we used to call RAD (rapid application development). They only have manual testing, nothing automated, and they tell me it will be years before they can build automated testing! And shouldn’t they be using object-oriented methods? Wouldn’t that provide more re-use, so that things can be built and changed more quickly? They have three tiers, but when I want to change something, the code always seems to be in the wrong tier and takes forever. They’re talking about re-writing everything from scratch using the latest  technology, but I’m afraid it will take a long time and there won’t be anything that benefits me.

Basically, he was saying that he wants more things, quicker, and better quality. He also advanced some theories for why he’s not getting those things and how they might be achieved, but of course he couldn’t push his theories too hard, because he lacked experience and in-depth knowledge of the newer methods. He even claimed, in classic “the grass is greener” style, that practically everyone accomplishes these things, and he was nearly alone in being deprived of them – not true!

The usual dynamics of a technology group explaining itself to “outsiders” was also at work here – if you just listen to the technology managers, things are pretty good. The methods are modern and the operation is efficient and productive. There are all sorts of improvements that could be made with additional money for people and tools, of course, but for a group that’s been under continual pressure to build new features, support cranky customers and meet accelerated deadlines with fewer resources, they’re doing amazingly well. The non-technology executives tend to feel that this is all a front, and that results really could be better with more modern methods and tools. The technology managers, for their part, feel like they’re flying passenger planes listening to a bunch of desk-bound ignoramuses complain about their inability to deliver the passengers safely and on-time while upgrading the engine and cockpit systems at the same time. These people have no idea what building automated testing (for example) really takes, they’re thinking. The non-technology people don’t really want to talk about automated testing, of course – they’re the ones taking the direct heat from customers who get hurt by bugs in the new release, and aren’t even getting proposals from the technology management of how this noxious problem can be eliminated. Well, if you can’t tell me how to solve the problem (and you should be able to), how about this (automated testing, object-oriented, micro-services, etc.)??

It goes on and on. The business executives put a cap on it, sigh, maybe throw a tantrum or two, but basically try to live with a situation they know could be better than it is. Inexperienced executives refuse to put with this crap, and bring in new management, consultants, do outsourcing, etc. Their wrath is felt! Sadly, though, the result is typically a dramatic increase in costs, better-looking reporting, but basically the status quo in terms of results, with success being defined downwards to make everything look good. The inexperienced executive is now experienced, and reverts to plan A.

The technology manager does his version of the same dance. The experienced manager tries to keep things low-key and leaves lots of room for coping with disasters and the unexpected. Inexperienced technology managers refuse to tolerate the tyranny of low expectations; they strive for real excellence, using modern tools and methods. Sadly, though, the result is typically a dramatic increase in costs, better-sounding reports, but basically the status quo in terms of tangible results. The new methods are great, but we’re still recovering from the learning curve; that was tense and risky, I’m lucky I survived, that’s the last time I’m trying something like that again!

The Hope

The non-technology executive is sure there’s an answer here, and it isn’t just that he’s dumb. He keeps finding reason to hope that higher productivity with high quality and rapid cycles can be achieved. In my experience, the most frequent (rational) basis for that hope is a loose understanding of the database concept of normalization, and the thought that it should enable wide-spread changes to be made quickly and easily. Suppose the executive looks at a set of functionally related screens and wants some button or style change to be applied to each screen. It makes sense that there should be one place to go to make that change, because surely all those functionally related screens are based on something in common, a template or pattern of some kind. What if the zip code needs to be expanded from five digits to nine? The executive can understand that you’d have to go to more than one place to make the change, because the zip code is displayed on screens, used in application code and stored in the database, but there should be less than a handful of places to change, not scores or hundreds!

But somehow, each project gets bogged down in a morass of detail. When frustration causes the executive to dive into “why can’t you…” the eyes normally glaze over in the face of massive amounts of endless gobbledy-gook. What bugs some of the more inquisitive executives is how what should be one task ends up being lots and lots of tasks? With computers to do all the grunt work, there’s bound to be a way to turn what sounds, feels and seems like one thing (adding a search function to all the screens) actually be one thing – surely there must be! And if everything you can think of is in just one place, surely you should be able to go to that one place and change it! Don’t they do something like that with databases?

There is a realistic basis for hope

I’ve spent more of my life on the programmers’ side of the table than the executives’, so I can go on, with passion and enthusiasm, about the ways that technology-ignorant executives reduce the productivity, effectiveness and quality of tech groups, not to mention the morale! The more technical detail they think they know, the worse it seems to be.

That having been said, the executive’s lament is completely justified, and his hope for better days is actually reasonable (albeit not often realized).

What his hope needs to be realized is there to be exactly one place in the code where every completely distinct entity is defined, and all information about it is stated. For example, there should be exactly one place where we define what we mean by “city.” This is like having domains and normalization in database design, only extended further.

The definition of “city” needs to have everything we know about cities in that one place. It needs to include information that we need to store it (for example, its data type and length), to process it (for example, the code that verifies that a new instance of city is valid) and to display it (for example, its label). The information needs to incorporate both data (e.g. display label) and code (e.g. the input edit check) if needed to get the job done. This is like an extended database schema; a variety of high-level software design environments have something similar to this.

It must be possible to create composite entities in this way as well, for example address. A single composite entity would typically include references to other entities (for example, city), relationships among those other entities and unique properties of the composite entity (for example, that it’s called an “address.” This composite-making ability should be able to be extended to any number of levels. If there are composites that are similar, the similarity should be captured, so that only what makes the entity unique is expressed in the entity itself. A common example of this is home address and business address.

Sometimes entities need to be related to each other in detailed ways. For example, when checking for city, you might have a list of cities, and for each the state it’s in, and maybe even the county, which may have its own state-related lists.

The same principle should apply to entities buried deep in the code. For example, a sort routine probably has no existence in terms of display or storage, but there should usually be just one sort routine. Again, if there are multiple entities that are similar, it is essential that the similarities be placed in one entity and the unique parts in another. Simple parameterization is an approach that does this.

Some of these entities will need to cross typical software structure boundaries in order to maintain our prime principle here of having everything in exactly one place. For example, data entities like city and state need to have display labels, but there needs to be one single place where the code to display an entity’s label is defined. Suppose you want a multi-lingual application? This means that the single place where labels are displayed needs to know that all labels are potentially multi-lingual, needs to know which the current language is, and needs to be able to display the current language’s label for the current entity. It also means that wherever we define a label, we need to be able to make entries for each defined language. This may sound a bit complicated at first reading, but it actually makes sense, and has the wonderful effect of making an application completely multi-lingual.

In order to keep to the principle of each entity defined once, we need the ability to make relationships between entities. The general concept of inheritance, more general than found in object-oriented languages, is what we need here. It’s like customizing a standard-model car, where you want to leave some things off, add some things and change some things.

There’s lots more detail we could go into, but for present purposes I just want to illustrate the principle of “each entity defined in one place,” and to illustrate that “entity” means anything that goes into a program at any level. By defining an entity in one place, we can group things, reference things, and abstract their commonality wherever it is found, not just in a simple hierarchy, and not limited to functions or data definitions or anything else.

While this is a layman’s description, it should be possible to see that IF programs could be constructed in this way, the layman’s hope would be fulfilled. What the layman wants is pretty simple, and actually would be if programs were written in the way he assumes. The layman assumes that there’s one way to get to the database. He assumes that if you have a search function on a screen, it’s no big deal to put a search function on every screen. He assumes that if he wants a new function that has a great deal in common with an existing function, the effort to create the new function is little more than the effort to define the differences. He assumes that look and feel is defined centrally, and is surprised when the eleventh of anything feels, looks or acts differently than the prior ten.

Because he has these assumptions in his mind, he’s surprised when a change in one place breaks something that he doesn’t think has been changed (the infamous side-effect), because he assumes you haven’t been anywhere near that other place. He really doesn’t understand regression testing, in which you test all the stuff that you didn’t think you touched, to make sure it still works. Are these programmers such careless fools that, like children in a trinket shop, they break things while walking down the aisle to somewhere else, and you have to do a complete inventory of the store when the children leave?

Programs are definitely not generally written in conformance with the layman’s assumptions; that’s why there’s such a horrible disconnect between the layman and the techies. The techies have a way of building code, generally a way that they’ve received from those who came before them, that can be made to work, albeit with considerable effort. They may try to normalize their database schemas and apply object principles to their code, but in the vast majority of cases, the layman’s assumption of a single, central definition of every “thing,” and the ability to change that thing and have the side-effects ripple silently and effectively through the application, does not exist, is not articulated, not thought about, and is in no way a goal of the software organization. It’s not even something they’ve heard talked about in some book they keep meaning to get to. It’s just not there.

I assert that it is possible to write programs in a way that realizes the layman’s hope.

I’ve done it myself and I’ve seen others do it. The results are amazing. It’s harder to do than you would ideally like because of a lack of infrastructure already available to support this style of writing, but in spite of this, it’s not hard to write. Moreover, once the initial investment in structure has been made, the ability to make changes quickly and with high quality quickly pays back the investment.

The main obstacle for everyone is that there is tremendous inertia, and the techniques that provide a basis for the hope, while reasonable and achievable, are far out of the mainstream of software thinking. I have seen people who have good resumes but are stupid or lazy look at projects that have been constructed according to the “one entity – one definition” principle and simply declare them dead on arrival, complete re-write required. But I have also encountered projects in domain areas where there is no tradition at all in building things in this way in which the people have invented the principles completely on their own.

The “principle of non-redundancy” has far-reaching technical consequences and ends up being pretty sophisticated, but at its heart is simple: things are hard to do when you have to go many places or touch many things to get them done. When the redundancy in program representation (ignoring for the moment differences between code, program data and meta-data) is eliminated, making changes or additions to programs is optimally agile. In other words, with program representation of this type, it is as easy and quick as it can possibly be to make changes to the program. In general, this will be far quicker than most programs in their current highly redundant form.

The layman’s hope that improvements can be made in software productivity, quality and cycle is realistic, and based on creating a technical reality behind the often-discussed concepts of “components” and “building blocks” that is quite different from the usual embodiments.

I have no idea why approach to building software, which is little but common sense, isn't taught in schools and widely practiced. For those who know and practice it, the approach of "Occamality" (define everything in exactly one place) gives HUGE competitive advantages.

How to Improve Software Productivity and Quality: Code and Metadata

In the long-fought war to improve software programming productivity, there have been offensives on many fronts, but precious little genuine progress made. We give our programmers the fanciest, most high-tech equipment imaginable – and it is orders of magnitude faster and more powerful than the equipment available to earlier generations – but this new equipment has made only marginal difference. While relieving programmers of the burden of punch cards helps, the latest generation of programmers are not getting the job done much better than their comparatively low-tech predecessors.

Form and Content

As usual, most efforts to improve the situation focus on one of two general approaches: form or content.

People who focus on form tend to think that the process of getting software written is what’s important. They talk about how one “methodology” is better than another. They think about how people are selected, trained, organized and motivated. As you might imagine, the spectrum of methodologies is a broad one. On one end of the spectrum is the linear approach, which starts from the generation of business requirements for software, and ends with testing, installation and ongoing maintenance. On the other end of the spectrum is the circular, interactive approach, in which a small working program is built right away, and gradually enhanced by programmers who interact closely with the eventual end-users of the program. There are any number of methodologies between these two extremes, each of which claims an ideal combination of predictable linearity with creative interactivity.

People who focus on content tend to think that the language in which programs are written and the structure and organization of programs are what’s important. Naturally, they like design and programming tools that give specific support to their preferred language and/or architecture. There tends to be a broad consensus of support at the leading edge around just a couple of languages and structures, while the majority of programmers struggle with enhancing  and maintaining programs originally written according to some earlier generation’s candidate for “best language” or “best architecture.” At the same time, many of those programmers make valiant attempts to renovate older programs so that they more closely resemble the latest design precepts, frequently creating messes. Regardless of the generation, programmers quickly get the idea that making changes to programs is their most time-consuming activity (apart, of course, from never-ending meetings), and so they focus on ways to organize programs to minimize the cost of change. This leads to a desire to build “components” that can be “assembled” into working programs, and naturally to standards for program components to “talk” with each other.

Lots of effort, not much progress

The net effect of all these well-intentioned efforts, on both the form and content sides, has been a little bit of progress and a great deal of churn. Having some agreed-on methodology leads to better predictability and generally better results than having none; it seems that having a beat to which everyone marches in unison, even if it’s not the best beat, leads to better results than having a great beat that many people don’t know or simply refuse to march to. What is the best methodology in any one case depends a great deal on both how much is already known about the problem to be solved and how smart and broadly skilled the participants in the project are. The more qualified the people and the less known about the target, the more appropriate it is to be on the “interactive” end of the spectrum; think highly qualified and trained special forces going after a well-defended target in enemy territory – creativity, teamwork and extraordinary skills are what you need. The more ordinary the participants and with better-understood objectives, the more appropriate being somewhere towards the “linear” end of the spectrum; think of a large army pressing an offensive against a broad front – with so many people, they can’t all be extraordinary, and you want coordinated, linear planning, because too much local initiative will lead to chaos.

As to content, while it is clear that the latest programming languages encourage common-sense concepts like modular organization and exposing clear interfaces, good programmers did that and more decades ago in whatever language they were writing, including assembler, and bad programmers can always find a way to make messes. And even if you use the best tools and interface methods, incredible churn is created by frequent shifts in what is considered to be the best architecture, practically obsoleting prior generations. Probably the single biggest movement over the last several decades has been the gradual effort to take important things about programs that originally could be discovered only by inspecting the source code of the program and making those things available for discovery and use by people and programs, without access to the source code. The first major wave of this movement led to exposure of much of a program’s persistent data, in the form of DBMS schemas; the other major wave of this movement (in many embodiments, from SAA to SOAP to microservices) exposes a program’s interfaces, in the form of callable functions and their parameters. This has been done in the belief that it will enable us to make important changes to some programs without access to or impact on others, and thus approach the dream of programming by assembling fixed components, like building blocks or legos, into completed programs or buildings. The belief in this dream has been affected very little by decades of evidence that legos don’t build things that adults want to live in.

I believe that while considerations of form are incredibly important, and when done inappropriately can drag down or even sink any programming project, there is little theoretical headway to be made by improvements in methodology. I think most of the relevant ideas for good methodology have already been explored from multiple angles, with the exception of how to match methodology to people and project requirements – no one methodology is the best for each project and each collection of people.  But even when you’ve picked the best methodology, the NBA players will always beat the high school team, as long as the NBA players execute well on the motivational and teamwork aspect that any good methodology incorporates. With methodology we’re more in need of good execution and somehow assembling a talented and experienced team than we are of fresh new ideas.

Ptolemy, Copernicus, Kepler, Newton

Content, however, is another story altogether. I think our current best languages and architectures will look positively medieval from the perspective of a better approach to content. I think there is a possible revolution here, one that can bring about dramatic improvements in productivity, but which requires an entirely new mind-set, as different as Copernicus and Ptolemy, as different as Einstein and Newton.

Having said that, let me also say that the “new” ideas are by no means completely new. Many existing products and projects have exploited important parts of these ideas. What is mostly new here is not any particular programming technique, but an overriding vision and approach that ties various isolated programming techniques into a unified, consistent whole. Kepler’s equations described the motion of planets with accuracy equal to Newton’s – in fact, you can derive one set of equations mathematically from the other; but Newton provided the vision (gravity) and the tools (calculus) that transformed some practical techniques (Kepler’s equations) into the basis of a new view of the world. That’s why physics up to the end of the nineteenth century was quite justifiably characterized as “Newtonian” and not “Keplerian.”

Ptolemy and Newton looked at the same set of objects; Ptolemy picked the closest one, the earth, to serve as the center of thinking, while still incorporating the rest.

His main goal was to describe the motions of what you could see, focused on the matter. Copernicus noted that things got simpler and more accurate if you picked one farther away, the Sun, to serve as the center of things.

Kepler made things better by noticing the curves made by the planets. Newton then took the crucial step: instead of focusing on matter, he focused on energy (gravity in this case), and wrote an equation describing how gravity works,

which creates changes in the location and velocity of matter. In programming, we have clear equivalents of matter and energy: matter is data (whether or not it is persistent), and energy is instructions (lines of code, regardless of the language it’s in). In COBOL, for example, this division is made explicit in the data division and the procedure division. In modern languages the two are more intermixed, but it remains clear whether any particular line describes data or describes an action to take (an if statement, assignment statement, etc.).

Now, in spite of what you may have been taught in school, Ptolemy’s method works. In fact, it would be possible (though not particularly desirable) to update his approach with modern observations and methods, and have it produce results that are nearly identical to those possible today; in fact, only when you get to relativity does Ptolemy’s method break down altogether -- but remember that the effects of relativity are so small, it takes twentieth century instruments to detect them. But no one bothers to do this, because the energy-centered approach developed by Newton and refined by his successors is so much simpler, cleaner and efficient.

Similarly, there is no doubt that the instructions-centric approach to programming works. But what comes out of it is complicated, ugly and inefficient. The Newtonian breakthrough in programming is replacing writing and organizing instructions (and oh, by the way, there’s some data too) as the center of what we do with defining and describing data (and oh, by the way, there are some instructions too). The instruction-centered approach yields large numbers of instructions with a small amount of associated data definitions; the data-centered approach yields large numbers of data definitions and descriptions operated on by a significant body of unchanging, standard instructions and small numbers of instructions specifically written for a particular program. In the instruction-centered approach, we naturally worry about how to organize collections of instructions so that we can write fewer of them, and arrive at concepts like objects, components and class inheritance (a subclass inherits and can override its parent’s methods (instructions)). In the data-centered approach, we naturally worry about how to organize collections of data definitions so that we can have the minimal set of them, and arrive at concepts like meta-data, standard operations (e.g. pre-written meta-data-driven instructions enabling create, query, select, update and delete of a given collection of data) and data inheritance (a child data definition, individual or group, inherits and can override its parent’s definitional attributes). In short, we separate out everything we observe into a small, unchanging core (like Newton's gravity) that produces ever-changing results in a diverse landscape.

We shift our perspective in this way not because it enables us to accomplish something that can’t be accomplished by the current perspective, but because it proves to be cleaner, simpler, more efficient, easier to change, etc.

Instructions, data and maps

Only by understanding the details of this approach can it really be appreciated, but the metaphor of driving instructions and maps is appropriate and should make the core idea clear.

Suppose your job is to drive between two locations, and the source and destination location are always changing. There are two general approaches for giving you directions:

1. Turn-by-turn directions (the instruction-driven, action-oriented approach)
2. A map, with source and destination marked (the data-driven, matter-oriented approach)

The advantage of directions is that they make things easy for the driver (the driver is like the computer in this case). You pick up one step, drive it, then pick up the next, drive it, and so on until the end. You don’t have to think in advance. All you have to do is follow directions, which tell you explicitly what to do, in action-oriented terms (turn here, etc.).

The problem with directions come in a couple of circumstances:

• You have to have a huge number of directions to cover all possible starting points and destinations, and there is a great deal of overlap between sets of directions
• If there is a problem not anticipated by the directions, such as an accident or road construction, you have to guess and get lucky to get around the problem and get back on track.

A map, on the other hand, gives you most of the information you need to generate your own directions between any two given points. The map provides the information; you generate the actions from that information. With a direction-generating program and some parameters like whether to use toll roads, a generic program can generate directions on the fly. With a program like Wayz, real-time changes due to updated traffic information can even be generated.

The Wayz program itself isn't updated very often -- it's mostly the map and road conditions. Same thing with a program written using this approach; the core capabilities are written in instructions, while the details of the input, storage, processing and output are all described in a "map" of the data and what is to be done with it.

Conclusion

It's pretty simple: programming today largely follows the method of Ptolemy, resulting in an explosion of software epi-cycles to get anything done. Attempts to keep things easily changeable sound promising but never work out in practice.

The way forward is to focus instead on what there is and what is to be done with it (data and metadata), with a small amount of Wayz-like code to take any user or data stream from its starting point to its destination with easy changes as needed.

Cryptocurrency and Crime

Cryptocurrency (Bitcoin, Ethereum and the rest) is fueling a new kind of crime wave. Computers and networking are the lawless continent on which criminals go wherever they want, going into factories, stores and homes, stealing data in massive amounts to sell and use to enable more crime. That crime continues to grow. Bitcoin, the software built on computers and networks, has added the element of anonymous payments to and between criminals. Criminals world-wide have been inspired by this near-instant, secret way to pay and accept money to ratchet up existing crimes and invent new ones.

Why do big, important people continue to deny a problems exists? As discussed in a recent WSJ article, this crime-enabling menace needs to be confronted head-on.

Burglary

Burglary is when a criminal steals something without a confrontation with the owner, for example breaking into your house when you’re away and taking your valuables. A great deal of cyber-crime has been burglary, things like hacking your computer system and stealing data. But then how do you sell the data? More important, how will you collect your money from the criminals who buy it?

Enter Bitcoin. The buyer can be anywhere in the world. They can be of any nationality, used to using any currency. Once an agreement has been made, payment is simple, fast and untraceable. The buyer and seller don’t need any direct contact. Any currency can be converted to Bitcoin to send, and converted to any currency on receipt. Or left in Bitcoin to use in other criminal enterprises. Bitcoin hasn’t transformed the huge field of criminal data, but it sure has greased the wheels.

Robbery/Ransomware

Robbery is worse than burglary. It’s when a criminal confronts you on the street, points a gun at you and says something like “your wallet and jewels or your life.” Most people do what the robber says and hope to live another day. The new wave of cybercrime is robbery a.k.a ransomware: not just sneaking into your computer but encrypting everything and “tying your computer up” until you pay the ransom.

Ransom attacks on computers have always existed, but they were fairly rare, because there was no way the robber could collect the victim’s money without revealing himself. Then Bitcoin came along. Bitcoin enables anyone to buy it from an exchange like Coinbase and then send it to the criminal’s anonymous Bitcoin address. The criminal, who could be anywhere, then has your money and may, if he feels like it, release your computers from their electronic shackles.

There wasn't much ransomware a decade ago. Then came Bitcoin.

“eCrime – a broad category of malicious activity that includes all types of cybercrime attacks, including malware, banking trojans, ransomware, mineware (cryptojacking) and crimeware – seized the monetization opportunity that Bitcoin created. This resulted in a substantial proliferation of ransomware beginning in 2012...

Bitcoin exchanges provided adversaries the means of receiving instant payments while maintaining anonymity, all transacted outside the strictures of traditional financial institutions.”

Then came a new generation of locking technology, 2048 bit private key. This led to a shift away from spraying malware to millions of little computers to infecting, locking and ransoming big institutions, Big Game Hunting.

Big Game Hunting

CrowdStrike

The criminals evolve quickly. They are generations ahead of the largely inept bureaucrats with huge budgets following security regulations that are typically obsolete by the time they are issued.

As a result, ransomware attacks were everywhere in 2021 and continue growing.

Double-extortion ransomware attacks rise: On average, a new organization becomes a victim of ransomware every 10 seconds worldwide.

Here is more and a recent example.

From suitcases of cash to Venmo for Criminals

Illegal national and international weapons trafficking has always existed. So has human trafficking. Likewise importing and selling addictive drugs like heroin. These are all human horrors.

For some strange reason, the people who import and sell innocent young girls want to be paid in cash. Lots of it. Same thing with fentanyl. It’s inconvenient and dangerous, carrying around huge stacks of hundred dollar bills! Bitcoin changes the game. Bitcoin is like Venmo for the criminal class only better. No records. No annoying banking regulations and reports sent by banks to snoopy government agencies. Computer-to-computer transfer. Yes there’s a record that a transfer of Bitcoin took place – but ZERO record of from whom or to whom.

On the other hand...

Cryptocurrency utilization is exploding, most of it unrelated to criminal activity. It is certainly true that crypto-related crime has grown; one respected vendor reports it nearly doubled from 2020 to 2021, reaching an all-time high of $14 billion. That same vendor reports even more dramatic growth of overall cryptocurrency transactions, which was more than five times in the same period. As the vendor says: “Transactions involving illicit addresses represented just 0.15% of cryptocurrency transaction volume in 2021 despite the raw value of illicit transaction volume reaching its highest level ever. As always, we have to caveat this figure and say that it is likely to rise as Chainalysis identifies more addresses associated with illicit activity and incorporates their transaction activity into our historical volumes. For instance, we found in our last Crypto Crime Report that 0.34% of 2020’s cryptocurrency transaction volume was associated with illicit activity — we’ve now raised that figure to 0.62%.”

Supporters or crypto are also quick to point out that fiat currency is also used by criminals, so no one should be surprised that crypto is used by them.

Conclusion

Cryptocurrencies are widely discussed. "Bitcoin Billionaires" are in the news; hosts of ordinary people hope to be like them. The crypto industry sponsors reports and generally promotes the idea that the criminal use of crypto is minimal and going down. Which it is, as a share of all crypto transactions. As we know from the growth of ransomware attacks, the use of crypto by criminals is in fact increasing.

It should be illegal for any regulated exchange to enable sending to or receiving from any address that fails to have full KYC and other identity disclosure with it. There are lots of exchanges that operate internationally for the criminals to continue using, as they will.

Cryptocurrencies are an amazing technical achievement. Computers and networking already provide rich ground for criminal activity; Bitcoin added a safe-for-criminals international payment method that has fueled computer-based crime.

Note: this was originally published at Forbes.

NNT for Benefits and for Harms

In a previous post, I described the difference between relative risk (efficacy), absolute risk and the related concept of NNT (number needed to treat). In that post I focused on the NNT to get the benefit of the treatment. In this post I will focus on the essential other half of NNT: the NNT to be harmed.

I will mostly focus on the direct harms of the treatment itself. However, in some cases, there are harms that come from other actions taken to treat or avoid a medical problem. Sometimes the harms can be large. The study of these indirect harms is not as advanced in the scientific literature as the direct harms, but given how large the scale of the indirect harms can be, they should be made standard. practice.

NNT for Harms

NNT is a simple way to understand how probable a given outcome is likely to be in absolute terms.

Sometimes there aren't any harms, as in this meta-analysis of over 240,000 patients in 18 studies.

What's important to note is that the researchers looked not only for the benefit of fever reduction, but also for the harms that had been suspected for one of the treatments.

Here is one where the NNT for harms is crucially important -- because the treatment that is supposed to prevent heart attacks caused more of them than it prevented!

The case above illustrates an important aspect of NNT: it should cover (if appropriate) multiple possible benefits and multiple types of harms.

Just because NNT harms outweigh benefits for a treatment doesn't mean that medical practice responds appropriately. For a long time, high blood cholesterol was thought to cause heart attacks. Statins became widely prescribed to lower the number. But now it is scientifically proven that blood cholesterol should not be lowered and therefore statins should not be taken. In spite of the fact that NNT harms are strong with no benefits, it remains standard practice for doctors to prescribe statins to lower the cholesterol level to meet now-disproven standards..

Sadly, this raises the issue of conflicts of interest and transparency in scientific research, and the readiness of the medical profession to update practices when the science demonstrates that it should. It's even trickier when a pharmaceutical company conducts studies to prove that a drug it developed has important benefits and minimal harms.

NNT Harms for covid vaccination

The FDA's EUA (Emergency Use Authorization) Issued in December 2020 for Pfizer's covid drug claimed 95% effectiveness, and listed minor side effects which lasted just a couple days. The FDA gave full approval for the drug in August 2021.

The full approval document stated that "the vaccine was 91% effective in preventing COVID-19 disease." No explanation was given for the reduced effectiveness. Unlilke the EUA document, the absolute numbers of infections were not disclosed, therefore giving a highly misleading impression of how likely any person who got the vaccine would be helped by it, implying that 90% of the vaxed would be protected vs. the actual number of under 1%.

For harms, most of the minor harms of the EUA were repeated. However, they disclosed that myocarditis and pericarditis were suffered by young males: "Available data from short-term follow-up suggest that most individuals have had resolution of symptoms. However, some individuals required intensive care support. Information is not yet available about potential long-term health outcomes." Sadly, they provided no data, no NNT for Harm.

I have yet to find good numbers for NNT Harms for covid. This should be easy, but as it turns out, the vast majority of the relevant data is secret. Yes, secret by approval of the FDA.

However, I've dug into a couple of issues based 100% on published scientific data. For example, I found a paper published in April 2021 in the New England Journal of Medicine on Vaccine Safety in Pregnant Persons. The paper showed that the mRNA vaccines were safe for pregnant people to receive. Here is Table 4 from the original paper, showing that there were 104 spontaneous abortions out of 827 vaccine recipients, about 12%, which is within a normal range.

Here is the footnote to the last column, about the numbers of people involved.:

A correction was published in October 2021 in the same journal, after the FDA's full approval had been issued. A casual reading of the correction, including the summary and abstract, makes it seem as though nothing significant was changed.

Here is Table 4 in the corrected paper:

The number of spontaneous abortions remained at 104, but the totals and percentages were dropped. The explanation is found in the footnote:

The footnote leaves the impression that nothing can be concluded. However, returning to the footnote in the original paper, we read "...based on 827 participants ... who received a Covid-19 vaccine ... A total of 700 participants (84.6%) received their first eligible dose in the third trimester..." So 700 participants could not have had spontaneous abortions, since all those took place in the first 20 weeks of pregnancy.

The arithmetic leads us to 827-700=127 participants were vaccinated earlier, and 104 of those participants had spontaneous abortions. The vast majority. This is clearly something that the authors should have pointed out and explained. Maybe my logic here is wrong.

This leads us to wondering what should happen:

What should happen

First of all, the authors should have made clear the implications of their correction. If indeed the data shows that spontaneous abortions were excessive, they should have said so, and promised further study to confirm.

Second, data about medical treatments of all kinds, including drugs, should be fully open source, the way some software is. That way, others could do the job that the authors of the study failed to do. The developer of the drug should open its data to the public, just like the source code to software like Linux is 100% open for copying, testing and use. This by itself will solve many problems. It will also enable problems to be surfaced quickly, so that a minimum of people are hurt by the problems. If drug makers were truly interested in safety and effectiveness, they would welcome the additional scrutiny.

Conclusion

NNT is an essential measure for treatment effectiveness. Every time a treatment is proposed to a patient, NNT should be part of the discussion. Certainly NNT for benefits is important -- that's the whole point of the treatment. But NNT for harms is regularly left out of the discussion. Instead, it should be brought to the forefront.

Better Software and Happier Customers with Post-hoc Design

What can you possibly mean by "post-hoc design?" Yes, I know it means "after-the-fact design," using normal English. It's nonsense! First you design something. Then you build it. Period.

Got your attention, have I? I agree that "post-hoc design" sounds like nonsense. I never heard of it or considered it for decades. But then I did. Before long I saw that great programmers used it to create effective high-quality, loved-by-customers software very quickly.

The usual way to build software: design then build

The way to build good software is obviously to think about it first. Who does anything important without having a plan? Start by getting requirements from the best possible source, as detailed as possible. Then consider scale and volume. Then start with architecture and drill down to design.

When experienced people do architecture and design, they know that requirements often "evolve." So it's important to generalize the design anticipating the changes and likely future requirements. Then you make plans and can start building. Test and verify as you drive towards alpha then beta testing. You know the drill. Anything but this general approach is pure amateur-hour.

I did this over and over. Things kept screwing up. The main issue was requirements "evolution," which is something I knew would happen! Some of the changes seemed like they were from left field, and meant that part of my generalized architecture not only failed to anticipate them, but actually made it harder to meet them! Things that I anticipated might happen which I wove into the design never happened. Not only had I wasted the time designing and building, the weren't-needed parts of the design often made it hard for me to build the new things that came along that I had failed to anticipate.

I assumed that the problem was that I didn't spend enough time doing the architecture and design thinking, and I hadn't been smart enough about it. Next time I would work harder and smarter and things would go more smoothly. Never happened. How about requirements? Same thing. The people defining the requirements did the best they could, but were also surprised when things came along, and embarrassed when things they were sure would be important weren't.

After a long time -- decades! -- I finally figured out that the problem was in principle unsolvable. You can't plan for the future in software. Because you can't perfectly predict the future! What you are sure will happen doesn't, and what you never thought about happens.Time spent on anything but doing and learning as you go along is wasted time.

The winning way to build software: Build then Design

Build first. Then and only then do the design for the software you've already built. Sounds totally stupid. That's part of why I throw in some Latin to make it sound exotic: "Post-hoc design," i.e., after-the-fact design.

When you design before you build, you can't possibly know what you're doing. You spend a bunch of time doing things that turn out to be wrong, and making the build harder and longer than it needs to be. When you build in small increments with customer/user input and feedback at each step, keeping the code as simple as possible, you keep everything short and direct. You might even build a whole solution for a customer this way -- purposely NOT thinking about what other customers might need, but driving with lots of hard-coding to exactly what THIS customer needs. Result: the customer watches their solution grow, each step (hopefully) doing something useful, guides it as needed, and gets exactly what they need in the shortest possible time. What's bad about a happy customer?

Of course, if you've got the typical crew of Design-first-then-build programmers, they're going to complain about the demeaning, unprofessional approach they're being forced to take. They might cram in O-O classes and inheritance as a sop to their pride; if they do, they should be caught and chastised! They will grumble about the enormous mountain of "technical debt" being created. Shut up and code! Exactly and only what's needed to make this customer happy!

When the code is shown to another customer, they might love some things, not need some other things and point out some crucial things they need aren't there. Response: the nearly-mutinous programmers grab a copy of the code and start hacking at it, neutering what isn't needed, changing here and adding there. They are NOT permitted to "enhance" the original code, but hack a copy of it to meet the new customer's need. At this point, some of the programmers might discover that they like the feeling of making a customer happy more quickly than ever before.

After doing this a couple times, exactly when is a matter of judgement, it will be time to do the "design" on the software that's already been built. Cynics might call this "paying off tech debt," except it's not. You change the code so that it exactly and only meets the requirements of the design you would have made to build these and only these bodies of code. You take the several separate bodies of code (remember, you did evil copy-and-modify) and create from them a single body of code that can do what any of the versions can do.

When you do this, it's essential that you NOT anticipate future variations -- which will lead to the usual problems of design-first. The pattern for accomplishing this is the elimination of redundancy, i.e., Occamality. When you see copy/modify versions of code, you replace them with a single body of code with the variations handled in the simplest way possible -- for example, putting the variations into a metadata table.

This isn't something that's done just once. You throw in a post-hoc design cycle whenever it makes sense, usually when you have an unwieldy number of similar copies.

As time goes on, an ever-growing fraction of a new user's needs can be met by simple parameter and table settings of the main code line, and an ever-shrinking fraction met by new code.

Post-Hoc Design

Ignoring the pretentious name, post-hoc design is the simplest and most efficient way to build software that makes customers happy while minimizing the overall programming effort. The difference is a great reduction in wasted time designing and building, and in the time to customer satisfaction. Instead of a long requirements gathering and up-front design trying valiantly to get it right for once, resulting in lots of useless code that makes it harder to build what it turns out is actually needed, you hard-code direct to working solutions, and then periodically perform code unification whose purpose is to further shorten the time to satisfaction of new customers. To the extent that a "design" is a structure for code that enables a single body of code to be easily configured to meet diverse needs, doing the design post-hoc assures zero waste and error.

What is the purpose of architecture and design anyway? It is to create a single body of code (with associated parameters and control tables) that meets the needs of many customers with zero changes to the code itself. The usual method is outside-in, gaze into the future. Post-hoc design is inside-out, study what you built to make a few customers happy, and reduce the number of source code copies to zero while reducing the lines of code to a minimum. The goal is post-hoc design is to minimize the time and effort to satisfy the next customer, and that's achieved by making the code Occamal, i.e., eliminating redundancies of all kinds. After all, what makes code hard to change? Finding all the places where something is defined. If everything is defined in exactly one place, once you've found it, change is easy.

Post-hoc design is a process that should continue through the whole life of a body of code. It prioritizes satisfaction of the customer in front of your face. It breaks the usual model of do one thing to build code and another to modify it. In the early days of what would normally be called a code "build," the code works, but only does a subset of what it is likely to end up doing. When customers see subsets of this kind, it's amazing how it impacts their view of their requirements! "I love that. I could start using it today if only this and that were added!" It's called "grow the baby," an amazing way to achieve both speed and quality.

New name for an old idea

All I'm doing with "Post-hoc design" is putting a name and some system around a practice that, while scorned by academia and banned by professional managers, has a long history of producing best-in class results. I'm far from the first person who has noticed the key elements of post-hoc design.

Linus Torvalds (key author of Linux, the world's leading operating system) is clearly down on the whole idea of up-front design:

Don’t ever make the mistake [of thinking] that you can design something better than what you get from ruthless massively parallel trial-and-error with a feedback cycle. That’s giving your intelligence much too much credit.

Gall's Law is a clear statement of the incremental approach:

A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system.

The great computer scientist Donald Knuth, author of the multi-volume Art of Computer Programming, was a master of shifting between assembler language programming and abstract algorithms and back, the key activities of the speed-to-solution and post-hoc abstraction phases of the method I've described here.

People who discover the power and beauty of high-level, abstract ideas often make the mistake of believing that concrete ideas at lower levels are worthless and might as well be forgotten. On the contrary, the best computer scientists are thoroughly grounded in basic concepts of how computers actually work. The essence of computer science is an ability to understand many levels of abstraction simultaneously.

Thanks to Daniel Lemire for alerting me to these quotes.

Conclusion

Post-hoc design is based on the idea that software is only "built" once, and after that always changed. So why not apply the optimal process of changing software from day one? And then alternate between as-fast-as-possible driving to the next milestone, with periodic clean-ups to make fast driving to the next goal optimal? Post-hoc design is a cornerstone of the process of creating happy customers and optimal code.  It also happens to conform to the goals of software architecture. Post-hoc design is like first fighting a battle, and then, once the battle is over and you've won, cleaning and repairing everything, incorporating what you learned from the battle just past so that everything is ready for the next battle. Post-hoc design is the way to win.

 

Does Vaccine Efficacy of 95% mean I won't get sick?

The Moderna and Pfizer Covid vaccines have 90-95% efficacy, but the studies submitted for their approval showed they helped only about 1% of the people who took them. This is news to most people. How can this be?

We are constantly told that vaccines are safe and highly effective, for example by the CDC. Numbers like 90% efficacy are thrown around, which most people understand to mean that getting vaccinated means there's only a 1 chance in 10 that you'll get sick. You're really protected!

What the CDC and major authorities fail to disclose is that standard statistical methods applied to the vax vendors' own data shows that only about one in a hundred people who get the jab would be protected from getting covid! The tests did indeed show 90% or better "efficacy" (relative risk improvement), but what's more relevant is "absolute risk" (AR), which their own data showed was around 1%.

Read on to understand these industry-standard measures that are mostly ignored; if widely understood and acted on, they would transform not just vaccines, but pharma and public health in general.

Winter Coats and Vaccines

Winter coats are a standard solution to protect people from getting cold when the weather outside is cold. Kind of like when the air is suffused with invisible vaccine particles, you want to help your body defend itself.

There are a wide variety of coats available to protect against the cold. What would happen to a new coat vendor that promoted its coats as being highly effective against the cold, protecting most people who wear them, but it turned out that the maker and seller knew that 99% of the people who wear them on a cold winter day wouldn't be helped by them -- word would get out quickly and the coat maker's reputation would be in the cellar.

What would happen if major authorities had subsidized the coat making, regulated their testing, and then promoted them as "safe and effective?" And then what would happen if all the authorities demanded that you buy and wear the coats, to the point refusing to let you enter a football stadium on a cold day unless you were wearing one of the approved coats? There would be mass revolt. Which is what would have happened with covid if people knew the facts that were so carefully concealed from them.

When locations like restaurants and performance halls opened, authorities in places like New York City declared that only people with proof of vaccination would be admitted. People were eager to eat out and be entertained, so this was another reason to get the jab. Vaccination cards were checked on entry so that everyone could be "safe."

While covid is the most current example of this grotesque propaganda/misinformation, it is all too common in healthcare and pharma, as I have shown for example here for saturated fat, here for cholesterol and here for hypertension. What's new in covid is the level of coercion involved.

Relative risk, absolute risk and Number Needed to Treat (NNT)

The widely used number for a vaccine called "efficacy" is technically "relative risk" (RR). In scientific papers, it's typically a number like .05, which means that compared to the number of people who got sick without the vax, just .05 of the vaxed got sick. This is translated to saying 95% of the vaxed avoided sickness compared to the unvaxed who got sick. While technically true, it is NOT about your chances of getting sick or staying well. It means relative risk, which is how much better the vax is compared to those who had no vax and got sick, independent of the number of people in the study.

Let's go back to winter coats. When people go out in the cold, they put something on to keep warm. Sometimes the coat doesn't keep some of them warm enough. Suppose the august health authorities got real worried about people dying of the cold without adequate protection. Huge amounts of time and money were spent developing what the developers thought was a great winter coat. Never mind that, for various reasons, the vast majority of people weren't getting cold. They went to a northern football stadium near the end of play-off season (winter). They got everyone entering at half the entrance gates to wear their wonderful coat and everyone who entered at the other half to wear a fake, ineffective version of the coat (the placebo) on top of whatever they were already wearing. At the end of the game, they briefly interviewed and temperature-measured everyone who left, noting which version of the coat they wore.

Let's suppose that 20,000 people went to the football game, with 10,000 getting fancy new coats and the other 10,000 getting fake coats. Suppose 10 people wearing the fancy new coat got cold, while 100 people in the fake coat group got cold.

First let's calculate the number everyone talks about, efficacy, technically known as Relative Risk (RR). RR in this case is 100 minus 10 divided by 100 = 90% efficacy. The wonderful coat did much better when added to what people were already wearing, about ten times better than the fake coat (placebo)! This is the number everyone thinks means that 90% of the people who take the vax won't get sick. Except it doesn't mean that. The key to understanding that is that RR has NOTHING to do with the size of the group, the number of people getting poked.

So let's calculate Absolute Risk (AR). In this case, of the 10,000 in the fake coat (placebo) group, 100 got cold, which is 1 in 100, for an AR of 1.0%. Your chances of avoiding getting cold without the fancy coat were excellent -- 99 out of 100! For the 10,000 people in the fancy coat group, just 10 got cold, which is 1 in 1,000, an AR of 0.1%. The relative difference between the fake and real coats was truly big -- ten times! But the absolute difference means that 10,000 people had to get the fancy coat in order to avoid just 90 of them getting cold. The reduction in absolute risk was 1.0% - 0.1% = 0.9%.

How many people have to get the fancy coat in order for one to benefit? Scientists have a name for this. It's NNT: Number Needed to Treat, sometimes called NNTV (Number Needed To Vaccinate) when a vax is involved. While "efficacy" focuses on "relative" risk, NNT turns the absolute risk (AR) into a more relevant number -- of those getting the treatment, how many will benefit? In this case, all 10,000 football fans would have to wear the fancy coat so that about 100 wouldn't be cold, ignoring the 10 who got cold anyway. In other words, in order for one person to benefit, 100 people have to get the treatment, an NNT of 100. For the other 99, the fancy coat made no difference -- they would have been warm without it.

Getting back to reality, this means that the coats most people choose to wear protect them from getting cold remarkably well. Anyone surprised? What's the normal reaction to being in the stands and getting cold? Doing something to warm up! Jump up and down. Wave your arms. Drink a cup of hot cocoa. Get hugged. Sit on someone's lap, get wrapped in their coat. If worse comes to worse, leave for someplace warm. There are "treatments" that work just fine.

Why would anyone bother accepting and wearing the authorized coat on top of what they already have? In the vast majority of cases, they'll be fine without it, and there are things they can do if they start to feel cold. Not to mention the risk of side effects of the fancy new thing. Here and here are more detailed explanations with examples.

ARR and NNT for Covid

I used round numbers above to make sure the concept was clear. But the whole point is the real world. There is a wonderful scientific website that provides NNT's for many treatments, based completely on scientific studies. For example, here is their article on cholesterol-reducing statins. which makes it clear that no one should be taking these widely used but destructive drugs.

Let's turn to the NNT for covid. What's amazing about this is that the information about NNT for covid is hidden in plain sight. Let's look at the FDA's announcement of their EUA (Emergency Use Authorization) for the Pfizer covid vaccination. The FDA states:

The FDA has determined that Pfizer-BioNTech COVID-19 Vaccine has met the statutory criteria for issuance of an EUA. The totality of the available data provides clear evidence that Pfizer-BioNTech COVID-19 Vaccine may be effective in preventing COVID-19. The data also support that the known and potential benefits outweigh the known and potential risks, supporting the vaccine’s use in millions of people 16 years of age and older, including healthy individuals.

Later in the same announcement, the FDA gives the details about how good the vaccine is. Here is the start of the key paragraph:

FDA Evaluation of Available Effectiveness Data 

The effectiveness data to support the EUA include an analysis of 36,523 participants in the ongoing randomized, placebo-controlled international study, the majority of whom are U.S. participants, who did not have evidence of SARS-CoV-2 infection through seven days after the second dose. Among these participants, 18,198 received the vaccine and 18,325 received placebo. The vaccine was 95% effective in preventing COVID-19 disease among these clinical trial participants ...

This gives the key point of (relative) effectiveness: it's 95% effective! Hooray, we've got it! See what happens when you keep reading:

... with eight COVID-19 cases in the vaccine group and 162 in the placebo group. Of these 170 COVID-19 cases, one in the vaccine group and three in the placebo group were classified as severe. At this time, data are not available to make a determination about how long the vaccine will provide protection, nor is there evidence that the vaccine prevents transmission of SARS-CoV-2 from person to person. 

First, let's look at the chance of getting covid without getting vaccinated; it's 162/18,325 = 1 in 113. Fewer than 1% of the placebo group got covid! And of those 162 cases, just 3 were classified as severe, so just 1 in over 6100 unvaxed people got severe covid. The numbers to achieve the benefit of vaccination aren't much different. The NNT is over 1 in 110 -- over 110 people had to take the vaccine for one person to avoid getting covid! Yes, the relative benefit is huge, but in absolute terms, less than 1% of people are actually helped by getting jabbed.

Note also that there was zero evidence that the vaccine prevents an infected person spreading the infection.

Is this the case only for Pfizer? A group of French scientists calculated ARR and NTT for the leading Covid drugs, based solely on the published studies of the trials of those drugs. Here is a summary and here is the study published in a scientific journal. It deserves much more attention than it seems to have gotten because of its focus on NNT.

Let's jump right to the key table.

The first drug, Pfizer, has a terrific efficacy (RR), listed there as 0.05, but normally reported as 95%. Everyone (including me, when I first saw it), thinks that means that taking the Pfizer vax means there's only a 5% chance of getting covid, right? It works great! Now look at the NNT, 141. That means that for each 141 people who are vaxed, just one benefits by not getting covid!! It makes common sense: there were 21,728 people in the control group (people who got shots that were placebos), and only 162 of them got covid,

You might think that relative and absolute risk are related, but the third drug, AstraZeneca, makes clear that they're not. AstraZeneca had efficacy (RR) of 0.30, normally reported as 70%, which is dramatically worse than Pfizer's -- why would anyone choose it? But AstraZeneca has an NNT of 83, which means that your chances of the AstraZeneca vax helping prevent covid were much better than the Pfizer vax. But even with the better NNT, chances are extremely high that you wouldn't get covid, with or without the vax.

The issues I describe here are not radical or new. The paper above was notable only in that it covered all the major covid vaccines; other doctors and scientists have publicly pointed out the same facts. For example, here is a note by a doctor published in the BMJ shortly after the trial results were first published.

Conclusion

After learning about efficacy, absolute risk and NNT, your understanding of what it means for a treatment to be "effective" changes radically. Absolute risk and NNT are at least as important. Authorities should discuss all these number prominently.

 

The Medical Treatment of Obesity

In a prior post I asked why there is no search for the origins of the widely-acknowledged obesity epidemic that harms so many people. I suggested that the data shows that there is an obvious cause: the government nutrition recommendations that pervade our society and prominently stated on packaged food. The overweight/obesity numbers started their steady growth shortly after these were promulgated and people followed the recommendations.

Ignoring overwhelming evidence, the authorities continue the health-destroying drumbeat of bad eating advice. Now, the medical people who are charged with dishing out this destructive nonsense are being criticized for making the people who follow their advice feel bad. When will it end??

Smoking

When I was growing up, I saw advertisements and commercials for smoking.

The Marlboro Man was particularly memorable. The ad campaign generated billions of dollars of sales.

The tobacco industry was always concerned about their image; throat irritation from smoking was a well-known side effect, not to mention the growing number of deaths by lung cancer. So ads were created and widely shown claiming the support of the medical profession for smoking, for example:

We know today that smoking causes lung cancer. It wasn't until 1964 that the Surgeon General declared it the cause, and many years passed before other measures were taken. For example, United Airlines was the first to create a non-smoking section of the plane, in 1971. It took until 1990 for smoking to be banned on domestic flights in the US, and later for international flights.

Obesity

So where do we stand with obesity compared to smoking? I would estimate we're at about 1960. The government is hard at work revising the nutritional guidelines most recently updated in 2017, and the drafts that have come out strongly resemble the equivalent for nutrition of what for smoking would be: "smoking unfiltered cigarettes is just fine, but don't smoke too many a day, and make sure you practice breathing exercises regularly to keep your throat and lungs healthy."

As a reminder, the science is solidly behind consuming whole-fat dairy, eggs and meat, while minimizing sugar and carbs. Here is an example of the current version of nutritional insanity:

Sugar-loaded Frosted Mini Wheats and Lucky Charms are better than a whole egg, and ice cream with nuts is better than ground beef. Sure! I wonder what the role of the processed food industry has been in all this...?

Doctors and obesity

Doctors are required to dish out their profession's broken nutritional recommendations to one and all. They are particularly supposed to give good advice to the obese people those recommendations continue to harm. But now there's a new twist -- doctors are being blamed for the on-going troubles of their obese patients!

Obese people are often “weight-shamed” by doctors and nurses — worsening their problem and causing them to wrongfully blame themselves for the condition, according to a new study.

Fat-shaming by medical professionals leads patients to feel humiliated and anxious about appointments — making them more likely to overeat, according to research from the University of London.

Researchers examined 25 previous studies centered on 3,554 health professionals and found evidence of “strong weight bias” — including that doctors and nurses tend to assume overweight people are lazy, according to the report, published in the journal of Obesity Reviews.

“[They] believe their patients are lazy, lack self-control, overindulge, are hostile, dishonest, have poor hygiene and do not follow guidance,” Dr. Anastasia Kalea, who authored the study, told the UK Guardian.

So what should physicians do?

The study concludes that medical professionals should be trained in “non-stigmatizing weight-related communication.”

Tam Fry, the chairman of the National Obesity Forum, said doctors and nurses should take responsibility for the role they play in the UK’s obesity epidemic.

“It is shameful that the condition continues to be regarded by health professionals as being solely a personal problem, little to do with them and it’s disgraceful that they stigmatize patients for being overweight,” said Fry, who was not involved in the study. 

“This is the last thing a patient wants to hear from professionals who they trust will help them.”

It's clear that physicians are stuck between a rock and a hard place. If they dish out their profession's nutritional advice, the obese person will stay over-weight. If they dish out the limit-calories-exercise-more stuff, most people just can't keep it up -- as we know from the obesity numbers. And if they bend over backwards to make sure to avoid giving obese people the slightest impression that their own actions might, just maybe, have something to do with their condition, then they've really blown it! Can't talk and what they're told to talk doesn't work.

Conclusion

Remember what happened with smoking -- the decades it took for the cancer-causing truth about it came out, got proven, and the more decades it took for it to be acted on. We're still in the early innings here with nutrition in general, and saturated fat in particular. We can only hope that sanity and science can move more quickly this time.

Samuel Johnson's Dictionary and Writing Software

Samuel Johnson's famous Dictionary of English was written hundreds of years ago. Nonetheless, it has a great deal to teach us about software.

English is a language, of course, and we know there are numerous computer languages. Each kind of language has words, and it's essential to spell the words correctly so that the reader of what you write knows what you mean to say. Similarly with grammar and usage. When explaining the use of words, it's helpful to give examples. Of course there are more words in human languages than in computer languages, so the book defining a human language tends to be longer, as you can easily see by picking up a copy of Kernighan & Ritchie's The C Programming Language and any English dictionary.

The Dictionary

In 1755, Samuel Johnson published "A Dictionary of the English Language."

His was far from the first such dictionary -- there were already dozens in existence. But his became the most influential dictionary of English for over 150 years.

Much like a top modern programmer, he furiously wrote away with great energy and concentration. He originally claimed he would finish it in 3 years, but it ended up taking him seven. He did everything himself, with just a small amount of clerical help.

Meanwhile, L'Academie Francaise had forty scholars working for forty years to do a similar job for French. Johnson is said to have commented on this:

This is the proportion. Let me see; forty times forty is sixteen hundred. As three to sixteen hundred, so is the proportion of an Englishman to a Frenchman.

In software, there is sometimes talk of programmers who are 10X more productive than the average programmer. Johnson claimed to be more than 500X! When he was "really" only over 200X. Such an exaggerator... Of course in programming, there are people and small teams that perform 100X better than the ancient, lumbering giants "competing" against them.

With such a vast amount of work, it's not surprising that some humor managed to find its way into the work. For example:

Lexicographer: A writer of dictionaries; a harmless drudge, that busies himself in tracing the original, and detailing the signification of words.

Oats: a grain which in England is generally given to horses, but in Scotland supports the people.

Software

If you're a high-productivity programmer, I suspect the parallels have leaped out at you. Here are a couple:

• Programmers have strong opinions, and sometimes express them with wonderful humor. Here are dictionary examples and language opinions.
  
• The productivity difference between a top-down typical big-company MBA-driven software project tends to take 10 to 100's of times more effort than a similar project done by a great programmer or small team. See this and this.
  

The lesson here is clear: when it comes to writing software, be more like Samuel Johnson and less like the French Academy.

Software Programming Language Evolution: Credit Card Software Examples 3

Credit card systems are among the earlier major enterprise software systems written. The early systems were written in assembler language or COBOL. If programming languages really did get more powerful and advanced, you would think that a wave of re-writes would have transformed the industry as card systems written in creaky old languages were streamlined and turbo-charged by being written in more modern languages. Generations of industry executives and technical experts and leaders have thought exactly this.

The earlier posts in this series have described two major such efforts that ended in face-plants, and other efforts that illustrate the ongoing power and productivity of supposedly decrepit approaches. In this post, I'll describe a couple not-famous examples of advances that actually took place.

Clarity Payment systems and TSYS: the first version in Java

In the late 1950's a group inside a local bank, Columbus Bank and Trust in Georgia, started building a system for processing credit cards. The division went public and eventually became known as TSYS, Total Systems, which is now one of the world's major card processing companies.

During the early 2000's a special kind of card with limited functionality called a pre-paid debit card started to be used. Unlike a credit card, which you use to make charges and then later pay it off or use revolving credit (called "pay later"), a prepaid debit card is just what it sounds like: you first pay in some money and then can make charges using the card until the amount you put in runs out (called "pay before"). This kind of card is vastly easier to implement in software than a credit card, but is still complicated because of all the bank and card interfaces. See this for more.

Meanwhile a small company called Clarity Payment Solutions had created a working prepaid card system. The technical founder of the company had bought into the rhetoric around the Java enterprise Object-oriented language, and had constructed the code using it. What everyone believed was that basing a program on objects would make it tremendously more flexible and easier to change than using traditional languages. Objects were thought of as being like Lego blocks with super powers, enabling you to pick out the ones you like and piece them together easily. A feature called inheritance promised the ability to make minor changes without much effort and no side effects.

The executives at TSYS needed to get into the rapidly growing market for prepaid debit cards. They put some effort into having their staff modify their internal systems to meet the need but weren't getting anywhere fast. When they encountered Clarity Payments they were pretty happy -- it's what we need! It's already working in the market! And best of all, it's written in Java, so we'll be able to make changes easily without all the trouble of systems written in prehistoric languages like the one we're stuck with! They bought the company.

The technical leader of Clarity was sobered by his experience of writing the software for prepaid debit. It was a lot harder than he thought it would be, and the ease of making changes because of the object orientation of Java proved to be little but hollow rhetoric. He had proven to himself that it was all b.s. through hard personal experience, learning what others have learned. He now had years of practical experience building a production system to make clear to him what the real obstacles were. He was glad to sell off the company and not have to struggle with the code any more. TSYS was welcome to it!

TxVia and Google: the second version in Java with help

The technical founder of Clarity set about building the software infrastructure he would have liked to have had when building the Clarity code in the first place. Java by itself didn't solve the problems. It needed lots of help, and he was going to build the system that would give it help. This is a response that some smart programmers have when they get their noses rubbed in the broken promises of some new programming fad. It led this fellow to build a system that took a significant step up the hierarchy of abstraction, as I describe here. Java would remain the core language, but given a huge practical boost by having the ability to make changes built in at critical points using a kind of workbench approach with cleverly chosen "user exits" to enable safe customization. Eventually he turned the power of his new system to building prepaid debit card software, whose requirements he understood so well. Reconnecting with his old business partners, they went into the same market again and started getting some real business.

Meanwhile a team at Google had been working on the same problem.They wanted a Google implementation of prepaid debit card functionality for the new Google Wallet. The leaders took at look at prepaid debit card functionality and immediately felt it was no big deal. It's nothing but putting money into an account and checking withdrawals to make sure they was enough money.  Adding and subtracting and a few interfaces. No biggie. But just to be safe they assembled a crack team of nearly 100 Google-level geniuses and put them on the job. They used languages and tools that were generations ahead of standard Java.

A year later they still had nothing working. It turned out to be harder than they had thought, even with the astounding power and flexibility of Google software resources. When one of the leaders heard about TxVia, he insisted on giving them a look. A group of Googlers came to the TxVia offices and threw down the specs of what they were trying to build on the table. They sneered, we heard you guys were real smart; our managers tell us you've gotten a lot further than we have. Sure. If you're so smart, prove it by making a system like what's in this document work.

Shortly after the TxVia team came back with -- a system that met Google's requirements. That the Google team had spent a year failing to meet. Skipping over the emotions of everyone involved, which pretty much covered the gamut from embarrassed to ashamed to denial to exultant to you-can-imagine, Google bought the company. It became a key part of Google Wallet.

This was a clear demonstration that Java or other modern languages are NOT the determining factor in programmer productivity and software effectiveness.

The Paysys Corecard system and Apple

I told the story of my time as CTO at Paysys in the late 1990's here, including the sale of its millions of lines of COBOL code to First Data. I gave details about how Paysys became a powerful player in the market for credit card software by increasing their breadth of automation here.

While I was CTO, after studying the COBOL code in detail with lots of help from the people who had written it, I came up with a way to re-create the system's functionality. The idea was to implement a core set of concepts written in a small amount of abstract code and then build extensive metadata as needed to support the product’s existing functionality and more. I wasn’t fully aware of it at the time, but I used the method described here and in the linked posts. It was written in C++ (mostly the C subset) and ran of a network of servers. One of the big national consulting groups ran tests that verified it could handle tens of millions of cards with linear growth. In addition a team at First Data in Omaha ran the code and modified the metadata to make it match the functionality of their existing system written in assembler language. The trouble they had modifying the assembler language to handle a variety of requirements already met by the Paysys COBOL code was the main reason they were buying Paysys. They decided they would really like to have the new code as well.

While his team urged the CEO to include the new code in the purchase, he decided he didn’t need it, and kept it out of the deal. The COBOL code solved immediate problems like supporting cards in Japan, and who cared what a bunch of nameless programmers babbled about the speed of making future changes?

Thus it happened that when First Data bought the Paysys VisionPLUS COBOL code in the year 2001, the new metadata-based system was left out of the deal and stayed with the remainder company, now called Corecard.

Years went by. Some leading people began to notice Corecard because they could make it do thing unanticipated things much more quickly and easily than with normal procedural systems. Then Apple decided to get into the credit card issuing business. Not the cheap and easy pre-paid kind, the full-featured, tough credit card kind. They took their requirements to the usual suspects who gave them the usual lengthy go-live times with the usual astronomical custom programming fees. Somehow they talked with Goldman Sachs, which had connected with Corecard through a small, adventurous group. They could get the job done, quickly and efficiently, when no one else in the industry could come close. A deal got done and the Apple card came out quickly, doing everything Apple wanted. And scaled quickly.

Normal parameters could not have accomplished this. The TxVia workbench approach didn't have even a fraction of the functionality needed. Only a software system that went far beyond the capabilities of procedural languages of any generation could have met the challenge. In the end, languages of ANY generation can only do so much. They're like prop planes. If you want to go REALLY fast you need a rocket engine, and that's what meta-data-based systems are.

One of the sobering lessons here is the very basic human one: No one will want a rocket engine unless they're trying to build a rocket. If you build a rocket engine, however powerful it may be, people will look at it, scratch their heads, express mild amazement, but walk away -- they don't need it. And won't until they decide they want to build a rocket.

Conclusion

These real-life examples demonstrate the limits of normal procedural languages, no matter how modern and fancy. They demonstrate how taking even small steps up the ladder of abstraction can yield amazing gains, as they did for Google, at least after they bought TxVia. And finally they demonstrate that there's a whole quantum leap further you can go to meet software requirements beyond what procedural languages alone can handle -- but no one will want to buy them until they have a problem that nothing else can solve.

Blood Pressure Pills can make you Blind

As a direct result of ridiculous, anti-scientific standards, pills to lower blood pressure are the mostly widely prescribed pills in the US, with over 100 million people supposedly cursed by the “disease” of hypertension. Did you know that there’s a never-refuted medical study published by the American Academy of Ophthalmology and sponsored by the National Eye Institute (part of NIH) showing that taking those pills greatly increases the risk of going blind? I didn’t think so.

AMD: Age-related Macular Degeneration

More than 11 million people in the US have this disease. It mostly affects people 60 and older. The most common variety of it – dry AMD – is progressive and has no cure. Eventually it leads to complete loss of vision. Here is the NEI description of the disease, its causes, prevention and non-cures. You will notice that there is NO mention of blood pressure medication.

I have described the largely suppressed side effects of blood pressure medication, and my path to freedom, with the result that I'm not taking the pills and I'm healthier. Two years ago I was diagnosed with early stage AMD. After resolving the issue with harmful blood pressure pills, I decided to see if the pills also impacted AMD. While it wasn't too hard to find out about the side effects of blood pressure medications, including the ones related to heart health I experienced, I hadn't seen anything about vision in general, much less AMD. I decided to look harder.

I mostly found things like this from the Cedars-Sinai website:

In other words, they don't really know. And they clearly state that "uncontrolled high blood pressure" -- in other words, failure to take blood pressure medicine when you "should" -- is a cause.

OK, let's go to the professionals. the American Academy of Ophthalmology. What do they say about blood pressure drugs and AMD?

This blows me away. The very first risk factor they list is the garbage about saturated fat. Totally wrong. This is the cornerstone of the explosion of obesity that harms so many and has nothing to do with AMD. I'm suspicious. Scanning down the list, I see one of the causes they list is "have hypertension (high blood pressure)." Not "treating" it or "taking blood pressure medications," but simply "have" it. In the linked article about high blood pressure, they simply declare that it can lead to big trouble, and "can cause permanent vision loss." OMG! I'd better start taking pills to get my blood pressure under control!

I guess it's clear. Whatever the cause of my AMD, it can't be the blood pressure pills I took for eight years.

The Beaver Dam Eye Study

Stubborn guy that I am, I kept looking. I found a little eye group in the DC area that promotes its services. I found them because my search engine surfaced two closely related blog entries on the site, one of them titled "The Link between Blood Pressure Drugs and AMD," a close match to my search string. Score! The second sentence of the post is: "If you take medication to lower your blood pressure, it’s important to know that you could be increasing your risk of developing AMD, or age-related macular degeneration." The bold was in the original!

Both blog posts give a reference to the 2014 study and extract some details, all of which I have verified. Here is the attention-grabbing sentence from the blog post: "For residents who were not taking blood pressure drugs, only 8.2 percent of them developed early AMD. For residents who took medication for high blood pressure, nearly 20 percent of them developed AMD."

The chances of getting AMD were more than doubled by taking the drugs.

Here are the highlights of the study.

T

In short, thousands of people in a Wisconsin town were followed over 20 years, tracking their use of blood pressure medication and the incidence of AMD. Here is the conclusion at the top of the paper:

Conclusions: Use of vasodilators is associated with a 72% increase in the hazard of incidence of early AMD, and use of oral b-blockers is associated with a 71% increase in the hazard of incident exudative AMD. If these findings are replicated, it may have implications for care of older adults because vasodilators and oral b-blockers are drugs that are used commonly by older persons. Ophthalmology 2014;121:1604-1611 ª 2014 by the American Academy of Ophthalmology.

Whatever the chances of you getting leads-to-blindness AMD are, you increase them by about three quarters by taking widely-prescribed blood pressure pills. Still think lowering your blood pressure is worth it, particularly considering the proven facts I describe here?

So where are the headlines? Where are the cautions about the vision-killing side effects of blood pressure drugs? Where are the follow-up studies? Where are they on the websites of major public and private healthcare organizations? Nowhere, that's where they are. Nowhere!!

It's clear that this isn't just ignorance. It's suppression. Just above I showed how there's no hint of a problem with blood pressure pills on the official AAO website. When I did a full search on Google for "AMD blood pressure," instead it showed me results for "And blood pressure." I corrected it and mostly found propaganda, but did find a reference to the Beaver Dam study. When I used my favorite non-Google search engine, which I like because they don't have thousands of engineers hard at work adding bias to the results, the very first result was a direct link into ... the AAO website! ... to a news item about the Beaver Dam study! The Expert-fueled AAO organization put a brief post on their site about the study, but failed to mention it anywhere else! Not only that, when you use their embedded (Google) search facility on the site, their own post fails to appear in the results!

Why do you suppose that is? Pharma money? What about the ethics of the healing profession, not to mention their self-respect? Given the near-total suppression of the information, I suppose simple ignorance could explain the actions of most providers, along with "standards of care" that demand regular taking of blood pressure and prescribing medications according to standards. Which are wrong, not to mention destructive.

I paid to get a copy of the full study. It had important information not included in the brief summaries. Look at this extract from Table 4 near the end of the paper:

The first line is the one often quoted. Let me show the math. Of the 2714 people in the study, 295 of them (more than 10%) got AMD because they were taking the BP pills.

I took two pills for eight years. One was Amlodipine, a calcium channel blocker, which in the study nearly doubled the chance of getting AMD. I also took Losartan, an ARB, which had zero percent AMD -- not because it was innocent, but because as shown in an earlier table, almost none of the participants took it. It could be awful, but the study was too small to know.

An earlier table also showed the incredible extent of BP medication use. About a third of the participants in the youngest age group (under 64 years) took medications, while over two thirds of those over 85 were taking them. Most of whom shouldn't have been taking them at all! I  wonder, just wonder, if this could have something to do with the increasing incidence of AMD with age -- you think that's a possibility that should be studied?

Conclusion

I used to think that the pharma and the industrial food industries make mistakes, like any industry, and you have to take the good with the bad. There is certainly some good. But the more I learn, the more I discover the all-too-widespread shameless self-dealing of the industries, strongly supported by government agencies and professional authorities. They force through regulation putting misinformation on our food and our diets in hospitals, and are making billions of dollars selling pills that are standard procedure for preventative care that, instead of keeping us healthy, actively make us sick -- even to the point of making us blind -- along with numerous other problems I have briefly touched on in prior posts.

 

The Dimension of Software Automation Breadth Examples

One of the major ways software evolves is by increasing along the dimension of automation breadth. A domain can be dominated by products at a given breadth of automation, and suddenly a existing or new competitor starts winning by increasing its breadth of automation, offering its customers more value for less effort and money. It's a classic move and a good way for new entrants to disrupt a market.

One of the most frequently given pieces of advice, including by me, is to “focus,” i.e. basically solve fewer problems, try to satisfy a narrower range of customers, etc. While this advice is applicable more often than not, the natural and recurring progression of products through the spectrum of “automation breadth” makes it clear that, sometimes, when the conditions are right, the winning strategy is to be among the first to increase the breadth of automation that you incorporate into your product.

Example: Athena Health

A clear example of this is shown by the story of AthenaHealth. At the time the founders started the company, a wide variety of products were already available to run physician offices, from small single-office practices to extended medical groups. These products generally ran on inexpensive machines that the practice would keep in some back room, and would support multiple users via a LAN or terminals. Most of the products were sold by license, so that the office had to pay only a modest price for the license, and then annual maintenance.

Along comes little Athenahealth, with a better way of doing things. Athena had a cool new practice management system (PMS). Unlike all PMS’s at the time, it was built using internet technologies, so that it could be operated as a service, with people at the office accessing the system using machines with browsers and internet connections. Athena took care of the computers, relieving the medical office of a burden it basically didn’t want.     

But they ran into a little problem: the people in charge of medical practices are doctors, and doctors really don’t care about PMS’s – they care about patients and medicine. A PMS is a necessary evil, something you should buy for as little as you can get away with and ignore until things get so bad you are forced to buy a new one. Money spent on the PMS is just money out of the doctors’ pockets, as far as they’re concerned. Oh, you have a “better” one, do you, whatever that means? Stop wasting my time.

The folks at Athena noticed that one little thing the PMS does is produce bills and claims, the purpose of which is to get patients and insurance companies to send them money. No claim, no money. Unfortunately, merely producing the claims rarely proves to be sufficient to get the money flowing. People are required to do special things to the claims, provide additional information, harass the payers, etc. This is so specialized and time-consuming that it either consumes the time of a number of people at the office, or is outsourced to a “billing service.”

The Athena folks went on to notice additional important things: (1) the chances of getting paid are a direct reflection of the quality and appropriateness of the information on the claim; (2) the PMS and how it is used is the main source of this information; (3) by actually performing the billing service, you can learn how to produce a better PMS that produces better claims, increasing the effectiveness of the billing service while reducing the cost of running it at the same time. Finally, they found out that there is something the doctors who run medical practices care about other than medicine – surprise, surprise, that something is money.

So Athena introduced an outsourced billing service, but required practices that use it to also use their practice management system – at no additional cost! And they got so good at collecting the money that doctors could essentially get more money and a really cool, state-of-the-art PMS (like they cared…) for free!

This is a nice story for Athena, but the point of telling it here is that it illustrates the principle of product automation breadth evolution. While products are evolving within a “level” of automation breadth (i.e., how many of an organization’s functions it automates), it is normally a good idea to maintain discipline, avoid distractions, and concentrate on automating that function. But at a certain point in the evolution of each product category, pretty much everyone in a space has automated everything within that function, and everyone is reduced to concentrating on sales strategies and niggling little details. At that point, and PMS’s were at that point when Athena came along, it makes sense to do what you’re normally supposed to avoid – look for another function inside the organization to automate, particularly if there are synergies in implementing the two functions within a single framework, as there certainly were in this case.

Example: Bank and Retail Credit Cards

In 1983 a small company called CCS (Credit Card Software, later called Paysys) released a body of COBOL code that would enable a bank to process credit cards. A number of small and regional banks bought copies of the code and ran it successfully. The code was enhanced over the years.

A major retailer, Michael's Jewelers, approached CCS and asked if they could make a version of the bank software that could handle purchases from their stores, including a variety of payment plans and financing options offered by the store that were not supported by bank card software.

The company's programmers quickly gave up on the idea of modifying the bank code to handle the problem. Many aspects of bank card processing, such as the difference between issuing and acquiring, were irrelevant to retail. In addition, the many financing options supported by retailers went far beyond anything banks did. So they borrowed from the bank code to the extent that it helped and ended up creating a separate body of software called Vision 21. Once it was available, it proved to be a big success in the market, and was quickly enhanced by customer demand to include all the options desired by retailers Before long it supported the needs of retailers in other countries as diverse as Japan and South Aftrica.

Finally, there was very large processor, Household International, that was running multiple copies of both products, separate because they had been customized for a variety of reasons, for example to support methods of credit that were unique to a market (for example “hire-purchase” in South Africa). While CCS, now called Paysys, had failed to create a generic bank/retail product when confronted with an example of the generic problem, unifying multiple bodies of related code into a single, highly parameterized code base proved to be a far more tractable problem, particularly with a single important customer who insisted that these variations were the only ones to worry about.

The industry quickly rallied to this new product, called Vision PLUS, that could be directed at so many different problems with such relative ease. For example, it enabled retailers to issue co-branded cards that worked like regular bank cards, except when used in the issuer's retail store, when it acted like a classic store card with features like "90 days same as cash" options that bank cards don't support. While “parameterized product” may sound like an abstract concept, it translates directly into business advantage compared to more primitive product types, by enabling the product to be customized, installed, upgraded and maintained with less labor, less time and lower risk of error.

The company that built Vision PLUS was bought by First Data, a major card processor. The reason is interesting: in spite of having thousands of programmers (Paysys had only dozens), First Data was unable to modify their US-centric code base to handle processing in Japan. At the time of the sale, Vision PLUS was processing about 150 million cards world-wide. The code lives on and currently runs over 600 million cards.

Conclusion

These are two examples of companies growing by broadening their focus -- in a strategic way, driven by just a couple of representative customers. In neither case did they address a whole new market at the beginning, though that was the eventual goal. They took their existing software along with a cooperative customer and met that customer's needs. Athena started with just one specialty in one state with a single payer. Paysys started with a single existing customer. In each case they grew the broadening of their focus a step at a time, making each customer happy as they went. As they grew, word got around in the industry, and they shifted to saying "no" to the vast majority of inquiries in order to maintain the step-wise customer success they were building.

This is a classic pattern of focus broadening that can bring transformative success to companies when handled well.

 

The Destructive Treatment of Hypertension

I’ve talked about how all the medical authorities are united in the importance of fighting the “silent killer” of blood pressure that’s too high, i.e., hypertension. I’ve described in detail that what doctors call ‘essential hypertension” is NOT a disease. Fighting the non-disease of hypertension is an ongoing bonanza for doctors and the drug companies while leading to serious problems for patients.

In this post I’ll describe my personal experiences that led me to these observations. What happened to me was not unusual, and others have had it worse than I have.

Getting Cancer and Hypertension

Eight years ago I developed a rare form of cancer, desmoid tumor, of which there are about a thousand cases a year. I was treated with drug infusions. The drugs sometimes have bad effects on the heart, so I received tests and a consultation with a NYC cardiologist. She told me I had high blood pressure that must be treated immediately. I was surprised since my reading had always been low, but complied, ending up taking daily doses of Amlodipine and Losartan.

This was unusual for me, because I normally dive in and check for myself everything that’s important to me. I didn’t in this case. I was consumed with my study of my rare cancer and the ineffective early advice I got. I found the one doctor in the country who knew how to treat it. The blood pressure seemed like a bump in the road at the time. My bad.

I kept up with the daily drugs after that, with new prescriptions issued by my primary care doctor with minor adjustments. Not once did any doctor mention anything about side effects. I felt OK and did no research.

About a year ago I started monitoring my blood pressure myself because I began experiencing symptoms it was hard to put a finger on. I knew the drugs I was taking were generics and had discovered the widespread corruption of generic drug makers and the ineffectiveness of the quality monitoring conducted by the FDA. I asked my primary care doctor for prescriptions for the branded versions of the drugs, which I hoped were more carefully monitored. I discussed in detail with my local CVS pharmacist, who ultimately was unable to get the drugs.

Symptoms of Heart Trouble

The symptoms increased. In March 2022 I had a tough time driving with symptoms that included being light-headed and a heart pulse rate that was high for me, as though I were exercising. I went to my primary doctor who gave me some tests including an EKG. With inconclusive results, she referred me to a cardiologist. The cardio guy gave me lots of tests, including an electro-cardiogram, a nuclear stress test, a week-long Holter monitor. This all took a few weeks.

Meanwhile, I did what I should have done eight years ago – dove in and studied heart function and blood pressure. It didn’t take long for me to discover – surprise! – that the symptoms I experienced were the same as side effects of the drugs I was taking, and were widely reported by patients online. I tried to get FDA data on them and discovered the great lengths the FDA goes into order to keep drug adverse reactions as secret as possible – kind of like the way medical offices say you have full access to your medical records, except that they prevent it, as I have described in detail.

I brought up the subject on my next visit with the cardiologist. He immediately dismissed the possibility. He refused to discuss it or take seriously the possibility that my symptoms were due to the drugs that he and all the other members of his profession profusely prescribe.

After that I took matters into my own hands. I stopped taking the drugs after the last test was conducted, in order to avoid confusing the results. I continued daily blood pressure readings, sometimes more often.

I finally got the results of all the tests. Nothing was wrong with me – except of course when you monitor for seven days straight, sometimes your heart beats fast. It’s scary! It’s called supraventricular tachycardia (SVT). Once or twice a day, for a dozen beats at a time and on the low end of the "fast" scale. Call 911! My valves are fine, no blockages, no Afib, etc. etc. His recommendation? Consult one of his friends to get either a pacemaker or a six hour operation to zap random bits of my heart in hopes that the scary SVT would go from 0.001% to zero. Maybe. NFW, thanks anyway, esteemed board-certified cardiologist.

My first step after stopping the drugs was to start taking a well-reviewed natural heart-health additive based on L-Arginine. After 3 weeks I was better, but not satisfied. So I just stopped messing with my body and its extremely complex mechanisms. After my body cleared out, I was much better.

The blood pressure numbers are interesting:

Average        Systolic         Diastolic

with drugs     137              64

L-Arg only     157              74

nothing          144              65

Taking no blood pressure or other drugs resulted in Diastolic numbers that were unchanged and Systolic numbers that were 7 points higher, well within a healthy range, though not according to current cardiologist drug-pushing fashion.

Side effects of blood pressure control drugs

There are lots of non-government places to learn about the side effects of the awful blood pressure drugs -- thanks, internet! No thanks at all, cardiologists! -- and even published studies that show 10% of participants in studies dropping out due to the intolerable side effects.

Here are a few samples of problems with Amlodipine from a data-rich site.

fast, irregular, pounding, or racing heartbeat or pulse

Common (1% to 10%): Palpitations, ankle edema

Amlodipine has an average rating of 3.7 out of 10 from a total of 571 ratings for the treatment of High Blood Pressure. 20% of reviewers reported a positive experience, while 61% reported a negative experience.

41% gave it 1 star out of 10

Common in reviews below: “feeling lightheaded, heart palpitations and arrhythmia”

After 35 years of taking it, “I took myself off of 2 years ago, but could not get through the withdrawals. It caused my heart to feel like it was beating out of my chest”

After starting “suddenly I felt dizziness I went to ER they admitted me to the heart hospital. I was told I needed a pacemaker. I declined.”

After strong heart beat “I attended a pre-operative assessment where I was given a routine ECG and this confirmed that I had become tachycardic while taking amlodipine. My pulse was racing at over 100 BPM”

After stopping “Five weeks later I still have the tinnitus”

After 6 years, “worse side effects have been; heart pounding/palpitations, fatigue, and increased anxiety…. I stopped taking the Amlodipine Besylate 10 mg. over 3 weeks ago and and have noticed that my energy level has increased, anxiety lessened and heart pounding decreased.”

After 18 months “… anxious. Couldn't sleep, couldn't concentrate. I noticed muscle tics all the time, heart palpitations, more joint pains, memory loss and more. …I had myself convinced I had contracted some fatal condition (ALS, MS, etc...). After every specialist I could find, we decided it was anxiety. Then one day I read someone's account of anxiety and amlodipine. …try a switch. Today I am back to my old self.”

Conclusion

I have re-learned one of life’s most important lessons: if you want to be healthy, take charge of your own health. It’s your health, no one else’s. You own it, you have to live with it. There are experts and authorities all over the place who are lined up to tell you what to do. They want you to pay them, take drugs and undergo invasive procedures. Most of these people are highly trained and well-meaning. They sometimes know things that are worth knowing. They can be of great help. It’s worth listening. But it is not worth mindlessly following their orders, because their profession’s best, standard advice is all too often wrong. WRONG. And not just wrong – actively harmful.

Once I took my health into my own hands eight years ago, I found a truly expert doctor who brought my nasty rare cancer into remission, a place where I hope it’s happy. On the path, I stupidly and without examination followed doctor’s orders about blood pressure, following advice given to nearly half the population of the US. How could it possibly be bad? Easy. The same way the nutrition advice given to ALL the US population was and remains highly destructive, leading to the ongoing obesity epidemic and widespread avoidable suffering. The same way bad science about blood cholesterol has led to the most profitable drug in pharma history, treatment which shortens lives and makes patients less healthy.

The good news is that you’re not alone. There are dedicated people devoted to discovering and putting out the facts so that diligent, self-reliant people can find out what’s best for their health, most importantly for those cases where the medical profession stubbornly clings to destructive error, as it has so often in the past. It’s your health. Own it!

How to Integrate AI and ML with Production Software

Most enterprises that build software are proudly flying the flag of AI/ML. "We're technology leaders!" their leaders crow in annual reports and at conferences. At the same time, any objective observer usually sees a lack of common sense in the operation of the company's systems. It often appears that, far from needing beyond-human artificial intelligence, they could use some insect-level functioning instincts that get things done. What's going on? Can it be fixed?

The Industry-standard way to fix the problem

The usual fix to the problem is to completely ignore the fact that there's a problem in public, while following something like these proven strategies:

• Brag, loudly and often, about you and your organization's commitment to AI/ML. The commitment is serious; it's deep and it's broad!
• Talk about the initiatives you've funded and the top experts you've hired.
• Talk about the promising things you've got in the works.
• Use extra phrases to demonstrate your seriousness, things like "1-to-1 personalization" and "adaptive processes" and "digital-first transformation."
• Put your top executives with fancy titles out there to follow the same strategy, using their own words.

I've given a detailed example of how a top healthcare insurance company follows this strategy while operating at a sophistication level that is best described as "hey, this electronic mail stuff sounds neat, let's give it a try."

Sometimes one of these organizations puts something in practice that works. It typically takes a great deal of time and effort to find and modify the relevant production systems. The efforts that are mostly likely to make it into production are those that can be done with the least amount of modification. For example, minimal-effort success can sometimes be achieved by extracting data from production systems, subjecting it to AI/ML magic and then either feeding a new system or making it effective with just a couple of insertion points.

The Obstacles to AI/ML Success

The obstacles to AI/ML success have two major aspects:

• The typical practice of leap-frogging all the predecessors to AI/ML to maximum sophistication.
• The extensive, incompatible existing production systems into which AI/ML power has to somehow be inserted.

A good way to understand these obstacles is to imagine that you're in a world in which boats are by far the most important means of bulk transportation. In other words, the world in which we all lived at the start of the 1800's. Suppose by some miracle a small group has invented nuclear power and has decided it would be a great way to provide locomotion to large boats instead of the sails and wind power then in use. What prevents the amazing new technology from being used?

Easy: the boats were designed for sails (with masts and all that) and have no good place to put a nuclear engine, and no way to harness its power to make the boat move.  The strong steel and other materials required to make a turbine and propellers doesn't exist. You can demonstrate the potential of your engine in isolation, but making it work in the boats available at the time won't happen. You can spend as much time as you like blaming the boats, but what's the point?

The solution is clear by studying boat locomotion: there were incremental advances in boat materials and design, and the systems used for powering them. Paddle wheelers have been around for over a thousand years. Here's a medieval representation of a Roman ox-powered paddle wheel boat.

For serious ocean travel, the choice became the large sail boat, as in this painting of boats near a Dutch fortified town:

Suppose you had a nuclear engine of some kind and were somehow able to make it with materials that were generally not available in the 1600's. How would you use it to power the sail boat? The very thought is ridiculous. The problem is that the boats have no way to accept or utilize the nuclear engine.

How to overcome the obstacles to AI/ML

What would a sensible person do? Exactly what real-life people did in history: incrementally make boats suitable for more powerful means of locomotion, and make more powerful means of locomotion that would make boats go more quickly. Practically. You know, in real life.

That means, among other things, once steam power was created, gradually make it suitable for powering ships with sails -- using the sails to conserve coal when the wind was strong, and using coal to power paddles when the wind wasn't blowing. Then, after materials advanced, invent the screw propeller -- which didn't happen until the late 1800's -- to make things even better. Eventually, the engine and the ship would converge and be suitable for the introduction of nuclear power.

This is an excellent model for understanding how to overcome the obstacles to powering existing enterprise applications with AI/ML:

• The AI/ML can only be jammed into existing systems with great effort and by making serious compromises.
  • With a few exceptions, simpler methods that can make real-life improvements should be devised and introduced first, with the portion of AI/ML gradually increasing.
• The existing enterprise applications are like wooden sailing ships, into which generation-skipping advanced locomotion simply can't be jammed.
  • Evolve the applications with automated decision-making in mind, first putting in simple methods that will produce quick returns.
  • The key to AI-friendly evolution is to center the application architecture on metadata in general, and in particular with metadata for workflow.

The important thing is this: increase the "intelligence" of your applications step by step, concentrating on simple changes for big returns. Who cares whether and to what extent AI/ML is used to make improvements? All that matters is that you make frequent changes to improve the effectiveness, appropriateness and personalization of your applications. Experience shows that relatively simple changes tend to make the greatest impact. See this series of posts for more detail.

 

The Facts are Clear: Hypertension is not a Disease

The medical community, organizations and government agencies couldn't be clearer: hypertension (high blood pressure) is a silent killer. You may not feel anything wrong, but if you've got it, your risk of strokes and heart failure goes way up. Therefore it's essential to monitor and treat this deadly condition.

They're all wrong. Hypertension is not a disease that needs to be cured. It may be a symptom of a problem, but not a problem itself, just like fever is a symptom, not the underlying problem. By treating it as a disease and giving drugs to lower blood pressure, the medical establishment makes patients less healthy and raises costs substantially. With a few exceptions, we would all be better off ignoring blood pressure and most of the associated advice.

Drugs for "Curing" Hypertension

The single most prescribed drug in the US is for lowering cholesterol. But most prescriptions for a disease are to reduce blood pressure.

Here's the story with blood pressure pills.

In fact, a majority of the most prescribed drugs in the U.S. are used to treat high blood pressure or symptoms of it. That’s because 108 million or nearly half of adults in the U.S. have hypertension or high blood pressure.

Is Hypertension a Disease?

There is no doubt that blood pressure can be measured and that it varies greatly. What is hypertension? As I describe here, currently it's a systolic pressure reading above 120 (until 2017 it was above 140). There are lots of things you can measure about people. What makes this measurement bad?

There's a clue buried deep in Doctor-language, a clue that is nearly always missed -- but it's one that doctors with a basic education should know. The official name for high blood pressure is essential hypertension. What's that? Let's ask Dr. Malcolm Kendrick, a long-experienced cardiologist:

At medical school we were always taught – and this has not changed as far as I know – that an underlying cause for high blood pressure will not be found in ninety per cent of patients.

Ninety per cent… In truth, I think it is more than this. I have come across a patient with an absolute, clearly defined cause for their high blood pressure about five times, in total, and I must have seen ten thousand people with high blood pressure. I must admit I am guessing at both figures and may be exaggerating for dramatic effect.

Whatever the exact figures, it is very rare to find a clear, specific cause. The medical profession solved this problem by calling high blood pressure, with no identified cause, “essential hypertension”. The exact definition of essential hypertension is ‘raised blood pressure of no known cause.’ I must admit that essential hypertension certainly sounds more professional than announcing, ‘oh my God, your blood pressure is high, and we do not have the faintest idea why.’ But it means the same thing.

Hypertension = your blood pressure number is high. Kind of like having a high temperature, which we call a "fever," right? Wrong. When you get a fever, doctors first make an effort to determine the cause of the fever! What an idea! The fever is a clue that something is wrong, not the problem itself! Here's the real, bottom-line clue: When you treat fever you treat the underlying cause e.g. bacterial infection, NOT the fever itself! If we treated fever the way we treat hypertension, we would give drugs whose sole purpose was to lower the body temperature, ignoring the underlying bacterial infection that caused the fever. Wouldn't do any good! Maybe we'd sweat less, but the bacteria would rage away inside our bodies. But high blood pressure? Doctors ignore the cause and "treat" the symptom, which can often do more harm than good -- except of course for the drug makers, who make out just fine.

Makes me sick.

Causes of hypertension

From Kendrick:

So, why does the blood pressure rise in some people, and not in others. It is an interesting question. You would think that, by now, someone would have an answer, but they don’t. Or at least no answer that explains anything much.

Just as fever is caused by an infection (or something else), could it be possible that hypertension results from some underlying problem? Kendrick again:

Looking at this from the other direction, could it be that cardiovascular disease causes high blood pressure. Well, this would still explain why the two things are clearly associated, although the causal pathway may not be a → b. It could well be b → a.

I must admit that I like this idea better, because it makes some sense. If we think of cardiovascular disease as the development of atherosclerotic plaques, leading to thickening and narrowing of the arteries then we can see CVD is going to reduce blood flow to vital organs, such as the brain, the kidneys, the liver, the heart itself.

These organs would then protest, leading to the heart pumping harder to increase the blood flow and keep the oxygen supply up. The only way to increase blood flow through a narrower pipe, is to increase the pressure. Which is what then happens.

Over time, as the heart is forced to pump harder, and harder, the muscle in the left ventricle will get bigger and bigger, causing hypertrophy. Hypertrophy means ‘enlargement.’ So, in people with long term, raised blood pressure, we would expect to see left ventricular hypertrophy (LVH). Which is exactly what we do see.

He goes on to give lots of detail about how this takes place, if you're interested.

Correlation and Causation

There's a little problem that everyone who knows about science and statistics is supposed to know. It's the difference between correlation and causation. Two things seem to happen at the same time. They are correlated. No problem. But does one of the cause the other? That's a whole other thing, and it's super-important. At McDonald's, burgers and fries are often seen together. They're correlated. Did the burger cause the fries? Fries cause the burgers? Nope. They're just listed together on the menu and lots of people like them together.

How about knife cuts and bleeding? Definitely correlated. Causation? By looking at repeated cases of knives making cuts, you can determine that putting a knife into someone's skin nearly always causes bleeding.

This is the problem at the heart of hypertension -- except perhaps in extreme cases, hypertension can be correlated with heart attacks and strokes -- but it can't be shown to cause them in the vast majority of cases.

The range of blood pressure

The authorities don't like to talk about this, but blood pressure varies HUGELY not just from person to person, but also by age and for a single person during the day!

Here's something to give you the idea from a scientific paper:

The range of pressure for a single person can be rather larger. I just took my pressure this morning. The systolic was 126. In the previous days the readings were 159 and 139.I have taken my pressure with different devices over a year, and that variation is not unusual. It can vary that much in a couple hours, depending on my activity level.

It is well-known in the medical community that blood pressure varies naturally with age, generally rising as you get older. Has anyone documented this statistically? If they have, I can't find it. Generally, what is normal is roughly 100 plus your age, so a 50 year old man would have 150, roughly 10 less for women. Here is an interesting description of the age factor from a former NASA astronaut and doctor.

The assumed causation fails to hold

A surprising amount of modern medical misinformation goes back to the diet-heart hypothesis put forward by Ancel Keys and supported by the seven countries study. It's what led to the obesity-causing fat-is-bad diet recommendations and the ongoing harm of reducing blood cholesterol using statins. Out of the same witch's brew came the notion that high blood pressure causes heart disease.This notion was supposedly locked down by the famous Framingham study, which continues to this day.

In the year 2000, the edifice crashed when a careful review was published in the journal of the European Society of Cardiology, "There is a non-linear relationship between mortality and blood pressure." It includes references to the original Keys study and many following journal articles.

The article is prefaced by a quote that is so appropriate, I can't help but share it with you:

"For every complicated problem there is a solution that is simple, direct, understandable, and wrong." H. L. Mencken

The authors start by explaining the current paradigm:

"the relation of SBP (systolic blood pressure) to risk of death is continuous, graded and strong..." The formulation of this "lower is better" principle ... forms the foundation for the current guidelines for hypertension.

They point out that Ancel Keys himself concluded that "the relationship of overall and coronary heart disease death to blood pressure was unjustified."

They went on to examine the detailed Framingham study data.

Shockingly, we have found that the Framingham data in no way supported the current paradigm to which they gave birth.

...

Systolic blood pressure increases at a constant rate with age. In sharp contrast to the current paradigm, we find that this increase does not incur additional risk. More specifically, all persons in the lower 70% of pressures for their age and sex have equivalent risk.

Cardiologist Kendrick in his recent book Doctoring Data points out

Has this paper ever been refuted? No, it has not. Sadly, it was given the worst possible treatment that can be dished out by the medical establishment. It was completely ignored.

...

The benefits of blood-pressure lowering, whatever the level, became so widely accepted years ago that it has not been possible, ethically,[viii] to do a placebo-controlled study for a long time. I am not aware of any placebo-controlled trials that have been done in the last twenty years, or so.

A bit of sanity

The same year (2017) the AHA and cardiologists were lowering the target blood pressure for everyone from 140 to 120, a group representing family physicians published an official guideline for treating hypertension in adults age 60 and over. Their method was rigorous, taking into account all available studies. Here is their core recommendation:

ACP and AAFP recommend that clinicians initiate treatment in adults aged 60 years or older with systolic blood pressure persistently at or above 150 mm Hg to achieve a target systolic blood pressure of less than 150 mm Hg to reduce the risk for mortality, stroke, and cardiac events. (Grade: strong recommendation, high-quality evidence).

What a breath of fresh air! And completely in line with this data-driven review that showed that a large number of people taking anti-hypertensive drugs just 1 in 125 were helped (prevented death), while 1 in 10 were harmed by side effects. Also in line with this careful study of people with elevated blood pressure in the range of 140-160; the study showed that none were helped by drugs, while 1 in 12 were harmed.

BTW, if you're not familiar with the concept of NNT, you should learn about it. It's crucial.

Hypertension Drugs can hurt you

Doctors dish out hypertension drugs like candy. It's often the case that two different kinds of drugs will be required to get your blood pressure to "safe" levels. For reasons that don't seem to be studied, it's rare indeed for doctors to mention side effects; yet in repeated studies, the generally data-suppressing researchers can help but mention that the side effects are so bad that roughly 10% of study participants drop out of the study! (See above for references.)

There are good lists of side effects at Drugs.com. Here's some information about Amlodipine:

Side effects requiring immediate medical attention

Along with its needed effects, amlodipine may cause some unwanted effects. Although not all of these side effects may occur, if they do occur they may need medical attention.

Check with your doctor immediately if any of the following side effects occur while taking amlodipine:

More common

• Swelling of the ankles or feet

Less common

• Chest tightness
• difficult or labored breathing
• dizziness
• fast, irregular, pounding, or racing heartbeat or pulse
• feeling of warmth
• redness of the face, neck, arms, and occasionally, upper chest

Rare

• Black, tarry stools
• bleeding gums
• blistering, peeling, or loosening of the skin
• blood in the urine or stools
• blurred vision
• burning, crawling, itching, numbness, prickling, "pins and needles", or tingling feelings
• chest pain or discomfort
• chills
• cold and clammy skin
• cold sweats
• confusion
• cough
• dark yellow urine
• diarrhea
• dilated neck veins
• dizziness or lightheadedness when getting up from a lying or sitting position
• extra heartbeats
• fainting
• fever
• itching of the skin
• joint or muscle pain
• large, hive-like swelling on the face, eyelids, lips, tongue, throat, hands, legs, feet, or sex organs
• numbness and tingling of the face, fingers, or toes
• pain in the arms, legs, or lower back, especially pain in the calves or heels upon exertion
• painful or difficult urination
• pale, bluish-colored, or cold hands or feet
• pinpoint red or purple spots on the skin
• red, irritated eyes
• redness of the face, neck, arms, and occasionally, upper chest
• redness, soreness or itching skin
• shakiness in the legs, arms, hands, or feet
• slow or irregular heartbeat
• sore throat
• sores, ulcers, or white spots on the lips or in the mouth
• sores, welting, or blisters
• sudden sweating
• sweating
• swelling of the face, fingers, feet, or lower legs
• swollen glands
• trembling or shaking of the hands or feet
• unsteadiness or awkwardness
• unusual bleeding or bruising
• unusual tiredness or weakness
• weak or absent pulses in the legs
• weakness in the arms, hands, legs, or feet
• weight gain
• yellow eyes or skin

Then there are the ones judged to be less severe:

Side effects not requiring immediate medical attention

Some side effects of amlodipine may occur that usually do not need medical attention. These side effects may go away during treatment as your body adjusts to the medicine. Also, your health care professional may be able to tell you about ways to prevent or reduce some of these side effects.

Check with your health care professional if any of the following side effects continue or are bothersome or if you have any questions about them:

Less common

• Acid or sour stomach
• belching
• feeling of warmth
• heartburn
• indigestion
• lack or loss of strength
• muscle cramps
• redness of the face, neck, arms, and occasionally, upper chest
• sleepiness or unusual drowsiness
• stomach discomfort, upset, or pain

Those are the issues with just one of the many hypertension drugs, one of the most widely prescribed!

Conclusion

Blood pressure varies greatly, reflecting the human body's amazing self-regulation systems. In the vast majority of cases, blood pressure goes up with age. Lowering it by drugs does more harm than good. Except perhaps in extreme cases, high blood pressure does not cause disease. When pressure is extremely high, a search for the cause should be made. The ongoing focus on hypertension as a disease reflects nothing but the stubborn refusal of the medical establishment to admit that they were wrong, and of the pharma companies to give up a lucrative market.

Flowcharts and Workflow in Software

The concept of workflow has been around in software from the beginning. It is the core of a great deal of what software does, including business process automation. Workflow is implicitly implemented in most bodies of software, usually in a hard-coded, ad-hoc way that makes it laborious and error-prone to implement, understand, modify and optimize. Expressing it instead as editable declarative metadata that is executed by a small body of generic, application-independent code yields a huge increase in productivity and responsiveness. It also enables painless integration of ML and AI. There are organizations that have done exactly this; they benefit from massive competitive advantage as a result.

Let’s start with some basics about flowcharts and workflow.

Flowcharts

Flowcharts pre-date computers. The concept is simple enough, as shown by this example from Wikipedia:

The very earliest computer programs were designed using flowcharts, illustrated for example in a document written by John von Neumann in 1947. The symbols and methods became standardized. By the 1960’s software designers used templates like this from IBM

to produce clean flowcharts in standardized ways.

Flowcharts and Workflow

Flowcharts as a way to express workflows have been around for at least a century. Workflows are all about repeatable processes, for example in a manufacturing plant. People would systematize a process in terms of workflow in order to understand and analyze it. They would create variations to test to see if the process could be improved. The starting motivation would often be consistency and quality. Then it would often shift to process optimization – reducing the time and cost and improving the quality of the results. Some of the early work in Operations Research was done to optimize processes.

Workflow is a natural way to express and help understand nearly any repeatable process, from manufacturing products to taking and delivering orders in a restaurant. What else is a repeatable process? A computer program is by definition a repeatable process. Bingo! Writing the program may take considerable time and effort, just like designing and building a manufacturing plant. But once written, a computer program is a repeatable process. That’s why it made sense for the very earliest computer people like John von Neumann to create flowcharts to define the process they wanted the computer to perform repeatedly.

What’s in a Flowchart?

There are different representations, but the basic work steps are common sense:

• Get data from somewhere (another program, storage, a user)
• Do something to the data
• Test the data, and branch to different steps depending on the results of the test
• Put the data somewhere (another program, storage, a user)
• Lots of these work steps are connected in a flow of control

This sounds like a regular computer software program, right? It is! When charted at the right level of detail, the translation from a flowchart to a body of code is largely mechanical. But humans perform this largely mechanical task, and get all wrapped up in the fine details of writing the code – just like pre-industrial craftsmen did.

Hey, that's not just a metaphor -- it is literally true! The vast, vast majority of software programming is done in a way that appears from the outside to be highly structured, but in fact is designing and crafting yet another fine wood/upholstery chair (each one unique!) or, for advanced programmers, goblets and plates made out of silver for rich customers.

Workflow

In the software world, workflow in general has been a subject of varying interest from the beginning. It can be applied to any level of detail. It has led to all sorts of names and even what amount to fashion trends. There is business process management. Business process engineering. And re-engineering. And business process automation. A specialized version of workflow is simulation software, which led early programmers to invent what came to be called "object-oriented programming." To see more about this on-going disaster that proved to be no better for simulating systems than it has been for software in general, see this.

When document image processing became practical in the 1980’s, the related term workflow emerged to describe the business process an organization took to process a document from its arrival through various departments and finally to resolution and archiving of the document. The company that popularized this kind of software, Filenet, was bought by IBM. I personally wrote the workflow software for a small vendor of document image processing software at that time.

Workflow in practice

There has been lots of noise about what amounts to workflow over the years, with books, movements and trends. A management professor in the 1980's talked about how business processes could be automated and improved using business process re-engineering. He said that each process should be re-thought from scratch -- otherwise you would just be "paving the cow paths," instead of creating an optimal process. As usual, lots of talk and little action. Here's the story of my personal involvement in such a project in which the people in charge insisted they were doing great things, while in fact they were spending lots of money helping the cows move a bit faster than they had been.

The Potential of Workflow

The potential of workflow can be understood in terms of maps and driving from one place to another. I've explained the general idea here.

Most software design starts with the equivalent of figuring out a map that shows where you are and where you want to get to. Then the craftsmanship begins. You end up with a hard-coded set of voluminous, low-level "directions" for driving two blocks, getting in the left lane, turning left, etc.

When the hard-coded directions fail to work well and the complaints are loud enough, the code is "enhanced," i.e., made even more complex, voluminous and hard to figure out by adding conditions and potential directional alternations.

Making the leap to an online, real time navigation system is way beyond the vast majority of software organizations. You know, one that takes account of changes, construction, feedback from other drivers on similar routes about congestion, whether your vehicle has a fee payment device installed, whether your vehicle is a truck, etc. Enhancements are regularly made to the metadata map and the ML/AI direction algorithms, which are independent of map details.

When software stays at the level of craftsmanship, you're looking at a nightmare of spaghetti code. Your cow paths aren't just paved -- they have foundations with top-grade rebar, concrete and curbs crafted of marble.

Conclusion

Metadata-driven workflow is the next step beyond schema enhancement for building automated systems to perform almost any job. It's a proven approach that many organizations have deployed -- literally for decades. But all the leaders of computing, including Computer Science departments at leading universities, remain obsessed with subjects that are irrelevant to the realities of building software that works; instead they stay focused on the wonders of craftsman-level low-level software languages. It's a self-contained universe where prestige is clearly defined and has nothing to do with the eternal truths of how optimal software is built.