Monday, September 6, 2021

Heuristics: Inappropriate high-confidence of "expert" decision-makers

Quite perversely, the more experience we acquire in decision-making the less accurately our mental models reflect how we actually make decisions.

Researchers in decision-making (Tversky, Kahnman, Slovic, Dawes and a host of others) studied expert decision-makers with the intention of improving diagnostic and financial decision making.

Financial decision making was researched, in part, because there is a wide body of non-volatile data that is readily available to the public. If you want to know the quarterly payroll and revenues of an obscure company in 1963 you can still find that data. That makes it possible to extract the data environment the decision maker was swimming in -after the fact- and review the results in real-time.

The hope was that insights gained by explicitly studying the thought processes of experts could be leveraged to other fields, like medicine, and could be taught to younger traders resulting in a much faster learning-curve. It was also hoped that computers could be programmed to make the same decisions stockbrokers made, but make them faster and more reliably (AI).

The earliest studies involved interviewing stock brokers in the top 5%. A typical high-end broker would wax eloquent about carefully evaluating twenty or more variables looking back five years or more. They said they evaluated the variables in highly non-linear ways. For example, growth in the number of employees was a positive when certain conditions were met but a negative when other conditions prevailed.

Young stockbrokers were also interviewed to define the starting point and get a picture of how much "learning" occurred. It was not surprising that novice stock pickers used simple, cook-book formulas. For example, a client might want to buy a position in a specific industry. The newbie would look at the candidate's Price/Earnings ratio less extraordinary flows, Revenue growth and future duration of Patent protections. The newbie would sort the candidates and present the client with his top three recommendations and tag "the best" choice.

That was the state-of-the-art when computers became widely available.

One of the first things those who studied decision making did was to "test" the model expert stockbrokers SAID they used against the data.

Performing linear regressions against their decisions and the data that was available at the time they made those decisions, it was determined that the decision-making process of "experts" was indistinguishable (given reasonable certainty bands) from the decision-making process of the newbies.

How can that be so?

The human brain is a computer with a very small buffer.

Experts used the same three, perhaps four variables that the newbies used. Then they used the remaining seventeen variables (remember, they said they used twenty or more) to CONFIRM their initial assessment.

At this point you should be thinking...Ah! This sounds like the First Impressions or Anchoring-and-Adjustment heuristic.

The balderdash about "...highly non-linear/configural..." arose from two sources.

More often than we care to admit, firms that appear great on paper falter and we do not get the financial return we predicted. Looking backwards, the stockbrokers would find some anomaly in the firm's statistics and create ad hoc "rules" to explain after-the-fact why that firm did not live up to their predictions.

The other origin of the "...highly non-linear/configural..." BS was that the experts always ignored statistics that contradicted their original assessment. But that caused cognitive dissonance so they eased the pain by rationalizing "...too many teraquarks per employe is usually a negative but this is such a great company and in this industry that, in this case, it is a positive..."

Most newbie stockbrokers were appropriately humble. They knew what they did not know. The "expert" stockbrokers were confident at levels far beyond what was appropriate for the actual processes he uses.

Do experienced stockbrokers add value?

One might think that an advisor who is grossly unaware of his actual decision making process and offers no measurable advantage over a newbie offers no value to the investor.

The advantage of an experienced advisor (financial, medical, etc.) is that lack of confidence is often very expensive.

Selling your stock holdings after the market pukes? Very expensive.

Waffling and not choosing a course-of-therapy when you have cancer? Very expensive.

The senior advisor is reassuring and helps you stay-the-course even as things get rocky.

The downside is that he is likely to keep you invested in buggy-whips when the less confident newbie would move you out.

3 comments:

  1. When do herd tendencies come into it?
    Many stockbrokers act with other brokers for safety in numbers to protect their job. Truly unique ones willing to act different and risk getting fired or losing customers are rare.

    ReplyDelete
    Replies
    1. That is a very hard question to answer.

      The usual canard is that stockbrokers want you to buy-buy-buy because you don't register a loss until you sell. The biggest place I see the herd effect is that they will almost never advise you to sell and cut your losses.

      It begs the question, where does all that money to buy come from? If I sell something then I have money to roll over. If I had huge income maybe I would not need to invest.

      Delete

  2. This is just a tangential thought to your subject, however:

    My daughter really helped me this year with a similar issue. A medication impacted me in ways I didn't think possible. I was hesitant to follow up on that due to the cost of a specialist. She said, "you don't mind paying tuition to learn a new subject or skill. Why not consider this 'tuition' to learn about yourself?" It made perfect sense, and I have a new diagnosis that has enabled me to function better.

    When a legal issue popped up this year, it didn't take me long to realize I needed to pay more 'tuition' and get a good counselor to help me navigate the issue.


    The financial market is difficult in my mind due to the wild card of .gov manipulations. You never know what they are going to do, but can almost be certain that it will backfire and cause harm to all involved. I spent some time learning about day trading a few years ago. Unfortunately, the program I used was more about taking my money than teaching me anything. I bailed out when that became apparent.

    ReplyDelete

Readers who are willing to comment make this a better blog. Civil dialog is a valuable thing.