• What We Do
    • Wealth Counseling
    • Investment Advisory Services
  • Who We Are
    • Our Team
    • Research Roundtable
  • How We Invest
    • Our Fiduciary Pledge
    • Investment Approach
    • Questions To Consider
    • Our Transparent Approach
  • Insights and Research

Insights and Research

Home  »  Managing The Market • Transparency   »   Stats About Statistics

Stats About Statistics

By Joachim Klement, CFA and Preston McSwain, August 13, 2021

Be humble about things that aren’t stable

Presentations of how various investment asset classes have moved as compared to each other are common.  Illustrations of standard deviations and correlations drive trillions of dollars in allocations to various asset classes and investment products.

Behind these investment risk and return charts are statistics.  It’s good to review statistics but it is also important to understand their limitations, especially in the financial world where the inputs used in models can be unstable.

To dive deeper into statistical issues that investors should keep in mind, we talked again with our friend Joachim Klement.  For those who don’t know him, Joachim is a trustee of the CFA Institute Research Foundation and was previously, the head of the UBS Wealth Management Strategic Research team.

Beyond being a well respected investment researcher, Klement also has a deep background in statistics, having studied mathematics and physics at the Swiss Federal Institute of Technology, graduating with a master’s degree in mathematics.

The following are excerpts from our talk.


McSwain

Thank you for spending time with us on this.

Like you, I’ve been in the investment business for the better part of 30 years.  During this time, I’ve seen many presentations that use the return streams of investment strategies and asset classes to create standard deviation and correlation charts that illustrate how one type of investment may move as it relates to another.

When pondering how they are presented, I often think back to early statistics classes that I took in undergrad when studying engineering.  I don’t have near the background in this that you do, but I’ve been noodling this around for quite some time wondering about the proper, or maybe sometimes improper, use of statistics in our industry.

Klement

I think we both feel the same unease, because what we’re doing in the finance world is using concepts that were not designed for this purpose.  They are sound mathematically, but were developed for natural sciences.

As an example, regression analysis was developed to investigate the motion of the planets around the sun and to test Newton’s law of gravity. That’s the origin of linear regressions.

McSwain

Designed for laws of nature.

Klement

Yes.  Based on the work of Kepler and Newton’s law of gravity.

Natural systems have stability and gravity is a constant.  When using these stable inputs, the outcomes of statistical models can be reliable.

Some natural systems can change, of course, but they change very slowly over millions and billions of years.

The problem when we use statistics in the financial markets is that the interactions between variables are constantly changing.

McSwain

This is what has been on my mind for years.

When we apply statistics to the financial world, we can create models that are not stable.

Klement

Correlations in the financial and investment world are never stable.

We might get some stability by accident and happen to find that standard deviation and correlation match up to the long-term average, but we only know this years later – not in advance.

McSwain

If you play this out, it makes you step back, especially related to correlation charts that can drive large investment allocations.

As you and many have written about, financial market inputs used in investment statistics may have more to do with human emotion than anything else, and us humans are not always emotionally stable about things related to money.

Klement

This gets into George Soros’s notion of reflexivity.  The moment people think they understand the market, they change their behavior.  This then changes the market – again.  It is a constant race between people changing to try to catch something that is changing.

McSwain

The only constant in the financial markets might be change.

Related to how these changing inputs are used, when speaking to groups I often ask this question:

What is the most important word in the term Modern Portfolio Theory?

Theory.

We hope that all of our theories and hypotheses can be valid but it is wise to remember that they are not laws of nature.

Klement

Harry Markowitz has talked about this before.  So has Bill Sharpe, as did Jack Treynor.

All were well aware that their work was based on rough approximations.  They knew that they were using volatility, for example, as a concept because it was easier to deal with based on the computing power in the 1950s and 60s.

Then along came the 1960s and 70s, and somehow Modern Portfolio Theory (MPT) got morphed.  Everyone in the industry started to apply it as if it was a law.

All know it has problems, but it has become almost canonical.

In reality, however, output from MPT models is just a first approximation.

McSwain

This does not mean that it is not useful.

Klement

Correct.

Approximations can be useful, as long as you just treat them as such.

McSwain

In putting together these statistical approximations, you need to have a series of observations.

For a model to be reliable, you need to have a lot of observations, though.  This is why natural systems work well, correct?

You have millions or billions of years of observations.

Klement

Yes.

For models to be reliable, like a law, you need a lot of observations.

In addition, the observations should be close together in series.

McSwain

Back in undergrad financial stats classes, I was taught that the further away you get from daily observations, the more unreliable a model becomes.

Let’s talk a little more about this, though, especially your second point about the connections of time periods or a series.

Klement

When you look at the correlation of an investment asset class to another using monthly, annual, or three- to five-year returns they can change dramatically.

Next, you need to keep in mind issues that can occur with projections that are made using overlapping time periods.

If you only have a few years of rolling monthly returns your correlations might look much more stable than they really are.

The problem is that you just don’t have enough data that represents truly different time periods, not to mention that, as we previously discussed, for stats to be more reliable, you need to be able to analyze decades and decades of data.

McSwain

We could quickly go down a technical rabbit hole here, but how can we explain this in lay language?

Klement

Not easy for sure.

But how about this…

Many professionals in our industry like to look at three-year rolling periods of performance.  This can be useful to spot trends but it can be problematic when running correlations.

As an example, assume that your first three-year period runs from January 2000 to the end of December 2002.

Some might then roll this forward monthly, with the next time period being February 2000 to January 2003, and so on.

The problem is that when doing this you create overlapping time periods that contain much of the same information.

Using our same example, if you start in January 2000 and go to December 2002, and then you go one month forward from February 2000 to January 2002, the data is identical for 35 of the 36 time periods.

Recycling can be good as it relates to the environment but, when running investment statistics, it can be bad.

McSwain

This gets to what we talked about off-line.  The differences between stability and reliability.

Klement

Exactly.

Using overlapping periods can make things look more stable.

But…

It can make correlation charts less reliable.

Back to our previous example, unless you have a lot of three year periods, and take care to analyze the data properly, you are going to get models that make the correlation between two asset classes look stable.  You just have too many overlapping periods and not enough new information.

McSwain

From my perspective as an adviser or allocator, this is where it gets tricky.

If you aren’t careful to look closely at how the data in a presentation is being put together, you might think you are making a decision based on something that is reliable when it is not.

As we’ve all talked about before, you can use data to fit the narrative that you want to promote.

Klement

That’s the problem with many new product or new asset class presentations.

How often have we seen wonderful looking, new product back tests that, when applied to real markets, look very different?

McSwain

Plenty.

Klement

Usually, this issue is put down to data mining, which can be a problem, but that is not always fair.

Back to where we started, in our industry we are just not working with stable data.  This should be a first principle that we talk about more often.

McSwain

Agreed – again.

We want to be careful about this because it is good to keep our eyes open to new ideas and to support innovation.

Often the time frames we are working with, however, are just too short to be making some of the proclamations that we tend to make in our industry.

As I’ve often joked, we are very good at making theories and hypotheses sound like laws of nature.

We don’t have millions or billions of years of data like we do with laws of nature, but to be helpful and work with what we have, how many years of data, which would ideally include monthly observations, should an investor look for when reviewing correlation charts or models?

Klement

If you would have asked me this a few years ago, I would have said at least two full economic cycles, which might cover 20 or more years.

The problem, though, is that cycles can be odd.  Take the last two for example.  A global financial crisis and now a global pandemic.  If you just looked at these two, how representative are they?

McSwain

Yes.

This last equity market cycle was up, down, up, all in about 12 months or less.

Klement

Exactly – again.

This last cycle might not have any predictive power at all unless there’s another pandemic.

Similarly, the financial crisis might have no predictive power unless we have another one that is caused by the exact same things.

McSwain

What I heard was this.

It is not only the number of cycles or time periods, but also how unique the time periods are.

Klement

Yes.

McSwain

If you listen to us too much, you might start to throw your hands up.

How about trying to come up with some rules of our own related to all of this?

Klement

This is tricky, as we will be proven wrong for sure.

But, how about these?

First, if something is being presented with too much conviction in our industry, beware – caveat emptor.

Next, and importantly, be humble.  No one knows how the markets really work.

McSwain

Indeed.

Klement

I think this leads me to my first true rule.

Never be too sure about anything.  The markets will always throw something at you that you didn’t expect or see coming.

Some people think they are better at seeing things coming than others but in general, we’re all pretty rubbish at that.

I think this one might actually stand the test of time.

McSwain

This all provides investors with a lot to ponder related to the reliability, or maybe unreliability, of some investment correlation and statistical models.

Any final thoughts?

Klement

I would again say, stay humble.

Also, don’t be dogmatic in the sense that the way things have worked in the past will be the way they work in future.

McSwain

Well, said.

Thank you, Joachim.

 


Related Reading:

The Double Life of Correlations – Kevin Kneafsey, Phd – Schroders

Are We Baking Portfolios with Bad Ingredients? – Tommi Johnsen, Phd

Fallible Forecasts? – Maneesh Shanbhag, CFA

Don’t Put Yourself in a Corner – Joachim Klement, CFA

Joachim Klement, CFA
Contributing Author | Website | + posts
  • Joachim Klement, CFA
    #molongui-disabled-link
    Don’t Put Yourself in a Corner
  • Joachim Klement, CFA
    #molongui-disabled-link
    How to Actively Add Value
Preston McSwain
+ posts
  • Preston McSwain
    https://fwpwealth.com/author/preston/
    Transparency, Simplicity and Peace of Mind®
  • Preston McSwain
    https://fwpwealth.com/author/preston/
    Fiduciary Wealth Partners Reading List
  • Preston McSwain
    https://fwpwealth.com/author/preston/
    If We Had A Chief Economist We Would Have to Pay Them
  • Preston McSwain
    https://fwpwealth.com/author/preston/
    Is Trying to Pick Active Managers a Loser's Game?
SHARE
Tags:
Forecasting, Investing, Statistics
RELATED ARTICLES
Staying Disciplined and Avoiding Unforced Errors
The track records of many successful professionals can be attributed to keeping it simple and avoiding unforced errors
Still Keeping A Steady Hand
Every year over the past 10 years we have written a similar letter.
Recessions – What Should Investors Do?
The evidence may surprise you.
All articles

Follow us on social media

Search Our Ideas

Subscribe to Our Posts


Important disclosures

Most Popular

  • The Simple Alternative – Keeps On Winning
  • The Simple Alternative
  • Say It Ain’t So, Joe
  • Are We Baking Portfolios with Bad Ingredients?
  • Crucial Elements in Wealth Management: Simplicity and Transparency

Browse by Theme

  • Fees
  • Fiduciary Duty
  • Investment Philosophy
  • Managing The Market
  • Peace of Mind
  • Performance Measurement
  • Private Investing
  • Quarterly Letter
  • Simplicity
  • Taxes
  • Transparency
  • Uncategorized
  • Values

FWP Logo Dark

Fiduciary Wealth Partners is a 100% employee owned firm that serves clients in a transparent, fiduciary manner.

We do not have any fee sharing arrangements with managers and do not have any broker-dealer conflicts. In addition, you will never see an arbitration clause in our contracts.

Everything we do is focused on assisting trustees, institutions and families with investment consulting, management and overall asset planning strategies.

  • Disclosures
  • ADV
  • Privacy Policy
  • Form CRS

Useful Links

  • Investment Advisory Services
  • Wealth Counseling
  • Our Team
  • Research Roundtable
  • Our Values
  • Questions To Consider
  • Insights and Ideas
  • Contact Us

Contact us

Phone

(617) 602-1900

Email

info@fwpwealth.com

Address

177 Huntington Avenue
20th Floor
Boston, MA 02115

 

View larger map

© 2020 FWP. All Rights Reserved. Fiduciary Wealth Partners Is An SEC Registered Investment Adviser.