Connect with us

Hi, what are you looking for?

News

Big data’s past is messing with our future

THERE is an inconvenient open secret that lurks behind many decisions we make every day. We live in a data-empowered era. When we plan a trip, Google estimates our travel time and recommends routes based on both current and past traffic data. Netflix suggests movies and shows we might like based on its data from people with similar taste. The Federal Reserve uses market and macroeconomic data to make projections about inflation and what it should do about interest rates.

The open secret is that data makes us slaves to the past. And when the world changes, data no longer tells us as much about the future, and we can end up making worse decisions. Coming out of the pandemic has caused significant chaos and changed many things about our lives. How it affects the data we use to make future decisions will matter for years to come.

To some extent, we have always relied on data to predict the future. Even when we were fully analogue, our past experiences — our personal “data” — drove how we perceived risk and made choices. But during the last century, computing power meant everyone’s past experience, in the form of large data sets, started influencing our lives. This was especially true in finance, which took securities prices from markets to make projections about where prices would go in the future and how to hedge or insure against risk.

Data meant we could make decisions more scientifically, based on many more experiences than just our own. It offered great promise and more accuracy. Many aspects of our lives became more predictable and efficient. We not only were told the best route for our road trip, our computer knew which shoes we wanted to buy before we did. Companies could better predict demand and spend less maintaining inventories, and airlines could sell more seats because they had a better sense of who would show up.

But this process was always less scientific than we told ourselves.

About 10 years ago, I was tasked with projecting future interest rates for a pension risk model. This problem raised many challenges. Normally in finance we take data from the past; when you look at a fund prospectus it tells you its past average return and volatility. But by 2011, interest rates had been falling for decades, and odds were such a period wouldn’t happen again since a repeat would mean rates would go very negative. Economists normally assume rates go back to their historical mean, but there was no sign that was going to happen anytime soon either. The world had changed, it was more global and there was more demand for US bonds, which drove rates down.

All of these concerns made me question if I could use historical rate data, and if so, what history? If I started in the 1980s, I’d bake into my model a bull bond market that would probably not happen again. If I used data going back to the 1970s, I’d assume rates would go to a higher level — that probably won’t happen, either. In the end, after much debate with senior leadership, we decided to use as much history as possible and worked out a process to update with new data frequently. It was the more conservative choice, but it was a hard call.

Data isn’t a replacement for wisdom. Without good judgment, data will fail you because it ties you to the past. Assuming housing prices wouldn’t fall, simply because they hadn’t in a long while, was a big contributor to the financial crisis. More recently, poor data judgment was one reason why the Fed was so certain inflation was transitory. Fed economists use mean-reversion models that, with the last 40 years of data, predicted inflation would quickly go away on its own. Changing data is one reason why Google searches aren’t as useful as they used to be.

Today, the practice of letting data guide us has become a bigger danger because our lives have become more data-driven. Our online lives generate more data than ever before. And greater computing power and technology enabled use of that data. Suddenly every trip we took, everything we bought and every website we visited added to data’s predictive power and guided our decisions — even if it was all a little creepy.

But even big powerful data is from the past. And now the world has changed and all that data isn’t so predictive anymore. Inflation is back and changing how and what we buy. Interest rates are going up, changing how we save, invest and buy homes. And it’s not just the economy. Traffic patterns from 2019 — when people went to work five days a week — are no longer as useful, nor are projections on what clothes we will need. We watch more movies at home and eat out earlier. Leisure travel is up, but there is still less business travel, which makes it harder for airlines to know how many seats to sell.

Today’s economy is chaotic for many reasons — primarily supply and labor shortages — so it’s hard to know how much our data-powered algorithms are failing us. If the world has truly changed, it may take years to have meaningful data to power our decisions again. Or, if it goes back to normal, which pandemic years should we use going forward?

It’s unusual for the world to change so much in such a short amount of time, but it offers a valuable lesson that we will probably forget: Data is only a guide, and never a replacement for our own judgment. Going forward, we’ll have to take all the predictions, big and small, with some skepticism and balance them with our own personal experience.

BLOOMBERG OPINION

Your information is secure and your privacy is protected. By opting in you agree to receive emails from us. Remember that you can opt-out any time, we hate spam too!

You May Also Like

News

BW FILE PHOTO GROSS BORROWINGS by the National Government reached P2.6 trillion as of end-September as it continued to raise funds to respond to...

News

REUTERS By Luz Wendy T. Noble, Reporter The country’s foreign exchange buffers slightly increased as of end-October as the value of the central bank’s...

News

COVID-19 has had a significant impact on the mental health of Filipinos across different groups all over the archipelago. From frontline workers, parents balancing...

Financial Advisors

The healthcare ecosystem is one that has thrived on the cusp of scientific progress, benefitting enormously from the winds of change in the technological...

Disclaimer: Respect Investment.com, its managers, its employees, and assigns (collectively "The Company") do not make any guarantee or warranty about what is advertised above. Information provided by this website is for research purposes only and should not be considered as personalized financial advice. The Company is not affiliated with, nor does it receive compensation from, any specific security. The Company is not registered or licensed by any governing body in any jurisdiction to give investing advice or provide investment recommendation. Any investments recommended here should be taken into consideration only after consulting with your investment advisor and after reviewing the prospectus or financial statements of the company.

Copyright © 2022 Respect Investment. All Rights Reserved.