A jarring number of Americans are steering clear of investing.

Only 1 in 4 Americans think now is a good time to invest, according to a new study from Allianz Life Insurance Company of North America published Tuesday.

That’s down from 34% last quarter and the most wary people have been in four years.

“While it’s understandable that market volatility can make people hesitant to invest, it’s important to put that uncertainty in context,” Kelly LaVigne, vice president of Consumer Insights at Allianz Life, told Yahoo Finance. “Historically, markets have rewarded patience.”

What got under their skin? Two things: concern about continued volatility and fear about a major recession.

Age is a factor. After all, older investors have weathered previous shaky markets and rising costs. Boomers are the least anxious about the specter of a pending recession, while Gen Zers are the most fearful.

Read more: How to protect your money during turmoil, stock market volatility

Gen Z pulls back

People are taking action, for better or worse, to ease their worries. Roughly half of Americans say they have shifted investments to make them less risky, moving from equities to bonds and other cash holdings.

Two-thirds of Gen Zers took it a step further. They cut back on their retirement savings contributions in the first half of the year due to stress over their finances, the economy, and having other demands for their cash.

Gen Z may feel like they can afford to cut back on retirement savings because they have so many decades until they will likely need to tap those funds, LaVigne said.

But that strategy ignores the time value of money. Even a small amount set aside today with a conservative return will have built up significantly decades down the road.

“For individuals with a long time until retirement, time is one of their greatest advantages, and staying out of the market during periods of volatility can mean missing out on potential recovery and growth,” LaVigne said.

Among those who are not panicking over the economy and market shifts, 47% say they’re comfortable taking on more risk with their retirement investments to offset inflation’s effects, but that confidence has slipped from 54% who expressed that confidence last quarter.

“Not all investors are in the same position,” LaVigne added. The key is balance and making sure “your strategy helps you stay on track, even when markets feel unpredictable.”

Kerry Hannon is a Senior Columnist at Yahoo Finance. She is a career and retirement strategist and the author of 14 books, including “Retirement Bites: A Gen X Guide to Securing Your Financial Future,” “In Control at 50+: How to Succeed in the New World of Work,” and “Never Too Old to Get Rich.” Follow her on Bluesky and X. You can reach her at kerry.hannon@yahooinc.com