Paying off Mortgage vs Investing

By: Matt Ward, CFP^®

^{(summary at bottom)}

One of the very common questions that I am asked is, “Should I invest or should I pay off my mortgage?”

There are 6 considerations I make when deciding if someone should pay off their mortgage.

Those 6 considerations are:

1. Mortgage Interest Tax-Deductions
2. Mortgage Interest Rate
3. Where are you paying your mortgage off from?
4. Time Value of Money
5. Inflation Rate
6. Non-Financial Considerations

1. Mortgage Interest Tax-Deductions (lost)

One benefit to holding a mortgage is that you may be able to claim the mortgage interest as a tax deduction, for those taxpayers that itemize*. Say you have $5,000 in mortgage interest, this is deducted against your income for the year. For example, if your adjusted income was $100,000 and you had $5,000 of mortgage interest, this would reduce your taxable income to just $95,000. Assume a 20% tax bracket, on $5,000 interest, that is $1,000 in tax savings.

2. Mortgage Interest Rate (after-tax)

As indicated above, what is your after-tax mortgage rate? Do you itemize your tax deductions? When considering and making your determination for paying off a mortgage, this is important. So, now you might want to think about how you earn and invest money. Assuming an individual had a 3.7% before-tax mortgage rate, itemizes, and has an after-tax interest rate of 3.1% on mortgage interest, this is very low.

It is very likely that someone contributing to retirement and investing this additional income in a 401K or SEP IRA will earn more than 3% over the same, annualized 30-year period. Maybe a self-employed person is growing their business income at over 10% per year, putting capital to work there would certainly be more efficient than paying off a 3.1% mortgage loan. Either way, determining human earning potential along with passive investment returns is a great starting point for determining the best use of your army of dollar bills.

3. Where are you considering paying your mortgage off from? (IRA or bank)

The next consideration after determining the after-tax mortgage interest rate is to determine where you’re paying your mortgage off from. Is it your checking account? Or a retirement account, like a 401(K) or IRA? This stuff matters. Maybe someone has more in IRAs than their bank. If you are considering taking money from an IRA to pay a large lump sum towards your mortgage, I recommend exercising caution. Think about this. You are jumping up in tax brackets when you pull from IRAs. So you’re paying say, 28% to save 4%. That would be 7 years of mortgage payments in just taxes alone. So again, it might make sense for your situation, but oftentimes, tying up liquidity at a higher tax rate is not the best use of your dollars. There are strategies for taking money from an IRA more efficiently than one lump sum.

4. Time Value of Money (compounding)

If you look back at historical stock market returns, you find that over every full 30-year periods, markets (after inflation) have been higher than average mortgage rates. So, for most people, holding low-interest mortgage debt allows them to compound investments (stocks, real estate, businesses) at a higher rate (10%) than the mortgage debt they hold (4%).

5. What is the current and expected inflation rate? (inflation = good for debt)

This one is important. Right now we are experiencing high inflation. Take your monthly payment of say $2000. Let’s talk about what happens to your monthly payment at a 5% inflation. If you take 1 year, $2000 x.95 (5% reduction to inflation for the year), this is worth $1900. So after 1 year, you’re only paying the equivalent of $1900 per month. Extend this out over 10 years and $2000 monthly payment is worth $1,227.83. So after the 10th year, your monthly payment is the equivalent of just $1,227.83, after inflation reduced the value of the dollar. Extend this out over 30 and the $2000 payment in those days will only be worth a fraction, now at $462.75. So either way you spin it, inflation is great for long-term, low-interest rate debt. It makes your monthly payment worth less in terms of real dollars.

6. Non-Financial Considerations

Lastly, consider non-financial aspects of the decision. Ultimately, our values and beliefs shape our perception and view of money. People view their investments, debt, liquidity, net worth, and other finances differently from one another. So determining what the right decision for you is unique to anyone else. If after weighing the non-financial considerations it makes sense to pay off a large mortgage balance, the next step is to start planning for taxes and how to most efficiently withdraw from various accounts. There are strategic ways to get money that are not as impactful as withdrawing a very large sum from an IRA, especially for those in retirement, but equally so for those under 59 ^1/2 because they will be penalized and taxed.

Think about this real-world scenario

A taxpayer has a 4% interest rate (3.5% after-tax because they itemize deductions), and this taxpayer faces the decision of whether they should pay off their mortgage or keep their portfolio invested. Assume this taxpayer, like many, has a large amount of their money saved inside of a Traditional IRA.

Here’s what I am thinking about in this scenario: First, what bracket is the taxpayer in? Second, what’s the interest rate on the mortgage? Do you itemize deductions? How much do you have remaining on the mortgage balance, in terms of both dollar amount and years remaining? What age is this individual? Do they have any other debts? How much do they have in liquid savings and investments? Are there any benefits? What about drawbacks?

Once I have analyzed each of these considerations, then I can help make an informed decision.

If the taxpayer had a $250,000 balance on their mortgage, a $1,000,000 IRA, 10 years left on their mortgage, they’re 65 years old, the interest rate is 4% (3.5% after-tax), no other debt, and they maintain $50,000 in their checking/savings account, then I would be reluctant to recommend paying off their house. Now, if they had $10,000,000, that’s different. In this scenario, however, this would be a reduction of 20-25% to their liquid net worth. They would jump up likely into the 24% tax bracket to save 3.5%. They would lose the mortgage interest rate deduction and the opportunity to continue earning a 5-7% return in the capital markets on this $250,000. Rather, I would suggest a couple of things. First, we know this individual can afford to pay off their mortgage. It just is not the most efficient way to allocate their dollars. Maybe the hassle of writing the check each month bothers this individual.

Now, typically, I don’t suggest anyone under the age of 30 dump lump sum contributions into a mortgage, especially if they locked in a low mortgage rate under 4%. The 30-year-old saver can compound their money aggressively at 7-10% and earn more on those dollars than paying off a mortgage at 4%. Think of it as a runway. It takes time for a plane to take off. But once you’re off the ground, it is really noticeable. Same thing here. In the beginning, a young investor might not notice the impact but look at the tables below. I have compared 2 time periods 1970-1996 (which included a big bear market) and 1928-1966 (which included the Great Depression). I have used a mortgage balance of $223,000 on a 30-year note. The total mortgage interest is the balance on the left along with the interest rate.

Exhibit 1. 1970-1996

You ended with 3 million more investing. After the 70s bear market, there was explosive recovery in the markets.

*Inflation adjusted net gain = $677,005

Exhibit 2. 1928-1966

You earned $820,000 more over the 26 years of investing, even going through the great depression era.

*Inflation adjusted net gain = $413,798

Summary Recap:

At the end of the day, there are legitimate reasons to pay off your mortgage. When you get close to retirement, being mortgage-free is not only a good feeling, but for many symbolizes the American dream. However, financially, if you were one that was fortunate enough to lock in a fixed, low-rate mortgage, then it’s probably not the best “financial” decision to pay off your mortgage in a lump-sum, if you believe you can capitalize those dollars at a higher rate of return. There are non-financial considerations to be made. If those outweigh the financial considerations, then the next step is to determine the most efficient way to pay off your mortgage.

Contact us today at 817-238-6300! If you, or anyone you know is facing the decision of whether to pay off their mortgage or invest, or you have any questions about paying off tax implications, please don’t hesitate to reach out.

Matt Ward, CFP^®

CLICK TO SCHEDULE APPOINTMENT

^{*Please always make sure to consult with your tax and financial advisor prior to making a decision. If you don’t itemize deductions then you might not have an after-tax interest rate that is lower than your current. itemizing deductions means you have more deductions than the standard deduction. So this would weigh into the determination. Each tax and financial situation is different, as well as non-financial situations. Tax rates discussed are for 2022 and may change in the future.}

About Matt

Matt Ward is a financial advisor and the president of New Century Investments, an independent investment advisory firm serving business owners, pre-retirees, and retirees in the Dallas-Fort Worth area and beyond. Matt is passionate about integrating investing, planning, and tax management into a holistic approach. Matt’s breadth of knowledge and experience in both taxes and investment management sets him apart, giving him the ability to design, advise on, and manage business strategies, tax efficiency, and retirement planning. He is known for his care and attention to detail and works hard to develop personal relationships with each of his clients so they can benefit from his customized service and guidance. He loves walking with his clients through their financial journey, supporting them and celebrating with them as they reach their goals.

Matt graduated from Texas Tech University with a bachelor’s degree and is a certified financial planner™ and chartered retirement planning counselor℠ professional. When he’s not working, you can find Matt hiking, playing the guitar, and spending time with his family. To learn more about Matt, connect with him today!