Borrowing against 401k for first home - things to watch

Question

There is a general advice that borrowing against 401k is to be avoided. When I was looking at related questions here in SE I found a few discussions where some experts challenged the generality. For ex, source vs collateral question. To me that somewhat makes sense.

In the recent past I have been thinking about this option more and more as I have been looking to buy my first home in the San Francisco Bay Area, California. Needless to say most houses are unreasonably highly priced (and that discussion could be out of scope for now).

I am currently faced with a sort of a dilemma/quandary and I am looking to evaluate my thought process and remove misconceptions.

Back story: In the last two years we had many expenses (unplanned medical, wedding, deferred medical, expenses from putting purchases on credit cards for a period close to a year when my now wife wasn't being paid by her professor for postdoc research).

So, currently, even though together we bring a decent pay home, we're aggressively paying down debt and we've been doing so by juggling balance transfer cards and scoring 0% APR cards. My own question - balance transfer vs loan - total interest paid for reference.

Until even a year ago I was not looking to buy a house in the Bay Area and was hoping to find a job in another state but that is seeming uncertain now. It also looks like the market is somewhat saturating with more houses staying longer on the market. So as we started talking about borrowing options it soon became evident that the biggest blocking factor for us is the down payment.

Constraints/Challenge: Because of our paying down the debt we aren't building/saving for the down payment. A few options where they need just 3% down seem a little unconvincing/shady, but even if we wanted to we can't do 3% currently.

With the stock market having bad and good days, my 401k balance is not showing consistent growth. I understand that over the long term with the assumption of a growth in stocks the 401k will grow and compound. But...

So the questions: Can I use this time of ups and downs and trade-war uncertainties to withdraw from 401k? Also I keep hearing fed rates are low. Will this help my case in securing lower APR mortgages, and thus help me in the long run?

I am also hazy on the 'you pay yourself the interest' because then what is the flip side? And what do the banks get for this?

Answer 1

Some questions to ask your 401(k) custodian: While you have an open loan, can you continue to make contributions as long as you keep up with the scheduled loan payments? Or are all paycheck deferrals treated as loan payments? (The reason this is important is that there is no employer match for loan payments, only for contributions, and lost match could far outweigh the other costs of the loan)

Of course, ask about the fees. The setup and funds availability time. Whether you need to prove hardship to get the loan.

Since you mentioned using balance transfer offers, be warned that these are increasing your credit utilization which is a major factor in your credit score and therefore your mortgage interest rate. And the fact you're using a 401(k) loan for your down payment will be apparent to your mortgage lender, who will adjust the monthly payment you can afford downward to account for 401(k) repayment.

If you haven't already, use freecreditscore.com (run by Experian) and creditkarma.com (data from Equifax and TransUnion) to get free access to scores from all 3 major bureaus.

You'd have to have a lower all-in (mortgage+insurance+taxes+HOA) monthly payment compared to renting in order to be worth making a home purchase while still paying down credit card debt (plus the student loans that your other question mentioned you married into).

With the stock market having bad and good days, my 401k balance is not showing consistent growth.

This makes it a really dangerous time to pull money out. If you initiate the loan before a dip, you come out ahead. But if you initiate it during a dip, you both draw down a bigger fraction of your 401(k) balance, and you permanently miss out on the recovery. Also, as long as you believe the market trends up long term, during a dip is the best time to contribute -- that money will grow in the recovery without having been subjected to the loss.

Answer 2

Can I use this time of ups and downs and trade-war uncertainties to withdraw from 401k?

Only if you believe you can time the market. You are hoping you can borrow from the account at the top, and then have most of the payment take place while the market is rising.

Also I keep hearing fed rates are low. Will this help my case in securing lower APR mortgages, and thus help me in the long run?

The rate you will be offered for the mortgage will depend on the general interest rates, your amount of down payment, the monthly payments you are already committed to, and your credit scores. But lowered federal rates will generally allow you to either borrow more money for the mortgage, or lower your monthly mortgage payment.

You face several issues even before looking at borrowing from the 401K. Your question and comments refer to credit card debt and student loans. Those will also limit how much you can borrow for the mortgage. Those plans you mention that allow you to borrow money even with less than 20% do come with either their own interest rate for the borrowed down payment, or with fees such as for PMI.

The loan from the 401K will be treated as a loan by the lender. That will be more debt which will be factored into how much you can borrow for the mortgage.

I am also hazy on the 'you pay yourself the interest' because then what is the flip side? And what do the banks get for this?

The argument that you pay yourself interest is the crux of the dispute. If you have no other way of getting the money, then it doesn't matter. But if you shouldn't be borrowing the funds then that argument that you are paying yourself the interest is a weak argument because you have to also factor into the calculation the reduced investments during that time, the loss of company match, a comparison of the rates, and what happens if you leave your job, or are fired.