Agree.The PDA (Premium Deposit Account) only works for funding the funding. It's not really part of the policy. You can't borrow against the PDA.
If at all possible, I wouldn't touch the borrowing on a policy until after the 5th year of a well-funded policy.
Generally, to me, a well-funded policy will have 50%+ of the premiums paid... as available cash values by the end of year 5. So, if you did $10,000 a year for 5 years, that's $50,000 in premiums. I'd want to see $25,000+ in available cash values for it to be a well-funded policy.
I just mentioned the PDA b/c of the OP. Most people don't say, "I have the money to buy a house, I want to do it in 10 years..." They want to buy a house now.
If someone has saved 100k to get a mortgage on a 500k house, putting it into WL instead of getting a mortgage is just dumb (IMO).
Now, if that person had put 10k/yr into a WL plan for the last 10 years and has close to 100k to borrow from, that strategy can work well.
But as you said, that takes some time.