Lots to consider here with your refinance lender question Lallo. Here's an overview:
1) If your loan is between $275k and 484k in many cases a small broker or a good sized mortgage banker is the right choice. Owning, Quicken, Fremont Bank, and others can get the job done well for you. Conforming Conventional loans have some base guidelines that can make the process very quick and simple. Most Banks have significant underwriting overlays that prevent seemingly "easy deals" (Pro tip: "easy deals" do not exist....) from closing efficiently. As well, Refinancing with a Broker/Banker can be accomplished in 20-40 days. Banks are now at 60-90 days.
On the plus side: Faster turn around. Better underwriting guidelines.
On the minus side: If you do not want your loan not to be sold, forget about it. Your loan will be sold, and re-sold, and sold again. If this is a concern, go with a Bank. Also - that rate you see... it often requires impounding for taxes and insurance to get that rate.
Bear in mind: Any written offer from a mortgage broker/banker will be matched by any bank out there. You have to wait 60 to 90 days to get the deal done, but the price/rate/terms will be matched. Why is $275,000 a floor? All lenders are "for profit" institutions and the economics aren't there for smaller loans. Yes, lenders are legally required to offer to small loan finance. Yes, as well, your pricing will be terrible because in reality lenders don't want these deals.
2) If your loan is between $484k and maximum conforming Jumbo of $726,525, it's a mixed bag. Most broker/bankers must use standard Jumbo Conforming pricing which at present blows. There are a few outliers able to price Jumbo Conforming loans in a market friendly place, but they are rare indeed. Banks use their Portfolio Jumbo pricing in this arena which then has rates closer to where Standard Conforming loans are priced. Once again, you have to expect a more rigorous underwriting challenge and the process will take 60 - 90 days.
On the plus side - Broker/Bankers still have the edge with faster turn times and more open guidelines.
On the minus side - Conforming Jumbo pricing is often terrible at a broker banker. Banks will take longer.
Bear in mind: To get the better Portfolio Loan rate, you must have 6 months or more of post closing cash reserves, not the 1-2 months you have with a Conforming Conventional loan.
3) If your loan is $750k to $3m+++ there are very, very few mortgage bankers who compete in this arena (Guaranteed Rate, LoanDepot, etc) but their bread and butter here is "Non QM" lending. If you have janky tax returns, or mystery money, or other non-conforming to lending norms within your credit profile, the mortgage banker might be the way to go. If you have a few bumps along the way, the major banks do these loans day in, day out, at rates and terms no banker can match.
On the plus side: Banks have a great number of getting a much lower rate than Broker/Bankers here. If you have unusual income, a Banker will have some options for you, but the terms will match the need for flexible lending.
On the minus side: For the time being, it will take 60-90 days to complete your refinance.
Bear in mind that Big Banks were caught understaffed with the push down in rates. What was 120 day turn times are now 60-90 days - 60 if you're well prepared. What is currently 60-90 days will shortly be 45 days, perhaps sooner. Also, not much of the information above applies to purchase loans. That's a different post altogether...
Hope this helps,
SGIP