What is the difference between a bank and a mortgage broker? A mortgage broker is basically a middleman. Brokers work with a variety of lenders to find loans for clients, but do not lend out money directly. ... The lender could be a mortgage bank, which specializes in mortgages; it could be a large commercial bank, a community bank or a credit union.