Is it true that large banks can borrow funds at close to 0% interest rates from the Fed and turn around and buy US bonds paying higher interest rates with the borrowed funds?
Yes, large banks can in principle borrow funds at close to 0% from the FED and turn around and invest them on higher paying US bonds. But, when large banks do this, they push up the market price of these US bonds (being large players in the market). With the same cash flows from a bond, a higher market price implies a lower yield to maturity or interest rate on the bond. Thus, with efficient markets, the kind of arbitrage opportunities you are pointing out are eliminated.