Vowing to recover "every single dime the American people are owed," President Obama today proposed a new tax on the nation's largest financial institutions to recover projected losses from the government's bailouts.

"We want our money back and we're going to get it," Obama said in a short but sharply worded White House speech. "If these companies are in good enough shape to afford massive bonuses, they surely are in good enough shape to afford to pay back every penny to taxpayers."

Obama's tough talk comes amid public anger about big profits and large bonuses at major financial firms as Wall Street -- with the help of taxpayer money -- has returned to robust fiscal health while much of the nation struggles to recover.

Obama said the bailouts, which were begun under the Bush administration, were "as necessary as it was unfortunate" because of the vital role large financial institutions play in the economy. Even though many of the firms have repaid the money, Obama said that's not good enough.

"My commitment is to recover every single dime the American people are owed," he said. "And my determination to achieve this goal is only heightened when I see reports of massive profits and obscene bonuses at some of the very firms who owe their continued existence to the American people, folks who have not been made whole and who continue to face real hardship in this recession."

The new Financial Crisis Responsibility Fee would have to be approved by Congress. It would hit about 50 of the nation's largest financial institutions and generate about $9 billion annually for at least 10 years, administration officials said. The tax would be an annual 0.15% fee on a company's liabilities, excluding insured deposits, and would be assessed on banks, insurance companies and other financial firms with at least $50 billion in assets.

The goal is to offset projected losses of about $117 billion from the $700-billion Troubled Asset Relief Program, or TARP. About 60% of the fee would be raised from the 10 largest financial firms, among them some of the biggest names from the financial crisis, including Bank of America, Citigroup, Goldman Sachs Group, JPMorgan Chase & Co. and Morgan Stanley.

But most of the large recipients of bailout money have repaid all of their infusions, and the Treasury Department has projected a profit from the TARP program that injected money into banks because of payments from dividends and stock warrants.

The tax, which would begin on June 30, would be assessed on firms that have repaid their money and some that never received any bailout. It would not be assessed on automakers General Motors Co. and Chrysler Group, which have received about $64 billion in bailout money and are projected to account for a large share of the losses. Such a fee would be logistically difficult to impose on a manufacturing company, according to a senior administration official.

The administration also is spending about $50 billion of TARP money on its plan to offer incentives to banks to modify home mortgages to reduce foreclosures. That program's cost is included in the projected losses for TARP.

"It is thus possible that legislation may result in financial institutions being charged for losses made on investments in two automobile companies and on foreclosure mitigation efforts," said a report released today by the Congressional Oversight Panel monitoring TARP. "On the other hand, it may be argued that many of the financial institutions that received TARP funds would not have survived absent such capital injections, or, even if they themselves were not short of capital, would have been vulnerable had other giants in the industry fallen, and therefore asking for these institutions to contribute to an overall TARP shortfall is appropriate."

Large banks said asking them to make up the TARP shortfall is unfair.

"Two-thirds of the TARP investment from banks has already been repaid with a large profit to the taxpayer," said Steve Bartlett, president of the Financial Services Roundtable, a trade group of large financial firms. "This proposed tax will do nothing more than stifle economic recovery and encumber more pressing concerns, such as covering new regulatory costs. . . . This tax is strictly political."

Speaking to reporters after a hearing Wednesday on the causes of the financial crisis, Jamie Dimon, chief executive of JPMorgan Chase, said the financial industry should not have to pay for losses caused by the auto companies.

"At some point, you've got to be fair," he said.

Some Republicans also said they were opposed to the new fee, echoing bankers' concerns that it will limit lending as the economy struggles to recover from the recession.

"This proposal is the latest folly in the Obama administration's failed attempt to borrow, spend and tax their way into economic prosperity," said Rep. Jeb Hensarling (R- Texas). "To think that banks will loan more money if you tax them more is fundamental economic ignorance. It is small wonder the nation remains mired in double-digit unemployment under such policies."

Obama said executives of large banks should take responsibility for the fee rather than fighting it.

"What I'd say to these executives is this. Instead of sending a phalanx of lobbyists to fight this proposal, or employing an army of lawyers and accountants to help evade the fee, I'd suggest you might want to consider simply meeting your responsibilities and I'd urge you to cover the costs of the rescue not by sticking it to your shareholders or your customers or fellow citizens with the bill, but by rolling back bonuses for top earners and executives," he said in some of his sharpest rhetoric yet about the role of large firms in the financial crisis.

Some congressional Democrats would like to tax the large financial firms even more. In addition to Obama's tax, they want to tax large bonuses paid to employees of those firms. Rep. Peter Welch (D-Vt.) has introduced legislation to impose a 50% tax on bonuses of more than $50,000 paid by companies that received bailout money.

Rep. Barney Frank (D-Mass.), chairman of the House Financial Services Committee, said Wednesday he supported the administration's bank tax and also would like to raise the marginal tax rate on those bonuses, which is now 36%. He plans hearings next week on limiting Wall Street pay.