Baltimore police arrested five people last week accused of impersonating city tax collectors and going to the homes of elderly residents to rob them.

These are disturbing allegations. But apparently, scam artists who pretend to have ties to the government so they can take advantage of older consumers aren't all that rare.

That's what senior advocates are telling the Consumer Financial Protection Bureau, the federal agency with a mandate to promote financial literacy among those 62 and older and protect them from fraud and abuse.

Since mid-June, the CFPB has been gathering public input on the financial exploitation of older Americans. It received more than 750 comments by last week's deadline.

Consumer advocates submitted reports on a wide range of the abuses they've witnessed, including con artists pretending to be affiliated with the government in a ploy to sell investments. Often, though, it's not a stranger ripping off seniors but a trusted family member, caregiver or financial adviser.

"It's a very difficult message," says Sally Hurme, an elder law attorney with AARP. "How do you tell seniors the person most likely to rip you off is your son and daughter?"

The CFPB says it will use the comments to develop tools for seniors so they can make sound financial decisions and safeguard their assets.

It better act soon. By many accounts, financial exploitation of older Americans is widespread — and growing.

"It's going to continue and get worse as more seniors move into their so-called golden years," says Don Blandin, president of the nonprofit Investor Protection Trust, or IPT. "They are still going to have a lot of wealth that could be taken from them."

Two years ago, the IPT reported that one in five people age 65 and older had been swindled. But Blandin says the problem is likely much worse, given the findings of the IPT's latest poll.

The online survey of more than 750 experts — including regulators, social workers, elder-law attorneys and financial planners — found that nearly 80 percent said the greatest financial threat to seniors comes from family members.

"Think of what that really means if family members are engaged in ripping off money from one another," Blandin says. "How much of that is really going to get reported?"

But, of course, relatives aren't the only bad actors. Many older investors are exploited by the financial advisers they hire to help them manage their money through retirement.

A recent poll of about 2,650 planners by the Certified Financial Planner Board of Standards found that more than half worked with an older client who had been a victim of deceptive or abusive practices by another adviser. Planners say they encourage victims to report abuse to authorities but estimate that only 5 percent do so.

These figures don't surprise Barbara Korenblit, chief of the individual and family services division at the Baltimore County Department of Aging.

Most often, she sees exploitation by family members, such as a relative dipping into a joint account shared with an older person, Korenblit says. Or a young adult might have a grandparent co-sign a loan and then not repay it.

"Sometimes older clients have lost their homes as the result of the debt they have taken out to help young relatives," she says.

Older consumers often don't complain because they don't want to get a relative into trouble, particularly if that person is the caregiver, Korenblit says. Even when an adviser takes advantage of them, she adds, seniors don't report the abuse because they aren't aware of it or are too embarrassed to admit it.

Senior advocates urge the CFPB to beef up protections.

The Virginia-based National Academy of Elder Law Attorneys suggests that stronger penalties be adopted against those exploiting seniors. It also suggested a law that would require doctors and bank employees to report financial elder abuse.