Bryan Garner, who dislikes the expression, cites an article in American Speech that theorizes that slurred speech led to "the loss of this alveolar closure." And fairness demands that I point out that Bill Walsh, whom I hold in great esteem, also advises against the idiom in Yes, I Could Care Less.
Even Messrs. Garner and Walsh concede that could care less is firmly established as a colloquial American English idiom. You probably do not want to use it in formal writing or speech, and you're perfectly free to shun it in your own casual conversation. But you can't pretend that it doesn't exist, or complain that it's illogical (grammar ain't algebra), or deny that you understand it when you come across it.
Ms. Weiss responded to the Language Log post with "Yes, and 'impactful' is a word."
Indeed it is, an adjective formed by normal pattern from a noun, with a spelling, a pronunciation, a meaning, an etymology, numerous citations, and an entry in dictionaries. That makes it a word.