A great many political debaters accuse their opponents of arguing ad hominem, and therefore losing the debate. Usually, these charges are wrong: the character of the person making the argument is an enormously important guide to its merits.
If I’m arguing with a smug Tory bastard, and I suggest that he’s talking out of his arse because he’s a smug Tory bastard, then this is indeed dodgy ad hominem argument. Bad me.
However, this is not the same as trying to gauge the truth of a statement by a known liar, or to judge how factually well-informed a notorious idiot is, or to work out whether a trade union regulation plan proposed by someone who hates trade unions will benefit union members.
A person’s background as a liar, idiot or scab doesn’t affect the *logical* validity of their argument. However, logical validity is only relevant if you accept the truth of an argument’s premises. Since you first need to work out whether you can trust anything they’re saying, assessing past behavior is a vital tool in the debating process.
(and yes, people who argue otherwise only do so because they’re liars and idiots.)