I should have thought that these were matters of political judgement, above all in telling the difference between the point at which the enemies of liberalism have been given only enough rope to hang themselves, and the point at which they have enough rope to hang someone else.
This piece by Jeremy Waldron in the LRB is, I think, excellent, articulating the same kind of view on the subject. The question which is at the centre of both Williams' and Waldron's critiques, which share the same concerns about the overly analytical style typical of much academic political theory, although not the illiberalism of some thinkers motivated by that worry, is what moral features of hate speech place in it a category where it is automatically protected? The intrinsic moral interest in systematically defaming whole categories of other members of the human race on the basis of the colour of their skin or their sexual orientation seems to me to fairly categorically small, while as Waldron points out, whatever virtue there is in the confrontation of such evils is as likely served by highlighting, through legal sanction, that evil as by allowing it a space and engaging with it in the forum. Summarising Mill's argument that
the truth could not emerge except through "the rough process of a struggle between combatants fighting under hostile banners", and that even where some doctrine is known to be true, we need combat with real opponents to maintain its vitality and keep us healthily on edge in its defence. "Both teachers and learners go to sleep at their post, as soon as there is no enemy in the field"
This seems to me rather unlikely. The best way to keep our minds sharp with the horrors of the Holocaust and other genocides is surely not to permit their existence to be denied, but to be brought to unavoidably face with those horrors, the ghastliness of the numbers, of the sheer administrative effort, of the weight of each and every individual who suffered through the attempt to exterminate a portion of the human race.
Waldron even allows himself a little jibe at those who seem to believe that hate speech is relatively costless, and hence to be protected:
[I]f [this belief] signifies anything, what it signifies is that the costs of hate speech, such as they are, are not spread evenly across the community that is supposed to tolerate them. The Robert Relfs of the world may not harm the people who call for their toleration, but then few of them are depicted as animals in posters plastered around Leamington Spa. We should speak to those who are depicted in this way, or those whose suffering or whose parents suffering is mocked by Frank Collin and his Nazi colleagues, before we conclude that tolerating this sort of speech builds character.
You do have to wonder, at times.