But when we actually stop to consider the phrase “as long as they do not harm other people”, it becomes pretty clear that this is quite a limitation. The only society of total freedom would be a one person society. Of course, much of this depends on how we define harm (and whether we allow for self-definition), but, as soon as we factor in any form of social relations, we must limit our actions quite considerably for the benefit of others.
Indeed Hegel argued that the basis of social relations is precisely the respect for the person and good of others, and the freedom of others therefore depended on our restraint. Nozick, of course, makes a similar point when he defines freedom in terms of side-constraints: one person’s freedom is determined by the constraints placed on others. Kant goes considerably further and argues that we actually should not do things that harm ourselves as well as other people, and that this is entirely consistent with a liberal view of freedom (Kant, of course, equated freedom to rationality or pure reason).
Therefore it seems to me that what matters is not just what freedom I have, but what responsibility others have, or, put in Nozick’s terms, what side-constraints are placed on others to provide me with the opportunity to determine my own ends. This means that social relations are competitive – libertarians like Rasmussen and Den Uyl are wrong to see freedom as indivisible – and we have to beware of how we act in order to protect others.
So what I want to argue for is a libertarianism based on personal responsibility rather than on a simple statement of free choice. This is more in accord with the reality of social relations and provides us with a much more nuanced view of how we can deal with complex issues like the use of addictive substances, sexual relations and so on.