Yet so natural to mankind is intolerance in whatever they really care about, that religious freedom has hardly anywhere been practically realized, except where religious indifference, which dislikes to have its peace disturbed by theological quarrels, has added its weight to the scale. In the minds of almost all religious persons, even in the most tolerant countries, the duty of toleration is admitted with tacit reserves. One person will bear with dissent in matters of church government, but not of dogma; another can tolerate everybody, short of a Papist or an Unitarian; another, every one who believes in revealed religion; a few extend their charity a little further, but stop at the belief in a God and in a future state. Wherever the sentiment of the majority is still genuine and intense, it is found to have abated little of its claim to be obeyed.
You can find the complete essay at the link: http://www.constitution.org/jsm/liberty.htm