"Nothing like this had previously existed in the long annals of human slavery."

An illuminating passage from Orlando Patterson's Slavery and Social Death:

The U.S. South shared with other slaveholding societies the exploitation of slave women and the inclination of masters to manumit their concubines and children. The intense shame that the master class felt about this sexual relationship was absolutely unique to the South, however. The guilt, with its disastrous consequence for the freedmen, had three sources. First, there was the puritanical tradition, which condemned fornication with the threat of fire and brimstone. Second, there was a highly developed sense of racial purity frequently codified in laws against miscegenation. And third, there was a strong moral commitment to a patriarchal family life, in which the women of the master class were placed on a pedestal and became symbolic not only of all that was virtuous, but as W. J. Cash has argued, of "the very notion of the South itself." The cult of southern womanhood was of course directly derived from slavery and the sense of racial superiority. Any assault on the dignity and honor of the idolized woman was an assault on the entire system. 

But southern males were no less pleasure-loving than the men of any other slaveholding society. Their hedonism, however, conflicted with their religious values, making the southern master alive to a deep sense of sin and wickedness: "the Southerner's frolic humor, his continual violation of his strict precepts in action, might serve constantly to exacerbate the sense of sin in him, to keep his zest for absolution always at white heat, to make him humbly amenable to the public proposals of his preachers, acquiescent in their demands for the incessant extension of their rule." Equally, his hedonistic exploitation of the slave women was an assault on the integrity of the idolized women, all of whom were constantly reminding him of his wickedness when they were not displacing their bitterness in acts of cruelty toward comely female slaves. 

The result of all this was that the freed group, with its disproportionate number of mixed-blood members, was a living reproof, a caste of shame, confronting the white males with the fact that they repeatedly violated not only their puritanical precepts but the the honor of their women. It was not guilt about slavery that accounts for exceptional hostility toward freedmen, as Berlin and others claim, or any real fear of them as a political threat, but guilt about their own violation of their own social order. The "zest for absolution always at white heat" made it imperative that the freedmen be scourged from their midst—or, if not scourged, punished, victimized, and defiled like scapegoats. 

Nothing like this had previously existed in the long annals of human slavery.