Highest Number of Slaves Owned: 118
Although James Madison knew slavery was wrong, he remained a large slaveholder throughout his adult life.
In private letters, Madison referred to slavery as a "dreadful calamity" and a "sad blot on our free Country," and he wrote to Frances Wright on September 1, 1825: "The magnitude of this evil among us is so deeply felt, and so universally acknowledged, that no merit could be greater than that of devising a satisfactory remedy for it."
Yet Madison never emancipated his slaves (despite a promise he made to do so), and he advocated sending Black people to Africa rather than abolishing slavery. Of freeing slaves in America, Madison wrote on October 20, 1789: "Neither the good of the Society, nor the happiness of the individuals restored to freedom is promoted by such a change in their condition."
In a written response to questions posed by Jedediah Morse in 1823, Madison described free Blacks as "Generally idle and depraved; appearing to retain the bad qualities of the slaves with whom they continue to associate, without acquiring any of the good ones of the whites." Elsewhere, Madison wrote that Black people were marked "by Physical & lasting peculiarities" and that they had a "natural and habitual repugnance to labour."
To extract slave labor, Madison instructed his overseer to "treat the Negroes with all the humanity and kindness consistent with their necessary subordination and work." The amount of work must have been high, for Madison told a British visitor that he could make $257 per Negro annually for only $12-13 in upkeep.
Perhaps most regrettable of all, Madison did not oppose the enslavement of children. For example, when Madison was informed that a French visitor had "procured a Negro girl, and only wants a boy in order that they may breed," he arranged for funds to purchase a "Negro boy."