The Story of African-Americans in Rabun County: Slaves, Segregated Work Camps and a Grandmother Who Was Purchased

African-Americans were among the earliest residents of Rabun County. They were slaves.

Rabun County was created by an act of the Georgia legislature in 1819. The new county’s land was offered to white settlers through the Land Lottery of 1820. Unique to Georgia, this lottery system distributed 250-acre parcels in the county’s river valleys, which contained the most arable land. Parcels in the mountainous portions of the county were surveyed into holdings of 490 acres. Regardless of location, both 250 and 490-acre lots were acquired for pennies an acre.

Like most mountain counties, Rabun’s topography prevented the formation of many large farms, which largely eliminated the need for a significant amount of slave labor. However, the availability
of cheap land nevertheless attracted slave owners, many from South Carolina.

205 Slaves in 1860

According to the “1860 Slave Schedule” compiled by James Bleckley, 49 families owned 205 slaves in Rabun County. This stands in stark contrast to the vast slave populations on the sprawling cotton plantations farther south in Georgia.

The county’s largest slave owners farmed large tracts of arable land in the Little Tennessee River Valley north of Clayton and the Warwoman Valley east of Clayton. Captain Samuel Beck owned 18 slaves, making him the largest slaveholder in Rabun County. He was followed by Hiram Gibson (17), G.A. Greenwood (16), John W. Scruggs (14) and Edley Powell (13). The county’s other slaveholders owned far fewer slaves. The Slave Schedule reported that many families owned only one or two, most likely for domestic work or house chores. Many of the county’s 205 slaves were children. The oldest was listed as 72. Most were in their 20s and 30s.

Samuel Beck came to Rabun County from Pickens, South Carolina without slaves to participate in the 1820 land lottery. He won 490 acres along Dick’s Creek in the Warwoman Valley. After serving as captain of a battalion of Georgia volunteers in Florida’s Seminole Wars during the 1830s, Beck started acquiring large tracts of land. He eventually owned more than 2,000 acres, stretching east from his initial homestead to the Chattooga River. To meet his labor needs, Beck
acquired slaves.

Slave Owner Voted Against Secession

Beck and Horace W. Cannon were chosen to represent Rabun County at Georgia’s secession convention in then-capital Milledgeville in January 1861. Reflecting the county’s generally pro-Union sentiment due to the absence of widespread slavery and their fear that war would wreak havoc on Rabun’s economy, slaveholder Beck and Cannon initially voted against secession. However, bowing to intense pressure from firebrands, they ultimately voted to secede.

Hiram Gibson, a South Carolina plantation owner and the county’s second largest slaveholder, purchased 1,000 acres of farmland in 1851 in the Little Tennessee River Valley in the vicinity of
present-day Mountain City. He moved his family and slaves to the site. His granddaughter eventually opened a boardinghouse that became the York House hotel on a portion of this land.

Slavery in Rabun County grew modestly over the two years following 1860, based on information compiled by Andrew Jackson Ritchie, author of the 1948 Sketches of Rabun County History. He
listed 60 families that owned 248 slaves in 1862. Although Ritchie did not provide a source for these statistics, it can be assumed that part of the increase in the county’s slave population
resulted from births. However, given the increase Ritchie reported in the number of slaveholding families, it is a virtual certainty that slaves had been purchased by new owners at auctions in
Atlanta or other southeastern cities.

Slaves Likened to Horses

Based on what Ritchie said was his personal acquaintance with former slave owners, he was certain Rabun County’s slaves had been treated kindly. “It was simply a matter of good business
management to treat the slaves kindly and keep them in healthy and comfortable condition. To be a good slave master was like being the owner of a good horse. An able-bodied slave man was
worth several horses. A slave child was always valued at $100 or more. An adult slave man in good physical condition was in some cases valued at $1,000.”

Following emancipation, some newly freed blacks continued working the land of their former owners, probably as sharecroppers. According to Ritchie, “There was something about them and
their experience as slaves that made them feel as if they were members of the old master’s family.” A Rabun County native, Ritchie was expressing beliefs that probably were widely held at
the time he wrote his history over 70 years ago.

Jobs for African-Americans

The coming of the Tallulah Falls Railroad in the late 1800s created job opportunities that drew large numbers of African-Americans to Rabun County. Black workers were hired to build the line’s
roadbed. The railroad spurred the development of tourism, first in what became the resort mecca of Tallulah Falls and then in other communities as the railway was extended north. Hotels
became a prime source of employment for African-Americans. The railroad also facilitated the logging industry, which employed many black workers.

From the construction of the Tallulah Falls hydroelectric dam that started in 1910 through the completion of the Nacoochee Dam in 1926, Georgia Railway and Power, the predecessor of
today’s Georgia Power, hired hundreds of African-Americans for dam construction labor. Work camps were segregated. The “Negro Workers Village” at the site of the Mathis Dam’s Terrora
powerhouse included 93 one-room shacks, 42 two-room shacks and a dance hall.

Segregated Schools

Work camps were not the only thing that was segregated in Rabun County. So were the public schools. According to a state survey of Georgia’s rural schools in 1914, black children attended
two of the county’s 29 one-room schoolhouses. The Well’s Chapel School was located in Rabun Gap, while Ivy Hill School in eastern Rabun County was near what was called “Colored Town.”
The 1914 survey indicated that 80 black children were enrolled at the two segregated schools.

In a Clayton Tribune article, Hattie Fortson, who attended Ivy Hill, said desks, books and playground equipment all came from the white schools. Many of the books were missing pages.
“It used to make me so mad. You never knew the beginning or the end of a story.” Sadie Owens added, “If you got into arithmetic, you had to start in the middle of the book and had no idea what
went on before.”

Motor oil coated the floor of the Ivy Hill School to keep the wood from rotting, and sawdust was used to keep students from slipping on the oil. Fortson said her shoes would be covered with oil
and sawdust by the time she returned home.

Wells Chapel School

The 1914 state survey included a glowing comment about the Well’s Chapel School for black children. “Pupils here read remarkably well and with thorough understanding…Children could
write well and answer intelligently all questions that were asked. The methods of teaching were far above the average.” Nothing remotely similar was written about the county’s 27 white schools.

Since African-American students could not attend Rabun County’s segregated high school, they were forced to travel to Cornelia or Toccoa, which had black high schools. Fortson said her father
drove her and her sisters to Tallulah Falls, where they caught the bus to Cornelia. Rabun County High School started admitting black students in 1965.

“The Whites Was Bad On The Coloreds”

In a 1977 interview conducted by the Foxfire Museum and Heritage Center in Mountain City, Harley Penland, an African-American, said, “My mother’s name was Georgia Scruggs. She was
from here. The Scruggs had bought my grandma and so my mama carried their name.”

“This place used to be tough country, you know,” Penland continued. “The whites was bad on the coloreds. It was almost as bad as the way they treated the Indians. I don’t know which was the
worst. They kept the coloreds to do the work and run the Indians onto the reservation.” He recalled the time a jury found an African-American guilty of a crime and recommended that he be
placed on a chain gang. “The judge said, ‘I oughn’t to do that but because he’s a (N word), I’m going to give him some time (on the chain gang). That’s the reason I tell you this place used to be

Depression Spurs Black Outmigration

With the onset of the depression in 1929, jobs vanished for African-Americans as well as whites. Harley Penland said many of his friends and relatives simply left Rabun County in search of
employment. “It got so we couldn’t get jobs at all, and then some scattered to Atlanta and up to New York and around to other places…They could get work then by going north.”

America started preparing for World War II by the late 1930s. Manufacturers of airplanes, tanks, guns and munitions were clustered in northern cities and along the West Coast. Lured to these
regions by jobs in defense plants, millions of African-Americans abandoned the South. Rabun County’s black population shrank dramatically.

Rabun County’s population of African-Americans is far smaller today than it was in the 1920s. The U.S. Census reported that 106 African-Americans were living in Rabun County in 2020. In his
1948 history, Andrew Jackson Ritchie guessed that the “negro colony” in eastern Rabun County also totaled about 100 residents. He wrote, “They are well behaved and are on good terms with
their white neighbors. We have no race trouble with these people.”

This article by Society member Richard Cinquina was originally published in the Laurel of Northeast Georgia magazine in July 2022.

About the Rabun County Historical Society 

The Rabun County Historical Society is dedicated to keeping alive Rabun County’s 200-year history in the Appalachian Mountains of Georgia. We collect, preserve and display important historic artifacts, photographs and records in our 2,300-square-foot museum and archives located at 81 North Church Street in downtown Clayton, Georgia. The Society is a not-for-profit organization under Section 501(c)(3) of the Internal Revenue Code, making membership dues and donations fully tax deductible. For more information, please contact us.