In the ghost town of Carrollville, Mississippi, there was once a dry goods store run by “Clayton & Walker.” The proprietors of this 1840’s establishment – Colonel Richard B. Clayton and his son-in-law, future Tishomingo County Sheriff Porter Walker – were original movers and shakers in this part of the world, and for almost two centuries now, their families have made an impact on Baldwyn, north Mississippi, and the southeast region of the country.
The late Annie Laurie Arnold, a direct descendent of Colonel Clayton, peaked my interest in this iconic Baldwyn-ite a few years ago with her extensive collection of family photographs, stories and documents. One of those documents was a copy of an interview given by former Baldwyn mayor Thomas G. Stocks to the Mississippi Historical Society in 1902.
“In 1836, R. B. Clayton took charge of the village tavern.”
That’s the enigmatic line found in the Stocks’ interview that prompted new research into the life of Mrs. Arnold’s great, great grandfather – Colonel R. B. Clayton.

Perhaps the most ornate marker in the Baldwyn Masonic Cemetery can be found at the gravesite of Col. Clayton and his second wife Margaret. Yet genealogical researchers still search for the definitive reason he was even called “Colonel.” Born in 1790, he would have been in his mid-twenties during the War of 1812, and given his later Post Master appointments, civic service and land acquisitions, a case could be made that he came out of the southern theater of that conflict on the “good side” of Andrew Jackson, John Coffee and other frontier leaders who ultimately landed in high seats of governance. But so far, just why the Colonel was a “colonel” is not known precisely.
What is certain about R. B. Clayton was that he had no qualms about taking charge. Before his days in Mississippi, Clayton began life as a son of the Appalachians, born in Person, North Carolina, his father an equally enigmatic mountain man named “Flat River” Clayton. Richard made his way west as a young man and found a niche in Winchester, Tennessee, where he was a respected merchant by 1819. There in Winchester, he met and wooed his first wife, Sarah “Sally” Rutledge. Sally was the daughter of General George Rutledge, the commanding officer of the Tennessee militia, a wealthy landowner, and second only to John Sevier in the Tennessee political hierarchy of the day.
After they married, Richard and Sally moved almost immediately into former Cherokee Indian territory in north Alabama, another indication that he may have served under Jackson in the War of 1812. Land in Alabama would have likely been granted to those who had fought on the southern frontier in that conflict against the British and the Creek Indians.
B. Clayton was a man who gained the respect of his neighbors. He became County Clerk in Jackson County, Alabama, for several terms, after having been among the five men appointed commissioner to purchase the land for the original county seat in 1827.
Richard and Sally had five children together. The first was Angerona Moore Clayton who eventually married J.O. Nelson in old Tishomingo County, Mississippi, and became ancestor to Baldwyn’s current Nelsons. Their second daughter Annis married William B. Hunt, a grandson of John Hunt. Hunt was the original settler of Huntsville, Alabama, where none other than Richard B. Clayton was an original land owner. It really was a small world in the south in the early 1800’s. The Clayton’s last child was their only son, George Rutledge Clayton. Sally died after giving birth to George on June 11, 1828.
With four daughters and a newborn son, Richard Clayton did not wait long before remarrying. He wed Margaret Rhea Weir on March 10, 1829. Like Richard, Margaret was herself a widow and a child of the Appalachians. She was descended from Hugh Weir who had fought at the Battle of King’s Mountain in the Revolutionary War alongside Sevier and Rutledge. Interestingly, Presidents George H.W. Bush and George W. Bush are also directly descended from Weir.
When Clayton’s term as Jackson County Clerk ended in August 1836, he, Margaret and ten children headed west to secure a large plot of plentiful land soon to be vacated by the Chickasaw Indians and where, as the interview said, Clayton “took charge of the village tavern.”
In Carrollville, Richard Clayton not only ran the tavern, he served as Post Master for a dozen years, ran a dry goods store and was an election commissioner. His Weir brothers-in-law were pastors at the local Presbyterian Church, and he was able to see his second son, 26 year-old Dick Clayton, elected Tishomingo County Sheriff and Tax Collector in 1860, continuing the family’s political prominence.
In November of 1860 as the Mobile & Ohio Rail Road approached north Mississippi, Colonel Richard B. Clayton disassembled his tavern, the Clayton Inn, and moved it two miles, from Carrollville into the emerging town of Baldwyn. He and Margaret actually lived in the depot for a time and coordinated local rail road and city construction efforts. He and the son-in-law with whom he ran the dry goods store – Porter Walker – may have even laid out the streets of Baldwyn, essentially just as they are today.
The destruction resulting from the Civil War, including the death of the Colonel’s son Dick at the Battle of Antietam, cued the final chapter of Richard B. Clayton’s life. Even the mountain man who had fought the Creek Indians with Andrew Jackson, the shrewd merchant who built businesses across Tennessee, Alabama and Mississippi, a man who married the daughters of generals and presidents could not escape the inevitable hands of time. On December 27, 1868, at age 78, Colonel Richard B. Clayton died. Richard Clayton was a patriot before he was a rebel. He was always a civilizer, a builder, a true mover and shaker. The town he helped build – Baldwyn – is alive and thriving 155 years after his passing.
Colonel Richard B. Clayton is the kind of man that should be remembered.
Talk of the Town is sponsored by Six Shooter Studios. Follow us at Facebook.com/sixshooterstudios