If this is your first time to my site, welcome! You’ve plopped into the middle of my family lineage obsession. Or rather, my extremely focused effort to organize facts and wrap my brain around one part of my tree’s ancestral slave ownership. I am giving one chapter to each of the Scott men from who I am descended and telling as much of their stories as I can figure out.
I do not typically write entries with 1800s census forms, old pictures, and seemingly random historical facts, so hopefully you aren’t scared away by the sight of what essentially appears to be my boring book reports {blogger pauses at the thought of expanding project to dioramas}.
Hang in there, I’ll get back to posting pictures of my dog, adventures with my Little Sister, and home DIY projects. I promise.

Robert Garland Scott, Sr. 1830-1909
My second great-grandfather Robert Garland Scott was born to William and Eliza on November 29, 1830. Thanks to more records and an article written about the Riverside Mansion, I can share more about him than I could about his predecessors. I did find some inconsistencies with dates between what I found in censuses (and gravestones) and what the article talks about, so in truth, I don’t know how much is accurate.
But first a recap! I’ve highlighted who I’ve covered so far (click to enlarge);
Predictably, I know little of what Robert’s childhood was like aside from understanding that they lived along the James River in Amherst County, VA and likely watched boats bring stuff in and out. The Riverside Mansion article read “He would have seen tobacco being “prized” (which means pressed) into hogsheads (barrels), inspected, stored in warehouses, and when the water level was ideal for travel, loaded by boatman onto bateaux for shipment”. When full of leaf tobacco, hogsheads weighed about one thousand pounds each!
I can also tell you that when Robert was nine-years old;
Louis-Jacques-Mandé Daguerre introduced the daguerreotype worldwide in 1839. The daguerreotype was the first publicly available photographic process, widely used during the 1840s and 1850s.
And in 1846, when Robert was around sixteen years old;
Elias Howe refined the sewing machine design concepts of his predecessors, and in September 1846, he was awarded the first United States patent for a sewing machine using a lockstitch design. He traveled to Europe for funding, and when he returned to MA, found others manufacturing sewing machines. Howe was forced to defend his patent in a court because he found that Isaac Singer with cooperation from Walter Hunt had perfected a facsimile of his machine and was selling it with the same lockstitch that Howe had invented and patented. He won the dispute and earned considerable royalties from Singer and others for sales of his invention.
Back to Robert.
The 1850 Census, taken when he was 20, says Robert was a Boatman, in addition to mentioning others in the household including his oldest brother William, 28, who is listed as being a contractor on the canal. Their sister Frances (who, I’ll point out, is noted as being 20 when she was actually 24 at the time) is also in the census count.
I tease about Frances but A) Maybe the census-taker was cute and she’d been single for a long time and she didn’t want to seem like a spinster so she lied. Or B) She had nothing to do with the mistake. I’m envisioning the scenario; The census-taker is tired. It’s been a long day out in the heat so he just asks the first person he spots on the property, 19-year-old John Black who is an overseer in the field “Who lives in the house? What do they do for work?” and so on. John isn’t really sure so he lifts his wide-brimmed hat to give his sweaty head a little scratch and takes his best guess. The results I share with you below, written forever in history as fact. Nice job, Mr, Black.
I do wonder where these five lived. They clearly didn’t live with William Waller and Eliza. Maybe Mom and Dad built them a house on their massive acreage so they didn’t have to listen to them party late into the night anymore. “But you have to pay your oil bill yourselves!” they’d tell their kids and their bad-influence friends.

Robert married Sallie Shelton in November 1859, when he was 29 and she was… wait for it…15.
Gross, Robert.
When I was 17, I was competing in marching band competition, making stupid sketch comedy videos with my friends, and getting ready to leave for Ohio University). When Sallie was 17 she had their first kid, Sarah Elizabeth “Lizzie” Scott. She had another child basically every two or three years for the next twenty years. In the end she had nine children; Sarah, Florence, Fannie, Robert, William, James, Hugh Roy, Waller, and Edwin.
In 1860, Robert and his brother James became partners in a grist mill business. Together they “obtained” 55 acres from a brother-in-law along the river which was likely used for shipping. The area they purchased included a ferry crossing, and a nearby lock was leased to provide power to the mill.
Robert also had slaves at this time according to the 1860 Slave Schedules. I don’t know if they worked at the mill, but in the 1870 census (which was after emancipation), he had a large farm with a lot of produce being harvested, so I think it’s possible that the farm existed during the time the 1860 Slave Schedule data was collected, but I’m speculating.



Civil War. 1861-1865 The economics of slavery and political control of that system that was central to the conflict. The Southern states wanted to assert their authority over the federal government so they could abolish federal laws they didn’t support, especially laws interfering with the South’s right to keep slaves and take them wherever they wished. The South also wished to take slavery into the western territories, while the North was committed to keeping them open to white labor alone. The election of Abraham Lincoln in 1860 sealed the deal. His victory, without a single Southern electoral vote, was a clear signal to the Southern states that they had lost all influence. They seceded, which lead to war.
Robert enlisted in the Confederacy, serving in Kirkpatrick’s Amherst Light Artillery in September 1861. In August 1861 he was promoted to 1st lieutenant. This sequencing makes little sense to me, but it’s what the records say.
He declined to reenlist and “mustered out” on 15 May 1862. I like to think it’s because he didn’t support the cause of The South, but since he was working as a canal contractor three months later, it’s more likely he was given exemption to help the cause in another way. Various tradesmen were offered deferments if their work contributed to the war effort by way of supplies or transportation. (Again, I got this from the article I linked to above. I don’t know these things myself. Not an historian, people)
James sold his part of the grist mill business to Hugh Roy Scott around this same time. It’s unclear which Hugh Roy this was. When I do a search within my Ancestry tree for “Hugh Roy” five of them pop up (this lack of creative naming is problematic for your descendents, people. Consider that when naming your kids.). It was either a brother or a cousin.
By December 1865 the war is over, the North has won (yay!), and the requisite three-quarters of the states had ratified the Thirteenth Amendment to the Constitution, which ensured that forever after “neither slavery nor involuntary servitude… shall exist within the United States.”
Over the next fifteen years there are multiple severe floods which kill people and damage the canal, a nearby dam, and the mill. Robert worked much of this time as a subcontractor helping repair the damage.
He also acquired, within this window of time, $15,000 worth of land.

Let’s talk about the 1870 census. At this point, Robert has been married to his inappropriately young wife for a little over a decade. One of his sons, my great-grandfather William Edwin, was born after Robert (the four-year-old listed in this census) and before James. So where is William Edwin? Also, James wasn’t born for another six months after this census was supposedly taken. Not to mention his name was James Pendleton Scott. So who is this James W clown listed up there at the bottom?
Were people just unconcerned about accuracy back then? I’m imagining it; The grown, grieving children allocate the brothers to the responsibility of meeting with the headstone carver. They stand in front of the bewildered, chisel-holding stone carver, and one brother turns to the other “When was Mom born exactly?” The other thoughtfully bites his lower lip “I don’t know. Give it your best guess.”). I’m not being fair to men, I realize, but sometimes this lack of info asks for storytelling, albeit lame storytelling.
Perhaps this census-taker performed their info-collecting duties while intoxicated. Perhaps my second great-grandparents were intoxicated when questioned about the makeup of the household. (Actually, with that many kids it’s not a terrible idea).
Who knows. The mystery of history!


The Riverside Mansion article states that in 1870 Robert headed a household of ten, with eighteen servants (ten of them children) listed as “under his care”. I’ll mention here that I only see seven people listed, including him, as living in the home. I’m sure there were servants, but the family member headcount is off for sure.



On Robert’s farm, wheat, corn, oats, and tobacco were harvested. He also owned some pigs, a few milking cows, horses and “other cattle” (what are those, exactly?).
Robert spends the next ten years dealing with the over-flooding of the canal, which happens a few times, and working to keep his mill running, which ultimately fails by 1880.
In 1880 the Richmond and Allegheny Railroad bought out the canal company and began laying steel on the path along the river to Bedford. Robert, along with other canal pros, was hired to work with surveyors and as contractors. That same year, Robert bought “the Bethel tract including the old Tinsley Mill on Salt Creek. This property is where the Scott family later moved into a large mansion and boarding resort they named Riverside.”
In 1884, Sallie died at 40 years old. Her youngest child, Edwin, was four years old. I can find no record of who her parents were or where she came from, socially-speaking. She married so young, and while I know this was common, I feel for her, a fifteen-year old bride, and wonder how she died at 40.
One year later, Robert married Ide E (Scott). Perhaps a marriage of necessity for a widow with nine children. Perhaps not.
At this time Robert was working as a land agent and had a sawmill operation. He later became postmaster at the Bethel/Salt Creek post office from ’88-’95. and again from 1897-1901. The couple continued to host visitors at the Riverside Resort, many of whom would arrive at Reusens depot, take a short walk up to Judith’s Dam, and hitch a ride on Elmer, Robert’s steamer, which was steered by an old boat captain. This unnamed captain and Robert would both bend the ears of the riders with bygone river stories.
People visiting were said to enjoy dancing, boating, bowling, and bathing. After dinner, watermelon feasts sometimes took place on the lawn.
The last census I can find is from 1900 ands contains more specific questions. It lists Robert, Ida, his daughter Frances, and a servant named Sallie Lee. It also shows that Robert’s most recent wife is 34 years younger than him. (Insert cringe emoticon here) Oh, Robert.
Seeing Sallie’s name of course makes me wonder if just four years prior, she was forced to work for them for free. How much was she making now? Did her two children hang out with her while she worked?
In 1904, Robert was forced to sell his property. It’s not clear to me why, but he retired to Lynchburg where he died on December 14, 1909 after a year long illness. He was buried in the Scott Cemetery.
Embarrassingly, this photo from 2013 is better than the one I took in June.
He was 79 years old.
Today there is a park called Monocan Park where the mansion once stood. Traces of the mansion include a stone retaining wall, and near the parking lot, two Nordmann Fir evergreen trees, native to Turkey, stand tall. The firs have been identified as being around a hundred years old, and for them to have taken root, they would have needed to be planted as seeds.
The Riverside Photo
Since I’m a photographer, I want to talk about this photo. It was supposedly taken in 1885. Robert is front and center and I believe Ida is to the left. In ’85, Robert was 55 years old and emancipation had taken place a full twenty years prior. The Scotts had owned Riverside for about five years at this point. Had his farm failed? If so, was it because Robert had to let his slaves go free? Or, if I dug around enough, would I find that he gave the farm to one or a few of his grown children to tend to? I know what three of his boys did as adults (to be continued…)
This photo interests me for a few reasons. Supposedly these are his family members seated with Robert. By now George Eastman of Eastman Kodak had invented film, but it was another five years or so before the Kodak camera was available to the public.
So this photo was shot during the sit-very-still era of photography. With that in mind, the photo feels very orchestrated in its effort to seem relaxed and candid. Look at the man on the porch reading the newspaper. Why is he there and not tucked back enjoying the shade of the porch itself? Because he’s been placed there. Also, I noticed he may actually be an African American.
All of the faces of the people in the photo can be seen, which to me means they were composed in this manner. My deepest interest comes from the two children. The one on the stairs appears to be white, but I’m not sure. She is wearing a rather flouncy dress, which implies wealth, or at the very least, middle class.
And then there is the little girl in the back at left. She stands out from the lattice work of the house, and appears to be wearing a simpler dress. She is clearly African American.
She is where my eye goes every time. Who was this little girl, with her body language that, even with the lack of clarity in the photo, feels, I don’t know, sad? What was she doing there? And I don’t mean to say she doesn’t belong, but rather point out that photographer clearly asked her to stand there specifically.
Compositionally-speaking, that left side is dead space. Putting a person there gives it weight.
To me the weight feels massive, even though its coming from this tiny child.