Born in Washington, D.C. and growing up in Alexandria, Virginia, Joe Naylor was blissfully unaware of his historic ties to Branford—until the inheritance of a family genealogy book turned the page.

“It was just pure coincidence,” says Joe.

After college, followed by a short stint acting in New York, a visit to his sister in New Haven led Joe to meet his wife, Barb. They were already living in Branford in 1982 when Joe inherited the book that held the family history.

“I was flipping through the book, and I discovered that I had an ancestor named Elizabeth Swayne who was born in Branford in 1646,” says Joe. “When you see something like that, you have to look into it.”

Joe reached out to the Branford Historical Society (BHS) to find out more.

“They were very helpful. I found out Branford was founded in 1644, so I thought my ancestors must have been some of the original settlers of Branford,” says Joe, who also joined BHS as a member.

That was nearly 40 years ago, at a time when Joe he was busy with family and his career as an IT manager for SNET/AT&T. But as soon as Joe retired in 2005, he began to give more of his time and talent to BHS. One of his first contributions was organizing volunteers at the society’s iconic, red 1724 Nathaniel Harrison House museum at 124 Main Street.

Through the years, Joe had also been digging into his family’s ties to Branford, leading back to a certain, wealthy Puritan relative, William Swayne (also spelled Swaine), who sailed from Brentford (sometimes spelled “Braynford”), West London, to settle near Boston, Massachusetts, in the 1600s. Swayne went on to help found Wethersfield in the Connecticut colony. From there, he became one of the first settlers—and leaders—of Branford.

Joe’s love of history has led him to share much of what he’s learned with others in town, most recently with Zoom lectures sponsored by BHS with Branford’s Blackstone Library (now on YouTube; search “Branford Beginnings: Presented by the Branford Historical Society”). He also likes to share what he’s learned about Branford in his membership with the Tuesday Discussion group of the Canoe Brook Senior Center, led by his friend Lloyd Buzzell.

As a speaker, Joe spins a fascinating recounting of the past. Meeting The Sound at the Harrison House recently, Joe even went back to his acting roots, donning a tri-cornered hat and donated, reproduction Colonial militia uniform (it’s a perfect fit). The uniform usually hangs on display in the museum’s second floor Exhibit Room, filled with local historic items from points throughout Branford’s past.

Taking a seat in the building’s rear lean-to (added about 1780, for the Harrisons and their eight children), Joe begins, “in about 1643, there was tension between the Puritan factions in Wethersfield. So [Rev.] John Davenport [a founder] of the New Haven Colony, came up to Hartford to try to settle the differences. He finally decided that about half the town was better off going to this area called Totoket and starting a new settlement.”

Swayne was a designated leader of the faction set to move to Totoket. The Native American name means “place of the tidal waters” and was part of a territory purchase by the New Haven Colony in 1638 for 11 coats “10 of trucking cloth and one of finer material,” Joe notes. Joe says warring factions, such as the Pequots, together with deaths due to disease brought by the first settlers, had greatly diminished the local native population by the time of the purchase.

In 1644, Swayne was among some 35 families arriving to settle the area as a town.

“Some came overland with their livestock from Wethersfield, and most came in boats, all the way to the Branford River. They settled where the town green is,” says Joe. “They built houses around what now is the green, which was pretty much in the same triangular shape it is now.”

The new settlement wasn’t called Totoket for long, and Joe’s ancestors may have had a hand in that. After researching histories of as many of the original settlers as he could locate, “I was only able to pin down about half the people,” Joe says. “But the [Swaynes] were the only family I could find that were from this city, Brentford. And I assume because William had originally come from this town of Brentford, that the town that was settled here soon was changed to Branford.”

Swayne had two sons, Samuel and Daniel, and a daughter, Mary (who, when living with the family in Wethersfield, was kidnapped by Native Americans, then rescued by a Dutch ship—but that’s another story). Swayne, in his 60s when he arrived here, was selected as one of three town leaders, together with leaders of the church.

“So you could say he was one of the first ‘selectmen’ of Branford,” says Joe.

What makes this family historical connection more remarkable for Joe, as someone who spends plenty of time at the Harrison House, was learning some of his ancestors went on to own the land on which the Harrison House is built.

Although Captain Samuel Swayne left Branford in 1666 with Reverend Abraham Pierson to found Newark, New Jersey, his younger brother, Daniel, stayed in Branford.

“What happened in Branford is they awarded homestead plots to build homes, starting up around the green and eventually moving down this way, which is called Canoe Brook,” says Joe, adding as a quick fun fact that “at one time, Canoe Brook was a common term, just like Short Beach or Stony Creek, but it’s faded out.”

Daniel Swayne was awarded a homestead in Canoe Brook.

“When he died, his wife sold the property to the same Harrison family who built this house. So as it happened, this property was once owned by Daniel Swayne,” says Joe.

Within that discussion, Joe spins off several more fascinating historical tidbits—Rev. Pierson, for example, was also one of the founders of the Collegiate School that later became Yale University. And Samuel Swayne’s daughter, Elizabeth, was the first to land on the shore with the Newark, New Jersey party, “gallantly” handed onto the bank by her fiancé.

Joe also marvels at the amount of travel undertaken by many early settlers.

“Richard Harrison was also one of the first settlers of Branford, and his family members eventually built this house. But he didn’t come here from Wethersfield,” says Joe. “He came from England to Virginia. And then, probably by ship, he came up to New Haven. And shortly thereafter, he heard they were settling this town out here, and came here.”

Helping History Live on with BHS

A member of the BHS Board since 2013, Joe has a great deal of appreciation for those who have stepped up to serve BHS through the years, helping Branford’s history to live on.

While there are simply too many to name, he thanks many of the past presidents for their hard work, including Ginny Page, a descendant of George Page (one of the signers of Branford’s original town charter in 1645), for her guidance of BHS including its fundraising to purchase Harrison House in 2016. There’s also the tireless work and contributions of Jane Bouley (also Branford’s town historian), as well as the leadership and active involvement of current BHS President Matt Radulski. As assistant treasurer for BHS, Joe also gives props to the contributions and exceptional efforts undertaken on behalf of BHS by Treasurer Josh Russo.

In addition to serving as BHS assistant treasurer, Joe’s also House & Grounds chair of the Harrison House property, which, in addition to the house, includes a barn and two acres of land featuring herb and Colonial gardens (tended by Branford Garden Club). As Joe aptly describes, the “two over two” Harrison House, recorded on the National Register of Historic places, was first occupied by Nathaniel Harrison and his family and descendants until 1800, when it became home to the Linsleys and their descendants, who lived there until 1938. The house was then purchased and restored by architectural historian J. Frederick Kelly.

Upon his passing, Kelly bequeathed it to Boston-based Society for the Preservation of New England Antiquities (now Historic New England or HNE), America’s oldest and largest regional non-profit historic preservation organization. Founded in 1960, BHS began leasing Harrison House in 1974 at a rate of a dollar per year from HNE, before purchasing it from HNE in 2016.

In past years, Joe also served BHS as Guides chair for Harrison House tours (he still loves to lead tours, too). Tours at Harrison House started up again in June and take place Saturdays from 1 to 4 p.m. through September (private tours by appointment; visit branfordhistoricalsociety.org to learn more). All tours are free; donations are welcome.

Joe especially enjoys taking Branford’s 3rd-graders through the house annually, as part of a cooperative learning unit that first brings BHS volunteers to the classroom, then brings students to the historic property.

“When you look back in history, you sometimes don’t appreciate how things collapse in time,” says Joe. “This house was built in 1724, 80 years after the settlement. When the kids come in, I always ask, ‘Who do you think was president when this house was built?’ And I’ll usually hear ‘George Washington,’ or sometimes, even, ‘Abraham Lincoln!’ And then I tell them, ‘George Washington wasn’t even born when this house was built.’”

Things like that tend to stick with you, once you hear them. Joe also notes it’s important to share that Branford’s history included slavery.

“People in Branford were farmers, so they would have had them as field hands, but also as house domestics,” he says. “In one census, almost 10 percent of Branford’s population were slaves. The population was 2,000 and there were 200 slaves. Think about it.”

As for today’s Branford residents, Joe says one of the things they can do to help keep this town’s history alive is to consider joining BHS as a member, and possibly as a hands-on volunteer.

“We really do need new members and volunteers,” says Joe.

Membership with non-profit BHS provides access to special events and programs and helps BHS bring the community educational lectures, presentations, and publications; give an annual scholarship to a high school senior pursuing studies in history; and, of course, maintain and operate the Harrison House museum, barn, and grounds.

Due to COVID-19 canceling the traditional BHS annual Strawberry Shortcake Fundraiser at the Branford Festival for the past two years, memberships and/or donations are needed, now more than ever, Joe adds.

“That’s actually our biggest fundraising event of the year,” says Joe. “In addition to the education programs, there are expenses—insurance is not cheap, and repairs cost money. So I would appeal to people to think about joining the historical society.”

For more information, visit branfordhistoricalsociety.org.