History & Landmarks

The First People

Long before European settlement, the land around Oyster Bay was home to the Matinecock people, one of the Algonquian-speaking nations of Long Island. They called the harbor and its abundant shellfish home, and their paths, fishing grounds, and relationships with the land shaped the landscape that later settlers would inhabit. The harbor's extraordinary oyster beds — which would define the town's identity for centuries — were a Matinecock resource long before they became a European commodity.

Dutch and English Settlement (1639–1690)

David de Vries, a Dutch explorer, was the first European to record the name "Oyster Bay" in 1639, noting the harbor's remarkable shellfish. Long Island was claimed by both the Dutch (as part of New Netherland) and the English simultaneously, and Oyster Bay sat directly on the boundary — a circumstance that shaped its unique character. When New Englanders arrived in present-day Oyster Bay in 1640, they were arrested by the Dutch as part of a boundary dispute.

By 1653, a group of English settlers — including the Townsend, Wright, Underhill, and Youngs families — had established a permanent community. These were largely Quakers and dissenters, drawn to the relatively tolerant atmosphere of the region. John Townsend, whose grave at the Fort Hill cemetery dates to 1668, was among the earliest. His descendants would play a crucial role in American history more than a century later.

Colonial Life and the Quaker Tradition

Oyster Bay developed as a prosperous colonial village, its economy built on oystering, farming, and trade. The Quaker Meeting, established in the late 17th century, gave the community a distinctly egalitarian character compared to other colonial towns. The Townsend family — merchants, politicians, and civic leaders — were central figures. Samuel Townsend purchased what would become Raynham Hall in 1740, expanding it into a town house whose land ran down to the bay.

It was a community of neighbors: the same family names — Townsend, Youngs, Underhill, Wright — appear on gravestones, street signs, and in Revolutionary War records across three centuries. Audrey Avenue itself is named for Audrey Townsend. Simcoe Street is named for the British colonel who would occupy Raynham Hall. The town's history is written in its geography.

The Revolutionary War and the Culper Spy Ring (1776–1783)

When the British occupied Long Island after their victory at the Battle of Brooklyn in August 1776, Oyster Bay was directly in their path. The hamlet spent the entire war under British occupation — a community divided between Loyalists and Patriots, forced to quarter soldiers in their homes and watch their orchards cut down for firewood.

Lieutenant Colonel John Graves Simcoe of the Queen's Rangers made Raynham Hall his headquarters during the winter of 1778–79. His men occupied the house however they wished, destroying Samuel Townsend's prized apple orchard and forcing the family to swear loyalty to the Crown or face imprisonment. The street that leads to the Townsend family cemetery still bears Simcoe's name.

From this very occupation, however, emerged one of the most consequential intelligence operations in American history. Robert Townsend — Samuel's third son — was working as a merchant in British-occupied New York City. In 1779, his childhood friend Abraham Woodhull (operating as "Samuel Culper Sr.") recruited him into Washington's secret intelligence network. Robert became "Samuel Culper Jr." — agent number 723 — and his position in Manhattan gave him access to British officers, shipping manifests, and military plans that no one else could reach.

The Culper Ring's greatest triumph came when Townsend's intelligence allowed Washington to warn the French fleet of a planned British ambush, enabling French forces to land safely in Newport. The ring also uncovered critical intelligence about the Benedict Arnold–Major John André plot to surrender West Point. Robert Townsend kept his secret identity for the rest of his life — 59 years after the war ended. He died in 1838 at age 84, having told no one. The identity of "Samuel Culper Jr." was not discovered until 1930, when historian Morton Pennypacker matched Robert's handwriting to the Culper letters preserved in Washington's papers.

Robert is buried at the Townsend Cemetery on Fort Hill — hidden at the end of Simcoe Street, nearly impossible to find, surrounded by suburban back gardens. His sister Sally is buried beside him. Sally Townsend is celebrated as the recipient of what is believed to be the first Valentine's card given in America, sent to her by a young British officer quartered at Raynham Hall. She never married, and lived at the family home until her death in 1842.

George Washington visited Oyster Bay personally during his 1790 tour of Long Island, staying with Thomas Youngs — in whose family cemetery Washington's most important spy would one day be buried, alongside a President of the United States.

The 19th Century: Oysters, Industry, and the Gilded Age

Through the early 19th century, Oyster Bay's harbor was one of the most productive oyster fisheries on the East Coast, shipping Bluepoint and other varieties by the barrel to New York City markets. The hamlet grew steadily, connected to the city by water routes and, after 1889, by the Long Island Rail Road's Oyster Bay Branch — whose original station, built that year, survives today as the Oyster Bay Railroad Museum.

The Gilded Age brought extraordinary wealth to the North Shore. The Gold Coast — running from Great Neck to Lloyd Neck — attracted industrialists, bankers, and financiers who built enormous estates on the hills above the Sound. Planting Fields, once the estate of insurance magnate William Robertson Coe, is one of the finest surviving examples: 409 acres of arboretum, greenhouse collections, and a Tudor Revival mansion now open to the public as a State Historic Park.

Theodore Roosevelt and the Summer White House (1885–1919)

No figure looms larger over modern Oyster Bay than Theodore Roosevelt, who made Sagamore Hill — his Queen Anne shingle-style home on Cove Neck Road — his permanent residence from 1885 until his death on January 6, 1919. During his presidency (1901–1909), Sagamore Hill served as the "Summer White House," the center of world attention for months at a time. Roosevelt negotiated the Treaty of Portsmouth here, ending the Russo-Japanese War and earning him the Nobel Peace Prize. He received diplomats, heads of state, and celebrities on the broad porch overlooking Oyster Bay.

Roosevelt was not merely a resident — he was embedded in the community. He attended Christ Church at 61 East Main Street, where his wife and children were active members, and where his funeral service was held. He rowed in the harbor. He knew the oystermen. When he died in his sleep of a coronary embolism at Sagamore Hill, former President William Howard Taft — once ally, then rival — traveled to Oyster Bay and stood alone at the grave in Youngs Memorial Cemetery, weeping.

The 26 steps up to TR's grave were not accidental. He was our 26th president. From the hilltop, you can look out over the waters of Oyster Bay exactly as he once did from his rowboat.

The 20th Century and Today

The hamlet's oyster industry declined through the early 20th century due to pollution and disease, but a revival is underway — local oyster farmers have returned to the bay, and the annual Oyster Festival celebrates that heritage each October at Theodore Roosevelt Memorial Park, drawing up to 200,000 visitors over two days.

The Seawanhaka Corinthian Yacht Club — founded in 1871 on Centre Island, one of the oldest in the Western Hemisphere — continues to anchor the hamlet's sailing tradition. The Waterfront Center at 1 West End Avenue maintains the oyster sloop Christeen (1883), the oldest known surviving work boat in America and a National Historic Landmark.

Downtown Oyster Bay today is a walkable, historic village anchored by independent businesses, the Oyster Bay Main Street Association (est. 2001), and the Oyster Bay–East Norwich Chamber of Commerce. Theodore's Books, opened in 2019 by former Congressman Steve Israel, has become a community hub and the founding institution of the Gold Coast Book Fair — Long Island's largest annual literary event.

It is, above all, a place that takes its history seriously. The same family names that appear on 17th-century gravestones appear on street signs today. The house where Washington's most important spy lived is open to the public on West Main Street. A president is buried on a hill a mile from his home, and you can walk up to see the view he loved. OysterBayLI.com has been documenting all of it since 1997.