The Town Center Community Association is housed in Historic Oakland. It is believed that Charles Sterrett Ridgely had a Baltimore housewright named Abraham Lerew build the mansion between 1810 and 1811. The name is likely to have come from a large oak tree that stood on the property at the time.

The Rouse Company purchased the property during the initial land acquisition and preserved it to help give a sense of history to the new town. The Rouse Company used it for offices for a short time. Over time, it served as a campus for Antioch College, the home of Dag Hammarskjöld College and then offices of the local Red Cross.

In 1989, Columbia Association restored the building and it became the setting for special events, monthly teas, home to the African Art Museum of Maryland and offices for the Town Center Community Association. The community association became managers of the historic site in 2002.

Although Historic Oakland is in Town Center, its setting off Vantage Point Road borders open space along the Little Patuxent River and gives visitors a taste of its rural beginnings.