The museum is adjacent to the Sutton Town Hall.  The building was constructed in 1823 by the Olive Branch of the Master Masons. After 1844 it was used as a school and for several years as a meeting place for the DAR. When the first floor school room was vacated it became space for the Sutton Free Library. In 1984 the library was moved into the lower level of the town hall and the town voted to allow the Sutton Historical Society use of the building. The building houses the Rufus Putnam Museum and Office.