The Brandon Village Hall and Library has played many roles in the history of its community. From its brief use as a commercial cold storage building to its long-term use as a village hall and public library, the building has housed activities that have been important in the development of Brandon's local government and public education.

Local entrepreneur Charles Ellis had this commercial vernacular building constructed in 1894 as a combination retail building and cold storage plant. A harness shop was located in the storefront, while the cold storage plant occupied the second floor. Cork or other thick insulation on the second floor made for deep window wells that remain in the building today.

The cold storage business was not a successful venture, and in 1897, Ellis sold the building to the Village of Brandon. This purchase was an important step for the village because it established a permanent location for village government and the fire department. The village renovated the second story cold storage plant into two meeting rooms and housed the fire department's horse-drawn engine and hoses in the rear half of the first floor. The village installed the bell tower on top of the building as a fire alarm.

The harness shop remained in the front half of the first floor until 1913, when the public library moved into the space. When the fire department had to move to larger quarters in the mid-twentieth century, that space became a city garage. In 1973, the library expanded into the garage space. Also during the mid-twentieth century, the smaller meeting room on the second floor was remodeled into a village office and remained there until 2001.

The former Ellis Block remains a public building today, with the library on the first floor and the Brandon Historical Society on the second floor. Charles Ellis' simple cold storage building, meant to add to the village's agricultural economic base, instead became the most important public building in the village. Its bell tower is often used as a village symbol.

The building is open to the public during normal business hours.