In 1899 Senator Leri Withee donated a traveling library to the expanding community of Arcadia, which formed the core of its first permanent free library. As in many small towns, the new library was initially housed in a room in city hall, but increasing demand and a growing collection soon led to the construction of a separate library building. In 1905 the Carnegie Foundation funded a new library building contingent upon the community providing a site, paying for maintenance and furnishing books. Bookshelves were filled with private collections including a large donation from United States Senator Spooner.

This simple one-story building uses details borrowed from the Neo-Classical Revival to evoke a sense of dignity and grandeur appropriate to a library. It was the final project of St. Paul architect Diedrik A. Omeyer before he died in 1905. An arched central entrance dominates the front of the building. Two monumental rusticated brick pilasters frame the entrance and appear to support a parapet-like ornamental railing on the roof. This rustication is repeated at the building corners. Ornamental brick arches above the windows mirror the decorative arch over the door. Modillion blocks with a plain cornice and frieze below visually support the slight overhang of the roof. The interior floor plan is U-shaped, but retains the typical library plan. A large reading room occupies the majority of the building with a central circulation desk and an office in the corner.

The library sits across the street from both the high school and the elementary school. It served many years as the primary research and study facility for both schools. The library is still in operation and is open during regular hours.