The Sturgeon Bay Library Association was formed in 1866 by a group of local citizens. They began weekly meetings to share books and engage in debates. This group was short-lived. In 1901 the Women's Association was formed in order to keep the library collection together. They laid the ground work for securing a grant from the Carnegie Foundation. The association turned the library over to the city in 1906 and a library board was appointed in order to meet requirements and conditions of the Carnegie Foundation.

Local architect Fred D. Crandell designed the library in the Classical Revival style to fit in with other commercial buildings surrounding it. Classical Revival was quite popular in Sturgeon Bay. Many of these buildings were constructed of Sturgeon Bay limestone giving the city a unique character. Trim and ornamentation is executed of stone pieces because the limestone is too hard for carving. This unique application creates simple designs with vague Classical stylistic references.

The one story with raised basement structure is constructed with Sturgeon Bay limestone and carved Bedford stone. The main level is defined by a continuous sill and beltcourse that is set slightly above the lintel. A smooth faced stone water table separates the basement from the main level. The symmetry of the main façade is demonstrated with tripartite window groupings on either side of a projecting entrance. The side elevations feature smaller windows located above book stack level in order to let light in. The interior is unique for its mosaic tile floor with a Greek key pattern and the word ¿Library¿ in the center of the vestibule. A broad oak staircase leads to the open floor plan of the main level.

The building was used as a library until 1974. It is now a professional office building and is open during regular business hours.