The Colleton County Memorial Library was established through the consolidation of two libraries, the historic Walterboro Library Society and the Colleton County Library.

• 1820 – Walterborough Library Society was founded by a group of young men as a subscription-based library.
• 1821- South Carolina General Assembly was petitioned by the Society for incorporation.
• 1822 – Constitution of the Walterborough Library Society adopted
• 1824 – Library building constructed on current site of St. Jude’s Episcopal Church, Walterboro
• 1836 – Library building was turned over to the town of Walterborough. Society disbanded and distributed the books among its members.
• 1845 – “Little Library” building was moved to its present day location at corner of Wichman and Fishburne Streets.
• 1866 – Effort was made to revive the library society. The society became known as the “Philomothian Library Society.”
• 1888- Claudia Stuart, a teacher revitalized the library as librarian. She was succeeded by Sally Webb Henderson with the help of Bertha Klein
• 1903 – The “Committee on the Education of the Young Men’s Business League” organized the Claudia Stuart Town Library Association.
• 1914 – Cornelia DeTreville named librarian
• 1920 – Amelia S. Fraser named librarian. Library maintained by the Walterboro Book Club.
• 1925 – Walterboro Town Council assumed control of Walterborough Library Society and maintained it as free public library.
• 1936 – Library improved under Elma S. Rogers with Federal Works Progress Administration (WPA) money.
• 1937 – Bookmobile service started by Isabel Patterson Heaton and Elma S. Rogers. Colleton and Dorchester counties organized a bi-county library located in the Colleton County Courthouse to serve the rural population of both counties. Services were later split. Colleton County Rural Library continued its bookmobile service.
• 1943-45 – Elma S. Rogers initiated and maintained a library at Walterboro Army Airfield.
• 1948 – Marguerite Gramling Thompson named librarian – Colleton County Rural Library
• 1955 – SC General Assembly provided funds for purchase of suitable location for new library.
• 1956 – Walterborough Library Society and Colleton County Rural Library merged into one library at its present day location at 600 Hampton Street, Walterboro, SC.
• 1962-1969 - Lila Morley (Smoak), Librarian
• 1970 – Amelia S. Fraser retired after 50 years of library service
• 1970-1986 - Lila Morley Smoak, Head Librarian
• 1984 – Library underwent renovations and additions
• 1987 – Sylvia Newton Rowland named Librarian.
• 2000 – Extensive renovations and additions to the library expanding its square footage from 8,859 square feet to 22,157 square feet.

Board Chairmen
R.A. Durham, Walterboro, 1946-49
C. Moye Padgett, Walterboro, 1949-76
Elma S. Rogers, Walterboro, 1976 - 1989
Jane McT. Brown, Walterboro, 1989 -

Librarians
Marguerite G. Thompson, 1947-61
Lila M. Smoak 1962-69; 1970- 1986
Sylvia N. Rowland 1987 -