Settlement began in the 1870s when Georgia native H.H. Peaster bought 160 acres (0.65 km2) of land and built a house. The community that developed near the site of his home was originally called Freemont; in 1885 the name was changed to Peasterville. Throughout the 20th century, Peaster has served area farmers as a school and church community. During the mid-1920s the town's population grew to approximately 300, but this was soon followed by a steep decline after the Great Depression. From 1975 to 1990, Peaster had around 80 residents. Current estimates put the population at approximately 100.
The Peaster Independent School District serves the community and home to the Peaster High School Greyhounds. Peaster High School has many sports which include: Basketball, Baseball, Golf, Tennis, Track and Field, Softball, Volleyball, and Cross-Country. The school is also considering starting Soccer and Football.