The Castle is located in south central Colorado along a paved public road, State Highway 165, approximately 13 miles (21 km) southeast of the junction of State Highways 96 and 165. This road is part of the Frontier Pathways Scenic and Historic Byway, and Bishop Castle is shown on the official map.
Construction of the castle began in 1969, when Bishop began construction on a family cottage, which he decided to surround with rocks. Several neighbors noted that the structure looked something like a castle. Bishop took this into consideration and soon began building his castle. He had bought the land when he was fifteen for a price of $450. In 1996, he was challenged by the local and state government over unsanctioned road signs that pointed to the site. They settled the dispute by issuing official road signs.