Ypres Castle is one of the oldest surviving buildings in Rye. It was built in 1250 by Roger of Savoy. The castle was later sold to John d' Ypres around 1430, from whom, it takes its present name.
One of Ypres Castle early uses was as a prison. The courtyard, now planted as a medieval garden, was the women prisoners exercise yard. Originally prisoners were kept 5 to a cell, male and female prisoners together. Later it was found that separating male and female prisoners was worked better. The castle now houses part of the Rye town museum.