There is an English Heritage priory located here. The priory is one of the best preserved and dates back to 1090. It is one of the most complete that I have sen so far. On the website there are a few activities you can print out for the children before you go.
I think you could spend a good half of a day here seeing the priory and castle, going around the village, and having lunch in the local pub.
Not too much to organise and a nice place to get away for the day.
