The de Grey family settled on this site by the time of the Domesday Book. In 1347, the 1st Lord de Grey, one of the original Knights of the Garter, was granted a license to crenellate. Most of the medieval fortifications are now in ruins, but the modest Tudor house which was built in the courtyard is still very much intact. Inside the house, there is a plain pine staircase, at the top of which is a window containing 5 panels of 16th- and 17th-century Swiss stained glass. The interiors contain very fine 18th-century plasterwork, perhaps by Roberts of Oxford. An elegant classical drawing room was added in c.1750 by the Stapleton family, who held Greys from 1724 until 1935. In 1969 Sir Felix Brunner gave the house to the National Trust.