The building was built between 1270 and 1282, following the typical square plan of the fortifications in Savoy. It was the property of the Counts of Gruyères until the bankruptcy of Count Michel in 1554. His creditors, the cantons of Fribourg and Bern, divided his earldom. From 1555 to 1798, the castle was the residence of the bailiffs and then to the prefects sent by Fribourg. In 1849, the castle was sold to the Bovy and Balland families, who used the castle as their summer residency and restored it, establishing an artists' colony. The castle was repurchased by the canton of Fribourg in 1938, made into a museum, and opened to the public. Since 1993, a foundation ensures the conservation as well as the highlighting of the building and the art collection.
Museum
The castle is home to three capes of the Order of the Golden Fleece. They were part of the war booty captured by the Swiss Confederates (which included troops from Gruyères) at the Battle of Morat against Charles the Bold, Duke of Burgundy in 1476. As Charles was then celebrating the anniversary of his father's death, one of the capes is a black velvet sacerdotal vestment with Philip the Good's emblem sewn into it.
An international collection of Fantastic Art is also exhibited (José Roosevelt, Gian Paolo Dulbecco, Vincent Amas, Patrick Woodroffe, Milan Goldschmiedt).