It was built as a guarding castle in the second half of the 13th century near the confluence of the Kysuca and the Váh, where tolls were collected. At the beginning of the 14th century, the originally royal fortress passed into the hands of Matthew III Csák and the castle, especially the towers, were fortified, and inside the fortress a new palace was built.
Beginning in 1487, the new owner of the castle was Gašpar Suňog (Hungarian:Gáspár Szunyogh), whose family owned it until the end of the 18th century. In the mid-16th century, the castle was rebuilt in the Renaissance style. The Turkish threat from the south, and more uprisings at the beginning of the 17th century forced the Suňogs to fortify the castle in the Renaissance style. At that time, the Gothic castle was changed into a comfortable Renaissance palace. During the Baroque period in the 17th century, efforts were concentrated on building a chapel and finishing various outbuildings of the castle and surrounding it by the park. After the Suňog family became extinct, the new owners were the Csákys, who owned the castle until 1945. During the revolutions of 1848/1849, imperial troops seriously damaged the castle, which was repaired in 1870, and was serving as a barracks. The park's size was reduced by the construction of Košice-Bohumín Railway in 1872. The last major reconstruction was done in the 1922–1923, based on the historical and romantic motives. Today, the castle houses three exhibitions of the Považie museum.