The site is first recorded in a document in 1021 as the property of the monastery of Saint-Victoir in Marseille. Later, the lords of Pontevès progressively developed the site. In 1233 there is mention of a gate to the courtyard and the buildings arranged in the U-shape characteristic of the early 13th century. Further additions between 1560 and 1580 included a new bedroom, a great hall and the northwest tower. Between 1580 and 1590, the castle was defended during the Wars of Religion. In 1626, a billiards room, tennis court, stables and chicken house were added. Gradually, the castle was developed as a comfortable residence rather than a defensive site.
In 1650, François de Pontevès sold the castle to a rich Aix-en-Provence financier, Pierre Maurel, nicknamed the Croesus of Provence. The site then underwent a massive upheaval with most of the ancient edifice being destroyed and the materials reused to construct a vast building containing more than 50 rooms on three levels. Three corner towers were added. A gallery was decorated with trompe-l'œil paintings by Jean Daret.
However, this splendour was not to last. Faults in its constructions and inheritance problems led to a rapid degradation. By the time of the French Revolution the castle was already crumbling and many houses in the village today show evidence of plundering of the site for building materials, particularly doorframes and windows.
The castle today is open to the public (free of charge). Visitors can see the gateway, four towers, parts of the curtain wall and other ruined structures.