The estate's connection with the Montmorency family began in 1484. The first mansion (no longer in existence, now replaced by the Grand Château) was built in 1528–1531 for the Constable Anne de Montmorency by Pierre Chambiges.