Top of VVD loses grip on the party