Despite a majority voting to abolish it in February, growers said there were mounting rumours that they would still need to pay the 2021/22 horticulture levy