The Efra Committee suggested the UK could see food prices go up unless more was done to resolve questions around labour