A County Hall report which concluded one or two Norfolk councils is the way forward has come under fire from political opponents.