The groups, including the UN refugee agency, are accused of trying to change Libya's "demographic composition".