The more you fragment social housing, the more you take away tenants rights the more you undermine social housing.
This means the whole sector will suffer, If you take dignity away from tenants - you take away dignity of those who work for and with tenants and in the sector. By depriving tenants of rights you deprive your whole sector from achieving progress.
The argument "we have to do the best with what we got" does not make sense if this involves taking rights and dignity away from tenants.
According to these "professional" thinkers we would have whole estates filled up with only mentally ill people, another estate filled up with only family with children, another estate filled up only by disabled, another estate filled up only with gay, or victims of violence, etc. maybe another estate filled up by people of the same race, etc...
social housing is more complex than that, you got to balance all the individual cases against a better and common good for all, otherwise you are just a nutcase.
Which other sector would say to their customers you have to live in a permanent state of anxiety? It only happens in the social housing sector, where tenants are seen as disposable goods that can be shifted, terminated, moved or left unmoved, etc. with very little control of the tenants themselves about their wishes.