A shift in focus since the Grenfell fire has moved tenant satisfaction further up the agenda for social landlords. But have they got the right tools to assess it? Robyn Wilson finds out. Illustration by Rami Niemi
One of the consequences of David Cameron’s ‘localism’ agenda nearly a decade ago was a fundamental shift away from consumer regulation in the social housing sector. But with the impact of the Grenfell Tower fire still sending ripples through the industry, the pendulum appears to be swinging back towards tenant engagement.
As Jane Porter, chief operating officer at Optivo, puts it: “Consumer regulation is coming down the tracks. The regulator is going to be taking much more of an interest in consumer standards than ever before.”
But exactly how to measure consumer satisfaction is not always clear – and even less clear is what should be done with the information once it has been gleaned. How can we compare provider to provider, and how useful are these comparisons?
“Consumer regulation is coming down the tracks. The regulator is going to be taking much more of an interest in consumer standards than ever before”
Jane Porter, chief operating officer, Optivo
One of the problems with measuring tenant satisfaction in a regulatory environment is consistency.
Since 2011, many providers have begun to develop their own ways of measuring tenant satisfaction. But as one industry source points out, this would be “useless for the regulator” today, given that the data is not comparable. Prior to 2011, the sector ensured consistency through regulated surveys known as the ‘Status’ framework, but these were restrictive in a number of ways. For example, landlords had to ask certain questions in a certain order and send them via post, which raised questions over the quality of the information being gathered.
Once these ceased to be a regulatory requirement, they evolved into a voluntary questionnaire: HouseMark’s ‘Star’ (Survey of Tenants and Residents) survey, which much of the industry still takes part in.
Star surveys include a core set of questions, covering overall satisfaction, quality of home, and satisfaction with repairs and maintenance, which providers send out to a random sample of tenants (also known as a perception survey).
In addition to using the core Star questions, providers must follow the framework methodology to be compliant. The results are then benchmarked.
“Star borrowed the best from the Status framework but improved it,” says Jonathan Cox, deputy director of business intelligence at HouseMark. “It is voluntary but is still widely adopted, with over 400 landlords adopting the Star methodology and benchmarking their results with HouseMark.”
Mr Cox says that of the 92% of landlords that currently do perception surveys, 77% are Star compliant.
Much like Status, Star offers a relatively high level of consistency, enabling providers to benchmark tenant satisfaction – something that has also proven useful for the regulator in Scotland, which adopted Star in 2015 as part of its housing charter.
Indeed, once Scottish landlords started pegging satisfaction to regulation, data trends showed it going up and service delivery improving, according to HouseMark.
On this, Mr Cox says: “This may be because the charter focused the regulatory framework in Scotland around the interests of tenants and other service users in a way that encouraged improvement.”
Understanding regional and demographic variations is critical in accurately interpreting the data. One example of this is what HouseMark calls “the London effect”, with residents in the capital being around 10% less satisfied than those elsewhere.
There are a number of theories behind why this is the case. “Age and urban environment may be a contributing factor,” says Mr Cox.
Methodology and the way providers engage with tenants, whether via phone or face-to-face, for example, can also impact survey data, he adds. HouseMark is researching the potential reasons behind this and how this can impact results. This research is part of a wider review of Star that was launched following the release of the Social Housing Green Paper last year, and the findings will be used to “modernise” and update the survey.
Perhaps because the sector is particularly heightened to the importance of customer satisfaction and engagement at the moment, HouseMark received feedback from 300 landlords and 8,000 residents as part of its consultation.
“Particularly with Star, there is a real suspicion among tenants that it is just a way to tick a box and do nothing with it other than put it in your annual report”
Jenny Osbourne, chief executive, Tpas
Those results show that 70% of landlords feel they do not make the best use of tenant feedback, while 81% of tenants do not know what the landlord does with survey results. This brings to light a general sense that more work could be done to make the survey a genuinely insightful and transparent process.
As Jenny Osbourne, chief executive of Tpas, which has been involved in the review, says: “Particularly with Star, there is a real suspicion among tenants that it is just a way to tick a box and do nothing with it other than put it in your annual report.
“There’s a lot of working going on in the sector about how consumer standard metrics might look in the future. Certainly, when we’ve spoken to tenants, they want things that can’t be gamed.”
Alistair McIntosh, chief executive of the Housing Quality Network, echoes this sentiment: “Sometimes people game satisfaction surveys by sending out the forms at a time when residents are likely to be happy.”
The review also highlights the challenge in finding a balance between consistency in order to measure the results, and allowing tenants flexibility in their responses, so the most valuable information can be drawn out.
As Mr Cox explains: “With the new framework, the challenge is should we limit the variables by applying consistency? So, where is consistency important and where is there flexibility to bring in some benefits? And if there is flexibility, how can we help people understand the impact of the variables?”
There are also some concerns about oversimplifying the process. This formed part of the argument from critics of the league table idea proposed by government in the green paper. Many also fear that these could further stigmatise the social housing sector, given that most tenants do not have a choice of provider.
“As a sector, it’s useful to know where other people are in the country but to put it so clearly in a league table it leads to [providers thinking] ‘let’s get up the league table’, rather than ‘let’s concentrate on getting better services’,” Ms Osbourne says.
“Homes England would only allocate resources on satisfaction at the point of a gun”
Alistair McIntosh, chief executive, Housing Quality Network
Katy Wilburn, director at research agency IFF Research, says the industry should not be too quick to dismiss the idea. “If you look at it from the tenant point of view, if you have no choice around the provision you receive, then surely they should be able to compare directly if they could have got a better service? So, they can challenge and have some level of power in the relationship; [it’s] empowering them and arming them with data.”
Ms Wilburn works with 42 providers that are looking at other ways of measuring satisfaction, including some who want to redesign their in-house tenant surveys. Peabody, Midland Heart and Black Country Housing Group are just a few of the housing associations Ms Wilburn says are “tending to veer away from core Star and do their own thing”.
She says there has been a real shift away from the general satisfaction metrics providers typically focus on, and towards metrics that cover trust, effort and engagement.
“We seem to be talking to everyone [about these three metrics] at the moment,” Ms Wilburn says. “It’s a drive towards treating tenants more like customers in other sectors. It’s not just about whether we deliver our service standard to the agreed level – it’s about whether that is the expectation of the customer.”
Some providers are also using a net promoter score (NPS), which tests customer loyalty by asking how likely it is that a customer would recommend a company, product or service to a friend or colleague.
Although a common tool in the private sector, this measurement is not as widespread in social housing. “It’s difficult because you’re asking someone a multi-layered question about recommending a service that they didn’t choose and their [friends and family] couldn’t opt to have,” Ms Wilburn offers as an explanation.
Still, an increasing number of providers, such as Optivo, are starting to look at NPS as another way to gauge tenant satisfaction. NPS is often based on a 0-10 scale, with those who respond with a score of 9 to 10 (‘promoters’) considered more likely to show value-creating behaviours.
Ms Porter says that Optivo adopted NPS when it researched non-housing, customer-facing organisations such as Amazon, which has a high NPS score. “We started to look outside the sector at who’s delivering good customer service, how are they doing it and why are they doing it, how they are using their customers to shape their service, and predictive analytics to get customer insight,” she explains.
But she admits that there are limitations to comparisons with the tech giant: “We found it was quite difficult to compare Amazon feedback on a speedy delivery to us around a transaction we had about arrears. That doesn’t mean we don’t deliver a great service, but with that context it’s harder to consistently get 9s and 10s.”
This touches on another key problem with measuring tenant satisfaction within social housing: how much value does all this really have if the tenants have no choice?
“On its own, customer satisfaction is a fairly blunt tool which provides subjective feedback and little about the factors that underpin or explain which aspects of services are effective and which need adjusting”
Rob Wray, chief innovation officer, HACT
As Mr McIntosh puts it: “No one is going to get to move property based on the satisfaction level of a landlord; no desperate homeless person is going to say, ‘I’ll go for that landlord that has 80% satisfaction.’ Homes England would only allocate resources on satisfaction at the point of a gun.”
Still, there is some consensus that providers should learn from satisfaction feedback to improve their service. “Lack of customer choice is not a reason for saying that we shouldn’t deliver great customer service,” says Ms Porter. “In fact we should be giving them even better service because they don’t have that flexibility of choice.”
Ms Wilburn notes that as providers become more commercially focused to cross-subsidise sub-market housing, tenant satisfaction will play an increasingly important role.
“Where housing providers are branching out to commercial products, things like positive reputation and customer experience become incredibly important because in the same areas where providers are going to try and sell leasehold properties for profit, they are going to have a large body of tenants who are influencing the perception of them as an organisation,” she says.
The more people Inside Housing speaks to about this topic, the more it becomes clear that relying on a single tool or customer satisfaction measurement would fail to drive real change.
For some, customer satisfaction alone is ineffective when it comes to improving services. “On its own, customer satisfaction is a fairly blunt tool which provides subjective feedback and little about the factors that underpin or explain which aspects of services are effective and which need adjusting,” says Rob Wray, chief innovation officer at HACT, which released an extensive report on customer satisfaction last year.
Instead, he argues, this information should be used in conjunction with behavioural data to create a profile of how residents actually interact with an organisation.
“This one-dimensional feedback has routinely told us the same things for more than two decades,” he concludes. “It describes the problem, but offers little to understand what can be changed.”
Number of social landlords that have adopted HouseMark’s ‘Star’ survey
Percentage of landlords that currently do perception surveys
Percentage of landlords that are Star compliant, of the above 92%
Percentage of landlords that feel they do not make the best use of tenant feedback
Percentage of tenants who do not know what their landlord does with survey results
All our Social Housing Green Paper coverage in one place:
Green paper measures are not enough to create May’s ‘new generation’ of council homes Green paper proposals are welcome but much more is needed to support councils to build, writes John Bibby
Green paper shows ministers now see associations as trusted partners Focusing on the failure of the green paper to address supply misses the point, writes Boris Worrall
Government should focus on building on what is already strong Philippa Jones considers the Social Housing Green Paper through a slightly different lens
We need more than a week of delayed announcements bundled together Jules Birch reflects on the government’s ‘Housing Week’ announcements
The regulator should monitor how associations assist homeless people Government announcements this week are positive, but any enhanced role for the English regulator should include looking at homelessness prevention work, argues David Bogle
The regulator’s role should be limited to dealing with systemic failures Julian Ashby suggests the Housing Ombudsman Service should deal with all complaints
The green paper shows ministers are in listening mode Despite some glaring omissions, the government appears to be in listening mode and it is important the sector takes advantage, argues Emma Maier
A short history of social housing league tables Attempts to create league tables for housing associations are nothing new. Mervyn Jones looks at how they have worked in the past
League tables could prove blunt and counter-productive, sector warns Housing figures criticise government proposals to measure social landlords against performance indicators
Government ‘must decide how proactive regulator should be’ on consumer standards Ministers now face a dilemma over the regulator’s focus, sector figures say
The Green Paper: a golden opportunity missed? Melanie Rees assesses the Social Housing Green Paper against recommendations drawn up by the Chartered Institute of Housing and finds the government comes up short
Longer strategic partnerships and guranteed debt to boost social housebuilding The Social Housing Green Paper outlines key ways of boosting supply
The green paper is remarkable progress but it is still not enough The green paper suggests the government appears to be re-writing much of its policy since 2010, but more needs to be done, writes Jules Birch
Green paper marks a ‘milestone’ on resident involvement The government’s recognition residents need clear information is to be welcomed, now it up to the sector to embrace tenant involvement, writes Paul Hackett
Ministers consider stock transfer programme to community-led associations The stock transfer programme could be revived under proposals in the housing green paper
Access to housing grant could be tied to new league tables Grant could be awarded according to how well landlords meet performance indicators, the paper suggests
Ofsted-style regulation of tenant services proposed The government is considering expanding the Regulator for Social Housing’s remit to intervene over tenant services and give it a more “proactive approach to enforcement”
Government proposes dropping one-for-one Right to Buy replacement commitment A consultation paper published alongside the green paper proposes a broader measurement to replace the one-for-one pledge
A list of recent housing policy U-turns The green paper confirms yet more housing policy U-turns from the government, which has spent the past two years dropping policy ideas developed under the David Cameron government. Here is a rundown of the major changes in policy direction
Sector welcomes green paper but calls for more ‘ambitious investment’ Reaction to the proposals, from the National Housing Federation, Chartered Institute of Housing and more
Morning Briefing: reaction to green paper announcements how the media reported the proposals trailed by the government overnight
Government drops plans to force councils to sell higher-value stock The government drops plans to force councils to sell higher value homes
League tables and ‘sharper teeth’ for regulator in social housing green paper Ministers reveal some of the things in the paper ahead of its publication
Grenfell survivors: green paper does not go far enough survivors of the Grenfell Tower fire have said the measures published in the Social Housing Green Paper do not do enough to rectify issues in the social housing sector