Ian Dean, Sales Director at Feedback Ferret, explains some key factors that housing associations should consider before embarking on constructing their own in house text analytics
I am often told by housing associations prospects that they understand the value of analysing free-form data using text analytics and are thinking about developing their own system in-house using a variety of available tools such as Microsoft fuzzy lookup and Python.
Over the years, we have seen a few organisations try and return to us once they realise the significance of what they are taking on. Don’t get me wrong, it is possible to achieve some level of accuracy and results from an in-house developed product; but there are challenges that need to be considered.
1 Processing power
The processing power required to do the initial labelling of words and phrases is quite considerable and has been known to crash systems. You will therefore need to take advantage of cloud processing from say AWS or Azure, which comes with its own cost and management overhead.
2 Categorisation
Labelling of the words and phrases is actually the simple part. Ordering them into topic groups that make sense to the business is more tricky – which topics are important, how should they be categorised, etc. We have, for instance, built out 472 housing-sector-specific topics by working with other HAs, so we know these is salient to the sector.
3 Applying sentiment
You then need to apply a sentiment to the sentences. Sentences like “I couldn’t recommend you to my friends and family” has one very negative connotation, until “more highly” is added to the end. Getting a contextual sentiment score is much more difficult than most imagine.
4 Clarifying sentiment
Contextual sentiment needs to account for things like slang, idioms, colloquialisms, and sarcasm – e.g., “Great job XXXXX….. NOT.” This is actually very negative, but we see some classing it as a positive as it says “great job”. Admittedly, we see less of this specific sarcasm, but there was quite a period where it was very common. Sarcasms will and do change, so the solution needs to take that into account and potentially some human intervention.
5 Problem words
There are ‘problem’ words – e.g., “outstanding.” In one context this is extremely positive; in another it is extremely negative. Machines need to make an allowance for this.
6 Building and refining
In order to train AI, an immense amount of data is required not only to process and see what the results look like but then ‘correctly’ classify data to check the results against. Invariably, this means there is a heavy human element to building the solution and refining algorithms.
7 Pre-coding
With outlying words, AI is not good. For example, with our Health and Safety solution we pre-code all the cancer conditions, all the cancer treatments, and all the numerous ways these can be misspelt. We also pre-code all the 1,000+ biocide products authorised for use in the UK, as well as many other topics. This kind of coding is not just coding: it is the research that goes into looking up all these words, phrases, names, before you can even get to the coding part.
8 Clarifying phrases
With outlying phrases, AI is not good. For example, the phrase “knocked-up” has been seen to mean a knocked-up worktop or window (repairs) or as a reference to a pregnancy. There are an enormous amount of phrases just like this that we know all about, having spent more than 20 years dealing with them.
9 Evolving situations
Language and situations change, so the system requires constant maintenance. Topical is Coronavirus; we now see 84 different ways this one word is spelt and then misspelt.
10 Staying alert
Despite having built a system over the past 20 years, we are still amazed at the need for constant refinement. As it is an ever-moving target, we employ a team of ‘lexicon’ agents to find the anomalies and constantly code new words and phrases.
As we have people who go home at night, see the terrible news reports that were coming in from Italy at the time, we immediately started coding. Before the end of February, we were already reporting on this to our clients. Remember, lockdown did not commence till 23 March in the UK, so our customers were already getting insights before they even conceived there might be a lasting impact to their organisations from the pandemic.
11 Data manipulation
Assuming your data teams can manipulate the data sufficiently to give a generic feel, they now need to build out rules for specific use cases. In our HHSRS (resident Health & Safety) solution, we look for a combination of topics – e.g., excess cold and elderly, cancer patients, cardio issues, mental health issues, children, etc. Excess cold can be as random as “thermostat broken”.
To give the HHSRS views, we needed to code 1,672 topics – each of which has hundreds or thousands of words and phrases associated (along with misspellings), which turns into hundreds of millions or even billions of lookups. In this way, we make sure we miss none.
12 Reporting
So, now you have all the data labelled and segmented by topics and sentiment – what views make sense to the various different stakeholders across the business. The repairs team want different views to the customer service team, wellbeing only want to hear from vulnerable customers, and the board will want to see how the outcomes and initiatives align to the company values.
There will be many other requirements besides which need to be built, supported, and maintained. We have the vast majority of those off the shelf, ready to go.
Housing Digital Stakeholder Feedback Ferret’s pre-coded housing sector-specific solutions capture what residents say across all feedback channels to give a 360° view of their experience.
Read next: Public take action as anti-social behaviour incidents go unreported
Are you a social housing professional? Sign up for a FREE MEMBERSHIP to upload news stories, post job vacancies, and connect with colleagues on our secure social feed.