Identifying mentions of pain in mental health records text: A natural language processing (NLP) approach

Jaya Chaturvedi recently won the Best Student Paper Award at MedInfo 2023, the 19th world congress on medical and health conference, in Sydney, Australia. Jaya is a KCL DRIVE-Health CDT PhD Student in the Department of Biostatistics and Health Informatics, at the Institute of Psychiatry, Psychology & Neuroscience, King’s College London. Jaya writes about the  process and findings from her winning paper.

Pain and mental health disorders – why study this?

Most of us have experienced pain at some point in our lives. Whether it's a fleeting moment of discomfort or a more enduring experience, pain is a condition that affects us all. Pain is a global problem and affects a significant portion of the world’s population, with about 1 in 5 adults worldwide struggling with it daily. In addition, 1 in 10 adults are diagnosed with chronic pain each year, making it a prevalent problem in society.

Likewise, mental health disorders affect millions of people, with about 1 in 4 adults and 1 in 10 children experiencing mental illnesses at some point in their lives .

Pain is a complex phenomenon, and this complexity is increased when intertwined with mental illnesses. While a substantial amount of research has been done on the onset of mental illnesses (such as depression) in individuals with chronic pain, the reverse scenario, for example, onset of pain in people with pre-existing mental health issues, remains relatively understudied. The start of electronic mental health records means that this interaction between pain and mental health problems can be studied at scale, through access to large amounts of data.

Using insight from those with lived experience to generate the key questions

With the aim of contributing to this understudied aspect of pain and mental health disorders, and to make my research more relevant, I talked to individuals who have lived experience of chronic pain and mental health disorders. I asked people what questions they would like answered through data about pain from mental health records. The insights they provided were invaluable, and generated a lengthy list of questions, which emphasises the need of having access to such data.

The service users I talked to were interested in finding out whether there were differences in pain experiences based on age, gender and ethnicity, as well as whether certain mental health diagnoses report more pain compared to others. They also wondered whether different body parts were affected based on these factors, and whether a mental health diagnosis affects the believability of the patient's pain by the clinician. While I can’t yet answer all these questions, I am attempting to answer some of them in my research.

Jaya presents her research at MEDINFO 2023

Current challenges in studying this intersection of pain and mental health disorders

Pain is a common reason for people to seek medical attention, and so their experiences are documented by clinicians within their medical records. However, a major challenge here is that pain, being a symptom rather than a diagnosis, often gets mentioned only in the detail of the clinical notes rather than in the more available and structured summary information. This is particularly evident in mental health records where clinicians tend to document their observations within the free-text fields of the health records, particularly as pain might not always be the primary complaint when visiting mental health services.

Consequently, when researchers want to identify patients who might be suffering from pain, it becomes a challenging and daunting task because this detail is hidden amidst a sea of documents and letters within the health record, and it is not feasible to screen this text manually to identify experiences of pain or painful conditions.

Overcoming the challenges

To help overcome this hurdle, my research focused on developing a method of extracting such information about pain from the text of clinical notes. This was achieved by using a computer-based artificial intelligence technique called natural language processing (NLP).

This approach facilitated the development of an application that made the clinical notes machine-readable, enabling the identification of patterns amongst the words to learn when a sentence might mention physical pain affecting the patient. This allowed us to then label sentences as relevant to pain or not, ultimately unlocking the necessary data required to study pain.

The application was built using the clinical notes from an anonymised version of mental health records from South London and Maudsley NHS Foundation Trust, called Clinical Record Interactive Search (CRIS) system. Now it has been developed, the application can be run over millions of documents and letters, and identify which ones record information about pain, thereby ‘unlocking’ information on a large scale about this important risk factor for worse mental health.

WATCH: In the video below, Jaya talks about the uses and importance of free text clinical data in healthcare research, the many challenges of working with such data, and the different NLP tools and approaches that can be used to assist in the analysis of this data, thereby aiding health research and care.

Future implications

This application has been extremely useful for my PhD project, where I am looking at differences in recorded pain experiences between different mental illnesses, and its relationship with age, gender, ethnicity and poverty; however, the implications extend beyond my own research.

By making the application available to other researchers, clinicians and epidemiologists who use the CRIS system, we can foster collaborative efforts in studying various aspects of pain. The text-processing application can be run on an ad-hoc basis whenever required.

However, challenges remain, particularly when studying pain in individuals who struggle to communicate their pain experiences, which consequently might not be recorded in medical records. For example, severe mental illnesses, such as schizophrenia and bipolar disorders, may pose a barrier to care if patients are not able to articulate their pain experiences clearly. Similarly, individuals with learning disabilities might rely on their carers to identify and report their pain experiences to the clinicians. By conducting more research in this area, we can offer proactive care measures for pain in these vulnerable groups.

In conclusion, research into pain and its interaction with mental health is crucial. Continued research into this complex relationship will provide valuable information that can lead to improved care and support for those affected by it. We can work towards a future where pain and mental health are better understood and efficiently managed in healthcare settings by leveraging technology and cooperative efforts.

Read Jaya’s paper ‘Identifying Mentions of Pain in Mental Health Records Text: A Natural Language Processing Approach’, which won first place at Medinfo 2023 for Best Student Paper.