Natural Language Processing (NLP) is a type of Artificial Intelligence, or AI, for extracting structured information from the free text of electronic health records. We have set up the Clinical Record Interactive Search (CRIS) NLP Service to facilitate the extraction of anonymised information from the free text of the clinical record at the South London and Maudsley NHS Foundation Trust. Research using data from electronic health records (EHRs) is rapidly increasing and most of the valuable information is sought in the free text. However, manually reviewing the free text is very time consuming. To overcome the burden of manual work and extract the information needed, NLP methods are utilised and in high demand throughout the research world.

CRIS Natural Language Processing

Welcome to the NLP Applications Catalogue. This provides detailed information about various apps developed within the CRIS service.

The webpage published and regularly updated here contains the details and performance of over 80 NLP applications relating to the automatic extraction of mental health data from the EHR that we have developed and currently routinely deployed through our NLP service.

This webpage provides details of NLP resources which have been developed since around 2009 for use at the NIHR Maudsley Biomedical Research Centre and its mental healthcare data platform, CRIS. We have set up the CRIS NLP Service to facilitate the extraction of anonymised information from the free text of the clinical record. Research using data from electronic health records (EHRs) is rapidly increasing and the most valuable information is sometimes only contained in the free text. This is particularly the case in mental healthcare, although not limited to that sector.

The CRIS system was developed for use within NIHR Maudsley BRC. It provides authorised researchers with regulated, secure access to anonymised information extracted from South London and Maudsley’s EHR.

The South London and Maudsley provides mental healthcare to a defined geographic catchment of four south London boroughs (Croydon, Lambeth, Lewisham, Southwark) with around 1.3 million residents, in addition to a range of national specialist services.

General Points of Use

All applications currently in production at the CRIS NLP Service are described here.

Our aim is to update this webpage at least twice yearly so please check you are using the version that pertains to the data extraction you are using.

Guidance for use: Every application report comprises four parts:

1) Description– the name of application and short explanation of what construct(s) the application seeks to capture.

2) Definition - an account of how the application was developed (e.g. machine-learning/rule-based, the terms searched for and guidelines for annotators), annotation classes produced and interrater reliability results (Cohen’s Kappa).

3) Performance – precision and recall are used to evaluate application performance in pre-annotated documents identified by the app as well as un-annotated documents retrieved by keyword searching the free text of the events and correspondence sections of CRIS. a) Precision is the ratio of the number of relevant (true positive) entities retrieved to the total number of entities (irrelevant -false positive- and relevant -true positive)) retrieved. b) Recall is the ratio of the number of relevant (true positive) entities retrieved to the number of relevant (true positive and false negative) entities available in the database. Performance testing is outlined in chronological order for either pre-annotated documents, unannotated documents retrieved through specific keyword searches or both. The latest performance testing on the list corresponds to results produced by the version of the application currently in use by the NLP Service. Search terms used for recall testing are presented, where necessary. Similarly, details are provided for any post-processing rules that have been implemented. Notes relating to observations by annotators and performance testers are described, where applicable.

4) Production – information is provided on the version of the application currently in use by the NLP Service and the corresponding deployment schedule.

Symptom scales (see proposed allocations)

As the number of symptom applications is increasing, we regularly evaluate how to make these available to researchers in a flexible and meaningful manner.

To this end, and in order to reduce the risk of too many and/or highly correlated variables in analyses, we are currently utilising symptom scales that group positive schizophreniform, negative schizophreniform, depressive, manic, disorganized and catatonic symptoms respectively.

The group of ‘other’ symptoms represent symptoms that have been developed separately for different purposes and that are intended to be used individually rather than in scales.

Each symptom receives a score of 1 if it’s scored as positive within a given surveillance period.

Individual symptoms are then summed to generate a total score of:

• 0 – 16 for positive schizophreniform

• 0 – 12 for negative schizophreniform

• 0 – 21 for depressive

• 0 – 8 for manic

• 0 – 8 for disorganized

• 0 – 4 for catatonic

We are encouraging researchers, unless there is a particular reason to be discussed with the NLP team, to use the scales for extracting and analysing data relating to symptom applications.

Version

V3.3

Contents

Symptoms

Physical Health Conditions

Contextual Factors

Interventions

Outcome and Clinical Status

Miscellaneous

Symptom Scales (see notes)