Overview
Incorporated as a part of a larger study on human immunodeficiency virus (HIV) testing in Emergency Departments in South Africa, the pilot symptom list (Appendix 1) was used for chief complaint coding in a large multi-center Emergency Department (ED) based observational study in South Africa wherein study staff (predominantly HIV counsellors or nurses, with research training), collected both free text chief complaints and then made a good-faith effort to match chief complaints to one of the pre-determined symptoms from the pilot symptom list [11].
The original prospective observational study, conducted between June 2017 and July 2018, was embedded in the larger Walter Sisulu Infectious Diseases Screening in Emergency Departments (WISE) Study that implemented point-of-care HIV testing in the ED and collected extensive demographic data on ED patients. This study collected data across three EDs in the Eastern Cape Province, South Africa, where each of the three EDs was sampled for a period of 6 weeks. Data was collected on convenience sample of 3357 patients, who enrolled in the WISE study, from across these three hospitals using paper case-report forms (CRFs). For the purposes of this analysis, the free-text chief complaints were then coded using the Medical Dictionary for Regulatory Activities (MedDRA©) nomenclature. Free-text chief complaints were then compared to those identified by study staff using the pilot symptom list and analyzed for clustering using factor analysis. The primary outcome of interest was to assess the adequacy and accuracy of the pilot symptom list in capturing and reflecting patients’ presenting complaints. The secondary outcome of interest was to assess redundancy in the pilot symptom list by observing categories that were never selected or selected significantly frequently. A modified Delphi methodology was used to review the outcomes and observations using these to make recommendations for modifications and amendments to the pilot symptom list.
Setting
The WISE study was conducted in the Eastern Cape Province in three hospital-based emergency departments. Nelson Mandela Academic Hospital (NMAH) and Mthatha Regional Hospital (MRH) are located in the rural town of Mthatha, and Livingstone Hospital (LH) is in the city of Port Elizabeth. NMAH and LH are both tertiary care centers, they receive referred patients from regional and district hospitals in addition to providing 24-h trauma care. MRH provides 24-h services for walk-in patients and ambulances, while trauma cases are transferred to NMAH. All hospitals maintain 20–50 beds in the ED and are staffed by 1–2 doctors, but are not staffed by physicians or other providers specializing in Emergency Medicine. Patients are seen on a first-come-first-serve basis unless determined to be critically ill or requiring immediate care. Handwritten logbooks and paper medical files are used to track all patients.
Recruitment and enrolment
Patients presenting for care to the hospital ED during the study period, aged 18 years and older, fully conscious, and clinically stable were eligible for enrollment in the study. Patients who met the inclusion criteria were approached by trained HCT staff as soon as they completed the triage process and were informed of the ongoing study and offered a point-of-care HIV test. Data was also collected on patient demographics, presenting complaint, presenting symptoms, past medical history, and reasons for accepting or declining the HIV test. Written informed consent was sought for all patients. Patients were enrolled 24 h a day throughout the duration of the study.
Data collection
Data were recorded using CRFs. Responses to demographic information, past medical history, and reasons for accepting or declining the HIV test were recorded using predetermined categorical options or as free text, presenting complaint was recorded as free text and later coded using the Medical Dictionary for Regulatory Activities (MedDRA©, MedDRA MSSO, Virginia). Chief complaints were recorded using the pilot symptom list. CRFs were scanned and entered using intelligent character recognition (ICR) DataFax software (DataFax©, Clinical DataFax Systems Inc., Hamilton, Ontario, Canada) and centrally double-verified by independent data technicians.
Data analysis and statistics
Data were analyzed using STATA v.15© (StataCorp, LLC, TX). The pilot symptom list was checked for accuracy against MedDRA-coded chief complaints for each patient. MedDRA-coded chief complaint terms were reviewed by two study team members to determine the frequency of concordance between the MedDRA term and the selected symptom(s) from the pilot symptom list. A ‘match’ was defined as a patient record with a MedDRA term that matched with the symptom(s) selected on the pilot symptom list. A ‘mismatch’ was defined as a patient record with a MedDRA term that was either not present in the pilot symptom list or did not align with the symptom(s) selected on the pilot symptom list. These mismatches were further defined as ‘true mismatches’ aka list errors (when the appropriate symptom(s) matching the MedDRA term did not exist on the pilot symptom list/needed to be added) and ‘false mismatches’ aka rater errors (when the appropriate symptom(s) matching the MedDRA term was available but not selected from the pilot symptom list). Clusters of chief complaints were identified using an exploratory factor analysis of the chief complaint list (48 complaints). Factorability of the chief complaints was determined by inspecting the correlation matrix (correlations > 0.4), the Kaiser-Meyer-Olkin measure of sampling adequacy (KMO > 0.6), and Bartlett’s test of sphericity (p < 0.05). The chief complaints were then subjected to factor analysis with an oblique rotation (oblimin), producing as simple a structure as possible while permitting correlations among factors. Factors were retained based on the Scree test (Cattell, 1966). Factor analysis is a statistical data reduction and analysis technique that strives to explain correlations among multiple outcomes as the result of one or more underlying explanations, or “factors.” The technique involves data reduction, as it groups a set of variables based on frequency of concurrence.
Modified Delphi process
Our algorithm for matching the free-text chief complaint (coded using MedRA ©) and the boxes ticked on the pilot symptom list identified potential chief complaints to be discussed further during a modified Delphi process (Fig. 1).
From this review, final decisions regarding changes to the list were reached systematically using the modified Delphi method. Using this method, each reviewer shared reflections from their independent review, in a round robin fashion, which was recorded and reflected on a whiteboard to the entire group, until no new ideas were forthcoming. Thereafter, reviewers had the opportunity to discuss and clarify each comment/idea shared until group consensus was reached. Notes were kept on rationale for response to each of the mismatches and a detailed review of each of the mismatches, the supporting discussion and resulting recommendations for changes is provided below.