User Research, Human Factors
The content of this project is hidden because of a NDA. Please contact me for the password.
The Introduction
A successful drug delivery system should be easy to use, intuitive and efficient. By understanding the needs and desires of the patient at various times during treatment, we can mitigate user-based error, and reduce any lapses in therapy associated with improper device use. Human factors plays an integral role in the user-centered design process and can help to ensure that the device and delivery system meets the cognitive, physical and emotional needs of the end user.
The Objectives
Our team conducted a late-stage formative usability evaluation. The objective aimed to evaluate if the BD Vystra™ disposable pen prototype and Information For Use (IFU) could be used by a user population exhibiting different properties without repeating patterns of user error. Our secondary goal was to evaluate the comprehension of the Instructions For Use (IFU) and whether prior training needed to be considered.
The Approach
The BD Vystra™ disposable pen is an intuitive and customizable disposable pen injector designed to support therapies that require frequent, low-volume self-injections or variable dosing. The device is entirely manual and similar in design to current insulin pen injectors on the market.
The BD Vystra™ disposable pen is intended as a device to be used across different medical indications and patient groups. Therefore, our focus was on selecting relevant user properties that can reasonably be expected from a wide range of users. As a result, our team defined six user groups.
Performance Anticipation & Explanation:
1. Group 1 – Since health care providers (HCPs) in group 1 have clinical knowledge and training, it will positively affect their performance with the device in comparison to users (groups 2–6).
2. Group 2 – Caregivers (CGs) in group 2 will have full command of their mental and physical abilities; therefore, they are likely to perform better than participants with impairments in groups 4–6.
3. Group 3 – Due to adolescents' developmental and educational factors, this may affect their abilities in understanding device use.
4. Group 4 – Diabetics with retinopathy (DR; group 4) may have visual impairments, which will affect their abilities in device use.
5. Group 5 – Diabetics with neuropathy (DN; group 5) may have tactile impairments, which will affect their abilities in device use.
6. Group 6 – Arthritic patients (AR; group 6) may have motor impairments, which will affect their abilities in device use.
Our goal was to recruit a set of eight participants per user group, with the aim of reaching a minimum of five active participants per user group.
Evaluations will be conducted in a test room in a hospital or medical center. The equipment for this study will include:
1) Mannequins, used for the injections performed by user groups 1 and 2
2) Injection pads, used for the injections performed by user group 3, 4 and 5
3) The BD Vystra™ disposable pen injector with water-filled cartridges
4) Mounted cameras
Participants will participate in an individual 60-minute session. Participant will be asked to:
1) Sign a consent form
2) Listen to background information & study brief
3) Conduct the handling evaluations
4) Answer questions about their experiences and background
1) Inject 60 units (to simulate a large injection) into the (mannequin or injection pad)
Subtasks: Attach Needle, Prime, Select dose, Inject, Hold and Dispose of needle.
2) Watch a 1-minute news summary
3) Inject 20 units (to simulate a small injection) into the (mannequin or injection pad)
Subtasks: Attach Needle, Prime, Select dose, Inject, Hold and Dispose of needle.
The collection measures will include a combination of qualitative and quantitative measures. The usability metrics for this evaluation are:
1) Injection Success Rate
2) Degree of Confidence when using the Device
3) Degree of Comfort when using the Device
4) Degree of Usefulness of the IFU
5) Degree of Difference In Error Rate Per Injection Among Volume & User Group
My team and I observed and noted all use errors, near misses, or deviations from the IFU procedure during the evaluation. The flow of the evaluation was as follow:
43 participants were recruited, of which 36 completed the evaluation. The minimum number of five participants per group was reached in 5 of the 6 groups. Overall, 78% of the participants were female, 81% had previous experience using pen injectors, and 28% suffered from some kind of impairment. The characteristics of the participants in the evaluation are presented below:
All participants except two in the Arthritic patients’ group (Group 6) were successful (i.e., required no assistance) in performing the first injection, which corresponded to a 94% success rate. Everybody succeeded in performing the second injection, meaning a success rate of 100% for this injection.
Overall, 69% of the participants reported that they strongly agree with the statement that they felt confident in using the BD Vystra™ disposable pen, whereas 22% reported that they would agree with this statement. The combined rating for the two categories was 92% of all participants.
Overall, 28% of participants rated the BD Vystra™ disposable pen as a very comfortable to use device, whereas 47% rated it as comfortable to use, giving a combined result of 75% of participants rating the device as at least to be comfortable to use. Only 8% of the participants rated it as uncomfortable to use, with the remaining 17% rating the degree of comfort as neutral.
Overall, 75% of the participants rated the IFU as very helpful and 25% as somewhat beneficial bringing the total to 100% of participants finding the IFU to varying degrees helpful.
1) The error rate varied significantly between user groups, ranging from 3.75 (highest) errors per participant for the first injection in the DN group to 1.00 (lowest) for the second injection in the CG group.
2) The overall rate was 2.39 for the first and 1.94 for the second injection, equivalent to an improvement of 19% between the injections.
3) Also, this improvement or learning effect varied across the user groups, with the CG group showing the largest (39%) and the HCP group showing the lowest (11%) learning effect.
The largest proportion (35%) of all potentially relevant errors occurred when disposing of the needle (step 6), and significant proportions were observed when priming (step 2, 24%) and holding after injection (step 5, 21%). Attaching the needle (step 1, 4%), selecting the dose (step 3, 4%), and injecting (step 4, 12%) were associated with fewer errors or deviations.
Although a substantial number of user errors and deviations from the IFU procedure were observed, most of the error was associated with the use of the needle rather than the pen injector itself. A noteworthy fraction of errors and deviations were also due to experienced users sticking to their (erroneous) habits rather than not understanding or misinterpreting the instructions. As a result, the BD Vystra™ disposable pen together with the IFU could be safely and efficiently used by all user groups without any training. Supported evidence includes:
1) An overall success rate in performing injections above 90% for the first injection and 100% for the second injection.
2) An overall degree of confidence in using the device above 90% across all participants and user groups.
3) An overall degree of comfort in using the device above 90% across all participants and user groups.
4) An overall IFU appreciation percentage of 100% by the participants.
Therefore, the observation that all tested user groups can safely and efficiently use the device provides security that the device and IFU in their current form will pass future summative testing. There are no further improvements for this device needed at this time.