Background: Informational and supportive care resources are needed for Hodgkin lymphoma (HL) survivors.
Objectives: The aim of this article is to adapt and evaluate a previously developed survivorship care website for HL survivors.
Methods: A mixed-methods, user-centered design, including one-time focus groups, was followed by iterative web design and user testing. Transcripts were content analyzed, and survey responses were summarized.
Findings: HL survivors need survivorship care education and support. Tailored web-based resources may be an effective adjunct to clinical care. The Survivor Wellness website is a usable web-based resource for HL survivors that may facilitate survivorship care.
Earn free contact hours: Click here to connect to the evaluation. Certified nurses can claim no more than 0.5 total ILNA points for this program. Up to 0.5 ILNA points may be applied to Oncology Nursing Practice OR Survivorship OR Psychosocial OR Health Promotion OR Care Continuum. See www.oncc.org for complete details on certification.
Survival rates for Hodgkin lymphoma (HL) have improved due, in large part, to advances in treatment during the past 30 years (Patel et al., 2018). The five-year survival rate for those diagnosed with HL is 78%–93%, varying with stage of disease (American Cancer Society [ACS], 2018). Despite the success of living longer, treatment for HL—often with anthracyclines or radiation therapy—is not without consequences. Many survivors are left to live with the risk of late and long-term effects from their cancer and cancer treatments, as well as the emotional and psychosocial implications of being a cancer survivor (ACS, 2018; Ng, 2014). The consequences of HL treatment include the development of secondary malignancies, cardiopulmonary disease, and endocrinopathies (ACS, 2018; Ng, 2014). In addition, survivors may face fear of recurrence, worry, or emotional changes while adjusting to cancer survivorship (Raphael et al., 2017).
Because of the impact of HL and its treatment, survivors are instructed to seek routine medical care for screening, prevention, and monitoring that is different from that recommended to the general population. These recommendations are tailored to survivors based on the type of treatment received and can include increased breast cancer screening, lung cancer screening, cardiac monitoring, and emotional and psychosocial support, as well as a host of other recommendations related to health and well-being (Sanft et al., 2019). HL survivorship care includes providing specialized care and addressing unmet needs (Cox et al., 2016). This context of care may be influenced by social, economic, and geographic factors. A large retrospective study of nearly 4,000 cancer survivors found that only 43% accessed a survivorship clinic for care, and this outcome was affected by survivors’ geographic location in comparison to the clinic (Nathan et al., 2016).
When HL survivors have access to resources associated with survivorship and follow-up medical care, their clinical care improves (Kvale et al., 2016). Web-based resources are available to cancer survivors and HL survivors through organizations such as the American Society of Clinical Oncology, ACS, and the Leukemia and Lymphoma Society. Tailored information that is interactive and customized to the individual may have an improved effect compared to static materials. Web-based options also may provide an effective way to reach a larger population. Therefore, a tailored web-based approach to providing survivorship care information to HL survivors may be a feasible, accessible, and scalable way to engage survivors on a large scale as an adjunct to clinical care (Corbett et al., 2018).
A prototype web-based intervention was developed by interprofessional survivorship and intervention development experts, based on current national guidelines of HL survivorship care, to inform HL survivors of the multifactorial late and long-term risks associated with their cancer and treatments, along with strategies to address those risks. The feasibility of providing the web resource to HL survivors was evaluated. In the initial study (Underhill et al., 2017), recruitment included all HL survivors who met eligibility criteria. These initial study data confirmed that outreach to HL survivors was feasible. Participants enrolled in the study proceeded to review content. Through patient feedback from the initial work, suggestions were made from participants about the need for further development (Underhill et al., 2017). Therefore, the purpose of the current study was adaptation and evaluation of the existing web resource.
Design and Setting
A user-centered model guided development and evaluation of the web resource (U.S. Department of Health and Human Services, 2019). The model is based on principles of web design, as well as health behavior, learning, and psychosocial theory, that integrates the individual user and environmental context when delivering health information in a web-based format. First, study investigators conducted focus groups of HL survivors to understand participants’ specification preferences for the web resource. Next, an iterative web-design process was followed with user testing by HL survivors to obtain patient perspectives on the adaptations. User testing is an iterative process in which users from the target population—in this case, HL survivors who meet the inclusion criteria—use the intervention alongside a research study member to provide feedback on the intervention and test the intervention functionality. The Dana-Farber/Harvard Cancer Center Institutional Review Board approved all study procedures. The study was conducted from April 2017 through December 2018.
Participant Identification and Recruitment
Eligible survivor participants included English-speaking adults who completed treatment for HL two years or more prior to study enrollment, and HL clinicians. The study team chose greater than or equal to two years post–HL treatment to ensure that participants were appropriate for the content delivered in the intervention. Individuals receiving end-of-life care were excluded. Eligible individuals were approached in the clinic. Providers were recruited through email. All survivor participants received a $40 gift card.
Focus groups: Focus groups were conducted in person and audio recorded. Screenshots of the previously developed prototype website were first reviewed at the beginning of the session. Pairs of participants were then asked to complete a card-sorting activity to rank the order of importance of specific content or to write in any content necessary that was not already part of the website. Index cards with the titles of each of the website’s subsection headers (e.g., cancer risks, care of your heart) were included, as well as blank cards to provide space to write new ideas. Using Wolpin et al. (2014) as a model, participants were then prompted to describe the most important content they would want to see on a website and how, as participants, they would like to engage with the content. The provider focus group participants discussed how the resource could be used clinically, how patients and providers could engage with the resource, and any potential barriers to clinical implementation. Based on HL survivor and provider focus group feedback, the investigators worked with the web design team to implement changes to the intervention and develop a final web resource, Survivor Wellness: Hodgkin’s Disease.
Development and individual usability testing: Iterative user testing was completed during the development phase. Three devices were tested: desktop computer, tablet, and smartphone. Cognitive interviews were conducted with HL survivors as they were using the website to learn of their perspective and experience with the resource. Once the interview was completed, an adapted version of the Acceptability E-scale (Tariman et al., 2011) was administered. Questions pertained to ease of use, understandability, enjoyment, time, value, and overall satisfaction. Responses ranged from 1 (lowest acceptability) to 5 (highest acceptability) on a Likert-type scale. Previously reported Cronbach alpha scores for the scale are acceptable at 0.79. The scale was adapted, with permission from the author, to include language reflective of the current study. Worry was assessed with a study-specific item that asked participants to rate, on a scale of 1 to 5, how much they agreed with the following statement: “The information that I read on the website made me worry.” If participants responded that the website prompted them to worry, participants were provided an option to describe their concerns in free text.
User testing was then completed, followed by final web development. Web development was led by a user-centered and web-based intervention development expert, and it was done by a team of content and intervention development experts. Web development included principles of health literacy to maximize success of patient materials and participant engagement, including keeping content brief, direct, and specific, with clear definitions of terms and use of illustrations to facilitate understanding (Foster et al., 2016).
De-identified transcripts of audio-recorded focus groups and interviews were transcribed and analyzed in NVivo, version 11.0. Demographic and survey data were summarized with descriptive statistics. An average Acceptability E-scale survey score at 24 or greater was considered acceptable (Tariman et al., 2011).
A total of 41 eligible HL survivors were identified and recruited for the focus groups, and 25 consented. Ten adult Caucasian HL survivors—six women and four men—aged 24–73 years completed participation in one of three focus groups. For the provider focus group, 20 eligible clinicians were identified. Nine providers participated, including six physicians and three nurse practitioners. In the cognitive interview phase, eight eligible participants were approached. Six user-testing interviews were completed with four female and two male participants aged 29–68 years; all were Caucasian.
Focus group participants prioritized physical and emotional well-being in the ranking session, as well as discussion about risk for second cancers. Survivors discussed a need to provide resources focused on psychosocial- and well-being. In addition, participants in the patient group were concerned about risk for second cancers, risk to family, and how to communicate with the family about the experience of being an HL survivor. In the clinician group, the conversation was focused on how the information provided on the website could be easily and automatically shared with other specialties, such as primary care. The clinicians also wanted to ensure that patients did not perceive the website as a replacement for clinical care. It was also discussed that, to ensure ease of use and access, the content would be integrated into existing systems, such as existing patient communication or electronic health communication platforms. It was discussed that members outside of the oncology team, such as primary care, should have access to this information to ensure comprehensive survivorship care.
Participants in the user testing phase discussed overall satisfaction with the website content and purpose and were able to test the site through all modalities (desktop computer, tablet, and smartphone) with minimal user interface issues identified. Suggestions were made by survivors that the website could be simplified and the purpose of the site clarified. Although the website was tailored to sex, age, and certain characteristics, participants did discuss that it would be more helpful to be more personalized to individual specific medical information. Participants encouraged removal of a formal username and login to increase ease of use, as well as a restructuring of the home page to be more appealing and informative. Multiple participants noted the use of the terms “Hodgkin’s disease” and “Hodgkin lymphoma” throughout the website and encouraged the use of Hodgkin’s disease because of familiarity to the participants, which was incorporated into the website for consistency. Finally, participants shared that a lot of information was provided in the web resource and that prioritizing what content to view would be helpful to the survivor. Based on the feedback obtained, the team revised the site as follows: the login was removed, the landing page was enhanced, prioritization was added to allow viewers to consider what content they wanted to view on this resource, and content focused on psychosocial well-being was added. Figure 1 presents an overview of the final website content, describing each section of the web resource, including a link to a demonstration video.
The mean Acceptability E-scale score for the six user-testing interview participants was 29.8 (SD = 5.7), confirming acceptability (score greater than 24). One participant scored a 21 on the acceptability scale, indicating below cutoff for acceptability. For this participant, scores were lower for the items asking how much the participant enjoyed the website, found it helpful, and felt the amount of time it took was acceptable. The median worry response for the single worry item was 3 of 5 (range = 1–3), indicating that participants had minimal to moderate worry, and no participants endorsed high worry caused by the website content.
This study reported on the user-centered development of a web-based resource for HL survivors using mixed methods. Overall, the website was found to be usable and acceptable. Specific changes were made to the website based on user input to improve content, functionality, and accessibility of the information. User testing in multiple modalities was successful, with minimal user interface issues identified. Clinicians reported that the resource would be most useful if integrated among health specialties and into the existing model of care.
Web-based resources can be an effective way to provide information to survivors in an accessible and scalable way (Kanera et al., 2016). Web resources have been shown to be acceptable and effective at improving a variety of medical and psychosocial outcomes for cancer survivors (Kanera et al., 2016). One benefit to a web resource is that it can be accessed anywhere at any time by a survivor with Internet access. In addition, from a study team perspective, web resources can be monitored and updated in real time as new evidence or guidelines become available. A benefit of the resource created through this project is that the platform can automatically ensure that links are still working. Research staff are charged with updating content based on guidelines or identifying new links, should previous web links be found to be no longer active. Although web resources are widely accepted, some individuals may still prefer print. Therefore, based on this project data, participants have the option of accessing web-based resources as print materials. As more educational materials are being provided electronically, web resources may be the preferred modality, particularly if the goal is to integrate the resource into the electronic health record or clinical workflow process.
Having an integrated model of cancer survivorship across practice settings has been documented as a national priority (McCabe et al., 2013). Discussion by providers and survivor participants in this study requested the same integration. Integration into clinical workflows and practice settings were outside the scope of this project. However, members of the study team are charged with the task of implementing this resource into clinical care. Barriers exist to this integration, including access to providers, capacity of primary care providers to be up to date with the many evolving cancer survivor guidelines, and no one successful or evidence-based model of care. In addition, varied access to electronic health records or lack of an ability to share data within electronic health records poses a challenge to sharing information across settings (Nekhlyudov et al., 2017). Partnering with primary care physicians and electronic health record administrators is critical to adapting such a resource as standard of care. In addition, principles of implementation science, such as incorporating metrics of organizational structure and systems, may facilitate the successful uptake of survivorship care resources in clinical practice (Hahn et al., 2017).
Patients were from a single academic cancer institute, and all participants who responded were Caucasian, which limits generalizability to more diverse patient populations. Because of scheduling constraints, not all consented participants were able to participate in a focus group, limiting the responses received. Health literacy was not measured as part of this study and should be considered for future studies. The authors were unable to address the need to incorporate the website into existing workflow, given the scope of this study. The goal of this integration is an important next step for the study team.
Implications for Practice and Research
HL survivors and providers who care for them desire a comprehensive resource that includes materials to promote physical and emotional well-being. Web-based resources that are tailored and interactive may be more effective than static information alone at promoting adoption of a recommendation. Therefore, use of resources, such as the Survivor Wellness website, along with standard of clinical care, can be a priority to those who care for individuals with or survivors of HL. Oncology nurses can advocate for these resources to be given to survivors and to be provided in collaboration with survivorship care and primary care experts. Ideally, the resources would be integrated through an existing clinical workflow.
Survivorship care is an important consideration for the growing number of HL survivors in the United States. Survivor Wellness is a web resource created to support the informational and psychosocial needs of HL survivors. The goal of the resource is to help HL survivors obtain information in a tailored and interactive format that will increase awareness of late and long-term consequences of HL and HL treatment, and to promote use of medical and psychosocial survivorship care resources.
The authors gratefully acknowledge the patient and provider participants who took the time to participate in this research. The study team also gratefully acknowledges the University of Washington Computer Informatics Research Group for contributing expertise in user-centered design to create this website. Finally, the authors gratefully acknowledge the administrative support provided through the Phyllis F. Cantor Center for Research in Nursing and Patient Care Services at the Dana-Farber Cancer Institute.
About the Author(s)
Laura N. Amweg, BSN, BA, RN, is a clinical research nurse at the Dana-Farber Cancer Institute and a doctoral student in the School of Nursing at the University of Massachusetts, both in Boston; Justin McReynolds, MS, is the technical program manager in the Department of Biomedical Nursing and Health Informatics at the University of Washington in Seattle; Kristina Lansang, RN, is a clinical nurse II at the Angeles Clinic and Research Institute Cedars Sinai Medical Center Affiliate in Santa Monica, CA, and was, at the time of writing, a research coordinator at Dana-Farber Cancer Institute; Tarsha Jones, PhD, RN, is an assistant professor in the School of Nursing at Florida Atlantic University in Boca Raton; Craig Snow, MHA, is a research project manager II at the Dana-Farber Cancer Institute; Donna L. Berry, PhD, RN, AOCN®, FAAN, is a professor in the School of Nursing at the University of Washington and a nurse scientist at the Dana-Farber Cancer Institute; Ann H. Partridge, MD, MPH, is the vice chair of medical oncology at Dana-Farber Cancer Institute and a professor in the Department of Medicine at Harvard Medical School in Boston, MA; and Meghan L. Underhill-Blazey, PhD, APRN, is an assistant professor in the School of Nursing at the University of Rochester in New York and a nurse scientist at Dana-Farber Cancer Institute. The authors take full responsibility for this content. Underhill-Blazey was funded by a 2017 two-year research grant from the Oncology Nursing Foundation Endowment. The article has been reviewed by independent peer reviewers to ensure that it is objective and free from bias. Underhill-Blazey can be reached at firstname.lastname@example.org, with copy to CJONEditor@ons.org. (Submitted October 2019. Accepted February 1, 2020.)
American Cancer Society. (2018). Late and long-term side effects of Hodgkin lymphoma treatment. https://www.cancer.org/cancer/hodgkin-lymphoma/after-treatment/lifestyle...
Corbett, T., Singh, K., Payne, L., Bradbury, K., Foster, C., Watson, E., . . . Yardley, L. (2018). Understanding acceptability of and engagement with Web-based interventions aiming to improve quality of life in cancer survivors: A synthesis of current research. Psycho-Oncology, 27(1), 22–33. https://doi.org/10.1002/pon.4566
Cox, C.L., Zhu, L., Ojha, R.P., Li, C., Srivastava, D.K., Riley, B.B., . . . Robison, L.L. (2016). The unmet emotional, care/support, and informational needs of adult survivors of pediatric malignancies. Journal of Cancer Survivorship, 10(4), 743–758. https://doi.org/10.1007/s11764-016-0520-0
Foster, J., Idossa, L., Mau, L.W., & Murphy, E. (2016). Applying health literacy principles: Strategies and tools to develop easy-to-read patient education resources. Clinical Journal of Oncology Nursing, 20(4), 433–436. https://doi.org/10.1188/16.CJON.433-436
Hahn, E.E., Munoz-Plaza, C.E., Garcia Delgadillo, J., Schottinger, J.E., Brasfield, F., Gould, M.K., & Parry, C. (2017). Building an embedded primary care survivorship model using an implementation science approach. Journal of Clinical Oncology, 35(5, Suppl.), 35. https://doi.org/10.1200/JCO.2017.35.5_suppl.35
Kanera, I.M., Willems, R.A., Bolman, C.A.W., Mesters, I., Zambon, V., Gijsen, B.C.M., & Lechner, L. (2016). Use and appreciation of a tailored self-management eHealth intervention for early cancer survivors: Process evaluation of a randomized controlled trial. Journal of Medical Internet Research, 18(8), e229. https://doi.org/10.2196/jmir.5975
Kvale, E.A., Huang, C.H.S., Meneses, K.M., Demark-Wahnefried, W., Bae, S., Azuero, C.B., . . . Ritchie, C.S. (2016). Patient-centered support in the survivorship care transition: Outcomes from the Patient-Owned Survivorship Care Plan Intervention. Cancer, 122(20), 3232–3242. https://doi.org/10.1002/cncr.30136
McCabe, M.S., Bhatia, S., Oeffinger, K.C., Reaman, G.H., Tyne, C., Wollins, D.S., & Hudson, M.M. (2013). American Society of Clinical Oncology statement: Achieving high-quality cancer survivorship care. Journal of Clinical Oncology, 31(5), 631–640. https://doi.org/10.1200/jco.2012.46.6854
Nathan, P.C., Agha, M., Pole, J.D., Hodgson, D., Guttmann, A., Sutradhar, R., & Greenberg, M.L. (2016). Predictors of attendance at specialized survivor clinics in a population-based cohort of adult survivors of childhood cancer. Journal of Cancer Survivorship, 10(4), 611–618. https://doi.org/10.1007/s11764-016-0522-y
Nekhlyudov, L., O’Malley, D.M., & Hudson, S.V. (2017). Integrating primary care providers in the care of cancer survivors: Gaps in evidence and future opportunities. Lancet Oncology, 18(1), e30–e38. https://doi.org/10.1016/s1470-2045(16)30570-8
Ng, A.K. (2014). Current survivorship recommendations for patients with Hodgkin lymphoma: Focus on late effects. Blood, 124(23), 3373–3379. https://doi.org/10.1182/blood-2014-05-579193
Patel, C.G., Michaelson, E., Chen, Y.H., Silver, B., Marcus, K.J., Stevenson, M.A., . . . Ng, A.K. (2018). Reduced mortality risk in the recent era in early-stage Hodgkin lymphoma patients treated with radiation therapy with or without chemotherapy. International Journal of Radiation Oncology, Biology, Physics, 100(2), 498–506. https://doi.org/10.1016/j.ijrobp.2017.09.048
Raphael, D., Frey, R., & Gott, M. (2017). Psychosocial distress in haematological cancer survivors: An integrative review. European Journal of Cancer Care, 26(6), e12640. https://doi.org/10.1111/ecc.12640
Sanft, T., Denlinger, C.S., Armenian, S., Baker, K.S., Broderick, G., Demark-Wahnefried, W., . . . Freedman-Cass, D.A. (2019). NCCN guidelines insights: Survivorship, version 2.2019. Journal of the National Comprehensive Cancer Network, 17(7), 785–794. https://doi.org/10.6004/jnccn.2019.0034
Tariman, J.D., Berry, D.L., Halpenny, B., Wolpin, S., & Schepp, K. (2011). Validation and testing of the Acceptability E-scale for web-based patient-reported outcomes in cancer care. Applied Nursing Research, 24(1), 53–58. https://doi.org/10.1016/j.apnr.2009.04.003
Underhill, M.L., Hong, F., Jones, T., Sprunck-Harrild, K., Walsh, S.K., Boyajian, R., . . . Partridge, A. (2017). Feasibility and acceptability of a web site to promote survivorship care in survivors of Hodgkin Disease. JCO Clinical Cancer Informatics, 1, 1–10. https://doi.org/10.1200/cci.17.00012
U.S. Department of Health and Human Services. (2019). Usability guidelines. https://www.usability.gov/what-and-why/user-centered-design.html
Wolpin, S.E., Halpenny, B., Whitman, G., McReynolds, J., Stewart, M., Lober, W.B., & Berry, D.L. (2014). Development and usability testing of a web-based cancer symptom and quality-of-life support intervention. Health Informatics Journal, 21(1), 10–23. https://doi.org/10.1177/1460458213495744