RERC: Research R-6

R-6: Investigation of WWW as Decision-support for Consumers

Task Leaders: Douglas Hobson, PhD (rehabilitation engineer)

Co-investigators: Bambang Parmanto, PhD (network computing specialist); Mary Ellen Buning, MS, OTR, ATP (clinician); Nigel Shapcott, MS, ATP (rehabilitation engineer)

Duration/Staging of Task: This research task will be conducted within the first 36 months of the 60-month RERC cycle, commencing January 1, 1999.

Rationale/Task Summary
It is our premise that the Internet and the WWW can serve as a powerful vehicle to inform, engage and empower consumers in improved decision-making. More specifically, we hypothesize that information gained from a WWW-based resource, such as WheelchairNet, can positively influence decisions related to acquiring new mobility devices. However, this hypothesis raises several questions.

Will people experience the personal relationship with the information source that research indicates is a prerequisite for the acceptance of advice (NCDDR, 1997)? As an RERC on wheeled mobility, will people trust and use the information found on its Internet site for decision-making? Since the WWW and virtual communities can offer information in several formats, what kind of information is the most useful to consumers? Is it high quality, impartial product comparisons, or is it the experiences of a community of others who have already dealt with this consumer decision?

The availability of traditional sources of information may affect how consumers use an Internet-based resource. Would consumers who live in regions devoid of wheelchair technology expertise make greater use of a "virtual community" to get information about mobility devices? In a related question, would consumers in metropolitan areas, rich with wheelchair technology expertise, use the resource in a different way?

This study will systematically investigate several of these currently unanswered questions. Specifically, we will focus on the effect of information presented via the WWW on the knowledge and decision-making competence of consumers who are preparing to acquire new mobility devices. This study has the potential to make a significant contribution to understanding how the Internet can support consumer's decisions and adaptive behavior, improve consumer partnerships with professionals, and improve dissemination of mobility-related research. Moreover, this new area of AT investigation will open new channels of research inquiry related to development of improved decision-making strategies.

Research hypothesis and objectives
Hypothesis 1: Consumers with access to both enhanced WWW-based resources and "traditional resources" will make wheelchair technology selection decisions that lead to greater levels of satisfaction than consumers with access to "traditional resources" only.

Hypothesis 2: Consumers with access to enhanced WWW-based resources will report both greater ease in gathering information and greater quantity and specificity of resources to support wheelchair technology decision-making than those with traditional resources only.

Hypothesis 3: Consumers who indicate successful use of WWW-based information for decision-making will possess specific attributes (age, educational level, prior experience, etc.) a different cluster of internal and external factors than those who do not.

The objectives and goals to be met through testing the above hypotheses are as follows.

To investigate the influence of enhanced WWW-based resources on the decision-making ability of primary users of wheelchair technology.
To compare the usefulness of information that is convenient and easy to acquire via the WWW versus traditional methods of getting information, i.e., from clinicians, rehab technology suppliers, and advertisements, etc.
To assess whether a certain kind of consumer is more likely to benefit from information and resources presented on a WWW site than a consumer with different characteristics.

Progress Report (12/31/99)

WheelchairNet, the virtual community that will be the "intervention" in this study, has been in active development this year. The community was built from the ground up with an eye on assembling the kind of infrastructure that would support a community of people concerned with wheelchairs. WheelchairNet contains sections named: Town Hall, Community Living, Products and Services, Wheelchair University and Discussion Groups. Since this task is directed toward understanding the role of the WWW in supporting and informing consumer decision-making about acquiring new wheelchairs, initial website development efforts focused on assembling useful resources for people who use wheelchairs.

The end of the first year brought the full implementation of the website's ability to determine the site resources and documents that are used most frequently. For example, we learned that PowerPoint lectures (converted to a format viewable on the WWW) have very high interest to site visitors. We are also able to identify our busiest periods, the sources of site traffic, and the frequency of returns to the site. We are currently focused on increasing the use of the discussion area of the site and increasing our ability to collect data from site visitors. Suggestions for improvement to the site are always welcome. The address for WheelchairNet is : http://www.wheelchairnet.org

The final activities leading up to the initiation of the parallel research study are underway. The outcome tool and the research protocol for the consumer-focused study are being finalized, and the tools for collecting data from study participants are being assembled. Subject recruitment and data collection will begin this year.

Expected Outcomes

This project will lay the groundwork for future investigations of consumer education and decision-support efforts using the Internet and the WWW. It is anticipated that it will reveal whether people with disabilities will use the Internet to support decisions and the factors that affect this process. If people with disabilities follow population trends, the Internet could be a valuable vehicle to deliver resources and to support communication with others on matters ranging from assistive technology to adaptation to disability. If consumers use information gained in this way to make better decisions and improve their quality of life, then organizations and consumer-oriented services can use this cost efficient and effective media with greater confidence. It is a natural step from WheelchairNet, which is focused on wheeled mobility technology, to the development of other on-line resources and communities.

On the other hand, if people with disabilities are skeptical about the information received via such a model, then the results may point to needed changes. Potential negative findings may call for increased education of people with disabilities in the use of the Internet or increased access to accessible and adapted computers. It may also indicate shortcomings in the presentation of information to consumers or their reluctance to alter the balance in their interactions with professionals.
This project could well be a model for how people in the twentieth century will learn about and communicate with others on a worldwide scale on matters related to assistive technology. Success with this proposed investigation will lead to additional investigations to explore additional questions such as those mentioned above.

Progress Report (02/02/01)

WheelchairNet's growth

WheelchairNet has had a year of strong growth. We invested considerable energy in saturating search engines so that anyone searching on a wheelchair-related topic will get referred to this site. More than 100 search engines are referring web-users to WheelchairNet.

The growth in the past year is seen when comparing visitor and download statistics. In January 1, 2000 1,457 visitors requested 6,075 pages and downloads. By December 31, 2000 these numbers had grown to 3,936 visitors who made 18,509 requests. Today, WheelchairNet, hosts on average 118 visitors per day. One fifth of the visitors to WheelchairNet are repeat visitors or people who know the site and return with the expectation of locating additional information.

WheelchairNet's usage pattern have also changed with more information being transmitted from the site on weekends. Perhaps this indicates and increase in consumers or end users using the website over those with a professional interest in wheelchairs. Many individuals are downloading PowerPoint Slide lectures from Wheelchair University or reviewing excerpts from books such as the Manual Wheelchair Training Guide and A Guide to Wheelchair Selection.

The last year saw increased efforts to develop the discussion area of WheelchairNet. A team of rehabilitation therapists and consumers are working with the WheelchairNet team and are replying to questions within their interest or experience level. Messages are replied to within a day of their being posted in the discussion area. An upgrade to the discussion software has been purchased, which will give email notification when a posted message receives a reply. This will replace manual notification and should further encourage dialog.

Research Protocol

A demographic questionnaires and three additional questionnaires (My life goals, Self-assessment of wheeled mobility knowledge, and My next wheelchair) were developed for use in measuring the effect of WheelchairNet on consumer readiness for decision-making. The three questionnaires were assessed for test-retest reliability using a group of 10 beta participants. Content validity was established for the Self-assessment of wheeled mobility knowledge during its development (Buning, M. E., 2001). The research protocol and questionnaires were approved by the Institutional Review Board of the University of Pittsburgh and data collection began in late August 2000.

Hypotheses

The following were established as the research study hypotheses and are guiding the R-6 task. They are stated here in their null form:

Hypothesis 1: There will be no difference between experimental and control groups of participants who use wheelchairs and have access to the Internet as measured by study pre and posttest scores. Scores on measures of locus of control, life goals, self-assessed knowledge, desire for device characteristics, and readiness to participate will be compared.

Hypothesis 2: There will be no difference between participants who use wheelchairs and have access to the Internet living in the local area and those geographically disbursed across the United States as measured by study pre and posttest scores. Scores on measures of locus of control, life goals, self-assessed knowledge, desire for device characteristics, and readiness to participate will be compared.

Hypothesis 3: There will be no pre-post difference between experimental and control groups on the frequency of rating the WWW as more helpful than traditional sources of information.

Hypothesis 4: There will be no relationship for the experimental group between online resources used and scores on posttest measures.

Seventy-five subjects representing all regions of the US have been recruited for the R-6 research tasks, which was entitled "Getting ready to choose your next wheelchair." Recruitment occurred through a designated study website which contained full information about the study, the admission criteria, the study consent forms and a form for submitting name and contact information. Announcements referencing this website were widely distributed through professional listserves such as RESNA, informal email distribution through groups of persons with mobility impairments such as the Paralyzed Veterans of America, postings on bulletin boards on disability-related websites, informal email word-of-mouth, and print announcements in local newspapers and Independent Living Center newsletters, and the Paralysis Society of America. A special effort was made to recruit persons from southwestern Pennsylvania.

Data collection

Data collection began in late August 2000 and continues until mid February 2001. Individuals have completed the study questionnaires by logging into a secure study website. Their responses are collected using interactive web technology which links forms containing the questionnaire items to the RERC's study database. Automated email messages are used throughout the participation period to provide printed directions, reminders to complete an unfinished questionnaire and instructions for activity during the intervening 6 week period between pre and posttest completion. The participants randomized to the experimental condition are asked to log onto a clone of the WheelchairNet website using assigned user ID and password. Software that analyzes the webserver log will provide data on the frequency and duration of website visits by members of the experimental group. This will provide the investigators with information about the amount of exposure to the information resources on WheelchairNet experimental participants actually had.

During the data collection period the RERC's webserver was relocated from the University of Pittsburgh Medical Center's network to the University of Pittsburgh's network. The disruption in Internet service was confined to December 23-25, 2000, which seemed to create minimal disruption for participants.

Expected outcomes

The data from the study is now being analyzed and results with be reported in a doctoral dissertation, to the research participants and to the research community through anticipated publications. The data is expected to shed information on the use of the WWW as a support for wheelchair selection and decision-making.

Publications

Buning, M. E. (2000). WheelchairNet. Paper presented at The 16th International Seating Symoosium, Vancouver, BC.
Buning, M. E. (2001, June 2001). The development of validity and test-retest reliability for measuring of the effect of WheelchairNet on wheelchair decision-making by consumers. Paper submitted Annual RESNA 2001Conference, Reno, NV.
Buning, M. E., & Hobson, D. A. (2000). WheelchairNet: a virtual community focused on wheeled mobility. Paper presented at the RESNA Annual 2000 Conference, Orlando, FL.

Final Report (12/31/01)

The Website

WheelchairNet has continued to grow astronomically. Whereas the traffic on the site last year (12/31/00) was 3,936 visitors who made 18,509 requests for information, the figures for this year 12/18/01) are 56,545 unique visitors who made 362,355 requests for information

Efforts were doubled this year to increase the accessibility of the site for persons using screen readers. We previously had used PDF format for our slide lectures and now we are moving to a text-based html format. The impetus for this was a flock of new slide lectures that were to be added in the Wheelchair University section of the website. We have added descriptions of the graphics on the slides so now they are accessible to everyone. The site has been upgraded and maintained this year through the significant efforts of Sandra Hubbard, MS, OTR, a research associate working for the RERC. A team of therapists and faculty members responded to many requests for technical assistance—more than 450 this year. These requests arrive at the site via email or telephone. The team consists of Sandra Hubbard, MS, OT, Barbara Crane, MS, PT, Laura Cohen, PT, Mary Ellen Buning, PhD, OTR, ATP, Gina Bertocci, PhD, Rory Cooper, PhD, and Doug Hobson, PhD.

The Research Study

The final data from the research study described above was collected on March 15, 2001. The next few months were spent in organizing the data and analyzing it with statistics. The data were used in the defense of a doctoral dissertation written by Mary Ellen Buning and defended successfully on July 18, 2001. The results of the study follow with the hypotheses listed and the results of each. Finally the conclusions and discussion will be present.

The Sample

The participants in this study had more education and higher employment rates than the population of person who use wheelchairs. However, they seem to be representative of those who have access to the Internet and use it to acquire information whether it is related to mobility products or how to use an advocacy service. They were evenly divided by gender and their ages ranged from 22 to 63 with a median age of 47 years. Their diagnoses were in rank order: quadriplegia, paraplegia, cerebral palsy, spina bifida, postpolio, arthritis, head injury, stroke, and osteogenesis imperfecta, arthrogyposis, amputation and several other rare conditions. The participants had an internal health locus of control and appeared to be more involved or better educated about the wheelchair selection process. All of these characteristics affect the generalizability of the results of this study. It is not known how the findings of this study would generalize to those without these characteristics.

The higher than expected ratio of persons with quadriplegia may have affected the study in several ways. Their need for a good fit with their mobility device is more critical than those with movement and postural management options, thus they may have been more highly motivated. It may also be more difficult for these individuals to locate the expertise they need to acquire a well-fitting wheelchair, thus they experience greater frustration. Finally, they may be more motivated as WWW users. Because quadriplegia so significantly affects most activities of daily living, an adapted computer linked to the Internet would be a highly adaptable and attractive activity. No longer are the issues of accessibility, travel complications, or dismissive attitudes a problem.

The study participants were actually more active than most participants in the Pew studies were speaks to the fact that this sample is strong Internet users (Howard, Rainie, & Jones, 2001; Rainie, 2000). For persons with significant impairment, the Internet offers incredible access to communication, information and resources. Other than the higher than expected size of this subgroup, it does not appear that any characteristic of the sample interacted with the research design to create the results found.

The Hypotheses

Hypothesis 1:

There will be no difference between experimental and control groups of participants who use wheelchairs and have access to the Internet as measured by study pre and posttest scores. Scores on measures of locus of control, life goals, self-assessed knowledge, desire for device characteristics, and readiness to participate will be compared.

Two significant findings resulted from the testing of this first hypothesis. First, there was an increase in self-assessed knowledge about wheeled mobility devices as measured by the Self-Assessment of Wheeled Mobility Knowledge. The second was a significantly greater readiness to participate in wheelchair decision-making as indicated by a subscore created from adding the value of two items on the posttest only questionnaire Decision-Making Readiness Survey. There were no changes detected in any dimension of health locus of control or on desire for or expectations about wheeled mobility devices.

It should be noted that the significant change detected in knowledge about wheeled mobility devices was a self-rating and not the same as an objective assessment of knowledge. However, the rating would seem to indicate that participants were feeling more informed and better exposed to the issues and information on the topic of wheeled mobility. This greater confidence and awareness of options is positive preparation for consumers especially when it is accompanied by questions about the relationship between their needs and product features.

Participants also reported feeling more ready to participate in wheelchair decision-making. This was an important goal of the study. Not only do professionals need more open and consulting approaches when working with consumers, but also consumers need a greater sense of empowerment about their essential contribution to the assessment process. Without prior thought about life goals, mobility needs and environmental and transportation realities, the partnership will be weak as it will not have access to essential information. It is hoped that greater readiness will allow participants to approach and use professionals more effectively at the time of actual wheelchair decision-making.

The My Next Wheelchair questionnaire did not prove useful in demonstrating change in desire for product features. The study participants, with their years of experience in using and replacing wheelchairs, were already quite knowledgeable about wheelchair features and their availability.

The ANCOVA statistical procedure was valuable to the research design. Through its use of the pretest as a covariate, it helped to account for the wide variability in knowledge and experience among study participants. In this same way, it controlled for any pre-study exposure to the WheelchairNet site that could not be accounted for by direct questioning.

Hypothesis 2:

There will be no difference between participants who use wheelchairs and have access to the Internet living in the local area and those geographically disbursed across the United States as measured by study pre and posttest scores. Scores on measures of locus of control, life goals, self-assessed knowledge, desire for device characteristics, and readiness to participate will be compared.

This hypothesis was designed to test the idea that sampling people about wheelchair decision-making and experiences is as effective with remote participants as it would be with local participants. It seemed important to determine if there was a difference between surveying local participants—a situation in which much is known about the density of disability and mobility device resources, the topography and in which people can be met face-to-face—and remote participants. It was hoped that a finding of no difference would validate the use of the Internet as a tool for future survey research.

For this hypothesis, the significant finding was that there is no difference between local and national participants, i.e., the local sample was just like the national sample, on all but one study measure. It showed one highly significant difference—a significantly high Powerful Others Health locus of control for SW Pennsylvania.

Initially, the meaning of this finding was puzzling since the local participants were equivalent to the national on internal and chance health locus of control constructs. The post hoc analysis, in segmenting the comparisons by geographical region, was helpful in explaining the meaning of this finding. It is completely plausible that Southwestern Pennsylvanians are different from Californians. After all, Berkeley, California is the birthplace of the Independent Living Movement, and is considered a model for accessibility and community inclusion (Shapiro, 1993). As compared to Southwestern Pennsylvanians, Californians have hospitable outdoor climate most of the year, can easily locate one-story housing and live in a predominantly level community with good accessibility for transportation and activities.

Hypothesis 3:

There will be no pre-post difference between experimental and control groups on the frequency of rating the WWW as more helpful than traditional sources of information.

The significant finding for this hypothesis was an increase in the frequency of citing the WWW as both “most helpful source of knowledge” and “second most helpful source” of information given a choice of 10 sources. The significant increase was noted in the experimental group following six weeks of exposure to WheelchairNet.

With this finding it would be misleading to focus on the change in ranking alone. As the most helpful source of information WheelchairNet increased from tenth to fourth position and from eighth to fourth as the second most helpful. In this fourth position, it ranked below personal experience, information from OTs and PTs, and other wheelchair users. This is considered an extremely appropriate placement. It means that as a resource WheelchairNet is placed ahead of other print, advertisements, and ancillary sources of information but behind personal experience, consultation professional OTs and PTs, and the experience of other wheelchair users. The value of experience in a group that has mean years of wheelchair use of 18.6 years must be highly regarded. This is especially important especially since this sample also had a high number of persons with tetraplegia (n=27), an impairment with complex health and mobility considerations.

Hypothesis 4:

There will be no relationship for the experimental group between online resources used and scores on posttest measures.

The significant finding related to this hypotheses is that a correlation of moderate strength exists between the gain scores on the experimental group’s measure of wheeled mobility knowledge and the two measures of their use of WheelchairNet—the number of pages viewed and total time online.

Unfortunately, reviewing the results of the FunnelWeb analysis of the webserver log indicates unexplained differences in participant tracking. Some undetectable client variable or perhaps a software bug allowed more complete data collection for some participants and not others. Even though the statistics report a significant finding the quality of the data makes it necessary to review this finding with caution.

Future Work

This study also opens the door for future work by others. First, replication of this study would add credibility to its findings and open the door to possible recruitment of more minority group participants. Modifying this study protocol to include only persons with progressive impairment such as Multiple Sclerosis, Muscular Dystrophy, etc. would create a useful contrast to this work. It would help to identify the wheelchair selection issues that arise when individuals are unable to predict their functional status in the short-term future.
Certainly more studies focused on persons with disabilities could be conducted using technology similar to that used in this study. Further work should investigate the factors that underlie the delivery of wheelchair and seating devices in the United States. Consumers know there are wide variations in therapist knowledge and ability to function as an empathetic professional comfortable with consulting models of service delivery and holistic notions of disability. This leads consumers to have unpredictable expectations that they will get the services they really want by working with a therapist. Expansion of therapist expertise in seating and wheeled mobility is needed. Currently there is not even a baseline figure for the amount instruction in seating and wheeled mobility assessment that is part of pre-professional preparation for occupational and physical therapy practitioners. Better documentation of the contribution made by certified rehabilitation professionals will increase demand for them.

Finally, the finding of a significantly different measure for local participants on the construct, Powerful Others Health Locus of Control reinforces the importance of recruiting study participants nationally. Sampling widely from this diverse population will avoid skewing samples with the unknown differences that are hidden within regional cultures. This finding may especially be important for local researchers who investigate topics with the local population of persons who use wheelchairs—especially in studies that involve social attitudes, health beliefs, and self-concept as a person with a disability.

References:

Howard, P. E., Rainie, L., & Jones, S. (2001). Days and nights on the Internet: The impact of a diffusing technology. American Behavioral Scientist, 45(Special Issue).
Rainie, L. (2000). Tracking online life: How women use the Internet to cultivate relationships with family and friends. Washington, DC: The Pew internet and American Life Project.
Shapiro, J. (1993). No pity: People with disabilities forging a new civil rights movement. New York: Time Books, a division of Random House, Inc.

Publications

Buning, M. E. (2002). Using the WWW to study the experiences of persons who use wheelchairs: local area versus distant participants. Submitted to the Proceedings of the RESNA 2002 Conference, June 27-July 1, Minneapolis, MN.

Buning, M. E. (2001). The development of validity and test-retest reliability for measuring of the effect of WheelchairNet on wheelchair decision-making by consumers. The proceedings of the RESNA 2001 Conference, June 22-26, Reno, NV. 260-263

Buning, M. E. (2000). WheelchairNet. Paper presented at The 16th International Seating Symposium, Vancouver, BC.

Buning, M. E., & Hobson, D. A. (2000). WheelchairNet: a virtual community focused on wheeled mobility. Paper presented at the RESNA Annual 2000 Conference, Orlando, FL.

Contact Person: Mary Ellen Buning