Main

In December 2000, 42% of the European females were using the Internet with a clear upward trend (Kögel and Longo, 2001). Notably, women older than 50 years turned out to be the most active group of Internet users (Pichler and Gilg, 2001). In April 2001, Harris Interactive published results indicating that 75% of all adults who had access to the Internet used the Internet to collect health and medical information (Taylor and Leitman, 2001). In the USA, 54% of the Internet users searched the Web for health-related information compared to 26% in Europe, as demonstrated by a survey of Health On the Net (HON) Foundation in 2001. More females (51%) than males (49%) used the Internet to search for health-related information (HON, 2002). The main obstacle for patients is ‘that there is so much information on the Net, big parts of it are incomplete, misleading, or inaccurate’ (Achenbach, 1996). Correspondingly, Silberg et al (1997) had criticised the information on the Internet as not peer reviewed and pointed out the dangers of relying on such invalidated information.

Due to the high incidence of mamma carcinoma (Weir et al, 2003), breast cancer web sites attract a high number of information seekers. In addition, breast cancer patients have gained a higher awareness of their disease during the last decades. They demand breaking taboos considering breast cancer, they refuse to hide their illnesses any longer, and many develop a different attitude towards their disease, which is known as ‘fighting spirit’ (Greer, 2000). Breast cancer patients, therefore, can be considered pioneers of a new type of self-confident patients (Biel, 1997).

Well-informed patients often scrutinise doctors' recommendations, search the Internet for specialists, and contact other patients all over the world (Degner et al, 1997; Blanchard et al, 2002). These patients want to participate actively in the therapeutic decisions and use the Internet ‘to acquire expertise to display competence in the face of serious illness’ (Ziebland et al, 2004). But, as Internet users, they play both an active and passive role. The passive role explains why these patients might be influenced by Internet information, and the active role – for example, patients as web authors – explains why the Internet reflects patients' needs. Both roles contribute to patients' individual belief and knowledge about cancer, which was shown to influence their coping and compliance.

But very little is known about intercultural differences between breast cancer patients (Nicholson, 1996; Smith et al, 2001) and, even less, about differences between English- and German-language web sites as mirrors of intercultural differences in patients' needs. Though the Internet is propagated as a ‘global medium’ theoretically enabling millions of people worldwide to communicate and share information electronically, the Internet as used by cancer patients emerged as a ‘local medium’, limited by language barriers. Analysing the internet use of 927 patients with breast cancer, we found that 260 (28.1%) among these are using the Internet to obtain information about their disease (unpublished data). More than a third of these patients with breast cancer (35.0%) turned out to be monolingual. Therefore, an increasing number of internet users are unable to use any foreign language web information autonomously. Conversely, their contributions to the Internet are inaccessible to monolingual members of any foreign language audience. This evokes questions about intercultural differences in disease management, pharmaceutical markets, and patients' behaviour and needs, mirrored by the Internet. Therefore, we analysed the Internet by evaluating a substantial number of breast cancer web sites in English and German language, representing the most and second-most frequent language (52.0 and 7.0%, respectively) used on the Internet (Pimienta et al, 2001).

Materials and methods

Search techniques to collect web sites for analysis

For evaluation purposes, a meta-search procedure was established and validated at the University Hospital in Freiburg, Department of Radiotherapy. To search for the keywords ‘breast cancer’, ‘Brustkrebs’ and ‘Mammakarzinom’, we modified (‘patched’) the meta-search programme Copernic© 5.0 (Kunst et al, 2002) to enable the programme to download up to 10 000 hits from 18 popular search engines (instead of the default limitation to 100 hits), representing the pooled content of about 1000 search engine result pages. Installed on local computers, Copernic© 5.0 provided a common interface to download, filter, and group web search results and check for duplicates.

Between March 2001 and February 2004, the web sites included in the study were repeatedly visited. The web sites were evaluated firstly between March 2001 and November 2001 and secondly between November 2003 and February 2004, to confirm the results and to prove the persistence of the web sites. The first evaluation was immediately followed by a second evaluation carried out by an independent observer to minimise the inter-rater variability. According to the study protocol, all web sites with a Cohen's kappa of less than 0.7 (thus proving a high interevaluator variability) were excluded, modifying the protocol of Bichakjian et al (2002). Finally, only those web sites which were accessible for at least 2 years and disseminated medical information pertaining to breast cancer were included in the analysis. Irrelevant web pages were excluded, as were link pages, non-English- or non-German web sites.

Definitions of the analysed subjects

Before the study started, the four observers (radiooncologist, gynecologist, and two researchers) received an identical training concerning the inclusion and exclusion criteria of the web sites to ensure the best possible data quality. Their training involved general aspects of computer and internet use, web search and programme-specific techniques, as well as education in breast cancer treatment, based on principles of evidence-based medicine, current guidelines, and consensus recommendations. Moreover, the observers were trained in judging web sites using the criteria and definitions of the study.

If a certain hit, which the search engines returned for the query, fulfilled the inclusion criteria, analysis was continued using an algorithm to select and combine the corresponding hits to ‘web projects’. The ‘extension’ of a web site was determined by locating the home page (where possible), and then exploring the web site by following the links to find all associated, lower hierarchical web pages. This procedure was used to identify all matching sites belonging to one ‘conclusive’ informational project. These complete web sites were called ‘web projects’ so as to avoid misunderstandings, because the term ‘web site’ lacked a generally accepted, content-related definition at the time of our study. The definitions used in our study are summarised in Table 1.

Table 1 Definitions of web-technical terms used in this study
Full size table

Approaching the web sites for evaluation

Primarily, the hyperlinks provided by the search engines were checked for relevance and categorised into ‘relevant’, ‘semirelevant’, and ‘nonrelevant’ according to Su et al (1998). Hyperlinks were considered ‘relevant’ if they offered information on several aspects of the disease regardless of the quality of the information provided. They were rated ‘partly relevant’ if they only dealt with one special aspect (e.g. web sites dealing with cancer prevention, aftercare, or mammography), and ‘irrelevant’ if they did not deal with any aspect at all (e.g. web sites reporting celebrities' battle with breast cancer). Nonrelevant web sites were excluded from the evaluation. The in-depth analysis was performed using items according to pre-defined criteria. The questionnaire used for the evaluation consisted of two parts: the ‘formal’ and the ‘content-related’ analysis – as we were aware of the limitation that no distinct differentiation between formal and content-related aspects is possible in nature. The formal analysis was based on Silberg's essential accountability criteria (Silberg et al, 1997) and included: site disclosure (ownership, advertising, commercial funding arrangements or potential conflicts of interest), site currency (dates of posting and updating), site authorship (authors and contributors, their affiliations and relevant credentials), and site attribution (references and sources). As described below, Silberg's criteria were applied in an extended version.

The ‘content-related’ aspects were evaluated by the in-depth analysis of the information provided by the web site using principles of evidence-based medicine, current guidelines, or consensus recommendations for the treatment of breast cancer.

Questionnaire Part I: Silberg's criteria

The first part of the analysis used Silberg's validated criteria list extended by further items abstracting the quintessence of other published criteria for information quality (Table 2). Silberg's criteria were rated as available or not. Thus, we additionally searched the web projects for aspects such as privacy protection, disclaimer to the links, usage of the presented information (e.g. the warning not to use the medical advice instead of contacting a physician), presence of a quality seal (HON, 1997), orthography, and grammatical errors. Further, a small size of each web page (measured in kbytes), leading to a fast loading time, is one important aspect of good web site usability. A maximum page size of around 40 kbytes is commonly recommended for quick access to the web site.

Table 2 List of initiatives to assess and improve quality of information
Full size table

The up-to-dateness of the web sites was assessed by repeated visits in order to check their content for currency and modifications. This procedure documented modifications and aimed to uncover sites which automatically set the current date as the ‘last update’, thus leading the Internet user to believe that the site is current and state of the art.

Questionnaire Part II: content-related analysis

The second part of the questionnaire analysed the presented information such as treatment options, alternative treatment options, screening, follow-up, treatment-related risks, side effects, treatment time, and information about self-help and support groups. The items of the validated questionnaire were analysed as per the protocol providing detailed instructions for content evaluation.

Statistical analysis was performed using SAS and JMP® (SAS Corp.). The data were evaluated with descriptive statistics, χ2 tests, and Cohen's kappa to ensure low inter-rater variability (Cohen, 1960).

Results

Hits returned to the web query

Search engines returned totally 10 616 unique hits related to breast cancer (Table 3). Of these, 6026 nonrelevant hits were excluded from the study, and 4590 relevant hits were analysed. Of these, 1888 web pages belonged to 132 English-language web sites and 2702 belonged to 65 German-language web sites. Correspondingly, English-language web projects consisted on average 14.3 web pages, whereas German-language web projects consisted on average of 41.6 web pages. In all, 13.7% (937 of 6804) of the English-language and 8.6% (328 of 3812) of the German-language web pages turned out to be semirelevant ones, providing only information about the partial aspects of breast cancer (mostly prevention or aftercare of breast cancer). No web site disappeared in the period of the 2 years of the study or was substantially modified. After the training period in the pre-study, the two independent raters achieved a kappa value of more than 0.7 for all web sites. Thus, all 197 web projects were included in the study.

Table 3 Number of hits contributed by each search engine
Full size table

In all, 81.2% (n=160) of the English- and German-language web projects targeted patients, whereas 5.1% (n=10) targeted physicians. Altogether, 6.6% (n=13) of the web projects targeted both patients and doctors, 6.6% (n=13) other groups (e.g. web sites only for patients' relatives or business-to-business communication sites).

Formal aspects

To analyse the formal criteria, we used Silberg's core criteria (Silberg et al, 1997).

(1) Authorship: We identified the author(s) and their qualifications significantly more often on German-language web projects than on English-language web projects (Table 4).

Table 4 Results of the content-related analysis (Chi-square test)
Full size table

(2) Attribution: Conversely, we found references, contact e-mail address, and information about privacy protection of the users more often on English-language web sites. A disclaimer, which warned that the site should not be used as a substitute for doctor's visit, was displayed in 40.2% of the English-language web projects compared to 13.8% (P<0.001) of the German-language web projects. In all, 14.0% of all English- and German-language web projects displayed the HON Seal (HON, 2002). More than 50% of these web projects had a commercial background. Only a few web projects had a remarkable accumulation of mistakes in orthography or grammar. Using an access with analogue transmission technique, the average download time of pages belonging to English web projects was 12.5 s (German web projects: 9.0 s). The average size of the English web pages was 70.4 kbytes (German web pages: 52.9 kbytes).

(3) Disclosure: Overall, information about the site owner could be obtained in 183 (92.9%) web projects. Web projects were provided by institutions such as universities or noncommercial organisations in 47.2% (n=92), industry, companies or commercial organisations in 41.6% (n=82), and private persons in 10.7% (n=21). Elements of advertising (e.g. banners, pop-ups) were detected on 80 (60.6%) and 45 (69.2%) of English- and German-language web projects, respectively (Table 4).

(4) Currency: According to the update information displayed, English-language web projects were significantly more frequently updated within the last 12 months than German-language web projects (Table 4). As revealed by repeated visits, the displayed information about updating was, in fact, accompanied by an observable modification of the web site in 25.8% (German web projects: 53.8%) of the web sites. Some web sites, however, made use of update scripts which automatically set the ‘last update’ field to the current date (Table 4).

Analysis of the web site content

Evaluating the content of the web sites, we found that information about the four main therapy modalities for breast cancer (surgery, chemotherapy, radiotherapy, hormone therapy) were more frequently seen on English-language web projects compared to German-language web projects (Table 4). We observed the same when we examined the presentation of new approaches such as immunotherapy (15.9 vs 12.3%) and stem cell transplantation (16.7 vs 6.2%). Conversely, palliative therapy (4.5 vs 16.7%) and methods of alternative medicine (18.2 vs 46.2%) were significantly more often presented on German-language web projects; pain therapy obviously showed the same trend (2.3 vs 4.6%). Side effects and risks of therapy were the domain of English-language web projects whereas German-language web projects more often dealt with disease-related information (like risk factors for breast cancer, prognosis, genetics, epidemiology, cancer aftercare, and self-help groups).

Discussion

Recent studies confirmed that patients demand high-quality health care information (Guadagnino, 2000; LoBuono, 2000; Chen and Siu, 2001; Fogel et al, 2002; Forbriger and Glattes, 2003). Hence, many efforts (Table 2) were made to improve the quality of online information with moderate success (Pandolfini et al, 2000). Any improvement needs to be monitored by reliable and valid evaluation tools for information quality. In our study, therefore, procedures and definitions of the evaluations were validated and fixed in a protocol, including detailed instructions for training of the raters. The items of the analysis were based on published and generally accepted instruments for the evaluation of information quality. However, the set of items that most accurately describe the quality of web sites (Gordon-Murnane, 1999; Jitaru et al, 1999) and the procedure of the analysis (Eysenbach et al, 2002) itself are controversially discussed.

We found the most striking differences regarding web sites dealing with ‘complementary alternative medicine’ (CAM) (Markman, 2001). We defined CAM as ‘practices not generally recognised by the medical community as standard or conventional’, including, for example, herbal preparations, megadose vitamins, magnet therapy, spiritual healing, and meditation. Significantly more German-language web sites (46.2%) than English-language web sites (18.3%) dealt with this aspect of breast cancer treatment (Table 4). Most of the German web sites dealing with CAM contained information directed against conventional therapies. Corresponding to our data, Ernst and Schmidt (2002) judged five out of 13 web sites dealing with alternative medicine as harmful for cancer patients if the advice provided was followed. Moreover, three out of these 13 web sites overtly discouraged cancer patients to employ conventional therapies.

The lower frequency of English-language web sites with alternative medicine compared to German-language web sites may be due to the legal practices in the US to hold content providers responsible for what can be regarded as ‘Internet fraud’, leading to high indemnities (Mack, 2001). Further tragic instances where misinformation brought death (Hainer et al, 2000) may provoke that, in future, ‘compensation for misinformation’ will be justifiable. Besides this judicial aspect, we assume differences in the needs for and susceptibility to alternative medicine between the audience of English- and German-language web sites. Anti-science attitudes meshed with New Age mysticism, belief in the superiority of ‘natural’ products, and inability to make an informed choice about a health care product (Beyerstein, 2001) were known to promote the popularity of alternative medicine, even or especially among otherwise intelligent patients (who are the typical ‘health seekers’ on the Internet). In all, 51.6% of the patients in England, however, use CAMs (Werneke et al, 2004), and the impact of the Internet on self-medication with herbal remedies and even on trafficking in controlled drugs is still on the rise (International Narcotics Control Board (INCB), 2004).

In general, we found information related to curative therapy (especially side effects and risks of therapy) and scientific-based aspects of breast cancer (e.g. current trials or results of recently published studies) significantly more frequently on English-language web projects. Conversely, we found information related to palliative therapy and disease-related information significantly more frequently on German-language web projects (Table 4). As most English-language web projects were of US origin, this may reflect the role of judiciary as well as the American superiority in terms of research and basic science. Apparently, the emphasis on therapy-associated risks and side effects also aim to hedge the risks of possible indemnities (similar to our finding that on English-language web sites alternative medicine is frequently disregarded, as described above).

Further, authors of English-language web projects obviously assume that their audience expects information about cutting edge therapies and studies evaluating new agents, though this information jeopardises the intelligibility and readability of their web sites (but may increase authority and credibility in the perception of the Internet users). On the contrary, the authors of German-language web projects seem to assume that their audience prefers the digest of such information rather than details about current trials or recently published studies. In fact, asking 213 patients for the most important detail of information they want to receive, we found that patients most frequently (more than 90%) answered ‘prognosis’ (unpublished data).

Similar to the US American audience, however, a recent study reported a change in access to information and patient behaviour over the past few years: Patients increasingly demand information on current clinical studies, new agents, genetic risk factors, diagnosis, and screening procedures (Hiller, 2004). These data base on 210 000 calls, e-mails, and letters, addressed to the German Cancer Information Service (German Cancer Research Center) since 1986.

Information about palliative therapy options (e.g. bisphosphonates administration or radiation of metastatic lesions) was more frequently found on German-language web projects (16.9%) compared to English-language web projects (4.6%), suggesting that English-language web projects more often focus on curative therapies (Weissenberger, 2004). Both English- and German-language web projects rarely mentioned pain management for metastatic breast cancer (English vs German web sites: 2.3 and 4.6%, respectively). Evidently, the majority of English- and German-language web sites focused on the curability of breast cancer, but neglected helpful information for a considerable number of breast cancer patients with metastatic disease.

Compared to German web projects, the authors of English-language web projects were convinced (if at all) of the necessity rather to ensure privacy than to disclose authorship. We found a privacy statement on 23% of the English and 14% of the German web sites. In all, 30.3% of the English web sites (German web sites: 55.4%) provided information about the authors, and 22.7% (German web sites: 49.2%) about their qualifications. Shon and Musen (1999) had found authorship and qualification statements in 20% of the medical web sites analysed (n=97). After 3 years, Meric et al (2002) found evidence of authorship in 57% of the web sites analysed (n=184), but only 17% displayed the author's name, qualifications, and affiliation.

In conclusion, analysing and comparing large English- and German-language breast cancer web sites, we found significant differences in web content and quality. English-language web sites provided better overall quality (often supported by scientific-based information, e.g. about current trials or new agents), but focused on curative treatment options and their risks and side effects. German-language web sites comprised significantly more often palliative and alternative medicine, but a substantial number of web sites were a serious potential danger to patients. As we pointed at the strengths and weaknesses of English- and German-language web sites and observed several intercultural differences, our results may help physicians to more accurately satisfy patients' information needs and to improve patient education. If cancer patients use the Internet for information and support needs that are not met through conventional health care (Ziebland et al, 2004), physicians should make efforts to correct these communicating deficiencies and provide their patients even these information that are not met through the Internet.