ࡱ > d e f c y q` ; bjbjqPqP
: : d [ r r r >
>
>
$ b
R R R P S T b
V W : X p X X ` h | 4k @
$ s h ۭ . >
3 _ | ` 3 3 . X X C c c c 3 X l X : >
X ^ c 3 c c R
, >
N X
V pQ> R Y 0 8 # 4 N >
N X tl pr F c v l "z tl tl tl . . p tl tl tl 3 3 3 3 b
b
b
D R b
b
b
R b
b
b
Accessibility
of Spains Regional Government Websites
Synthetic Version
Table of Contents
TOC \o "1-3" \h \z \u HYPERLINK \l "_Toc128911458" Accessibility of Spains Regional Government Websites PAGEREF _Toc128911458 \h 1
HYPERLINK \l "_Toc128911459" 1. The Information Accessibility Observatory at Discapnet PAGEREF _Toc128911459 \h 1
HYPERLINK \l "_Toc128911460" 2. Selection of the sample PAGEREF _Toc128911460 \h 1
HYPERLINK \l "_Toc128911461" 3. Aspects of accessibility to be analyzed on the pages PAGEREF _Toc128911461 \h 2
HYPERLINK \l "_Toc128911462" 3.1 Technical evaluation of web accessibility PAGEREF _Toc128911462 \h 2
HYPERLINK \l "_Toc128911463" 3.2 Assessing website accessibility based on user feedback PAGEREF _Toc128911463 \h 3
HYPERLINK \l "_Toc128911464" 4. Analysis of the results PAGEREF _Toc128911464 \h 3
HYPERLINK \l "_Toc128911465" 4.1 Analysis of the results of the technical evaluation of web accessibility PAGEREF _Toc128911465 \h 3
HYPERLINK \l "_Toc128911466" 4.2 Ranking of technical accessibility scores, in % success PAGEREF _Toc128911466 \h 9
HYPERLINK \l "_Toc128911467" 4.3 Analysis of User Feedback Scores PAGEREF _Toc128911467 \h 11
HYPERLINK \l "_Toc128911468" 4.4 Combined Scores PAGEREF _Toc128911468 \h 15
HYPERLINK \l "_Toc128911469" 5. Conclusions PAGEREF _Toc128911469 \h 16
ACCESSIBILITY OF SPAINS REGIONAL GOVERNMENT WEBSITES
December 31, 2005 marked the legal deadline for all web-based public services to comply with generally acknowledged criteria for accessibility. Spains Autonomous Communities (Regional Governments) have thus had more than three years to adapt their sites to meet the new regulation as set forth in the Law of Services for the Information Society and E-Business passed in June 2002.
The present study attempts to show their status based on how they fared on a technical analysis of 12 criteria for accessibility and a user feedback survey of task performance on the websites of 17 Regional Government websites.
The results of the study reveal a disheartening view in which much remains to be done. Only one website (from the Region of Murcia) achieved a combined score of over 50%, squeezing by with 50.75%. Furthermore, none of the sites scored even 50% on the criteria for accessibility on the technical analysis, where success rates were found to be the lowest of all previous web studies published of a similar kind.
User feedback scores were likewise dismal, the lowest to date found by the Information Accessibility Monitor here at Discapnet. The highest score was achieved by the Principality of Asturias website at 58.89%.
We hope that the statements made by regional government officials regarding their commitment to complying with the criteria for web accessibility will soon be acted upon to enhance the services provided on their websites.
1. The Information Accessibility Observatory at Discapnet
In 2004, the Discapnet Project, co-financed by the ONCE Foundation of Spain and the European Regional Development Fund (ERDF), started up the Info-accessibility Observatory to generate and publicize information on website accessibility, by both analyzing specific sectors as well as comparing across sectors to monitor their development over time. The reports on the accessibility of university websites and the general State Administrations e-services are a result of that line of work.
The purpose of the reports by the Discapnet Info-accessibility Observatory is to inform and highlight not only the degree of compliance with current norms, but also point out the good features and main obstacles on the websites, including assessments from user feedback. It is hoped that a deeper understanding of the strengths and weaknesses identified by web experts and users alike will lead to a better understanding in webmasters and web designers of what constitutes web accessibility, what tools and services are available. Greater interaction, then, will lead to improving the ever-increasing usefulness of such sites.
The Observatory employs an innovative methodology designed by Fundosa Teleservicios. The methodology follows on the W3C/WAI guidelines for combining the technical analysis of accessibility with an assessment of the usability and accessibility based on feedback from the users own experiences.
Evaluation of the technical aspects takes the Web Content Accessibility Guidelines 1.0 from the W3C/WAI web as a framework and synthesizes them in a set of indicators applied to a sample of web pages by website. Verification is carried out by professionals running automatic and manual checks.
Assessment is done by a panel of users of varying functional abilities who undertake a set of tasks and then answer a questionnaire on their perception of each site. This procedure helps identify both the barriers and the aids in using each site, check the information architecture (i.e., how the content is organized, how to navigate around the site, perform searches, etc.) as well as determine how individual users interact with the websites.
The combination of both approaches provides information that is relevant, systematic, and qualified regarding accessibility in the sectors subject to this study. With it, insight is gained into how to correct and improve the Internet medium.
2. Selection of the sample
The present study on web accessibility carried out by the Discapnet Web Info-accessibility Observatory centered on the institutional websites of the 17 Regional Governments of Spain.
The websites contain a great number of pages each. Therefore, the scope of the study was tightened to center on the subject of employment as the main focus of the study sample.
For each of the 17 websites under study, five representative pages were selected for analysis in the following manner:
The institutional websites home page.
The page displayed by searching the word empleo (jobs) in the websites search engine.
The home page of whichever government department is in charge of job opportunities in that region.
A general information page on jobs (services, help desks, legislation, etc.).
The page for contact information for further requests, either within the Jobs section or, if none is available, for the website in general).
3. Aspects of accessibility to be analyzed on the pages
As in the previous studies carried out by Fundosa Teleservicios for the Discapnet Info-accessibility Observatory, the tests for verifying the degree of accessibility of the sites under study fall into two categories: a technical evaluation and a user feedback survey.
Details on the procedure for each are given below.
3.1 Technical evaluation of web accessibility
In order to evaluate the technical aspects of accessibility, twelve aspects were used which synthesize most of the Web Content Accessibility Guidelines on the W3C/WAI 1.0 website (WCAG 1.0) corresponding to levels A and AA. The experts at Fundosa Teleservicios, who led the study, consider the WCAG criteria able to provide a synthetic view closely matching the degree of accessibility of websites and web-based services. Included most are priority 1 aspects, though in some cases those of priority 2 were also used. The points of verification, itemized further on in the section on the analysis of the results, are as follows:
Validation of W3C technologies (priorities 1 and 2 in WCAG 1.0).
Frames (priorities 1 and 2 in WCAG 1.0).
Forms (priorities 1 and 2 in WCAG 1.0).
Text-only alternatives to multimedia elements (priority 1 in WCAG 1.0).
Headers (priority 2 in WCAG 1.0).
Units in Style Sheets (priorities 1 and 2 in WCAG 1.0).
Understandable links (priority 2 in WCAG 1.0).
Contrast (priority 2 for images in WCAG 1.0).
Semantic use of colors (priority 1 in WCAG 1.0).
Alignment of content in tables for layout (priority 2 in WCAG 1.0).
Data tables (priority 1 in WCAG 1.0).
Scripts (priority 1 in WCAG 1.0).
The technical analysis of accessibility was carried out on the sample web pages during the first two weeks of November 2005.
3.2 Assessing website accessibility based on user feedback
As WAI suggests and Fundosa Teleservicios firmly concurs, the results of the technical evaluation on accessibility should be complemented with a consideration of the users own experience. The purpose is to assess each web-based public services ability to be used by persons with differing limitations from a strongly practical point of view.
The standard way to record the users feedback is by means of a Users Test. For this study, a self-administered questionnaire format was chosen. This type of test consists of a series of tasks and questions answered by the users themselves after receiving prior instructions so that they may carry out the tasks on their own, without the presence of an outside observer or interviewer. Questions were asked regarding simple tasks that are commonplace in the websites visited.
The sample profile of the users, presented in Table 1, includes persons with different types and degrees of functional limitations, various kinds of technical aids, and differing degrees of technical mastery.
Table 1
Profile of users who undertook the assessment.
Functional impairmentTechnical aid employedInternet skillHighly reduced mobility in handsTrackball IntermediateDeafnessNone usedIntermediateBlindnessJAWS 5.0 screen readerAdvancedBlindnessJAWS 6.20 screen readerIntermediateSerious visual impairmentZoom Text screen magnifierIntermediateNo discernible impairmentNone usedAdvancedThe results were interpreted and tabulated by an accessibility expert from Fundosa Teleservicios. The tabulation compares and weighs the users assessments against the objective results of the tasks specific to each service in terms of efficiency, effectiveness, and satisfaction.
The user feedback surveys were carried out in the first half of November 2005.
To complement the self-administered user feedback surveys, the users also took part in a discussion group to share their experiences.
4. Analysis of the results
The scores were tabulated from the tests done on each of the points, and then analyzed one by one.
4.1 Analysis of the results of the technical evaluation of web accessibility
This section gives general remarks on the results obtained on each of the twelve technical criteria used in the technical evaluation of the web accessibility of the 85 pages reviewed from the 17 Regional Government websites under study.
It should be pointed out that the web pages of the services studied here undergo frequent changes. Therefore, the results presented here refer exclusively to their state during the testing period.
4.1.1 Validation of HTML code and CSS
Priorities 1 and 2 in WCAG 1,0
This section establishes that both the HTML code used in the web pages and the Style Sheet Code must be expressed correctly and validated by formal grammar, namely, in compliance with W3C guidelines for HTML and CSS2 .
Faulty code may cause pages to be displayed differently in different web browsers, since not all elements are supported by all browsers. According to W3C, correctly written HTML code ensures complete compatibility on any browser. Most HTML code errors arise from using attributes and elements incorrectly.
While five of the seventeen websites analyzed in the study received some positive scores, every single page on the other twelve sites contained faulty HTML and CSS code.
Of the 85 pages reviewed for HTML and CSS code validity, seven (8.24%) passed and 78 (91.76%) failed.
4.1.2 Frames
Priorities 1 and 2 in WCAG 1.0.
Frames are no longer needed to lay out a website, since other elements such as shared borders can be used instead. Nevertheless, if frames are used, they should be given meaningful names so that a user with a text-only browser can see that they contain and how they are related. Some alternatives to frames should also be provided for users whose browsers do not support frames.
The pages in the sample were checked for the existence of frames and whether or not they had a title or name that describes them. Any title or name found was also examined for its helpfulness to users.
Seven of the sites were found to use frames on at least some pages. The remaining ten sites made no use of frames for layout. A total of sixteen pages (18.22%) in this study used frames. Though few in number, it would be preferable for them to avoid using frames entirely and rely on other, less troublesome layout features instead.
It should be noted that the five pages studied from the Comunidad Valenciana all used frames but none did so correctly for accessibility. Navarra and La Rioja also used frames on 3 of the pages in the sample but none were used correctly, nor were they used correctly on the other four sites where frames were found (Asturias, Balearics, Canary Islands, and Castilla-La Mancha).
Of the 16 sample pages using frames, none (0%) made correct use of them.
4.1.3 Forms
Priorities 1 and 2 in WCAG 1.0.
Some users may have trouble dealing with forms because they are not sure what information to enter or choose in each field. That situation is most likely due to improper structuring of the form or an incompatibility with Javascript in some web browsers.
Problems with incorrect structure on forms occur when the control tags are not properly related. In some cases, they may have been omitted entirely, mistakenly being considered self-evident.
Further difficulty ensues if movement through the form with the tab key does not match up with the order of the fields in the form, or that the fields in lengthy forms are not grouped into orderly groups by concept or topic.
All the websites under study here were found to use forms. Forms were used on 57 of the 85 pages sampled (67%). Only 2 of the 57 used forms correctly: one on the Castilla-La Mancha website, and the other on the Basque Country website.
Of the 57 pages found using forms, 2 (3.51%) were designed correctly whereas the remaining 55 (96.49%) did not.
4.1.4 Images
Priority 1 in WCAG 1.0.
Images have become fundamental to web design. Images may act as a clickable link, provide complementary information to a text, or simply make the pages more pleasing to look at.
There are users who, for different reasons, are unable to see images: the blind, users of text-only browsers, those who stop downloads of pictures because their internet connection is slow, etc. For them, it is essential to be given an alternative to the images, especially for images that convey relevant information for using the page. For example, if no alternative text is available, both the screen readers used by the blind to access the net as well as text-only web browsers will only reveal the path of where the image is stored, which can become annoying for the user.
The following process was used to determine compliance with this point:
Check to see whether images on sample pages offer any textual alternative for users unable to see the images themselves.
Verify that any existing textual alternatives match the real needs of different user profiles.
Almost 96.5% of the pages in the sample (82 out of 85) used images. This massive use of images has become standard practice in web layout today. The Navarra website is noteworthy in this section for using images correctly on 4 of its 5 pages with images. Other websites with more successes than failures were Catalonia, the Basque Country, and Murcia (3 correct, 2 incorrect).
Of the 82 pages analyzed, 24 (29.27%) made correct use of tagging images with text alternative labels, and 58 (70.73%) did not.
4.1.5 Headers
Priority 2 in WCAG 1.0.
An essential feature to have on any website is its headers (also known as section titles) to show how information is structured on each web page. These headers must also be named to fit the level of depth appropriately. For example, a level one header
should not be followed by a level 3 header without good reason.
Some web designers do not respect the order of headers because they do not like the default font size when using a given level, unaware that font size can be changed through a style sheet.
Using headers to lay out a page is indeed helpful. For example, browsers such as Opera and screen readers such as Jaws allow the cursor to move from block to block across the page. This is especially useful for the blind and for people with impaired motor control.
The present analysis of headers, then, examined the sample pages to check for both the presence of headers as well as their correctness.
Headers were used on 72 pages (84.70% of the sample). The Madrid Regional Government website was the only one to score favorably on this point (on 5 pages); none of the other websites received a favorable score.
Of the 72 pages analyzed for this criterion, 5 (6.94%) complied with headers for accessibility whereas 67 (92.06) did not.
4.1.6 Cascading Style Sheets (CSS)
Priorities 1 and 2 in WCAG 1.0.
Some people with impaired vision may need to be able to change the text size in order to access a website. For users to be able to change the font size requires using relative units of either em or percentage (%). Style sheets, however, should only use relative units, not absolute units.
Furthermore, style sheets should not state obsolete elements which could make browsers display the page incorrectly and thus lead to a loss of the information they were meant to convey.
Some web browsers cannot interpret style sheets. Consequently, web designers should check to be sure that the page content can be displayed correctly even without style sheets.
All the pages (100%) sampled in this study made use of cascading style sheets (CSS) for page layout. The Madrid Regional Government website was the only one to do so in accordance with the guidelines mentioned above. Murcia (3 pages out of 5), Valencia and the Basque Country (2 out of 5), and the Balearics and Catalonia (1 out of 5) were the only other websites to use style sheets correctly.
The technical evaluation of cascading style sheets found on 85 pages was passed on 14 (16.47%) and failed on the remaining 71 (83.53%).
4.1.7 Accurate and meaningful links
Priority 2 in WCAG 1.0.
Links make up the most important component of a website. They are the means a user can navigate between pages and choose what contents s/he wishes to see.
From an accessibility point of view it is vital that text and/or image links be self-explanatory. In other words, the link should provide the user with a clear indication of what will be shown on the next page if s/he clicks on the link.
Having the right text or a helpful alternative description to an image in each link is essential because, for some users, it is the only guide they use to find the information they want. This is most frequent among users who navigate on small-screen devices, those who use text-to-speech screen readers, etc.
Therefore, special attention was given to evaluating links on the sample pages, for both their purpose as well as any additional information they may provide (such as telling the user what format the linked document is in).
All 85 pages in the sample (100%) used links and thus were included in the analysis. The highest scores went to the websites from Aragon (5 out of 5) and Murcia (4 out of 5). Only 4 websites (Andalusia, Cantabria, Castilla y Len, and La Rioja) failed this point on all of their pages.
Accessibility guidelines for links were correctly applied on 31 (36.47%) of the 85 sample pages; errors were found on 54 pages (63.53%).
4.1.8 Contrast
Priority 2, for images, in WCAG 1.0.
Some people may not be able to see colors correctly. Reading the text in such cases becomes difficult if not downright impossible. This problem can be remedied by changing the Style Sheet to a personalized sheet the user can apply to make reading easier. Yet some websites do not allow this, and some users do not know how to do it. Thus, the web designer must be sure to provide clear contrast between the text and the background.
As for the contrast between images and the background, this feature becomes particularly relevant, since the user cannot modify the images themselves. Good contrast must be provided by the web designer.
On 63 of the 85 pages reviewedmore than 74% of the samplecontrast was analyzed for images that conveyed relevant information and the background color. The Murcia Government website fared the best, with 2 of the 4 pages analyzed passing the contrast test. Other pages with positive scores were the Balearics, Cantabria, the Basque Country, La Rioja, and Navarra.
Of the 63 pages reviewed for contrast between images and background color, 7 (11.11%) passed the test; the remaining 56 (88.89%) failed the test.
4.1.9 Semantic use of colors
Priority 1 in WCAG 1.0.
It is essential for any information that is conveyed through the use of colors to be equally available to the blind and the color-blind, not to mention anyone using a black and white monitor. It is not that color should not be used: many people with other types of impairments find color a great help to navigating through a website. Nevertheless, colors should be used wisely (by means of context or markers, for example).
A web page may use colors semantically (e.g., using red to indicate negative numbers in an account) as long as meaningful complementary elements are also included (such as a negative sign in the previous example).
Only 4 pages of the 85 in the sample (4.7% of the total) made semantic use of colors to convey information. Andalusia, Asturias, Castilla y Len, and Catalonia each did so, and in each case it was correct.
Of the only 4 pages making semantic use of color, the technical evaluation was successful on all of them (100%).
4.1.10 Tables for layout
Priority 2 in WCAG 1.0.
At present, tables are being used both to lay out the contents of a page as well as to organize data into categories. For correct semantic use of tables, it is recommended that they be used only to categorize data; positioning for page layout can be done through CSS.
Leaving data tables aside for the moment (they are analyzed in the next section), we have reviewed tables used for layout. Using tables for layout can create havoc for screen readers (and thus for their users), which may not be able to render them correctly. This also holds true of text-only browsers.
Though the Web Accessibility Guidelines 1.0 at W3C do not restrict using tables for layout purposes, they do make it essential that table contents be properly aligned.
We found 66 pages in the sample (almost 78%) that used tables for laying out content. This feature, though not recommended by W3C for layout purposes, is still considered by designers as the most reliable way to display page contents on different browsers because of how they render the design in layers.
The scores on this aspect of accessibility were overwhelmingly positive, since only 3 pages (from the Balearics, Extremadura, and Navarra websites) used this feature incorrectly.
The alignment of content through tables received a passing score on 63 pages (95.45%) using this feature, whereas only 3 pages (4.55%) failed in their use.
4.1.11 Data tables
Priority 1 in WCAG 1.0.
For the blind or otherwise visually impaired, something even harder than reading data gathered in a table is knowing which heading or headings a piece of data falls under. To that end, it is vital that the tables have certain special features. For example, column and row headers should be properly marked as such; markers can be used to associate header cells with data cells in tables with two or more logical tiers of headers.
This feature should be implemented according to the W3C guidelines. Otherwise, a screen reader will be unable to give the user the information needed to make sense of the table.
Row and column headers should not be tagged by modifying the content (e.g., by using boldfaced text), but by tagging them with the right attribute so that browsers will render them correctly as headers.
In the sample for the study only 1 page (barely 1%) used data tables. The page was on the Catalonia website and made correct use of it following the W3C guidelines.
The one page using data tables in our sample (100%) did so according to W3C specifications.
4.1.12 Scripts
Priority 1 in WCAG 1.0.
Care must be taken so that there is no loss of functionality should the user for some reason not have scripts turned on in their browser.
Users of browsers that do not support scripts (e.g., Lynx) are unable to view their contents or activate the links.
Some search engines such as Google cannot follow links inserted in scripts to index the pages and thus cannot index the search results. It is essential, then, to have some content alternative if scripts and other programming objects are not turned on or supported in a web browser.
A total of 59 pages of the 85 making up the study sample (almost 70%) made uise of scripts in their page design. The Catalonia website is a good example of how scripts can be used without deterring accessibility (3 of the 4 pages using scripts did so correctly). Andalusia, Murcia, the Basque Country, Navarra, and Valencia are the other Regional websites where scripts were used correctly for accessibility. The only website found not to use scripts at all was that of the Canary Islands.
Of the 59 pages using scripts, 10 (16.95%) were scored favorably; the scores on the other 49 (83.05%) were unfavorable.
4.2 Ranking of technical accessibility scores, in % success
Table 2 below provides an overview of the Regional Government website scores on the technical accessibility evaluation, ranked from highest to lowest.
Table 2Ranking of Regional Government websites, by percent success rate of correct application of the criteria analyzed in the tasks on the technical evaluation of accessibility.
REGION% SUCCESSRegion of Murcia47.62 Madrid44.44 Catalonia37.50 The Basque Country37.50 The Canary Islands27.27 Andalusia26.09 Comunidad Foral de Navarra25.58 Castilla-La Mancha25.00Generalitat Valenciana23.81 Aragon22.73 Galicia19.35 Balearics17.07 Castilla y Len15.00 Principality of Asturias14.29 Extremadura13.51 Cantabria11.43 La Rioja7.89 Average mean score of all regions:24.48 We should point out that these compliance scores take into account only some of the aspects of accessibility and should in no way be considered absolute indications of the overall accessibility of the websites. Several aspects in WCAG 1.0 were not included in the study for reasons of scope yet are not without their relevance as well.
The website which performed the best on the accessibility tests was that from the Region of Murcia, low though its overall score may be (47.62%). More than 3 percentage points below is the Region of Madrid website (44.44%), at second place in the technical accessibility evaluation. Tied for third place at almost 7 points below Madrid are the Regions of Catalonia and the Basque country, at 37.50%.
At the bottom end, the lowest-ranking websites were from La Rioja (7.89%), Cantabria (11.43%), and Extremadura (13.51%).
Graph 1 displays the percentage of errors per criterion.
Tellingly, we found a success rate of less than 50% on 9 out of the 12 criteria used in the technical analysis.
The highest scoring criteria (data tables and semantic use of color) have a very low incidence (only 1 page and 4 pages, respectively), which may diminish the value of both criteria. Nevertheless, it should be noted that they fared little better on earlier studies.
Another high-scoring criterion was Table alignment, with a success rate of 95.5%. Since this aspect fared quite well on previous studies, its successful implementation here was not unexpected.
What did come as a surprise, though, was the poor showing of alternate text for images (29.3%) and understandable links (36.5%). In the latter, the links were deemed not understandable precisely because the images offered no alternative text. Being simple to resolve with the editing tools available today, the problem seems to be due more to a lack of awareness of its importance for certain kinds of users than a technical one. With luck, improved professional training and studies such as this one will raise awareness enough to overcome such obstacles.
Correct use of frames was the bottom-ranking criterion (0.0%). This layout technique has been outmoded by the many features provided by cascading style sheets; the use of frames is hardly justifiable today, and even less so at the cost of accessibility.
Also near the bottom of the list were forms (3.5%), whose use is indispensable when searching, contacting the site, or filling out an online application. The repercussions forms can have on the websites usability and the ability for users to interact with website personnel more than justify the need to make correct and accessible use of this feature.
Yet another surprisingly poor score was found for headers (6.9%). Headers make it faster and easier to navigate around a page, and are one of the most basic layout features in web design. Using styles to put text in bold is no substitute for a correctly used header. Again, the simplicity of how to avoid the problem leads us to think it is likely a problem of awareness of the consequences of not complying with web design standards.
Falling short of the 10% success rate is code validation (8.2%). Not following the established technical norms for encoding web pages may mean that the information on the page may not be available with all the different ways users access the web. Obsolete labels, carelessness when opening and closing them, and non-standard code are the main mistakes detected in this study. Some of the errors may be due to using content management tools that do not follow the standards or that fail to replace contents without leaving junk code behind such as labels without their opening and closing pairs.
Incorrect color contrast on images conveying relevant information is, at 11.11%, another of the most noticeable problems uncovered. Some 10% of the population has eyesight trouble with color contrasts. Finding good contrast between the text and the background is important, but it can be done by the user through strategies such as personalized style sheets to change background colors, highlighting foreground text, etc. Contrast is even more important for text within the images themselves, since the user cannot change them. This is especially true when no alternate text for the images is given. The choice of soft, pastel colors predominant in web design today may actually be harmful in that regard, and many web designers are still unaware of the criteria for color contrast algorithms recommended by W3C.
Cascading Style Sheets were used correctly on 16.5% of the pages reviewed. The presence of obsolete or incorrect elements, and the use of absolute instead of relative units, were the errors under examination. Web designers are advised to learn more about how to use style sheets: the correct placing and sizing fonts and structural elements in relative units would suffice to clear up most of the problems encountered. Seeing how a page looks without applying any style sheets is one test every web designer should perform before concluding his/her work.
Scripts, used more and more often in programming, scored low at 16.9% of the pages tested. It is not that scripts should not be used. Rather, we recommend that they be used correctly so as to ensure that the page renders correctly even when the scripts are not used. Some web browsers especially designed for impaired users are unable to interpret scripts, while other users prefer not to run scripts for fear of viruses.
4.3 Analysis of User Feedback Scores
For the feedback assessment of the Regional Government websites, six users each received a self-administered multiple choice test with instructions for its completion.
4.3.1 Instructions for user feedback
The instructions given to users to carry out the user feedback assessment of each of the 17 websites were as follows:
Surf around the site and find the indicated places.
Perform 5 tasks for each of the assessed services.
Jot down the answer to each task along with the time needed to carry it out.
Make note of any time you gave up a task due to accessibility issues.
Fill out a satisfaction survey of 10 multiple-choice questions (4 options each) and explain your reason(s) why for each.
The resultant scores were tabulated to aid in drawing comparable and measurable conclusions in percentages.
After all the tests were completed and handed in, the users took part in a focus group discussion to share and record their overall impressions on the accessibility and usability of the websites under study.
4.3.2 Successes, mistakes, and defeats during the tasks
This first step consists of determining the users total number of successes, mistakes, and defeats upon performing the requested tasks for each website in the sample.
Table 3 presents the results of the 6 users for each of the 5 tasks they were to carry out on the 17 websites making up the sample under study. There were thus 30 task scores for each website.
Of the 510 tasks carried out by the 6 users on the 17 websites in the sample, some 260 were finished successfully (50.98%), and 96 were done incorrectly (18.82). The number of defeats (when a user gave up before completion due to trouble with the accessibility or usability of the site) jumped to 154 (30.20), which brings the percentage of mistakes and defeats to the highest level ever witnessed on Observatory studies.
Table 3
Successes, mistakes, and defeats during user feedback assessment tasks, in absolute numbers.
WebsiteSuccessesMistakesDefeatsLa Rioja2307Castilla -La Mancha2046Extremadura2028Galicia2028Madrid1839Asturias17103Aragn1686Balearics16410the Basque Country1587Andalusia1479Canary Islands1497Cantabria1497Murcia14313Navarra14412Castilla y Len12711Valencia7716Catalonia6915Total:26096154As shown in Table 3, the Regional Government website that fared best in successes was La Rioja, at 23 of the 30 tasks completed. It is also the website with the fewest mistakenly completed tasks (0), though there were 7 defeated attempts as well. the Asturias site had the fewest defeats (3) but the highest incidence of mistakes (10). The fewest successes were recorded on the Catalonia site (6), followed closely by the Valencia site (7). Defeats were also top scorers on these site: Valencia at 16 and Catalonia at 15.
It should be pointed out again that the high number of errors and defeats found on the user feedback assessment stands out far beyond other results from earlier studies. A glance at the remarks made by users (shown in Table 6) reveals one problem again and again: the difficulty of changing languages, or the inability to perform searches in more than one of the official languages of the website. This may be one reason for the poor scores on the Catalonia and Valencia websites.
There is no apparent explanation for the poor showing overall other than because of unduly complicated layouts and general user-unfriendliness. All the tasks were designed to be able to be completed in fewer than 5 clicks; furthermore, the users all had enough experience of the medium (the web) and the tool (the browser) to perform the tasks.
4.3.3 Results of the User Feedback Survey
This section examines the results of each Regional Government website on the ad hoc user feedback survey filled out by each user in the study upon performing the requested tasks.
Each user was asked to answer 10 questions on specific aspects of their experience while performing the tasks. Each question focused on accessibility and user-friendliness, and was to be answered by choosing one of the four responses (on an increasing scale) provided. In addition, users were asked to write a remark with their reasoning for each. The questionnaire contained the following items:
Were you able to finish the tasks as requested?
Were you able to navigate around the site?
What is your overall impression of the site?
Did the correct labeling and alternate text for images help you understand the sites contents?
Was the text on the website helpful and clear enough fir you to perform your tasks?
What do you think of the websites overall organization?
Were the names of the links clear about where they led to?
Do you think the page titles were good indicators of the pages content?
Would you use this website because of the easy way it lets you access its information and services?
Does it seem like a good website to you?
The results from the survey were converted into percentages. Graph 2 presents the percentage scores of the set of 6 users who carried out the assessment. It should be remembered that 5 of the users had some form of impairment and one did not.
On the whole, users gave their highest satisfaction ratings to the Regional Government of Asturias website (58.89%). Close behind were Castilla-La Mancha (57.22%), Murcia and La Rioja (tied at 53.89%), Galicia (52.22%), Cantabria (51.67%) and the Basque Country (51.11%).
The remaining websites did not reach 50% satisfaction on the user feedback survey. The Catalonian website had the lowest satisfaction rating (26.67%), more than 7 percentage points below the Valencia site (33.89%). Both websites recorded the highest number of defeats for users attempting to perform their tasks. At a jump of some 14 percentage points above the lowest scorer were the websites from the Balearics (40.56%), Castilla y Leon (41.11%), and Navarra (41.67%).
Situated near the average score for all the websites (47.03%) were the portals from Aragon (45%), the Canary Islands (46.67%), Andalusia and Extremadura (both at 47.78%) and Madrid (49.44%).
Again, we point out that the scores from this user feedback study are significantly lower than those from earlier studies on state-run websites: Spanish universities scored between 43.75% and 71.25%, while Spains General Public Administration Services ranged from 26.11 to 79.44%. All these scores come just as the January 1st 2006 deadline approached for the Law on Services of the Information Society and E-business to make all public websites compliant with generally acknowledged criteria for accessibility.
It is noteworthy that user opinions coincided on the lowest-ranking website (Catalonia) in terms of user satisfaction. No such consensus exists for the top-ranking site (Asturias for users with some kind of impairment, Murcia for the user without any impairment). For the rest, there is relative consensus on some (such as for Castilla-La Mancha, Cantabria, and Valencia), while others such as the Basque Country, Galicia, and Navarra show greater variability among users in their ranking.
4.4 Combined Scores
As in our earlier studies on the accessibility of Spanish university websites and General State Administration websites, the scores from the user feedback assessment were noticeably higher than the scores on their technical evaluation. Even so, the differences between the two are lesser in degree in the present study than in prior ones.
Thus, while the top score on the technical evaluation was 47.62%, the highest degree of user satisfaction was 11 points higher, at 58.89%. This difference increases among the bottom-ranking sites: the lowest score on the technical evaluation was 7.89%, in contrast to 26.67% as the lowest on the user feedback assessment, a spread of almost 19 percentage points.
As mentioned earlier, it may be inferred that, despite whatever functional limitations they may have, users develop alternative skills to overcome theoretical barriers and attain their goal. In many cases, doing so requires greater effort and large amounts of patience and imagination to get through the pages and finish the task. One additional snag observed in this study was regarding how to change languages. Websites for Regional Governments where more than one official language exists are supposed to provide a channel in each language. In practice, however, the same contents are not always available in both languages. Thus, a user may begin navigating the site in one language and end up following links to pages in another language completely, with no other recourse. Even more distressing are the search engines that return results in only one language, and even that they do poorly.
Generally speaking, the results on both assessment techniques leave much to be desired, and were especially disappointing on the technical evaluation. The imminent legal deadline for accessibility at the time of the tests (Dec. 31, 2005) for applying the Fifth Additional Disposition of Law 34/2002 on Services of the Information Society and E-business raised our expectations that the degree of compliance for their accessibility would be notably higher.
In some cases, there is a curious discrepancy on a websites scores depending on which assessment technique was used. A case in point is the Catalonian website, which, though low, still came in third from the top on the technical analysis (37.50%) and yet took last place in the user feedback assessment (26.67%). Problems stemming from the use of the two co-official languages are likely at the root of the discrepancy.
For the La Rioja website, however, the situation is reversed. Its very low score on the technical analysis (7.89%) puts firmly it in last place, yet it ranked third from the top on the user feedback assessment (53.89%). It may be that the site affords certain characteristics of user-friendliness taken into account by the users.
There remains one final classification that gathers the scores attained on both assessment techniques used to evaluate the Regional Government websites in this study. The averages were determined for each service, presented in Graph 3. The purpose of this classification is to provide an overview at a glance of each websites relative accessibility and user-friendliness by combining their scores on technical criteria and user feedback. The resultant scores are not meant as absolute assessments of a websites degree of accessibility. Graph 3 reveals the following features:
The scores are, on the whole, quite disheartening. None of the websites under study may be said to meet even minimum standards of accessibility.
Only one of the websites analyzed obtained a combined score above 50% (Murcia, at 50.75%). Three sites had combined scores above 40%: Madrid (46.94%, the Basque Country (44.31%), and Castilla-La Mancha (41.11%).
Three websites in the sample did not even reach the 30%-mark on their combined scores: Valencia (28.85%), The Balearic Islands (28.81%), and Castilla y Leon (28.06%), making them the lowest scorers so far on any studies done by this Observatory on state-run websites.
5. Conclusions
The purpose of these studies undertaken by the Discapnet Observatory for Info-accessibility is not only to show the present state of accessibility on public websites. Rather, they also attempt to furnish information for their improvement to fit the needs of the users of their web-based services. To that end, this section offers the conclusions we find most pressing. Though the harsh reality of the objective criteria may be unflattering, what is meant is to provide guidelines for improvement in this sector.
In general, the websites of Spains Regional Governments are a long way from complying with the requisites of the Fifth Additional Disposition of Law 34/2002 on Services of the Information Society and E-business. Indeed, even the highest-scoring website (Murcia) on the technical evaluation failed to reach the 50% mark of compliance with the criteria of evaluation. The poor showing of all the websites is particularly disturbing given that the Regions had more than three years warning before the law was to come into effect on Dec. 31, 2005, to make their sites comply with the guidelines for accessibility. And yet, so much remains to be done. Some of these websites had been updated very recently before the testing; though we assume the new versions offer better accessibility than previous versions, they clearly do not qualify for a double-A rating (AA) from the Web Content Accessibility Guidelines 1.0 from W3C/WAI, the generally acknowledged technical point of reference.
As in previous studies, the scores from the user feedback assessment were more positive than the technical evaluation. In this study, however, we find a much smaller distance between the two. Users, especially those with some kind of functional impairment, found particular difficulty in the poor use some webs make of versions in different languages. This became even more acute when the websites search engine displays results in only one of the languages or performs happenstance searches in one of them. Users also complained about the bureaucratic jargon being hard to understand, more than on previous studies. The high number of defeats (more than 30%, and hitting 34.12% for impaired users) during the tasks done by the users is significant of the hardships users faced, well beyond the figures from previous studies.
Only three of the twelve criteria used in the technical analysis of the sample reached a percent rating above 50%. The semantic use of color and data tables both saw 100% success, though very few pages in the sample actually contained those features and thus may not be representative. Table alignment scored 95.5%, making it the most favorably ranked criterion.
The rest of the features in the evaluation scored far lower (all fell below the 36.5% success rate), which suggests that all of the websites in question are in serious need of revision. We are particularly disturbed by the lack of accessibility on forms (3.5% success rate), since forms are the users main nexus for interaction with the government administration via enquiries, complaints, applications, etc. Another notable failing is the alternate text for images rate of 29.3%, which makes it hard for users to understand the contents of the webpage and navigate the site if they cannot see the images or do not download them, whether to boost their speed or to cut down on connection costs. This is especially true when browsing on wireless devices such as cell phones and PDAsa trend on the rise.
We thus make the following recommendations to help make Regional Government websites more accessible:
Adjust the HTML and CSS code to meet W3C guidelines. Good code can be read any users web browser, even special devices used by persons with functional impairments.
Stop using frames for layout, or, if frames must be used, make them W3C compliant. Many people use devices that do not support frames to connect to the net, which could prove an insurmountable barrier otherwise.
Forms, which are the tool for interaction between user and website, can and must be implemented in an accessible manner. Otherwise, users are unable to make enquiries, file complaints, fill out online applications, search the sites contents, and generally interact, which is often the whole point of their visit.
Providing text alternatives for images is the most highly stressed recommendation. Yet it is not enough to give just any old text to get by. Currently available design tools provide many ways to add alternative text correctly. The quality of the text itself, though, depends on the standards of the web design and maintenance team and may be indicative of their overall level of training.
Due should be given to the correct use of headers, since headers help users understand particular web tools as well as clarify page contents in general.
For style sheets, relative units should be used. Relative units let users control how the page is displayed on screen. At the same time, be sure that the page is correctly rendered even without style sheets, which may not be supported in all browsers (though more and more do).
Links (and their hypertext structure) are basic features of the web. Particular attention should be paid to any images used to display the link. Alternative text, which should always be provided, should let the user know about where the link will lead to. Care should also be taken not to make links open new browser windows without previously informing the user. Persons who navigate the site without actually seeing the pages can get confused, if not totally lost, if they are unaware that a new window has been opened.
The contrast between content and background, especially where images are involved, should ensure that all users will be able to tell them apart. Some visual impairments (such as color-blindness) as well as the progressive deterioration of our eyesight over time make this criterion one not to be underestimated.
Data tables are good ways to convey a lot of information in little space. But people who cannot visualize the whole table at a glance may find their contents incomprehensible if they are not designed for accessibility. Web developers should heed this section if they do not want to leave out any user unable to see the table.
Programming objects such as scripts enhance several aspects of web design. However, it should be born in mind that not all web browser tools can read them. Hence, the best solution is always to provide an alternative for such users as well.
Web-based public services have already passed the legal deadline for adapting their contents to the criteria for accessibility as generally acknowledged. Indeed, though they have had more than three years to accomplish that task, it does not seem to have ranked high as a priority in the Regional Government administrations. Some last-minute effort was noticed and several Regional Government websites were updated in their last year. We assume that the accessibility of these remodeled sites meant an improvement over the earlier versions, but the stark reality of the data in this study reveals that they are still a far cry from reaching their goal.
These papers are available at: HYPERLINK "http://www.discapnet.es/Discapnet/Castellano/Observatorio_infoaccesibilidad/observatorio04.htm" http://www.discapnet.es/Discapnet/Castellano/Observatorio_infoaccesibilidad/observatorio04.htm
HYPERLINK "http://www.discapnet.es/Discapnet/Castellano/Observatorio_infoaccesibilidad/observatorio05.htm" http://www.discapnet.es/Discapnet/Castellano/Observatorio_infoaccesibilidad/observatorio05.htm
W3C/WAI: Web Accessibility Initiative of the World Wide Web Consortium. For further information, see HYPERLINK "http://www.w3.org/WAI" http://www.w3.org/WAI
Available in English ( HYPERLINK "http://www.w3.org/TR/WCAG10/" http://www.w3.org/TR/WCAG10/), on the W3C/WAI website, and in Spanish ( HYPERLINK "http://www.teleservicios.es/accesibilidad/recursos/documentos/index.html" http://www.teleservicios.es/accesibilidad/recursos/documentos/index.html) on the Fundosa Teleservicios website.
The studies are available on the following website: HYPERLINK "http://www.discapnet.es/Discapnet/Castellano/Observatorio_infoaccesibilidad/" http://www.discapnet.es/Discapnet/Castellano/Observatorio_infoaccesibilidad/
INCLUDEPICTURE "http://www.discapnet.es/inc/infoaccesibilidad/html/img/image001.jpg" \* MERGEFORMAT
Accessibility of Spains Regional Government Websites
PAGE 14
PAGE 1
EMBED Excel.Chart.8 \s
EMBED Excel.Chart.8 \s
EMBED Excel.Chart.8 \s
" = F N ` s ԽwԐhTE:E h` hb OJ QJ j h` hb OJ QJ U'h` hb 5CJ OJ QJ aJ mH sH h` h*4u OJ QJ mH sH 0h` hH 5B* CJD OJ QJ aJD mH ph)b sH h` h^ CJ OJ QJ mH sH h` h^ CJ OJ QJ h` hH OJ QJ mH sH -h` hH B* CJD OJ QJ aJD mH ph sH -h` hH B* CJD OJ QJ aJD mH ph)b sH (j h` hH OJ QJ UmH nH u
G H I J K L M N ` a $a$gd*4u gd^ $a$gdH gdH d : a b c d e f g h i j k l m n o p q r s j
W X
.!
.!
$a$ 5 6 ש镃mUmm<