Made by :
Accessibility of Spain’s Regional Government Websites
December 31, 2005 marked the legal deadline for all web-based public services to comply with “generally acknowledged” criteria for accessibility. Spain’s “Autonomous Communities” (Regional Governments) have thus had more than three years to adapt their sites to meet the new regulation as set forth in the Law of Services for the Information Society and E-Business passed in June 2002.
The present study attempts to show their status based on how they fared on a technical analysis of 12 criteria for accessibility and a user feedback survey of task performance on the websites of 17 Regional Government websites.
The results of the study reveal a disheartening view in which much remains to be done. Only one website (from the Region of Murcia) achieved a combined score of over 50%, squeezing by with 50.75%. Furthermore, none of the sites scored even 50% on the criteria for accessibility on the technical analysis, where success rates were found to be the lowest of all previous web studies published of a similar kind.
User feedback scores were likewise dismal, the lowest to date found by the Information Accessibility Monitor here at Discapnet. The highest score was achieved by the Principality of Asturias website at 58.89%.
We hope that the statements made by regional government officials regarding their commitment to complying with the criteria for web accessibility will soon be acted upon to enhance the services provided on their websites.
Table of Contents.
In 2004, the Discapnet Project, co-financed by the ONCE Foundation of Spain and the European Regional Development Fund (ERDF), started up the Info-accessibility Observatory to generate and publicize information on website accessibility, by both analyzing specific sectors as well as comparing across sectors to monitor their development over time. The reports on the accessibility of university websites and the general State Administration’s e-services are a result of that line of work.[1].
The purpose of the reports by the Discapnet Info-accessibility Observatory is to inform and highlight not only the degree of compliance with current norms, but also point out the good features and main obstacles on the websites, including assessments from user feedback. It is hoped that a deeper understanding of the strengths and weaknesses identified by web experts and users alike will lead to a better understanding in webmasters and web designers of what constitutes web accessibility, what tools and services are available. Greater interaction, then, will lead to improving the ever-increasing usefulness of such sites.
The Observatory employs an innovative methodology designed by Fundosa Teleservicios. The methodology follows on the W3C/WAI guidelines [2], for combining the technical analysis of accessibility with an assessment of the usability and accessibility based on feedback from the users’ own experiences:
The combination of both approaches provides information that is relevant, systematic, and qualified regarding accessibility in the sectors subject to this study. With it, insight is gained into how to correct and improve the Internet medium.
The present study on web accessibility carried out by the Discapnet Web Info-accessibility Observatory centered on the institutional websites of the 17 Regional Governments of Spain.
The websites contain a great number of pages each. Therefore, the scope of the study was tightened to center on the subject of employment as the main focus of the study sample.
For each of the 17 websites under study, five representative pages were selected for analysis in the following manner:
As in the previous studies carried out by Fundosa Teleservicios for the Discapnet Info-accessibility Observatory, the tests for verifying the degree of accessibility of the sites under study fall into two categories: a technical evaluation and a user feedback survey.
Details on the procedure for each are given below.
In order to evaluate the technical aspects of accessibility, twelve aspects were used which synthesize most of the Web Content Accessibility Guidelines on the W3C/WAI 1.0 website (WCAG 1.0) corresponding to levels A and AA. The experts at Fundosa Teleservicios, who led the study, consider the WCAG criteria able to provide a synthetic view closely matching the degree of accessibility of websites and web-based services. Included most are priority 1 aspects, though in some cases those of priority 2 were also used. The points of verification, itemized further on in the section on the analysis of the results, are as follows:
The technical analysis of accessibility was carried out on the sample web pages during the first two weeks of November 2005.
As WAI suggests and Fundosa Teleservicios firmly concurs, the results of the technical evaluation on accessibility should be complemented with a consideration of the user’s own experience. The purpose is to assess each web-based public service’s ability to be used by persons with differing limitations from a strongly practical point of view.
The standard way to record the user’s feedback is by means of a User’s Test. For this study, a self-administered questionnaire format was chosen. This type of test consists of a series of tasks and questions answered by the users themselves after receiving prior instructions so that they may carry out the tasks on their own, without the presence of an outside observer or interviewer. Questions were asked regarding simple tasks that are commonplace in the websites visited.
The sample profile of the users, presented in Table 1, includes persons with different types and degrees of functional limitations, various kinds of technical aids, and differing degrees of technical mastery.
| Functional impairment | Technical aid employed | Internet skill |
|---|---|---|
| Highly reduced mobility in hands | Trackball | Intermediate |
| Deafness | None used | Intermediate |
| Blindness | JAWS 5.0 screen reader | Advanced |
| Blindness | JAWS 6.20 screen reader | Intermediate |
| Serious visual impairment | Zoom Text screen magnifier | Intermediate |
| No discernible impairment | None used | Advanced |
The results were interpreted and tabulated by an accessibility expert from Fundosa Teleservicios. The tabulation compares and weighs the users’ assessments against the objective results of the tasks specific to each service in terms of efficiency, effectiveness, and satisfaction.
The user feedback surveys were carried out in the first half of November 2005.
To complement the self-administered user feedback surveys, the users also took part in a discussion group to share their experiences.
The scores were tabulated from the tests done on each of the points, and then analyzed one by one.
This section gives general remarks on the results obtained on each of the twelve technical criteria used in the technical evaluation of the web accessibility of the 85 pages reviewed from the 17 Regional Government websites under study.
It should be pointed out that the web pages of the services studied here undergo frequent changes. Therefore, the results presented here refer exclusively to their state during the testing period.
Priorities 1 and 2 in WCAG 1,0.
This section establishes that both the HTML code used in the web pages and the Style Sheet Code must be expressed correctly and validated by formal grammar, namely, in compliance with W3C guidelines for HTML and CSS2.
Faulty code may cause pages to be displayed differently in different web browsers, since not all elements are supported by all browsers. According to W3C, correctly written HTML code ensures complete compatibility on any browser. Most HTML code errors arise from using attributes and elements incorrectly.
While five of the seventeen websites analyzed in the study received some positive scores, every single page on the other twelve sites contained faulty HTML and CSS code.
Priorities 1 and 2 in WCAG 1.0.
Frames are no longer needed to lay out a website, since other elements such as shared borders can be used instead. Nevertheless, if frames are used, they should be given meaningful names so that a user with a text-only browser can see that they contain and how they are related. Some alternatives to frames should also be provided for users whose browsers do not support frames.
The pages in the sample were checked for the existence of frames and whether or not they had a title or name that describes them. Any title or name found was also examined for its helpfulness to users.
Seven of the sites were found to use frames on at least some pages. The remaining ten sites made no use of frames for layout. A total of sixteen pages (18.22%) in this study used frames. Though few in number, it would be preferable for them to avoid using frames entirely and rely on other, less troublesome layout features instead.
It should be noted that the five pages studied from the Comunidad Valenciana all used frames but none did so correctly for accessibility. Navarra and La Rioja also used frames on 3 of the pages in the sample but none were used correctly, nor were they used correctly on the other four sites where frames were found (Asturias, Balearics, Canary Islands, and Castilla-La Mancha).
Priorities 1 and 2 in WCAG 1.0.
Some users may have trouble dealing with forms because they are not sure what information to enter or choose in each field. That situation is most likely due to improper structuring of the form or an incompatibility with Javascript in some web browsers.
Problems with incorrect structure on forms occur when the control tags are not properly related. In some cases, they may have been omitted entirely, mistakenly being considered self-evident.
Further difficulty ensues if movement through the form with the tab key does not match up with the order of the fields in the form, or that the fields in lengthy forms are not grouped into orderly groups by concept or topic.
All the websites under study here were found to use forms. Forms were used on 57 of the 85 pages sampled (67%). Only 2 of the 57 used forms correctly: one on the Castilla-La Mancha website, and the other on the Basque Country website.
Priority 1 in WCAG 1.0.
Images have become fundamental to web design. Images may act as a clickable link, provide complementary information to a text, or simply make the pages more pleasing to look at.
There are users who, for different reasons, are unable to see images: the blind, users of text-only browsers, those who stop downloads of pictures because their internet connection is slow, etc. For them, it is essential to be given an alternative to the images, especially for images that convey relevant information for using the page. For example, if no alternative text is available, both the screen readers used by the blind to access the net as well as text-only web browsers will only reveal the path of where the image is stored, which can become annoying for the user.
The following process was used to determine compliance with this point:
Almost 96.5% of the pages in the sample (82 out of 85) used images. This massive use of images has become standard practice in web layout today. The Navarra website is noteworthy in this section for using images correctly on 4 of its 5 pages with images. Other websites with more successes than failures were Catalonia, the Basque Country, and Murcia (3 correct, 2 incorrect).
Priority 2 in WCAG 1.0.
An essential feature to have on any website is its headers (also known as “section titles”) to show how information is structured on each web page. These headers must also be named to fit the level of depth appropriately. For example, a level one header <h1> should not be followed by a level 3 header <h3> without good reason.
Some web designers do not respect the order of headers because they do not like the default font size when using a given level, unaware that font size can be changed through a style sheet.
Using headers to lay out a page is indeed helpful. For example, browsers such as Opera and screen readers such as Jaws allow the cursor to move from block to block across the page. This is especially useful for the blind and for people with impaired motor control.
The present analysis of headers, then, examined the sample pages to check for both the presence of headers as well as their correctness.
Headers were used on 72 pages (84.70% of the sample). The Madrid Regional Government website was the only one to score favorably on this point (on 5 pages); none of the other websites received a favorable score.
Priorities 1 and 2 in WCAG 1.0.
Some people with impaired vision may need to be able to change the text size in order to access a website. For users to be able to change the font size requires using relative units of either em or percentage (%). Style sheets, however, should only use relative units, not absolute units.
Furthermore, style sheets should not state obsolete elements which could make browsers display the page incorrectly and thus lead to a loss of the information they were meant to convey.
Some web browsers cannot interpret style sheets. Consequently, web designers should check to be sure that the page content can be displayed correctly even without style sheets.
All the pages (100%) sampled in this study made use of cascading style sheets (CSS) for page layout. The Madrid Regional Government website was the only one to do so in accordance with the guidelines mentioned above. Murcia (3 pages out of 5), Valencia and the Basque Country (2 out of 5), and the Balearics and Catalonia (1 out of 5) were the only other websites to use style sheets correctly.
Priority 2 in WCAG 1.0.
Links make up the most important component of a website. They are the means a user can navigate between pages and choose what contents s/he wishes to see.
From an accessibility point of view it is vital that text and/or image links be self-explanatory. In other words, the link should provide the user with a clear indication of what will be shown on the next page if s/he clicks on the link.
Having the right text or a helpful alternative description to an image in each link is essential because, for some users, it is the only guide they use to find the information they want. This is most frequent among users who navigate on small-screen devices, those who use text-to-speech screen readers, etc.
Therefore, special attention was given to evaluating links on the sample pages, for both their purpose as well as any additional information they may provide (such as telling the user what format the linked document is in).
All 85 pages in the sample (100%) used links and thus were included in the analysis. The highest scores went to the websites from Aragon (5 out of 5) and Murcia (4 out of 5). Only 4 websites (Andalusia, Cantabria, Castilla y León, and La Rioja) failed this point on all of their pages.
Priority 2, for images, in WCAG 1.0.
Some people may not be able to see colors correctly. Reading the text in such cases becomes difficult if not downright impossible. This problem can be remedied by changing the Style Sheet to a personalized sheet the user can apply to make reading easier. Yet some websites do not allow this, and some users do not know how to do it. Thus, the web designer must be sure to provide clear contrast between the text and the background.
As for the contrast between images and the background, this feature becomes particularly relevant, since the user cannot modify the images themselves. Good contrast must be provided by the web designer.
On 63 of the 85 pages reviewed—more than 74% of the sample—contrast was analyzed for images that conveyed relevant information and the background color. The Murcia Government website fared the best, with 2 of the 4 pages analyzed passing the contrast test. Other pages with positive scores were the Balearics, Cantabria, the Basque Country, La Rioja, and Navarra.
Priority 1 in WCAG 1.0.
It is essential for any information that is conveyed through the use of colors to be equally available to the blind and the color-blind, not to mention anyone using a black and white monitor. It is not that color should not be used: many people with other types of impairments find color a great help to navigating through a website. Nevertheless, colors should be used wisely (by means of context or markers, for example).
A web page may use colors semantically (e.g., using red to indicate negative numbers in an account) as long as meaningful complementary elements are also included (such as a negative sign in the previous example).
Only 4 pages of the 85 in the sample (4.7% of the total) made semantic use of colors to convey information. Andalusia, Asturias, Castilla y León, and Catalonia each did so, and in each case it was correct.
Priority 2 in WCAG 1.0.
At present, tables are being used both to lay out the contents of a page as well as to organize data into categories. For correct semantic use of tables, it is recommended that they be used only to categorize data; positioning for page layout can be done through CSS.
Leaving data tables aside for the moment (they are analyzed in the next section), we have reviewed tables used for layout. Using tables for layout can create havoc for screen readers (and thus for their users), which may not be able to render them correctly. This also holds true of text-only browsers.
Though the Web Accessibility Guidelines 1.0 at W3C do not restrict using tables for layout purposes, they do make it essential that table contents be properly aligned.
We found 66 pages in the sample (almost 78%) that used tables for laying out content. This feature, though not recommended by W3C for layout purposes, is still considered by designers as the most reliable way to display page contents on different browsers because of how they render the design in layers.
The scores on this aspect of accessibility were overwhelmingly positive, since only 3 pages (from the Balearics, Extremadura, and Navarra websites) used this feature incorrectly.
Priority 1 in WCAG 1.0.
For the blind or otherwise visually impaired, something even harder than reading data gathered in a table is knowing which heading or headings a piece of data falls under. To that end, it is vital that the tables have certain special features. For example, column and row headers should be properly marked as such; markers can be used to associate header cells with data cells in tables with two or more logical tiers of headers.
This feature should be implemented according to the W3C guidelines. Otherwise, a screen reader will be unable to give the user the information needed to make sense of the table.
Row and column headers should not be tagged by modifying the content (e.g., by using boldfaced text), but by tagging them with the right attribute so that browsers will render them correctly as headers.
In the sample for the study only 1 page (barely 1%) used data tables. The page was on the Catalonia website and made correct use of it following the W3C guidelines.
Priority 1 in WCAG 1.0.
Care must be taken so that there is no loss of functionality should the user for some reason not have scripts turned on in their browser.
Users of browsers that do not support scripts (e.g., Lynx) are unable to view their contents or activate the links.
Some search engines such as Google cannot follow links inserted in scripts to index the pages and thus cannot index the search results. It is essential, then, to have some content alternative if scripts and other programming objects are not turned on or supported in a web browser.
A total of 59 pages of the 85 making up the study sample (almost 70%) made uise of scripts in their page design. The Catalonia website is a good example of how scripts can be used without deterring accessibility (3 of the 4 pages using scripts did so correctly). Andalusia, Murcia, the Basque Country, Navarra, and Valencia are the other Regional websites where scripts were used correctly for accessibility. The only website found not to use scripts at all was that of the Canary Islands.
Table 2 below provides an overview of the Regional Government website scores on the technical accessibility evaluation, ranked from highest to lowest.
| REGIONAL GOVERNMENT | % SUCCESS |
|---|---|
| Region of Murcia | 47,62 |
| Madrid | 44,44 |
| Catalonia | 37,50 |
| The Basque Country | 37,50 |
| The Canary Islands | 27,27 |
| Andalusia | 26,09 |
| Comunidad Foral de Navarra | 25,58 |
| Castilla-La Mancha | 25,00 |
| Generalitat Valenciana | 23,81 |
| Aragón | 22,73 |
| Galicia | 19,35 |
| Balearics | 17,07 |
| Castilla y León | 15,00 |
| Principality of Asturias | 14,29 |
| Extremadura | 13,51 |
| Cantabria | 11,43 |
| La Rioja | 7,89 |
| Average mean score of all regions: | 24,48 |
We should point out that these compliance scores take into account only some of the aspects of accessibility and should in no way be considered absolute indications of the overall accessibility of the websites. Several aspects in WCAG 1.0 were not included in the study for reasons of scope yet are not without their relevance as well.
The website which performed the best on the accessibility tests was that from the Region of Murcia, low though its overall score may be (47.62%). More than 3 percentage points below is the Region of Madrid website (44.44%), at second place in the technical accessibility evaluation. Tied for third place at almost 7 points below Madrid are the Regions of Catalonia and the Basque country, at 37.50%.
At the bottom end, the lowest-ranking websites were from La Rioja (7.89%), Cantabria (11.43%), and Extremadura (13.51%).
Graph 1 displays the percentage of errors per criterion.
[D]
Tellingly, we found a success rate of less than 50% on 9 out of the 12 criteria used in the technical analysis.
The highest scoring criteria (data tables and semantic use of color) have a very low incidence (only 1 page and 4 pages, respectively), which may diminish the value of both criteria. Nevertheless, it should be noted that they fared little better on earlier studies.
Another high-scoring criterion was “Table alignment,” with a success rate of 95.5%. Since this aspect fared quite well on previous studies, its successful implementation here was not unexpected.
What did come as a surprise, though, was the poor showing of “alternate text for images” (29.3%) and “understandable links” (36.5%). In the latter, the links were deemed not understandable precisely because the images offered no alternative text. Being simple to resolve with the editing tools available today, the problem seems to be due more to a lack of awareness of its importance for certain kinds of users than a technical one. With luck, improved professional training and studies such as this one will raise awareness enough to overcome such obstacles.
Correct use of frames was the bottom-ranking criterion (0.0%). This layout technique has been outmoded by the many features provided by cascading style sheets; the use of frames is hardly justifiable today, and even less so at the cost of accessibility.
Also near the bottom of the list were forms (3.5%), whose use is indispensable when searching, contacting the site, or filling out an online application. The repercussions forms can have on the website’s usability and the ability for users to interact with website personnel more than justify the need to make correct and accessible use of this feature.
Yet another surprisingly poor score was found for headers (6.9%). Headers make it faster and easier to navigate around a page, and are one of the most basic layout features in web design. Using styles to put text in bold is no substitute for a correctly used header. Again, the simplicity of how to avoid the problem leads us to think it is likely a problem of awareness of the consequences of not complying with web design standards.
Falling short of the 10% success rate is “code validation” (8.2%). Not following the established technical norms for encoding web pages may mean that the information on the page may not be available with all the different ways users access the web. Obsolete labels, carelessness when opening and closing them, and non-standard code are the main mistakes detected in this study. Some of the errors may be due to using content management tools that do not follow the standards or that fail to replace contents without leaving “junk code” behind such as labels without their opening and closing pairs.
Incorrect color contrast on images conveying relevant information is, at 11.11%, another of the most noticeable problems uncovered. Some 10% of the population has eyesight trouble with color contrasts. Finding good contrast between the text and the background is important, but it can be done by the user through strategies such as personalized style sheets to change background colors, highlighting foreground text, etc. Contrast is even more important for text within the images themselves, since the user cannot change them. This is especially true when no alternate text for the images is given. The choice of soft, pastel colors predominant in web design today may actually be harmful in that regard, and many web designers are still unaware of the criteria for color contrast algorithms recommended by W3C.
Cascading Style Sheets were used correctly on 16.5% of the pages reviewed. The presence of obsolete or incorrect elements, and the use of absolute instead of relative units, were the errors under examination. Web designers are advised to learn more about how to use style sheets: the correct placing and sizing fonts and structural elements in relative units would suffice to clear up most of the problems encountered. Seeing how a page looks without applying any style sheets is one test every web designer should perform before concluding his/her work.
Scripts, used more and more often in programming, scored low at 16.9% of the pages tested. It is not that scripts should not be used. Rather, we recommend that they be used correctly so as to ensure that the page renders correctly even when the scripts are not used. Some web browsers especially designed for impaired users are unable to interpret scripts, while other users prefer not to run scripts for fear of viruses.
For the feedback assessment of the Regional Government websites, six users each received a self-administered multiple choice test with instructions for its completion.
The instructions given to users to carry out the user feedback assessment of each of the 17 websites were as follows:
The resultant scores were tabulated to aid in drawing comparable and measurable conclusions in percentages.
After all the tests were completed and handed in, the users took part in a focus group discussion to share and record their overall impressions on the accessibility and usability of the websites under study.
This first step consists of determining the users’ total number of successes, mistakes, and defeats upon performing the requested tasks for each website in the sample.
Table 3 presents the results of the 6 users for each of the 5 tasks they were to carry out on the 17 websites making up the sample under study. There were thus 30 task scores for each website.
Of the 510 tasks carried out by the 6 users on the 17 websites in the sample, some 260 were finished successfully (50.98%), and 96 were done incorrectly (18.82). The number of defeats (when a user gave up before completion due to trouble with the accessibility or usability of the site) jumped to 154 (30.20), which brings the percentage of mistakes and defeats to the highest level ever witnessed on Observatory studies.
| Website | Successes | Mistakes | Defeats |
|---|---|---|---|
| La Rioja | 23 | 0 | 7 |
| Castilla-La Mancha | 20 | 4 | 6 |
| Extremadura | 20 | 2 | 8 |
| Galicia | 20 | 2 | 8 |
| Madrid | 18 | 3 | 9 |
| Asturias | 17 | 10 | 3 |
| Aragón | 16 | 8 | 6 |
| Balearics | 16 | 4 | 10 |
| The Basque Country | 15 | 8 | 7 |
| Andalusia | 14 | 7 | 9 |
| Canary Islands | 14 | 9 | 7 |
| Cantabria | 14 | 9 | 7 |
| Murcia | 14 | 3 | 13 |
| Navarra | 14 | 4 | 12 |
| Castilla y León | 12 | 7 | 11 |
| Valencia | 7 | 7 | 16 |
| Cataluña | 6 | 9 | 15 |
| Total: | 260 | 96 | 154 |
As shown in Table 3, the Regional Government website that fared best in successes was La Rioja, at 23 of the 30 tasks completed. It is also the website with the fewest mistakenly completed tasks (0), though there were 7 defeated attempts as well. the Asturias site had the fewest defeats (3) but the highest incidence of mistakes (10). The fewest successes were recorded on the Catalonia site (6), followed closely by the Valencia site (7). Defeats were also top scorers on these site: Valencia at 16 and Catalonia at 15.
It should be pointed out again that the high number of errors and defeats found on the user feedback assessment stands out far beyond other results from earlier studies. A glance at the remarks made by users (shown in Table 6) reveals one problem again and again: the difficulty of changing languages, or the inability to perform searches in more than one of the official languages of the website. This may be one reason for the poor scores on the Catalonia and Valencia websites.
There is no apparent explanation for the poor showing overall other than because of unduly complicated layouts and general user-unfriendliness. All the tasks were designed to be able to be completed in fewer than 5 clicks; furthermore, the users all had enough experience of the medium (the web) and the tool (the browser) to perform the tasks.
This section examines the results of each Regional Government website on the ad hoc user feedback survey filled out by each user in the study upon performing the requested tasks.
Each user was asked to answer 10 questions on specific aspects of their experience while performing the tasks. Each question focused on accessibility and user-friendliness, and was to be answered by choosing one of the four responses (on an increasing scale) provided. In addition, users were asked to write a remark with their reasoning for each. The questionnaire contained the following items:
The results from the survey were converted into percentages. Graph 2 presents the percentage scores of the set of 6 users who carried out the assessment. It should be remembered that 5 of the users had some form of impairment and one did not.
[D]
On the whole, users gave their highest satisfaction ratings to the Regional Government of Asturias website (58.89%). Close behind were Castilla-La Mancha (57.22%), Murcia and La Rioja (tied at 53.89%), Galicia (52.22%), Cantabria (51.67%) and the Basque Country (51.11%).
The remaining websites did not reach 50% satisfaction on the user feedback survey. The Catalonian website had the lowest satisfaction rating (26.67%), more than 7 percentage points below the Valencia site (33.89%). Both websites recorded the highest number of defeats for users attempting to perform their tasks. At a jump of some 14 percentage points above the lowest scorer were the websites from the Balearics (40.56%), Castilla y Leon (41.11%), and Navarra (41.67%).
Situated near the average score for all the websites (47.03%) were the portals from Aragon (45%), the Canary Islands (46.67%), Andalusia and Extremadura (both at 47.78%) and Madrid (49.44%).
Again, we point out that the scores from this user feedback study are significantly lower than those from earlier studies on state-run websites: Spanish universities scored between 43.75% and 71.25%, while Spain’s General Public Administration Services ranged from 26.11 to 79.44%. All these scores come just as the January 1st 2006 deadline approached for the Law on Services of the Information Society and E-business to make all public websites compliant with “generally acknowledged” criteria for accessibility.
It is noteworthy that user opinions coincided on the lowest-ranking website (Catalonia) in terms of user satisfaction. No such consensus exists for the top-ranking site (Asturias for users with some kind of impairment, Murcia for the user without any impairment). For the rest, there is relative consensus on some (such as for Castilla-La Mancha, Cantabria, and Valencia), while others such as the Basque Country, Galicia, and Navarra show greater variability among users in their ranking.
As in our earlier studies [4] on the accessibility of Spanish university websites and General State Administration websites, the scores from the user feedback assessment were noticeably higher than the scores on their technical evaluation. Even so, the differences between the two are lesser in degree in the present study than in prior ones.
Thus, while the top score on the technical evaluation was 47.62%, the highest degree of user satisfaction was 11 points higher, at 58.89%. This difference increases among the bottom-ranking sites: the lowest score on the technical evaluation was 7.89%, in contrast to 26.67% as the lowest on the user feedback assessment, a spread of almost 19 percentage points.
As mentioned earlier, it may be inferred that, despite whatever functional limitations they may have, users develop alternative skills to overcome theoretical barriers and attain their goal. In many cases, doing so requires greater effort and large amounts of patience and imagination to get through the pages and finish the task. One additional snag observed in this study was regarding how to change languages. Websites for Regional Governments where more than one official language exists are supposed to provide a channel in each language. In practice, however, the same contents are not always available in both languages. Thus, a user may begin navigating the site in one language and end up following links to pages in another language completely, with no other recourse. Even more distressing are the search engines that return results in only one language, and even that they do poorly.
Generally speaking, the results on both assessment techniques leave much to be desired, and were especially disappointing on the technical evaluation. The imminent legal deadline for accessibility at the time of the tests (Dec. 31, 2005) for applying the Fifth Additional Disposition of Law 34/2002 on Services of the Information Society and E-business raised our expectations that the degree of compliance for their accessibility would be notably higher.
In some cases, there is a curious discrepancy on a website’s scores depending on which assessment technique was used. A case in point is the Catalonian website, which, though low, still came in third from the top on the technical analysis (37.50%) and yet took last place in the user feedback assessment (26.67%). Problems stemming from the use of the two co-official languages are likely at the root of the discrepancy.
For the La Rioja website, however, the situation is reversed. Its very low score on the technical analysis (7.89%) puts firmly it in last place, yet it ranked third from the top on the user feedback assessment (53.89%). It may be that the site affords certain characteristics of user-friendliness taken into account by the users.
[D]
There remains one final classification that gathers the scores attained on both assessment techniques used to evaluate the Regional Government websites in this study. The averages were determined for each service, presented in Graph 3. The purpose of this classification is to provide an overview at a glance of each website’s relative accessibility and user-friendliness by combining their scores on technical criteria and user feedback. The resultant scores are not meant as absolute assessments of a website’s degree of accessibility. Graph 3 reveals the following features:
The purpose of these studies undertaken by the Discapnet Observatory for Info-accessibility is not only to show the present state of accessibility on public websites. Rather, they also attempt to furnish information for their improvement to fit the needs of the users of their web-based services. To that end, this section offers the conclusions we find most pressing. Though the harsh reality of the objective criteria may be unflattering, what is meant is to provide guidelines for improvement in this sector.
We thus make the following recommendations to help make Regional Government websites more accessible:
Web-based public services have already passed the legal deadline for adapting their contents to the criteria for accessibility as “generally acknowledged.” Indeed, though they have had more than three years to accomplish that task, it does not seem to have ranked high as a priority in the Regional Government administrations. Some last-minute effort was noticed and several Regional Government websites were updated in their last year. We assume that the accessibility of these remodeled sites meant an improvement over the earlier versions, but the stark reality of the data in this study reveals that they are still a far cry from reaching their goal.
Note: All the external URL´S are opened in a new window.
[1] These papers are available at: http://www.discapnet.es/Discapnet/Castellano/Observatorio_infoaccesibilidad/observatorio04.htm
http://www.discapnet.es/Discapnet/Castellano/Observatorio_infoaccesibilidad/observatorio05.htm
[2] W3C/WAI: Web Accessibility Initiative of the World Wide Web Consortium. For further information, see: http://www.w3.org/WAI
[3] Available in English (http://www.w3.org/TR/WCAG10/), on the W3C/WAI website, and in Spanish (http://www.teleservicios.es/accesibilidad/recursos/documentos/index.html) on the Fundosa Teleservicios website.
[4] The studies are available on the following website: http://www.discapnet.es/Discapnet/Castellano/Observatorio_infoaccesibilidad/