Evaluation Criteria

Evaluation Criteria

The evaluation criteria used by e-Gov Watch represent the range of best e-government practices and to allow comparison among websites. The criteria are tailored to the responsibilities of individual agencies. The criteria represent targets for complete user satisfaction rather than the minimum standards necessary.

Each site is evaluated by a process of web page sampling at a depth relevant to each criterion. A rating between 0 and 10 is set to reflect the quality found in the pages sampled. Comments are noted, often with links to web page examples from the site concerned. Each of the criteria is also weighted from 1 to 10 to emphasise its relative importance and relevance. The sum of the rating and weightings for all the criteria produce a 0 to 100 total score for the criteria area. Any criteria deemed inapplicable to the website under assessment are removed from calculation.

This extract shows the approach in action:

Sample Client One
RefCriterionWeight excl NAsRating (out of 10)Wghtd RatingComments
6.1 Any publicly available reports the agency is required to produce by statute are provided 5 10 50  
6.2 Consultation documents are provided and labelled 'Currently Consulting on..." 7 6 42 Consultations docs are provided but labelled "view consultations" only and also are linked to general MoF consultations
6.3 Press notices from the agency and links to press notices from the Minister where they set the context for a specific release of information are provided 7 5 35 Link to general Ministry news only - no unique news yet
  Totals 301   2172  
  Total Score Possible (excluding Not Applicables) 3010      
  Overall Rating for Required Government Content     72.2  

The criteria are organised into ten criteria areas and are described as follows:

Required Government Content
These 14 criteria are drawn entirely from the external requirements of the NZ Usability Standard 1.2. The criteria assess the minimum content requirements that users expect a government site to include, often for reasons of identity, accountability or purpose. Common website features are emphasised, as are site policies that each site should have as standard.

These 36 criteria test the breadth of access provided to all website users, particularly those with impaired sight and mobility. The criteria are entirely from the NZ Accessibility Standard 1.0, which combined with Required Government Content give a Web Standards compliance level. Government has a special obligation to provide information and services to all citizens and online accessibility is a key to meeting this obligation.

Site Usability
This set of 142 criteria assesses whether users will find a website easy to use irrespective of their experience with that site. The site should be easy to learn, navigate and allow users to achieve goals and objectives. 'Don't make me think' is an important usability principle; if fully adopted in government sites this would enable users to find content with minimum difficulty. These criteria were collated from leading usability experts and selected for their application to government sites.

Information Delivery
This area checks how well static information is delivered online by the organisation. Information areas expected to be available online are selected from the organisation's deliverables as expressed in Statements of Intent, Output Statements, Business Plans or from the website concerned. The relative importance of each information area is assessed, along with 13 information delivery quality criteria for each of the information areas listed. No coverage of an expected listed area is rated "0".

e-Services Delivery
This area checks the delivery by the organisation of dynamic, two-way interactive services or transactions. e-Services expected to be available online are selected from the organisation's deliverables as expressed in Statements of Intent, Output Statements, Business Plans, comparable organisations internationally or from the website concerned. The relative importance of each e-service is assessed along with 16 e-services delivery quality criteria for each of the e-services listed. No coverage of an expected e-service is rated "0".

Search Readiness
This 12 criteria area checks the findability of the site overall especially via the increasingly important search engines. Elements of search engine optimisation that can be tested at a point in time are included as well as cross-linking from other relevant websites.

This 19 criteria area determines the level to which website content is made relevant to individual users. The growth of content on many websites increases the risk of making users wade through too much irrelevant material to get what they want. Initial instances of personalisation are subscriptions and notifications, then navigation sorted by user group, toward the pinnacle of one-to-one delivery (typically a one-stop user account). Ultimately, users should be able to describe their online needs and have content delivered how, when and in what style and format they prefer.

Collaboration & Integration
Website users like to achieve their online objectives irrespective of where the useful information is located. Links can often be made to relevant and useful content on the websites of other organisations. Users also like to be engaged and to collaborate on issues of importance to them. This criteria area combines these related desirable qualities and assesses how well a website enables co-creation at a sector scale.

This criteria measures the opportunity for users to offer feedback comments on the delivery of information, services or the online experience. Users should be able to get the satisfaction of responses to legitimate concerns.

Separate from the general feedback criteria, e-Consultation assesses how well the website supports the organisation's consultation on new policies and plans. Government websites can expand from supporting traditional consultation processes through use of the unique capabilities of the Internet and open up participation in decision-making. As a cost-effective channel for stakeholder involvement, e-consultation improves the quality of decision-making and buyin.

The criteria are also weighted by percentages to reflect their relevance to the type of host organisation concerned, as follows:

 Assessment Criteria AreasOperational Organisation WeightingsMixed Organisation WeightingsPolicy Organisation Weightings
1 Required Government Content 2% 3% 3%
2 Accessibility 6% 7% 7%
3 Site Usability 12% 12% 12%
4 Information Delivery 16% 34% 48%
5 e-Services Delivery 35% 15% 0%
6 Search Readiness 2% 3% 3%
7 Personalisation 4% 4% 4%
8 Collaboration & Integration 10% 7% 5%
9 Feedback 3% 3% 3%
10   e-Consultation 10% 12% 15%
  Overall e-Gov Watch Rating 100% 100% 100%

Please contact us to discuss pricing options.

All enquiries for countries outside New Zealand should contact us to enquire about relevant pricing and inclusion of relevant government web standards.