Optimal Experience have just completed the third and final round of user testing on the beta site. Their findings reassured us that we’re on the right track.
Why test a beta site?
The industry feedback we’ve received is invaluable and we’ve used it to refine a number of aspects of the site — thank you. Beta.govt.nz is designed for our end users — ordinary New Zealanders — and it’s their needs we’ve been striving to meet. User testing works: it addresses functionality, interface design and content. We’re learning more all the time as we integrate our findings into our ongoing development.
We were looking for:
- task completion time
- an increase in number of people following the correct path
- search functionality — ease of use
- observational feedback on the site overall.
The results: some great, some good … still more to do
The report has given the team reason to yell “yay”, it’s earned a few pats on the back, and given us a few reminders that there’s still more to do.
The top line of the report included:
- 22 positive findings
- 2 severe issues that require immediate attention
- 25 major issues requiring urgent attention
- 16 minor or cosmetic issues which are less pressing
- 24 observations which are not problematic but important to note as well
- 12 key findings and recommendations.
A round of applause please…
Our “boom” moment this round came from our SUS result. SUS stands for "system usability scale" and is the score that measures the usability of a site. It takes into account:
- effectiveness — can users successfully achieve their objectives
- efficiency — how much effort and resource is expended in achieving those objectives
- satisfaction — was the experience satisfactory.
A SUS score greater than 68% is considered above average. We scored 70.3% this round, an improvement on round 2’s score of 68.6%.
We were also happy that:
- users liked the website purpose:
The mother ship of all government departments.
Definitely use it — would go on and learn things even though I didn't need to use it.
- content was easy to understand:
It's right there. Bullet points, short sentences — done!
- content highlights also included success with relevant search results (a hard one to achieve when users’ reference point is Google), relevant search result descriptions and helpful headings and synopses.
- navigation highlighted users taking the most common paths and easily navigating homepage categories. The high use of popular pages also had us congratulating each other.
Issues already ticked off
We’re constantly iterating, so over a third of the issues raised from round 2 had already been completed by round 3. Developing in Agile allows our team to be exactly that, which meant we already had a plan of attack to fix:
- visual design: some users noted the website looked “plain”. Our latest release incorporates an “NZ” feel with a change in colour, layout and (soon to be included) images.
- style/layout: refined layout design now includes a prominence of popular pages (which are dynamic) listed on the homepage, plus hub pages now feature the most relevant pages hosted within a hub.
- external links not leading where expected: link styles were not added prior to round 3 testing. Design alterations are underway and will be completed for production release.
- search/filter improvements: design, help text and content have been updated to assist with this.
What to expect when Govt.nz goes live
Juggling priorities and resources leaves some issues lower down the “to do” list — fortunately we'd pre-empted many of the issues and the work is already underway to resolve them (e.g. search improvements).
Current developments planned for go live include:
- further updates on site search (currently “severe” so this is our highest priority)
- refinement on Government A–Z listings and search through filter options
- creating a quicker path for users “to complain about government”.
Talk to us — we love to share
Feedback is valuable, from whatever source. We're looking forward to getting feedback from users once the site is fully live — we know this will give us more things to look at, and the site will get even better.
We use open source code and our API is public because we believe sharing is important.
