The Ministry of Justice ran a Proof of Concept (PoC) to understand how technology could support the management of court cases from start to finish (caseflow management) and explore future ways of working.
The Proof of Concept
Technology has advanced significantly since the Ministry last invested in case management tools in the 2000s.
In 2018, the Ministry identified 2 technology options that might facilitate more effective caseflow management:
- Business Process Management Solution (BPMS) - (offering flexible components that can be used to develop a complete solution)
- Commercial Off the Shelf (COTS) solution - (caseflow management tools that have been developed for the court system)
The Ministry decided to run a Proof of Concept (PoC) to test these 2 options and see how they could be configured to meet the requirements of a court process and the judiciary, and support participants like lawyers.
The purpose of the PoC was to:
- understand what trade-offs there are—if any—between the 2 options
- enable the Ministry, Judiciary and participants to see what could be possible in the future
- see if technology opens up new ways to manage the changes associated with moving to different ways of working.
While we understand the current state of today’s world, we do not know the art of the possible … the PoC provides a unique opportunity for us, the Judiciary and our stakeholders to tangibly see the future of managing cases rather than a system that replicates today’s paper-based world.
Market and user research
The Ministry integrated market research into the PoC process by sending a Request for Information (RFI) to a range of suppliers around the world. Their answers informed the selection of the suppliers for the PoC and provided rich information about the breadth of available tools and the approaches that others have taken.
The Ministry then conducted user research interviews with the clients of suppliers. This qualitative research gathered valuable insights across the experiences of different justice systems that were undertaking a similar change to the one the Ministry was contemplating.
Defining the requirements for the PoC
The Ministry developed a non-traditional requirements ‘PoC book’ that very simply described the business and user contexts. It outlined:
- outcomes the Ministry were looking for
- key rules and legislation the system must support
- what was to be tested in each part of the process.
The Ministry did not use business analysts to create the PoC book. Instead 2 people with no process experience—a junior Court Registry Officer and a staff member from the operational part of the business—developed the requirements.
How the PoC was run
The Ministry engaged 2 suppliers, one to represent the BPMS option and one to represent the COTS option. The suppliers were asked to use their technology to create a system for managing a simplified Category 3 Criminal process.
The 2 suppliers worked on-site at the Ministry of Justice. They started on the same day after a joint induction when they were given the PoC book. They had 3 weeks to configure their system and be ready to demonstrate it.
The suppliers were not focussing on a predetermined solution, but were free to explore, to validate the problems that came up, and to design a system based on their insights.
During the configuration period, internal technical architects shadowed both suppliers to allow the Ministry to fundamentally understand the amount of effort that was needed to do this work. The technical architects only gave limited guidance to the suppliers and spent equal time with each supplier to avoid bias.
Demonstrating the PoC
Following the 3-week configuration period, the Ministry demonstrated the PoC to over 80 people, including: the Ministry, Judiciary, Police, Department of Corrections, professional bodies, central agencies and other stakeholders.
This gave potential users a valuable opportunity to understand how technology could support caseflow management, and how a PoC could test preconceptions and help see what could be possible in the future.
Evaluating the PoC
The intent of the PoC and the market research was to compare the options rather than the specific technology products. This meant each option was evaluated by what it could do differently from the other, rather than by what it had in common with the other. For example, if both options offered a specific functionality, then the Ministry excluded this component from evaluation.
The Ministry also gathered user experience feedback throughout the PoC demonstrations to understand what users valued and didn’t value.
Results of the PoC
The Ministry learnt a lot from undertaking a different process. The PoC dispelled myths, provided new insights and proved the Ministry could do things differently.
Finding the right solution for caseflow management
The findings from this exercise are now being used to inform a business case regarding the future of caseflow management for New Zealand’s courts.
Understanding capability and appetite for change
The design of the PoC process allowed the Ministry to understand more about its own capability and appetite for change. For example, when the PoC book was developed using non-traditional methods, it tested whether this type of requirements book could be:
- created by people with no process experience
- used by a supplier with limited additional support to both configure their system and customise their tool.
The PoC book was well received by both suppliers, who found it easier to work from than traditional requirements.
Informing future design and investment decisions
Technology options available today are more modular and configurable, enabling an expansion of functionality over time as maturity increases, and iterative development involving prototyping, piloting and implementation.
Iterative development
Detailed description of Figure 1
This diagram shows the process of iterative development the Ministry plans to use. Rather than designing the process 'on paper', the Ministry will describe the requirements, and then enable users to refine the design by engaging with the tool as it is developed, prototyped and tested.
In summary, this iterative process:
- reveals flaws at the early stages when the cost of eliminating mistakes is lower
- allows you to reduce usability issues before you build and launch a product.
Understanding this, the Ministry has found that by undertaking a PoC exercise (or creating a prototype), it can make better investment decisions.
The PoC really challenged the thinking of those involved and established a true partnership between the business, the Judiciary and ICT, as well as fundamentally changing how we will approach the design and implementation of caseflow management.
