Step 4 – Defining the success criteria

The key objective in defining the success criteria is to document what a good solution should look like for the project to succeed and become production-ready.

You need to clearly define the elements that need to function correctly to move from POC to POT and then into a pilot phase, before finally deploying into production. You need to fully document what these elements are and get the end users or other project stakeholders to sign up to them. It's almost like creating a statement of work with a clearly defined list of tasks. If you don't have any success criteria in place, then you should not start the testing!

Another important factor is to ensure that during this phase of the project, the success criteria don't start to grow beyond the original scope, which is commonly known as "scope creep." This means that any additional elements should not get added to the success criteria, or at least not without discussing them first. It may well transpire that something key was missed; however, if you have conducted your assessment thoroughly, this shouldn't happen. This is another reason for having the success criteria in place, otherwise you won't know what is on the list, and the list will never end with things continually being added.

Another thing that works well at this stage is to, once again, involve the end users. Set up a steering committee or advisory panel by selecting people from different departments to act as sponsors within their area of business. Actively involve them in the testing phases but get them on board early to get their input in shaping the solution from the outset.

Too many projects fail when a user tries something that didn't work. However, the thing that they tried is not actually a relevant use case or something that is used by the business as a critical line of business app, and therefore shouldn't derail the project.

I once saw a VDI project failing due to the unresponsiveness of the mouse when using Microsoft Paint. This knocked the project way off course while the issue was investigated. In the end, it was shown that Paint was not used by anyone, and so was totally irrelevant to the business or use case. However, it still ended up burning precious cycles while the team tried to enhance performance. As this customer had no success criteria defined beforehand, it was difficult to move the project on. If there had of been a list and this application was not on that list, then it could simply be ignored, and the project could have moved on.

If we have a set of success criteria defined up front that the end users have signed up to, anything outside that criteria is not in scope. If it's not defined in the document, it should be disregarded as not being part of what success should look like.