Business Analysis Technique #2 – Benchmarking

Posted on May 27, 2011 by



benchmarking reveals the fingerprint of a new system

Benchmarking is a valid, powerful and concrete way to compare a new system or process to the current, “as-is” state OR to compare multiple systems in a vendor selection process. To ensure each of those adjectives (valid, powerful and concrete) are met here’s some suggestions:

  1. Valid – ensure a level playing field in all systems evaluated – think “sameness”!  The same approach, conditions and evaluation guides need to be used.  Many years ago, while benchmarking systems for Nebraska’s first Automate Fingerprint Identification System for the Nebraska State Patrol (circa 1994) we included fingerprints that would match, partially match and had zero probability of matching.  One vendor objected to the zero match fingerprints (called latent fingerprints).  We held firm in the benchmark and onsite testing – in part that two other vendors had already complied.
  2. Powerful – benchmarks of the current process, user interaction and system performance ensure that new systems can be fairly and completely verified. This is powerful and makes for an apples to apples comparison.  If the current system provides 99.99% up-time and less than 2 hours for return to operational state, then all replacement systems can be held to that standard, OR BETTER!
  3. Concrete performing benchmarks is a concrete way to see a set of requirements in action.  A few years back (2004) I facilitated a vendor review for a modernization of a large state agency unemployment insurance system.  Three leading vendors had pitched all sorts of cutting edge development frameworks, tool sets and approaches – it was at benchmarking that the complexity (or over complexity) and mismatch was found.

Benchmark away!  Capture metrics for quality, quantity, speed of your current system and maintain those metrics. They are the seeds for benchmarking.

Enhanced by Zemanta