Selecting COTS software under pressure (Forrester and Gartner)
This happens on many projects: everyone agrees that the core of a solution (e.g. workflow engine) will be purchased instead of built from scratch. This is an absolutely rational decision for most enterprise applications. The next question is which vendor to select.
The rational approach is to identify the core functional requirements and key quality attributes that the solution must exhibit. This is an absolute must; because without understanding what you need you will not be able to objectively judge your options. The criteria on which you should evaluate you COTS software candidates must be granular and quantifiable to the degree that makes sense. Each criteria element needs to be weighed against all other chosen consideration elements so that all weights add to a whole. E.g. In case of workflow COTS software the number of concurrent workflow processes that the solution must support has to be at least 400 and this criterion has a 15% importance in relation to other selection elements.
The next step is to conduct evaluation of the market space for available vendor solutions. A number of solutions will not meet the top selected criteria based on the marketing information from the vendor, but few will emerge as potential candidates. Since COTS software will be an integral part of the solution a targeted proof of concept should be implemented and results judged against the established criteria. The outcome of the test along with other evaluation criteria is used to make an objective and rational decision.
If a decision on the COTS software absolutely has to be made in a period of time that doesn’t allow a proof of concept, then you should buy the intelligence from a third party that has done COTS software evaluation for you. Both Gartner (Magic Quadrant) and Forrester (The Forrester Wave) provide analysis of various COTS products that fulfill a particular segment of business applications (e.g. BPM tools). Each vendor solution is evaluated against a wide range of criteria. These reports provide very valuable insight, but without objectively understanding the needs of the solution you’re architecting it’s not possible to fully utilize the insight provided by a third party.
In general a Gartner, Forrester, or another 3rd party report should be used in any COTS software evaluation whether you’re pressed for time or not.