What Makes a BPMS Good?

I’m in the process of updating my 2006 BPMS Report series on BPMInstitute.org to the new and improved 2007 version. A major change from last year is a beefed-up evaluation scoring. I’ve discovered that many users (and most vendors) are happy to skip the 25-page walkthrough of the product and go straight to the scorecard at the end. Which product “won”? I haven’t figured out the presentation - it will probably be some 2-dimensional thing like the Forrester Wave or Gartner MQ - but I’m close to having a finished scoring methodology. It’s probably asking for trouble, but I’m publishing it right here so that you can comment upon it.

The basic plan is this. I define 4 process types: Task Routing (basic workflow), Production Workflow, Case Management (emphasizes content, collaboration, and unstructured processes), and Integration-Centric. The characteristics of each type are explained in the report overview, but most of you can imagine what they are. Each BPMS is scored against all 4 process types using 12 sets of criteria, but the weightings of each set may differ from one process type to the next. Also, the capabilities affecting the individual criteria may be process type-specific.

Here are the 12 sets of criteria, things I’m looking for in each (some are process type-specific), and the percentage weighting of the set for each process type:

1. Architecture and Environment (Weightings: task routing 10%, production workflow 10%, case management 8%, integration-centric 10%)
· Unified environment for workflow, integration, rules, BAM
· Minimal programming required
· Scalable
· Clustering, hi availability
· Leverage J2EE or equiv platform services
· Web-hostable/SaaS-compatible runtime
· Support for standards
· RAD/iterative support
· Component discovery and reuse

2. Modeling (Weightings: task routing 10%, production workflow 10%, case management 4%, integration-centric 5%)
· Usable by business process analysts (not developers)
· Support for BPMN (full support)
· Integration of BPA (ARIS, ProForma, MEGA, etc)
· Stay in sync w/design (shared or roundtripping)
· KPI modeling
· Model repository, publishing
· Support for industry models (SCOR, ITIL, etc)
· Team collaboration
· Simulation

3. Human Workflow (design) (Weightings: task routing 10%, production workflow 10%, case management 8%, integration-centric 5%)
· Dynamic task assignment
· Flexible routing
· Form design
· Screenflow design
· Worklist design (columns for business data)
· Internationalization support

4. User Experience (runtime) (Weightings: task routing 15%, production workflow 7%, case management 13%, integration-centric 5%)
· Ajax web forms
· Offline task participation
· Guided task performance
· Portal design and integration
· Reassign/delegate task
· Production workflow support (getNext item)
· User sort/filter/query worklist
· Instance status tracking

5. Content/Collaboration/Case Management (Weightings: task routing 10%, production workflow 7%, case management 13%, integration-centric 5%)
· OOTB attachment support in task UI
· Doc upload, annotate, edit
· Viewers
· Checkin checkout, versioning
· Integ with 3d party ECM
· Archiving
· Retention mgmt
· Team room support
· Discussion, chat, presence detect
· Process knowledge repository
· Case mgmt support

6. Business Rule Management (Weightings: task routing 5%, production workflow 7%, case management 8%, integration-centric 10%)
· Rule repository
· Rule design
· Rule maintenance app
· Rule change impact analysis
· Rule engine
· Rule-process integration

7. Integration (Weightings: task routing 5%, production workflow 7%, case management 8%, integration-centric 10%)
· Adapters – Introspectng/self-generating; packaged - Mainframe, pkg app, b2b, .Net etc
· Async integration, callback, reliable messaging, WS-Addressing, security etc
· Data transformation mapping, XSLT/XQuery engine
· ESB, mediation
· Registry/repository
· Metadata library
· B2B support – trading partner gateway, EDI

8. Events and Exceptions (Weightings: task routing 5%, production workflow 6%, case management 13%, integration-centric 10%)
· Event listeners and adapters
· Full bpmn event support
· Wait for event
· Interrupt by event + exception flow
· Manual suspend/resume instance
· Error propagation, handling without programming
· BAM events and actions
· Transaction rollback and compensation

9. Performance Mgmt/BAM (Weightings: task routing 5%, production workflow 10%, case management 4%, integration-centric 10%)
· Metrics and kpis – OOTB and User-defined
· Dashboard design - Charts, reports, alerts and notifications
· OLAP-style breakdowns
· Bam rules and actions
· Drilldown to root cause
· Instance monitoring
· Optimization actions
· Predictive analytics, e.g. expected finish

10. Governance (Weightings: task routing 5%, production workflow 10%, case management 4%, integration-centric 10%)
· Enterprise repository of models and components
· Link processes to goals and KPIs
· Role based access control and authorization
· Version control of model and implementation components
· Change request, approval, and implementation workflows
· Audit trail on component changes
· Change impact analysis

11. Solutions and Services (Weightings: task routing 5%, production workflow 6%, case management 4%, integration-centric 10%)
· Industry solutions – documented, QA’ed, supported
· Partner industry solutions
· Professional services

12. Installed/reference customers (Weightings: task routing 15%, production workflow 10%, case management 13%, integration-centric 10%)
· Task routing implementations
· Production workflow implementations (volume, industry)
· Case management implementations
· Integration-centric implementations (volume, industry)

Products will be scored in each of the 12 categories from 0-5, as in the Forrester Wave, based on the bullets listed here (as amended); in some cases the scoring will be process type-specific. Then these scores will be weighted as shown here (or as amended) for each category. One or more categories may be split off to form the second axis of the final result, as Gartner and Forrester both do in theirs.

So there you have it. If you see something missing or improperly weighted here, please let me know, either by comment or by private email.