Managing Process Innovation

Applying performance goals to business-process management drives growth and innovation.

When image document routing first appeared in the 1990s, the idea of applying computer technology to this kind of labor-intensive business process was considered cutting edge. Everyone in IT understood the potential of centralized computing for numerical computation and transaction processing, but few envisioned that this type of application would fit a broader set of distributed business processes.

Today, leading companies such as Southwest Airlines, British Petroleum, and Bank of America, are finding more innovative ways to automate their business processes. E-forms, process modeling, simulation, EAI, integration services, rules engines, event services, real-time monitoring, and process analytics are among the systems being applied to processes that include order management, billing, financial reporting, credit-card issuance, product returns, and dispute resolution.

Although these companies are still in the minority, more IT and operations executives are beginning to understand that their current technology does little to link the processes that run their companies to the transactions that result from those processes—transactions at the heart of corporate growth and profitability. This disconnect is rooted in a basic misunderstanding of the purpose of enterprise resource planning and the role of business-process management—relationships that we're now examining more carefully.

It's no secret that work gets done by people through business processes and that technology only supports those processes. Whether distributing goods to customers, collaborating with suppliers, or coordinating employee efforts, business processes add value to products and brands.

Yet most ERP systems have a functional focus and lack process models that explain business operations. As a result, when managers try to innovate or solve business problems—customer satisfaction is one of the most widespread—the "fix" is often myopic and transaction-centric. What's missing is good business-process management.

Business-process management provides methods to automate and/or improve activities and tasks for particular business purposes. Its goal is not only efficiency and productivity, but also control, responsiveness, and improvement. Control assures that company resources are aligned to execute strategies. Responsiveness and improvement support the competitive differentiation that enables a company to excel.

IT executives can assert control by basing the direction and flow of transactions on a predefined set of rules and work flows—for example, determining how a purchase order is acknowledged or merchandise is returned. Responsiveness enables individuals to react quickly to business events and maximize interactions, as when expediting a critical customer order across the customer-service and warehouse teams. As for improvement, you want to systematically measure and monitor processes; doing so will lead to innovation and optimal performance.

An integral part of business-process management is performance management, which is intended to steer the organization and its partners toward corporate goals. Performance management focuses on the collaboration and empowerment of all individuals in the business network or value chain. It enables them to work across strategic, tactical, and operational levels to align actions that produce rapid, effective responses to business challenges.

In the same way you define and document processes, you need to detail performance-management objectives. These objectives are the analytics used to measure all process-improvement projects. The metrics will provide the basis for an ongoing cycle of measurement, evaluation, and improvement. It's also critical that your company tie these process-improvement metrics to high-level performance-improvement goals, not low-level or transaction-oriented metrics.

Done right, performance management should shed light on why some processes don't function well and how to go about improving them. During analysis, the tools will provide the project teams with data to assess the productivity impact of proposed solutions. That data should also help business and IT departments arrive at a common understanding of particular business needs and their solutions. Collaboration between these groups is particularly important in good business process management.

Once you've addressed the overall issue of performance management and built the framework for a sustainable business-process management practice, you can begin to assess requirements for the IT systems that will support it. For this part of the project, five components are necessary: assessing current systems, building a business case, developing and communicating the plan, evaluating software and architecture options, and, lastly, deploying the initiative. Here is further detail on each of the steps.

First, conduct an independent assessment of the process you want to innovate and the systems that currently support it. Establish a benchmark for the current levels of efficiency and effectiveness, and then identify areas for improvement. Of course, you'll need to evaluate financial and operational requirements in this approach, including ROI and total-cost-of-ownership calculations.

Next, build a business case to demonstrate the value and results that the project will deliver, citing clear definitions of the value and cost of your program, as well as compelling productivity and financial reasons for going ahead. Address the cultural, business, and technology barriers to ensure you have support for your initiative.

Third, create a well-defined program and plan, and communicate it to the process owners and participants. The program will have to be articulated at different levels of the organization, so make clear to all stakeholders what's in it for them. The program should also show how the effectiveness of operations will improve through this process innovation.

After that, architecture and software needs should be identified in several ways. First, evaluate solutions with appropriate criteria to ensure that the program is timely and responsive to the organization. Consolidation in the business software market has changed the landscape of business-process management systems significantly. Ventana Research recommends that you evaluate all viable options including service-oriented architectures, but be wary of promises from vendors of single solutions that do only business-activity monitoring (BAM) or only process modeling—these products may not be functionally rich enough.

During your vendor evaluations, keep performance management capabilities in mind. Good business-process management requires both simulation and BAM tools. Simulation aids process design and modeling by letting designers preview how a process flows and look at how the logic, events, and rules will work together—before the process is rolled out into a production environment. Using such a tool, you should discover and remove bottlenecks and accurately predict process performance. The best process models allow multiple simulation scenarios to be performed across subprocesses. The engine should be able to track resource usage, including cost and time analysis, and monitor usage that exceeds preset thresholds.

A robust simulation tool will allow you to deploy new versions of processes without interrupting those already in use and the best solutions allow controlled migration from old processes to new ones. This capability is critical not only for safety reasons but also for benchmarking and measuring results.

Business-activity monitoring aggregates, analyzes, and presents relevant and timely internal information as well as data involving your customers and partners. BAM solutions can alert individuals to changes in the business that may require action from them. Once again, the purpose is to produce rapid insight into process innovations by identifying issues in real time, improving process performance, and reducing operating costs.

Most BAM solutions provide postprocess metrics, such as when and how many times the process was executed and which user performed which tasks. Some go further to provide visual representations of business activity with maps, technical drawings, charts, blueprints, or graphs.

However, keep in mind that real-time process monitoring requires considerable development and integration work. Consider tying these activities to your company's performance management objectives, measuring and tracking them in an active, balanced scorecard. This may look like a project inside a project, but it's well worth the effort. After all, how will you know if you're aligning current process activities to your performance objectives if you don't properly score the results?

Once you've evaluated all software and architecture options, the fifth and final step is to roll out your solution and ensure widespread adoption of your business-process management initiative. To succeed, you must understand how to minimize interruptions to your current business processes, culture, and technology usage.

Make sure you don't skip any of these steps—especially the second, where you benchmark your current process performance and build a business case. Doing so will enable you to showcase the value of your innovation after adoption. Following these steps will increase efficiency and effectiveness, and improve the alignment of your operational processes. Through a simple organizationwide approach, you can transform your people, processes, and systems in an efficient manner that ultimately will be reflected on the company's bottom line.

Colin Snow, VP and research director for Ventana Research, is the leader of the company's Operational Performance Management practice.

What Matters Most

To measure and monitor business activities, software has to perform analysis and integration with underlying systems. Business-process management software must also allow for rules and definitions of business processes and for monitoring and leveraging work-flow exceptions. Findings from Ventana Research's recent Operational Performance Management Research Study indicate that all of these capabilities are critical.

In the survey, more than 1,300 business- and IT-management respondents were asked to rank in importance six major software capabilities for measuring and monitoring business activities and processes. The most highly valued capability was "integrate data and application sources," with 61% of respondents saying that integration is somewhat or very important. More than half the respondents scored all capabilities as important except "define work flow," which drew 48% of responses.

To determine what software capabilities are important to them, companies must know how specifically and widely to measure their business activities and processes. The level of sophistication required will determine capabilities such as work flow, rules, or defining and visualizing business processes. The basic set of software capabilities for integration and analysis are available in most enterprises with business-intelligence solutions, but the additional capabilities outlined in the survey must be included for successful business-process management.

What's Driving Process Innovation?

For many enterprises, the first driver of business-process innovation was the need to improve operational efficiency. Today, the focus has expanded in two directions.

First, regulatory-compliance mandates from the Sarbanes-Oxley Act and the European Union are forcing companies to document processes and procedures in financial reporting, product design, supply chains, and service delivery. In addition, companies are looking for ways to make operations more effective, particularly by automating operational decision-making. This requires systems that can analyze, select, and execute several courses of action. To perform such operations, the systems need to apply explicit policies and procedures—as enforced by rules engines—and execute in a hands-off fashion. Here are examples of how specific industries might deploy automated process innovations:

• Insurance: consumer underwriting, pricing
  
  
• E-commerce: promotional-offer personalization
  
  
• Retail: inventory replenishment
  
  
• Transportation: asset deployment
  
  
• Brokerage: trading, fraud detection, pricing
  
  
• Energy: service-delivery management
  
  
• Telecom: provisioning optimization
  
  
• Government: fraud detection, security

---

Action Plan

Achieving optimal performance from operational processes is essential to an organization's business-process innovation efforts. Follow these guidelines to align process with business objectives:

• To begin a process-innovation project, start with these performance-management principles:

• Develop a performance plan that describes how your operational processes contribute to strategic goals.
  
  
• Determine the key analytics that you'll use to measure all process improvements.
  
  
• Set priorities for improving operational processes that will advance your strategic objectives.

When these guidelines are in place, begin to assess requirements for the IT systems that will support your project. Ventana recommends taking the following five steps:

• Assess your core strategic and operational processes.
  
  
• Build a business case to demonstrate the value and results (through benchmarking) that the project will deliver.
  
  
• Create a well-defined program and plan, and communicate it to all process owners and participants.
  
  
• Evaluate solutions by considering both users' needs and performance management requirements.
  
  
• Model, simulate, and deploy your solution.

Once you've determined these requirements, continue to measure your process with a performance scorecard. Your metrics will compare the degree of improvement against the benchmarks and simulations. Applying the metrics rigorously will guide your ongoing actions toward greater productivity and stronger financial results.

---

http://www.optimizemag.com/

BPM For The Masses

Mariano Benitez12/13/2006

Abstract

A business process is a formal representation of the work done by people and systems of an organization and its partners, and is aimed at providing a product or service to internal or external customers. Business Process Management (BPM) is the art of formalizing and automating business processes. AquaLogic BPM Suite (ALBPM) is a Business Process Management system that allows the complete management of end-to-end business processes.
This article gives you an introduction to the functionality of the product, along with some very important things you need to consider when moving to BPM. It does not provide a complete description of the concepts of BPM, but it will give you the vision of BPM that the AquaLogic product proposes for it.
The article is aimed at anyone who wants to get an initial view of the product and the technology; it presents important information for solutions architects and for those responsible for designing software systems that are to be moved to BPM-based solutions. Developers and executives can benefit from the topics covered here as well.
Something to Do (Before You Start)
Before starting, I want you to ask yourself these questions:
What is BPM?
How do you think business processes get implemented in the real world?
Do you see the processes in your organization reflected clearly in your applications?
Can you easily follow the processes inside your organization?
Would using a BPM tool change the way people work in your organization?
When you’re done reading the article, ask yourself these questions again. Then look at the answers, and see if you may have a better way to tackle these issues. My intention is not to answer these questions directly, but to let you elaborate on them, drawing your own conclusions.
Policy of Truth (Let’s See Some Definitions)
A business process is a formal representation of the work done by people and systems of an organization and its partners, with the aim to provide a product or service to internal or external customers. In its simplest form, a process is a set of activities, representing different steps of the process, linked together through transitions. Activities can require human intervention or they can be totally automatic. For human interactive activities, you define a role in the process that identifies who is allowed to interact with the process at this point. The process acts as a definition, where instances of the process are the actual items moving through it, transitioning from activity to activity. Instances always start at the Begin activity of the process and finish in the End activity. The path the instances take depends entirely on the data of the instance and the external environment.
A transition is a directed link between activities, and many transitions may come in or out of an activity. Once an instance has completed an activity, the outgoing transitions are evaluated, and one is selected that will move the instance to the next activity. Conditional transitions contain a boolean expression that is evaluated and must be true to allow the instance to follow that transition. Some transitions are time-based, which means that they will trigger an automatic routing to the destination activity if the instance is still there at the due time. Processes can have state, too: You can define attributes to a process that will take a value for each instance, helping you keep the state of the instance, to transition to different activities (for example, if quote amount > X, then go to this activity).
This is what BPM is from a pure modeling perspective. This definition is not enough, however, to execute the process; you still need to talk to the other systems, reflecting the process into the infrastructure. Here is where integration comes into play. For each activity in the process, you can define tasks, which basically consist of code being executed when an instance reaches the activity. Tasks can be implemented in a variety of ways. When an instance reaches an automatic activity, the task defined for the activity will be executed, and, depending on the result of that execution, the instance can move to the next activity.
For human interactive activities, the execution of the task is triggered by the end user, from the client workspace. The AquaLogic product has built-in integration capabilities that can be used in the tasks code. (This feature is an important one in the product, which I will cover extensively in further articles.) Tasks update the process state, accessing and changing external systems, interacting with end users, and more. Tasks are what turn a simple process model into a process-driven application.
Business Process Management is the art of formalizing and automating business processes. To successfully evolve into a process-oriented organization, you need tools to design, execute, and monitor business processes. This is what Business Process Management Systems are built for. This is what AquaLogic BPM was designed for.
A Question of Time (Before You Move to BPM)
As organizations try to get more agile, they need to adapt the software they run their businesses with. From a conceptual point of view, the thing that changes most often is not the back-end applications, or the Web service definition to access a customer, or the location of the EJBs, or the version of ERP they are based on. The single most important thing that changes when businesses need to adapt is the process itself: Businesses must add a faster path to automatically approve orders based on some criteria, to modify a reference value that raises or lowers the approval rate. They need process approval and check activities in parallel, they need to change the role that performs an activity, they need to handle exceptions manually, and they need to enforce new business rules. Handling these kinds of changes is at the core of BPM, allowing modifications to be made in the business process without altering any of the underlying components.
In this next section I bring up some things that I consider very important to understanding the true value of BPM.
Behind the Wheel (the Process is in Control)
By embracing BPM, you are now externalizing one important piece of your business. The problem is that if your process model is not executable (it is a pure model), eventually you will have to map it to an application, transforming it, just like you did when moving from "analysis" to "design" to "implementation." This makes it really hard to map the business process to what really is being run in production. With AquaLogic BPM Suite, you build a complete process-driven application, attaching tasks to activities, which are the real operations that take place inside the process. A task is the code of the process and can be either user-driven or system-driven. The execution of tasks is the main driver of state changes in the process. So, the process really drives the application, directly enforcing the flow, tasks, and operations allowed on the process.
Writing process-driven applications does not change the way you write code or implement services; you will still write Java classes, Web services, and JSPs in the same way. The main difference is that the application is process driven. This means that all the code is triggered from activity tasks, so you just need to focus on the code for these tasks; the rest is handled for you. The basic process operations are provided out of the box, in the user workspace.
Walking in My Shoes (or in Your End User's Shoes)
The main shift for your end users (the users of the solutions you build) is that they have a single list of things to do. Users have an inbox where they see the pending instances and the tasks available to execute for each instance.
Before, they had to look on different pages, systems, and applications for actions to perform. Luckily, they received a notification that they had some new work to do, but it was very unlikely they would find it in a single view. BPM makes job assignment very intuitive for users.
The most important change for end users is that they are presented with a more task-oriented view of the work. They easily understand that they have a pending order waiting to be approved by executing the tasks enabled for that activity. By clearly defining the processes, you are creating a unit of work that is clear for all participating entities (other systems, other people).
With processes and instances, you need a user front-end for managing the instances (CRUD-like basic operations). For BPM, these operations would be create, search, execute, abort, delegate, and route. AquaLogic BPM Suite provides all these features out of the box, either from the HiPer Workspace (in any flavor) or directly, using APIs (Java- or Web service-based). You don’t need to build this functionality; it’s already there.
BPM is part of the big SOA movement and plays a very important role in this movement. Let me explain how.
World in My Eyes (the SOA World in the Eyes of BPM)
A number of well known people have defined how BPM and SOA relate to each other.
Bruce Silver’s BPMS Watch has a nice article on BEA’s take on BPM-SOA.
Alfred Chuang’s article on the Exec2Exec newsletter also provides a good description.
My view of SOA is that it is about building services that are encapsulated, reusable, and decoupled. SOA has two main "aspects"?: building decoupled services, and binding them together in a simple way. In a SOA enterprise, all services are ultimately linked together from top to bottom at different levels (otherwise they would not be used). Some bindings are pretty static, and others are very dynamic. BPM tries to provide a tool to model the level that changes the most—the business level. Processes use the SOA services as building blocks that will reflect your business process in the underlying systems. These core services are reused throughout all the processes, providing the real value of reuse. The processes bind the business, data, and presentation services into the value generator: the real business process.
Without a process layer, eventually you will wire the invocation of these services in the presentation layer or in the business layer, and some changes in the business logic will definitively imply finding what services to modify, and rewiring services to adjust to the new business definition. Let me elaborate with an example: You have a Web-application, with a two-tier architecture, with business components and presentation. Some presentation component contains the sequencing logic of the application. The business user decides to make a change in the business process, changing the order of some operations and the usage of a business component. How do you map the business requirement to the exact code you need to change? How do you know where to modify? Why do you need to know the implementation to change the process?
With the process layer, you have a clear space to make use of the available services, acting as the glue for those services, making them useful inside the scope of a business process. So, when your business changes, just one place changes: the process, not the services.
When talking about reuse in the context of BPM, multiple levels are open to consideration:
Low-level reuse of external systems, where many business processes will access the same Web service.
Medium-level reuse of BPM, where you can share process and component templates, reusing the best practices of the code and process models.
High-level reuse of processes, where you expose a process that can be used by other parts of the organization.
Now that I’ve painted a picture of what BPM is and how it fits in the new world, I'll go deeper into the product I want you to meet.
Something to Do (with AquaLogic BPM Suite)
AquaLogic BPM Suite is a product that embraces the whole lifecycle of business processes. It provides tools for all solution times: development time, runtime, and evolution time. For each stage, a number of tasks must be performed to complete the lifecycle, I will get into the tasks next, when I describe what can you do with ALBPM.
Enjoy the Silence (and What BPM Can Do for You)
AquaLogic BPM allows you to design, simulate, develop, execute, monitor, operate, and change business processes. I will explain what each of these things mean.
Development time
This is where you transform your ideas into an application, something executable that includes all things necessary to make it alive: presentation, processes, integration, everything all bundled in a BPM project source.
· Business Process Design—ALBPM Designer allows you to build in a graphical way the real business process, defining process roles, activities, and the flow between the activities. You can also document the operations that need to be performed on other systems. Here you are defining the way your business will work, establishing dependencies, SLAs, exception handling, and more. The result of the design is your business process model. This model will remain as the core of all other steps in the process lifecycle, even at runtime. This model can be visible to end users.
· Business Process Simulation—ALBPM Designer can simulate the execution of the business process. For each activity in the process, you can define resources, costs, times, and the distribution of the different transitions. The simulation will then use that information and generate information about times, bottlenecks, total costs, and so on. This allows you to understand where the process issues will be.
· Business Process Development—With ALBPM Studio you build a complete process-driven application. You define tasks for each activity in the process and what is executed on each task. Task implementations contain business process logic, integration code, and a presentation. The result of this is an executable application. This is where you declare and use the integration points to external system or resources that the process requires. You are defining here the application that will support your business process, the process UI, and the interaction with the external systems. Of course, you can use any component of your SOA infrastructure, including other BEA products. ALBPM has extensive integration capabilities that allow you to use any external software that has an interface to interact with. The result of the development is your business process-driven application. For simplicity sake, for the rest of the article I will refer to this as The Process.
Runtime
Once you have your application source, you can transform them into a living entity, exposing the BPM project to end users, creating instances, connecting with the other systems involved, and so on.
· Business Process Execution—ALBPM Enterprise is the runtime environment for business processes. It provides the process engine that runs the processes and a number of client applications that are the end user front end. This product is all you need to build a BPM environment, with running processes and users connecting to them. Infrastructure management applications are also provided. This is our core product—the "heart" of a BPM solution. The architecture of the product ensures that all enterprise-level features are provided, like scalability, performance, and availability. The product is built to support large-scale installations, with many users, processes, and more.
· Business Process Operation—The process engine keeps a record of the events that occur with each instance of the processes. It stores time, user, and results of any operation performed by users or systems. This information can be reviewed later to audit the steps that the instance has been through. This capability manifests cause/effect relations that, in many cases, were untraceable. You can also search for instances using flexible criteria to see the state of an instance at any moment, or you can override the default flow and grab an instance that is in a wrong state and send it to the proper place. Basic process-oriented operations are provided out of the box, like instance creation, cancellation, routing, delegation, and assignation. All these operations are enabled based on the permissions assigned to the users, and on the process definition.
· Business Process Monitoring—Based on the information available on the process engine, you can easily build dashboards that give you information that is relevant to the business process owner, in the owner's terms—like how much time the entire process takes, or how much money is pending to be collected because of delayed processing. For each process, you can define what measures are relevant to the business and what dimensions need to be defined to ensure proper drill down. All this information is used to build the dashboards that are later exposed at runtime, and provide information about the current state of the process.
Business process evolution
Inevitably, once your business process is alive, you will need to change it. The reason for change can be a result of changes in the market, or new business definitions, or you may have detected inefficiencies in the existing process. You go back to design and development, and implement the change in the process. ALBPM Enterprise handles multiple versions of processes, or replaces an existing process with another version. Different versions of the processes are kept isolated and don’t interfere with each other.
Business process management
Based on what I’ve described, you can manage the whole lifecycle of a business process, from inception to runtime. The ALBPM Product Suite was designed with this in mind. Its basic concepts have been there for more than 10 years and represents a pure BPM product.
Figure 1 provides an overview of the main components found in AquaLogic BPM Suite.

Figure 1. BEA AquaLogic BPM Suite components
Everything Counts (Even the Last Words)
If you are looking for tools to help you embrace a process-oriented way of building software for your organization, then ALBPM is the perfect way to let the tools drive you through the process lifecycle. ALBPM was designed from the ground up, with processes and integration being absolutely critical to a successful application. I want to emphasize two things that are fundamental to a complete understanding of BPM:
ALBPM transforms the way you think about solutions.
ALBPM transforms the way your end users work.
This article describes what BEA envisions as a fundamental piece in the software map of the future, about the way applications, development environments, and, most important, businesses will be driven in the future.
In further articles I will discuss the architecture of a BPM solution, including the product architecture, process modeling, and many other interesting topics. Stay tuned.

Using Microsoft Tools for Business Process Management

May 3, 2005 - For the latest information, please see http://www.microsoft.com/biztalk

Introduction

Business processes are dependent and ordered activities that result in predictable and repeatable outcomes. Consisting of an organization’s operating procedures, institutional working knowledge, and information resources, business processes are designed to satisfy defined business objectives in an efficient and timely manner. In an efficient environment, the functional components of a process can be readily identified, adapted, and deployed to address ever-changing corporate requirements—a capability termed as business agility. By definition, business agility is an organization’s systemic ability to fluidly marshal and reconfigure resources in response to business requirements and opportunities. Business Process Management (BPM) tools are designed to provide for such agility by facilitating the creation and execution of highly transparent and modular process-oriented workflows that meet the operational performance standards IT organizations demand.

Automated business processes developed and executed within such an environment are characterized by the following attributes:

• Visibility of end-to-end process activities
• Process components and functionality that are exposed and self-describing
• Ability to integrate disparate information source and application functionality into a process
• Information flow and event notification that can be automated and monitored throughout a process
• Workflow participation that makes the most of desktop productivity and communication tools
• Service level agreements that can be specified, monitored, and enforced for activities in a process
• Ability to add, remove, or reconfigure any process activity or component, without disrupting the process
• Processes that can be monitored in real time or near real time
• Process designs that can accommodate any exception handling requirement
• Processes that can be easily replicated, extended, and scaled

With the support of XML and Web Services, BPM systems are transforming the way in which IT organizations are implementing and executing workflow components. XML applies structure to information, freeing it from any functional dependency on the software that operates on it. Web Services on the other hand provide the framework for application-to-application messaging and invocation over an unbounded network. BPM tools provide the additional support infrastructure to harness these capabilities to create, deploy, and execute the entire scope of workflow management, enterprise application integration (EAI), and trading partner integration (TPI).

This document examines how Microsoft’s tools for Business Process Management and supporting technologies facilitate the creation of processes that share the characteristics defined previously. The paper also describes how XML and Web Services are deployed within a BPM solution to achieve unprecedented modularity and extensibility. Finally, the document illuminates the gains in development and operational productivity that BPM technology engenders, which in turn enables real-time business agility.

A new paradigm

Just as standards-based Web servers and browsers facilitated the communication and distribution of information between people, BPM tools that employ standards-based XML and Web Services technologies will facilitate the wide-scale proliferation of automated and distributed business processes.

A defining characteristic of BPM technology is the elevation of design and development functions from the program layer to the information (document) and transport (messaging) layer. An application is no longer an opaque data-centric procedural construct; it is a messaging event or messaging agent capable of processing the exposed declarative properties of rich (XML) documents. Workflow processes, integration scenarios, or trading partner interactions consist of an orchestrated flow of messages that are routed, transformed, and processed according to message content, formatting requirements, and business rules.

Transparency and modularity are also defining characteristics of BPM technology. Not only are documents and messages exposed, self-describing, and extensible, but so are the communicating endpoint definitions, services, business rules, transformation maps, and process execution instructions that exchange and act upon the messages and documents. A process component can be “loosely coupled” with any other component so that a modification made to one activity or component in a process does not necessitate changes to other activities or components. Each component is functionally independent of any other component.

Microsoft Technology for BPM

Business Process Management technology represents a major conceptual reorientation of the methodologies of workflow development and deployment for business processes. As with any paradigm shift, it must result in significant benefits to be justified. Substantial evidence already indicates that users of BPM technology are achieving dramatic development efficiencies, accelerated returns on their investment, and most importantly, reduced resource requirements. Although XML, Web Services, and BPM platforms impose a new conceptual model on business process development and execution, the technologies required to do so are proven Microsoft® products that have been augmented to support this new paradigm. This section describes the core and supporting component technologies that constitute Microsoft’s process development and execution suite.

BizTalk® Server is Microsoft’s central platform for Enterprise Application Integration (EAI) and BPM and embodies the integration and automation capabilities of XML and Web Services technologies. BizTalk Server functions as a process execution engine and as a multitransport hub for messaging and document transformations.

Visual Studio® .NET is Microsoft’s flagship integrated development environment. The Orchestration Designer module found in previous versions of BizTalk Server is now an integral part of Visual Studio .NET with significantly more functionality. It is a visual development tool for building sophisticated workflows and processes that incorporate business rules, events, transactions, and exceptions and for linking these elements to implementation objects and messaging events. The assembled process generates an XML-based run-time script (BPEL) of the process that is executed in BizTalk Server.

SQL Server is tightly coupled with BizTalk Server and functions as its real-time data store for document tracking information and dehydrated instances of long-running processes.

Office 2003 redefines the functional concept and capabilities of Word and Excel by making XML the native file format. The applications can now behave like network clients, in the manner of a Web browser or e-mail client, but are capable of far more sophisticated and automated interactions with any source of XML information. Furthermore, in Office 2003, Microsoft also introduces InfoPath, an XML-based form application designed to address complex workflow documentation requirements. Because these applications can generate and decode XML documents with their respective schema definitions and processing instructions, they can directly engage in event-level interactions with BizTalk Server.

Active Directory® provides automated and federated authentication and authorization facilities as part of the process development and execution architecture. Its authentication and authorization capabilities facilitate sophisticated workflow processes that involve multiple participants, applications, documentation flows, and information sources. Active Directory also defines role-based participation attributes for workflow activities.

Host Integration Server and BizTalk Server Adapters facilitate integration with enterprise applications, legacy networking and transport protocols, and numerous data formats.

Microsoft Operations Manager and Application Center provide data center class system tools for building, monitoring, and scaling high-performance, mission-critical deployments of BizTalk Server and its supporting technologies.

Microsoft is at the forefront of XML, Web Services, and Business Process Management development and is committed to the implementation of these enabling technologies throughout its products. Nowhere are the potential capabilities of XML and Web Services more evident and maximized than within Microsoft’s integration, development, and productivity technologies. The core XML and Web Services capabilities found in the new releases of BizTalk Server, Visual Studio .NET, Visio, and Microsoft Office 2003 demonstrate a coherent vision for distributing EAI and BPA development and deployment activities, both along functional lines and among stakeholders. The details of this vision become apparent through an examination of the new features and functions of the component technologies described previously.

BizTalk Server 2004

BizTalk Server is the central product of the Microsoft Enterprise Application Integration (EAI) and Business Process Management (BPM) toolset and embodies the application integration and process automation capabilities of XML and Web Services technologies. BizTalk Server has two core functions:

• A process execution engine that manages the steps, applies the business logic, and invokes the supporting applications of a complex process and/or transaction set.
• A multitransport messaging hub for transforming and routing information to and from applications and process steps.

The release of BizTalk Server 2004 represents Microsoft’s third generation of BPM technology. The initial introduction of BizTalk Server 2000 demonstrated Microsoft’s early leadership in defining BPM functionality and supporting XML. BizTalk Server 2002 provided feature set refinements and performance enhancements. The new version of BizTalk Server is a major upgrade that incorporates the recommendations of thousands of users. BizTalk Server 2004 offers many new features and has been reengineered to provide substantial upgrades including improved performance and monitoring of process execution, robust Visual Studio .NET integration for programmatic control, and enhanced workflow modeling capabilities.

By leveraging the application integration and process automation capabilities of BizTalk Server and Microsoft Office, Microsoft has adopted the W3C XML Schema Recommendation. XML Schema is a specification that formally defines an extensive array of data type primitives and structural components for creating XML documents; it serves as a dictionary of abstract elements, attribute entities, and organizational rules. Creating XML documents that conform to the schema delivers a significant advantage: the meaning, function, and application of the document’s content is comprehensible to, and operable by, any XML-enabled application that can access the document’s underlying schema.

Every industry-specific initiative to develop a common vocabulary and set of procedures for the exchange and processing of information is based on XML Schema. By using XML Schema internally, BizTalk Server can import any XML schema definition natively without translation. This significantly reduces the time and effort required to facilitate sophisticated, interactive business scenarios—particularly those that involve the exchange of multiple document types. For example, consider the emergence of generic business-to-business (B2B) framework specifications for standardizing document content and data exchange procedures and behavior. There are typically hundreds of document types and exchange scenarios identified in each initiative specification, all of which are defined using XML Schema. By supporting XML Schema natively, the BizTalk Server messaging infrastructure and process execution engine can effectively support any B2B framework initiative.

Office 2003

XML Schema also figures prominently in the release of Microsoft Office 2003 as Word and Excel adopt XML as a native document format using an XML schema definition file. Also in Office 2003, Microsoft introduces InfoPath™, an XML-based form tool designed to propagate automated workflow capabilities throughout an organization. InfoPath allows workflow participants who create new information or perform analytical or content collection functions to generate, interact with, and exchange structured information. Typically, these activities are paper-bound or use a digital representation of paper. A form created in a word processing or spreadsheet program can be filled out easily enough, but the information that is entered in the form cannot be understood or processed without programmatic or manual intervention. Generating, conveying, extracting, manipulating, and reorganizing unstructured information in and from these formats is extremely labor intensive, inefficient, and costly.

InfoPath addresses this problem by creating smart forms that generate structured XML information, including processing instruction metadata. Underlying an InfoPath form is a template that incorporates one or more XML schemas, XSLT style sheets, embedded controls, and business logic instruction sets. The template controls the behavior of a form in the following ways:

• Generates multiple form views of the same information
• Constrains data types and values that can be entered in a form
• Defines and controls the dependencies for entering information
• Generates automatic, derived, and computed values
• Invokes events, prompts, and instructions based on dependencies
• Provides access to remote information sources
• Enables the incorporation of digital signatures

Form templates are created in a WYSIWYG design tool and require no procedural programming, predefined XML schemas, or XSLT style sheets. The schema and processing instructions are implicitly defined as the form template, which is represented in XML, is constructed from a palette of drag-and-drop controls, wizards, and dialog boxes. When an InfoPath form is populated, it generates an XML document that contains the entered and derived information, required metadata, processing instructions, and digital signatures of the participants who have accessed the form. The document also includes references to the template schemas and XSLT files required by other XML-enabled applications.

By enabling Microsoft Office applications to automatically recognize and process the information and metadata contained in a document, they become capable of engaging in event-level interactions. These “content-responsive” applications have the ability to facilitate automated workflow processes that take place between applications, and between applications and participants. This is the same fundamental premise of Web Services, and it redefines the functional concept and capabilities of these applications. They now behave as network clients, in the manner of a Web browser or e-mail client, but are capable of engaging in sophisticated and automated interactions with any source of XML information. Participants using these tools still engage in their workflow functions, but much more efficiently because of the elimination of manual processing tasks that are irrelevant to the effective execution of those functions.

Exchanging Information

BizTalk Server provides extensive transport and messaging services such as receive and send locations, adapters, pipelines and publish and subscribe distribution capabilities through a messagebox data store—all of which support content-based routing and processing. These facilities combined with the BizTalk Server document tracking and process monitoring capabilities, provide an overlay infrastructure for managing workflow processes. With BizTalk Server as the messaging and process hub and Office 2003 applications as XML-processing clients, participant involvement in workflows can be orchestrated, monitored, and qualified for reliability and performance metrics. This has the potential to radically alter the dynamics and efficiencies of workflow processing on an enterprise-wide basis.

The methodologies and technologies of process design, implementation, and execution are also undergoing a paradigm shift. As such, Microsoft is introducing key innovations in its BPM tools that will significantly improve process development and deployment. One of these innovations is the introduction of a set of high-level process-design and implementation tools that correspond to the roles of the participants involved in process development. These tools make it possible to graphically construct the business logic model of a process, link the steps in the model to actual implementation agents and components, and then generate an executable run-time instruction set of the finished process model in XML.
Roles of Business Analyst and Developer

Automating business processes is a collaborative activity that takes place between line-of-business professionals and programmers. Because each discipline has its own language and development issues, a communication and procedural divide separates their understanding of the development objectives. Consequently, software development is characterized by recursive revision cycles and ambiguity. Its methodology requires interpreting the apparent objective of a specification and translating it into a highly abstract form. While development notation systems such as UML provide business analysts with a structured approach for documenting use cases and specifications, programmers still have to interpret the documentation and translate its intent into a different language and format.

To take advantage of a business process paradigm that is exposed, loosely coupled, and document-driven, development tools and methodologies that incorporate these concepts are required. In creating these tools, Microsoft offers an alternative methodology for developing process-oriented applications. This methodology eliminates the inefficient interpretation and translation cycles that presently characterize application development.

Microsoft Visio remains the ideal tool for the business analyst. Designed for line-of-business professionals, Visio enables users to create diagrams of a business process by arranging, ordering, labeling, and linking various symbols that represent activities, events, decisions, flow, and transactions. When a process diagram is completed, Visio generates an XML representation of it in a Business Process Execution Language (BPEL) document. The BPEL representation of the designed process is then imported into Visual Studio .NET where each design object in the process will be implemented.

The corresponding product for developers remains Visual Studio .NET. Offering an extensive set of functionality, Visual Studio .NET contains a special project overlay template that provides programmers with a visual workspace. This workspace exposes the “binding” that exists between implementation objects, documents, and messaging infrastructure and the process steps created in the design stage. Though Visual Studio .NET is a programming environment, the method for implementing the process design does not resemble procedural programming application development. Instead, the visual workspace provides a collaborative environment where the programmer and process designer can work together on a diagrammatic object model of the process that both can understand.

Bringing the pieces together

In the workspace, Visual Studio .NET reconstitutes the diagrammatic representation of the business process. In addition to the primitive element design palette found in Visio, it also contains a palette of implementation objects: application integration components, COM objects, Web Services, XML documents, messaging and transport facilities, and transformations. These objects are visually “bound” to the design objects through messaging events or method calls using drop, drag, and connect activities. Furthermore, the implementation mechanisms for highly complex functions, such as transactions requiring two-phase commit, rollback, and so on, are built-in functions— thus eliminating the need to write complicated procedural code.

Because the primary dynamic of BPM exchanges is based on sending, receiving, inspecting, and transforming exposed XML documents, a BPM development tool should be capable of visually mapping the flow and exchange of information in a messaging event associated with a process step. This is facilitated through a special “Data” template that graphically represents the message flow in a process as well as the transformation of, and operations performed on, document elements through the process flow.

This assembly approach to building process applications incorporates the modular, loosely coupled, and exposed paradigm for creating any type of interaction or event. Each implementation binding or messaging event is functionally independent of any other binding or messaging event. A change on the design side does not affect the operation, function, structure, or integrity of the objects on the implementation side. In conventional process development, a complex integration or process scenario is embodied in opaque programming code. That code incorporates the structure of the endpoint objects, the process flow logic, the conversion of data formats, business rules, and the bindings to transport infrastructure. If a modification is required to any one facet of the integration module, the integrity of the entire exchange implementation is compromised. The risk of introducing unexpected behavior when modifying code has always been a pitfall of software development and it accounts for the hesitancy to make ongoing process changes in response to business requirements.

It is no accident that the tools and methodologies for developing business processes exemplify the modular, loosely coupled, and exposed paradigm that characterizes the processes themselves. This paradigm is fundamental to every aspect of the Microsoft BPM environment. The ability to visualize and comprehend complex business logic and its implementation mechanisms makes process development and maintenance infinitely more efficient and manageable—enabling organizations to flexibly adapt to new business requirements and opportunities.

Understanding BPEL

Real-world business processes are complex and incorporate numerous integrity controls: ACID transaction support, stateful persistence of long-running interactions, nested and parallel operations, compensation and exception mechanisms, acknowledgements, and correlation capabilities. As process complexity increases, the value of being able to design, implement, and document highly dependant and sophisticated behavior using a visual assembly methodology becomes more apparent. This is especially the case when processes require modification or one process design serves as the basis for another.

The Business Process Execution Language for Web Services (BPEL4WS or BPEL) is another key standard supported by Microsoft in BizTalk Server 2004. Developed collaboratively by Microsoft, IBM, and BEA Systems, BPEL was designed to orchestrate and coordinate Web Services so they can be engaged in collaborative and transactional behavior. The BPEL specification has been submitted to the OASIS standards body for review and eventual designation as a protocol standard.

While Web Services provide the methodology for application-to-application messaging and method invocation over an unbounded network, by themselves they cannot satisfy the operational requirements of a business process. A business processes is a set of dependant and ordered activities, the execution of which results in a predictable and repeatable outcome in a timely manner. BPEL will enable Web Services to ultimately meet these requirements. BPEL formally defines basic and structured activities that are used to compose sophisticated business processes. Because a BPEL instruction set is an XML representation of a process with a precise language and grammar structure, it provides a readable and understandable instruction set for documenting a process. In fact, the object primitives used in BizTalk Server Orchestration Designer are direct representations of basic and structured BPEL elements such as receive, invoke, sequence, flow, switch, partners, role, link, and source.

Business Rules in Business Processes

Business processes are driven by business rules, and the majority of modifications to a business process life cycle pertain to changes in business rules (as opposed to technology-related modifications). However, because business rules in conventional applications are embodied in opaque programming code, they cannot be accessed or modified easily and without potential disruption to running processes. There’s no argument that isolating business rules from procedural code or any process implementation mechanisms dramatically improves the efficiencies of managing and adapting business processes in response to new requirements or business conditions.

It is therefore of particular significance that, in BizTalk Server 2004, Microsoft introduces the Business Rules Composer. The Business Rules Composer consists of a business rule editor and engine for creating and processing sophisticated rule sets using a forward-chaining inference model. A rule set (or “Policy”) that drives a specific activity or function is created with the Business Rules Composer and becomes a resource object that is referenced in a BizTalk Server orchestration. Transparency and loose coupling governs the creation and implementation of business rules. A rule set incorporated within a BizTalk Server orchestration can be viewed, modified, or replaced both at design and run time, without affecting any other operational aspect of a process or interrupting running instances of the affected process.

Isolating, exposing, and publishing business rule sets as services that can be accessed by any application or process provides one of the most compelling value propositions for the Services Oriented Architecture paradigm. As such, the Business Rules Composer module is one of Microsoft’s most noteworthy products.

BPM and the Service Oriented Architecture

The next era of computing will be characterized by the detachment of information from applications, leading to a widely distributed Service Oriented Architecture.
• The meaning, function, relationships, and presentation of information will be self-describing and embedded in the information itself, using schema vocabularies and style sheet references.
• Information will be generated and published without knowledge of how it will be consumed or used.
• Applications will be capable of consuming information and the methods of other applications, while being consumed themselves.
• Processes will be self-configurable and self-modifying based on event-level interactions between rule sets and information.
• Entirely new applications and business models will evolve from this paradigm.

Microsoft’s BPM tools and the XML technologies on which they are built introduce the next era of computing based on the Services Oriented Architecture paradigm. In examining the innovations being introduced in Microsoft’s BPM toolset, it becomes apparent how cumulative XML functionality accrues: XML Schema enables Web Services and InfoPath. XML technologies embedded in BizTalk Server enable an entirely new model for application integration and process management. Each manifestation by itself has significant value, but combined, they offer the potential to facilitate wholesale efficiencies and innovative solutions. It is an object lesson in the whole being greater than the sum of the parts.

Operational Support for BPM Technology

It is one matter to have a vision of how things can work; it is another to actually make that vision work. The innovations engendered by XML require operational support context. On their own, they cannot be simply inserted into an organization’s existing infrastructure and expected to provide functional efficiencies, or to conform to the operational performance standards to which IT organizations are accustomed. Their value is actualized when implemented within a framework of complementary and supporting technologies that facilitate their use within an embedded infrastructure. Just as InfoPath and BizTalk Server complement each other, Microsoft’s array of proven enterprise applications (SQL Server, Active Directory, Host Integration Server, Microsoft Operations Manager, and Application Center) complement and support the implementation of these applications in a real-world, mission-critical context.

SQL server provides the transaction record data store for all BizTalk Server messaging events as well as dehydrated instances of long-running processes. Because SQL Server directly supports XML, it can maintain a record of the document instance associated with each message event as well. In this capacity SQL server provides the persistence capabilities of BizTalk Server.

The BizTalk Server application integration methodology is based on converting the native file formats of legacy and packaged applications into an internal XML representation, which is then transformed according to the requirements of the integration exchange. A prerequisite of this conversion function is knowledge of the native file formats. Microsoft’s Host Integration Server and other third-party adaptors provide this information to BizTalk Server in a comprehensible and usable way. Furthermore, these products provide transport gateways between supported transport facilities in BizTalk Server and the transport protocols used by legacy systems.

Lastly are the requirements for system performance, availability and scalability. Any enterprise-class application must be engineered to accommodate performance metrics monitoring, operational redundancy, and scalability at a granular level. This fine grain level of performance management, failure prevention, and scalability can be achieved with the support of Microsoft Operations Manager (MOM) and Applications Center. These tools are tightly integrated with all Microsoft server products and provide the operational assurances necessary for enterprise class information processing.

Conclusion

Over the last two years, based on thousands of successful deployments, BizTalk Server has clearly demonstrated that a platform based on XML technologies can dramatically improve the efficiencies and economics of application integration development. With the innovations introduced in BizTalk Server 2004, Office 2003, Visio, and Visual Studio .NET XML technologies will demonstrate an even greater potential to deliver operational efficiencies and benefits—and will, no doubt, assume a prominent role in enterprise computing.

Business Intelligence Meets Business Process Management

Powerful technologies can work in tandem to drive successful operations - Ventana Research 2006

The Corporate Challenge

Whether they involve distributing goods to customers, collaborating with suppliers or coordinating the efforts of employees, business processes are the foundation on which rest a business’s products, services and brands. They are the essential underpinning of an organization’s ability to function. In successful companies, business processes align resources to support the achievement of goals.

Business process management (BPM) is a methodology to ensure that those processes support a common set of goals and objectives. It involves automating and/or improving the effectiveness of process activities, tasks and outcomes to accomplish particular business purposes. Its goals include not only efficiency and productivity but, beyond them, control, responsiveness and improvement. Efficiency enables individuals to maximize the time they can devote to priority business tasks and to maximize throughput. Control assures that company resources are aligned optimally to execute strategies. Responsiveness and improvement support the competitive differentiation that enables a company to excel over others.

BPM, then, is about improving processes and implementing systems that enable the improvement. In pursuit of this mission, BPM starts with the following assumptions:

* As a business changes, so must its processes; as a result, they need to be revisited periodically.
* Processes are used by multiple organizations and stakeholders.
* Processes interact with systems and people. Those people can be employees, customers, suppliers or other stakeholders.

To attain the process goals of efficiency, productivity, control, response and improvement, companies first must understand their processes, the needs and skills of the people who use them, the changes that affect them and which areas need improvement. To understand these things, they need relevant information and the ability to analyze and apply it. Business intelligence (BI) tools provide this information, its contexts and appropriate analytics. BI delivers information that, when linked with BPM, gives people the input they require to improve business processes.

This last point is key: Both processes and tools are subordinate to the end of empowering people, at whatever level they occupy, to make informed business decisions that execute corporate strategy, improve performance and ultimately produce the best possible results.

Separation Inhibits Decision-Making

To benefit from improved process management and decision-making, however, companies first must bring together these two business enablers. Most organizations use BI and BPM technologies to serve separate purposes that seldom overlap. For the most part, BI deployments don’t focus on process, and BPM technology doesn’t provide metrics or an aggregate view of business. This situation reflects the predominant view that these are different technologies that each stands alone, delivering value to the business each in its own way.

This view is shortsighted, and soon it will be antiquated. Dynamic enterprises are beginning to forge links between processes and data, particularly in the areas of operational BI and decision-intensive processes. They are linking these two seemingly incompatible technologies in three distinct ways.

First, when initially implemented, most processes are sketchily defined. BI can generate analytics that help analysts tune the process to serve its purposes better. For example, process analytics can calculate measures such as the average process cycle time, durations of individual activities and time lapses between activities. Typically, BI used in this way will report on data about both transactions and process instances. This data becomes post-process feedback that can guide the process of tuning.

Second, most business processes manage the flow of data through an organization and involve decision points at which human interaction is required. For example, sales order processors need to obtain account approval before orders can be fulfilled. Business intelligence coupled with enterprise business process management can place the right data in the hands of the right users exactly when they must make a decision.

Third, many business processes are kicked off by information generated from BI. For example, trade promotion managers use BI to track the effectiveness of promotional activities. Their measurements help shape the next steps of the process. If the activities prove ineffective, managers can act to create more awareness or decide to bring the promotion to an end. In any case, decision-makers initiate the chosen business response process based on the BI information. The inputs gained through BI and BPM empower groups and individuals to execute strategy effectively and streamline tactics.

BI Sharpens Business Processes

The ultimate goal of business process management is performance management: managing the performance of the organization and its business network by using all assets in ways that achieve a common set of goals and objectives. Performance management enables all individuals to work across strategic, tactical and operational levels to align actions so that the organization produces rapid, effective responses to business challenges.

To connect processes with performance goals, companies need business intelligence capabilities, including metrics, key performance indicators (KPIs), executive dashboards and advanced reporting capabilities. They must go beyond just providing reports of basic operational metrics to facilitating access to aggregate data definitions and real-time information. Without BI, it is impossible to correlate process outcomes to corporate performance goals or to apply operational metrics to continuous process improvement.

BI is also a critical tool for analyzing process data. Without it, it’s virtually impossible to measure, evaluate and control business processes. Through process performance reports, BI gives business process managers the means to measure process execution as well as to gain insight into future workflow design improvements. BI tools map out process metrics such as throughput and flow rates in process modeling and then can be used to measure actual process execution. This data can be aggregated from the process implementation and displayed in real time in performance dashboards and reports.

Like corporate performance management systems, BI in a BPM environment supports high-level strategic metrics as well as drill-down analytics and sends alerts when process performance results begin to deviate from targets. In addition, by providing a platform for rule-triggered actions, BI can turn alerts into automated procedures for escalation and remediation in real time, allowing the organization to respond without delay to changes in the business environment. Ultimately, statistics distilled from actual operations can be fed back into the process model to begin the next cycle of performance improvement.

In many cases, business data is far more important than process data for analyzing and optimizing business processes. BI is the critical element of BPM here. To use the example of the mortgage loan approval process, the real-time value of credit scores is critical data for calculating the loan interest rate to be offered. In this situation, integrated business intelligence data is the major ingredient to support key decisions in the process.

BI is also critical to BPM because day-to-day business decisions must be made in the context of business process. BI not only harnesses the power of transactional data created by processes but also creates understanding that leads to action. Order fulfillment is an example of this intersection. In such a case, BI helps make these and other decisions:

* Should we process this order even though it exceeds the customer’s credit limit?
* Should we escalate this order to the regional sales manager because the credit score is low and this customer is a priority client?
* Should we extend more credit to this customer even though he or she has outstanding payments?

These key processing decisions cannot be made efficiently unless the data from each application and each process is incorporated in a BPM application. Integrating process and data on a readily available platform enables decision-making in context and in real time.

Integration of BI and BPM also addresses an issue in the relationship of enterprise software to business processes. An application represents a logical grouping of tasks that has been automated with the objective of reducing or supplementing human interaction. A business process is a set of activities performed by people and machines that are necessary to bring about a desired result. Thus, an application represents a subset of the tasks of a business process. The remaining steps of any business process continue to be human activities and interactions. The bottom line is that most IT applications automate only a subset of the tasks in a business process, and often they do so in an inflexible way.

Most applications in widespread use today do not help to streamline processes. In many companies, departments buy niche technology to solve their particular problems. Even though in recent years the application footprint of enterprise resource planning (ERP) has expanded, companies still find themselves with separate applications addressing customer relationship management (CRM), supply chain management (SCM) and supplier resource management (SRM). The problem with these applications is that they are focused on function, not business process. They provide no real linkage between transactions and processes when business processes cross individual systems and departments.

Another problem with functional applications is that they lack integration with each other. Even though enterprise application integration (EAI) technology can connect systems at critical data points, it really helps automate only transactions – not processes. Thus, integration, if used alone, actually can become a barrier to getting desired results.

Limits of Traditional BI

BI tools are widely used to complement enterprise applications. Application-specific BI tools do a great job of surfacing after-the-fact data captured in, say, an ERP or SCM application. Transactional data is stored and aggregated in a repository, such as an operational data store, for reporting, analysis and visualization. Application BI comes preconfigured to support the data schemas, application programming interfaces (APIs), security mechanisms and business logic of the particular transactional system. This approach speeds the extraction of information and helps maintain the integrity of source applications. Unfortunately, it also limits the scope of BI in decision-making.

BI reporting functionality often is included with an ERP or CRM solution, for example, in predefined or “canned” reports specific to that one application. The reports come from data without its process context, which typically spans multiple applications. Processes such as mortgage approval or call center routing, for example, require information from multiple applications and data sources. So the usefulness of the application-based reports is limited. Reporting embedded within each solution cannot satisfy the requirements of combining data and processes from disparate applications.

Another example of unnecessarily limited BI utilization occurs when data from multiple applications is homogenized in a data warehouse. In this situation, reporting and analytics provide enterprise-wide visibility, but the data loses business applicability – that is, the ability to facilitate collaboration and support decision-making in the context of a specific business process. An example of this is the different ways customer data appears within a data warehouse and outside of it. Data homogenized from sales force automation software (sales opportunity data) and a separate order management system (customer order data) provides good insight into customer buying trends and fits well in the structure of a data warehouse. However, it is missing the granular customer contact data required for the business process of escalation.

Traditional BI also has a latency problem: The information is not available quickly enough to support individuals making immediate decisions. Often the stored data is out of date for operational response; for example, a bank today needs to complete loan approvals in hours, not days. Hourly and near-real-time information is important for operational responsiveness to such competitive pressures.

Also, traditional BI information is not delivered in the context of a business process, but rather at the individual, entity or activity level. Information should be in the context of causal activities preceding a situation and consequential activities following it.

The latency and context issues compound another problem: BI information is not always actionable. Individuals receiving analytic information also need options for possible next steps. For example, call center managers must be given information that is timely (such as that call volumes are in excess now), and notification must be in a form that supports an active response (telling where to route those calls).

Even if the problems of integration, collaboration, latency, context and actionability in business intelligence usage were solved, corporate executives still would not have the complete foundation they need. The ultimate goal of BPM is performance management, and a key component of performance management is alignment: the process of linking strategy with corporate goals and objectives in a way that makes best use of the company’s resources by coordinating the efforts of every member of the organization. Alignment links strategies and goals across the organization from the present into the future. In other words, performance management is about managing the business moving forward, not just looking at past results.

Here traditional BI falls short in that it looks backward at results and analyzes what happened. While understanding the past is a necessary part of the performance management cycle, dynamic businesses need to look also at what is happening now – that is, to get real-time data about events and processes. And they need to project what will happen – to be able to anticipate the consequences of future choices, actions or plans. All these time-oriented steps are needed for good performance management, and so for proper business process management.

Plan for BI with BPM

Companies should do more with their BI deployments. Most businesses do not set performance targets and goals for processes and then monitor those. Nor do they map strategy to business processes.

Henry Ford did it. His strategy was to produce cars at a lower cost by increasing the throughput of production. He divided automobile production into specialized tasks and set time standards as target goals for each, to be able to monitor production rates. Concurrently, he employed accounting methods to measure the favorable and unfavorable cost impacts of faster or slower production speeds. By employing these methods he could optimize production to meet his cost goals.

Ford’s methods illustrate how setting corporate performance targets and goals trickle down to performance goals and targets for business processes. Today’s companies can use modern methods to align their own processes with goals.

To assure that process performance is connected to results, organizations need to include BI at each stage of the process performance life cycle. That life cycle includes process planning, execution and management.

At the first stage, “planning performance,” managers need to define, in order, the desired outcomes of a business process, the resources needed to support it and the strategy by which to achieve the outcomes. This should be a cross-functional exercise that involves all managers and line workers who participate in the process. They should set expectations via the creation of goals and planned outcomes and record these goals and expected outcomes for future plan-vs.-actual comparison. The next step is to determine resources and support requirements. Plans will become operational once the business rules and individual process steps that make up the overall process all are defined. In this first phase, managers make sure that the resources, structure and support are in place for proper process execution.

At stage two, “executing performance,” each step of the business process is carried out exactly as the plan and the rules specify. At each step, inputs, outputs, messages and process data are recorded to monitor the behaviors of users of the process.

At stage three, “managing performance,” managers analyze process outcomes and process behavior data for deviations from the performance plan and to determine what action needs to be taken. This action can be corrective, preventive or sustaining. Typically, business process managers give feedback to the process participants about the execution, and the process is changed as appropriate to pursue the targeted outcomes more effectively.

Most process managers stop here. But they ought to go further and fully close the performance management loop by providing feedback to stage one (planning performance). If processes are to be changed, new business rules, expectations, resources and support also must be determined and new process outcome expectations recorded. All these steps must be followed if optimal performance is to be attained.

To ensure that the goals are met, business unit and IT process managers should apply BI and BPM technology jointly at each stage of the process performance life cycle. In stage one, business and IT process planners need a flexible BPM modeling tool that meets both their needs. While the logic of an automated business process can be implemented directly in a programming language or script, successfully creating an automated business process usually requires collaboration between software developers and business users. For the most part, business-side people should be responsible for designing, maintaining and managing business processes. But since they aren’t programmers, they need to be able to do the design work in a simple, graphical environment in which they define process steps (workflow) and process parameters (basic business rules).

Integral to the effective design of these processes is the incorporation of parameters that will capture the performance data or events for later business intelligence reporting, analysis and business activity monitoring. Business and IT process planners must know this and work together to bake these key capture points into the process. They also must work together to create visual representations of how strategies map to business processes. Here, the same process designers must incorporate BI technology to map key performance indicators and corporate initiatives into the process flow diagram.

At stage two (executing performance) the automated business intelligence data capture defined in the process records key inputs, outputs and process data. It is important to remember that most business processes span multiple applications, so any process messages and transactional data have to be synchronized.

BI technology plays another key role is stage three (managing performance). Here, technology helps close the loop by summarizing performance outcomes. Scorecards, dashboards, alerts and portals play supporting roles with visual representations of the resulting data and events. Scorecards in particular provide an effective way to display process KPIs; they provide easy-to-understand icons such as traffic lights (in red, yellow and green), gauges and heat maps that show the performance of a KPI metric against a target process benchmark. Desktop office applications, particularly spreadsheets, also play a role since they extend performance analysis by serving up detail data, allowing drilling down to more detailed levels of information to spot exceptions and trends.

Convergence Improves Decision-Making

When BI and BPM are linked, they enable people to make better decisions – about business processes, within business processes and that initiate business processes. This linkage brings content and context to individuals and teams across the organization. Let’s look at some common situations in which the technologies can work together to improve business decision-making.

Companies can use BI and BPM to analyze and tune the efficiency of a business process. For example, businesses that process insurance claims must establish connections within a multisource data entry process that deals with thousands of claims per day. Claim processing moves a claim application from entry to validation to authorization to approval and ultimately to payment. The company needs to manage the time it takes to collate data from its many source points and to incorporate rigorous screening checks. Process designers should identify all points at which to customize claims, to perform additional checks and to establish what authorizations and business rules apply. To measure efficiency and effectiveness, process planners need to analyze the business model, identify bottlenecks and make improvements before executing the modified business process. BI provides these analytics for process efficiency and effectiveness by reporting on cycle times and process constraints.

A second example of BI and BPM convergence can be found in the order fulfillment process. Here, some process decisions are based on business data. When an order processor has determined that a priority customer has exceeded his credit limit, a trigger initiates an automated escalation process in which an alert is sent to the account manager and regional sales manager, notifying them of a delay in fulfillment. The alert notice can point to the open order (in the transaction system) and the sales history of the account (in the BI application) for them to review. The same alert also can be sent to the credit manager – the decision-maker – who sees not only the open order and the sales history but also the customer’s payment history. In this way business process and business intelligence work together in ways determined by the business role and data security level defined for each individual.

A third example of convergence occurs when BPM manages a process that is kicked off by information generated by BI. The trade promotion management process, mentioned earlier, includes creating a promotion, selling it and evaluating its effectiveness. When the promotion is launched, its process infrastructure should include the steps needed to measure its effectiveness. These measurements help shape reactions to the process as it continues. If the promotion proves to be effective, then a process must be started to deliver more product units to the retailers. If the promotion proves ineffective, another process has to begin to create more awareness of the promotion or, alternately, bring it to an end. In any case, decision-makers need to see all the appropriate business intelligence reports on the promotion, and from the same screen the marketing manager needs to be able to initiate the chosen business response process.

The Right Approach

To automate and improve business processes involves not only business process decisions but technology choices. Bringing together BI and BPM capabilities need not require a large investment in a single enterprise-wide technology. It is possible to procure a solution that can be implemented quickly without sacrificing the ability to scale and then built out incrementally. For many companies, this will represent the most manageable and cost-effective choice.

Business process management software should be able to integrate systems, trading partners and business users through flexible business processes. But as information technology continues to evolve, companies acquire an increasing number of systems and need to add more business users – and both these steps increase the challenges of implementing BPM.

The challenges begin with trying to get the several systems to work compatibly, a task that expands with each new business need and software upgrade. In addition to an increasing number of points of connectivity, business drivers such as just-in-time inventory and real-time reporting have introduced demands for accessibility and visibility into the processes that span these connected systems. BPM products have been in the market for more than a decade, but many are expensive and difficult to use and address only a single aspect of a company’s needs. What’s more, often they are hard to integrate with other BPM and BI technologies.

Typically, a company realizes the best return on investment when BI and BPM are brought together and integrated with a technology platform that enables joint development, management and interaction. Customers that desire to realize the benefits of BPM and BI technologies should consider the extensibility of the software platform as well as the degree to which the BI and BPM components are integrated. The platform also should extend to tools in everyday use, such as Microsoft Office. The best solution will make the most of both existing and new investments.

Business Process Management Overview

July 12, 2006 - Microsoft Corporation

What is Business Process Management?
A business process is a set of linked steps or activities that taken together result in a specific business outcome, either internal or external to the organization. Documenting a business processes involves describing what is done, why it is done, how it is done, who (or what system) does it, as well as how well it is done.
Business processes may be structured or unstructured, depending on the extent to which the underlying steps are fixed and therefore automated or changeable and generally executed by people or people interacting with systems.
Business processes underlie all organizational efforts, and the effectiveness with which they are carried out contributes directly to critical business goals such as customer retention, length of time it takes to fulfill a product order or service, or regulatory compliance.

BPM Defined
Business process management (BPM) is a management discipline that combines a process-centric and cross-functional approach to improving how organizations achieve their business goals. A BPM solution provides the tools that help make these processes explicit, as well as the functionality to help business managers control and change both manual and automated workflows.
Business process management has its origins in total quality management and business process reengineering. While it adds to these a technological framework, it is more than just the combination of these disciplines. BPM is an IT enabled management discipline that promotes organizational agility and supports the efforts of people to drive process change and rapid innovation. As such, BPM supports the alignment of IT and business activities both within the organization and with business partners and suppliers.
Although early implementations of BPM have unfolded in large enterprises, managing business processes is critical for any sized organization that would benefit from greater visibility into and control over the processes that support their business goals.

Who does BPM?
Business process management is inherently a cross-disciplinary exercise involving personnel from all areas of the organization—from the process owners who are responsible for getting the day to day operational work done, to the department heads who are responsible for managing divisional areas, to the CXOs of the organization providing oversight and direction. That said, most organizations appoint or hire a point person to oversee the BPM process. That person, often referred to as a business analyst or process architect, generally comes from the business side of the organization, but nevertheless has a strong understanding of IT, critical to effective liaison with the IT department.

What BPM Is Not
While business process management endeavors make use of IT technologies to automate and increase efficiencies, BPM is not an amalgam of existing technologies cobbled together and labeled BPM. Nor is it a form of application development which in which business requirements are thrown to IT to interpret and develop, a solution emerging only after extended time and often with poor success translating business requirements into functionality.
BPM is a fast, agile process in which business and IT work together using tools that enable them to arrive at solutions that not only support the organization’s efforts right now, but provide the framework enabling rapid adaptation to new challenges.

What is the BPM lifecycle?
Every organization’s day to day work practices—from shipping clerk to CEO—support the organization’s ability to remain competitive. Workflow practices that support organizational goals are valuable assets to the organization, a principle made more obvious when you consider the knowledge that needs to be exchanged for successful outsourcing or mergers. Not all workflow practices are valued for the same thing: in some cases the most effective practice is also the most streamlined and efficient; in others, the practice that best supports organizational goals is one that enables the greatest ingenuity and innovation.
Driving improvement in business processes has the advantage of carrying the same underlying logic of process improvement and total quality management initiatives. The difference, as we have already seen, is tying together processes that span unstructured to highly structured work—from people to systems—with agile software support.
We see the iterative stages of BPM lifecycle as divided into the stages shown in the diagram: planning, model and design, develop and deploy, manage and interact, analyze and optimize. Preceding the onset of the BPM lifecycle with a planning stage helps to ensure success.

Planning
Business process improvement doesn’t just happen, and as a strategist in the business world, it’s evident that attempting to implement organization-wide changes without first culturing buy-in and support is almost guaranteed to simply waste resources and sour employee good feeling.
Businesses need to focus their initial efforts on understanding their current business practices and planning on the best course of action to improve their current situation.
The initial planning stage consists of identifying and prioritizing a short list of candidate BPM projects, identifying key players whose input is critical to project success, and establishing the governance to ensure that the BPM project stays on track throughout all of the iterative stages of the cycle.

Choosing the Business Process to Improve
Which business processes are good candidates for an initial BPM project? Look for two things. Those processes that have considerable impact on an organization’s ability to achieve its goals either in the short or long term. And those processes, which if improved, will enable the organization to realize a high return on investment.
Business processes related to productivity can have considerable impact on organizational efficiencies in the near term. Look at areas where there are known complaints, whether from customers, trading partners or internal staff. At the root of these complaints are processes that are ineffective and require reworking, streamlining, or better management.
In those business areas where today’s actions have an impact 12 to 18 months down the road, again prioritize those processes that are in greatest need of rework to better support such goals as innovation, growth, and regulatory compliance.
Because organizational buy-in is so critical to the success of business process management, the project that has the greatest support for the BPM process may be the one your organization chooses to go with. A successful BPM experience in one area of the organization will help create enthusiasm for subsequent projects in other areas.
Having completed the preliminary planning, the iterative stage of the BPM lifecycle begins with formally modeling the business process and beginning design work to build an effective solution.

Model and Design
Modeling business processes that span people and systems within the organization, as well as those that reach across to the business partners in the supply chain, is the critical first step of the iterative cycle of BPM. Why? Because modeling directly supports a number of goals, first and foremost of which is to make explicit the opportunities for how the business process might be improved.
Modeling enables you to clearly and explicitly lay out each step in a business process, including the critical touch points across people and disparate systems. It is this mapping that enables informed process improvement; in addition, detailing processes through rigorous documentation enables organizations to meet other goals, including effective outsourcing and regulatory compliance.
A business analyst—usually someone with a foot both in business and IT—oversees the model and design process. During the modeling phase, the analyst works first with process owners and end users to formally define and document the target business process. This stage maps the process from end-to-end, not only across functional divisions (such as finance or marketing) within the organization, but also across to relevant business partners outside of the organization. More than simply a mapping process, modeling also makes explicit the business rules that underlie the various steps in the process—how each of the steps relate, constrain or derive from others. During this modeling stage, stakeholders are encouraged to share suggestions on how changes to the workflow process can better support business goals.
Once all the steps of the target business process has been defined and documented (and often graphically represented in a model), the analyst works with IT to determine how IT can design improved support. This means looking at the entire business process to assess where integration, automation or workflow redesign will improve efficiencies or increase agility. Obviously not all steps in a business processes (particularly those steps that represent highly unstructured human workflow processes) will benefit from being linked to an underlying technology; for the many that will, such linkage is the primary focus of the develop and deploy stage of the BPM lifecycle.

Develop and Deploy
Armed with the detailed model of the business process and the underlying business rules, the IT developer maps the business needs onto the underlying technologies that contribute to the complete solution. These technologies include line of business applications and messaging systems, the former supporting highly structured workflows and the latter unstructured workflows.
Using BPM tools to uncouple business rules from their underlying technologies, the IT developer abstracts these rules to a layer independent of the systems and applications, then joins the logical components into a “composite application” that combines the functionality of underlying systems. Not only can this composite application be rapidly assembled and reassembled, it provides outputs that individual systems alone cannot (such as real-time metrics across the entire business process).
Much of the IT developer’s job during this stage is to ensure that the solution incorporates the functionality, performance metrics, and user interface to meet business user needs. Once the solution is tested and deployed, business users and IT alike begin the manage and interact stage of the BPM lifecycle.

Manage and Interact
With the BPM solution for a specific business process in place, end users interact with the process as it runs though the various process stages, providing or requesting information (though form-based interfaces) as appropriate. At the same time, business users can monitor for potentially disruptive events along the various steps of the workflow process (such as an event indicating low stock of a production item), and take appropriate action as required.
In the background, IT staff manage the entire automated business process solution to ensure that the running process continues to meet capacity and availability standards.

Analyze and Optimize
The BPM solution provides a number of different means through which the business analyst or manager can continuously monitor and measure improvements both within single and across multiple business processes.
Benchmarks are established using service level agreements (SLAs) and key performance indicators (KPIs). Information derived from performance metrics is critical in driving the iterative process of optimizing the business practices and policies that support organizational goals.
The most effective BPM systems enables IT staff or non-technical business users to optimize of business rules in real-time, an iterative process that enables rule change, versioning and simple execution.

What are the benefits of BPM?
The reasons for embarking on a business process management effort are as varied as the organizations that undergo the endeavor, but most organizations are driven by the following benefits:
• Increased customer retention, gained through better, faster customer sales and services, as well as providing customers better access to resources and information through the various access channels.
• Reduced process time, gained through process optimization and efficiencies. Shorter process cycle times speeds time to market and time to service.
• Improved regulatory compliance, gained through improved process control, regulation and monitoring. One of the most effective ways to move toward compliance is to replace manual processes with automated ones, lowering the risk of errors and security breaches.
• Improved efficiencies across organizational boundaries such as departments, branches and trading partners (including both supply chain and outsourcing). Efficiencies are gained through improved visibility and control, as well as automation and integration.
• Reuse and create new IT assets, through integration with legacy applications and the creation of new composite applications that help to overcome their limitations.
• Greater personal productivity and satisfaction, resulting from greater insight into processes and improved workflow.
• Reduced risk, reduced waste and more profitable allocation of human resources.
• Increased agility through compression of BPM lifecycle, allowing for more rapid process innovation and response to changing business conditions.

What are the challenges associated with BPM?
Successful business process management initiatives require both near and long term planning and goal setting, and the goals and means by which to achieve them must be supported by executives across the organization. Without clear alignment on the goals and commitment to the BPM process, organizational resistance will defeat the initiative.
BPM requires strong communication both within and across departmental boundaries. Handoffs between departments are notorious for providing the greatest challenge to process improvement, and the BPM process is no exception in this regard.
A strong business analyst or process architect, key to ensuring that the BPM process remains on track, is critical for ensuring effective communication throughout the initiative. It is the individual in this role who ensures continuous alignment between business and IT; failure to keep business personnel engaged in the process can too readily result in IT solutions that fail to meet business goals.
Both the business analyst and the IT developer must ensure that the chosen BPM system works with existing systems and is capable of supporting current business practices. Since one of the primary goals is to integrate processes across both people and systems, the BPM solution must be flexible enough to ensure that integration efforts can meet across the divide that has traditionally separated business and IT.
Having outlined these challenges, it is obvious that the first BPM project will be the hardest: familiarity and expertise with the method is most likely limited, and organizational reluctance (to yet another initiative) will likely be at its greatest. But with the completion of the BPM lifecycle and the delivery of measurable improvements, subsequent iterations to fine tune the process will be smoother, as will the undertaking of new BPM projects.
As the iterative process of BPM unfolds, workers develop a more nuanced understanding of their workflow processes. At the same time, IT gains a more sophisticated understanding of the business processes they are supporting. For their part, business managers develop a better understanding of how technologies can support their needs, which increases their abilities to suggest targeted changes to the functionality of the composite application supporting their needs.

How can your organization get started with BPM?
It is critical that the organization appoint or hire a BPM process architect who understands both business requirements and technical solutions. In addition to garnering organizational buy-in and choosing an initial project that targets a weak process directly impacting the customer, as well as one that offers a high return on investment, it is imperative to choose the right business process management solution to support the BPM process.

Choosing a BPM Solution
The BPM solution market is highly fragmented. BPM enabling technologies span a broad spectrum of activities, but can be generalized as supporting either activities that are unstructured (such as ad-hoc or collaborative tasks) or activities that are highly structured and often transactional in nature. Unstructured or human-workflow activities are supported by tools that center on the information worker; structured (or straight through processing) activities are supported by traditional IT business applications and integration middleware.
Until recently, solutions have been focused on one area or the other. Because of that, it’s important to look for comprehensive BPM solutions that:
• support both human-centric and straight through processing activities
• offer a solution framework rather than simply a point solution (necessitating multiple point solutions)
• support process standards enabling integration across different platforms and with different line of business applications
• support integration across business partners
• support solutions for different industries
• provides business users with the ability to define the business rules that make up business processes, without IT programming
• provides visibility into business processes, enabling real-time monitoring and event management