Using Microsoft Tools for Business Process Management

May 3, 2005 - For the latest information, please see http://www.microsoft.com/biztalk

Introduction

Business processes are dependent and ordered activities that result in predictable and repeatable outcomes. Consisting of an organization’s operating procedures, institutional working knowledge, and information resources, business processes are designed to satisfy defined business objectives in an efficient and timely manner. In an efficient environment, the functional components of a process can be readily identified, adapted, and deployed to address ever-changing corporate requirements—a capability termed as business agility. By definition, business agility is an organization’s systemic ability to fluidly marshal and reconfigure resources in response to business requirements and opportunities. Business Process Management (BPM) tools are designed to provide for such agility by facilitating the creation and execution of highly transparent and modular process-oriented workflows that meet the operational performance standards IT organizations demand.

Automated business processes developed and executed within such an environment are characterized by the following attributes:

• Visibility of end-to-end process activities
• Process components and functionality that are exposed and self-describing
• Ability to integrate disparate information source and application functionality into a process
• Information flow and event notification that can be automated and monitored throughout a process
• Workflow participation that makes the most of desktop productivity and communication tools
• Service level agreements that can be specified, monitored, and enforced for activities in a process
• Ability to add, remove, or reconfigure any process activity or component, without disrupting the process
• Processes that can be monitored in real time or near real time
• Process designs that can accommodate any exception handling requirement
• Processes that can be easily replicated, extended, and scaled

With the support of XML and Web Services, BPM systems are transforming the way in which IT organizations are implementing and executing workflow components. XML applies structure to information, freeing it from any functional dependency on the software that operates on it. Web Services on the other hand provide the framework for application-to-application messaging and invocation over an unbounded network. BPM tools provide the additional support infrastructure to harness these capabilities to create, deploy, and execute the entire scope of workflow management, enterprise application integration (EAI), and trading partner integration (TPI).

This document examines how Microsoft’s tools for Business Process Management and supporting technologies facilitate the creation of processes that share the characteristics defined previously. The paper also describes how XML and Web Services are deployed within a BPM solution to achieve unprecedented modularity and extensibility. Finally, the document illuminates the gains in development and operational productivity that BPM technology engenders, which in turn enables real-time business agility.

A new paradigm

Just as standards-based Web servers and browsers facilitated the communication and distribution of information between people, BPM tools that employ standards-based XML and Web Services technologies will facilitate the wide-scale proliferation of automated and distributed business processes.

A defining characteristic of BPM technology is the elevation of design and development functions from the program layer to the information (document) and transport (messaging) layer. An application is no longer an opaque data-centric procedural construct; it is a messaging event or messaging agent capable of processing the exposed declarative properties of rich (XML) documents. Workflow processes, integration scenarios, or trading partner interactions consist of an orchestrated flow of messages that are routed, transformed, and processed according to message content, formatting requirements, and business rules.

Transparency and modularity are also defining characteristics of BPM technology. Not only are documents and messages exposed, self-describing, and extensible, but so are the communicating endpoint definitions, services, business rules, transformation maps, and process execution instructions that exchange and act upon the messages and documents. A process component can be “loosely coupled” with any other component so that a modification made to one activity or component in a process does not necessitate changes to other activities or components. Each component is functionally independent of any other component.

Microsoft Technology for BPM

Business Process Management technology represents a major conceptual reorientation of the methodologies of workflow development and deployment for business processes. As with any paradigm shift, it must result in significant benefits to be justified. Substantial evidence already indicates that users of BPM technology are achieving dramatic development efficiencies, accelerated returns on their investment, and most importantly, reduced resource requirements. Although XML, Web Services, and BPM platforms impose a new conceptual model on business process development and execution, the technologies required to do so are proven Microsoft® products that have been augmented to support this new paradigm. This section describes the core and supporting component technologies that constitute Microsoft’s process development and execution suite.

BizTalk® Server is Microsoft’s central platform for Enterprise Application Integration (EAI) and BPM and embodies the integration and automation capabilities of XML and Web Services technologies. BizTalk Server functions as a process execution engine and as a multitransport hub for messaging and document transformations.

Visual Studio® .NET is Microsoft’s flagship integrated development environment. The Orchestration Designer module found in previous versions of BizTalk Server is now an integral part of Visual Studio .NET with significantly more functionality. It is a visual development tool for building sophisticated workflows and processes that incorporate business rules, events, transactions, and exceptions and for linking these elements to implementation objects and messaging events. The assembled process generates an XML-based run-time script (BPEL) of the process that is executed in BizTalk Server.

SQL Server is tightly coupled with BizTalk Server and functions as its real-time data store for document tracking information and dehydrated instances of long-running processes.

Office 2003 redefines the functional concept and capabilities of Word and Excel by making XML the native file format. The applications can now behave like network clients, in the manner of a Web browser or e-mail client, but are capable of far more sophisticated and automated interactions with any source of XML information. Furthermore, in Office 2003, Microsoft also introduces InfoPath, an XML-based form application designed to address complex workflow documentation requirements. Because these applications can generate and decode XML documents with their respective schema definitions and processing instructions, they can directly engage in event-level interactions with BizTalk Server.

Active Directory® provides automated and federated authentication and authorization facilities as part of the process development and execution architecture. Its authentication and authorization capabilities facilitate sophisticated workflow processes that involve multiple participants, applications, documentation flows, and information sources. Active Directory also defines role-based participation attributes for workflow activities.

Host Integration Server and BizTalk Server Adapters facilitate integration with enterprise applications, legacy networking and transport protocols, and numerous data formats.

Microsoft Operations Manager and Application Center provide data center class system tools for building, monitoring, and scaling high-performance, mission-critical deployments of BizTalk Server and its supporting technologies.

Microsoft is at the forefront of XML, Web Services, and Business Process Management development and is committed to the implementation of these enabling technologies throughout its products. Nowhere are the potential capabilities of XML and Web Services more evident and maximized than within Microsoft’s integration, development, and productivity technologies. The core XML and Web Services capabilities found in the new releases of BizTalk Server, Visual Studio .NET, Visio, and Microsoft Office 2003 demonstrate a coherent vision for distributing EAI and BPA development and deployment activities, both along functional lines and among stakeholders. The details of this vision become apparent through an examination of the new features and functions of the component technologies described previously.

BizTalk Server 2004

BizTalk Server is the central product of the Microsoft Enterprise Application Integration (EAI) and Business Process Management (BPM) toolset and embodies the application integration and process automation capabilities of XML and Web Services technologies. BizTalk Server has two core functions:

• A process execution engine that manages the steps, applies the business logic, and invokes the supporting applications of a complex process and/or transaction set.
• A multitransport messaging hub for transforming and routing information to and from applications and process steps.

The release of BizTalk Server 2004 represents Microsoft’s third generation of BPM technology. The initial introduction of BizTalk Server 2000 demonstrated Microsoft’s early leadership in defining BPM functionality and supporting XML. BizTalk Server 2002 provided feature set refinements and performance enhancements. The new version of BizTalk Server is a major upgrade that incorporates the recommendations of thousands of users. BizTalk Server 2004 offers many new features and has been reengineered to provide substantial upgrades including improved performance and monitoring of process execution, robust Visual Studio .NET integration for programmatic control, and enhanced workflow modeling capabilities.

By leveraging the application integration and process automation capabilities of BizTalk Server and Microsoft Office, Microsoft has adopted the W3C XML Schema Recommendation. XML Schema is a specification that formally defines an extensive array of data type primitives and structural components for creating XML documents; it serves as a dictionary of abstract elements, attribute entities, and organizational rules. Creating XML documents that conform to the schema delivers a significant advantage: the meaning, function, and application of the document’s content is comprehensible to, and operable by, any XML-enabled application that can access the document’s underlying schema.

Every industry-specific initiative to develop a common vocabulary and set of procedures for the exchange and processing of information is based on XML Schema. By using XML Schema internally, BizTalk Server can import any XML schema definition natively without translation. This significantly reduces the time and effort required to facilitate sophisticated, interactive business scenarios—particularly those that involve the exchange of multiple document types. For example, consider the emergence of generic business-to-business (B2B) framework specifications for standardizing document content and data exchange procedures and behavior. There are typically hundreds of document types and exchange scenarios identified in each initiative specification, all of which are defined using XML Schema. By supporting XML Schema natively, the BizTalk Server messaging infrastructure and process execution engine can effectively support any B2B framework initiative.

Office 2003

XML Schema also figures prominently in the release of Microsoft Office 2003 as Word and Excel adopt XML as a native document format using an XML schema definition file. Also in Office 2003, Microsoft introduces InfoPath™, an XML-based form tool designed to propagate automated workflow capabilities throughout an organization. InfoPath allows workflow participants who create new information or perform analytical or content collection functions to generate, interact with, and exchange structured information. Typically, these activities are paper-bound or use a digital representation of paper. A form created in a word processing or spreadsheet program can be filled out easily enough, but the information that is entered in the form cannot be understood or processed without programmatic or manual intervention. Generating, conveying, extracting, manipulating, and reorganizing unstructured information in and from these formats is extremely labor intensive, inefficient, and costly.

InfoPath addresses this problem by creating smart forms that generate structured XML information, including processing instruction metadata. Underlying an InfoPath form is a template that incorporates one or more XML schemas, XSLT style sheets, embedded controls, and business logic instruction sets. The template controls the behavior of a form in the following ways:

• Generates multiple form views of the same information
• Constrains data types and values that can be entered in a form
• Defines and controls the dependencies for entering information
• Generates automatic, derived, and computed values
• Invokes events, prompts, and instructions based on dependencies
• Provides access to remote information sources
• Enables the incorporation of digital signatures

Form templates are created in a WYSIWYG design tool and require no procedural programming, predefined XML schemas, or XSLT style sheets. The schema and processing instructions are implicitly defined as the form template, which is represented in XML, is constructed from a palette of drag-and-drop controls, wizards, and dialog boxes. When an InfoPath form is populated, it generates an XML document that contains the entered and derived information, required metadata, processing instructions, and digital signatures of the participants who have accessed the form. The document also includes references to the template schemas and XSLT files required by other XML-enabled applications.

By enabling Microsoft Office applications to automatically recognize and process the information and metadata contained in a document, they become capable of engaging in event-level interactions. These “content-responsive” applications have the ability to facilitate automated workflow processes that take place between applications, and between applications and participants. This is the same fundamental premise of Web Services, and it redefines the functional concept and capabilities of these applications. They now behave as network clients, in the manner of a Web browser or e-mail client, but are capable of engaging in sophisticated and automated interactions with any source of XML information. Participants using these tools still engage in their workflow functions, but much more efficiently because of the elimination of manual processing tasks that are irrelevant to the effective execution of those functions.

Exchanging Information

BizTalk Server provides extensive transport and messaging services such as receive and send locations, adapters, pipelines and publish and subscribe distribution capabilities through a messagebox data store—all of which support content-based routing and processing. These facilities combined with the BizTalk Server document tracking and process monitoring capabilities, provide an overlay infrastructure for managing workflow processes. With BizTalk Server as the messaging and process hub and Office 2003 applications as XML-processing clients, participant involvement in workflows can be orchestrated, monitored, and qualified for reliability and performance metrics. This has the potential to radically alter the dynamics and efficiencies of workflow processing on an enterprise-wide basis.

The methodologies and technologies of process design, implementation, and execution are also undergoing a paradigm shift. As such, Microsoft is introducing key innovations in its BPM tools that will significantly improve process development and deployment. One of these innovations is the introduction of a set of high-level process-design and implementation tools that correspond to the roles of the participants involved in process development. These tools make it possible to graphically construct the business logic model of a process, link the steps in the model to actual implementation agents and components, and then generate an executable run-time instruction set of the finished process model in XML.
Roles of Business Analyst and Developer

Automating business processes is a collaborative activity that takes place between line-of-business professionals and programmers. Because each discipline has its own language and development issues, a communication and procedural divide separates their understanding of the development objectives. Consequently, software development is characterized by recursive revision cycles and ambiguity. Its methodology requires interpreting the apparent objective of a specification and translating it into a highly abstract form. While development notation systems such as UML provide business analysts with a structured approach for documenting use cases and specifications, programmers still have to interpret the documentation and translate its intent into a different language and format.

To take advantage of a business process paradigm that is exposed, loosely coupled, and document-driven, development tools and methodologies that incorporate these concepts are required. In creating these tools, Microsoft offers an alternative methodology for developing process-oriented applications. This methodology eliminates the inefficient interpretation and translation cycles that presently characterize application development.

Microsoft Visio remains the ideal tool for the business analyst. Designed for line-of-business professionals, Visio enables users to create diagrams of a business process by arranging, ordering, labeling, and linking various symbols that represent activities, events, decisions, flow, and transactions. When a process diagram is completed, Visio generates an XML representation of it in a Business Process Execution Language (BPEL) document. The BPEL representation of the designed process is then imported into Visual Studio .NET where each design object in the process will be implemented.

The corresponding product for developers remains Visual Studio .NET. Offering an extensive set of functionality, Visual Studio .NET contains a special project overlay template that provides programmers with a visual workspace. This workspace exposes the “binding” that exists between implementation objects, documents, and messaging infrastructure and the process steps created in the design stage. Though Visual Studio .NET is a programming environment, the method for implementing the process design does not resemble procedural programming application development. Instead, the visual workspace provides a collaborative environment where the programmer and process designer can work together on a diagrammatic object model of the process that both can understand.

Bringing the pieces together

In the workspace, Visual Studio .NET reconstitutes the diagrammatic representation of the business process. In addition to the primitive element design palette found in Visio, it also contains a palette of implementation objects: application integration components, COM objects, Web Services, XML documents, messaging and transport facilities, and transformations. These objects are visually “bound” to the design objects through messaging events or method calls using drop, drag, and connect activities. Furthermore, the implementation mechanisms for highly complex functions, such as transactions requiring two-phase commit, rollback, and so on, are built-in functions— thus eliminating the need to write complicated procedural code.

Because the primary dynamic of BPM exchanges is based on sending, receiving, inspecting, and transforming exposed XML documents, a BPM development tool should be capable of visually mapping the flow and exchange of information in a messaging event associated with a process step. This is facilitated through a special “Data” template that graphically represents the message flow in a process as well as the transformation of, and operations performed on, document elements through the process flow.

This assembly approach to building process applications incorporates the modular, loosely coupled, and exposed paradigm for creating any type of interaction or event. Each implementation binding or messaging event is functionally independent of any other binding or messaging event. A change on the design side does not affect the operation, function, structure, or integrity of the objects on the implementation side. In conventional process development, a complex integration or process scenario is embodied in opaque programming code. That code incorporates the structure of the endpoint objects, the process flow logic, the conversion of data formats, business rules, and the bindings to transport infrastructure. If a modification is required to any one facet of the integration module, the integrity of the entire exchange implementation is compromised. The risk of introducing unexpected behavior when modifying code has always been a pitfall of software development and it accounts for the hesitancy to make ongoing process changes in response to business requirements.

It is no accident that the tools and methodologies for developing business processes exemplify the modular, loosely coupled, and exposed paradigm that characterizes the processes themselves. This paradigm is fundamental to every aspect of the Microsoft BPM environment. The ability to visualize and comprehend complex business logic and its implementation mechanisms makes process development and maintenance infinitely more efficient and manageable—enabling organizations to flexibly adapt to new business requirements and opportunities.

Understanding BPEL

Real-world business processes are complex and incorporate numerous integrity controls: ACID transaction support, stateful persistence of long-running interactions, nested and parallel operations, compensation and exception mechanisms, acknowledgements, and correlation capabilities. As process complexity increases, the value of being able to design, implement, and document highly dependant and sophisticated behavior using a visual assembly methodology becomes more apparent. This is especially the case when processes require modification or one process design serves as the basis for another.

The Business Process Execution Language for Web Services (BPEL4WS or BPEL) is another key standard supported by Microsoft in BizTalk Server 2004. Developed collaboratively by Microsoft, IBM, and BEA Systems, BPEL was designed to orchestrate and coordinate Web Services so they can be engaged in collaborative and transactional behavior. The BPEL specification has been submitted to the OASIS standards body for review and eventual designation as a protocol standard.

While Web Services provide the methodology for application-to-application messaging and method invocation over an unbounded network, by themselves they cannot satisfy the operational requirements of a business process. A business processes is a set of dependant and ordered activities, the execution of which results in a predictable and repeatable outcome in a timely manner. BPEL will enable Web Services to ultimately meet these requirements. BPEL formally defines basic and structured activities that are used to compose sophisticated business processes. Because a BPEL instruction set is an XML representation of a process with a precise language and grammar structure, it provides a readable and understandable instruction set for documenting a process. In fact, the object primitives used in BizTalk Server Orchestration Designer are direct representations of basic and structured BPEL elements such as receive, invoke, sequence, flow, switch, partners, role, link, and source.

Business Rules in Business Processes

Business processes are driven by business rules, and the majority of modifications to a business process life cycle pertain to changes in business rules (as opposed to technology-related modifications). However, because business rules in conventional applications are embodied in opaque programming code, they cannot be accessed or modified easily and without potential disruption to running processes. There’s no argument that isolating business rules from procedural code or any process implementation mechanisms dramatically improves the efficiencies of managing and adapting business processes in response to new requirements or business conditions.

It is therefore of particular significance that, in BizTalk Server 2004, Microsoft introduces the Business Rules Composer. The Business Rules Composer consists of a business rule editor and engine for creating and processing sophisticated rule sets using a forward-chaining inference model. A rule set (or “Policy”) that drives a specific activity or function is created with the Business Rules Composer and becomes a resource object that is referenced in a BizTalk Server orchestration. Transparency and loose coupling governs the creation and implementation of business rules. A rule set incorporated within a BizTalk Server orchestration can be viewed, modified, or replaced both at design and run time, without affecting any other operational aspect of a process or interrupting running instances of the affected process.

Isolating, exposing, and publishing business rule sets as services that can be accessed by any application or process provides one of the most compelling value propositions for the Services Oriented Architecture paradigm. As such, the Business Rules Composer module is one of Microsoft’s most noteworthy products.

BPM and the Service Oriented Architecture

The next era of computing will be characterized by the detachment of information from applications, leading to a widely distributed Service Oriented Architecture.
• The meaning, function, relationships, and presentation of information will be self-describing and embedded in the information itself, using schema vocabularies and style sheet references.
• Information will be generated and published without knowledge of how it will be consumed or used.
• Applications will be capable of consuming information and the methods of other applications, while being consumed themselves.
• Processes will be self-configurable and self-modifying based on event-level interactions between rule sets and information.
• Entirely new applications and business models will evolve from this paradigm.

Microsoft’s BPM tools and the XML technologies on which they are built introduce the next era of computing based on the Services Oriented Architecture paradigm. In examining the innovations being introduced in Microsoft’s BPM toolset, it becomes apparent how cumulative XML functionality accrues: XML Schema enables Web Services and InfoPath. XML technologies embedded in BizTalk Server enable an entirely new model for application integration and process management. Each manifestation by itself has significant value, but combined, they offer the potential to facilitate wholesale efficiencies and innovative solutions. It is an object lesson in the whole being greater than the sum of the parts.

Operational Support for BPM Technology

It is one matter to have a vision of how things can work; it is another to actually make that vision work. The innovations engendered by XML require operational support context. On their own, they cannot be simply inserted into an organization’s existing infrastructure and expected to provide functional efficiencies, or to conform to the operational performance standards to which IT organizations are accustomed. Their value is actualized when implemented within a framework of complementary and supporting technologies that facilitate their use within an embedded infrastructure. Just as InfoPath and BizTalk Server complement each other, Microsoft’s array of proven enterprise applications (SQL Server, Active Directory, Host Integration Server, Microsoft Operations Manager, and Application Center) complement and support the implementation of these applications in a real-world, mission-critical context.

SQL server provides the transaction record data store for all BizTalk Server messaging events as well as dehydrated instances of long-running processes. Because SQL Server directly supports XML, it can maintain a record of the document instance associated with each message event as well. In this capacity SQL server provides the persistence capabilities of BizTalk Server.

The BizTalk Server application integration methodology is based on converting the native file formats of legacy and packaged applications into an internal XML representation, which is then transformed according to the requirements of the integration exchange. A prerequisite of this conversion function is knowledge of the native file formats. Microsoft’s Host Integration Server and other third-party adaptors provide this information to BizTalk Server in a comprehensible and usable way. Furthermore, these products provide transport gateways between supported transport facilities in BizTalk Server and the transport protocols used by legacy systems.

Lastly are the requirements for system performance, availability and scalability. Any enterprise-class application must be engineered to accommodate performance metrics monitoring, operational redundancy, and scalability at a granular level. This fine grain level of performance management, failure prevention, and scalability can be achieved with the support of Microsoft Operations Manager (MOM) and Applications Center. These tools are tightly integrated with all Microsoft server products and provide the operational assurances necessary for enterprise class information processing.

Conclusion

Over the last two years, based on thousands of successful deployments, BizTalk Server has clearly demonstrated that a platform based on XML technologies can dramatically improve the efficiencies and economics of application integration development. With the innovations introduced in BizTalk Server 2004, Office 2003, Visio, and Visual Studio .NET XML technologies will demonstrate an even greater potential to deliver operational efficiencies and benefits—and will, no doubt, assume a prominent role in enterprise computing.