Content Based Routing: EAI Challenge, what's the best solution

Recently I was confronted with the following scenario, where I asked the authors of BizTalk Pro for advise:

I am a Microsoft BizTalk Consultant in the Netherlands and I would like to ask you a question about Content-based Routing. The question is does Content Based Routing (CBR) exists inside BizTalk Server 2006 as functionality by itself of is it a design pattern, practice as you wrote in your book (Chapter 6 BizTalk Design Patterns and Practices page 198). The reason I am asking this, because a customer we are doing a BizTalk implementation is wondering if a CBR could be a solution to their specific problem and if this is standard functionality in BizTalk or do they need another solution. Problem they are having is that their flex worker ERP systems exports different kinds of flat-file and xml files (between 20k and 20 Mb large) for the Dutch tax department, bank and so on (3rd parties). Flat files contain a header with metadata what kind of file it is and where it is originated inside the system. The XML files are exported with same metadata in an envelope. These files are send to 3rd parties using FTP, SMTP or SOAP protocol. All these protocols and maybe other protocols in the future need to be supported. We suggested a orchestration with one receive location (folder) to pick up any kind of exported file (flat or xml) and based on its metadata a lookup is done in a database table to find address data and other data like authentication data so a dynamic port is created and the file is send by FTP, HTTP and SMTP or all if necessary. This could also be done using BRE instead of using a database lookup. What we want to achieve through this solution was that with the Meta data any kind of address the customer want can be placed inside a database and the BizTalk application (orchestration) could be left untouched. What would you suggest this solution of CBR?

George Dunphy answered:

Hi Steef,CBR is a feature of BizTalk. This is something that comes with the product when you purchase it. The situation you describe seems like a classic example of where you would use CBR. Chapter 3 we discuss this as well. CBR should be able to parse your document and route it to the appropriate destiantion based on what properties are promoted.Hope that helps.

I also asked some other experts in BizTalk field like Stephen W Thomas, Tomas Restrepo and Richard Seroter. Tomas replied this:

Technically speaking, it is definitely possible to do this using CBR. What you’d need to do is create a property schema and promote the relevant field in the message schema to a property in your property schema, thus making it available for routing. Once that is done, you could just create your send ports and add Filter expressions built on your custom promoted property to enable the routing scenarios.

So definitely this is possible, and in some cases it is far more efficient because all the work is done by the biztalk Pub/Sub engine. That said, there are tradeoffs. Depending on your scenario, using pure CBR might be more complex to manage (all the configuration work/changes needs to be done by the biztalk administrator), and in some scenarios it might be more complex to debug/tracking because any configuration error (or missing configuration) would generate a routing failure, compared to your custom orchestration/db which could implement more advanced error handling/retry mechanisms.

And also Richard Seroter replied:

Your solution description below is perfectly viable. That is, accept data in via any format, use metadata to determine routing. As you said, you could do a straight database lookup in a pipeline, map or orchestration, or, you could use the BRE.Making sure that you could add new lookups without modifying the orchestration (or ANY deployed component) would be key in making this purely content based routing. That said, you'd still have the flexibility to add an orchestration that only subscribes to specific data elements.

Well from these answers I can tell that to my scenario that strictly CBR, without an orchestration but simple filter options is not the best option, besiedes that the customer is looking for routing using different protocols, it also wants logging, errorhandling and audit trail.

I must say that reading the BizTalk Pro book really gave some more understanding of BizTalk. This book and the recipes from APRESS are the best on BizTalk 2006 so far.

Here are the book details:

By George DunphyAhmed Metwally
ISBN: 1-59059-699-4
528 pp.
Published: Oct 2006