Debatching(Splitting) XML Message in Orchestration using DefaultPipeline - BizTalk

{tocify} $title={Table of Contents}

Introduction


In this post we will walk through the process of debatching
an xml
message in Orchestration using pipeline in Biztalk.

I have used the Default XML Receive pipeline to achieve it, but it can also be done by creating a custom pipeline which uses XML disassembler (where you can set the Envelope and Document Schema).


Pipeline is not available in Orchestration like other shapes, thus to use it we need to add reference to following assemblies(which will allow us to use methods in those assemblies) :



  • Microsoft.XLANGs.Pipeline.dll
  • Microsoft.BizTalk.Pipeline.dll

You can browse to the location BizTalk Installation Directory to find above dll's.


Most of the part of this post is borrowed from my earlier post which talks about debatching xml at receive port:

https://tech-findings.blogspot.in/2013/07/debatchingsplitting-xml-message-biztalk.html


Scenario


We receive many item information but its wrapped (Enveloped) , so to process each item we need to unwrap it (remove the envelope and split individual Item message).


Below is what we receive (Input) :

<ns0:Items xmlns:ns0="http://TestingSchemas.ItemEnvelope">
  <ns1:Product xmlns:ns1="http://TestingSchemas.Item">
  <ID>ID_0</ID> 
  <Name>Name_0</Name> 
  <Quantity>100</Quantity> 
  <UnitPrice>100</UnitPrice> 
  </ns1:Product>
  <ns1:Product xmlns:ns1="http://TestingSchemas.Item">
  <ID>ID_0</ID> 
  <Name>Name_1</Name> 
  <Quantity>200</Quantity> 
  <UnitPrice>100</UnitPrice> 
  </ns1<:product gt="" span="">
  <ns1:Product xmlns:ns1="http://TestingSchemas.Item">
  <ID>ID_2</ID> 
  <Name>Name_2</Name> 
  <Quantity>300</Quantity> 
  <UnitPrice>100</UnitPrice> 
  </ns1:Product>
<ns1:Product xmlns:ns1="http://TestingSchemas.Item">
  <ID>ID_3</ID> 
  <Name>Name_2</Name> 
  <Quantity>300</Quantity> 
  <UnitPrice>100</UnitPrice> 
  </ns1:Product>
<ns1:Product xmlns:ns1="http://TestingSchemas.Item">
  <ID>ID_4</ID> 
  <Name>Name_2</Name> 
  <Quantity>300</Quantity> 
  <UnitPrice>100</UnitPrice> 
  </ns1:Product>
<ns1:Product xmlns:ns1="http://TestingSchemas.Item">
  <ID>ID_5</ID> 
  <Name>Name_2</Name> 
  <Quantity>300</Quantity> 
  <UnitPrice>100</UnitPrice> 
  </ns1:Product>
<ns1:Product xmlns:ns1="http://TestingSchemas.Item">
  <ID>ID_6</ID> 
  <Name>Name_2</Name> 
  <Quantity>300</Quantity> 
  <UnitPrice>100</UnitPrice> 
  </ns1:Product>
<ns1:Product xmlns:ns1="http://TestingSchemas.Item">
  <ID>ID_7</ID> 
  <Name>Name_2</Name> 
  <Quantity>300</Quantity> 
  <UnitPrice>100</UnitPrice> 
  </ns1:Product>
<ns1:Product xmlns:ns1="http://TestingSchemas.Item">
  <ID>ID_8</ID> 
  <Name>Name_2</Name> 
  <Quantity>300</Quantity> 
  <UnitPrice>100</UnitPrice> 
  </ns1:Product>
<ns1:Product xmlns:ns1="http://TestingSchemas.Item">
  <ID>ID_9</ID> 
  <Name>Name_2</Name> 
  <Quantity>300</Quantity> 
  <UnitPrice>100</UnitPrice> 
  </ns1:Product>
  </ns0:Items>

But we want (Output) :




      <ns0:Product xmlns:ns0="http://TestingSchemas.Item">

           <ID>ID_0</ID>
          <Name>Name_0</Name>
          <Quantity>100</Quantity>
          <UnitPrice>100</UnitPrice>
      </ns0:Product>
             .
             .
             .
             .
             .


     <ns0:Product xmlns:ns0="http://TestingSchemas.Item">
        <ID>ID_9</ID>
        <Name>Name_9</Name>
        <Quantity>100</Quantity>
        <UnitPrice>100</UnitPrice>
     </ns0:Product>


All right, let's see how we do it:

Steps in creating Solution


1. Create schema - document schema 
create document schema

2. Create the wrapper - Envelope Schema
  • Name the root node, I have named it Items.


Envelope Schema




  • Click on "Schema" and go to Properties Window
  • Set the property "Envelope" as Yes, as this schema will be used as envelope and this is the property which helps disassembler to recognize it.
  • So far good, envelope schema is ready but what about the document which will be wrapped.
properties window
  • In the Property window select "Imports" and click on the ellipsis 
  • You will get Imports wizard pop-ed  out  .
  • Click on add and select the document schema which we created in step 1.(Here it is Item schema)

select schema

  • Click on the "Items" and go to property window and select the property Body XPath and set it to /*[local-name(<)='Items' and namespace-uri()='http://TestingSchemas.ItemEnvelope']. Doing so allows an Items node to contain any number of Item Document.
  • Cool... Now we are ready with the resources, next is to validate schema, build it, sign it  and deploy.


envelope schema

3. Now lets create Orchestration which will receive the enveloped item and debatch/split it using the pipeline.


Debatching Orchestration

  •         Receive shape is configured to receive untyped message i.e. of type System.Xml.XmlDocument (which will receive enveloped message -ItemEnvelope Type).
  •         Then we have a "DebatchScope"  which is of type "Atomic" and Orchestration  is of type Long running. Why Atomic scope? Because we will be calling/executing Pipeline within it and the pipeline is of non- serializable type.
  •         Next we have
  •         It contains following line:  GetPipelineOutput=Microsoft.XLANGs.Pipeline.XLANGPipelineManager.ExecuteReceivePipeline(typeof(Microsoft.BizTalk.DefaultPipelines.XMLReceive),ItemsIn);
  •         GetPipelineOutput is a DebatchScope variable of type: Microsoft.XLANGS.Pipeline.ReceivePipelineOutputMessages
  •       So we are executing the method ExecuteReceivePipeline()  method of XLANGPipelineManager class  which belongs to Microsoft.XLANGs.Pipeline and its output assigned to  GetPipelineOutput.


  •      Then we have a loop shape "UntilLastMessage" and its same as  while loop, below is the condition(till spillited messages are available): GetPipelineOutput.MoveNext(
  •         Next is Construct shape with Message Assignment within it, which has following code:
 ItemOut = null;
          GetPipelineOutput.GetCurrent(ItemOut);

It is here where the splitted single  message is assigned to ItemOut , where ItemOut is a Message variable which is of type Item.xsd
  •        At last we have Send shape which accepts message of type ItemOut and sends it .
  •        Now what, build the project ,sign it and deploy.
4.Now create a Receive and Send port :
ports


  •        Receive Pipeline is PassThruReceive  as we don't want its message type to be detected until the message reaches to ExecutePipeline shape in Orchestration.
Passthrough Pipelin


  •        Now after both the ports are ready, it's time to bind it to the logical ports of our orchestration:
binding ports

5. Start the application and test it, I will drop a envelope message at receive location which we saw at start  . So I should be getting 10 individual xml message at the destination location:

testing



        Will keep on posting as an when I find something to share!!!!!!!!!!!!

If you have questions or suggestions, feel free to do in comments section below !!!


Do share if you find this helpful .......
 
                          Knowledge Sharing is Caring !!!!!!





Related Post 





20 Comments

If you have any suggestions or questions or want to share something then please drop a comment

  1. Hi Mahesh, when i try your example, i get an error... "Use of unconstructed message ItemsOut" when i have declared in the construct message shape the ItemsOut... what can be the issue.. please comment

    ReplyDelete
    Replies
    1. Hi,

      Did you create message "ItemsOut" ?
      And check whether you have done following in MessageAssignment shape:
      ItemsOut = null;
      GetPipelineOutput.GetCurrent(ItemsOut);

      Also check if all the shapes are in Atomic scope.

      Delete
  2. Yes, i created it... I got it resolved now... the issue was, accidentally i had the send shape out of the loop. it working... thanks for your response

    ReplyDelete
    Replies
    1. Hi,

      Even after doing all the above mentioned, am getting error as"Use of unconstructed message ItemsOut" can you please let me know what could be the issue

      Delete
  3. hi
    I am new in biztalk. I follow you step and try to import schema but after I import in envelope schema I can not see imported schema like it is show on you picture. Is there something what have to done to see imported schema?

    dan

    ReplyDelete
  4. Wonderful article.
    It woks fine for me.
    may be a missing detailed explanation (Under items we need to create another record of type any to hold the debached elements) but it should be known to every one

    ReplyDelete
  5. Hi Mahesh,
    Thanks for the such a nice post.But I have one doubt : when I set the receive shape with envelope msg instead of untype msg then I got a error in admin console :
    The published message could not be routed because no subscribers were found. This error occurs if the subscribing orchestration or send port has not been enlisted, or if some of the message properties necessary for subscription evaluation have not been promoted.
    I am confused why it is happening ?

    ReplyDelete
    Replies
    1. Hi,
      Getting the routing failure error .......can anyone help to resolve this please...

      The published message could not be routed because no subscribers were found. This error occurs if the subscribing orchestration or send port has not been enlisted, or if some of the message properties necessary for subscription evaluation have not been promoted.

      Kept Passthru pipeline in Receive end and XML pipeline on Send end....

      In Visual Studio the solution was built successfully but in admin console getting routing failure

      Delete
  6. Hi Mahesh,
    Thanks for the such a nice post.But I have one doubt : when I set the receive shape with envelope msg instead of untype msg then I got a error in admin console :
    The published message could not be routed because no subscribers were found. This error occurs if the subscribing orchestration or send port has not been enlisted, or if some of the message properties necessary for subscription evaluation have not been promoted.
    I am confused why it is happening ?

    ReplyDelete
  7. Hi Prakash,

    Please check the pipeline in receive port. It should be of type XMLReceive.

    Thanks

    ReplyDelete
  8. My Message is not being de-batched

    ReplyDelete
  9. hello Dear,while sending the message to message box (direct binding in send port),i am getting infinite message means infinite loop occuring .Do u know the reason y?

    ReplyDelete
  10. Hi Piyush,

    This is very common scenario -- it happens when you are subscribing to same message which you are sending.
    Say, you are sending msgXYZ to MessageBox and your orchestration first receive shape is also looking msgXYZ. So you send a message and it instantiate another instance,thus this process goes on.

    ReplyDelete
  11. This comment has been removed by the author.

    ReplyDelete
  12. I got the following problem.

    The published message could not be routed because no subscribers were found. This error occurs if the subscribing orchestration or send port has not been enlisted, or if some of the message properties necessary for subscription evaluation have not been promoted. Please use the Biztalk Administration console to troubleshoot this failure.

    I kept Receive pipeline as "XML Receive". As I was not using any Item.xsd(input) variables in orchestration I have not promoted any of them. But there is no way out.

    I tried SEND PORT Filter expression, Mentioned BTS.ReceivePortName == "MyReceivePort" it resolved the issue. As per my understanding this step is not required as we are associating Receive port and Send port to orchestration. But it resolved.

    Any logical explanation will help me to understand better.

    Thanks.

    ReplyDelete
  13. Hi SRA,

    You faced this problem, as you set the pipeline to xml on Receive Port instead you had to set it in Orchestration, as you have to debatch in Orchestration(In orchestration we use xml receive pipeline)

    ReplyDelete
  14. Thanks for your Response. Newbie to BizTalk, pl. bear with me. I have Kept PassThruReceive for Receive Pipeline and Kept XMLTransmit for Send Pipeline as shown in the article. Also executed the following stepGetPipelineOutput=Microsoft.XLANGs.Pipeline.XLANGPipelineManager.ExecuteReceivePipeline(typeof(Microsoft.BizTalk.DefaultPipelines.XMLReceive),ItemsIn). I think is what you mean by "In orchestration we use xml receive pipeline". But the result are not as expected. I don't know where I am committing the mistake

    But the results are not as expected. When I add Send Filter Expression for BTS.Receive Portname results are as expected.

    Thanks.

    ReplyDelete
  15. Hi Mahesh,

    Agree with SAR as I am also facing the same issue.
    Called in Orchestration :
    Set ReceivePipeline as Passthrough
    if not applied any filter on physical send port then it throws no subscriber found error where else if added filter of receiveport name works fine but no instance of orchestration getting completed in dashboard query window

    ReplyDelete
  16. How to use scope and decide shape in orchestration

    ReplyDelete
Previous Post Next Post