E-Book "BizTalk Mapping Patterns & Best Practices" free at Biztalk360

If you're into Biztalk mapping, you need to check Sandro Pereira's book, "BizTalk Mapping Patterns & Best Practices".

In 350 pages (367, but you can skip the first 15), you can learn in detail several tricks of 12 mapping patterns that will help you to be more productive (only if you want, of course).

To download the book, you have to give a valid email.

Replace a schema in a map: the easy way

We had a little problem on the first day in production of an EDI project, when our partner says they have to send their orders in EFACT_D96A_ORDERS_EAN008, instead of the EFACT_D93A_ORDERS_EAN007 we tested for months. Not a big stress. Those schemas are similar, so the change was easy.


To redo a map like this one and properly test it in the same day, was impossible. But the task was simplified in the fastest way. No XSLT knowledge needed, just basic XML and logic.

Supposing the change is in the input

1. Add the new schema or reference to the project.

2. Open the btm as xml.

3. Find and replace all references
3.1. Source Tree Root Node and Location

<SrcTree RootNode_Name="EFACT_D93A_ORDERS_EAN007">

<Reference Location="*.EFACT_D93A_ORDERS_EAN007" />

</SrcTree> 
3.2. Some on the Link elements (LinkFrom attribute).

<Link LinkID="9" LinkFrom="10" LinkTo="/*[local-name()='<Schema>']/*[local-name()='ORDERS01']/*[local-name()='E1EDK03']/*[local-name()='DATAHEADERREC']/*[local-name()='DOCNUM']" Label="" />

        <Link LinkID="10" LinkFrom="/*[local-name()='<Schema>']/*[local-name()='EFACT_D93A_ORDERS_EAN007']/*[local-name()='DTM']/*[local-name()='C507']/*[local-name()='C50702']" LinkTo="9" Label="" />

        <Link LinkID="11" LinkFrom="/*[local-name()='<Schema>']/*[local-name()='EFACT_D93A_ORDERS_EAN007']/*[local-name()='DTM']" LinkTo="11" Label="" />

4. Close the xml view and open the same .btm with the regular view.

4.1. If no Map Loading Errors popped up, it’s done.
4.2. Otherwise, keep following the procedure.

5. Copy all the errors (no Ctrl+A here, just the old mouse dragging).

(1) Link from Source Tree Node with XPath: '/*[local-name()='']/*[local-name()='EFACT_D96A_ORDERS_EAN008']/*[local-name()='LINLoop1']/*[local-name()='QTY_3']/*[local-name()='C186_3']/*[local-name()='C18601']' to Equal functoid (with FunctoidID: 30).
...

6. Close the errors screen but DO NOT save the map.

7. Locate the missing field and take note of the correct path
In this case there was no EFACT_D96A_ORDERS_EAN008/LINLoop1/QTY_3/C186_3/C18601 , but instead, EFACT_D96A_ORDERS_EAN008/LINLoop1/QTY_2/C186_2/C18601

8. Close the map (don’t save!) and open again with the xml view.

9. Replace those fields. [local -name()='QTY_3']/*[local-name()='C186_3'] to [local-name()='QTY_2']/*[local-name()='C186_2']. In case of doubt, use the FunctoidId or any additional info from the errors screen to locate the Link Id.

10. Repeat from step 4 until all the errors are gone.

Changes in Destination schema are in TrgTree instead of SrcTree, and LinkTo instead of LinkFrom.

Please note that this is only an advanced starting point. If the schema was replaced, probably it was because the previous one no longer supported the information needed. It’s very likely new fields are filled and some others have changed, including the ones used within the map. A new batch of tests will show it very quickly.

Extra: Two orchestrations sharing a receive port

If they enter by the same port, use the Message property to receive only one of them.
If they arrive by Direct Binding –independently of the port, from the MessageBox - use the Filter Expression.

Be careful with:

Typename
Don’t give the same name to both. Not only it is unauthorized, but also is the name by which the orchestrations will be known after deployment and so should be different for whoever is managing.

Ports types
The Orchestration 2 will try to create a port type that already exists. Delete the Operation on every port and configure to use the type from Orchestration 1. If the input message is different (source schema changed) the receive port will have a different type. The same goes for the output message (when the target schema changed) at the send port.

Copy name and value with XSLT

I run across a situation where I had to map a huge list of sibling nodes. I was required to send tuples of name (in upper case) and value. If they were few, I would use a Table Looping like always, but this time it was a lot of fields and I was feeling lazy. Here is the solution.

<xsl:variable name="smallcase" select="'abcdefghijklmnopqrstuvwxyz'" />

<xsl:variable name="uppercase" select="'ABCDEFGHIJKLMNOPQRSTUVWXYZ'" />

            <xsl:for-each select="//Data/item/*">

                  <xsl:element name="ns0:GenericDataContainerEntry">

                        <xsl:element name="ns0:GenericDataContainerEntryIndex">

                              <xsl:attribute name="value">

                                    <xsl:value-of select="./text()" />

                              xsl:attribute>

                              <xsl:attribute name="name">

                                    <xsl:value-of select="translate(name(.),$smallcase,$uppercase)" />

                              xsl:attribute>

                        xsl:element>

                  xsl:element>

            xsl:for-each>

      xsl:for-each>

xsl:element>

Do to all nodes

<xsl:for-each select="//Data/item/*">

Copy value

<xsl:value-of select="./text()" />

Copy name

<xsl:value-of select="name(.)" />

Copy name in upper case in XSLT 1.0 (found it here)

<xsl:variable name="smallcase" select="'abcdefghijklmnopqrstuvwxyz'" />

<xsl:variable name="uppercase" select="'ABCDEFGHIJKLMNOPQRSTUVWXYZ'" />

<xsl:value-of select="translate(name(.),$smallcase,$uppercase)" />

Using SAP Dates in Biztalk

Half the maps I've been doing have SAP in one side or the other. One of the biggest problems are in map transformation.

Making all those substring concatenations is ugly. Creating a functoid is the best thing to do, but in case it is needed just in a map or two, how can it be made?

Convert .NET Date to SAP using C#

dateVariable.ToString("yyyyMMdd", System.Globalization.CultureInfo.InvariantCulture);

Convert SAP Date to .NET using C#

DateTime.ParseExact(val1, "yyyyMMdd", System.Globalization.CultureInfo.InvariantCulture);

(the CultureInfo attribute is optional but recommended in both methods)



This problem appeared when I had to calculate a difference between two SAP dates. It end up by being just this:

public int DaysToPay(string val1, string val2) {
DateTime startTime = DateTime.ParseExact(val1,"yyyyMMdd", System.Globalization.CultureInfo.InvariantCulture);
DateTime endTime = DateTime.ParseExact(val2,"yyyyMMdd", System.Globalization.CultureInfo.InvariantCulture);

TimeSpan span = endTime.Subtract ( startTime );

return span.Days-1;
}

Untyped Messages and WCF Services in a real scenario

Context:

This project was my first big Biztalk experience. It started more than two years ago (is still in Biztalk 2006 R2) and most of the work was made based on pre-conceived ideas of what was good, what was easy, what was trustable.
The project kept on growing and at a certain point the desire to trash it all and restart was too strong. This is an example of how to slowly modify a monster-like orchestration. The images and names aren’t always the same because they are not from a case-study, they were all taken from a real project. It consisted of several orchestrations and this post was made when each one of them was in a different stage of transformation so that all situations are represented.

---

In this project we had to deal with a bunch of different XML files, different types, different schedules, and, worst of all, different actions on the destination server.



We could make this by using maps on the send port, but the tracking of the messages was critical so we needed to keep track of the message on every step of the journey. So, an orchestration was the choice.

To reunite everything, the receive port that collects them (from FTP, SAP idocs, etc.) is the entry of the orchestration. The Message schema in the orchestration is System.Xml.XmlDocument so all schemas are accepted with no distinction.
A Decide component help us creating the basic structure for it, with a lot of conditions of type

OriginalMessage(BTS.MessageType) == http://Company/Schemas/BusinessPartnerActivityTypes#ActivityTypes

and a two-way destination port prepared for send/receive schemas of type System.Xml.XmlDocument, we could make it by repeating a lot of components.

For each flow we make

1. Transform Xml (Received message) to original schema

FCBusinessPartnerActivityTypes = OriginalMessage;

2. Map original schema to service schema


3. Transform ServiceSchema to Xml (port schema)

XmlMessage = SMUVASActivityType;

4. Send to Service

5. Receive Response

6. And after some flows made the general picture would be something like

How to make it in a lighter way?

The first step is to remove the transformation.

1. Declare a string variable for the map, called mapName

1.1. Use an expression editor to select the map based on the message type

1.2. Use the Fully Qualified Name of the map

mapName="";
if(OriginalMessage(BTS.MessageType) == "http://Company/Schemas/OrderTypes#OrderTypes"){ mapName="Company.Biztalk.SmuasCommercialModule.Messaging.map_FCOrderTypes_to_SMUVASOrderTypes";
}
if(OriginalMessage(BTS.MessageType) == "http://Company/Schemas/OrderStatus#OrderStatus"){
mapName="Company.Biztalk.SmuasCommercialModule.Messaging.map_FCOrderStatus_to_SMUVASOrderStatus";
}
…

2. Then in a message assignment use the transform method.

mapSystemType = System.Type.GetType(mapName);
transform(XmlMessage) = mapSystemType(OriginalMessage);

3. The orchestration will be much simpler.

The next optimization is to remove the multiple actions.

At the if that defines the map, you can also define the action. Something like

if(OriginalMessage(BTS.MessageType) == "http://Company/Schemas/Banks#Banks"){
mapName="Company.Biztalk.SmuasFinancialModule.Messaging.map_FCBanks_to_SMUVASBanks"; action="http://SmuAs/Services/BackendServices/CommonBackendSyncService/ICommonBackendSyncService/SetBankList";
}

And when the request message is created, give it the correspondent action.

mapSystemType = System.Type.GetType(mapName);
transform(RequestMessage) = mapSystemType(OriginalMessage);
RequestMessage(WCF.Action)= action;

The look of the orchestration is much lighter and pleasant to work with.

XSLT in Any situation

XSLT: The Basics

Here are some basics XSLT commands that will be useful for the next step.

Setting a variable

Variables can only be set once. For loops consider recursive functions (a.k.a. templates).
MyVariable=5
<xsl:variable name="MyVariable">5</xsl:variable>

Reading a variable

<xsl:value-of select="@MyVariable" />

Read element

Read an element called Element from a node Node2 inside a node Node1
<xsl:value-of select="/Root/Node1/Node2/Element" />
Remember to close the tag

Setting a variable with the value of an element

<xsl:variable name="ValueOfElement">
<xsl:value-of select="/Root/Node1/Node2/Element" />
</xsl:variable>

Read attribute

Read an attribute called Attribute from a node Node4 inside a node Node3
<xsl:value-of select="/Root/Node3/Node4/@Attribute" />

Setting a variable with the value of an attribute

<xsl:variable name="ValueOfAttribute">
<xsl:value-of select="/Root/Node3/Node4/@Attribute" />
</xsl:variable>

If condition

Remember that XSLT is used to construct a XML document so the return of a value is not mandatory. The easiest way to think of it is like a print command, not like a function.
if(MyVariable>10) print MyVariable (> is a reserved char so we use &gt; instead)
<xsl:if test="@MyVariable &gt; 10">
<xsl:value-of select="@MyVariable" />
</xsl:if>

When the comparition is not against variables or numbers (string comparition for instance) the method is very similar, just delimit the string with ' '.
if(MyVariable="test") print "this is a test"
<xsl:if test="@MyVariable='10'">
this is a test
</xsl:if>

If, Elseif, Else

Does not exist an equivalent approach. We use a kind of switch instead.

if(MyVariable=10) print 1;
else if (MyVariable=10) print 10;
else print 100;


<xsl:choose>
<xsl:when test="@MyVariable &lt; 10">1</xsl:when> <!-- if --!>
<xsl:when test="@MyVariable = 10">10</xsl:when> <!-- else if --!>
<xsl:otherwise>100</xsl:otherwise> <!-- else --!>
</xsl:choose>

<Any> and <AnyAttribute>

A schema can be created without knowing the name of fields to fill. The menu Add Element has the option <Any>, and the same applies for Add Attribute (<AnyAttribute>). Like this, you can add virtually any node or attribute even after compilation. Be sure to define 0 at minimum occurrencies or it will fail while you don't have values to fill.

How to Map to <Any>

Create a Script Functoid of type "Inline XSLT". On the code area write the new tag and the desired value.

<NewField>
42
</NewField>

<NewField>
<xsl:value-of select="@MyVariable" />
</NewField>

<NewField>
<xsl:choose>
<xsl:when test="@MyVariable='X'">1</xsl:when>
<xsl:otherwise>0</xsl:otherwise>
</xsl:choose>
</NewField>

To give a namespace instead of writing directly the element, write the tag element:
<xsl:element name="ns2:ElementName">

and so on...

How to Map to <AnyAttribute>

Create a Script Functoid of type "Inline XSLT". On the code area write the new tag and the desired value between xsl:attribute tags.


<xsl:attribute name="NewAttribute">
<xsl:value-of select="$MyVariable" />
</xsl:attribute>
or any other case.

The Scripting Fuctoid is not linked to source schema, but be careful on linking Scripting Functoid to the destination schema. Point to the desired parent node.

Pipelines in Biztalk

Creating pipelines in ports: knowing when to use PassThru Pipelines, XML Pipelines, and when to use Custom Pipelines.

A PassThru pipeline is straightforward. On a ReceiveLocation it receives a file without further treatment, validation or field promotion. The clear text message can be captured by a Send Port with filter by Receive Port.

A XML Pipeline is more complex. It analyses the received file and determines its structure. Properties are promoted and therefore more kinds of filters become available (specially the useful namespace).

In a XML pipeline there are no limitations. The schemas can by converted, envelopes can be splitted, all options are available under the properties section. To avoid time-consuming configurations in that area – and reduce error-proneness – a Custom Pipeline should be made. It stores the configuration inside the DLL and thus can be reused in another port, or even exported to another environment without dues.

The same logic applies to SendPipelines. Usually a PassThruTransmit is enough because the message is already known in Biztalk. If a format change is needed (to Flat File i.e.) an XMLTransmit with the adequate assembler is the only option. And to place that configuration at development level, a Custom Pipeline.


Hint: For the Flat File Assembler there is no need to select a schema. According to this MSDN page If no schema is specified, run-time schema discovery is done. It was tested and confirmed. It makes development a lot easier:

by reducing the number of pipelines. There's no need for multiple assembler pipelines. Instead of one pipeline for each schema, a single pipeline can be used by the entire application.

by making them faster to upgrade. When refering schemas from external DLL's, the assembly version is required. In the past if that DLL was changed (it happens in our dev environment on a weekly basis) we had to manually correct each map and pipeline. Now the pipelines are not a problem.

Mapping several elements to a single element

Frequently we have one block of XML that we want to split to diferent blocks. For instance, when exporting to SQL stored procedures. It is easy to do. Sometimes the opposite happens and the tables are imported from SQL: we have the information divided by several blocks on one XML and we want it grouped.

Let's imagine a simple scenario: one table included the name of a person, the other the professional information. How can we group it to a single element?

We use XSLT.

Supposing that one of the segments has all the keys, the trick is to loop that segment to read the keys and pull the information from all. This way the order of the elements is not important, the association is made by key.

This template compiles the information:

 <xsl:template name="NameValueTemplate">
  <xsl:param name="param1" />
  <xsl:for-each select="//ElementKey">
   <xsl:if test="Id/text()=$param1">
    <xsl:attribute name="Id">xsl:value-of select="Id/text()" /></xsl:attribute>
    <xsl:element name="Name"><xsl:value-of select="Name/text()" /></xsl:element>
   </xsl:if>
  </xsl:for-each>
  <xsl:for-each select="//ElementInfo">
   <xsl:if test="$Id=$param1">
    <xsl:element name="Mail"><xsl:value-of select="Mail/text()" /></xsl:element>
    <xsl:element name="Job"><xsl:value-of select="Job/text()" /></xsl:element>
    <xsl:element name="Phone"><xsl:value-of select="Phone/text()"/></xsl:element>
   </xsl:if>
  </xsl:for-each>
 </xsl:template>


And this is how it is called:

 <xsl:template match="/s0:Root">
  <ns0:Root>
   <xsl:for-each select="//ElementKey">
    <xsl:element name="Element">
     <xsl:call-template name="NameValueTemplate">
      <xsl:with-param name="param1"><xsl:value-of select="Id/text()"/></xsl:with-param>
     </xsl:call-template>
    </xsl:element>
   </xsl:for-each>
  </ns0:Root>
 </xsl:template>

This is it. It includes attributes and elements to include a broad range of situations. More complex cases can be made from this.

Ports and correlations

You must specify at least one already-initialized correlation set for a non-activation receive that is on a non-selfcorrelating port.

There is not a better error to start whin than this wonderful classic. This error happens to everyone all the time. When you start making an orchestration you may be thinking of this, but it will vanish from your mind before the first component is set.

Solution:

The beggining of the orchestration - usually a Receive component - must have the property Activate as True. Why isn't this a default in Biztalk? Good question...