Welcome to the HL7 Soup and Integration Host Workflow Designer.
With it, you can Transform HL7 Messages too and from different formats such as JSON, CSV, or XML. You can also send and receive messages via TCP, HTTPS, Databases, Webservices and more, even enable HL7 in your products.
This video delves into many places that none of our other videos do, and it’s essential viewing for mastery of HL7 Interfaces. But once you’ve reached the end, be sure to check our many great tutorials that give step by step instructions on how to perform various tasks.
The workflow designer, is divided into three. On the very left is the list of steps that will be performed. We call the individual steps activities, and the list we call a workflow [some call this a channel]. Activities can be added by clicking the “Add New Button”. You can insert activities between others, reorder them by dragging and dropping, delete them, disable them so they aren’t validated or executed, and enable them again. (TCP Reciever, TCP sender, File write)
The first activity is a special one. It can’t be moved or deleted because it defines the action that starts off the execution of the workflow. We call this first activity the Receiver activity. [some others call it the source activity].
The center section shows the details of the activity that’s currently selected, clicking another activity jumps to its details. (jump to second activity)
The action that will be performed by each activity is selectable at the top of the details panel. Common use cases such and TCP, HTTPS, Webservices, and Databases are all supported, and there’s even a Code activity if you think some C# would help. The list of available activities is also extensible and can be expanded by the team at HL7 Soup, or you can even create your own in .net [Link to tutorial]. It’s worth noting that the available list of activity types differs for the Receiver Activity, as these are all trigger actions that will start an instance of the workflow running and acquires the inbound message. Activities after the first often focus on sending the message to other systems, but some can also obtain additional data too. Nothing at this level represents how we will be altering or converting the data; we’ll get to that shortly.
As we switch between activity types, we get a set of options relevant to performing the action. Other tutorials go into the configuration of these, but I have just a couple of standard settings I wish to point out. Firstly there is the “Add incoming message to current list” checkbox available on all receiver activities. If selected the inbound message will be loaded into the HL7 Soup Editors messages when executed by that application. It has no function in Integration Host as it uses logs instead.
As we switch between activity types, we get a set of options relevant to performing the action. Other tutorials go into the configuration of these activities in detail, so I will skip most of the settings, but jump down to the Message Template, a key field which is in almost every activity type.
For the receiving activities, it represents an example of the inbound message, so if you have one, paste it in. It’s not required to proceed, but the ease of mapping your messages will be significantly improved. Some samples are available in the right-click menu if you don’t have a message handy (OBX one). Notice as I add in the Message Template, the bindings panel on the right is populated with a tree structure representing this message. We’ll see what can be done with the binding panel shortly, but for now, notice how clicking on a field in the message template selects the corresponding item in the bindings panel and vice versa. Also, if we edit the message and add or delete a field it will be reflected almost immediately in the binding panel, so if a field you need is missing from the binding panel, just add it to the message template.
For any activity after the Receiver, the Message Template represents the outbound message. For instance, it’s the message being sent on a TCP sender, or the file being written in a file writer.
When the activity was created, it automatically had its message template populated with a binding to the Receivers inbound message. That’s what this blue highlighted text represents. This activity will send on the same message that the workflow initially received [Though alterations are possible – more soon]. I can, however, delete this binding just like any other piece of text, or add it back again by right-clicking, “Inserting Activity Message”, then selecting which activity. It needn’t be the Receiver activity either; it could just as easily utilise the message sent or returned by any proceeding activity. Using message templates with bindings is a great way to relay a message between systems with minor alterations (bind to source)
If you are converting between different message types or making substantial changes with the same message type, you should start by deleting the binding, inserting an example of the message you will be converting to, and setting the Message type. If we run this workflow now it would wait for an inbound message, then send off this message exactly as it’s written here. That’s great, that’s our general plan, and along with Relays, is a very common workflow scenario. But what’s missing is how we alter or map the incoming values into this outbound message.
There are several ways to map values from one message to another. One of the simplest is to use the Binding Tree that populated earlier. See how I can drag fields from the binding tree directly into a message template. At runtime these purple placeholders will be replaced with the source value.
Notice this drop-down at the top of the binding tree; this enables you to select which activity you want to get your data from. It defaults to the Receivers inbound message, but you can mix and match from any previous activity, or even from values stored in memory called variables - more on those shortly. But first let’s click on the Transformers tab of the activity details and take a look at other ways of mapping values. Each activity has its own list of transformers that are executed just before the message is sent out. Transformers are a list of steps that define how values are to be mapped between messages.
This center “Transformers” panel will list all the steps and mappings that will be performed, while the tree’s on either side represent where you can acquire data from, as well as where into your outbound message it can go [Trees are constructed from your message templates]. Note, the Source Tree is functionally just like the Binding tree and you can use them interchangeably. If you don’t have a source tree available in your version of the designer just use the binding tree instead.
By dragging a field from the source tree to a field in the destination tree, we have created a transformer that maps the value across. You can potentially map all of your fields this way, mapping one field to another till you have mapped them all. Notice that clicking a field in a tree will highlight all transformers that use it, alternatively selecting a mapping transformer will highlight the corresponding field in the trees.
Details of the current Transformer is shown down here in the details panel. See that the “To” and “Source Path” are populated with a syntax representing where in the message to get and set the values. You can change the paths by dragging in different paths from the relevant tree (destination path to xml name), or by manually typing too (PID-5.1). The path needn’t be in your message template either (PID-50). The source path might represent a value that may not always exist. If the “To” path doesn’t exist in your Message Template, it will be automatically appended with the appropriate message structure that’s required.
Just quickly, the paths syntax are common to other systems and based on common standards.
HL7 is the Segment name, dash, Field, dot, Component, dot, Subcomponent. Each of these can include an index too for repeated segments or fields. If indexes are omitted then the first instance will be assumed.
FHIR, XML and JSON use a simplifies XPATH syntax separating the hierarchy of nodes with a forward slash, and with attributes signified by their name starting with an @. If you require strict .net XPATH, then preface the path with “xpath:”, but this is generally not required.
A CSV path is just the index of the field in square brackets.
If you take a look at the end of the Source Path, you can see an anchor button, clicking it allows you to select which activity your path is acquiring the data from. Of note is “Text and variables” at the top. Selecting Text and Variables means that this text now represents a literal value rather than a path. Also notice how the text's colour has changed to blue. A standard throughout the workflow designer is that any text that supports binding is coloured:
Green if it’s a Path bound to an activity
Red if it's bound to an activity, but the path is invalid
Blue if it’s Text and Variables, with the variables shown in purple.
So now I’ve selected Text and Variables, whatever I type here will be literally put in the message. If you start finding paths in your outbound message, it’s because you need to set the source activity via the anchor button to an activity rather than Text and Variables. However, Text and Variables does allow you to insert variables into the text. If I right-click, I can select a variable to insert. Some default variables are always available such as the date of the message, but any variable you create will also be listed here.(Insert LastName Variable) What’s more I can add multiple variables along with literal text (-) to build up complex strings of text. I’ll add another variable for example, but this time from the source tree(last name).
Then is the “Allow messages structure to change” checkbox. By default it’s not selected, and that means that any characters reserved for that message type [such as & for XML or HL7] will be automatically encoded/escaped so that it doesn’t corrupt the message. If you instead check this box, you can include message structure altering characters and build the message on the fly.
There are many transformer types available.
Set Variable Value – Either creates a variable or alters the value on an existing one. See now that I’ve created a variable it becomes available to be used in mappings or other transformers. A shortcut to create a variable populated from a message field is to drop a Source Value directly into the transformers list, and a variable will be generated already bound to that source. You may want to edit the name, though it’s not essential. Also, remember earlier when we dragged from the Binding Tree directly into the message template, well that was behind the scenes creating a Set Variable Transformer populated by its source, then putting a variable placeholder into the message. The same thing could have been done manually too. You set the values of variables using much that same techniques as we did in mappings, though it’s worth noting this addition Text Box call “Sample Value”. This is a design-time helper that allows the binding tree’s to include fields that are created by variables, which is convenient if you are altering the message structure with a variable but still want to easily mappings with a drag and drop.
The Begin and End Conditional transformers add as a pair, and transformers placed between only execute if conditions are met. Note how you can add in multiple criteria arguments and adjust if they are “And” or “Or” comparisons. Complex criteria can be created by adding groups which act like brackets in your criteria logic.
The “For Each” and “Next” transformers allow looping over repeated values (OBX). For any transformer executed between the For/Next pair, any binding or mapping that references something in the loop item will take the value of the iterated item instead (OBX-5). There is an example of this in our JSON to HL7 and XML to HL7 tutorials if you want to see it in action.
The Comment transformer allows you to add comments into your transformers tree, which is helpful when things get complicated. (Required by Customer X)
Adding a code transformer lets you write C# code to perform just about anything you want. If you are a c# developer, you may wish to do all or some of the transformations in code. There is a fantastic API for HL7 included too, with intellisense more. It’s beyond this video, but check out the Coding Cheat Sheet in our tutorials if you want to know more.
You can even create your own transformers with Visual Studio that are easily distributable. See our Custom Transformers tutorial for details.
At times you may need to format your data. Dates are almost always formatted with HL7 integrations. Fortunately everywhere that accepts a mapping or is bindable supports easy formatting, as does every variable. I can right-click and select the format of my choice. (format DOB) There is even the ability to easily create your own format if it’s not on the list. Dates are not the only thing either, you can make sure your ID’s are padded with zeros, and your names are capitalized correctly. Using the McName-casing even formats surnames and addresses with complex capitalization correctly, that will keep Mr McDonald happy.
You saw earlier that mapping transformers were encoded\escaped correctly by default, well, you can also escape variables too with a simple right-click option.
Filters are another tab available in the activities details. Filters can be used to control which activities execute based on the data. Providing that the criteria evaluates to true, then the activity will be executed. An activity that is filtered won’t execute and the workflow will continue to the next activity, however, if a Receiver activity is filtered, then no further activities will be executed. Filters have the same logic creation abilities as the Conditional Transformers. “And’s”, “Or’s” and groups are available.(delete filters) Using the Binding Tree is a great way to specify which fields are going to be examined, dragging in creates criteria already mapped to the path, while dragging in holding shift will map that path to a variable and base the criteria on that. Also, don’t forget the autogenerated Variables, the WorkflowError is always available. It’s “false” when everything is fine and “true” if something has failed.
The filters happen before the main function of the activity executes, so have their own Transformers lists in case you need to create variables or some other task. You probably won’t need to touch these transformers, but it’s worth knowing they are there.
Now is probably the perfect time to show you the execution order.
As you can see the message comes in, any filter transformers are executed, then the receiving activities filters. Providing that the message wasn’t filtered and there are other activities we proceed to the first activity. It’s Filter Transformers are process, ready for the filters to follow. Providing the Activity isn’t filtered the main transformers execute which maps your messages together, and the activity is processed or sent depending on its function. If it’s configured to get back a response, then it will wait for that to be returned.
There can, of course, be many activities, so the green section will repeat for each of them till all activities have been processed or filtered. Finally, we return control back to the Receiving Activity, execute the response Transformers preparing the message to return, and finally send it back. Not all Receiver Activities respond back [Directory scanner], and some others that can respond, default to automatically generated responses [MLLP Reciever]. These will not have response transformers and won’t have the Transformers tab shown on the details panel. Remember that Transformers in a Receiver activity execute after all activities, so don’t accidentally try transforming data for other activities here.
A navigation tip: When navigating activities you may also click these icons to go directly to the Transformers or Filters of the activity.
Two message types are perhaps less obvious than the others – Text and Binary. You can’t use these in message mappings because they don’t have any message structure to parse them into fields; however they can be populated with variables or direct activity binding. They may be very helpful to you. Text is just a simple blob of text, which can be built up by appending variables into it. Because it has no structure it isn’t validated for correctness. This allows you to construct other message types with more freedom. For instance, you can create multiline CSV from OBX values found in a single message, or build JSON message structure with variables.
Binary messages are used to preserve binary files like PDF’s or Images that would be corrupted if converted to text. The File Writer will also write Base64 encoded data to binary file too.
Finally, in the right column, we have the Message Logs. They list the most recent 1000 workflow executions since that app was loaded in the HL7 Soup Editor, or if Integration Host, the logs are persisted and can be a much more extensive history. Message logs record the messages your workflow and activities send and receive, variable values when activities execute, error messages, and the duration and time of the execution. They are critical for debugging your workflows, so it’s helpful to keep them handy while editing and designing. If an HL7 message is clicked in the logs, the current path is highlighted in the Message Template, the Binding Tree, and Transformers. For instance, clicking on a faulty binding value in the logs will automatically highlight any transformer that refers to that field.