Thursday, 2 July 2020

Notes: Module 10: Mule 3 Fundamentals: Writing DataWeave Transformations


DataWeave is a JSON-like language.

Objectives include:
- Store DataWeave transformations in external files
- Coerce and format strings, numbers, and dates
- Call MEL functions and Mule flows from DataWeave transformations

10-1) Introducing DataWeave Transformations

DataWeave transformation expressions:
- The header contains directives - high level info about the transformation
- The body contains a DataWeave expression that generates the output structure.

Data model of the produced output can consist of 3 different types of data:
- Objects: As a collection of key value pairs
- Arrays: As a sequence of comma separated values
- Simple literals

Output Directive - sets output type of transformation:
- Specified using content/type
- Structure defined in DataWeave body

Example:
%dw 1.0
%output application/xml
---
{
  a: payload
}

MIME type: You may get an error unless the MIME type for the input data has been set.

The code can be placed in your XML as a CDATA type. Sample data is stored in src/test/resources as JSON.

Image: DataWeave code in XML

Reusing transformation:
- Click ‘Edit current target’ and set source code to file. Saved as ‘.DWL’ in ‘src/main/resources’.
- To reference an existing DWL, specify in the XML:
set-payload resource="classpath:user_transform.dwl"

Wakthrough 10-1: Write your first DataWeave transformation
- Create a new flow that receives POST requests
- Write a DataWeave expression to transform the data to Java, XML, or JSON
- Save the transformation in an external file

10-2) Transforming Basic Data Structures

Writing Expressions for JSON or Java input and output

+----------------------+
| 1) INPUT             |
+----------------------+
| {                    |
|   "firstname":"Max", |
|   "lastname":"Mule"  |
| }                    |
+----------------------+

+-----------------------------+-----------------------------|
| 2) TRANSFORM                | 3) OUTPUT                   |
+-----------------------------+-----------------------------|
| fname: payload.firstname    | {"fname": "Max"}            |
+-----------------------------+-----------------------------|
| {fname: payload.firstname}  | {"fname": "Max"}            |
+-----------------------------+-----------------------------|
| user: {                     | {"user": {                  |
|  fname: payload.firstname,  |   "fname": "Max",           |
|  lname: payload.lastname,   |   "lname": "Mule",          |
|  num: 1                     |   "num": "1"                |
| }                           | }}                          |
+-----------------------------+-----------------------------+
| [                           | [                           |
|  {fname: payload.firstname, |  {"fname": "Max","num": 1}, |
|   num: 1},                  |  {"lname": "Mule","num": 2} |
|  {lname: payload.lastname,  | ]                           |
|   num: 2}                   |                             |
| ]                           |                             |
+-----------------------------+-----------------------------+

Writing Expressions for XML Output

XML can only have one top-level value and that value must be an object with one property.

Image: Writing expressions for XML output

Image: Writing expressions for XML input (use @ to reference attributes)

Walkthrough 10-2: Transform basic Java, JSON, and XML data structures
- Write expressions to transform the JSON payload to various Java structures,
- Create a second transformation to store a transformation output in a flow variable.
- Write expressions to transform the JSON payload to various XML structures.

Image: Transform: Toggle between Payload (Java) and FlowVar - XML

10-3) Transforming complex data structures with arrays

The map operator (working with collections):
- To apply a transformation to each element in a collection (JSON or Java Array, or XML repeated elements).
- Applied to each element in an array or each value in an object.
- Returns an array of elements.

Image: Example: Using payload map
$$ refers to index (or key)
$ refers to the value

Image: Example: To set the index as a new key surround it with () or '' (2 x ')

Walkthrough 10-3: Transform complex data structures
- Create a new flow that receives POST requests of a JSON array of objects
- Transform a JSON array of objects to Java

Image: Transform Example

10-4) Transforming complex XML data structures

Image: Example: When mapping array elements (JSON or JAVA) to XML, wrap the map operation in {(...)}
{} are defining the object
() are transforming each element in the array as a key/value pair

Image: Example (continued I)

Image: Example (continued II): Use * to reference repeated elements

Warning: Make sure you look at the raw response and not the “pretty” response.

Walkthrough 10-4: Transform to and from XML with repeated elements
- Transform a JSON array of objects to XML
- Transform XML with repeated elements to Java

Image: Transform application/xml (Example 1)

Image: Transform application/xml (Example 2)

Image: Transform application/java (Example 3)

10-5) Formatting and Coercing Data

DataWeave has extensive documentation:
‘https://docs.mulesoft.com/mule-user-guide/v/3.?/dataweave’ (especially checkout: ‘type conversion table’.)

Using the ‘as’ operator for type coercion:
price: payload.price as :number
price: $.price as :number {class: "java.lang.Double"},

Defined types include:  string, number, Boolean, object, array, date, time, timezone, datetime, localdatetime, period, regex, and more...

Use metadata format schema property to format numbers and dates:
someDate as :datetime {format: "yyyyMMddHHmm"}

Walkthrough 10-5: Coerce and format strings, numbers, and dates
- Coerce data types
- Format strings, numbers, and dates

Note: format: "###.##" (for 3 digits and 2 decimal places.). Use "###.00" to give everything without decimals .00.

Image: Formatting Example

Image: Date Formatting Example (date formatted as string is YYYY-MM-DD)

10-6) Using DataWeave operators

Conditional logic operators:
default
when
unless
otherwise
==
!=
~= (equal regardless of type)

Additional operators (check out the documentation):
flatten, orderBy, concat, distinctBy, groupBy, replace, matches, regex...

Walkthrough 10-6: Use DataWeave operators
- Replace data values using pattern matching
- Order data, remove duplicate data, and filter data

Replacing BOING with BOEING:
upper $.planeType replace /(BOING)/ with "BOEING",

Order by price and date (date inside price), just distinct array items, and remove flights with 0 seats free:
} orderBy $.date orderBy $.price distinctBy $
   filter ($.seats != 0)

Image: The DataWeave expression

10-7) Using Custom Data Types

Use type header directive:
- name has to be all lowercase letters

Example: Specifying custom data types
%dw 1.0
%output application/json
%type ourdateformat = :datetime {format: "yyyMMddHHmm"}
---
someDate: payload.departureDate as :ourdateformat

Transforming objects to POJOs:
- Use:
as :object
- Specify inline:
customer:payload.user as :object {class: "my.company.user"}
- Or define a custom data type to use:
%type user = :object {class: "my.company.user"}
customer:payload.user as :user

Walkthrough 10-7: Define and use custom data types
- Define and use custom data types
- Transform objects to POJOs

Image: Example using type (compare with DW expression above)

Image: Example using custom data type

10-8) Referencing other data besides the payload

i) Reference message variables and properties:
flowVars, inboundProperties, outboundProperties, p (System or Spring properties)
Example: {n: flowVars.username}
Note: Do not preface with "message." or use #[]

ii) Call global MEL functions from DataWeave:
{"foo " : newUser(),
 "bar ": upperName(newUser())}

Image: Defining global MEL function in the configuration element you used to specify a default global exception handler

iii) Call other Mule flows
- Whatever the flow returns is what the expression returns
{a: lookup("mySecondFlow",{b: "Hello"}) }
- The 1st argument is the name of the flow to be called
- The 2nd argument is the payload to send to the flow as a map

Walkthrough 10-8: Call MEL functions and other flows
- Define a global MEL function in global.xml
- Call a global MEL function in a DataWeave expression
- Call a flow in a DataWeave expression

Image: Define a global MEL function in global.xml (Note: Should have a ; after return 300)


Image: Call a global MEL function in a DataWeave expression

Image: Or, with a new flow ‘getTotalSeatsFlow’ made up of an expression

Image: The POST in Postman

No comments:

Post a comment