DataWeave Memory Management
|
DataWeave 2.1 is compatible with Mule 4.1. Standard Support for Mule 4.1 ended on November 2, 2020, and this version of Mule will reach its End of Life on November 2, 2022, when Extended Support ends. Deployments of new applications to CloudHub that use this version of Mule are no longer allowed. Only in-place updates to applications are permitted. MuleSoft recommends that you upgrade to the latest version of Mule 4 that is in Standard Support so that your applications run with the latest fixes and security enhancements. |
When processing large files through DataWeave in Mule runtime engine, there are a few things you can set up to fine-tune how much memory will be used and when.
RAM vs Disk Usage
DataWeave uses disk data storage to avoid running out of memory. The files created are placed in a default temporary directory. If you want to store those files in a custom directory instead, you can specify the directory by using the java.io.tmpdir property.
Two types of DataWeave files are generated, both with names beginning with dw-buffer-:
-
dw-buffer-output-${count}.tmp
Used to store the output of a transformation when the result is bigger than the threshold 1572864 bytes. To change this threshold value, add the system propertycom.mulesoft.dw.max_memory_allocationand assign it the number of bytes you want as your new threshold. Because you can define system properties in several ways, see system properties for further details. Mule runtime engine deletes the file when the value is no longer referenced, JVM GC collects it or when the Mule Event finishes executing. -
dw-buffer-index-${count}.tmp
Used to store index information of a value being read. This file helps DataWeave access data quickly. Mule runtime engine deletes the file when the execution of the transformation ends or, in a streaming use case like the foreach loop, when the stream ends (when foreach finishes its execution).