Hello Thomas,

I cannot replicate the problem here. I am using a newer CTT that may in fact have the problem resolved already. I would like to send the new CTT to you. Please email me at compliance@opcfoundation.org.

No feedback.
Furthermore, we have released a newer CTT 1.02.336.248 that is a rebuild and should be more stable.

This "bug" still exists in CTT 1.2.336.254

Example: 1 subscription, 1,250 monitored items, 250 ms sampling rate, 1,000 ms publishing interval, queue size = 5.

It takes only a few minutes until the CTT's memory increaes up to 1.7 GB and crashes.

If I use the Java toolkit (from OPC Foundation website) as a client the memory isn't increasing but I get a lot of subscription timeouts.
If I use the .Net toolkit (from OPC Foundation website) everything is working perfectly.

This happens due to expected behavior of the CTT. The CTT is keeping for verification purposes all received DataChangeNotifications in memory. This allows test scripts to validate their content afterwards. Of course at some point in time it will reach a memory limitation of the system which will cause it to crash because it can't use any more memory. Because we need to keep that buffer available in the CTT we can't change that behavior and freeing old data. The limitation will be increased from time to time due to changes of the redistributables etc. In my environment the CTT can use up to 2 GB of Memory.

Could you please point out the intend of this test case so we can think about a possible solution by changing the script but still doing the same verification?

This test is intended as a performance test.

How can I access all received DataChangeNotifications in memory as you mentioned?
What I do is also to store all received data in memory, and after the subscription has ended I check if I have received all expected data.

I have several sets of test configurations: number of monitored items and updates rates, to measure the performance capabilities of our server. E.g. expected rate is of ~80K data values per second. And I also want to check if the performance is constant over a period of 1h , 8h or 24h (load test).
Today the tests can run savely only up to 25s without crash.

You can access them via the PublishHelper object. This Helper object has two elements for the DataChangeNotifications which are
=== PublishHelper.ReceivedDataChanges ===
This element will save all received data change notifications until the subscription is been deleted.
=== PublishHelper.CurrentDataChanges ===
This element will only show the data changes of the last PublishResponse.

I hope this helps a bit. Unfortunately this will only make the CTT run a bit longer because the data then is only kept in the memory once. But I'm pretty sure it still won't run for 24h with this change.
I'll check for an opportunity to empty the array. This would ensure that the CTT stays up and running even though you're going to lose the data in the memory.
Because JavaScript works pretty well with strings I would then recommend to save the desired Information in a file and clean the PublishHelper.ReceivedDataChanges list and do the validation afterwards by interpreting the content of the file.

That's good news to me! So if I manually (by script) empty the list of ReceivedDataChanges, or remove my own variables, then it could be what I need!?!

Sorry for my poor knowledge. I program user defined scripts with CTT for about 4 years now. All I know is what I understood from studying existing scripts. But from a lot of scripts and built-in objects I don't know their intention.

I have fixed my performance tests, but the storage of values of monitored items is not the only issue. After each call of PublishHelper.Execute() I reset the value of ReceivedDataChanges to "[]" (empty array). Unfortunately the CTT is still crashing. So I did more investigation and I found 3 issues:

1. Call of addError()
2. Call of writeFile()
3. Data type conversion using UaVariant.toXxx() methods.

The first line I can accept. CTT needs to show the errors occured. The other two I haven't expected. That piece of code is used to analyze received data and give error if some data is missing.

<code>
var c = mItem.Values[0].Value.toInt32() - 1; // initialize with the gap in the beginning
for (var i=1; i<mItem.Values.length; i++) // look for further gaps
{
c = c + mItem.Values[i].Value.toInt32() - (mItem.Values[i-1].Value.toInt32() + 1);
}
c = c + expectedNumOfValues - mItem.Values[mItem.Values.length-1].Value.toInt32(); // add the gap at the end
if (c > 0)
{
addError("Missing " + c + " values! Expected <" + expectedNumOfValues + "> but got <" + mItem.Values.length + ">. NodeId=\"" + mItem.NodeId + "\"");
writeFile(logDir + "/" + mItem.NodeId.getIdentifierNumeric() + ".txt", mItem.Values.toString());
success = false;
}
</code>

The variable "c" is used to calculate the gaps. The underlying data simulation increases values from "0" to <expectedNumOfValues> by "1". If "c > 0" I know how many values are missing.
Only after removing 3 lines (the for loop, the call of addError() and the call of writeFile()) the memory will not further increase.

I'm happy to hear that we already made some progress.

I can see where the increase might comes from. The addError() function does print the error in the result window and therefore the necessary memory increases with each call of this function. Due to your test scenario I assume there are many addError calls in one test run.
If you do not need the information of each failure in the CTT I would suggest to remove the addError() and only save the information in the file. Then checking the success flag at the end and if it is set to false, print one error only.
Let me know if it worked out well.