View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0010196Compliance Test Tool (CTT) Unified Architecture1 - Script Issuepublic2025-03-06 10:592025-03-18 11:37
ReporterV. Monfort Assigned To 
PrioritynormalSeverityminorReproducibilityrandom
Status newResolutionopen 
Product Version1.04.11-01.00.508 
Summary0010196: Alarm and Conditions tests should not use CallMethod response timestamp as expected Event Time minimum
Description

In most of the Alarm and Conditions tests, the Call method response time is used as a strict reference to consider if an Event is part of the events to consider based on the Event time.
It means that some tests will fail based on the fact that a newly received Event Time value is less than the response time, e.g. Acknowledge method call response and event containing the AckedState/Id=TRUE.

Regarding the specification there is no such guarantee, quite the contrary:

  • ResponseHeader timestamp is "The time the Server sent the response"
  • Event Time value "provides the time the Event occurred."

Since the event is generated during the method call treatment in case of Acknowledge method call, it actually occurred during the method call treatment and thus prior to the call method response is sent. There is a strong possibility depending on server implementation that the expected event Time is less than the response timestamp for this reason.

Thus the tests should not fail for such reason and should use other criteria to discriminate event if needed (response timestamp of the PublishResponse ?).

The following tests are using such timestamp, most time for such restriction on Event Time value:
maintree/Alarms and Conditions/A and C Comment/Test Cases/Test_003.js: localTestCase.CommentTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Comment/Test Cases/Test_002.js: localTestCase.CommentTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Comment/Test Cases/Err_007.js: localTestCase.CommentTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Comment/Test Cases/Test_004.js: localTestCase.CommentTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Comment/Test Cases/Test_001.js: localTestCase.CommentTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Enable/Test Cases/Test_003.js: localTestCase.DisableTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Enable/Test Cases/Test_003.js: this.ActualRefreshDeviceTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Enable/Test Cases/Test_002.js: localTestCase.ConditionDisableTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Enable/Test Cases/Test_002.js: localTestCase.ConditionEnableTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Enable/Test Cases/Test_002.js: localTestCase.InstanceDisableTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Enable/Test Cases/Test_002.js: localTestCase.InstanceEnableTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Confirm/Test Cases/Test_003.js: localTestCase.AcknowledgedTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Confirm/Test Cases/Test_003.js: localTestCase.ConfirmedTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Confirm/Test Cases/Test_002.js: localTestCase.AcknowledgeTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Confirm/Test Cases/Test_002.js: localTestCase.ConfirmedTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Confirm/Test Cases/Err_005.js: localTestCase.ConfirmedTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Confirm/Test Cases/Err_009.js: localTestCase.ConfirmedTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Confirm/Test Cases/Err_004.js: localTestCase.AcknowledgeTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Confirm/Test Cases/Err_004.js: localTestCase.ConfirmTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Confirm/Test Cases/Err_006.js: localTestCase.ConfirmedTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Confirm/Test Cases/Err_007.js: localTestCase.ConfirmedTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Confirm/Test Cases/Test_001.js: localTestCase.ConfirmedTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Acknowledge/Test Cases/Test_003.js: localTestCase.AcknowledgementTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Acknowledge/Test Cases/Test_002.js: localTestCase.AcknowledgementTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Acknowledge/Test Cases/Err_005.js: localTestCase.AcknowledgementTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Acknowledge/Test Cases/Err_009.js: localTestCase.AcknowledgementTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Acknowledge/Test Cases/Err_004.js: localTestCase.AcknowledgementTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Acknowledge/Test Cases/Err_006.js: localTestCase.AcknowledgementTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Acknowledge/Test Cases/Err_007.js: localTestCase.AcknowledgementTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Acknowledge/Test Cases/Err_008.js: localTestCase.AcknowledgementTime = collector.GetCallResponseTime();
maintree/Alarms and Conditions/A and C Acknowledge/Test Cases/Test_001.js: localTestCase.AcknowledgementTime = collector.GetCallResponseTime();

TagsNo tags attached.
Files Affected

Activities

V. Monfort

2025-03-18 11:37

reporter   ~0022543

In a more general way, the tests often expect that the next event will contain an expected value when it might not be the case regarding the specification part 9. Thus several events might occur prior to obtaining the expected state change.

E.g. when acknowledging an A&C, it is expected to have the AckedState/Id state and ConfirmedState/Id state change in the same Event Notification in UACTT test but it is not explicitly required by the specification.
Specification indicates: "In general, changes to Variables of the types TwoStateVariableType, ConditionVariableType, StateMachineType or any of their subtypes trigger Event Notifications and are not explicitly described in subtypes ."
Since 2 TwoStateVariableType variables are changed in this case, it might lead to 2 different Event Notification containing each one a state change (AckedState/Id = True and then ConfirmedState/Id=False).

Issue History

Date Modified Username Field Change
2025-03-06 10:59 V. Monfort New Issue
2025-03-18 11:37 V. Monfort Note Added: 0022543