Expression on limit (mixin) does not use updated ouptut value set by test step

Technical
1

I have 3 rise-time double values that a test step outputs from a scope, one represents the 28V rise time going into a DUT and then a 3V and 5V rise time exiting the DUT and I have added 2 limits mixins on the step to verify that the 2 output rise times are within limits based on the rise time of 28V, so one effective limit check is:

  • RiseTime28V + 0.02 < RiseTime5V < RiseTime28V + 0.15

When I run the step, the output values are:

  • RiseTime28V: 0.3
  • RiseTime3V: 0.4
  • RiseTime5V: 0.4

However the output of the Limit test shows:

  • Limit check for ‘Settings \ Rise Time 3p3V’ in step ‘ExpressionBugTestStep’ failed: Expected 0.02 < ‘Settings \ Rise Time 3p3V’ < 0.15. (Was 0.4)

So a value of 0 is returned for RiseTime28V when evaluating the expression and not the value of 0.3 that the test set. The second time I run it passes because 0.3 is the value of the output before running the step.

I assume the issue is that the expression is being evaluated too early or it is getting a cached value somehow. This could be a bug or just a limitation in the current design.

I tried to find the expressions code to potentially debug but it is not accessible here: https://github.com/opentap/expressions

Here is a link to the test and a test plan that calls it: ExpressionsBug.zip

2

Hi @scottri,

Thanks for raising the issue.

The normal use case for an expression is to provide a value for a property to be used as an input to the test step, but there are definitely other use cases, like yours.

In your case the expression is assigned to the limit itself, so it should be possible to order evaluation based on the order of dependencies, but unfortunately this is not currently implemented. But I guess we need to add some API to make it possible for a mixin to declare this information and then order the evaluation accordingly.

It’s related to this: Execution Order of Mixins · Issue #1820 · opentap/opentap · GitHub

1 Like
3

Thanks for explaining, makes sense, and for the link to the open issue. At least I know not to use expressions that pull in other output values for limits. I am glad I caught that as tests could have passed when they should have failed.

Was the source for expressions once publicly available open source, but now is not?