Hi, I’m using Keysight Test Automation (KS8400B).
I’m creating a tap plan that needs to be run on different machines, each one with its own user and file structure.
I think that a good way of handling this would be by creating a “generic” tap plan which imports paths (working folders, application path, etc) in all steps from somewhere.
One possible way would be using environment variables, but i’m not sure if they are supported in OpenTAP, although my tests seems to suggest that they are not (see below).
Are environment variables supported in any way?
if so, how can i work with them?
If not, how do you suggest to proceed?
Tests
Set and get environment variables
Expected behavior: MY_VARIABLE1 should be in env_variables1.txt, MY_VARIABLE 1 and 2 should be in env_variables2.txt
Result: MY_VARIABLE2 is found in env_variable2.txt, but MY_VARIABLE1 is not found in neither files
<?xml version="1.0" encoding="utf-8"?>
<TestPlan type="OpenTap.TestPlan">
<Steps>
<TestStep type="OpenTap.Plugins.BasicSteps.ProcessStep" Id="ad64b898-4446-400e-9ab4-d4954c6c1b0a">
<Application>powershell.exe</Application>
<Arguments>$Env:MY_VARIABLE1 = "test"; Get-ChildItem -Path "env:" > env_variables1.txt</Arguments>
<WorkingDirectory>C:\path\to\target\folder</WorkingDirectory>
<EnvironmentVariables />
<Timeout>0</Timeout>
<AddToLog>false</AddToLog>
<CheckExitCode>false</CheckExitCode>
<RegularExpressionPattern>
<Value>(.*)</Value>
<IsEnabled>false</IsEnabled>
</RegularExpressionPattern>
<VerdictOnMatch>Pass</VerdictOnMatch>
<VerdictOnNoMatch>Fail</VerdictOnNoMatch>
<ResultRegularExpressionPattern>
<Value>(.*)</Value>
<IsEnabled>false</IsEnabled>
</ResultRegularExpressionPattern>
<ResultName>Regex Result</ResultName>
<Behavior>GroupsAsDimensions</Behavior>
<DimensionTitles></DimensionTitles>
<Name Metadata="Step Name">Run Program</Name>
</TestStep>
<TestStep type="OpenTap.Plugins.BasicSteps.ProcessStep" Id="66f2e5c7-2780-4f67-9051-08eb0ccc371a">
<Application>powershell.exe</Application>
<Arguments>Get-ChildItem -Path "env:" > env_variables2.txt</Arguments>
<WorkingDirectory>C:\path\to\target\folder</WorkingDirectory>
<EnvironmentVariables>
<EnvironmentVariable>
<Name>MY_VARIABLE2</Name>
<Value>test</Value>
</EnvironmentVariable>
</EnvironmentVariables>
<Timeout>0</Timeout>
<AddToLog>false</AddToLog>
<CheckExitCode>false</CheckExitCode>
<RegularExpressionPattern>
<Value>(.*)</Value>
<IsEnabled>false</IsEnabled>
</RegularExpressionPattern>
<VerdictOnMatch>Pass</VerdictOnMatch>
<VerdictOnNoMatch>Fail</VerdictOnNoMatch>
<ResultRegularExpressionPattern>
<Value>(.*)</Value>
<IsEnabled>false</IsEnabled>
</ResultRegularExpressionPattern>
<ResultName>Regex Result</ResultName>
<Behavior>GroupsAsDimensions</Behavior>
<DimensionTitles></DimensionTitles>
<Name Metadata="Step Name">Run Program (1)</Name>
</TestStep>
</Steps>
<Package.Dependencies>
<Package Name="OpenTAP" Version="^9.22.3+76719a3b" />
</Package.Dependencies>
</TestPlan>
Using environment variables from the GUI
Expected behavior: write “test” in "MY_VARIABLE\test.txt
Result: Error running “Run Program”: The directory name is invalid.
<?xml version="1.0" encoding="utf-8"?>
<TestPlan type="OpenTap.TestPlan">
<Steps>
<TestStep type="OpenTap.Plugins.BasicSteps.ProcessStep" Id="d0a23e5a-ae98-427e-9d2a-e7e0b8a2a71c">
<Application>powershell.exe</Application>
<Arguments>echo "test" > test.txt</Arguments>
<WorkingDirectory>%MY_VARIABLE%</WorkingDirectory>
<EnvironmentVariables>
<EnvironmentVariable>
<Name>MY_VARIABLE</Name>
<Value>C:\path\to\target\folder</Value>
</EnvironmentVariable>
</EnvironmentVariables>
<Timeout>0</Timeout>
<AddToLog>false</AddToLog>
<CheckExitCode>false</CheckExitCode>
<RegularExpressionPattern>
<Value>(.*)</Value>
<IsEnabled>false</IsEnabled>
</RegularExpressionPattern>
<VerdictOnMatch>Pass</VerdictOnMatch>
<VerdictOnNoMatch>Fail</VerdictOnNoMatch>
<ResultRegularExpressionPattern>
<Value>(.*)</Value>
<IsEnabled>false</IsEnabled>
</ResultRegularExpressionPattern>
<ResultName>Regex Result</ResultName>
<Behavior>GroupsAsDimensions</Behavior>
<DimensionTitles></DimensionTitles>
<Name Metadata="Step Name">Run Program</Name>
</TestStep>
</Steps>
<Package.Dependencies>
<Package Name="OpenTAP" Version="^9.22.3+76719a3b" />
</Package.Dependencies>
</TestPlan>
Hi @mattia.lecci,
We dont have support for this at the moment, but it could be added to the expressions package (supported from 9.22). Then you’d be able to assign it from an expression like envvar(“MY_VARIABLE”).
Maybe you could install 9.22 and play abit around with expressions. In-depth description of the feature can be found in the developer guide: OpenTAP
Thank you for the reply @rolf_madsen.
I wasn’t aware of expressions, it’s definitely a good and useful addition to opentap!
Starting from there, i would also suggest to add the possibility to create variables, as I think they are essential to handle a structured test plan.
Maybe adding a test step called Variable, with a name and value which can be referenced in later test steps with, e.g., expressions.
Imagine the creation of multiple subfolders based on the repetition and or parameterization of a test step, each one with an easily referenceable name coming from, e.g., a variable.
Something like test{i} with i=0,1,2,....
Basically, right now I am missing an essential way to structure my test plan.
Be it an environment variable, or just simply a variable put at the beginning of the test plan, i just want a single point where I can insert some relevant information (e.g., paths for the specific testbed on which the test plan is running) and passing information from one test plan to the other, as if i was programming graphically.
Is there anything like this on the OpenTAP roadmap?
Yes! Actually we already have it, but it is called “mixins”.
You can use that to add a Number “i” which you can refer to from the expression. You can use that with parameterizations to create any kind of calculation or sweep i 0,1,2,3,… .
Thank you for the pointer, it is really appreciated!
I was looking for “variables” or “constants”, i was not familiar with the concept of “mixins”.
Looking at the doc it seems to be what I’m looking for.
I don’t understand how to make it work, though.
I installed both the Expressions and the Basic Mixins packages
09:36:48.110 Installation Adb Agents - 1.2.16
09:36:48.110 Installation Android Debug Bridge - 1.0.7
09:36:48.110 Installation Basic Mixins - 0.0.1-beta.10+9c78a5a2
09:36:48.110 Installation CSV - 9.12.0+2e85cff0
09:36:48.110 Installation Developer's System - 9.22.0+284da967
09:36:48.110 Installation Editor - 9.22.0+284da967
09:36:48.110 Installation Expressions - 1.0.1+36dc954d
09:36:48.110 Installation Keysight Licensing - 1.4.4+14f363d3
09:36:48.110 Installation KS8000 - 9.22.0+284da967
09:36:48.110 Installation OpenTAP - 9.22.3+76719a3b
09:36:48.110 Installation OSIntegration - 1.4.2+15f32a31
09:36:48.110 Installation Results Viewer - 9.22.0+284da967
09:36:48.110 Installation SDK - 9.22.3+76719a3b
09:36:48.110 Installation SQLite and PostgreSQL - 9.4.2+3009aace
09:36:48.110 Installation SSH - 1.0.0+f4ad9710
09:36:48.110 Installation Timing Analyzer - 9.22.0+284da967
09:36:48.110 Installation Visual Studio SDK - 1.8.0+d5099542
09:36:48.110 Installation WPF Controls - 9.22.0+284da967
09:36:48.110 Installation Keysight Fixed License Manager - 1.6.2+d58f8a89
09:36:48.110 Installation PathWave License Manager - 2.7.0+398d3a87
I cannot find anywhere a way to add Mixins to my test plan. The doc says to look into the “Settings context menu” but i only see Editor, Engine, Results, Bench>Connections, Bench>DUTs, Bench>Instruments.
Am i missing something?
I also tried a couple of small tests related to expressions but they are not behaving as expected.
<?xml version="1.0" encoding="utf-8"?>
<TestPlan type="OpenTap.TestPlan">
<Steps>
<TestStep type="OpenTap.Plugins.BasicSteps.LogStep" Id="5067ed56-9581-45ec-a2a3-0e6181042c75">
<LogMessage>The sum 1+2={1+2}</LogMessage>
<Severity>Info</Severity>
<Name Metadata="Step Name">Log Output</Name>
</TestStep>
<TestStep type="OpenTap.Plugins.BasicSteps.ProcessStep" Id="a93d8816-8096-4714-a583-9cab38dd599f">
<Application>powershell.exe</Application>
<Arguments>echo 'The sum 1+2={1+2}' > test.txt</Arguments>
<WorkingDirectory>C:\path\to\working\directory</WorkingDirectory>
<EnvironmentVariables />
<Timeout>0</Timeout>
<AddToLog>false</AddToLog>
<CheckExitCode>false</CheckExitCode>
<RegularExpressionPattern>
<Value>(.*)</Value>
<IsEnabled>false</IsEnabled>
</RegularExpressionPattern>
<VerdictOnMatch>Pass</VerdictOnMatch>
<VerdictOnNoMatch>Fail</VerdictOnNoMatch>
<ResultRegularExpressionPattern>
<Value>(.*)</Value>
<IsEnabled>false</IsEnabled>
</ResultRegularExpressionPattern>
<ResultName>Regex Result</ResultName>
<Behavior>GroupsAsDimensions</Behavior>
<DimensionTitles></DimensionTitles>
<Name Metadata="Step Name">Run Program</Name>
</TestStep>
</Steps>
<Package.Dependencies>
<Package Name="OpenTAP" Version="^9.22.3+76719a3b" />
</Package.Dependencies>
</TestPlan>
Results:
11:22:10.191 TestPlan -----------------------------------------------------------------
11:22:10.191 TestPlan Starting TestPlan 'expressions_test' on 11/29/2023 11:22:10, 2 of 2 TestSteps enabled.
11:22:10.194 KS8000 KS8000 installation id 7a6e7dc9edb76cf17c9fadfff3018bbf3bb0ccab [302 us]
11:22:10.196 CSV Resource "CSV" opened. [36.4 us]
11:22:10.210 TestPlan Saved Test Plan XML [17.8 ms]
11:22:10.227 TestPlan PrePlanRun Methods completed [38.1 us]
11:22:10.227 TestPlan "Log Output" started.
11:22:10.227 TestStep The sum 1+2={1+2}
11:22:10.228 TestPlan "Log Output" completed. [166 us]
11:22:10.228 TestPlan "Run Program" started.
11:22:10.228 TestStep Starting process powershell.exe with arguments "echo 'The sum 1+2={1+2}' > test.txt"
11:22:11.507 TestPlan "Run Program" completed. [1.27 s]
11:22:11.507 TestPlan Test step runs finished. [1.28 s]
11:22:11.513 Summary ----- Summary of test plan started 11/29/2023 11:22:10 -----
11:22:11.513 Summary Log Output 166 us
11:22:11.513 Summary Run Program 1.27 s
11:22:11.513 Summary ------------------------------------------------------------
11:22:11.513 Summary -------- Test plan completed successfully in 1.31 s --------
11:22:11.513 CSV OnTestPlanRunCompleted for CSV. [45.6 us]
11:22:11.514 CSV Resource "CSV" closed. [24.6 us]
11:24:28.429 Main Saved test plan to C:\path\to\test\plan\expressions_test.TapPlan [26.9 ms]
expected:
[...]
11:22:10.227 TestStep The sum 1+2=3
[...]
The sum 1+2={1+2}
expected:
The sum 1+2=3
I installed the packages but they don’t seem to have any effect.
What am i doing wrong?
PS: this discussion seems to be going off-topic, do you prefer to close this thread and open a more approriate one?
What you need is available in 9.22 with the mixins package installed. You can add mixins and assign expressions through the right-click dialog in the test step / test plan settings.
In the XML, it does not look exactly like what you expect, so this wont work, but you wont need to as you should be able to do it through the editor.
Hi @rolf_madsen , thank you again for your input.
You can find an example attached, it works great and this workflow will definitely improve our test plans!
You can find attached an example test using mixin to define and manipulate folders, if anybody will ever need an example to start from.
mixin_test.TapPlan (5.8 KB)
My only complaint is that the documentation could largely be improved for such an essential feature.
The Help > How To… does not report Mixins and Expressions at all, and the online documentation only mentions some basic knowledge about synthax, but not that you need to right-click and select expression or how to add a mixin.
With those additions I’m sure you’ll make lots of users happy about the product 
Hi @mattia.lecci,
Thanks for the feedback! It is an interesting way to use powershell, expressions and mixins together! Also I am very happy you find this feature useful!
The expressions feature brings a lot of new flexibility that we still need to discover the range of.
Regarding help, improved help pages are on the way.
1 Like