I’m using Test Plan Reference step and plans parametrization a lot, and it seems to me there’s a big memory/performance issue there. I’m not sure whether it’s related to the combination of the two features or just one of them.
Let’s say I have a test plan with 15 exposed parameters. If I do
tap run <test plan path> --list-external-parameters -v I can see it takes around 100ms for the parameters to be listed. Now, if create a new test plan, add a Test Plan Reference step that utilizes this plan, and propagate parameters to the newly created plan it already takes around 6s for the parameters to be listed. If the initial test plan also have references to other plans it takes dozens of seconds for the parameters to be listed and just the listing can consume hundreds of megabytes of memory. More complex plans take minutes and gigabytes of memory for the parameters to be listed. These numbers are just to give you an idea.
Listing parameters is not the issue per se, I just find it to be a good indicator of memory/performance issues when parsing/loading test plans which in the end makes it impossible to run them.
Has anyone experienced issues like that?