ResultSettings in new Session behaviour

Just returning to the software for Parallel dynamic testplans - Technical - OpenTAP Community Forum after some time working on other aspects of the project.

I was hoping to use a ResultListener to synchronize some unit tests of the concurrent runs, but Session.Create() doesn’t seem to clone the ResultSettings in the way I was expecting.

I’ve got a ResultListener defined in my test project as follows

class SyncListener : ResultListener
    private readonly SyncMessageStore messageStore;
    private string dut;

    public SyncListener(SyncMessageStore syncMessageColletor)
        this.messageStore = syncMessageColletor;

    public override void OnTestPlanRunStart(TestPlanRun planRun)
        dut = (new MacroString { Text = "<DUT>" }).Expand(planRun);

    public override void OnTestStepRunStart(TestStepRun stepRun)

        messageStore.ActiveTestName.AddOrUpdate(dut, stepRun.TestStepName, (key, oldVal) => stepRun.TestStepName);

    public override void OnTestPlanRunCompleted(TestPlanRun planRun, Stream logStream)
        base.OnTestPlanRunCompleted(planRun, logStream);
        messageStore.Complete.AddOrUpdate(dut, true, (key, oldVal) => true);

An object to store the bits I want to capture from each test run, at the moment just the name of the test that’s running and if the run’s complete.

class SyncMessageStore
    public ConcurrentDictionary<string, bool> Complete { get; } = new();
    public ConcurrentDictionary<string, string> ActiveTestName { get; } = new();

My test code (stripped down for brevity ) is something like this

SyncMessageStore messageStore = new();
ResultSettings.Current.Add(new SyncListener(messageStore));

serialNumberProvider.SendSerialNumber("000000001"); // This kicks off the first test run

serialNumberProvider.SendSerialNumber("000000002"); // This kicks off the second test run

while (!messageStore.ActiveTestName.All(x => x.Value == "Wait")) { }

Assert.Equal("Waiting", singleViews[0].StatusText);
Assert.Equal("Waiting", singleViews[1].StatusText);

I have a test manager which deals with getting test plans, and starting runs etc.

using (Session.Create(SessionOptions.RedirectLogging | SessionOptions.OverlayComponentSettings))
    TestPlan = testPlanProvider.GetTestPlanFromMaterialNumber(MaterialInfo.MaterialNumber);
    TestPlan.BreakOffered += TestPlan_BreakOffered;

    List<ResultListener> resultListeners = new(ResultSettings.Current.Cast<ResultListener>())
        new SimpleListener(this),
        new LogResultListener() { FilePath = new MacroString() { Text = "TestRunLogs\\<Serial> <Date>.txt", Context = TestPlan } },

    List<ResultParameter> metaDataParameters = new()
        new("", "DUT", TestPosition, "DUT"),
        new("", "Serial", SerialNumber, "Serial")
    CurrentRun = TestPlan.ExecuteAsync(resultListeners, metaDataParameters, null, CancellationToken.None);

I was expecting that using OverlayComponentSettings would give me a new instance of for ResultSettings.Current based off the one set up in the test but that doesn’t seem to be the case.

The API docs say about OverlayComponentSettings , “Component settings are cloned for the sake of this session. Instrument, DUT etc instances are cloned. When this is used, test plans should be reloaded in the new context.” Do ResultListeners not fall under “Instrument, DUT etc”?

ResultListeners does fall under ‘etc’

You say " I was expecting that using OverlayComponentSettings would give me a new instance of for ResultSettings.Current based off the one set up in the test but that doesn’t seem to be the case."

So you are saying that when you call ResultSettings.Current, it uses the same instances as before you created the new session?

Hi Rolf,
No, it looks like it creates a new empty ResultSettings.
Stepping through it, right before creating the new session, I queried the ResultSettings via the immediate window and got the following:

ResultSettings.Current.Select(x => x.Name)
Count = 1
    [0]: "MyListener"

Then stepping into the using block and running again, I get

ResultSettings.Current.Select(x => x.Name)
Exception thrown: 'System.Exception' in OpenTap.dll
Count = 0
    Empty: "Enumeration yielded no results"

I’m not sure what that exception is, I’ll try and investigate further.

But just so I understand correctly, I should be seeing a single (new) instance of “MyListener” in that list, right?

OK, strange… Could you try saving the ResultSettings before creating the new session? It could be that we need to add some extra handling for unsaved setting modifications when creating the new session.

I’ve been having a look this morning and here’s what I’ve come up with so far. Hopefully, it’ll help.

  • Saving the ResultsSettings before creating the new Session had no effect. I got the same output in the Immediate Window.

  • I tried adding to InstrumentSettings. This worked as expected, I had one instance of “SimpleInstrument” in InstrumentSettings.Current both before and after creating the Session. This was without saving.

  • I tried adding to ResultSettings, saving, aborting the execution, re-running, and adding to ResultSettings again. I expected to see two instances of my Listener, but there was only the one I just added.

  • Next I inspected Results.xml. It did have the Listener but I noticed the path was different from Instruments.xml (which seems to be working correctly).
    Though this may well be correct, as the listeners probably wouldn’t change with different bench settings. I only mention it because it didn’t seem to have loaded from that file.

This sounds like a bug to me. Can you open an issue on Issues · opentap/opentap · GitHub?

ComponentSettings not being cloned when creating new Session · Issue #710 · opentap/opentap (