I am making an operator GUI for OpenTAP for a company that has decided to re-focus on test security. Instead of using files, they would like me to load instructions from a database.
I have this working for a single file currently. However, our tests are broken into 3 layers:
- Main Sequence
- Test Segment (a single topic)
- Test Case (a single test)
The way we do this is via TestPlanReferences from the BasicSteps - A single Main Sequence contains many Segments which contain many Test Cases, all of which are their own tapplan files. Now that I need to load from the database, it’s looking like I can’t use the standard TestPlanReference since it is looking for a file directly.
I was looking at making my own TestCase reference step that would inherit from TestPlanReferences and override the loading step, but nothing in it seems overrideable.
What would you all suggest for the next best solution? The TestPlanReference code seems to be very complicated and I’m not sure that re-producing it myself is a good idea. Is there a standard operating procedure for referencing tapplans inside of other tapplans without using the filesystem for storage?
Hi @matthewdumas,
If you want something fully featured, it is going to be quite complicated.
Also, note that it is not generally supported to modify the test plan itself during test plan execution.
If you want something more simple, you could load test plans on the fly and run them in an isolated context. This is done so it seems like the sub plan is just one test step.
This is a feature implemented in test plan reference which has not been made available in the public API (however its still in the source), basically it goes like this:
public override void Run()
{
// Run the test plan in an isolated context:
LogForwardingTraceListener forwarder = null;
var xml = GetTheTestPlanXML(); // plan.SerializeToString();
if(OpenTap.Log.Context is ILogContext2 ctx2)
forwarder = new LogForwardingTraceListener(ctx2);
// This creates a session. All resources gets deserialized again
// All results gets forwarded via the SubPlanResultListener
using (Session.Create())
{
var plan2 = Utils.DeserializeFromString<TestPlan>(xml);
plan2.PrintTestPlanRunSummary = false;
if(forwarder != null)
OpenTap.Log.AddListener(forwarder);
var subRun = plan2.Execute(new IResultListener[] {new SubPlanResultListener(Results)});
UpgradeVerdict(subRun.Verdict);
}
}
Forwarding of results and log messages.
class SubPlanResultListener : ResultListener
{
readonly ResultSource proxy;
public SubPlanResultListener(ResultSource proxy) => this.proxy = proxy;
public override void OnResultPublished(Guid stepRunId, ResultTable result)
{
base.OnResultPublished(stepRunId, result);
proxy.PublishTable(result);
}
}
class LogForwardingTraceListener : ILogListener
{
readonly ILogContext2 forwardTo;
public LogForwardingTraceListener(ILogContext2 forwardTo) => this.forwardTo = forwardTo;
public void EventsLogged(IEnumerable<Event> Events)
{
foreach (var evt in Events)
{
if (evt.Source == "TestPlan" || evt.Source == "N/A")
if(evt.EventType != (int)LogEventType.Error)
continue;
forwardTo.AddEvent(evt);
}
}
public void Flush() { }
}
Hi rolf,
Thanks for the help, again. I did start to delve deeply into how the references work and I was thinking that run function might have been useful, but I couldn’t readily figure out how to use that portion from the main api without modifying BasicSteps.
However, after playing around with it yesterday, I decided that it might be better to reconcile the child plans into a single plan on insert into the database. That way, I avoid having to re-write a lot of the code and play around with triggering the sub runs.
I was planning to update the ticket with my decision after I made a simple prototype work.
Got it. In that case, you might like a new feature which enables you to convert test plan references directly into sequence steps.
You can find it in the release note her: Release 9.27.1 · opentap/opentap · GitHub
Actually, stumbling over that feature is exactly what made me think to do this.