Access test step result before it is published

Hi there,
I am currently trying to achieve retry of test step for few times if it fails. Also, I want to only publish the last retry result of the test step. I have tried using Retry test step but it seems to publish all the retry results. Has anyone already done that before or know how to do it?
Thanks,
Chandrima

2 Likes

Hi Chandrima,

I have been using the test step below to run all child steps until all verdicts are passing or the max iteration is reached.

I hope this helps,
Diego

using System.ComponentModel;
using System.Xml.Serialization;
using OpenTap;

namespace TapExtensions.Steps.FlowControl
{
    [Display("Retry",
        Groups: new[] { "TapExtensions", "Steps", "FlowControl" },
        Description: "Run all child steps until all verdicts are passing or max iteration is reached.")]
    [AllowAnyChild]
    public class Retry : TestStep
    {
        #region Settings

        [Display("Max repeat count", "Maximum number of iteration attempts for repeating child steps")]
        public uint MaxIteration { get; set; } = 3;

        [XmlIgnore]
        [Browsable(false)]
        public int Iteration { get; set; }

        #endregion

        public override void Run()
        {
            Iteration = 0;
            while (Iteration < MaxIteration)
            {
                Iteration++;

                if (Iteration > 1)
                    Log.Warning($"Retrying attempt {Iteration} of {MaxIteration} ...");

                ResetResultOfChildSteps();
                RunChildSteps();
                Verdict = Verdict.NotSet;
                UpgradeVerdictWithChildVerdicts();

                // Exit loop if retry attempt is successful
                if (Verdict == Verdict.Pass)
                    break;
            }
        }

        private void ResetResultOfChildSteps()
        {
            var enabledChildSteps = RecursivelyGetChildSteps(TestStepSearch.EnabledOnly);
            foreach (var step in enabledChildSteps)
                step.Verdict = Verdict.NotSet;
        }

        private void UpgradeVerdictWithChildVerdicts()
        {
            var enabledChildSteps = RecursivelyGetChildSteps(TestStepSearch.EnabledOnly);
            foreach (var step in enabledChildSteps)
                UpgradeVerdict(step.Verdict);
        }
    }
}
3 Likes

Hi diego,

Thanks for your response. Currently OpenTAP already provides one Repeat test step for the same. But even there it still publishes all the results(3 results if there were 3 retries done). What I am looking for is to only publish the final result of the retries. Did you by any chance done that before?

Thanks,
Chandrima

1 Like

Hello @chandrima.dasgupta and Welcome to the Opentap forum!

Based on your description I think you will need to implement a custom Result Listener, the Result Listener will receive the Publish for all 3 retries during the OnResultPublished:

    public override void OnResultPublished(Guid stepRun, ResultTable result)

In here you can validate if you have received a result for a given Guid and only add the values if the result has a verdict of Pass.

Another idea is to filter and only log results with Verdict equal to pass during the OnTestStepRunCompleted:

    public override void OnTestStepRunCompleted(TestStepRun stepRun)

but the point is that you need a custom result listener to achieve what you are looking for.

Carlos Montes

2 Likes