Access test step result before it is published

Technical
1

Hi there,
I am currently trying to achieve retry of test step for few times if it fails. Also, I want to only publish the last retry result of the test step. I have tried using Retry test step but it seems to publish all the retry results. Has anyone already done that before or know how to do it?
Thanks,
Chandrima

2 Likes
2

Hi Chandrima,

I have been using the test step below to run all child steps until all verdicts are passing or the max iteration is reached.

I hope this helps,
Diego

using System.ComponentModel;
using System.Xml.Serialization;
using OpenTap;

namespace TapExtensions.Steps.FlowControl
{
    [Display("Retry",
        Groups: new[] { "TapExtensions", "Steps", "FlowControl" },
        Description: "Run all child steps until all verdicts are passing or max iteration is reached.")]
    [AllowAnyChild]
    public class Retry : TestStep
    {
        #region Settings

        [Display("Max repeat count", "Maximum number of iteration attempts for repeating child steps")]
        public uint MaxIteration { get; set; } = 3;

        [XmlIgnore]
        [Browsable(false)]
        public int Iteration { get; set; }

        #endregion

        public override void Run()
        {
            Iteration = 0;
            while (Iteration < MaxIteration)
            {
                Iteration++;

                if (Iteration > 1)
                    Log.Warning($"Retrying attempt {Iteration} of {MaxIteration} ...");

                ResetResultOfChildSteps();
                RunChildSteps();
                Verdict = Verdict.NotSet;
                UpgradeVerdictWithChildVerdicts();

                // Exit loop if retry attempt is successful
                if (Verdict == Verdict.Pass)
                    break;
            }
        }

        private void ResetResultOfChildSteps()
        {
            var enabledChildSteps = RecursivelyGetChildSteps(TestStepSearch.EnabledOnly);
            foreach (var step in enabledChildSteps)
                step.Verdict = Verdict.NotSet;
        }

        private void UpgradeVerdictWithChildVerdicts()
        {
            var enabledChildSteps = RecursivelyGetChildSteps(TestStepSearch.EnabledOnly);
            foreach (var step in enabledChildSteps)
                UpgradeVerdict(step.Verdict);
        }
    }
}
3 Likes
3

Hi diego,

Thanks for your response. Currently OpenTAP already provides one Repeat test step for the same. But even there it still publishes all the results(3 results if there were 3 retries done). What I am looking for is to only publish the final result of the retries. Did you by any chance done that before?

Thanks,
Chandrima

1 Like
4

Hello @chandrima.dasgupta and Welcome to the Opentap forum!

Based on your description I think you will need to implement a custom Result Listener, the Result Listener will receive the Publish for all 3 retries during the OnResultPublished:

    public override void OnResultPublished(Guid stepRun, ResultTable result)

In here you can validate if you have received a result for a given Guid and only add the values if the result has a verdict of Pass.

Another idea is to filter and only log results with Verdict equal to pass during the OnTestStepRunCompleted:

    public override void OnTestStepRunCompleted(TestStepRun stepRun)

but the point is that you need a custom result listener to achieve what you are looking for.

Carlos Montes

3 Likes
5

Thank you @diego_tacconi

Saved my day

3 Likes