Segmentation Fault error

Technical
1

I’m facing an issue when running test plans via CLI:

/home/qdtest/bin/tap: line 2: 742475 Segmentation fault      dotnet "$(dirname "$(readlink -f "$0")")/tap.dll" "$@"

It happens after a while when going through the same referenced plans using Repeat step.

System info:

  • Ubuntu 20.04.1 LTS
  • .NET Core SDK 2.1.816
  • .NET Core runtimes 2.1.28
  • OpenTAP 9.15.2+39e6c2a2
  • Python 3.7.10
  • OpenTAP Python plugin 2.2.0+31a47a25

I’m happy to provide further info about my system or plan if necessary. Any help in resolving this issue would be highly appreciated.

1 Like
3

Can you see if this is repeatable with a very simple referenced test plan like Delay/Log? We will probably need to take a deeper look on our end, so identifying something we can reproduce will help.

1 Like
4

@brennen_direnzo I’ll try to reproduce it using built-in steps.

If I run the same plan on Windows it causes the following exception:

Unhandled Exception: System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at Python.Runtime.Runtime.PyObject_Call(IntPtr pointer, IntPtr args, IntPtr kw)
   at Python.Runtime.PyObject.Invoke(PyObject[] args)
   at Python.Runtime.PyObject.InvokeMethod(String name, PyObject[] args)
   at Keysight.OpenTap.Plugins.Python.PythonStep.<Run>b__13_0()
   at Keysight.OpenTap.Plugins.Python.PyThread.<>c__DisplayClass10_0.<Invoke>b__0()
   at Keysight.OpenTap.Plugins.Python.PyThread.pyThreadMain(Action startup)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()

To give you an idea about the structure of my plan:

  • high-level plan contains Repeat step with a dozen of referenced plans
  • referenced plans are using built-in steps and custom Python steps
  • custom Python steps are consolidated in subplans that run in two parallel threads using Parallel / Sequence approach

High-level test plan:

Repeat
     Test Plan 1 Reference
     ...
     Test Plan N Reference

Each referenced plan:

Arbitrary Steps
Parallel Step
    Sequence Step
         Test Plan X Reference
    Sequence Step
         Test Plan X Reference
Arbitrary Steps
Parallel Step
    Sequence Step
         Test Plan Y Reference
    Sequence Step
         Test Plan Y Reference
Arbitrary Steps

I hope it gives more insights.

1 Like
5

Thanks @pythonized. Since you are using the Parallel Step, I think there is a good chance something in your Test Plan is not thread safe. It could be something fundamental within Python.NET or could be related to your implementation, so the simple test plan with basic steps to reproduce this will help guide us.
I’ll try to find someone on our team to look into this.

2 Likes
6

@brennen_direnzo I used Python PluginExample as a basis and created a sequence with a structure similar to the real one I have.

I added an intentionally invalid statement to Run() method in DischargeStep.py at line 48:
self.UpgradeVerdict(OpenTap.Verdict.PASS)

I also added an exception to Run() method in ChargeStep.py at line 51:
raise Exception("Exception from ChargeStep")

main.TapPlan crashes consistently within a minute of execution. I’m attaching Instruments profile, as well as main and sub plans. Let me know if there’s something else you might need from my side.

For now I’ll make sure to handle exceptions in the code of my plugins, hoping it will prevent crashing.
Instruments.xml (1.9 KB)
sub.TapPlan (2.0 KB)
main.TapPlan (17.5 KB)

1 Like
7

Thanks @pythonized. This will be very helpful to help solve this. Just to make sure I understand correctly, the crash seems to be caused by an exception that is not being caught and handling the exceptions prevents the crash?

On the C#, even exceptions that are not handle in the plugins are caught later. It is possible the Python side misses some, but hopefully we can handle these more gracefully.

1 Like
8

@brennen_direnzo Yes, it seems that exceptions that aren’t handled in the code but handled by OpenTAP itself cause this crash at some point. It doesn’t happen immediately though. Generally, exceptions in Python seem to be caught properly like in C# but probably in certain cases something goes wrong.

I’ll need to verify whether exception handling really prevents this issue within longer tests but I have high hopes it does :crossed_fingers:

1 Like
9

Ok great. I’ve submitted this issue to track things, so let us know if you have any updates:

1 Like