Test Plan Loading Error when new packages are being installed

Hi all,

We are currently in the midst of upgrading our OpenTAP version to 9.21.1 from 9.18.5.

There is an issue where we try to load the test plan, after doing a fresh installation of our tap packages. The error thrown shows that it required the dependency packages, even though it has already been installed and can be seen in the directory. In addition, after restarting the application, it will start to work with no errors. We compared with 9.21.1 and 9.18.5 without any code changes, and it is only reproducible in 9.21.1.

This is the error:

This is how we are loading the test plan:

Any ideas what could be the issue?

Hi @antonio.quizon,

It is a bit hard to figure out without more context.

I’d check if there are any duplicate assemblies with different version in your installation folder. This is a very common issue - There should be a warning in the log if that is the case.

Hi @rolf_madsen ,

I do not see any duplicate assemblies. Also, after restarting, it will start to work. So I doubt it would be that issue. Is there a way I can provide more context?

You are right, that is probably not the case then.

You said restarting the application helps. Are you installing new plugins while the application is running?

Hi @rolf_madsen,

Yes we are installing as part of our application during runtime.

I double checked, and saw there are duplicate assemblies in the nested directories in the installation folder. After changing to the same version, it seems the problem was resolved.

May I know why this was an issue? And why it worked for 9.18.5 and not for 9.21.1?

Thanks again, for the help.

I am not sure, it depends a bit on which DLLs you changed. It could be as simple as the resolution order has been changed a bit, but I don’t really remember we did something like that.

Hi @rolf_madsen,

I did a further check and tested it out. I was not able to face the error because my test plan xml did not have the following dependency. After using a test plan that have the dependency, I am unable to load the test plan again, and it is giving the same error, even with matching version of assemblies. Again, it will be resolved after restarting the application.

image

Do you know what could be the issue?

Do you install the packages in the dependencies when you load the plan?

As we are using Tap Packages for our application, and then extracting the test plan, we are installing the tap plugins which are included as part of the tap package dependency in the package.xml.

How does it work if the package is already installed in a different version that is already loaded?

Generally, it is not supported to load a new version of a dll while the application is running.

It can work if you did not use the DLL yet, so if you update a package immediately after starting it should work, but if you loaded anything from it should fail.

If there is a new version of packages available while the application is running, generally, it will require a restart of the application as the DLLs will be locked. After restarting, that would be fine and can upgrade normally. But the issue in this case, it is only happening when there is a fresh installation of packages, and it is only 1 version being used.

OK, that makes sense. There is another thing we do in Editor related to when packages are installed. it is set up during startup.

                Installation installation = new Installation(Directory.GetCurrentDirectory());
                installation.PackageChangedEvent += () =>
                {
 
                  // New packages has been installed. Update the plugin manager and  notify the UI that this has happened
                    PluginManager.SearchAsync().Wait();
                    OnNotifyPluginsChanged();
                };
                // Type data cache is invalidated when new plugins are installed
                // around the plugin manager. Notify the UI that this has happened
                TypeData.TypeCacheInvalidated += (s, e) =>   OnNotifyPluginsChanged();

Something like this is required for the serializer to know about the new plugin types.

Hmmm. We are doing our own installation of the packages, instead of using the Installation() class. In any case, we always call the PluginManager.SearchAsync() when a new package is installed, or at launch of the application.

May I know what triggers the PackageChangedEvent? And can you share more on the OnNotifyPluginsChanged() and the EventHandler TypeCacheInvalidated? Isn’t PackageChangedEvent sufficient enough to trigger the OnNotifyPluginsChanged()?

Thanks for the continous response.

The PackageChagedEvent is triggered by package being installed. OnNotifyPluginsChanged is just whatever the UI needs to do when a package has been instaleld.

The type cache is related to new virtual types being added by other plugins. usually dynamic types that does not belong to any specific package.

We are already calling PluginManager.SearchAsync() during package installation. Based on those information, do you think there are more necessary actions to be done?

No, that is probably ok. Just to summarize, everything works after removing the duplicate dlls, right?

Nope, the issue still persists even if the dlls are the same version. And after restarting, it works, so I highly doubt that is the issue.

Ok, I think I understand why this is happening.

You have a custom way of installing the package right?

There is a cache inside OpenTAP which keeps track of installed packages. When you install the new package this cache does not get invalidated because you have your own way of installing packages. That means it thinks the same packages are installed even though you have changed things.

To fix it, you can try bumping the package ‘changeid’.

How OpenTAP handles the change ID you can see in this code: https://github.com/opentap/opentap/blob/16230ee2ec3aeb826859c9d0360d29c1a01e7e86/Package/PackageActions/IsolatedPackageAction.cs#L81C36-L81C36

Basically there is a file in Packages called .changeId. It contains a 64bit integer. If you increment that integer and write it back to the file, OpenTAP should recognize the cache as invalidated.

Maybe something you could try.

Hi @rolf_madsen,

Thanks for the response again. Yes we have a custom way of installing packages. This seemed to have fixed the issue.

May I know more on this in order to understand the impacts and how to implement it correctly?

  1. When is the correct time to increment the changeid? Before PluginManager.SearchAsync or Before loading the test plan?
  2. Any reason that it only failed for fresh installation, and it worked for upgrade installations? And also, how it worked for 9.18.5 and not 9.21?

Hi @antonio.quizon,

  1. As soon as you unpack the files increment the version.
  2. I think we did some performance improvements relying more on the cache ID. That might be reason. I have no idea about fresh install vs upgrade.