Multi-target Nunit 3 test project that test AssemplyAttributes pass for net45 and fail for netcoreapp2.0

[moh-hassan]

I have multi-target NUnit Test Project (Project1.Test) (net45;netcoreapp2.0)
The project reference project1 which is also multi-target (net45;netcoreapp2.0)

The libraries used:

           <ItemGroup>			
		<PackageReference Include="nunit" Version="3.9.0" />
		<PackageReference Include="NUnit3TestAdapter" Version="3.9.0" />
		<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.9.0" />
	  </ItemGroup>

Project1 have a static class StartUp with a property that use CompanyAssemplyAttribute :

     public class StartUp
     {
  private static string CopyRight= CopyrightInfo. Default; // it retrieve the assemplyAttribute   AssemplyCompany
     ....
    }

The test method

    public class StartUpTest
    {
       [Test]
    public void Test1()
    {
      var copyright=StartUp.CopyRight;
      Assert.AreEqual(copyRight,"xyz Company");
    }
     }

I use dotnet CLI command dotnet test.

I run for both frameworks the following command:

    dotnet test

For net45 framework: Test Run Successful

For netcoreapp2.0: Test Fail with an exception:

Failed DefaultSettingTest()
Error Message:
System.AggregateException : One or more errors occurred. (The type initializerfor 'project1.StartUp' threw an exception.)
----> System.TypeInitializationException : The type initializer for 'project1.StartUp' threw an exception.
----> System.ArgumentException : author
Stack Trace:
at System.Threading.Tasks.Task1.GetResultCore(Boolean waitCompletionNotification) at System.Threading.Tasks.Task1.get_Result()
..... other messages-----
--- End of stack trace from previous location where exception was thrown ---
at Project1.StartUp.RunOptionsAsync(String[] args) in F:\project1\Program.cs:line 103
--ArgumentException
at CommandLine.Text.CopyrightInfo..ctor(Boolean isSymbolUpper, String author, Int32[] copyrightYears)
at CommandLine.Text.CopyrightInfo.get_Default()

All assemply attributes are defined in the project properties

The same result for netcoreapp2.1

It seems that netcoreapp2.0 can't initialize the CompanyAssemplyAttribute.

What i missed for netcoreapp2.0 to run successful like net45 when accessing AssemplyAttributes.?

Repro project

[mikkelbu]

Hi @moh-hassan
Can you provide some more information about the CopyrightInfo class. It looks like that this class is failing and that the problem is the author argument passed to the constructor (see the System.ArgumentException : author).

If the code is this, https://github.com/commandlineparser/commandline/blob/0158942eb2e3b5f9322b21b7fadfadb53671a92a/src/CommandLine/Text/CopyrightInfo.cs#L75, then I guess that the problem is that the Company information in the attribute is empty. And as far as I can see you only end up in this case if the code cannot find a AssemblyCopyrightAttribute, but can find a CompanyAssemplyAttribute, so the reflection code must be different.

[mikkelbu]

I just notice that you already have created commandlineparser/commandline#354 (I was going to tell you that the problem probably was in commandline).

Ps have you tried to see if this is a NUnit problem or a general problem? (e.g. if you just run the production code directly via a console app or similar)

[moh-hassan]

Thanks @mikkelbu for reply.

Can you provide some more information about the CopyrightInfo class

CopyrightInfo provide Copyright string based on AssemblyCopyrightAttribute if not null or generated from AssemblyCompanyAttribute . At least one of them should be not null, otherwise it raise an exception.
It reads these Attributes using Reflection: Assemply.GetExecutingAssemply()

Problem Investigation:
I provided Copyright and Company as a project properties in csproj SDK new style.

I inspected the auto generated AssemblyInfo.cs class from MSBuild and both AssemblyCompanyAttribute and AssemblyCopyrightAttribute are exist (they are included in obj\debug folder in the attached file)
The following code is the auto generate AssemplyInfo.cs diring MSBuild:

			/------------------------------------------------------------------------------
			// <auto-generated>
			//     This code was generated by a tool.
			//     Runtime Version:4.0.30319.42000
			//
			//     Changes to this file may cause incorrect behavior and will be lost if
			//     the code is regenerated.
			// </auto-generated>
			//------------------------------------------------------------------------------

			using System;
			using System.Reflection;

			[assembly: System.Reflection.AssemblyCompanyAttribute("MyCompany")]
			[assembly: System.Reflection.AssemblyConfigurationAttribute("Debug")]
			[assembly: System.Reflection.AssemblyCopyrightAttribute("Copyright ©  2018")]
			[assembly: System.Reflection.AssemblyDescriptionAttribute("MyDescription")]
			[assembly: System.Reflection.AssemblyFileVersionAttribute("1.0.0")]
			[assembly: System.Reflection.AssemblyInformationalVersionAttribute("1.0.0")]
			[assembly: System.Reflection.AssemblyProductAttribute("Project2015To2017")]
			[assembly: System.Reflection.AssemblyTitleAttribute("CommandParserDemo")]
			[assembly: System.Reflection.AssemblyVersionAttribute("1.0.0")]

			// Generated by the MSBuild WriteCodeFragment class.

have you tried to see if this is a NUnit problem or a general problem? (e.g. if you just run the production code directly via a console app or similar)

I run project.exe for both net45 and netcoreApp2.0 and they display the valid copyright (as in the generated assemplyinfo) without exception.

I inspect other property (HeaderInfo.Default) which Access other AssemplyAttributes:
// The title is retrieved from "AssemblyTitleAttribute"
// The version is retrieved from AssemblyInformationalVersionAttribute

            HeaderInfo.Default = $" {title}  {version}"

I was surprised:
in net45: HeaderInfo.Default = CommandLine 2.3.0
in netcoreapp2.0: HeaderInfo.Default = testhost 15.9.0 (that is Microsoft.NET.Test.Sdk Version="15.9.0" !!)

It seems that in unit test, the testhost get the assemplyAttributes of its testhost.dll not from the system under test.
It has different behavior in both net45 and netcoreapp2.0

I inspect the testhost.dll which is located in:

                %UserProfile%.packages\microsoft.testplatform.testhost\15.9.0

by right clicking the file -> property\details, I found:

          Copyright is null

I inspected the testhost.dll by dotPeek (.NET Decompiler and Assembly Browser) and found that:

             company Attribute is null

My Conclusion:

• Problem is in TestHost.
• Testhost of Microsoft.NET.Test.Sdk has different behavior in reading AssemplyAttributes in net45(full .NetFramework) and netcoreapp2.x
• It's working fine in net45
• Because these assemply attributies are null in aseemply of testhost.dll, an exception is raised in netcoreapp2.x (it's working fine in net45)

Suggested Solution:
Testhost should unify the reading of assembly attributes for both .NET FullFramework or netcoreapp2.x, especially all the attributes are generated by default and has a value either in the new project style SDK or the old one.
It's nice if Copyright is provided to the Testhost.dll instead of null (also company).

[mikkelbu]

I looked a bit more into this and the commandlineparser code tries to find the attributes via ReflectionHelper.GetExecutingOrEntryAssembly (Source code). This in turn calls Assembly.GetEntryAssembly.

On netcoreapp2.0 Assembly.GetEntryAssembly() returns the assembly in testhost.dll, but on e.g. net461 it returns null. So on netcoreapp2.0 it will try to find the attributes in the testhost.dll, but on net461 it will end up calling Assembly.GetExecutingAssembly() which will return the dll that contains the tests (at least in my tests). On net461 Assembly.GetExecutingAssembly() also returns the dll that contains the tests. Why commandlineparser is calling Assembly.GetEntryAssembly() first intead of Assembly.GetExecutingAssembly() I don't know.

There are a lot of discussion about this on the net

• Assembly.GetEntryAssembly() returns null during integration tests when targeting net462 microsoft/vstest#649
• https://developercommunity.visualstudio.com/content/problem/315285/assemblygetentryassembly-returns-null-in-unit-test.html
• ...

[moh-hassan]

Why commandlineparser is calling Assembly.GetEntryAssembly() first intead of Assembly.GetExecutingAssembly()

AFAIK, commandlineparser use GetEntryAssembly because it is used for parsing exe application arguments so it need to handle GetEntryAssembly.

commandlineparser in Netstandard1.5 (not yet netstandard2.0) is calling Assembly.GetEntryAssembly(), but in net4x is calling Assembly.GetExecutingAssembly()

In the other side w.r.t testhost:
Testhost On netcoreapp2.0 Assembly.GetEntryAssembly() returns the assembly in testhost.dll, but on net4x it returns null.

I read the links you provided, but the workaround to SetAssembly was depending on AppDomainManager which isn't available in netcore.

Is there a technical reason for testhost to have different EntryAssembly in net4x/netcore?
What is the best workaround to resolve this problem.?
It's nice if NUnit can provide (as a new feature) an API to set the EntryAssembly for the Unit test purpose or injecting the AssemblyAttributes of the system under test.

[mikkelbu]

Is there a technical reason for testhost to have different EntryAssembly in net4x/netcore?

I don't know. Perhaps this is a question for MS (aka https://github.com/Microsoft/vstest/)

What is the best workaround to resolve this problem.?

Again, I don't know.

It's nice if NUnit can provide (as a new feature) an API to set the EntryAssembly for the Unit test purpose or injecting the AssemblyAttributes of the system under test.

Personally, I feel that this is too specific to be part of NUnit, but rather it should be implemented as an extension or just in the tests that require this. Also if it is not possible to use AppDomainManager then perhaps it cannot be done in NUnit. But I'm ready to be persuaded by someone who knows more about this 😄.

[moh-hassan]

I post that issue to vstest project

After a discussion with vstest team and the good support of @singhsarab, they considered the error exception is a bug and published a new package Microsoft.NET.Test.Sdk version 16.0.0-preview-20181205-02
Using this new test.sdk package the exception is removed

[ChrisMaddock]

Great, thanks for the feedback @moh-hassan, I'll close this issue now!