Will nspec support netcoreapp1.1?

[johnnymo87]

I've added the following lines to my .csproj file:

    <PackageReference Include="dotnet-test-nspec" Version="*" />
    <PackageReference Include="nspec" Version="*" />

When I run dotnet restore && dotnet test, it builds, but it cannot detect any tests. This blog post mentions that dotnet test works differently on netcoreapp1.1. Will you all add support for netcoreapp1.1?

[BrainCrumbz]

As you experienced yourself, test runner is not (yet) ready for the new tooling that comes with MSBuild/csproj and abandons project.json/xproj.

Yes, we definitely want to support the new tooling. As hinted by Nate McMaster post there are some changes going on under the covers.

Will keep this issue open until porting is done. Please stay tuned!

[martin-alexander-msc]

I am currently considering adding support for .NET Core 1.1+ either by upgrading this NSpec adapter or writing a new one, and wanted to ask if there has already been some work done in this direction

[BrainCrumbz]

Hi @vladimir-aleksandrov-cko , thanks for your interest (and sorry for delayed reply).

TL;DR: no, there has not been work in this direction.

Longer version:

we did some work to upgrade another "runner" of us, NSpec Visual Studio Adapter, which actually is a VS Test Adapter available both as an extension and as a Nuget Package. The aim was to make that fully compatible with VS 2017 Test Framework, both for .NET Framework and .NET Core SDK-style projects. We hit a roadblock there due to how .NET Core currently loads assemblies differently than .NET Framework (see https://github.com/dotnet/coreclr/issues/13277). As per that thread, the issue could be avoided either with future .NET Core versions, or working your way out through a dedicated assembly load context. Or completely changing the way you communicate between runner and library.

Just to be sure we're on track: this present project instead is about the "runner" working specifically for .NET Core xproj-style projects, either from command line or from VS 2015. In the meantime we did not have a chance to upgrade this runner to .NET Core SDK-style projects, hence no VS 2017 either. Actually we haven't got a chance to thoroughly look at differences between xproj-style and SDK-style, as well as between VS 2015 and 2017 test framework.

---

It'd be great if you contributed to this project. If you feel like we can find a way to work on the same branch/PR/..., or give feedback, explain bits and pieces.

Feel free to share your feedback!

[amirrajan]

My unsolicited two cents (which I know @BrainCrumbz loves).

I'd strongly recommend not integrating with dotnet and just build a vanilla console app that leverages NSpec (which is more generally a "delegate orchestration thingy").

runs away so that I don't get yelled at

[martin-alexander-msc]

Thanks a lot for a detailed reply @BrainCrumbz !

@amirrajan thank you for your comment.
My idea was actually to create a test adapter, so that it can also be consumed out of the box either by Visual Studio 2017 or Visual Studio Code (eg using a test extension like this one) as both these IDEs leverage the VS Test Platform (dotnet test). This way nspec tests could be both discoverable and runnable out of the box in VS or VSCode.

[BrainCrumbz]

Yep, rest assured Amir is well aware of all those implications. But it's a lost battle on this topic 😆

That would be great for everyone used to work with VS Test Explorer (and VS Code too!), so yes please, feel free to move the discussion forward on this. Here's the state-of-the-art on current projects:

DotNetTestNSpec adhered to the contract initially set for .NET Core 1.x projects, xproj-style, and thus able to work with those projects both in dotnet command line and in VS 2015 Test Explorer. It is available as a NuGet package as that is the way to be consumed by those environments.

NSpec.VsAdapter adhered to the classic VS test adapter contract and works for .NET Framework 4.5+ projects, plain old csproj format. It is available first as a VSIX extension, being the way test adapters used to be consumed up to VS 2015. It is also available as a NuGet package, as at some point briefly during VS 2015 it was possible to distribute a kind of tooling-package, available for all projects in solution, without having them installed as a VS extension, so each solution could have its own version.

One direction to move forward could be upgrade NSpec.VsAdapter so that it can run under VS 2017 for plain old .NET Framework projects. There's loads of people still working on that framework, so they could make use of that. Not sure how it should be consumed and what has changed in Test Adapter contract.

Another direction is, as you mention at the beginning, to upgrade/evolve either one of the two to work with .NET Core 1.1+ SDK-style projects (and thus under VS 2017). Not sure from which one to pick up as in our team we have not invested a lot of time in figuring out the new test framework. It depends on which one is closer to the current requirements (e.g. does it still work with requests coming from a local TCP port or something, in a certain order? Is it back to expecting an ITestDiscoverer and ITestExecutor implementations?)

-- ok, this was long, sorry about that!

[amirrajan]

@vladimir-aleksandrov-cko please ignore my silly rants :-P @BrainCrumbz and I have debated the merits of both approaches a whole lot. Now I just give him a hard time about it :-)