Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FSI integration is broken #465

Closed
baronfel opened this issue Oct 3, 2019 · 16 comments
Closed

FSI integration is broken #465

baronfel opened this issue Oct 3, 2019 · 16 comments

Comments

@baronfel
Copy link
Contributor

baronfel commented Oct 3, 2019

Right now FSI fails in Ionide because we're computing the .Net Framework reference set for parsing script files instead of the proper reference set for .Net Core.

We should fix this, and the route we've chosen for now is to investigate the use of the BackgroundCompiler.GetProjectOptionsFromScript method in FCS, specifically using the useSdkRefs and assumeDotNetFramework parameters.

This works, mostly. Results are summarized in the following table:

TFM assumeDotNetFramework useSdkRefs Ref Set Ok?
net462 true false Yes
net462 false true No (no netstandard.dll
netcoreapp2.1 true false No (netfx references are no bueno)
netcoreapp2.1 false true Sorta (missing mscorlib reference then ok)
netcoreapp3.0 true false No (same as netcoreapp2.1)
netcoreapp3.0 false true Yes (all refs from pack directory, which includes mscorlib)

On .net core, at least 2.1. The code paths all converge onto this function. This function seems to work for netfx and netcoreapp > 3.0 TFMs, because either desktop refs are ok or all necessary refs are in the 'pack' directories starting with 3.0.

For 2.x, once an mscorlib is added to the project options, then typechecking works. The problem seems to be that some deps are not available that the user might expect, ie System.Diagnostics.Process. I did some work to find the set of assemblies found in dotnet fsi:

dotnet fsi reference set
 [("/usr/local/share/dotnet/shared/Microsoft.NETCore.App/3.0.0",
    ["System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e";
     "System.Runtime, Version=4.2.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Runtime.Extensions, Version=4.2.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Console, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Threading, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Text.Encoding.Extensions, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "netstandard, Version=2.1.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51";
     "System.Collections, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "Microsoft.Win32.Primitives, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Diagnostics.Process, Version=4.2.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.ComponentModel.Primitives, Version=4.2.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.IO.FileSystem, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Collections.Concurrent, Version=4.0.14.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Resources.ResourceManager, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Threading.Thread, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Memory, Version=4.2.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51";
     "System.Threading.Tasks, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Threading.ThreadPool, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Linq, Version=4.2.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Buffers, Version=4.0.4.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51";
     "System.Diagnostics.TraceSource, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Runtime.InteropServices, Version=4.2.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Reflection.Emit, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Reflection.Emit.ILGeneration, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Reflection.Primitives, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Reflection, Version=4.2.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Runtime.Numerics, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Diagnostics.Debug, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Linq.Expressions, Version=4.2.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Net.Requests, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Net.WebClient, Version=4.0.1.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51";
     "System.ComponentModel.EventBasedAsync, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Diagnostics.Tracing, Version=4.2.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Text.RegularExpressions, Version=4.2.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Reflection.Metadata, Version=1.4.4.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Collections.Immutable, Version=1.2.4.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Collections.NonGeneric, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Collections.Specialized, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.ComponentModel, Version=4.0.3.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.Diagnostics.StackTrace, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.IO.MemoryMappedFiles, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "System.IO.Compression, Version=4.2.1.0, Culture=neutral, PublicKeyToken=b77a5c561934e089"]);
   ("/usr/local/share/dotnet/sdk/3.0.100/FSharp",
    ["fsi, Version=10.6.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "FSharp.Core, Version=4.7.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "FSharp.Compiler.Private, Version=10.6.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a";
     "FSharp.Compiler.Interactive.Settings, Version=10.6.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"]);

and the ones discovered in netcoreapp2.1 _that aren't in dotnet fsi`:

netcoreapp2.1 - dotnet fsi assemblies
System.ComponentModel.TypeConverter
System.Data.Common
System.Diagnostics.Contracts
System.Diagnostics.DiagnosticSource
System.Diagnostics.FileVersionInfo
System.Diagnostics.TextWriterTraceListener
System.Diagnostics.Tools
System.Drawing.Primitives
System.IO.Compression.Brotli
System.IO.Compression.ZipFile
System.IO.FileSystem.DriveInfo
System.IO.FileSystem.Watcher
System.IO.IsolatedStorage
System.IO.Pipes
System.Linq.Parallel
System.Linq.Queryable
System.Net.Http
System.Net.HttpListener
System.Net.Mail
System.Net.NameResolution
System.Net.NetworkInformation
System.Net.Ping
System.Net.Primitives
System.Net.Security
System.Net.ServicePoint
System.Net.Sockets
System.Net.WebHeaderCollection
System.Net.WebProxy
System.Net.WebSockets
System.Net.WebSockets.Client
System.Numerics.Vectors
System.ObjectModel
System.Private.DataContractSerialization
System.Private.Uri
System.Private.Xml
System.Private.Xml.Linq
System.Reflection.DispatchProxy
System.Reflection.Emit.Lightweight
System.Resources.Writer
System.Runtime.CompilerServices.Unsafe
System.Runtime.CompilerServices.VisualC
System.Runtime.InteropServices.RuntimeInformation
System.Runtime.Serialization.Formatters
System.Runtime.Serialization.Json
System.Runtime.Serialization.Primitives
System.Runtime.Serialization.Xml
System.Security.Claims
System.Security.Cryptography.Algorithms
System.Security.Cryptography.Csp
System.Security.Cryptography.Encoding
System.Security.Cryptography.Primitives
System.Security.Cryptography.X509Certificates
System.Security.Principal
System.Security.Principal.Windows
System.Threading.Overlapped
System.Threading.Tasks.Parallel
System.Threading.Timer
System.Transactions.Local
System.Web.HttpUtility
System.Xml.ReaderWriter
System.Xml.XDocument
System.Xml.XPath
System.Xml.XPath.XDocument
System.Xml.XmlSerializer

And that's where we are right now. We have a working proof of concept, but we'd like to
a) PR adding mscorlib to the default reference set, and
b) figure out which assemblies need to be added to the netcoreapp2.x reference set.

Solving this in a way that includes FSharp.Compiler.Interactive.Settings in the references set all the time would also fix #227.

@baronfel
Copy link
Contributor Author

baronfel commented Oct 3, 2019

@cartermp thoughts on if mscorlib should be part of the reference set? the dll exists in the netcoreapp2.x sdk paths, and in the 3.0 'pack' directories. (Also the fact that nothing works without it in-editor is a compelling argument...)

@Krzysztof-Cieslak
Copy link
Member

OK, so I've thought about it a bit...

  1. Is netcoreapp2.1 even viable scenario? We've resolved that in my proof of concept but that's only because FSAC itself is targeting 2.1. As far as I understand there doesn't exist FSI that targets it - fsi.exe is using Full Framework and dotnet fsi is targeting 3.0. Main problem here is that FCS is using current assembly location to get some of the paths, and FSAC targets 2.1 at the moment - hence, it resolved 2.1 references. I'm not sure if I want to update FSAC to 3.0 so quickly... but on the other hand - if that's the price for getting back good scripting experience I'm willing to do that.

  2. My current proof of concept was using #IF to determine which path we should hit - that's probably wrong.
    I can see a scenario where someone using FSAC .Net Core wants to target fsi.exe and we should be able to provide tooling for such context - this means we need to be able to switch between targeting net462 and 3.0 as part of usual workflow.

@Krzysztof-Cieslak
Copy link
Member

What we want is (I think):

  • For net FSAC (hopefully it will be dead soon anyway) - support just net462 and fsi.exe
  • For netcore FSAC - support both net462 and netcoreapp3.0

@baronfel
Copy link
Contributor Author

baronfel commented Oct 3, 2019

Cool. I think we can do both of those things just by using the FCS members with different args for useSdkRefs and assumeDotNetzframework. That'd be great. Then it just turns into a question of 'what API surface needs to change to allow us to choose an execution mode?'.

@cartermp
Copy link
Contributor

cartermp commented Oct 3, 2019

Generally speaking the uptake on new .NET Core versions is super fast, so requiring .NET Core 3.0 isn't unreasonable, especially since by all accounts it's been a stable release.

@Krzysztof-Cieslak
Copy link
Member

@cartermp Do we have any stats for that that you could potentially share?

@cartermp
Copy link
Contributor

cartermp commented Oct 4, 2019

None that I know as sharable. But given that VS will give you the latest, it's quite high. For example, you get .NET Core 3.0 with VS 2019 16.3.

@Krzysztof-Cieslak
Copy link
Member

Fixed by #466

@HoraceGonzalez
Copy link
Contributor

HoraceGonzalez commented Oct 9, 2019

For some reason, intellisense does not seem to be working in *.fsx files. No tooltips or anything.

I noticed the following error in the output logs:

[Checker] ParseAndCheckFileInProject - /home/tracy/code/scripts/test.fsx completed with errors [unknown (1,1)-(1,1) parameter error Assembly reference 'mscorlib.dll' was not found or is invalid]

Perhaps it's related to this PR?

I'm currently running ubuntu 18.04, dotnet sdk 3.0.100, mono 6.0.0.319, and ionide 4.2.0. I'm also using the netcore version of fsac and "FSharp.useSdkScripts": true in my ionide settings.

Also my dotnet install path is /usr/share/dotnet. Also, /usr/share/dotnet/shared/Microsoft.NETCore.App/3.0.0/mscorlib.dll does in fact exist.

Interestingly, installing FAKE seems to fix the problem. FSAC appears to detect that it's a fake script and handles it differently.

@baronfel
Copy link
Contributor Author

baronfel commented Oct 9, 2019

The initial error you saw is because we assume the default dotnet sdk install location of /usr/local/share/dotnet. We'd need to add a way to discover your dotnet root before this would get fixed for you.

@HoraceGonzalez
Copy link
Contributor

HoraceGonzalez commented Oct 9, 2019

Aww, geez. I totally missed that local in the path. I can reinstall dotnet as a workaround.

Thanks, @baronfel!

@baronfel
Copy link
Contributor Author

baronfel commented Oct 9, 2019

It's something we can do better at for sure. Part of the nature of the SDK is that it can be installed anywhere, so to be 100% correct we'd need a discovery mechanism :-/

@HoraceGonzalez
Copy link
Contributor

You know, while I was working with the previews, I ended up setting an env var called, DOTNET_ROOT in my .bashrc. I think it was necessary for the dotnet-install.sh script.

@baronfel
Copy link
Contributor Author

baronfel commented Oct 9, 2019

Yeah, any discovery mechanism we did would probably look at

  1. A user-set LSP setting (maybe fsharp.dotnetSdkRoot?)
  2. The DOTNET_ROOT env variable
  3. Us invoking dotnet --info from your project root and parsing the output of that 😱

And maybe that's enough to cover most folks use cases. The hard part is doing that in ionide/other clients and passing that info into FSAC

@Krzysztof-Cieslak
Copy link
Member

Yeah, adding setting and checking DOTNET_ROOT sounds like reasonable options and shouldn’t be too difficult to add. Keep in mind both should flow through other parts of Ionide when we use dotnet (spawning FSAC, Ionide, dotnet build integration etc)

@baronfel
Copy link
Contributor Author

baronfel commented Oct 9, 2019

@HoraceGonzalez please check out #475 and leave any additional comments there

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants