Port DatasetExtensions to .NET Core

[hugosmarques]

Anyones knows if .AsEnumerable() propertie will be available for datatable on .net core

[danmoseley]

This is a request for DataTableExtensions.AsEnumerable which is in System.Data.DataSetExtensions.dll.

@saurabh500 is this available for Core? I don't see it in our sources or NuGet.

[saurabh500]

@danmosemsft DataSetExtensions was not ported to Core.

[danmoseley]

Is it something worth considering - would it be a fit for CoreFX? We have had success with dumping desktop sources and community folk driving the work.

This is the only ask I have seen for it though.

[hugosmarques]

@saurabh500 did you know if it will be available in the future? I have to know because if this is not available I have to abandon the .net approach.

[saurabh500]

The source for DataSetExtensions is at https://github.com/Microsoft/referencesource/tree/master/System.Data.DataSetExtensions

We haven't planned porting this particular assembly to .Net Core. However I do agree with @danmosemsft idea about having the community bring the code to .Net Core. I don't see any platform specific dependencies in the code so this should be relatively simple.

[danmoseley]

Would we be open @saurabh500 to having it in CoreFX? If so, we can invite a PR to get things started.

[hugosmarques]

Well guys I do not have words for your good vibe in helping me. I will be very thankful if this is implemented, because right now for me is impossible to Port all my code to EF, I have thousands of line code and I use datatable with Linq. If is simple as @saurabh500 says I'm praying that it's implemented.

[danmoseley]

@karelz fyi another port-to-core request.

[karelz]

Moving to Future.

Next step: If there are folks interested, we can dump the code and let community make the code build, add tests, then ship it. Marking as up-for-grabs.

[hugosmarques]

If the integration of this is relatively simple i hope that someone do this ASAP please.

[hugosmarques]

is not supose that if .net core have DataTables properties gave the AsEnumerable propertie too, otherwise how i can use LINQ with datatables

[karelz]

@saurabh500 @NickCraver @FransBouma do you think this is must-have for 2.0? Or can it wait for Future?

[FransBouma]

I use it with meta-data filled DataTable instances (obtained from e.g. GetSchemaTable, stored procedures or ad-hoc sql queries) so I can write Linq queries consuming the meta-data. I use this method as it was available on in .NET full (the code is in .NET full targeting drivers), however I think datatable.Rows.Cast<DataRow>() would do just as well as .AsEnumerable(). EnumerableRowCollection, the class handling the enumeration of AsEnumerable(), has more logic than just a yield return or a cast of Rows, but in AsEnumerable() none of that is used, so it basically comes down to datatable.Rows.Cast() (as far as I can see by reading decompiled code, not tested with running code so I could be mistaken!)

So TL;DR: no, not a must have, as the workaround is .Rows.Cast(). I think the group of people targeting a DataTable with Linq is pretty small (ORM devs, using metadata in datatables with linq), and therefore there's likely not a lot of people hurt by this missing method, but as it's a simple .Rows.Cast(), it might not be a burden to port the method over as that instead of the whole EnumerableRowCollection class.

[hugosmarques]

@FransBouma tell me one thing. I see that i can use:

datatable.Select().Select(x => ... instead of datatable.AsEnumerable().Select(x => x.....

Exist any diference between the two cases?

[karelz]

Thanks for advice @FransBouma!

[FransBouma]

I see that datatable.Select() creates a new array and datatable.AsEnumerable() wraps the rows in the datatable, similar to datatable.Rows.Cast<DataRow>(), so the former copies data, the latter wraps it, which can have a huge impact.

[hugosmarques]

@FransBouma so please if as you said is a very simple feature can we have this for the 2.0 version?

[FransBouma]

@FransBouma <https://github.com/fransbouma> so please if as you said is a very simple feature can we have this for the 2.0 version?

I don't work for Microsoft, so I can't decide over that :)

[karelz]

@hugosmarques unless it is super-critical for many customers, we will not add more APIs into .NET Core 2.0, sorry. See details in https://github.com/dotnet/corefx/issues/17619#issuecomment-301937346

[hugosmarques]

@karelz thanks for all but i cannot understand this decision.... as we all know this is a simple integration and when .net core team decide to put the DataTable on .Net Core i did not understand why does not put the AsEnumerable propertie. People that use DataTable needs this. This is a disapoiting final for me because i cannot move forward with my migration to .Net Core without this.....

[hugosmarques]

@FransBouma if i use dt.Rows.Cast().Where(a => instead of dt.AsEnumerable().Where(a =>
is the same or almost the same right?

I make a test with 1000000 rows on datatable and the time for both are the same. Did you see any incovenient of using dt.Rows.Cast() instead of dt.AsEnumerable()

[FransBouma]

No, I think they're the same, functionality wise, as far as I can see.

[hugosmarques]

@FransBouma this is what I need from beginning... with this I don't need to port AsEnumerable to .net core
dt.Rows.Cast() is available on .net core 2 right?

[FransBouma]

Yes. Just add using System.Linq; at the top of your own code file to make Cast available.

[hugosmarques]

@karelz thanks again. We have a solution for this:)

[danmoseley]

@saurabh500 @corivera @weshaggard any objection to me finding someone to just go ahead and do this now?

Also I assume corefx is a good place for it, given it's small and has no home right now - also closely goes with S.Data.

@terrajobst

[saurabh500]

I don't have any problems. Are you gonna own porting it Dan?

[danmoseley]

If it's simple we can port it. You guys would own it after that (which doesn't sound like much of a burden)..

[joetherod]

The reference source doesnt seem to have the latest extensions code. The other source, I had trouble getting the LingDataView class to get all it needs. I think it was probably because it was 4.7 and my system,data is a different version. Are there links to the previous versions of the framework source code or do I have to download the whole thing?

[danmoseley]

@joetherod which code do you believe is out of date? I just diffed the latest internal .NET Framework sources with https://github.com/Microsoft/referencesource.git and the relevant sources seem to match

[joetherod]

This file, https://github.com/Microsoft/referencesource/tree/master/System.Data.DataSetExtensions
Where are the extension classes?
Is there a chance you port this over in the preview release of System.Data.DataSetExtensions.
And fix the nuget packages?

[savornicesei]

@joetherod You can find the package here along with instructions on setting up the myget feed in your environment.
I have added the System.Data.DataSetExtensions project (from this repo) into one of my projects and it complained about DataSetExtensions class so I removed it. I also replaced the whole thing with the NuGet preview package and it worked fine.

All the best,
Simo

[joetherod]

@savornicesei I was able to install the package, but it still doesnt have support for AsDataView for EnumerableRowCollection

[danmoseley]

@joetherod which specific type or file is missing for you? eg
https://github.com/Microsoft/referencesource/blob/90b323fe52bec428fe4bd5f007e9ead6b265d553/System.Data.DataSetExtensions/System/Data/LinqDataView.cs

Is there a chance you port this over in the preview release of System.Data.DataSetExtensions.

If we merge the assemblies it is possible, but priority wise you are the only person who has asked for these to date as far as I know. Is anyone else blocked on them?

[joetherod]

LinqDataView.cs and the extension methods for AsDataView() in DataTableExtensions.
When bringing in the LinqDataView.cs, the base class isnt in sync with my version of Dataview.cs in core.

[danmoseley]

I linked to LinqDataView.cs above, AsDataView is here:
https://github.com/Microsoft/referencesource/blob/90b323fe52bec428fe4bd5f007e9ead6b265d553/System.Data.DataSetExtensions/System/Data/DataTableExtensions.cs#L258

I checked these sources for DataSetExtensions exactly match the .NET 4.7.1 sources. .NET Core sources are often different at least for formatting.

[joetherod]

The code in core doesnt exist for dataextensions, so not sure what you mean. I copied over all of the classes from 4.6 and 4.7. The LinqDataView.cs has errors with it base class ctor implementation which is DataView.cs.

internal LinqDataView(
DataTable table,
Func<DataRow, bool> predicate_func,
Predicate predicate_system,
Comparison comparison,
Func<object, DataRow, int> comparerKeyRow,
SortExpressionBuilder sortExpressionBuilder)

            //Parent constructor
        : **base(table,
            predicate_system,
            comparison,
            DataViewRowState.CurrentRows)**
    {
        this.sortExpressionBuilder = (sortExpressionBuilder == null) ? this.sortExpressionBuilder : sortExpressionBuilder;    
        this.comparerKeyRow = comparerKeyRow;
    }

[danmoseley]

The code for DataSetExtensions is in .NET Core -- it's at https://github.com/dotnet/corefx/blob/master/src/System.Data.DataSetExtensions/src

If you copy over the code from reference source, including LinqDataView, you will hit the issue I mentioned that it requires access to internal members of the base class, and the base class is in System.Data.Common.dll. In .NET framework this was OK, because the assembly of the base class (named System.DAta.dll there) exposes its internal members to System.Data.DAtasetExtensions.dll.

Does that help? I apologize because I feel like I'm not udnerstanding well.

[joetherod]

@danmosemsft I understand now. So, this cant be done by me. Can someone take care of this in the prerelease package of DataSetExtensions then?
Thanks

[danmoseley]

@joetherod since it requires merging the assemblies it's not trivial work and will probably not happen for 2.1 given the low usage of this API unless it emerges that a bunch of other customers are hitting it also. Hopefully you can find a workaround.

[danmoseley]

Since this issue is closed I opened https://github.com/dotnet/corefx/issues/27610 to track it. Lte's continue any conversation there.

[divega]

Since this issue is closed I opened dotnet/corefx#27610 to track it. Lte's continue any conversation there.

@danmosemsft actually this issue doesn't appear to be closed. Was the intention to close it?

[ToddBradley]

@danmosemsft Low usage? Every single Stack Overflow explanation of how to create a subset DataTable from an existing one relies on AsEnumerable, which doesn't exist in .NET Core. The only workaround seems to be rolling your own subsetting code in a big for loop, which is ugly and messy.

[OHAVM]

Since this issue is closed I opened dotnet/corefx#27610 to track it. Lte's continue any conversation there.

With respect, this is going to be a pain point for me and it appears that the other issue was closed with a "this is too hard and we're lacking resources. Sorry". I will say though, @danmosemsft , if your plan is to not implement this causing .NET Framework developers converting to .NET Core to code around the lack of Data Table extensions, causing the need for DataTableExtensions to drop off meaning that your team won't have to actually address this issue, then I have to respect your logic even if I'm at the unfortunate end of it.

[danmoseley]

@OHAVM the only reason this was not done is convoluted dependency problems. You can see some discussion in dotnet/corefx#27610 . I have driven many ports of NET Framework features and it is pretty rare that we encounter such problems.

We don't have plans right now to have another run at the problem but I reopen this so we can continue to get feedback. As well as leaving a comment I encourage folks interested in this to do a thumbs up of the top comment. That's a datapoint we've used in the past.

[MSiccDev]

We are currently running into issues with the AsDataView() extension method as well. One of our private third-party NuGet packages needs this method internally and is, of course, throwing MethodNotFoundException there.

Microsoft wants us to port our code base over to newer versions of .NET, but puts not only stones, but rocks in our way. In my opinion the low number of complaints is just because others have thrown the towel on porting their code base already earlier, so they never reached here. Just because you do not see a lot of telemetry data regarding to certain code parts, it does not mean they are not essential. There are effective methods to block telemetry data submission, which are often used in enterprise applications.

[bmperdue]

Really need the AsDataView() Added this is causing a lot of issues getting our code to work.

[don4of4]

@danmosemsft Based on a cursory review, it looks like this relatively simple feature is causing a lot of pain.

Could you update us on the status of this? The ticket you referenced is a dead link.

[danmoseley]

Fixed link - it was https://github.com/dotnet/corefx/issues/27610. There are more details in the older PR https://github.com/dotnet/corefx/issues/31764.

[ericstj]

Fixed in dotnet/corefx#36528

[danmoseley]

@don4of4 @hugosmarques @OHAVM @bmperdue and all other folks above looking for this package, I'd be grateful if you'd try it out in our Preview 4 release (not too far away) so that we can be sure it solves your problems (and if not , we can fix it)

Thanks @ericstj for porting.