-
-
Notifications
You must be signed in to change notification settings - Fork 855
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Creating a new Image<TPixel> by Wrapping Memory<byte> #1097
Comments
I edited the title, since public class Image
{
+ public static Image<TPixel> WrapMemory(Memory<byte> memory, int width, int height);
} |
Yes, that's exactly what I would need. But I'm wondering how would you implement it. It's certainly non trivial to cast a So I pressume ImageSharp would need to do some work under the hood. |
## PR Type What kind of change does this PR introduce? <!-- Please uncomment one or more that apply to this PR. --> - Feature ## What is the current behavior? <!-- Please describe the current behavior that you are modifying, or link to a relevant issue. --> Right now there is no (easy) way to cast a `Memory<TFrom>` instance to a `Memory<TTo>` instance. There are APIs to to do that for `Span<T>` instances, but not for `Memory<T>`. The reason for that is that with a `Span<T>` it's just a matter of retrieving the wrapped reference, reinterpreting it and then adjusting the size, then creating a new `Span<T>` instance. But a `Memory<T>` instance is completely different: it wraps an object which could be either a `T[]` array, a `MemoryManager<T>` instance, etc. The result is that currently there are no APIs in the BCL nor in the toolkit to just "cast" a `Memory<T>`. This feature has been requested by a number of developers, including in a well known library such as `ImageSharp`: > Yes, that's exactly what I would need. But I'm wondering how would you implement it. > It's certainly non trivial to cast a `Memory<byte>` to a `Memory<TPixel>` and if there's an API for that I would gladly want to know... > So I pressume `ImageSharp` would need to do some work under the hood. (_`ImageSharp` issue, [here](SixLabors/ImageSharp#1097 (comment)) To solve that, I created a very simplified version of the code included in this PR, into a PR [here](SixLabors/ImageSharp#1314). Having this available right out of the box in the `HighPerformance` package would be helpful in a number of similar situations, especially with `Memory<T>` APIs becoming more and more common across libraries now (as they've been out for a while). ## What is the new behavior? <!-- Describe how was this issue resolved or changed? --> This PR includes 4 new extensions for the `Memory<T>` and `ReadOnlyMemory<T>` types that enable the following: ```csharp // Cast between two Memory<T> instances... Memory<byte> memoryOfBytes = new byte[128].AsMemory(); Memory<float> memoryOfFloats = memoryOfBytes.Cast<byte, float>(); // ...any number of times is needed Memory<int> memoryOfInts = memoryOfFloats.Cast<float, int>(); Memory<byte> backToBytesMemory = memoryOfInts.Cast<int, byte>(); // Or just convert into bytes directly Memory<int> sourceAsInts = new int[128].AsMemory(); Memory<byte> sourceAsBytes = sourceAsInts.AsBytes(); // Want to get a stream from a string? Why not! 😄 using (Stream stream = "Hello world".AsMemory().AsBytes().AsStream()) { // Use the stream here, which reads *directly* from the string data! } ``` Here is the full list of the new APIs introduced in this PR: ```csharp namespace Microsoft.Toolkit.HighPerformance.Extensions { public static class MemoryExtensions { public static Memory<byte> AsBytes<T>(this Memory<T> memory) where T : unmanaged; public static Memory<TTo> Cast<TFrom, TTo>(this Memory<TFrom> memory) where TFrom : unmanaged where TTo : unmanaged; } public static class ReadOnlyMemoryExtensions { public static ReadOnlyMemory<byte> AsBytes<T>(this ReadOnlyMemory<T> memory) where T : unmanaged; public static ReadOnlyMemory<TTo> Cast<TFrom, TTo>(this ReadOnlyMemory<TFrom> memory) where TFrom : unmanaged where TTo : unmanaged; } } ``` ## Notes Marking as draft as this is still being worked on, but feedbacks and reviews are welcome! 😄 ## PR Checklist Please check if your PR fulfills the following requirements: - [X] Tested code with current [supported SDKs](../readme.md#supported) - [ ] ~~Pull Request has been submitted to the documentation repository [instructions](..\contributing.md#docs). Link: <!-- docs PR link -->~~ - [ ] ~~Sample in sample app has been added / updated (for bug fixes / features)~~ - [ ] ~~Icon has been created (if new sample) following the [Thumbnail Style Guide and templates](https://github.com/windows-toolkit/WindowsCommunityToolkit-design-assets)~~ - [X] Tests for the changes have been added (for bug fixes / features) (if applicable) - [X] Header has been added to all new source files (run *build/UpdateHeaders.bat*) - [X] Contains **NO** breaking changes
## PR Type What kind of change does this PR introduce? <!-- Please uncomment one or more that apply to this PR. --> - Feature ## What is the current behavior? <!-- Please describe the current behavior that you are modifying, or link to a relevant issue. --> Right now there is no (easy) way to cast a `Memory<TFrom>` instance to a `Memory<TTo>` instance. There are APIs to to do that for `Span<T>` instances, but not for `Memory<T>`. The reason for that is that with a `Span<T>` it's just a matter of retrieving the wrapped reference, reinterpreting it and then adjusting the size, then creating a new `Span<T>` instance. But a `Memory<T>` instance is completely different: it wraps an object which could be either a `T[]` array, a `MemoryManager<T>` instance, etc. The result is that currently there are no APIs in the BCL nor in the toolkit to just "cast" a `Memory<T>`. This feature has been requested by a number of developers, including in a well known library such as `ImageSharp`: > Yes, that's exactly what I would need. But I'm wondering how would you implement it. > It's certainly non trivial to cast a `Memory<byte>` to a `Memory<TPixel>` and if there's an API for that I would gladly want to know... > So I pressume `ImageSharp` would need to do some work under the hood. (_`ImageSharp` issue, [here](SixLabors/ImageSharp#1097 (comment)) To solve that, I created a very simplified version of the code included in this PR, into a PR [here](SixLabors/ImageSharp#1314). Having this available right out of the box in the `HighPerformance` package would be helpful in a number of similar situations, especially with `Memory<T>` APIs becoming more and more common across libraries now (as they've been out for a while). ## What is the new behavior? <!-- Describe how was this issue resolved or changed? --> This PR includes 4 new extensions for the `Memory<T>` and `ReadOnlyMemory<T>` types that enable the following: ```csharp // Cast between two Memory<T> instances... Memory<byte> memoryOfBytes = new byte[128].AsMemory(); Memory<float> memoryOfFloats = memoryOfBytes.Cast<byte, float>(); // ...any number of times is needed Memory<int> memoryOfInts = memoryOfFloats.Cast<float, int>(); Memory<byte> backToBytesMemory = memoryOfInts.Cast<int, byte>(); // Or just convert into bytes directly Memory<int> sourceAsInts = new int[128].AsMemory(); Memory<byte> sourceAsBytes = sourceAsInts.AsBytes(); // Want to get a stream from a string? Why not! 😄 using (Stream stream = "Hello world".AsMemory().AsBytes().AsStream()) { // Use the stream here, which reads *directly* from the string data! } ``` Here is the full list of the new APIs introduced in this PR: ```csharp namespace Microsoft.Toolkit.HighPerformance.Extensions { public static class MemoryExtensions { public static Memory<byte> AsBytes<T>(this Memory<T> memory) where T : unmanaged; public static Memory<TTo> Cast<TFrom, TTo>(this Memory<TFrom> memory) where TFrom : unmanaged where TTo : unmanaged; } public static class ReadOnlyMemoryExtensions { public static ReadOnlyMemory<byte> AsBytes<T>(this ReadOnlyMemory<T> memory) where T : unmanaged; public static ReadOnlyMemory<TTo> Cast<TFrom, TTo>(this ReadOnlyMemory<TFrom> memory) where TFrom : unmanaged where TTo : unmanaged; } } ``` ## Notes Marking as draft as this is still being worked on, but feedbacks and reviews are welcome! 😄 ## PR Checklist Please check if your PR fulfills the following requirements: - [X] Tested code with current [supported SDKs](../readme.md#supported) - [ ] ~~Pull Request has been submitted to the documentation repository [instructions](..\contributing.md#docs). Link: <!-- docs PR link -->~~ - [ ] ~~Sample in sample app has been added / updated (for bug fixes / features)~~ - [ ] ~~Icon has been created (if new sample) following the [Thumbnail Style Guide and templates](https://github.com/windows-toolkit/WindowsCommunityToolkit-design-assets)~~ - [X] Tests for the changes have been added (for bug fixes / features) (if applicable) - [X] Header has been added to all new source files (run *build/UpdateHeaders.bat*) - [X] Contains **NO** breaking changes
As discussed in gitter:
With the current API, it is possible to create a new image in two ways:
The problem is that in many cases, the source data is provided as a plain byte array. this is specially true when dealing with devices at low level, like cameras and sensors.
In these cases, it would be desirable to do have Image,WrapPixelData(Byte[]) or something like that:
I am aware that with the current ImageSharp architecture this might be difficult, but I believe it's a case scenario that might show quite often, so if you guys can find a solution for this...
The text was updated successfully, but these errors were encountered: