Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: use rust native webp en/decoding #31

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,4 @@ Cargo.lock

test.png
.idea
.DS_Store
rust-toolchain.toml
.DS_Store
12 changes: 9 additions & 3 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ homepage = "https://github.com/jay3332/ril"
readme = "README.md"
keywords = ["ril", "imaging", "image", "processing", "editing"]
categories = ["encoding", "graphics", "multimedia", "visualization"]
rust-version = "1.67"

[dependencies]
num-traits = "0.2"
Expand All @@ -22,11 +23,15 @@ libwebp-sys2 = { version = "^0.1", features = ["1_2", "mux", "demux"], optional
fontdue = { version = "^0.7", optional = true }
color_quant = { version = "^1.1", optional = true }
colorgrad = { version = "^0.6", optional = true, default_features = false }
# todo: switch back to crates release once
# https://github.com/image-rs/image-webp/commit/4020925b7002bac88cda9f951eb725f6a7fcd3d8
# is released
image-webp = { git = "https://github.com/image-rs/image-webp", optional = true }

[features]
default = ["resize", "text", "quantize", "gradient"]
all-pure = ["resize", "png", "jpeg", "gif", "text", "quantize"]
all = ["all-pure", "webp"]
all-pure = ["resize", "png", "jpeg", "gif", "text", "quantize", "webp-pure"]
all = ["resize", "png", "jpeg", "gif", "text", "quantize", "webp"]
png = ["dep:png"]
jpeg = ["dep:jpeg-decoder", "dep:jpeg-encoder"]
gif = ["dep:gif"]
Expand All @@ -36,10 +41,11 @@ text = ["dep:fontdue"]
quantize = ["dep:color_quant"]
gradient = ["dep:colorgrad"]
static = ["libwebp-sys2?/static"]
webp-pure = ["dep:image-webp"]

[dev-dependencies]
criterion = "^0.4"
image = "^0"
image = "^0.24"
imageproc = "^0.23"
rusttype = "^0.9"

Expand Down
14 changes: 7 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ Additionally, we also plan to support the following pixel formats:
have actual support 16-bit pixel formats in the future.

## Requirements
MSRV (Minimum Supported Rust Version) is v1.61.0.
MSRV (Minimum Supported Rust Version) is v1.67.1.

## Installation
Add the following to your `Cargo.toml` dependencies:
Expand Down Expand Up @@ -128,12 +128,12 @@ image format support, but adds a lot of dependencies you may not need.

For every image encoding that requires a dependency, a corresponding feature can be enabled for it:

| Encoding | Feature | Dependencies | Default? |
|--------------|---------|--------------------------------|----------|
| PNG and APNG | `png` | `png` | no |
| JPEG | `jpeg` | `jpeg-decoder`, `jpeg-encoder` | no |
| GIF | `gif` | `gif` | no |
| WebP | `webp` | `libwebp-sys2` | no |
| Encoding | Feature | Dependencies | Default? |
|--------------|-----------------------|--------------------------------|----------|
| PNG and APNG | `png` | `png` | no |
| JPEG | `jpeg` | `jpeg-decoder`, `jpeg-encoder` | no |
| GIF | `gif` | `gif` | no |
| WebP | `webp` or `webp-pure` | `libwebp-sys2` or `image-webp` | no |

Other features:

Expand Down
4 changes: 4 additions & 0 deletions rust-toolchain.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
[toolchain]
channel = "1.67.1"
components = ["clippy", "rustfmt", "rust-src"]
targets = ["wasm32-unknown-unknown"]
2 changes: 2 additions & 0 deletions src/encodings/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ pub mod jpeg;
pub mod png;
#[cfg(feature = "webp")]
pub mod webp;
#[cfg(feature = "webp-pure")]
pub mod webp_pure;

/// Represents an arbitrary color type. Note that this does not store the bit-depth or the type used
/// to store the value of each channel, although it can specify the number of channels.
Expand Down
221 changes: 221 additions & 0 deletions src/encodings/webp_pure.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,221 @@
use crate::{
encode, ColorType, Decoder, Encoder, Frame, FrameIterator, Image, ImageFormat, OverlayMode,
Pixel,
};
use std::{
io::{Cursor, Read, Write},
marker::PhantomData,
num::NonZeroU32,
result::Result,
time::Duration,
};

#[derive(Copy, Clone, Debug, PartialEq, Default)]
pub struct WebPEncoderOptions {}

pub struct WebPStaticEncoder<P: Pixel, W: Write> {
native_color_type: ColorType,
writer: W,
marker: PhantomData<P>,
}

impl<P: Pixel, W: Write> Encoder<P, W> for WebPStaticEncoder<P, W> {
type Config = WebPEncoderOptions;

fn new(
dest: W,
metadata: impl encode::HasEncoderMetadata<Self::Config, P>,
) -> crate::Result<Self> {
Ok(Self {
native_color_type: metadata.color_type(),
writer: dest,
marker: PhantomData,
})
}

fn add_frame(&mut self, frame: &impl encode::FrameLike<P>) -> crate::Result<()> {
let data = frame
.image()
.data
.iter()
.flat_map(P::as_bytes)
.collect::<Vec<_>>();
let encoder = image_webp::WebPEncoder::new(self.writer.by_ref());

encoder
.encode(
&data,
frame.image().width.into(),
frame.image().height.into(),
match self.native_color_type {
ColorType::L => image_webp::ColorType::L8,
ColorType::LA => image_webp::ColorType::La8,
ColorType::Rgb => image_webp::ColorType::Rgb8,
ColorType::Rgba => image_webp::ColorType::Rgba8,
_ => unreachable!(),
},
)
.map_err(|e| crate::Error::EncodingError(e.to_string()))?;
Ok(())
}

// no-op
fn finish(self) -> crate::Result<()> {
Ok(())
}
}

pub struct WebPDecoder<P: Pixel, R: Read> {
marker: PhantomData<(P, R)>,
}

impl<P: Pixel, R: Read> Default for WebPDecoder<P, R> {
fn default() -> Self {
Self::new()
}
}

impl<P: Pixel, R: Read> WebPDecoder<P, R> {
#[must_use]
pub const fn new() -> Self {
Self {
marker: PhantomData,
}
}
}

impl<P: Pixel, R: Read> Decoder<P, R> for WebPDecoder<P, R> {
type Sequence = WebPSequenceDecoder<P>;

fn decode(&mut self, stream: R) -> crate::Result<Image<P>> {
let mut decoder = image_webp::WebPDecoder::new(Cursor::new(
stream.bytes().collect::<Result<Vec<u8>, _>>()?,
))
.map_err(|e| crate::Error::DecodingError(e.to_string()))?;

let mut image_buf: Vec<u8> = create_image_buffer(&decoder);
decoder
.read_image(&mut image_buf)
.map_err(|e| crate::Error::DecodingError(e.to_string()))?;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Consider implementing a more structured error handling approach

Instead of converting errors to strings, consider defining custom error types that wrap the underlying errors from the image_webp library. This would preserve more information about the errors and make it easier for users of your library to handle them appropriately.

            .map_err(|e| crate::Error::DecodingError(WebpDecodingError::from(e)))?;

// In the error module:
#[derive(Debug)]
pub enum WebpDecodingError {
    ImageError(image_webp::Error),
    // Add other specific error variants as needed
}

impl From<image_webp::Error> for WebpDecodingError {
    fn from(err: image_webp::Error) -> Self {
        WebpDecodingError::ImageError(err)
    }
}


let (width, height) = decoder.dimensions();

let data = image_buf_to_pixeldata(&decoder, image_buf).unwrap();

Ok(Image {
width: NonZeroU32::new(width).unwrap(),
height: NonZeroU32::new(height).unwrap(),
data,
format: ImageFormat::WebP,
overlay: OverlayMode::default(),
palette: None,
})
}

fn decode_sequence(&mut self, stream: R) -> crate::Result<Self::Sequence> {
let decoder = image_webp::WebPDecoder::new(Cursor::new(
stream.bytes().collect::<Result<Vec<u8>, _>>()?,
))
.map_err(|e| crate::Error::DecodingError(e.to_string()))?;

Ok(WebPSequenceDecoder::<P> {
marker: PhantomData,
decoder,
})
}
}

pub struct WebPSequenceDecoder<P: Pixel> {
marker: PhantomData<P>,
decoder: image_webp::WebPDecoder<Cursor<Vec<u8>>>,
}

impl<P: Pixel> FrameIterator<P> for WebPSequenceDecoder<P> {
fn len(&self) -> u32 {
image_webp::WebPDecoder::num_frames(&self.decoder)
}

fn is_empty(&self) -> bool {
self.len() == 0
}

fn loop_count(&self) -> crate::LoopCount {
match image_webp::WebPDecoder::loop_count(&self.decoder) {
image_webp::LoopCount::Forever => crate::LoopCount::Infinite,
image_webp::LoopCount::Times(n) => {
crate::LoopCount::Exactly((Into::<u16>::into(n)) as u32)
}
}
}

fn into_sequence(self) -> crate::Result<crate::ImageSequence<P>>
where
Self: Sized,
{
let loop_count = self.loop_count();
let frames = self.collect::<crate::Result<Vec<_>>>()?;

Ok(crate::ImageSequence::from_frames(frames).with_loop_count(loop_count))
}
}

impl<P: Pixel> Iterator for WebPSequenceDecoder<P> {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (performance): Consider optimizing the Iterator implementation for WebPSequenceDecoder

The current implementation allocates a new buffer and converts pixels for each frame. Consider pre-allocating the buffer and reusing it across frames, and look into ways to optimize the pixel conversion process. This could significantly improve performance for animated WebP images.

impl<P: Pixel> Iterator for WebPSequenceDecoder<P> {
    type Item = crate::Result<crate::Frame<P>>;

    fn next(&mut self) -> Option<Self::Item> {
        // Pre-allocate buffer here
        let mut buffer = Vec::with_capacity(self.width * self.height * 4);
        // Implement frame decoding and pixel conversion logic here
        // Reuse buffer for subsequent frames
    }
}

type Item = crate::Result<crate::Frame<P>>;

fn next(&mut self) -> Option<Self::Item> {
let mut image_buf: Vec<u8> = create_image_buffer(&self.decoder);
let (width, height) = self.decoder.dimensions();

let frame = self.decoder.read_frame(&mut image_buf);

match frame {
Err(image_webp::DecodingError::NoMoreFrames) => return None,
Err(_) | Ok(_) => (),
}

let data = image_buf_to_pixeldata(&self.decoder, image_buf).unwrap();

let frame_duration = self.decoder.loop_duration() / self.decoder.num_frames() as u64;

let frame = Frame::from_image(Image {
width: NonZeroU32::new(width as _).unwrap(),
height: NonZeroU32::new(height as _).unwrap(),
data,
format: ImageFormat::WebP,
overlay: OverlayMode::default(),
palette: None,
})
.with_delay(Duration::from_millis(frame_duration))
.with_disposal(crate::DisposalMethod::Background);
Some(Ok(frame))
}
}

/// Creates a preallocated [Vec<u8>] for the decoder to write to.
fn create_image_buffer(decoder: &image_webp::WebPDecoder<Cursor<Vec<u8>>>) -> Vec<u8> {
let image_buf_len = decoder
.output_buffer_size()
.ok_or(crate::Error::DecodingError(
"Failed to preallocate buffer for image data".to_string(),
))
.unwrap();
vec![0; image_buf_len]
}

/// Converts the imagebuf from [create_image_buffer()] into a [Result<Vec<P: Pixel>>].
fn image_buf_to_pixeldata<P: Pixel>(
decoder: &image_webp::WebPDecoder<Cursor<Vec<u8>>>,
image_buf: Vec<u8>,
) -> crate::Result<Vec<P>> {
let (color_type, pixel_bytes) = if decoder.has_alpha() {
(ColorType::Rgba, 4)
} else {
(ColorType::Rgb, 3)
};

image_buf
.as_slice()
.chunks_exact(pixel_bytes)
.map(|chunk| P::from_raw_parts(color_type, 8, chunk))
.collect::<crate::Result<Vec<_>>>()
}
11 changes: 11 additions & 0 deletions src/format.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,14 @@ use crate::encodings::jpeg;
use crate::encodings::png;
#[cfg(feature = "webp")]
use crate::encodings::webp;
#[cfg(feature = "webp-pure")]
use crate::encodings::webp_pure;
#[cfg(any(feature = "png", feature = "gif", feature = "jpeg", feature = "webp"))]
use crate::{Decoder, Encoder};

#[cfg(all(feature = "webp-pure", feature = "webp", not(doc)))]
compile_error!("features `ril/webp-pure` and `ril/webp` are mutually exclusive");

/// Represents the underlying encoding format of an image.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]
pub enum ImageFormat {
Expand Down Expand Up @@ -167,6 +172,8 @@ impl ImageFormat {
Self::Gif => gif::GifEncoder::encode_static(image, dest),
#[cfg(feature = "webp")]
Self::WebP => webp::WebPStaticEncoder::encode_static(image, dest),
#[cfg(feature = "webp-pure")]
Self::WebP => webp_pure::WebPStaticEncoder::encode_static(image, dest),
_ => panic!(
"No encoder implementation is found for this image format. \
Did you forget to enable the feature?"
Expand Down Expand Up @@ -229,6 +236,8 @@ impl ImageFormat {
Self::Gif => gif::GifDecoder::new().decode(stream),
#[cfg(feature = "webp")]
Self::WebP => webp::WebPDecoder::default().decode(stream),
#[cfg(feature = "webp-pure")]
Self::WebP => webp_pure::WebPDecoder::default().decode(stream),
_ => panic!(
"No encoder implementation is found for this image format. \
Did you forget to enable the feature?"
Expand Down Expand Up @@ -261,6 +270,8 @@ impl ImageFormat {
Self::Gif => Box::new(gif::GifDecoder::new().decode_sequence(stream)?),
#[cfg(feature = "webp")]
Self::WebP => Box::new(webp::WebPDecoder::default().decode_sequence(stream)?),
#[cfg(feature = "webp-pure")]
Self::WebP => Box::new(webp_pure::WebPDecoder::default().decode_sequence(stream)?),
_ => panic!(
"No encoder implementation is found for this image format. \
Did you forget to enable the feature?"
Expand Down
Binary file added tests/animated_lossless.webp
Binary file not shown.
Binary file added tests/animated_lossy.webp
Binary file not shown.
Binary file removed tests/animated_sample.webp
Binary file not shown.
16 changes: 16 additions & 0 deletions tests/common/mod.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
use ril::prelude::*;

pub const COLORS: [Rgb; 12] = [
Rgb::new(255, 0, 0),
Rgb::new(255, 128, 0),
Rgb::new(255, 255, 0),
Rgb::new(128, 255, 0),
Rgb::new(0, 255, 0),
Rgb::new(0, 255, 128),
Rgb::new(0, 255, 255),
Rgb::new(0, 128, 255),
Rgb::new(0, 0, 255),
Rgb::new(128, 0, 255),
Rgb::new(255, 0, 255),
Rgb::new(255, 0, 128),
];
Binary file added tests/out/webp_pure_encode_output.webp
Binary file not shown.
Binary file added tests/reference/random_lossless-1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added tests/reference/random_lossless-2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added tests/reference/random_lossless-3.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added tests/reference/random_lossy-1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added tests/reference/random_lossy-2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added tests/reference/random_lossy-3.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 2 additions & 2 deletions tests/test_gif.rs
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
mod test_png;
mod common;

use common::COLORS;
use ril::prelude::*;
use std::time::Duration;
use test_png::COLORS;

#[test]
fn test_gif_encode() -> ril::Result<()> {
Expand Down
Loading