Large Images Load Very Slowly from Examples or Outputs

[JohannesAck]

Describe the bug

Hi, I'm trying to build a demo for a high-resolution image editing approach using gradio currently.
Overall it works well, however loading large images from the examples or outputs is very slow for some reason.

For example, loading a 4896 x 3264 takes 30 seconds when loading it from gr.Examples and almost a minute from the output of a function.
When using the "upload" function of gr.Image instead, it is almost instantaneous.

Is there an existing issue for this?

• I have searched the existing issues

Reproduction

I wrote a small colab that reproduces the issue but for downloads and examples

https://gist.github.com/JohannesAck/802af4af6a89d6a273b6732cb3ce7629

Screenshot

Logs

No logs because no error produced

System Info

Gradio 3.9.1
Using Colab free tier

Severity

serious, but I can work around it

[abidlabs]

Thanks @JohannesAck will take a look next week!

[himmetozcan]

Same problem. It is very slow when the image resolution is high.

[abidlabs]

Hi @JohannesAck the gist you provided no longer is a valid URL. Would you mind providing another example we can use to reproduce please?

[JohannesAck]

Hi @abidlabs ,
Thanks for taking a look at this again!
Here's a new link: https://gist.github.com/JohannesAck/e4f3c975772d32c86c957e5e42daebc2

Here's also the code (after installing gradio in a previous cell), in case my gist gets lost again:

import requests
from io import BytesIO
from PIL import Image
import gradio as gr 

def get_img_from_url(url):
    print(url)
    response = requests.get(url)
    pil_img = Image.open(BytesIO(response.content)).convert("RGB") 
    print('finished download')
    return pil_img

blocks =  gr.Blocks()
with blocks as demo:
    image = gr.Image(source="upload", type="pil")
    url_box = gr.Textbox (label='url', value='https://images.unsplash.com/photo-1589085947445-a491beee038d?ixlib=rb-4.0.3&dl=nat-weerawong-0cZgvYHirBg-unsplash.jpg') 
    btn = gr.Button("Run")
    btn.click(fn=get_img_from_url, inputs=[url_box], outputs=image)
    
    demo.launch(debug=True)

As you will see, the function finishes execution after after one or two seconds, but it takes another 30-40 seconds for the image to display.

[abidlabs]

Thanks @JohannesAck for providing the code example. I can confirm that I can reproduce the problem (though the total inference time on my machine is about 20 seconds). I looked into this and the reason that this is happening is because we convert the PIL image to a byte format here in order to serialize it into base64 format so that it can be read by a browser, and this process is very slow.

There are faster ways to convert PIL images to byte formats, but they rely on 3rd party libraries, such as opencv or pyvips. We are unlikely to add these either of these libraries as dependencies in gradio -- opencv is too heavy and pyvips is not vetted enough.

However, if you would like, you could monkey patch your local installation of gradio to use either of these libraries to speed up the processing time. You could, for example, define a new method:

import numpy as np
import cv2
from cv2 import imencode
import base64

def encode_pil_to_base64_new(pil_image):
    print("using new encoding method")
    image_arr = np.asarray(pil_image)[:,:,::-1]
    _, byte_data = imencode('.png', image_arr)        
    base64_data = base64.b64encode(byte_data)
    base64_string_opencv = base64_data.decode("utf-8")
    return "data:image/png;base64," + base64_string_opencv

And then redefine:

import gradio as gr

gr.processing_utils.encode_pil_to_base64 = encode_pil_to_base64_new

If you then launch your gradio interface, it should use the faster processing method. On my machine, this brought down the total processing time from ~19 seconds to ~5 seconds.

Finally, I'll mention that we will consider removing serialization for images altogether when we release Gradio 4.0 (just created an issue to track this: #3158), which will hopefully make this point moot. I'll go ahead and close this issue since there isn't anything else we can do here for now.

[JohannesAck]

Thanks for looking into this and providing the monkey patch.
I'm looking forward to Gradio 4.0!

[gitgithan]

How does my Gallery of 50 images load within a few seconds, but display 3 images in 3 Image components already take up to 20 secs? I thought it scales at least in the same direction if not linearly.

I see not much difference using the opencv version above (checked prints are using opencv converter).
If I already have the images in the server, can I directly refer to them using local url instead of base64 to speed things up? How can that be done since gradio always does the base64 encoding?

If base64 encoding is the bottleneck, why is running on localhost much faster than the url from demo.launch(share=True) since I devtools inspect my Image output to see locally is also base64 src

Is using image cdn for their compression capability going to help? I'm doubting it since many of my images are already low resolution at 300x250px

In Killing tunnel 127.0.0.1:7860 <> https://f3c2e05c759639e326.gradio.live what tunnel is this refering to? Which is the tool used to create this (i've used ngrok)

[abidlabs]

How does my Gallery of 50 images load within a few seconds, but display 3 images in 3 Image components already take up to 20 secs? I thought it scales at least in the same direction if not linearly.

The Gallery returns paths to files on the server whereas the Image component returns serialized versions of the images, which likely explains the discrepancy

If base64 encoding is the bottleneck, why is running on localhost much faster than the url from demo.launch(share=True) since I devtools inspect my Image output to see locally is also base64 src

Hmm probably the base64 encoders take much longer to transmit through your network than when you run things locally. Agreed that having a way to transmit Images as files instead of base64 would be the solution, which we'll tackle as part of #3158

Is using image cdn for their compression capability going to help? I'm doubting it since many of my images are already low resolution at 300x250px

Not sure in your specific case, but in general, the smaller the sizes of the images, the smaller the base64 encoding, which means they should load faster.

In Killing tunnel 127.0.0.1:7860 <> https://f3c2e05c759639e326.gradio.live what tunnel is this refering to? Which is the tool used to create this (i've used ngrok)

Fast reverse proxy, https://github.com/fatedier/frp

[matthew-walters]

Is it possible to use some sort of 'double-buffering' to have two images and hide the visibility of the one that's currently getting redrawn? I didn't see a function that would tell me when the pic is ready though

[betterze]

@abidlabs Is there a way to use JPEG encoding instead of base64? Jpeg is smaller than base64. Where I should place this function?

def encode_image(image: Image.Image) -> bytes:
    """Encodes a PIL image in JPEG format."""
    buffer = io.BytesIO()
    image.save(buffer, format="JPEG")
    return buffer.getvalue()

[abidlabs]

Hi @betterze if you upgrade to Gradio 4.x, we don't use base64 anymore. Images are uploaded, which generally should be much faster.