Add download_chunk_size property to HTTPRequest.

[Faless]

This allows setting the read_chunk_size of the internal HTTPClient.
This is important to reduce the allocation overhead and number of file
writes when downloading large files, allowing for better download speed.

Fixes #32807 (the user will need to raise the download_chunk_size to a higher value. 64KiB is enough).

extends Control

var done := false
var last_bytes := 0
var time := 0.0 # timer that is restarted ~every second
var http

func _ready():
	http = HTTPRequest.new()
	# Sets a 64KiB chunk size
	http.download_chunk_size = pow(2,16)
	add_child(http)
	http.use_threads = true
	http.download_file = "100MB.bin"
	http.connect("request_completed", self, "_on_file_request_completed")

	# 100MB test file
	http.request("https://speed.hetzner.de/100MB.bin")

func _process(delta):
	if not done and time >= 1:
		var speed_in_bytes = http.get_downloaded_bytes() - last_bytes
		var speed_in_mega_bytes = speed_in_bytes / 1000000.0
		
		print(speed_in_mega_bytes, " MB/s")
		
		last_bytes = http.get_downloaded_bytes()
		time = 0
	time += delta

func _on_file_request_completed(result, response_code, headers, body):
	done = true
	print("File downloaded")

In general, and like in other cases in Godot networking, we are doing some extra buffering, which could be cleaned up in the future.

[Faless]

Note I think we should raise the default value to something higher, 32-64KiB, to better cover the download file use case by default. In any case, we should do that for editor-related HTTPRequest where download speed is quite important.

[akien-mga]

Note I think we should raise the default value to something higher, 32-64KiB, to better cover the download file use case by default.

Sounds good to me. Are there any drawbacks to using a value like 64 KiB by default?

[Faless]

Are there any drawbacks to using a value like 64 KiB by default?

Well, there might be some memory overhead, especially when reading in non-blocking mode from slow servers (or servers that sends little data at a time).

But in many other case (like the referenced issue) it would mean a much lower number of allocations.

In general, HTTPClient and HTTPRequest likely need some refactoring to use pre-allocated buffers instead of allocating memory every time (unless when strictly talking to GDScript). But that's 4.0 material.

[akien-mga]

Thanks!

[akien-mga]

I think a default of 64 KiB would make sense. Or 32 KiB if you think 64 KiB is too high as all-rounder value (knowing that most users will use the default). Can you make a follow-up PR?

[neqs20]

Now it works like a charm, Thanks Faless.
I think download_chunk_size should be by default 32 KiB because downloaded files are often small sizes.
In this case there's no need to occupy additional memory for such a file. If a user wants to download larger files, he just needs to increase the value. But the user has to read docs in a first place...