Custom Shading System & Vune Shading Language (v0.1)

[TheNachoBIT]

WARNING: At the moment, this PR contains functionality from (#801), but it'll be replaced with (#803) once that PR is approved.

This PR is a month worth of work, so this probably will be a long read lol.

Custom Shading System

This new Shading System allows Vello to have the power of OpenGL, Vulkan, Metal, etc. when it comes to shaders, by allowing you to run custom code on the GPU for graphical purposes, similar to how Vertex & Fragment Shaders work.

Since Custom Shading overall in a Tiling Renderer is kind of unknown territory, there's a lot of questions that have to be answered.

Three of the biggest ones are:

• How should the stages & shaders be called?
• What shading language should i use for these?
• How should the custom shaders be designed syntax-wise?

These questions come into mind, because Vello internally is not a Traditional Raster Renderer, but a Tiling Renderer. So, even if you can code a system that replicates the traditional Vertex & Fragment Shader system, that design can end up introducing limitations, or forcing a situation where the system has to translate something that works well in raster, into tiling, causing potential performance drops.

However, that doesn't mean that you cannot make something that is "equivalent to" or "similar to". So, to make things a bit easier to design, i decided to take this approach that answers the first question:

• If paths and transforms are treated in the "Flatten" stage of Vello's pipeline, the custom shaders will be called "Flatten Shaders".
• If colors and gradients are treated in the "Fine" stage, the shaders are going to be called "Fine Shaders".

And so on...

I'm going in this path so i don't have to follow any sort of standard/rules that aren't made by me, that could potentially put me in a situation where i have to ask myself constantly "Should i add 'x' feature or funcionality in the Fragment/Vertex Shader? Because Fragment/Vertex Shaders don't allow it."

So now, let's go to the second question.

The Shading Language.

Here's another big and tricky design decision i had to make, which is deciding what shading language i'll have to use for this system.

At first, i tried with WGSL being the main language, then i tried Slang, and later on a bunch of other tests... But there's one big problem that i had with every shading language that i tried using.

None of them know about the existence of Vello.

This was a big and constant problem, because in order to connect the shaders i was writing to Vello, the shader had to understand that it will be manipulated/executed in some way by Vello's pipeline. And to do that inside of the language of choice, you had three options.

1. Deal with it and be forced to write convoluted code in some places in order to connect to Vello properly.
2. Make a preprocessor that reads the content of the shader file and make modifications.
3. Make a compiler that compiles from (insert shading language here) to Vello-compatible (insert shading language here).

Eventually, as Vello's system grows in power, features and capabilities. Having only a preprocessor is not going to be enough, and eventually you'll have no choice but developing option 3 sooner or later.

But even if you choose one of the options, all of them have the same problem that question number one had, which is having to follow standards/rules that aren't made by me.

And don't forget about CPU fallback... yeah, that's another problem that gets added to the mix...

So, to fix these issues & more, ladies and gentlemen, allow me to introduce:

The Vune Shading Language.

Vune is a version of Rune that aims to compile Vello-compatible shaders at runtime.

Vune uses Rune's parser to create an AST that can be used for Code Generation. At the moment, its only target is WGSL, but the idea is that it could also compile to Rune so it can run shaders on the CPU. This could also help in the future with the problem of having to write the same shader twice.

Vune, because of Rune, has a syntax identical to Rust, so, to put it short, its "Rust-GPU, but it's a Just In Time Compiler for Vello shading".

Once a Vune Shader finishes compiling, the Shading System will inject it inside of the pipeline by replacing the default shaders with custom ones.

Despite the execution being a bit primitive as you're probably going to see in the code, it allows the custom code to have zero performance penalties of any kind.

Now, what custom shaders can you make so far?

The Flatten Shader.

Here's the only shader type that is able to do so far, which is the Flatten Shader. This shader is the one that allows you to handle paths and transforms, similar to the Vertex Shader.

This shader contains two functions that act as entry points.

• flat_main, which stands for "FLAtten Transform", is the function that allows you to handle the transforms.
• flap_main, which stands for "FLAtten Path", is the function that allows you to handle the individual paths (curves, lines, etc.).

So far, the only one that works is flat_main, but flap_main will be implemented soon in this PR.

Here's a tiny example of a Vune Flatten Shader in action:

2025-01-29.20-33-46.mp4

Vune also allows you to send custom uniforms with any type of data that you want (as long as its bytemuck compatible). Here's another example where i'm sending the mouse's position & the window's resolution to the GPU:

2025-01-30.12-53-35.mp4

Flatten Shader Code:

use core::transform;

struct MouseGPU {
	x: f32,
	y: f32,
	res_x: f32,
	res_y: f32,
}

uniform!(6, mouse: MouseGPU);

fn get_distance_between(a: vec2f, b: vec2f, div_by: vec2f) -> f32 {
	return 
		pow(sqrt(b.x - a.x), 2.0) / div_by.x + 
		pow(sqrt(b.y - a.y), 2.0) / div_by.y;
}

fn flat_main(input: Transform) -> Transform {

	let mut final_input = input;

	let mouse_vec = vec2f(mouse.x, mouse.y);
	let res_vec = vec2f(mouse.res_x, mouse.res_y);
	let distance = clamp(get_distance_between(Transform::get_position(final_input), mouse_vec, res_vec), -0.5, 0.5);

	final_input = Transform::then_scale(final_input, distance);

	return final_input;
}

The next video will demonstrate the amount of performance you get with this system: This is a stress test that contains 5000 shapes being scaled up and down individually the closer the mouse gets. Because the GPU takes care of the transformation instead of the CPU, this scene runs at 60+ fps on a GTX 1050 Ti (i still have to test with VSync off, that's why the "+").

2025-01-30.18-05-15.mp4

These are examples that are currently made in the Bella Engine, but i'll make a port of these to pure Vello as "with_winit" scenes.

Adding a Vune Shader.

WARNING: This API design is subject to change.

Adding a Vune Shader is pretty easy, first, you load & compile the shader you want by using the Renderer's add_vune_shader function.

renderer.add_vune_shader("name", "path/to/shader.vune", WgpuVuneBindings::flatten(vec![
	/* Insert custom bindings here..., or send an empty vec if you don't want to send any. */
]));

If you're planning to send custom data, like a struct for example, you have to set its binding:

#[repr(C)]
#[derive(bytemuck::Zeroable, Copy, Clone, bytemuck::Pod)]
struct CustomStruct {
	a: f64,
	b: f64,
}

/* ... */

renderer.add_vune_shader("name", "path/to/shader.vune", WgpuVuneBindings::flatten(vec![
	BindType::Uniform
]));

Once that's done, you can get its reference via get_vune_shader.

let my_shader = renderer.get_vune_shader("name");

And add it to your scene:

scene.flatten_shader.set(my_shader);

If you want to send the CustomStruct that we've made earlier to the Flatten Shader, you can use set_uniform.

scene.flatten_shader.set_uniform::<CustomStruct>(0, CustomStruct { a: 1.0, b: 1.0 });

Aaaaand that's it! You've added a Vune shader to your scene successfully :D

Now you can draw the scene and let the shading system take care of the rest.

If you want different shader types in different scenes, at the moment, you'll have to split the scenes into two draw calls for that to take effect, but in the future, that's probably not going to be necessary.

This is heavily WIP.

There's still A TON to do and add before this shading system can have a "full version" of it, but for this PR, i'll stop at adding the Flatten Shader and see what to do next, because the PR is getting insanely huge to review.

To-Do (before setting it up for review):

• Implement flap_main for the Flatten Shader.
• Ask: Are automatic bindings a good idea?
  EDIT: Yes, in theory, but i still have to test a bit more.
• Implement transform & path buffers.
• Upload the examples to Bella Engine.
• Port the examples to "with_winit".
• Fix custom data indexing.
• Clean up render_full.
• See if i can add CPU fallback.
• Add error handling to Vune.
• Document everything.
• Fix issues found by clippy.

Finally...

I want to thank @raphlinus , @DJMcNab , @waywardmonkeys and @xorgy for helping me by answering a ton of questions and giving support, i appreciate it a lot ❤️

Also, i would like to thank @udoprog for helping a ton with Rune related stuff ❤️

If there's any questions, or design choices that you want to discuss, feel free to do so! c:

Thank you for reading, and i hope y'all are having a good day/night, Cheers! :D

[TheNachoBIT]

More progress!

flap_main's first iteration is completed! Now you can control the positions of the path segments with precise detail! Here's a tiny example of the shader moving the square's shape:

2025-02-04.11-44-49.mp4

Shader code:

use core::transform;
use core::points;

struct ExampleConfig {
	time_elapsed: f32,
}

uniform!(6, example_cfg: ExampleConfig);

fn flat_main(input: Transform) -> Transform {
	return Transform::then_scale(input, 1.5);
}

fn flap_main(input: CubicPoints) -> CubicPoints {
	let mut final_input = input;

	final_input.p0.y = input.p0.y + sin(example_cfg.time_elapsed * 2.0 + input.p0.x) * 0.25;
	final_input.p0.x = input.p0.x + sin(example_cfg.time_elapsed * 2.0 + input.p0.y) * 0.25;

	final_input.p1.y = input.p1.y + sin(example_cfg.time_elapsed * 2.0 + input.p1.x) * 0.5;
	final_input.p1.x = input.p1.x + sin(example_cfg.time_elapsed * 2.0 + input.p1.y) * 0.5;
	
	final_input.p2.y = input.p2.y + sin(example_cfg.time_elapsed * 2.0 + input.p2.x) * 0.5;
	final_input.p2.x = input.p2.x + sin(example_cfg.time_elapsed * 2.0 + input.p2.y) * 0.5;

	final_input.p3.y = input.p3.y + sin(example_cfg.time_elapsed * 2.0 + input.p3.x) * 0.25;
	final_input.p3.x = input.p3.x + sin(example_cfg.time_elapsed * 2.0 + input.p3.y) * 0.25;

	return final_input;
}

[TheNachoBIT]

So, based on the suggestions of office hours, i'm going to close this PR and turn this into a fork.

Here it is if you want to check it out :D https://github.com/bella-project/vello-phoenix