Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add virtual video.canvas for positioning video elements independently of the frame size. #3656

Merged
merged 16 commits into from
Jan 30, 2024

Conversation

toots
Copy link
Member

@toots toots commented Jan 23, 2024

This PR adds a notion of virtual vs. actual video canvas that makes it possible to add video elements independently of the frame size.

Since there is no way of knowing how many pixel videos will have in the future, the general idea is to create a conversion from a higher number of virtual pixels to a smaller number of actual pixels. This is done via video.canvas.make.

We also add a suggested syntactic sugar: x@y = y(x) to make it more readable!

Then, we define standard canvas based on a 10k virtual canvas for 720p, 1080p and 4k resolutions. Those can be used as follows:

let {
  px,
  rem,
  vh,
  vw,
  width,
  height
} = video.canvas.virtual_10k.actual_1080p

video.frame.width := width
video.frame.height := height

background =
  video.add_image(
    x=0.3@vw, y=0.01@vh, width=1562@px, height=1562@px, file=cover_files, background
  )

background =
  video.add_text(
    color=0xFCB900,
    font=myfont,
    speed=0,
    x=234@px,
    y=4437@px,
    size=1.5@rem,
    {nowplaying()},
    background
  )

You can then change the actual resolution by using a different canvas on the top of the script!

@toots toots requested a review from smimram January 23, 2024 21:58
@Moonbase59
Copy link

Moonbase59 commented Jan 24, 2024

@toots: Some quick remarks (unprioritized, just what comes to mind):

  • Would have to check how other tools do this (thinking of ffmpeg), to stay congruent if possible.
  • Will "obscure" canvas sizes be possible, i.e. ultrawide monitors, 4:3, 16:10, etc.?
  • Is fontSize something like the browser rem, i.e. 1 rem = 16 px? The "base" char size?
  • Probably no one will be able to remember things like ten_k, seven_20p, ten_80p, four_k. Suggest using a common prefix and the numbers we all have in our heads, like:
    • video_720p, video_1080p, video_4k, or
    • canvas_720p, canvas_1080p, canvas_4k.
  • Maybe add some more common canvas sizes (and aliases) like
    • canvas_360p (640 x 360, 16:9)
    • canvas_640x480 (?) (640x480, 4:3)
    • canvas_480i, canvas_480p, canvas_ntsc (720x480, 3:2, NTSC TV, DVD)
    • canvas_854x480 (?) (854x480, 16:9, YouTube)
    • canvas_576i, canvas_576p, canvas_pal (720×576, 5:4, PAL TV, DVD)
    • canvas_576i_wide, canvas_576p_wide, canvas_pal_wide (1024x576, 16:9, PAL DVD non-square pixel format converted to square pixels)
    • canvas_800x600 (800x600, 4:3, historic monitor resolution)
    • canvas_720p (1280x720, 16:9)
    • canvas_1024x768 (1024x768, 4:3, historic EGA monitor resolution)
    • canvas_1080p, canvas_fhd (1920x1080, 16:9)
    • canvas_1080p_uw(2560x1080, 21:9/64:27 Ultrawide consumer electronics)
    • canvas_1440p (2560x1440, 16:9, YouTube)
    • canvas_4k, canvas_2160p (3840x2160, 16:9, consumer 4k, Rec.2020 4k)
    • canvas_4k_dci, canvas_2160p_dci (4096x2160, ~1.90:1, DCI 4k, for cinema production)
    • canvas_8k (7680x4320, 16:9, Rec.2020 8k)
    • These are the more common ones (and those used by YouTube).
  • I think we also need a "percent" function, not just px and rem. Think of text/image backgrounds or offsets like "5% off the top left corner". This is easier to grasp than calculating pixels on an idealized 10000x10000 square pixel canvas. It should have horizontal and vertical percentage (as in CSS vw, vh), so things like "make a full-width half-transparent background in the lower-third of the canvas" become easily possible. This would also ease changing the canvas size to one with a different aspect ratio and still keep existing code intact. Think of changing 4:3 → 16:9, or going from FHD to an Ultrawide monitor: Change canvas size and the code can stay intact and you’d still get a full-width "lower third". Or think of rotating from landscape to portrait mode—one canvas change and voilà: "responsive video"!
  • For text display at least, there should be some "overflow" functionality with a "horizonal ellipsis" function. With proportional fonts, you don’t know the rendered length before, which makes it extremely hard to shorten long text, say a "now playing" title text that is awfully long. Would be great to be able to specify "shorten the overflow with an ellipsis" if wider than … (virtual pixels, rem, or percentage).
  • Maybe even definable "text areas" where text flows in and gets wrapped? Recognition of Unicode soft hyphens (German has bloody long words)? Think of Chuck Lorre Vanity Cards…

For the rest, we’ll have to experiment ;-)

@toots
Copy link
Member Author

toots commented Jan 24, 2024

@toots: Some quick remarks (unprioritized, just what comes to mind):

Thanks for these this is really helpful! I agree that following the web practices will make it familiar to most people. However, it's important to remember, in web, the viewport increases with the screen but the pixels are not scaled up and down. Typically, 16px is the size of a 1rem unit (most of the time) regardless of the size of your viewport (which is similar to the idea of actual_size here).

In our case, we are in fact scaling pixels down from a 10k length. Or, more precisely, we are downsampling.

  • Would have to check how other tools do this (thinking of ffmpeg), to stay congruent if possible.

I'm not sure I've seen anything like this. OBS has a notion of virtual vs. actual frame size, which is where I got the idea from originally but it's a visual applicaiton.

  • Will "obscure" canvas sizes be possible, i.e. ultrawide monitors, 4:3, 16:10, etc.?

Definitely.

  • Is fontSize something like the browser rem, i.e. 1 rem = 16 px? The "base" char size?

Yes. The default value I have put is my guess as to what a base size should be from this advanced video example I'm working at cleaning up.

  • Probably no one will be able to remember things like ten_k, seven_20p, ten_80p, four_k. Suggest using a common prefix and the numbers we all have in our heads, like:

    • video_720p, video_1080p, video_4k, or
    • canvas_720p, canvas_1080p, canvas_4k.

I've done:

video.canvas.virtual_10k.actual_720p

I'm happy with it.

  • Maybe add some more common canvas sizes (and aliases) like

I've added the most common one:

# Standard video canvas based off a `10k` virtual canvas.
# @category Source / Video processing
def video.canvas.virtual_10k =
  def make(height, width) =
    video.canvas.make(
      virtual_height=10000,
      actual_size={width=width, height=height},
      font_size=160
    )
  end

  {
    actual_360p=make(640, 360),
    actual_480p=make(640, 480),
    actual_720p=make(1280, 720),
    actual_1080p=make(1920, 1080),
    actual_1440p=make(2560, 1440),
    actual_4k=make(3840, 2160),
    actual_8k=make(7680, 4320)
  }
end
  • canvas_360p (640 x 360, 16:9)

  • canvas_640x480 (?) (640x480, 4:3)

  • canvas_480i, canvas_480p, canvas_ntsc (720x480, 3:2, NTSC TV, DVD)

  • canvas_854x480 (?) (854x480, 16:9, YouTube)

  • canvas_576i, canvas_576p, canvas_pal (720×576, 5:4, PAL TV, DVD)

  • canvas_576i_wide, canvas_576p_wide, canvas_pal_wide (1024x576, 16:9, PAL DVD non-square pixel format converted to square pixels)

  • canvas_800x600 (800x600, 4:3, historic monitor resolution)

  • canvas_720p (1280x720, 16:9)

  • canvas_1024x768 (1024x768, 4:3, historic EGA monitor resolution)

  • canvas_1080p, canvas_fhd (1920x1080, 16:9)

  • canvas_1080p_uw(2560x1080, 21:9/64:27 Ultrawide consumer electronics)

  • canvas_1440p (2560x1440, 16:9, YouTube)

  • canvas_4k, canvas_2160p (3840x2160, 16:9, consumer 4k, Rec.2020 4k)

  • canvas_4k_dci, canvas_2160p_dci (4096x2160, ~1.90:1, DCI 4k, for cinema production)

  • canvas_8k (7680x4320, 16:9, Rec.2020 8k)

  • These are the more common ones (and those used by YouTube).

  • I think we also need a "percent" function, not just px and rem. Think of text/image backgrounds or offsets like "5% off the top left corner". This is easier to grasp than calculating pixels on an idealized 10000x10000 square pixel canvas. It should have horizontal and vertical percentage (as in CSS vw, vh), so things like "make a full-width half-transparent background in the lower-third of the canvas" become easily possible. This would also ease changing the canvas size to one with a different aspect ratio and still keep existing code intact. Think of changing 4:3 → 16:9, or going from FHD to an Ultrawide monitor: Change canvas size and the code can stay intact and you’d still get a full-width "lower third". Or think of rotating from landscape to portrait mode—one canvas change and voilà: "responsive video"!

vh and vw added, thanks for the suggestion!

  • For text display at least, there should be some "overflow" functionality with a "horizonal ellipsis" function. With proportional fonts, you don’t know the rendered length before, which makes it extremely hard to shorten long text, say a "now playing" title text that is awfully long. Would be great to be able to specify "shorten the overflow with an ellipsis" if wider than … (virtual pixels, rem, or percentage).

This is out of the scope of this PR, this tool is for downsampling only.

  • Maybe even definable "text areas" where text flows in and gets wrapped? Recognition of Unicode soft hyphens (German has bloody long words)? Think of Chuck Lorre Vanity Cards…

Same here.

For the rest, we’ll have to experiment ;-)

Yeah!

@toots toots force-pushed the video.canvas branch 5 times, most recently from 4625ac0 to b6b8d19 Compare January 28, 2024 00:51
@toots toots added this pull request to the merge queue Jan 30, 2024
Merged via the queue into main with commit 0e72a00 Jan 30, 2024
30 checks passed
@toots toots deleted the video.canvas branch January 30, 2024 07:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants