Join with empty dataframe causes a panic

[daBlesr]

Checks

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of Polars.

Reproducible example

let right = df!("s" => Vec::<u32>::new()).unwrap();
let left =  df!("s" => vec![1u32, 2u32, 3u32]).unwrap();

let df = left
    .clone()
    .lazy()
    .join(
        right
            .clone()
            .lazy()
            .with_column(lit(true).alias("p")),
        [col("s")],
        [col("s")],
        JoinArgs {
            how: JoinType::Left,
            ..Default::default()
        }
    );
println!("{:?}", df.collect());

Log output

'polars-4' panicked at /polars/crates/polars-core/src/frame/column/scalar.rs:141:9:
assertion failed: !series.is_empty() || length == 0

Issue description

Very recently introduced bug, it works on the commit with tag 0.44.2, but not on latest commit.

Expected behavior

It should be able to do the left join, and show the added column p with null values.

Installed versions

default-features = false, features = ["lazy"]