Host local storage support

[torchiaf]

Summary

PR Checklist

• Is this a multi-tenancy feature/bug?
  • Yes, the relevant RBAC changes are at:
• Do we need to backport changes to the old Rancher UI, such as RKE1?
  • Yes, the relevant PR is at:
• Are backend engineers aware of UI changes?
  • Yes, the backend owner is:

Related Issue harvester/harvester#6057
Related Issue harvester/harvester#5953

Occurred changes and/or fixed issues

Technical notes summary

Areas or cases that should be tested

Areas which could experience regressions

Screenshot/Video

• Add a LVM disk

• Add LVM storage class

[mergify]

This pull request is now in conflict. Could you fix it @torchiaf? 🙏

[tserong]

I've done some initial testing of this, and provisioning and unprovisioning LHv1 and LHv2 disks seems to work, but I've found two things so far that need tweaking:

1. This PR sets a storageclass parameter named engineVersion with value LonghornV1 or LonghornV2, but it should instead be dataEngine with value v1 or v2 (see https://raw.githubusercontent.com/longhorn/longhorn/v1.7.1/examples/v2/storageclass.yaml for an example of a V2 storage class).

2. spec.fileSystem.provisioned is deprecated in favour of spec.provision, but it seems we're still setting spec.fileSystem.provisioned here. If I use this PR to provision a Longhorn V2 disk for example, then check the BD CR, I get this:

> kubectl -n longhorn-system get bds/989754e4e66edadfd3390974a1aba3f8 -o yaml 
[...]
spec:
  devPath: /dev/sdb
  fileSystem:
    mountPoint: ""
    provisioned: true
  nodeName: harvester-node-0
  provision: false
  provisioner:
    longhorn:
      engineVersion: LonghornV2
[...]

...but it should be:

spec:
  devPath: /dev/sdb
  fileSystem:
    mountPoint: ""
  nodeName: harvester-node-0
  provision: true
  provisioner:
    longhorn:
      engineVersion: LonghornV2

[torchiaf]

Thanks @tserong , the code is now updated to handle the v1/v2 engineVersion in Storage Class page and use the new spec.provision field in blockDevice crd in Disks assignement.

[mergify]

This pull request is now in conflict. Could you fix it @torchiaf? 🙏

[tserong]

Thanks @torchiaf, the spec.provision field is working correctly now.

The StorageClass is not quite right - the v1/v2 value is correct, but the parameter name engineVersion needs to be changed to dataEngine (sorry if this wasn't clear earlier, and I know it's probably a pain that this is a different name than what's in the block device CR).

I also have a question: previously there was a "force formatted" checkbox for Longhorn V1:

...but this isn't present anymore, although there's still a tip about it:

Is this correct?

---

I also have a problem provisioning Longhorn V1 disks. They seem to add successfully, but shortly thereafter, they aren't mounted and are unschedulable. I suspect this is actually a problem in node-disk-manager. I'll report back once I've figured out what's going on with this.

[tserong]

...oh also, one more thing: After a disk is provisioned, do we need to show what provisioner is used when viewing the disks on that node? Because there's no indication of that currently - you just see the disk like this:

@Vicente-Cheng, WDTY? Also, I haven't tested what's shown here in the LVM case.

[torchiaf]

@tserong I've fixed the dataEngine parameter, thanks.

For the other points:

• We don't show the 'Force Formatted' checkbox if the disk has already been formatted. This mechanism is unchanged from master:
  • We show up the Force Formatted warning message if any of the new disks is formatted -> blockDevice.spec.fileSystem.forceFormatted. Could you verify this value? I can't replicate it in my environment.

• Not sure what was the root cause of provisioning errors, let me know if any API parameter from the UI is wrong.

[Vicente-Cheng]

...oh also, one more thing: After a disk is provisioned, do we need to show what provisioner is used when viewing the disks on that node? Because there's no indication of that currently - you just see the disk like this:

@Vicente-Cheng, WDTY? Also, I haven't tested what's shown here in the LVM case.

I checked with the latest changes. It looks like @torchiaf already added the provisioner fields.

I thought the current display looked good to me. WDYT? @tserong

We show up the Force Formatted warning message if any of the new disks is formatted -> blockDevice.spec.fileSystem.forceFormatted. Could you verify this value? I can't replicate it in my environment.

I thought we could keep that, but only for the v1 engine.
For v2 engine, we should not let users select any formatted options.

I also have a problem provisioning Longhorn V1 disks. They seem to add successfully, but shortly thereafter, they aren't mounted and are unschedulable. I suspect this is actually a problem in node-disk-manager. I'll report back once I've figured out what's going on with this.

I will check this... It is not related to UI issue. The CRD fields looks good.
Fixed, PR here: harvester/node-disk-manager#139

BTW, I checked the whole LVM flow, and everything looks fine. Thanks!

[tserong]

Thanks @Vicente-Cheng, @torchiaf! Will re-test this morning.

[torchiaf]

@tserong @Vicente-Cheng I 've fixed the accessMode property in Virtual Machine and Volumes create pages, following this rule:

- LonghronV1: RWX
- LonghornV2, Lvm: RWO

[tserong]

The LHv2 changes are all behaving correctly in my testing. Thanks for all your work on this @torchiaf!

[Vicente-Cheng]

Tested with LVM part (with addon enable/vg creation/volume creation)
Thanks!

[a110605]

Are Volume Group Name and Volume Group Type required when creating Storage Class ? cc @Vicente-Cheng
I can successfully create one even NOT choose any volume group name / type

[a110605]

Seems we don't clean the previous disk setting when deleting disk in host edit page. I need to press save first and edit again to modify the disk provisoner. Is that the limitation from backend ? @Vicente-Cheng

remove_disk.webm

Delete unsaved disk and then add disk back gets previous settings. Do we want to fix this behavior ?

[a110605]

When adding new disk, if I didn't choose any volume group and got error. I'm unable to modify anything.

[a110605]

Does Longhorn V2 volume support resize ? cc @tserong @Vicente-Cheng
I got the LHv2 volume stuck in resizing status while LHv1 volume is good on resizing.
If not, we should disable from frontend or block from backend webhook.

[a110605]

Does LHv2 volume support Export to Image ? @tserong @Vicente-Cheng
Seems not, we should hide the expot to image action for LHv2 volume.

[a110605]

Maybe we should disable LHv2 volume to Take a snapshot. cc @tserong , @Vicente-Cheng
There is no restore action for LHv2 volume snapshot to do restore.

[Vicente-Cheng]

Hi @a110605,
Thanks for these questions, here are some update:

Are Volume Group Name and Volume Group Type required when creating Storage Class ? cc @Vicente-Cheng
I can successfully create one even NOT choose any volume group name / type

We already have webhook on v0.1.3. I thought you were using the v0.1.2 on the lab environment.
The latest behavior would block the empty vgName/type/nodeName by webhook like below:

Delete unsaved disk and then add disk back gets previous settings. Do we want to fix this behavior ?

As I mentioned before, this behavior is related to UI behavior.
Delete the disk w/o saving would not change any CR, so adding this disk back will get the previous settings.
So, If we would like to delete the disk w/o saving and adding back with an empty provisioner, we need to handle it on the UI side.

When adding new disk, if I didn't choose any volume group and got error. I'm unable to modify anything.

We can remove and re-add it to handle this user error, but I will also create a webhook to check it.

Does Longhorn V2 volume support resize ? cc @tserong @Vicente-Cheng
I got the LHv2 volume stuck in resizing status while LHv1 volume is good on resizing.
If not, we should disable from frontend or block from backend webhook.

I think v2 volume does not support resize. (REF: longhorn/longhorn#8022)
So, as you mentioned, we should block this operation. cc @tserong

Does LHv2 volume support Export to Image ? @tserong @Vicente-Cheng
Seems not, we should hide the expot to image action for LHv2 volume.

+1, cc @tserong

Maybe we should disable LHv2 volume to Take a snapshot. cc @tserong , @Vicente-Cheng
There is no restore action for LHv2 volume snapshot to do restore.

Looks reasonable to me if we cannot restore the v2 volume. cc @tserong

[tserong]

Agree with @Vicente-Cheng to disable expand and export to image. As for snapshots, I have created a LHv2 volume snapshot and (seem to have) successfully restored it to a new volume, so IMO we should leave this enabled.

My one question is: can we do this extra little bit of work separately later, or should we do it as part of this PR, bearing in mind we really want to get this PR merged as soon as possible for RC1? (My preference is to merge as-is if there's no other issues)

[Vicente-Cheng]

Agree with @Vicente-Cheng to disable expand and export to image. As for snapshots, I have created a LHv2 volume snapshot and (seem to have) successfully restored it to a new volume, so IMO we should leave this enabled.

My one question is: can we do this extra little bit of work separately later, or should we do it as part of this PR, bearing in mind we really want to get this PR merged as soon as possible for RC1? (My preference is to merge as-is if there's no other issues)

Yeah, I agree!
I thought the most work looked great. We could fix others after the rc1.
We could get more feedback if we can ship this UI change with RC1.
WDYT? cc @a110605, @torchiaf, @bk201

[torchiaf]

Let's open a new Issue/PR to change the remaining functionalities.

[torchiaf]

@mergify backport release-harvester-v1.4

[mergify]

backport release-harvester-v1.4

✅ Backports have been created

• #1183 Host local storage support (backport #1076) has been created for branch release-harvester-v1.4