Provision a guest with multiple disks

[guazhang]

we need multiple disks server for some testing, like raid.
but looks tmt don't support it, so could we add this function in provision ?

maybe like --hardware "disk.size >= 500GB" --hardware "disk.count >= 4"

[happz]

Possible workaround:

disk:
  - size: ">= 0"
  - size: ">= 0"
  - size: ">= 0"
  - size: ">= 0"

This would turn into requesting 4 disks of any size.

[happz]

As proposed in #2721, we could have keywords in collections:

disk:
  count: ">= 4"

[skycastlelily]

We have many testcases,which need more than two disks,the disk.count param would benefit us a lot:)

[happz]

The suggested feature, with its specification proposed in #2789, and with changes proposed by me in #2790 (comment), did not seem to attract any real support from the rest of tmt community. Quite frankly, it was a bust.

@guazhang @skycastlelily would you mind commenting on how frequently you and your test face this kind of request in your tests and related work? Plus, more real-live use cases and demonstration of the added value, compared to the "copy & paste" workaround from #2536 (comment), are needed to get the necessary traction.

Cc @psss @lukaszachy @thrix

A note to the "copy and paste" workaround: it's equivalent to the == operator, but cannot handle >= or <= operators; it's not possible to express "at least N disks" by listing size: >= 0 entries under disk. That may limit the usability of such a solution - it's always harder to find the exact match, and when "at least" is acceptable to the test, this would artificially constrain the set of suitable machines.

[lukaszachy]

A note to the "copy and paste" workaround: it's equivalent to the == operator, but cannot handle >= or <= operators; it's not possible to express "at least N disks" by listing size: >= 0 entries under disk. That may limit the usability of such a solution - it's always harder to find the exact match, and when "at least" is acceptable to the test, this would artificially constrain the set of suitable machines.

Thanks for this note @happz. I didn't realize that use case and with it in mind I have to scratch all my complains .

[psss]

Sorry, when discussing the proposal yesterday I did not know about this issue so I completely missed the motivation mentioned here.

We have many test cases, which need more than two disks, the disk.count param would benefit us a lot:)

@skycastlelily, @martinhoyer, what's the common number of disks you usually request in your test cases? I would just like to better understand what is a common scenario here.

A note to the "copy and paste" workaround: it's equivalent to the == operator, but cannot handle >= or <= operators; it's not possible to express "at least N disks" by listing size: >= 0 entries under disk.

Shouldn't the list of two disks be mapped to any guest which has at least two disks? I think it would be much more reasonable default. For example, when I just need a larger disk I would state:

disk:
  - size: >= 4 GB

In this case I'd say it does not make sense to limit search for single-disk guests only. Does it?

[happz]

A note to the "copy and paste" workaround: it's equivalent to the == operator, but cannot handle >= or <= operators; it's not possible to express "at least N disks" by listing size: >= 0 entries under disk.

Shouldn't the list of two disks be mapped to any guest which has at least two disks? I think it would be much more reasonable default. For example, when I just need a larger disk I would state:

disk:
  - size: >= 4 GB

In this case I'd say it does not make sense to limit search for single-disk guests only. Does it?

That's hard to tell, given how disk works now. Suggests that the index might matter - we should fix that and state it doesn't, which would be much easier once the topology records what has been provisioned - and it's also a list. Yes, some might expect that the listing is "at least this and the rest doesn't matter", some might expect "exactly this".

I suppose we should not leave space for such guessing, and define what exactly disk means. Is it "at least N disks"? Ok, we state that. Can I request "at least N disks, with 1 of them at least this big"? yes, I can by repeating - size: >= 0 enough times. Or, is it "exactly this"? Ok, we state that instead. Can I request "more than N disks"? Yes, I can, by repeating - size: >= 0. Can I request "less than N disks"? Yes, by the same repetition, till the list has N-1 entries. Doable? Yes. User friendly? Nope.

That's why I like the idea of adding count, we could turn disk into "at least this, the rest does not matter" key explicitly while being able to control the number of disks, or, with possibly all keyword, apply requirements to all disks without repetition. It might look like a syntax sugar to some extent, removing the need for pretty pointless size: > 0 entries, but it would also remove some unambiguity from the specification, and the "number of disks" would not be driven by enumerating them.

[martinhoyer]

@skycastlelily, @martinhoyer, what's the common number of disks you usually request in your test cases? I would just like to better understand what is a common scenario here.

Tough question. I think disk count is only valid if you need a specific type of physical storage, so that exclude all tests cases that need just n of any block devices, as there are ways the test can create block devices from existing storage space.
Therefore I imagine the use-cases would be among the lines of: "4 SATA drives", "2 rotational disks; 1 SSD", "2 tape devices", or something more complex like "5 self-encrypting nvme drives connected to specific raid/hba/sas controller".

@fsuba might have better insight on what is a usual requirement for Stratis/VDO/LVM testing.

From my (distant) point of view, if a tester has a complex hardware requirements, they probably know specific hosts that fits them, so it should just be a matter of specifying a list of hostnames to choose from. So I wouldn't go crazy with hard-to-implement filters.
I was also thinking if there is a way to tag specific hosts with custom descriptor at least in some of the provisioning methods, so users could use something like hardware: tag: 'something', but that's probably a bad idea.

[guazhang]

Hi,
We don't have to think about this complicated scenario.
for example, how many more than 500G disks in the server? we don't need consider it's SATA/SAS/SSD .

maybe we can add another parameter to filter disk model.

[happz]

maybe we can add another parameter to filter disk model.

disk already defines model-name and driver keys, both are unfortunately not implemented in TF yet. But this kind of filtering is going to be supported.

[martinhoyer]

we don't need consider it's SATA/SAS/SSD

We do, but this can be already done with driver I think.

[skycastlelily]

Hi, sorry for the late reply, I was on sick leave.

Plus, more real-live use cases and demonstration of the added value, compared to the "copy & paste" workaround from > #2536 (comment), are needed to get the necessary traction.

what's the common number of disks you usually request in your test cases?

Three. Without count, we need specify - size > 8G three times.

And we do have some testcases ( besides the three-disks ones)needing more than two disks(disk.count >= 2) , so, disk.count would really benefit us:)