Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add IPMI pools to /api/address/pools endpoint #521

Merged
merged 5 commits into from
Apr 7, 2017
Merged

Add IPMI pools to /api/address/pools endpoint #521

merged 5 commits into from
Apr 7, 2017

Conversation

michaeljs1990
Copy link
Contributor

@michaeljs1990 michaeljs1990 commented Mar 9, 2017

This adds support for programmatically discovering the currently configured pools. If no pools are configured and you have one top level IPMI pool nothing is returned such as with this config.

 ipmi {
   randomUsername = false
   username = "root"
   passwordLength = 16
   network="172.16.32.0/20"
   startAddress="172.16.32.20"
}

I believe this is the right action since you likely don't care about ipmi pools if you have not configured any and makes handling edge cases easier. Such as if you had pools configured as well as having a network set in the top level of the ipmi config. Also any default name we would give the top level pool could potentially be overwritten by a user leading to some strange cases.

Closes #515

todo

@byxorna

IpmiInfo.AddressConfig.map(_.poolNames).getOrElse(Set())
} else {
// TODO: Change this to only return active pools?
IpmiInfo.AddressConfig.map(_.poolNames).getOrElse(Set())
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the one thing I don't like about this currently. How IpAddress pools are handing this is just querying the DB and returning all unique fields. Since this is not in the DB at all for IPMI pools we would have to run a very expensive query to figure out what all the active pools are. The other option is writing a migration to go with an evolution which I believe is the correct path forward.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer reading the configured pools from the config for IPMI tbh. However, we would need to take into account the fact that IPMI addresses can be "loose", and not adherent to any pool in the config.

I am open to either, however an evolution will definitely complicate things. I also think if we track pool with an ipmi_info entry, we will need to update it when people hit /api/asset/:tag/ipmi to create a new address or update existing...

@michaeljs1990 michaeljs1990 changed the title Pool config Add IPMI pools to /api/address/pools endpoint Mar 9, 2017
@byxorna
Copy link
Contributor

byxorna commented Mar 9, 2017

When #513 lands, this will need to be rebased, as I split out the dev_base config from the test config.

I was thinking having a separate API endpoint for IPMI pools would make more sense, because the semantics of how IPMI pools and IP pools are reported/configured/tracked/updated are different. If we are only reporting the IPMI pools as configured in the play config, I would prefer them to be handled by another endpoint with a singular purpose :)

@michaeljs1990
Copy link
Contributor Author

That sounds good to me, i'll split this out into another endpoint. Do you have any preference on the name?

@byxorna
Copy link
Contributor

byxorna commented Mar 9, 2017

Not really sure. Perhaps similar to https://tumblr.github.io/collins/api.html#api-ipam-get-address-pools, GET /api/ipmi/pools? Perhaps there can be some params to that endpoint to include/omit default pool, or include/omit actual IPs grouped by pools, and hopefully some way to indicate non-managed IP pools? (maybe the last thing is a huge timesink; maybe just stub out a param to show unmanaged IPMI network utilization and we can implement it later?)

@@ -85,4 +88,8 @@ trait IpmiApi {
)
}(Permissions.IpmiApi.UpdateIpmi)

// GET /api/ipmi/pools
def getIpmiAddressPools(all: String) =
GetPoolsAction(Truthy(all), Permissions.IpAddressApi.UpdateAddress, this)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this permission seems incorrect. We should use a separate "ipmi.get address pools" perm, and perhaps a separate one for ipmi network view from normal address pool view?

format(pools)
val ipmiPools: Set[String] =
if (all) {
IpmiInfo.AddressConfig.map(_.poolNames).getOrElse(Set())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check that this handles mapping the default pool to the proper name correctly? I.e. if you have a config that uses an implicit network (not using the pools = {...} syntax)

Also, does this stuff need to be reverted now that you have a separate endpoint for ipmi pools?

case ActionDataHolder(all) =>
val pools: Set[String] =
if (all) {
IpmiInfo.AddressConfig.map(_.poolNames).getOrElse(Set())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

both arms of the conditional seem the same. Whats the purpose of the all param then?

}
)

protected def formatNetworkAddress(network: String): String = network match {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would be cool to refactor this to return an Option instead of a string error message, and do the user-view formatting at a higher level with flatMap or something to handle the nones


override def validate(): Validation = {
var pools: Set[String] = {
val p = IpmiInfo.AddressConfig.filter(_.poolNames != Set())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could probably be _.poolNames.isNonEmpty or something

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is much nicer. I thought i tried this originally but I must have had some other error popping up the first time :(

override def validate(): Validation = {
var pools: Set[String] = {
val p = IpmiInfo.AddressConfig.filter(_.poolNames != Set())
if (p.isDefined) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think this could be more idiomatic like

IpmiInfo.AddressConfig.filter(_.poolNames.isNonEmpty) match {
case Set() => Set(IpmiInfo.AddressConfig.flatMap(_.defaultPoolName).getOrElse(""))
case _ => IpmiInfo.AddressConfig.map(_.poolNames).getOrElse(Set(""))
}
``` (syntax may be screwed up, but you get the drift)

Just a suggestion, feel free to disregard

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am a pattern matching noob but this does make things look way nicer.

if (p.isDefined) {
IpmiInfo.AddressConfig.map(_.poolNames).getOrElse(Set(""))
} else {
Set(IpmiInfo.AddressConfig.flatMap(_.defaultPoolName).getOrElse(""))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, if you do not specify any pools to pull back, it only returns the default pool? What is the deal iwht the getOrElse?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah if no pools are defined in the configuration it trys to pull back the DEFAULT pool. The get or else was just because this returns an option and I felt dirty just using get.

args(sequential = true)

"the REST API" should {
"Support getting ipmi pools" in new WithApplication with AssetApiHelper {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should probably test with and without the optional parameters?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes that would be best, I'll see if i can figure out how to switch out or change the configuration file it's reading from so both cases can be tested.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The WithApplication can use something like FakeApplication and provide individual parameters? However, ive not had great experience overriding a config this way (i gave up overriding the IPAM pool configs :()

@byxorna
Copy link
Contributor

byxorna commented Mar 22, 2017

@michaeljs1990 this looks pretty awesome. It needs a) doc updates, and b) updates to the ruby client and collins-shell to make this accessible.

@michaeljs1990
Copy link
Contributor Author

Added collins shell support in the form of...

$ ./bin/collins-shell ipmi pools
Pool       Network         Gateway      Broadcast      Possible Addresses  Start Address  Specified Gateway
OOB-POD01  172.16.32.0/20  172.16.32.1  172.16.47.255                4094  172.16.32.20   Unspecified

@michaeljs1990
Copy link
Contributor Author

michaeljs1990 commented Mar 23, 2017

didn't bump the version since that would cause a conflict with the diff you are about to land...

edit: oops i accidentally did bump the version when testing but will update after #513 lands

This adds support for IPMI pools to be returned from the /api/ipmi/pools
API endpoint. See comments above /ipmi/GetPoolsAction toAddressPools method
for a better understanding of how this works.

Also added a simple unit test based on the dev config.
Copy link
Contributor

@byxorna byxorna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Awesome work! @roymarantz

@michaeljs1990
Copy link
Contributor Author

michaeljs1990 commented Mar 25, 2017

Thanks for suffering through me learning some Scala 😆 The last rebase I did just fixes a lint error where i had a backet off by one space in test/collins/controllers/IpamApiSpec.scala

@byxorna byxorna requested a review from roymarantz March 28, 2017 00:18
@byxorna
Copy link
Contributor

byxorna commented Apr 6, 2017

@michaeljs1990 looks like my changes i landed created merge conflicts :( soz!

@michaeljs1990
Copy link
Contributor Author

Thats ok :) was expecting it. Going to test locally github gave me a cool GUI conflict resolver that I had to try out.

I believe after that is done though someone else will have to land this as even before the conflict the only option I had was to close the PR.

@michaeljs1990
Copy link
Contributor Author

hmmm, something broke with how the pool name is determined. Looking into it right now... yeah for unit tests.

update the test config file to match what the unit test is expecting.
@michaeljs1990
Copy link
Contributor Author

Ok, this should be all set for landing 🔥 🔥

@byxorna
Copy link
Contributor

byxorna commented Apr 7, 2017

@michaeljs1990 dope, thanks! Landing now. cc @roymarantz @defect

@byxorna byxorna merged commit 2683135 into tumblr:master Apr 7, 2017
@michaeljs1990
Copy link
Contributor Author

awww yis! thanks for landing!

@michaeljs1990 michaeljs1990 deleted the pool-config branch April 8, 2017 02:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants