Skip to content
This repository was archived by the owner on Nov 15, 2019. It is now read-only.

Extending event catcher and adding targeted refresh for availabilities #27

Merged
merged 3 commits into from
Jun 27, 2017

Conversation

israel-hdez
Copy link
Member

@israel-hdez israel-hdez commented Jun 14, 2017

  • A targeted refresh is added.
    • This targeted refresh only updates the status field of deployment entities when an availability update is received by the refresh worker.
    • Targeted refresh also updates server entities with the fetched new status.
    • No other field is updated, nor entity creation or deletion is performed. That work is left to the full graph refresh.
  • Event catcher is extended to retrieve availability metrics from Hawkular for all deployments in inventory. Availabilities are "parsed" and compared against the status stored in MiQ database. Deployments whose status has changed are enqueued to be updated in the refresh worker.
  • Event catcher is extended to fetch availability metrics of each inventoried server in MiQ. Also, its server state is fetched from Hawkular's inventory. A targeted refresh is queued for servers that have changed their availability or status.

With these changes, status of deployments and servers will be updated very frequently avoiding any manual refresh.

…ails

* Event catcher is extended to retrieve availability metrics from
  Hawkular for all deployments in inventory. Availabilities are "parsed"
  and compared against the status stored in MiQ database. Deployments
  whose status has changed are enqueued to be updated in the refresh
  worker.
* A targeted refresh is added. This targeted refresh only updates the
  status field of deployment entities when an availability update is
  received by the refresh worker. No other field is updated, nor entity
  creation or deletion is performed. That work is left to the full graph
  refresh.

These changes is work towards covering ManageIQ#5
@miq-bot miq-bot added the wip label Jun 14, 2017
* Event catcher is extended to fetch availability metrics of each
  inventoried server in MiQ. Also its server state is fetched from
  Hawkular's inventory. A targeted refresh is queued for servers that
  have changed their availability or status.
* Targeted refresh "availability updated" is also extended to support
  updating server entities with the fetched new status.
@israel-hdez israel-hdez force-pushed the worker-update-avails branch from 1181377 to 24f7c82 Compare June 16, 2017 19:23
@abonas
Copy link
Member

abonas commented Jun 18, 2017

@miq-bot add_label events

@miq-bot miq-bot added the events label Jun 18, 2017
@abonas abonas self-assigned this Jun 18, 2017
end

def id
"Collection: #{@targets.map(&:manager_ref)}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should the id be such a long string?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't need to be long. But, as name, it's only used when writing logs here. I found no other usage of it. I'm trying to be verbose for logging.

end

def name
"Collection of availabilities to update on inventory entities"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is the name displayed somewhere? if yes, it needs translation

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's only usage is for writing it in logs.

fetch_deployment_availabilities
end

def fetch_server_availabilities
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it looks like there's an opportunity here for code reuse.
because this method and the one below look exactly the same

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, I'll try to think a way to make it reusable.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't think a way to refactor it :(

@israel-hdez
Copy link
Member Author

I think this is ready. Only reviews left. So, I will do:
@miq-bot remove_label wip

@miq-bot miq-bot changed the title [WIP] Extending event catcher and adding targeted refresh for availabilities Extending event catcher and adding targeted refresh for availabilities Jun 21, 2017
@miq-bot miq-bot removed the wip label Jun 21, 2017
@abonas abonas requested a review from jkremser June 22, 2017 18:13
@abonas
Copy link
Member

abonas commented Jun 22, 2017

@Jiri-Kremser please review. thanks

Copy link
Member

@jkremser jkremser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, my only concern is whether it was tested with the domain mode and deployments on server groups. From the code I think it should work, but it's hard to say.


def fetch_server_availabilities(parser)
# For servers, it's also needed to refresh server state from inventory.
$mw_log.debug("#{log_prefix} Retrieving server state from Hawkular inventory")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: perhaps, s/state/states/g would be better

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is now changed :)

end

def server_updates
@target.select { |item| item.association == :middleware_servers }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you test your code for the WF running in domain mode?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I tested domain mode. There is no need to make any distinction.

Components being tested are:
* Persister and parser of targeted refresh. Collector is implicitly
  tested with parser.
* New fuctions of event catcher's stream. Some existing test are
  slightly complemented.

Some existing tests were adjusted due to small refactors for code
reusability and avoid repeating stuff.
@israel-hdez israel-hdez force-pushed the worker-update-avails branch from e5054c2 to 7e68ff0 Compare June 26, 2017 16:50
@miq-bot
Copy link
Member

miq-bot commented Jun 26, 2017

Checked commits israel-hdez/manageiq-providers-hawkular@6cce4f8~...7e68ff0 with ruby 2.2.6, rubocop 0.47.1, and haml-lint 0.20.0
17 files checked, 0 offenses detected
Everything looks fine. 🍰

@jkremser jkremser merged commit 8bb5611 into ManageIQ:master Jun 27, 2017
@abonas
Copy link
Member

abonas commented Jul 11, 2017

@Jiri-Kremser there's no milestone set for this PR

@abonas abonas added this to the Sprint 64 Ending Jul 10, 2017 milestone Jul 11, 2017
@israel-hdez israel-hdez deleted the worker-update-avails branch August 29, 2017 15:16
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants