Import error for Azure Monitor even if all requirements are installed

[pun-ky]

SUMMARY

Upgrading requirements causing a problem with Monitor dependency.

IMO - https://stackoverflow.com/a/70717476/3360007 - this is still true, it looks for me also buggy.

ISSUE TYPE

• Bug Report

ANSIBLE VERSION

2.8.x

COLLECTION VERSION

1.10.0

OS / ENVIRONMENT

RHEL-based / CentOS

EXPECTED RESULTS

Be able to install recent requirements for Ansible Azure Collection.

Upgrade failed and I needed to downgrade:

ACTUAL RESULTS

  azure-arm.arm: MSG:
    azure-arm.arm:
    azure-arm.arm: Failed to import the required Python library (ansible[azure] (azure >= 2.0.0)) on pkrvmbih6j4veli's Python /usr/bin/python3.6. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter
    azure-arm.arm: An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ModuleNotFoundError: No module named 'azure.mgmt.monitor.version'
    azure-arm.arm: failed: [aem_single] (item=cq-quickstart-6.5.0.jar) => {
    azure-arm.arm:     "ansible_loop_var": "item",
    azure-arm.arm:     "changed": false,
    azure-arm.arm:     "item": "cq-quickstart-6.5.0.jar"
    azure-arm.arm: }
    azure-arm.arm:
    azure-arm.arm: MSG:
    azure-arm.arm:
    azure-arm.arm: Failed to import the required Python library (ansible[azure] (azure >= 2.0.0)) on pkrvmbih6j4veli's Python /usr/bin/python3.6. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter
    azure-arm.arm: An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ModuleNotFoundError: No module named 'azure.mgmt.monitor.version'

[fiskhest]

I was able to resolve this by executing pip install azure>=4.0.0 which in turn installs azure-mgmt-monitor==0.5.2 rather than the faulty requirement specified in this repository.

On my machine, pip install 'ansible[azure]' only installs ansible and none of the azure dependencies.

[pun-ky]

thx for sharing @fiskhest ; I will try this out

[Fred-sun]

I was able to resolve this by executing pip install azure>=4.0.0 which in turn installs azure-mgmt-monitor==0.5.2 rather than the faulty requirement specified in this repository.

On my machine, pip install 'ansible[azure]' only installs ansible and none of the azure dependencies.

Does your problem still exist?

[fiskhest]

Well, if I follow the officially documented steps required to install this collection:

To install Azure collection hosted in Galaxy:

ansible-galaxy collection install azure.azcollection

Install dependencies required by the collection (adjust path to collection if necessary):

pip install -r ~/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt

the problem still very much exists, because that file contains azure-mgmt-monitor==3.0.0 while the collection currently seem to only work if used with azure-mgmt-monitor==0.5.2.

I can reproduce this on two separate laptops.

[Fred-sun]

@fiskhest First, ansible-collection requires ansible version higher than 2.9, please update it to the latest; If python3 is used, install in PIP3 mode. Could you try again? Thank you very much!

[fiskhest]

@Fred-sun:

❯ pip --version
pip 21.2.3 from /home/johan/.pyenv/versions/3.10.0/lib/python3.10/site-packages/pip (python 3.10)

❯ ansible-playbook --version
ansible-playbook [core 2.12.5]
  config file = None
  configured module search path = ['/home/johan/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/johan/.pyenv/versions/3.10.0/lib/python3.10/site-packages/ansible
  ansible collection location = /home/johan/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/johan/.pyenv/versions/3.10.0/bin/ansible-playbook
  python version = 3.10.0 (default, Apr 19 2022, 12:07:22) [GCC 11.2.0]
  jinja version = 3.1.2
  libyaml = True

Got any other terribly misguided suggestions dodging my very clear reproduction and fix?

[Fred-sun]

@fiskhest @pun-ky Sorry for the trouble caused to you, the pull request related to the upgrade of 'azure mgmt-monitor' is #701, which contains azCollection 1.11.0 or the higer version. Use the accompanying version test. Thank you very much!

Or you can install the latest version tests like this

ansible-galaxy collection install azure.azcollection --force
pip3 install -r ~/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt

[nleiva]

I also run into this in Ansible with Python 3.9.7: "ModuleNotFoundError: No module named 'azure.mgmt.monitor.version'
ModuleNotFoundError: No module named 'azure.mgmt.monitor.version'" I believe, this is indeed a bug as @kohtala explains here: #441 (comment) (newer Azure library versions are incompatible with some imports in this collection)

@l3ender suggested can also be a sequence issue in some cases, which is plausible: #715 (comment)

azure_rm_common.py#L244

from azure.mgmt.monitor import MonitorManagementClient

dir(MonitorManagementClient)
['DEFAULT_API_VERSION', 'LATEST_PROFILE', '_PROFILE_TAG', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__enter__', '__eq__', '__exit__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_get_api_version', '_models_dict', 'action_groups', 'activity_log_alerts', 'activity_logs', 'alert_rule_incidents', 'alert_rules', 'autoscale_settings', 'baseline', 'baselines', 'close', 'data_collection_endpoints', 'data_collection_rule_associations', 'data_collection_rules', 'diagnostic_settings', 'diagnostic_settings_category', 'event_categories', 'guest_diagnostics_settings', 'guest_diagnostics_settings_association', 'log_profiles', 'management_group_diagnostic_settings', 'metric_alerts', 'metric_alerts_status', 'metric_baseline', 'metric_definitions', 'metric_namespaces', 'metrics', 'models', 'operations', 'predictive_metric', 'private_endpoint_connections', 'private_link_resources', 'private_link_scope_operation_status', 'private_link_scoped_resources', 'private_link_scopes', 'scheduled_query_rules', 'service_diagnostic_settings', 'subscription_diagnostic_settings', 'tenant_activity_logs', 'vm_insights']

[Fred-sun]

@nleiva Since the problem you encountered has not been copied yet, the problem you mentioned @jaydestro has been resolved through the new version. Can you reinstall it and try? Thank you very much!
#715 (comment)

[benjdudas]

@pun-ky @fiskhest I just ran into this as well. From what I can tell, it looks like the actual issue is being masked since any import failure below will just return that generic Failed to import the required Python library (ansible[azure] (azure >= 2.0.0)).

azure/plugins/module_utils/azure_rm_common.py

Lines 227 to 285 in 0a7eb5e

	try:
	from enum import Enum
	from msrestazure.azure_active_directory import AADTokenCredentials
	from msrestazure.azure_exceptions import CloudError
	from msrestazure.azure_active_directory import MSIAuthentication
	from azure.cli.core.auth.adal_authentication import MSIAuthenticationWrapper
	from msrestazure.tools import parse_resource_id, resource_id, is_valid_resource_id
	from msrestazure import azure_cloud
	from azure.common.credentials import ServicePrincipalCredentials, UserPassCredentials
	from azure.mgmt.network import NetworkManagementClient
	from azure.mgmt.resource.resources import ResourceManagementClient
	from azure.mgmt.managementgroups import ManagementGroupsAPI as ManagementGroupsClient
	from azure.mgmt.resource.subscriptions import SubscriptionClient
	from azure.mgmt.storage import StorageManagementClient
	from azure.mgmt.compute import ComputeManagementClient
	from azure.mgmt.dns import DnsManagementClient
	from azure.mgmt.privatedns import PrivateDnsManagementClient
	import azure.mgmt.privatedns.models as PrivateDnsModels
	from azure.mgmt.monitor import MonitorManagementClient
	from azure.mgmt.web import WebSiteManagementClient
	from azure.mgmt.containerservice import ContainerServiceClient
	from azure.mgmt.marketplaceordering import MarketplaceOrderingAgreements
	from azure.mgmt.trafficmanager import TrafficManagerManagementClient
	from azure.storage.blob import BlobServiceClient
	from adal.authentication_context import AuthenticationContext
	from azure.mgmt.authorization import AuthorizationManagementClient
	from azure.mgmt.sql import SqlManagementClient
	from azure.mgmt.servicebus import ServiceBusManagementClient
	import azure.mgmt.servicebus.models as ServicebusModel
	from azure.mgmt.rdbms.postgresql import PostgreSQLManagementClient
	from azure.mgmt.rdbms.mysql import MySQLManagementClient
	from azure.mgmt.rdbms.mariadb import MariaDBManagementClient
	from azure.mgmt.containerregistry import ContainerRegistryManagementClient
	from azure.mgmt.containerinstance import ContainerInstanceManagementClient
	from azure.mgmt.loganalytics import LogAnalyticsManagementClient
	import azure.mgmt.loganalytics.models as LogAnalyticsModels
	from azure.mgmt.automation import AutomationClient
	import azure.mgmt.automation.models as AutomationModel
	from azure.mgmt.iothub import IotHubClient
	from azure.mgmt.iothub import models as IoTHubModels
	from msrest.service_client import ServiceClient
	from msrestazure import AzureConfiguration
	from msrest.authentication import Authentication
	from azure.mgmt.resource.locks import ManagementLockClient
	from azure.mgmt.recoveryservicesbackup import RecoveryServicesBackupClient
	import azure.mgmt.recoveryservicesbackup.models as RecoveryServicesBackupModels
	from azure.mgmt.search import SearchManagementClient
	from azure.mgmt.datalake.store import DataLakeStoreAccountManagementClient
	import azure.mgmt.datalake.store.models as DataLakeStoreAccountModel
	from azure.mgmt.notificationhubs import NotificationHubsManagementClient
	from azure.mgmt.eventhub import EventHubManagementClient
	from azure.mgmt.datafactory import DataFactoryManagementClient
	import azure.mgmt.datafactory.models as DataFactoryModel
	from azure.identity._credentials import client_secret, user_password
	except ImportError as exc:
	Authentication = object
	HAS_AZURE_EXC = traceback.format_exc()
	HAS_AZURE = False

It looks like the newer versions of the azure-mgmt-monitor, azure-mgmt-network, azure-mgmt-storage, and azure-mgmt-compute made the changed from version.py to _version.py. I ended up making the following changes to the imports in azure_rm_common to get past those import failures.

from azure.mgmt.monitor._version import VERSION as monitor_client_version
from azure.mgmt.network._version import VERSION as network_client_version
from azure.mgmt.storage._version import VERSION as storage_client_version
from azure.mgmt.compute._version import VERSION as compute_client_version

[nleiva]

@Fred-sun please check out the reply by @benjdudas highlighting the problem with the python library imports in the collection. Users will continue to hit this issue until this is resolved.

[Fred-sun]

@nleiva Sorry for replying to you so late. Are you using the old version and installing a new dependency? Please try upgrading the current latest version“ansible-galaxy collection install azure.azcollection --force ” ! In another Pull Request （relate link: #751) ! have removed these unused imports. Thank you very much!

[nleiva]

Thank you for the pointer to the PR @Fred-sun. That solves the problem. So, in summary, this issue was introduced in 1.11 and later fixed in 1.12. We were stuck at 1.10, but we now upgraded successfully to 1.13.

[Fred-sun]

Greate, I will close this issue! Thank you very much!