Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: Make dependancy adal optional #668

Merged
merged 1 commit into from
Feb 7, 2019

Conversation

iamneha
Copy link
Contributor

@iamneha iamneha commented Oct 28, 2018

No description provided.

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Oct 28, 2018
requirements.txt Outdated
@@ -9,4 +9,4 @@ ipaddress>=1.0.17;python_version=="2.7" # PSF
websocket-client>=0.32.0,!=0.40.0,!=0.41.*,!=0.42.* # LGPLv2+
requests # Apache-2.0
requests-oauthlib # ISC
adal>=1.0.2 # MIT
adal>=1.0.2;python_version=='2.7' # MIT
Copy link
Contributor

@micw523 micw523 Oct 29, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will cause adal to be installed automatically for Python 2.7.* , but not Python 3.* with setuptools. Since python-base does not have its own requirements.txt, it may cause kube_configto fail.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@micw523 can you tell me how to test this locally.

requirements.txt Outdated
@@ -9,4 +9,5 @@ ipaddress>=1.0.17;python_version=="2.7" # PSF
websocket-client>=0.32.0,!=0.40.0,!=0.41.*,!=0.42.* # LGPLv2+
requests # Apache-2.0
requests-oauthlib # ISC
adal>=1.0.2 # MIT
adal>=1.0.2;python_version=='2.7' # MIT
adal>=1.0.2;python_version=='3' # MIT
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am afraid this won’t work - see Travis CI information. Supported environment markers are documented in PEP 496 (
https://www.python.org/dev/peps/pep-0496/).

To test your changes, you’ll need to have a Python 2.7 and a python 3.4, 3.5, 3.6 environment (as specified in Travis CI, but usually one of 3.4/5/6 is enough). Use setuptools to install your package from your local source, and use python -m unittest discover to test your code. Changing adal to be optional will at least require restructuring of existing code to let it be allowed as an extra install component and will break existing user experience using Azure. I suggest you consult the owners of the repo on how they want to do it.

Copy link
Contributor Author

@iamneha iamneha Oct 29, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@micw523 I'm testing the same way you said.
Adding this in requirements.txt

adal>=1.0.2;python_version=='2.7' # MIT
adal>=1.0.2;python_version=='3.5' # MIT
adal>=1.0.2;python_version=='3.6' # MIT
adal>=1.0.2;python_version=='3.7' # MIT

All the test are passing locally but I am not sure if this is a good way to do.
Let me talk to the maintainer of this project.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You probably have a local copy of adal. You can try using pip to list your installed packages. Try start with a fresh python environment and install from your local source with setuptools.

I think you probably has misunderstood me - I was simply saying python_version='3' is not a valid environment marker. You'll need to restructure the code in kubernetes/python-base. Most likely there will be only a one-line change in the requirements.txt.

@roycaihw
Copy link
Member

I'd expect marking the dependency as optional would require some breaking change, since users depending on the feature have to opt-in (or we could default on, and allow users to opt-out to be maximum backward compatible-- new optional dependency should be defaulted off)

At any rate, we should document the change well

@tomplus
Copy link
Member

tomplus commented Oct 30, 2018

https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies

It was suggested in the issue as a twitter comment and it looks like what we need.

@micw523
Copy link
Contributor

micw523 commented Oct 30, 2018

@tomplus Yes that’s where it is and our requirements are stored in the requirements.txt and are being broken up into regular requirements and extra requirements. We are currently specifying setuptools >= 0.20 so we could not use env markers directly in the regular installs (introduced in 0.36.0)

Since the adal is used in python-base/config, it’s highly likely that we’ll need to segment it out and define a new entry point for azure users, otherwise it would probably break the CI. I haven’t tested it though since Neha is on this issue.

@iamneha
Copy link
Contributor Author

iamneha commented Oct 30, 2018

@micw523 you are right. I went through the code whole day and seems like changing requirement is not going to solve this issue.

@tomplus
Copy link
Member

tomplus commented Oct 30, 2018

I thought about simpler solution:

  1. remove adal from requirements
  2. add extras_require to setup.py if setuptools>=0.36
  3. handle import error or use lazy loading
try:
   import adal
except ImportError:
   pass
...
...
def _refresh_azure_token():
    if 'adal' not in globals():
        raise Error('refresh token error, adal library not imported')
  1. update doc/readme that adal have to be installed manually if you are integrated with MS Active Directory

@micw523
Copy link
Contributor

micw523 commented Oct 31, 2018

@tomplus That sounds good to me. We don’t need setuptools version 0.36 or up - it’s only that we can put everything in the requirements.txt into the installs_require block and simplify the setup.py logic. If we’re using adal for extras require I think 0.20 should be fine.

It’s relatively simple for user to install adal by themselves, see the end of the page at https://packaging.python.org/tutorials/installing-packages/#installing-setuptools-extras

@iamneha iamneha changed the title Fix: Make dependancy adal optional [WIP]Fix: Make dependancy adal optional Dec 3, 2018
@k8s-ci-robot k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 3, 2018
@iamneha iamneha changed the title [WIP]Fix: Make dependancy adal optional Fix: Make dependancy adal optional Dec 11, 2018
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 11, 2018
@iamneha
Copy link
Contributor Author

iamneha commented Dec 11, 2018

@tomplus can you look into this as well. The test is failing after I added adal into the extra_required
The possible fix for this is to handle import error in the test as well.

@tomplus
Copy link
Member

tomplus commented Dec 11, 2018

@iamneha There are not tests dedicated for the adal integration. These tests will pass if your previous PR (kubernetes-client/python-base#108) is merged.

@iamneha
Copy link
Contributor Author

iamneha commented Dec 12, 2018

@tomplus that make sense to me. Thanks for the help :)

@k8s-ci-robot
Copy link
Contributor

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please follow instructions at https://git.k8s.io/community/CLA.md#the-contributor-license-agreement to sign the CLA.

It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.


  • If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
  • If you signed the CLA as a corporation, please sign in with your organization's credentials at https://identity.linuxfoundation.org/projects/cncf to be authorized.
  • If you have done the above and are still having issues with the CLA being reported as unsigned, please email the CNCF helpdesk: [email protected]

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@k8s-ci-robot k8s-ci-robot added cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. and removed cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Feb 6, 2019
@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Feb 6, 2019
@iamneha iamneha force-pushed the adal branch 11 times, most recently from d660448 to 6e11e5f Compare February 7, 2019 21:51
@roycaihw
Copy link
Member

roycaihw commented Feb 7, 2019

/lgtm
/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: iamneha, roycaihw

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Feb 7, 2019
@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 7, 2019
@roycaihw roycaihw merged commit 2c02595 into kubernetes-client:master Feb 7, 2019
@ktdreyer
Copy link

Thank you for merging this!

Would you please release a new version of this library to PyPI with the adal requirement removed? It would be great to have this fixed in an official version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants