No credentials created for Microsoft services when warning raised Warning: Scope has changed

[jaknel]

~/.config/oauth2token/microsoft/config.json

{
    "web": {
        "client_id": "someclientid",
        "client_secret":  "someclientsecret",
        "auth_uri": "https://login.microsoftonline.com/common/oauth2/v2.0/authorize",
        "token_uri": "https://login.microsoftonline.com/common/oauth2/v2.0/token"
    }
}

~/.config/oauth2token/microsoft/scopes.json

["https://outlook.office365.com/IMAP.AccessAsUser.All", "https://outlook.office365.com/POP.AccessAsUser.All", "https://outlook.office365.com/SMTP.Send", "offline_access"]

Running oauth2create microsoft <account_name> raises the following warning:

Warning: Scope has changed from "https://outlook.office365.com/IMAP.AccessAsUser.All https://outlook.office365.com`/POP.AccessAsUser.All https://outlook.office365.com/SMTP.Send offline_access" to "https://outlook.office365.com/IMAP.AccessAsUser.All https://outlook.office365.com/POP.AccessAsUser.All https://outlook.office365.com/SMTP.Send".`

Note that the scope rewrite removes offline_access from the list of scopes. This warning interferes with properly configuring the account. If offline_access is omitted from scopes.json then the initial create procedure succeeds, however, refresh_token is not set on the returned credentials leading to: RefreshError: The credentials do not contain the necessary fields need to refresh the access token. You must specify refresh_token, token_uri, client_id, and client_secret. after the initial authorization expires and oauth2get attempts to refresh the token.

The fix I found is mentioned in oauthlib/oauthlib#562 Rewriting the scope is allowed by the RFC so the warning should not be interfering with the create process. The workaround is to run the command while setting an environment variable OAUTHLIB_RELAX_TOKEN_SCOPE.

OAUTHLIB_RELAX_TOKEN_SCOPE=True oauth2create microsoft <account_name>

This leads to an additional problem when oauth2get runs: RefreshError: Not all requested scopes were granted by the authorization server, missing scopes offline_access.

flow.credentials stores the original scopes as flow.oauth2session.scope instead of the rewritten scopes returned by the server available in flow.oauth2session.token['scope']

A change to create_user_credentials in oauth2token/token_mgmt.py fixes this

def create_user_credentials(app=None, user=None, **kwargs):
    flow = InstalledAppFlow.from_client_config(*get_json_config(app=app))

    flow.run_local_server(port=0)

    creds = flow.credentials
    creds._scopes = flow.oauth2session.token["scope"]

    pickle.dump(creds, get_credentials_file(app=app, user=user, override=True))

[jaknel]

This is likely the root of the issue referenced at #4 (comment) but I didn't want to mix this issue with #4 as I haven't tested against Google's services.

The user is missing offline_access from their scope.json but would run into the above problem if they added it.

[VannTen]

Good catch 👍 Would you mind opening a PR or sending a patch ?

[VannTen]

To clarify :

The fix I found is mentioned in oauthlib/oauthlib#562 Rewriting the scope is allowed by the RFC so the warning should not be interfering with the create process. The workaround is to run the command while setting an environment variable OAUTHLIB_RELAX_TOKEN_SCOPE.

OAUTHLIB_RELAX_TOKEN_SCOPE=True oauth2create microsoft <account_name>

This leads to an additional problem when oauth2get runs: RefreshError: Not all requested scopes were granted by the authorization server, missing scopes offline_access.

flow.credentials stores the original scopes as flow.oauth2session.scope instead of the rewritten scopes returned by the server available in flow.oauth2session.token['scope']

A change to create_user_credentials in oauth2token/token_mgmt.py fixes this

def create_user_credentials(app=None, user=None, **kwargs):
    flow = InstalledAppFlow.from_client_config(*get_json_config(app=app))

    flow.run_local_server(port=0)

    creds = flow.credentials
    creds._scopes = flow.oauth2session.token["scope"]

    pickle.dump(creds, get_credentials_file(app=app, user=user, override=True))

Do you still need the env var with your fix ?

[VannTen]

Could you test the 0.0.3 ? I've commited your fix.

[javiertury]

I run into the same problem and fixed it with OAUTHLIB_RELAX_TOKEN_SCOPE=True oauth2create microsoft <account_name>

Could you test the 0.0.3 ? I've commited your fix.

For me it worked

Thank you both @jaknel @VannTen

Hi @VannTen,

Thank you for your work on this! I am running with 0.0.3 and unfortunately still have this exact same problem: running against Microsoft authentication, if I do not provide the "offline_access" scope then I do not get a refresh token, but if I do then the server's answer does not contain the "offline_access" scope, which leads to a warning that actually stops the creation process. Using the environment variable "workaround" still works, but the above code modification appears to be insufficient... I would love to have sugestions but I know very little about all of this yet, so for now I cannot do much more than report the problem. Please tell me if you need any additional information!

[VannTen]

Please tell me if you need any additional information!

Hi ! Could you copy here your config.json and scopes.json ? (with client_id/client_secret scrubbed ofc) + the full output you get. Also, are those Microsoft account corporate ones / do you know when I can create one for testing if needed ? Thanks /reopen

Here is my config.json:

{
  "web": {
    "client_id": "<scrubbed>",
    "client_secret": "<scrubbed>",
    "auth_uri": "https://login.microsoftonline.com/common/oauth2/v2.0/authorize",
    "token_uri": "https://login.microsoftonline.com/common/oauth2/v2.0/token"
  }
}

Here is my scopes.json:

["offline_access", "https://outlook.office365.com/SMTP.Send", "https://outlook.office365.com/POP.AccessAsUser.All", "https://outlook.office365.com/IMAP.AccessAsUser.All"]

Here is the output of oauth2create <app> <user>:

Please visit this URL to authorize this application: https://login.microsoftonline.com/common/oauth2/v2.0/authorize?response_type=code&client_id=<scrubbed>&redirect_uri=http%3A%2F%2Flocalhost%3A34247%2F&scope=offline_access+https%3A%2F%2Foutlook.office365.com%2FSMTP.Send+https%3A%2F%2Foutlook.office365.com%2FPOP.AccessAsUser.All+https%3A%2F%2Foutlook.office365.com%2FIMAP.AccessAsUser.All&state=Y9CnisB1mouSRJ8AaddXrjirOFtjph&access_type=offline
Traceback (most recent call last):
  File "/usr/bin/oauth2create", line 11, in <module>
    create_user_credentials(user=args.user, app=args.app)
  File "/usr/lib/python3.10/site-packages/oauth2token/token_mgmt.py", line 16, in create_user_credentials
    flow.run_local_server(port=0)
  File "/usr/lib/python3.10/site-packages/google_auth_oauthlib/flow.py", line 505, in run_local_server
    self.fetch_token(authorization_response=authorization_response)
  File "/usr/lib/python3.10/site-packages/google_auth_oauthlib/flow.py", line 300, in fetch_token
    return self.oauth2session.fetch_token(self.client_config["token_uri"], **kwargs)
  File "/usr/lib/python3.10/site-packages/requests_oauthlib/oauth2_session.py", line 366, in fetch_token
    self._client.parse_request_body_response(r.text, scope=self.scope)
  File "/usr/lib/python3.10/site-packages/oauthlib/oauth2/rfc6749/clients/base.py", line 448, in parse_request_body_response
    self.token = parse_token_response(body, scope=scope)
  File "/usr/lib/python3.10/site-packages/oauthlib/oauth2/rfc6749/parameters.py", line 441, in parse_token_response
    validate_token_parameters(params)
  File "/usr/lib/python3.10/site-packages/oauthlib/oauth2/rfc6749/parameters.py", line 471, in validate_token_parameters
    raise w
Warning: Scope has changed from "https://outlook.office365.com/SMTP.Send https://outlook.office365.com/POP.AccessAsUser.All offline_access https://outlook.office365.com/IMAP.AccessAsUser.All" to "https://outlook.office365.com/SMTP.Send https://outlook.office365.com/POP.AccessAsUser.All https://outlook.office365.com/IMAP.AccessAsUser.All".

Whereas prefixing the above command with the mentioned environment variable exits correctly. If ran after the create command that exits with an error, the get command returns RefreshError: The credentials do not contain the necessary fields need to refresh the access token. You must specify refresh_token, token_uri, client_id, and client_secret.. If ran after the create command with the workaround, then the get command returns a token as expected.