When using CLI we try to do a token exchange at the login. This prevents us from using offline tokens for login. Some discussion happened in #326 where we have added way to use offline token but this requires manual edits in the config.