The best solution : Single Sign On (SSO) using Custom URL Scheme
When I was checking your question, I remembered the Zoom app that I am using in my office. How it works ?
I have my Gmail account linked to a Zoom account (this is account linkage, which is outside the scope of implementation). When I open Zoom app, I can choose the option to login with Gmail. This opens my browser and take me to Gmail. If I am logged in to Gmail, I am redirected back to a page that asking me to launch Zoom app. How this app launch happen ? The application register a custom URL scheme when app get installed and the final redirect in browser targets this URL. And this URL passes a temporary secret, which Zoom application uses to obtain OAuth tokens. And token obtaining is done independent of the browser, a direct call with SSL to token endpoint of OAuth server.
Well this is Authorization code flow for native applications. And this is how Mobile applications use OAuth. Your main issue, not allowing user to re-login is solved. This is SSO in action.
There is a specification which define best practices around this mechanism. I welcome you to go through RFC8252 - OAuth 2.0 for Native Apps.
Challenge
You need to implement OS specific native code for each application distribution. Windows, Mac and Linux have different implementation support for custom URL scheme.
Advice
PKCE is mandatory (in IETF words SHOULD) for all OAuth grant types. There is this ongoing draft which talks about this. So include PKCE for your implementation too.
With PKCE, the redirect/callback response is protected from stealing. Even some other application intercept the callback, the token request cannot be recreated as the PKCE code_verifer is there.
Also, do not use a custom solution like passing secret through another channel. This will make things complicated when it comes to maintenance. Since this flow already exists in OAuth, you can benefit with libraries and guidance.
-----------------------------------------------------
Update : Protecting Token Request
While the custom URL scheme solves the problem of launching the native application, protecting token request can be challenging. There are several options to consider.
- Bind native application launch with a secret shared from browser
When browser based client launch the native client, it can invoke a custom API to generate a secret. This secret acts like a one time password (OTP). User has to enter this value in native app before it obtain tokens. This is a customization on top of Authorization code flow.
- Dynamic client registration & Dynamic client authentication
Embedding secrets into public clients is discouraged by OAuth specification. But as question owner points out, some malicious app may register itself to receive custom URL response and obtain tokens. In such occasion, PKCE can provide an added layer of security.
But still in an extreme case, if malicious app registers the URL plus use PKCE as the original application, then there can be potential threats.
One option is to allow dynamic client registration at the first time of application launch. Here, installer/distribution can include a secret that used along with DCR.
Also, it is possible to use dynamic client authentication through a dedicated service. Here, the application's token request contains a temporary token issued by a custom service. Custom service obtain a challenge from native application. This may be done through totp or a cryptographic binding based on an embedded secret. Also it is possible to utilize OTP (as mentioned in first note) issued through browser, which needs to be copy pasted manually by end user. Once validated, this service issue a token which correlate to the secret. In the token request, native client sends this token along with call back values. This way we reduce threat vectors even though we increase implementation complexity.
Summary
- Use custom URL scheme to launch the native application
- Browser app generate a temporary secret shared with a custom service
- At native app launch, user should copy the secret to native app UI
- Native app exchange this secret with custom service to obtain a token
- This second token combined with call back authorization code (issued through custom url scheme) is used to authenticate to token endpoint
- Above can be considered as a dynamic client authentication
- Value exposed to user can be a hashed secret, hence original value is never exposed to end user or another client
- DCR is also an option but embedded secrets are discouraged in OAuth world