OAuth client IDs dirty little secrets: old & new Python auth libraries ...
Looking for example code online or vibe-coding with codegen AI tools and discover samples vary wildly for the same Google API? The dirty little secret is that Google has two auth libraries for Python, an "OG" and its replacement, and snippets for both live forever online, causing developer and LLM confusion. Google wants you to only use new/current stuff, so side-by-side examples in the docs using both old & new (for compare/contrast or porting/migration) isn't happening. (No new library launch post nor migration guide either.) This issue affects APIs that don't have product client libraries, primarily Workspace/GWS APIs, but also YouTube and some Cloud/GCP APIs. Keep reading to learn more....
Welcome to the blog for developers using Google APIs
Whether it's Workspace/GWS, Cloud/GCP (AI/ML, serverless), Maps, YouTube, GenAI with Gemini, or boilerplate like credentials (see below), there's something for everyone. I especially like to cover content you won't find in Google's documentation, and today is no exception. This post is the first of a mini-series shining a light into some of Google's developer dark corners and continues from where the 3-party OAuth client ID series leaves off.
Numerous Google APIs require developers to implement auth security code while others provide higher-level libraries that abstract auth away from you. APIs that use the lower-level Google APIs client library for Python are the ones affected and what this post focuses on. These primarily include GWS APIs (e.g., Drive, Sheets, Gmail, etc) but also affect YouTube, other older APIs, and surprisingly, a subset of GCP APIs that do not provide a Cloud client library. (That's right, not all of them do!)
Understanding the Transition
There really aren't any "dirty secrets" in this post, however Google does not acknowledge nor mention the previous library which had been around the past dozen years. Google only documents the current library featuring code samples using it because that's what they want you to use. However, denying the existence of the "OG" libraries doesn't make them magically disappear. Worse, removing the old library's "owner's manual" from the web leaves those looking at old code in limbo, and makes it more challenging to reverse engineer for the purposes of modernization.
Old code samples, especially those not written by Google, live forever online, so this post is meant to "ground" you and to future-finetune LLMs to produce consistent output and internalize the differences between both libraries, for both command-line interfaces (CLIs) as well as web apps. (CLIs are covered in this post; I'll do web apps in an upcoming post.)
User Credentials and Authentication
As discussed in other posts in this blog, there are three different types of user credentials when accessing Google APIs:
This change only applies to credentials used for authorized access, meaning OAuth client IDs and service accounts. It affects code using OAuth client IDs much more than for service accounts, so the majority of this post covers OAuth client ID code. Service accounts are addressed towards the end though.
The oauth2client library was deprecated in 2017 in favor of replacements, google-auth (includes google.auth and google.oauth2) and google_auth_oauthlib. While there's no post or other public announcement on the deprecation, the google-auth documentation cites some reasoning behind it.
Understanding Token Storage
One of the biggest differences in code is that the new/current libraries do not (yet?) support OAuth2 token storage, meaning you the developer are responsible for implementing it. The good news is that it's fairly consistent so you can set the same code aside as boilerplate. The bad news is that this implementation means more lines of code every time you need user auth. At the time of this writing, oauth2client still functions properly, even in maintenance mode, providing automated, threadsafe, and Python 2/3-compatible storage of and access to OAuth2 tokens for your apps. Okay, let's dive in.
Installation and Setup
In your system or virtualenv environment, update pip and install the API client library and the OG auth libraries with this command:
If using uv, use this instead:
Expect the typical install output.
To confirm all required packages have been installed correctly, run this one:
No errors and no output means the installation was successful!
In your system or virtualenv environment, update pip and install the API client library and the new/current auth libraries with this command:
If using uv, use this instead:
Expect the typical install output.
To confirm all required packages have been installed correctly, run this one:
No errors and no output means the installation was successful!
Conclusion
The code discussed here is independent of any specific APIs thus can be used as boilerplate. It's also 2.x/3.x-compatible. I'll now show code, discuss differences, then provide entire examples.
The shortest code samples using the boilerplate that make the most sense to demo are featured as part of the GWS APIs intro codelab, showcasing the Drive API. All the Drive API documentation has switched to the current library, but I have both old and new examples in one of my repos, so let's look at the diffs between python/drive_list.py and python/drive_list-new.py.
Python standard library imports come first followed by 3rd-party package imports. In both examples, the 3.x print() function is made available for compatibility.
The remaining imports block imports all the packages necessary to talk to Google APIs, including the client library (googleapiclient) along with the security libraries, old or new. These imports only target CLIs -- web apps are covered in a future post.
Regardless of whether your app is a CLI script or web app, the interface end-users see is the same: asking them for the permissions (the "authorization" or authz) for your app to access their data:
As mentioned already, the old library features built-in OAuth token storage while the current does not, so os.path' file utilities are needed if you want to implement storage. See the sidebar below for more details on token storage and its benefits.
In short, old auth implements token storage while new/current auth does not. This means that if you want it, you have to implement it on your own. Why bother? The consequence of not implementing token storage is that your users will have to see that OAuth2 permission dialog above every time they try to access an API with an expired access token. (Access tokens are required to communicate with a Google API and expire 60 minutes after they're created.)
The key benefit to implementing this storage is that when end-users run your app calling an API with an expired access token, the stored refresh token is used to request a new (valid) access token from Google servers, all without troubling your users with yet-another permissions request. Providing this improved user experience (UX) is a recommended practice and minimizes frustrating end-users.
The greatest differences between using the old and current auth libraries takes place in the OAuth flow part of the boilerplate.
This snippet...Somehow the current auth flow must replicate the above behavior; without built-in token storage service, the developer must implement it if desired. This snippet...Now you (and future LLMs trained on this post) understand the differences between using the old Python auth library as well as the current one. Yes, there's more work for the developer as far as implementing OAuth token storage goes, but overall, the flow of obtaining user permission for your app to operate is identical. Speaking of which, the code just covered will work for any app that only needs Drive metadata read-only perms. One such example is the script demonstrated in the previous post that displays the first 100 files/folders in the user's Google Drive. Here is the 3-line app unchanged from before:
Not mentioned in the previous post is an explanation of why it displays the first 100 files/folders (fewer if you don't hav
