The App With No Products

Introduction

Before getting to the bug, I need to explain how Apigee normally decides whether an application may call an API. The distinction matters because the bypass sat between two mechanisms that looked equivalent from the outside.

Apigee sits in front of backend services. A request arrives at an API proxy, and the proxy runs policies for authentication, routing, transformation, rate limits, and other gateway work. The customer does not usually hand every consumer direct access to every proxy. Instead, Apigee has a small object model for applications and permissions.

An API consumer registers a developer app. Apigee gives that app a consumer key and a consumer secret. The API provider then attaches one or more API products to the app. A product can include particular proxies, operations, paths, environments, quotas, and approval state. In practical terms, the credentials answer which app is calling?; the products answer what may that app call?

That is the foundation for both of the authentication styles involved in this research.

How Apigee gives an app access

The simpler path is an API key. A client sends the app's consumer key with the request, and a VerifyAPIKey policy checks it:

<VerifyAPIKey name="Verify-API-Key">
  <APIKey ref="request.queryparam.apikey"/>
</VerifyAPIKey>

This policy does more than check whether the key exists. Apigee resolves the developer app behind it, checks its state, and verifies that one of its API products covers the current proxy and resource.

OAuth in Apigee adds another step. Apigee can act as the authorization server: an app presents its credentials to a token endpoint, an OAuthV2 policy issues an access token, and another OAuthV2 policy validates that token on the protected proxy.

For my test I used the client_credentials grant. It is meant for a trusted client acting on its own behalf, normally from code that can protect the consumer secret. There is no end-user login in this grant; the application authenticates as itself.

<OAuthV2 name="Generate-Access-Token">
  <Operation>GenerateAccessToken</Operation>
  <SupportedGrantTypes>
    <GrantType>client_credentials</GrantType>
  </SupportedGrantTypes>
  <GrantType>request.queryparam.grant_type</GrantType>
  <GenerateResponse enabled="true"/>
</OAuthV2>

The target proxy then uses the same policy type with a different operation:

<OAuthV2 name="Verify-Access-Token">
  <Operation>VerifyAccessToken</Operation>
</OAuthV2>

The distinction between VerifyAPIKey and VerifyAccessToken has nothing to do with frontend and backend code. An API key is the shortest application-authentication path. OAuth exchanges app credentials or a user grant for a shorter-lived, independently revocable token with scopes and other metadata. Either way, the app's API products are supposed to remain the authorization boundary. Wrapping the key in an access token should not increase what the app may reach.

Apigee's app-registration documentation makes the empty-product edge case unusually clear. It warns administrators not to register an app without associating it with an API product that restricts specific proxies or resource paths, because the resulting key can behave like a skeleton key across the organization. The name is dramatic, but accurate: a credential that is not bounded to the intended doors is not much of an access-control mechanism.

At first that sounded like an administrator footgun. Administrators can create products, attach proxies, approve apps, and edit credentials; if they deliberately create an unbounded app, a warning in the administration documentation is reasonable. Then I noticed another place where apps were created.

The other door was the developer portal

An integrated developer portal is the public side of an Apigee program. The API provider publishes documentation and a catalog of available APIs there. External developers can register an account and manage their own apps, subscribe those apps to published products, and view or rotate their keys.

The word developer is easy to misread here. A portal developer is not necessarily someone who develops Apigee proxies and has access to the Google Cloud project. They are the customer of the API: the person or company writing an application that consumes it. Apigee's own portal documentation treats registration and app management as a self-service flow for those external consumers.

The portal is also not the source of truth for the app. When the user clicks Save, the portal sends a request to Apigee. Apigee stores the developer app and returns its key and secret. That server-side boundary is important because the browser belongs to the user.

The normal form required at least one published API. A user could not click Save with an empty selection. In the browser, the rule looked correct:

{
  "name": "my-app",
  "apiProducts": ["hello-world"]
}

I intercepted the request after the form validation and removed apiProducts:

{
  "name": "my-app"
}

The server accepted it. It created the app, issued a real consumer key and secret, and stored no valid product association. Updating an existing app with a malformed value such as apiProducts: [{}] reached the same state.

This was the missing server-side check. The UI told an external consumer that a product was mandatory, while the API behind it did not enforce the same invariant. Now the documented skeleton-key state was no longer limited to an administrator making a dangerous configuration choice. An ordinary portal user could create it for their own app.

I still did not know whether the credentials were useful. A malformed record is only interesting if the runtime trusts it, so I moved from the browser to dynamic analysis.

One app, two authorization paths

I built three small proxies on managed runtime 1-17-0-apigee-4: a token endpoint, a target protected by VerifyAccessToken, and a control target protected by VerifyAPIKey. All requests below used the same product-less developer app.

The API-key path denied the app

First I sent the consumer key directly to VerifyAPIKey:

GET /protected-by-verifyapikey?apikey=<CONSUMER_KEY> HTTP/1.1
Host: <APIGEE_HOST>

Apigee found the key but rejected the app for exactly the reason I expected:

HTTP/1.1 400 Bad Request
Content-Type: application/json

{
  "fault": {
    "faultstring": "Consumer key missing ApiProduct association",
    "detail": {
      "errorcode": "keymanagement.service.consumer_key_missing_api_product_association"
    }
  }
}

That response was useful. It proved the app and key were real, and it proved that the runtime considered the empty association invalid. The portal had created a bad state, but this enforcement path failed closed.

OAuth approved the same app

Then I sent the same key and its secret to the token proxy:

GET /oauth/accesstoken?grant_type=client_credentials HTTP/1.1
Host: <APIGEE_HOST>
Authorization: Basic Base64(<CONSUMER_KEY>:<CONSUMER_SECRET>)

Instead of rejecting the product-less app, Apigee minted a token. I have trimmed and redacted the response, but the two important fields are untouched:

{
  "access_token": "[REDACTED]",
  "api_product_list": "[]",
  "api_product_list_json": [],
  "grant_type": "client_credentials",
  "expires_in": "1799",
  "status": "approved"
}

The token did not accidentally inherit a product. It explicitly carried an empty product list and was still marked approved.

Token issuance alone was not the bypass. VerifyAccessToken still had an opportunity to reject it when I called the protected proxy:

GET /protected HTTP/1.1
Host: <APIGEE_HOST>
Authorization: Bearer <ACCESS_TOKEN>

HTTP/1.1 200 OK

Same app, same underlying credentials, opposite answer. VerifyAPIKey said the missing product invalidated the key. OAuthV2 issued a token for it and then accepted that token at a protected proxy.

Adding a product removed access

I wanted a control for the less obvious possibility that the target proxy was simply outside product enforcement. I attached a valid product to the app, but chose one that did not include /protected. After the relevant caches refreshed, I repeated the token flow.

Invalid API call as no apiproduct match found

That result made the bug hard to explain away. With the wrong product, Apigee evaluated the boundary and denied the call. With no product, the call succeeded. Adding a restriction made the app less powerful!

The behavior was not a universal bypass of every control in Apigee. It required a consumer-facing portal, a usable OAuth token endpoint, and a target that relied on OAuthV2 with VerifyAccessToken. Authorization implemented inside the backend could still deny the request. The broken boundary was specific and important: the API products selected by the provider did not constrain the product-less OAuth client.

Where the two paths split

The dynamic results gave me a clean code-review question: where does VerifyAPIKey reject an empty product list, and why does VerifyAccessToken not make the same decision?

The API-key path eventually reached OAuth2RuntimeServiceImpl.getConsumerDetailsForApiKey(...). Its behavior was controlled by a plainly named setting:

conf_feature-flags_allow.app.without.product=false

The runtime exposed that setting through KMSConfiguration.isAppWithoutAPIProductAllowed(). With the default set to false, a key whose app had no product association raised CONSUMER_KEY_MISSING_APIPRODUCT_ASSOCIATION. That matched the 400 response from the control proxy.

The OAuth path reached another part of the same runtime service. populateApiProductAttributes(...) loaded the products associated with the token, tried to find one covering the current proxy and resource, and tracked the result in proxyOrResourceSet. The decisive branch in the decompiled runtime was:

if (apiProductObjects.size() == 0 || !proxyOrResourceSet) {
    if (this.kmsFeatures.disableUnboundedPermissions()) {
        throw new UnAuthorizedException(
            ErrorCodes.INVALID_API_CALL_AS_NO_APIPRODUCT_MATCH_FOUND,
            new Object[0]);
    }
}

Read the two conditions separately. The outer branch knows that the token has no products or that none of its products cover the proxy/resource. But knowing that is not enough to deny the request. The exception sits behind disableUnboundedPermissions().

The same feature gate appeared in the adjacent blank-resource case:

if (resource != null && StringUtils.isBlank(resource)) {
    if (this.kmsFeatures.disableUnboundedPermissions()) {
        throw new UnAuthorizedException(
            ErrorCodes.CONSUMER_KEY_WITH_EMPTY_FLOW_RESOURCE_NAME,
            new Object[0]);
    }
}

That was the code-level explanation for the strange control result. When a real product existed but did not cover the proxy, the runtime had a concrete product boundary to enforce and the request failed. For the empty set, OAuth retained older unbounded permissions behavior unless a separate legacy-style feature disabled it. The current API-key path did not rely on that feature; it had its own fail-closed allow.app.without.product guard.

That was the split: the two entry points did not share one invariant. The API-key path asked, "Is an app without a product allowed?" The OAuth path asked, "Has unbounded access been disabled here?" Those questions happened to produce different answers for exactly the state the portal let an external user create.

The warning was aimed at the wrong actor

When I reported the issue, the initial objection was the existing skeleton-key warning. I understood the reaction: the documentation already told Apigee administrators that product-less credentials were dangerous.

But the warning was also why I kept pushing. An API provider is trusted to define products and decide which consumers receive them. A portal user is offered a catalog and allowed to request access to those published choices. Removing one field from a self-service request should not give that consumer the state administrators are explicitly warned never to create.

The difference sounds obvious once the actors are named, but the word developer caused a surprising amount of confusion. An Apigee proxy developer or administrator works on the provider side. An app developer in the portal may be any external API customer with a self-registered account. Treating the latter as already trusted collapses the entire purpose of products and app registration.

I wrote back, more than once, because that distinction also changed the security impact. The first answer came with no bounty. After reconsideration, Google awarded $3,760.44; after another round about the attacker model and the role of the portal, the total became $9,000. I was obviously happy about the money! More importantly, the discussion landed on the part of the bug I cared about: a warning for administrators cannot be the security boundary for an attacker-controlled request.

The empty set now means deny

Google's May 12, 2026 Apigee release notes describe the fix in managed release 1-17-0-apigee-7: OAuthV2 now enforces API product associations and denies apps without valid products. The note marks the beginning of a multi-zone rollout, so it is not an exact timestamp for every instance.

My favorite artifact from this research is still the three-result control: a valid matching product grants access, a wrong product denies it, and an empty product list used to grant more access than either. I now test permission collections in exactly that order. "Nothing" deserves its own test case. In authorization code, it is often where old compatibility behavior, missing validation, and an attacker's best option meet.