fix: replace instanceof checks with duck-typing for NextRequestHint compatibility

[marioesho]

• All new/changed/fixed functionality is covered by tests (or N/A)
• I have added documentation for all new/changed functionality (or N/A)

📋 Changes

Problem

When server actions trigger token refresh in middleware, the token refresh fails because NextRequestHint objects fail instanceof NextRequest checks in the updateSession() and saveToSession() methods.

This causes the code to incorrectly fall through to the Pages Router path, which tries to call appendHeader() - a method that doesn't exist on NextResponse, resulting in runtime errors.

Error:

TypeError: pagesRouterRes.appendHeader is not a function
    at Auth0Client.saveToSession (webpack-internal:///(rsc)/../../node_modules/.pnpm/@auth0+nextjs-auth0@4.13.3_next@15.1.3_react@19.0.0/node_modules/@auth0/nextjs-auth0/dist/server/client.js:531:29)

Affected Users

• All users whose server actions trigger token refresh in middleware
• Applications using both default cookie-based sessions and stateful session stores (DynamoDB, Redis, custom stores)
• Applications with proactive token refresh in middleware

Solution

Replace instanceof NextRequest && res instanceof NextResponse checks with duck-typing that validates the presence of required methods:

const isMiddlewareRequest = req.cookies && res.cookies &&
    typeof req.cookies.get === 'function' &&
    typeof res.cookies.set === 'function';

Why Duck-Typing?

1. Structural compatibility: Validates actual capabilities rather than type lineage
2. Works with Next.js internals: NextRequestHint from server actions has the same cookie interface as NextRequest
3. Future-proof: Will work with any request/response wrapper that implements the cookie interface
4. TypeScript best practice: Aligns with TypeScript's structural type system
5. Backward compatible: All existing NextRequest/NextResponse objects pass the check

Code Changes

File: src/server/client.ts

1. updateSession() method (~line 936)

Before:

if (req instanceof NextRequest && res instanceof NextResponse) {
    // middleware usage

After:

// Check if req/res have the right structure for middleware usage
// (supports both NextRequest and NextRequestHint from server actions)
const isMiddlewareRequest = req.cookies && res.cookies &&
    typeof req.cookies.get === 'function' &&
    typeof res.cookies.set === 'function';

if (isMiddlewareRequest) {
    // middleware usage (NextRequest or NextRequestHint)

2. saveToSession() method (~line 1148)

Before:

if (req instanceof NextRequest && res instanceof NextResponse) {
    // middleware usage

After:

// Check if req/res have the right structure for middleware usage
// (supports both NextRequest and NextRequestHint from server actions)
const isMiddlewareRequest = req.cookies && res.cookies &&
    typeof req.cookies.get === 'function' &&
    typeof res.cookies.set === 'function';

if (isMiddlewareRequest) {
    // middleware usage (NextRequest or NextRequestHint)

Impact

Before

• ❌ Token refresh triggered by server actions fails with appendHeader is not a function
• ❌ Both default cookie sessions and stateful session stores are affected
• ❌ Users experience authentication errors mid-session

After

• ✅ Token refresh triggered by server actions works correctly
• ✅ Works with all session store types (default cookies, DynamoDB, Redis, custom stores)
• ✅ Seamless authentication experience
• ✅ No breaking changes for existing code

Breaking Changes

None. This is a backward-compatible bug fix that maintains all existing functionality.

📎 References

Fixes #2253
Related to #2124, #1934

🎯 Testing

Verified With

• ✅ Default cookie-based session store (StatelessSessionStore)
• ✅ DynamoDB session store (StatefulSessionStore)
• ✅ Server actions triggering token refresh via middleware
• ✅ Next.js 14 and 15
• ✅ Production workload with proactive token refresh (90-second buffer)
• ✅ Both NextRequest and NextRequestHint objects

Test Scenario

// Middleware with proactive token refresh
export async function middleware(request: NextRequest) {
  const response = NextResponse.next();
  const session = await auth0.getSession(request);

  if (session) {
    const now = Math.floor(Date.now() / 1000);
    const needsRefresh = session.tokenSet.expiresAt - now <= 90;

    // This now works correctly with server actions!
    await auth0.getAccessToken(request, response, { refresh: needsRefresh });
  }

  return response;
}

Note on Tests:

• Existing tests continue to pass without modification (backward compatible)
• Tested locally with default cookie-based sessions
• Tested in production with DynamoDB session store
• The duck-typing approach maintains all existing functionality for NextRequest/NextResponse