feat(request): add cloneRawRequest utility for request cloning

[kamaal111]

Problem

After using Hono validators, the raw Request obejct gets consumed
during body parsing, making it unusable for external libraries like
better-auth.
This results in the error:

TypeError: Cannot construct a Request with a Request object that has already been used.

Root Cause

The issue occurs in the #cachedBody method in HonoRequest. When parsing request
bodies (json, text, etc.), the method directly calls parsing methods on the raw Request
object, which consumes its body stream. Once consumed, the Request cannot be cloned or
reused by external libraries.

Solution

Adding a utility function to clone HonoRequest's underlying raw Request
object, handling both consumed and unconsumed request bodies.

The author should do the following, if applicable

• Add tests
• Run tests
• bun run format:fix && bun run lint:fix to format the code
• Add TSDoc/JSDoc to document the code

[codecov]

Codecov Report

❌ Patch coverage is 90.32258% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.31%. Comparing base (4b796cf) to head (d4ceece).
⚠️ Report is 3 commits behind head on next.

Files with missing lines	Patch %	Lines
src/request.ts	90.32%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             next    #4382      +/-   ##
==========================================
- Coverage   91.32%   91.31%   -0.01%     
==========================================
  Files         173      173              
  Lines       11084    11115      +31     
  Branches     3201     3204       +3     
==========================================
+ Hits        10122    10150      +28     
- Misses        961      964       +3     
  Partials        1        1              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[yusukebe]

Hi @kamaal111

Why I didn't implement cloning the request is that it will introduce significant performance degradation.

The app:

import { Hono } from '../../src'

const app = new Hono()

app.post('/', async (c) => {
  const data = await c.req.json()
  return c.json(data)
})

export default app

Benchmark command:

bombardier -c 125 --fasthttp --method=POST --body='{"foo":"bar"}' http://localhost:3000/

Result:

We should find a good way to resolve the problem without performance delegation.

[kamaal111]

I understand, thank you @yusukebe!
What about instead of cloning the Request object, I instead instantiate a new Request object with what is available in HonoRequest, and construct that in a new method?

Something like this:

export class HonoRequest {
    // ...
    new = () => {
        return new Request(this.url, {
            // Collect request init from this class without accessing `raw`
        })
    }
}

That would then get consumed as the following:

const app = new Hono()

app.post(
  '/login',
  zValidator(
    'json',
    z.object({
      username: z.string(),
    })
  ),
  (c) => {
    const validated = c.req.valid('json')
    // `auth` from better-auth for example
    const response = await auth.handler(c.req.new());
  }
)

Would it be worth it to contribute this to this project?

[yusukebe]

@kamaal111

Would it be worth it to contribute this to this project?

I'm also thinking of familiar things. Please wait until my vacation ends.

[yusukebe]

@kamaal111

I'm working on this issue, but I can't find a good solution. Can you share more details of your idea?

[kamaal111]

@yusukebe I have workaround for it in my own project that only requires a Request object with JSON and it looks something like this:

import type { HonoContext } from '../api/contexts.js';

export async function makeNewRequest(c: HonoContext, options?: { withBody?: boolean }): Promise<Request> {
  let body: string | undefined = undefined;
  if (options?.withBody) {
    const rawBody: unknown = await c.req.json();
    body = JSON.stringify(rawBody);
  }
  const requestInit: RequestInit = { method: c.req.method, headers: c.req.header(), body };

  return new Request(c.req.url, requestInit);
}

But for Hono I am thinking something like this:

// ...

// Type safe transformer for request init
const BODY_TYPE_TO_TRANSFORMER_MAPPING: { [K in keyof Body]: (body: unknown) => Body[K] } = {
  json: (body) => JSON.stringify(body),
  text: (body) => body as string,
  arrayBuffer: (body) => body as ArrayBuffer,
  blob: (body) => body as Blob,
  formData: (body) => body as FormData,
}

export class HonoRequest {
    // ...

  new = async (): Promise<Request> => {
    const requestInit: RequestInit = { method: this.method, headers: this.header() }
    const cachedKeys = Object.keys(this.bodyCache) as Array<keyof typeof this.bodyCache>
    // If no body is cached but the raw request has a body and it's not consumed, consume it
    if (cachedKeys.length === 0 && this.raw.body && !this.raw.bodyUsed) {
      const body = await this.raw.text()
      this.bodyCache.text = body
      requestInit.body = body
      return new Request(this.url, requestInit)
    }

    const firstKey = cachedKeys[0] as keyof Body | undefined
    // If no cached body, return request without body
    if (firstKey == null) return new Request(this.url, requestInit)

    const cachedBody = await this.bodyCache[firstKey]
    requestInit.body = BODY_TYPE_TO_TRANSFORMER_MAPPING[firstKey](cachedBody) ?? cachedBody

    return new Request(this.url, requestInit)
  }
}

I haven't tested this in my application yet though, just drafted it quickly, but I will write proper tests for this and also see how it behaves in my app.
Will get back to you after I have validated this change.

[kamaal111]

Just confirming that the following works fine for my project now.

const app = new Hono()

app.post(
  '/login',
  zValidator(
    'json',
    z.object({
      username: z.string(),
    })
  ),
  (c) => {
    const validated = c.req.valid('json');
    //  New Request object constructor method, that is valid even after consumption.
    const request = await c.req.new();
    // `auth` from better-auth for example
    const response = await auth.handler(request);
  }
)

[kamaal111]

I have pushed my updated solution, let me know if its alright

[yusukebe]

Hi @kamaal111

Adding a new() function is a great idea that I also think is similar! And the implementation seems to be fine. But it's not good that the logic/code has increased. The core object, HonoRequest, should be simple and small. We have to consider them more deeply. I'll think it again.

[kamaal111]

I understand @yusukebe
I have another idea, what if .raw was an getter property on HonoRequest?

Excuse me if I am making mistakes writing the following, because I can only write this from my phone for the coming 2 weeks.

class HonoRequest {
  // ...
  // internal raw Request object which was previously `raw`
  private _raw: Request

  get raw() {
    // check if raw has been consumed
    // if it has been consumed then return a new Request object with cached values
    // else return raw in this class
  }
}

Note that we don't need a promise with the getter, because we have 2 simple cases and we don't need to think about forcing a new RequestObject if the raw object has not been consumed.

This should fix the issue of using raw while it is not consumed, but to not cause regressions in performance we likely need to reference the non getter raw prober and the getter raw property would be used for public applications.

What do you think about this solution?

[yusukebe]

@kamaal111

I think adding a raw getter is good, but how to create a new Request object if it has been consumed? The problem is that cached values are Promise. And the raw getter can not be an async (right?).

  get raw() {
    if (this._raw.bodyUsed) {
      // get body from the cached values?
      // but the values in this.bodyCache are Promise
      return new Request(this._raw, {
        ...this._raw,
        body, // how to get this?
      })
    }
    return this._raw
  }

[kamaal111]

@yusukebe you're right I missed that fact.
Looking at how #cachedBody is used in HonoRequest, what if we cache the resolved values instead of caching the body promise?

We will then also have access the body cache directly without the convenience method.

Then we can make a raw property getter work

[yusukebe]

Hey @kamaal111

Nice suggestion. The performance degradation is not big with the solution.

I've created the new PR: #4425

What do you think of it?

[kamaal111]

Hey @yusukebe
The change looks good at a glance, but I can't give a thorough enough review with verification yet, because I am traveling for 1 more week.
I did my best to give a mobile review on the PR

[yusukebe]

@kamaal111

Thanks! I'll check it.

[usualoma]

Sorry to interrupt.

What is raw?

I think c.req.raw is expected to return the “original object”. Returning something newly created with new Request() might not be ideal.

Potential for utility methods

While having too many utility methods has its pros and cons, the scenario of “consuming the body with c.req and then consuming it again with c.req.raw” is relatively rare. Therefore, adding a utility method might be preferable to increasing the HonoRequest code.

Even in the current state, it's not necessarily required to change the HonoRequest code. I believe we could add a utility method with an implementation like the following:

const cloneRawRequest = async (req: HonoRequest) => {
  if (!req.raw.bodyUsed) {
    return req.raw.clone();
  }

  return new Request(req.raw.url, {
    method: req.raw.method,
    headers: req.raw.headers,
    body: await req.blob(),
  })
}

const app = new Hono()

app.post('/', async (c) => {
  const text = await c.req.text()
  const req = await cloneRawRequest(c.req)
  return c.json({
    textViaReq: text,
    textViaCloneRawRequest: await req.text(),
  })
})

[kamaal111]

Thank you @usualoma for your comment!

The main problem I have been facing is that I would like to run validating middleware before I enter my handler, and my handler body would invoke better-auth's handler, as described here:

https://hono.dev/examples/better-auth

I know my use case is quite unique, but my goal is to have the endpoints documented with OpenAPI.

So for this cloning it outside of HonoRequest would not work, because the body has already been consumed before I get access to the raw Request object in my handler.

I think either the solution that @yusukebe presented in his PR would work or the solution in this PR (mine) if we want to keep changing it in Hono.

Alternatively we can also make better-auth work with a "consumed" Request object, but this is a very short term solution as there might be other third party integrations in the future that would like to use the Request object somehow.

[yusukebe]

@usualoma

Interrupting is no problem! Thank you for the idea. Making a helper is nice to keep the HonoRequest clean. I may make it without the problem @kamaal111 mentioned. I'll work on it later.

[yusukebe]

Hi @usualoma

Sorry! Your cloneRawRequest works fine. I think that is expected.

@kamaal111 For example, the following works without errors.

app.post(
  '/',
  validator('json', (data) => {
    return data
  }),
  async (c) => {
    const rawRequest = await cloneRawRequest(c.req)
    auth.handler(rawRequest)
    // ...
  }
)

Is this what you want to have? I think it's a good idea to make a utility function rather than adding logic to HonoRequest.

[kamaal111]

Hi @yusukebe @usualoma,
Sorry of the late response, I just verified in my application that the cloneRawRequest also works for my use-case, thank you 🙏

Is this something that can be added to this repo, or is there another repo better suited for it?
I can also just keep it in my own project

[yusukebe]

Hi @kamaal111

We can include cloneRawReques into this repo! It's good to make a helper for it. Like his:

import { cloneRawRequest } from 'hono/request-helper'

However, I'm unsure if hono/request-helper is the best naming choice. hono/request would be a suitable alternative, but it's already used in src/request.ts. Another idea is hono/clone.

@usualoma Do you have any idea for the naming?

[usualoma]

Well, generally speaking, it seems natural that both the “class” and “class-related utility methods” are exported from "hono/request".

import { HonoRequest, cloneRawRequest } from 'hono/request'

However, I recognize that Hono's existing helpers aren't structured that way. So, as you say, maybe hono/request-helper?

I feel like putting it in hono/request would be fine too.

[kamaal111]

@yusukebe I have updated this PR to export the cloneRawRequest util from /request

[yusukebe]

LGTM!

[yusukebe]

Hey @kamaal111

Let's go with this!

I'll merge this into the next branch for the next minor release v4.10.0 (maybe released soon!). Thank you for the hard work!

[kamaal111]

Awesome thank you for the support 🙏