fs.createReadStream sometimes does not emit data or end

[spthiel]

What version of Bun is running?

1.0.6+969da088f5db3258a803ec186012e30f992829b4

What platform is your computer?

Linux 5.15.0-86-generic x86_64 x86_64

What steps can reproduce the bug?

1. bun add pm2
2. start any pm2
3. bun node_modules/pm2/lib/binaries/CLI.js logs and bun node_modules/pm2/bin/pm2 logs

What is the expected behavior?

I expect both to show all logs of pm2 since bin/pm2 is just require('../lib/binaries/CLI.js')

What do you see instead?

When calling CLI.js directly the createReadStream from node_modules/pm2/lib/API/Log.js never sends any data

Additional information

This seems to be related to the {start: size} argument.
Because following code also does not produce any output with bun but does with node:

const fs = require("fs");
const path = require("path");

const file = path.resolve("test-out.log");

const size = Math.max(0, fs.statSync(file).size - (15 * 200));
console.log(size);

var fd = fs.createReadStream(file, {start: size});
fd.on('data', () => console.log("DATA"));
fd.on('end', () => console.log("END"));

with test-out.log

[Edgar-P-yan]

Hi everybody!
I stumbled upon the same bug when was implementing a JS solution for the 1brc challenge and created a small reproduction https://github.com/Edgar-P-yan/bun-fs-stream-issue-reproduction/blob/master/index.ts.

In short, Bun.file(...).slice(...).stream() does not work, wheras Bun.file(...).slice(...).text() seems to work as expected, and the node compatable fs.createReadStream({ start: ..., end: ...}) hangs if start is greater than 65536 (which is the size of the underlaying buffer I suppose).

Aside from that, first time working with Bun, and i am very happy with it's performance. The JS solution in the single thread mode with Bun works 40% faster then on node.js. I still can't get to make it multithreaded because of this bug, but the speed is already amazing 👍

[wzulfikar]

I'm on bun v1.1.20 and the issue still occur. Noticed it when trying to upload to a Tika server. How to repro:

1. Run tika server (e.g. using docker): docker run -p 127.0.0.1:9998:9998 apache/tika:2.9.2.1-full
2. Run the code to upload a sample file: bun upload-tika.js (the code and the test file are available at the end of the note)

With bun, the fetch code will trigger error in Tika server and returns "Unprocessable entity". With node, the code will work:

The error in Tika server when using bun:

---

• Sample file: test.txt (contains "Hello world")
• upload-tika.js file:

const { createReadStream } = require("node:fs");

(async () => {
  const fileData = createReadStream('./test.txt')
  const response = await fetch('http://localhost:9998/tika', {
    headers: {
      Accept: 'text/plain'
    },
    method: 'PUT',
    duplex: 'half',
    body: fileData
  })
  const text = await response.text();
  console.log('status:', `${response.status} ${response.statusText}`);
  console.log('text:', text || '(empty)');
})();