Slow read from blob stream

Version

16.14.0

Platform

Darwin Alans-MacBook-Pro.local 21.1.0 Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:01 PDT 2021; root:xnu-8019.41.5~1/RELEASE_ARM64_T6000 arm64

Subsystem

No response

What steps will reproduce the bug?

import crypto from 'crypto'
import { Blob } from 'buffer'

async function main () {
  const bytes = crypto.randomBytes(1024e6)
  const blob = new Blob([bytes])
  const reader = blob.stream().getReader()
  console.time('read')
  const result = await reader.read()
  console.timeEnd('read')
  console.log(result.value.length, 'bytes')
  reader.cancel()
}

main()

Output:

$ node test.js  
read: 203.812ms
65536 bytes

How often does it reproduce? Is there a required condition?

No response

What is the expected behavior?

No response

What do you see instead?

Why does it take 203ms to read 65kb from an in memory buffer?

blob.slice(0, 65536) takes 0.148ms so maybe some copying is happening?

const buf = await blob.arrayBuffer()
const slice = buf.slice(0, 65536)

Takes 105ms, which is still half the time stream.read() takes.

Additional information

No response

I looked up what's happening behind the scenes:

node/lib/internal/blob.js

Lines 323 to 339 in 9be1dcf

    
           return new lazyReadableStream({ 
        
             async start() { 
        
               this[kState] = await self.arrayBuffer(); 
        
             }, 
        
             pull(controller) { 
        
               if (this[kState].byteLength <= kMaxChunkSize) { 
        
                 controller.enqueue(new Uint8Array(this[kState])); 
        
                 controller.close(); 
        
                 this[kState] = undefined; 
        
               } else { 
        
                 const slice = this[kState].slice(0, kMaxChunkSize); 
        
                 this[kState] = this[kState].slice(kMaxChunkSize); 
        
                 controller.enqueue(new Uint8Array(slice)); 
        
               } 
        
             } 
        
           });

fix: tests in CI (#1034)

chore: disable node native blob (#49)

buffer: improve blob read performance

Navigation Menu

Repository navigation

Slow read from blob stream #42108

Version

Platform

Subsystem

What steps will reproduce the bug?

How often does it reproduce? Is there a required condition?

What is the expected behavior?

What do you see instead?

Additional information

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	return new lazyReadableStream({
	async start() {
	this[kState] = await self.arrayBuffer();
	},

	pull(controller) {
	if (this[kState].byteLength <= kMaxChunkSize) {
	controller.enqueue(new Uint8Array(this[kState]));
	controller.close();
	this[kState] = undefined;
	} else {
	const slice = this[kState].slice(0, kMaxChunkSize);
	this[kState] = this[kState].slice(kMaxChunkSize);
	controller.enqueue(new Uint8Array(slice));
	}
	}
	});

Slow read from blob stream #42108

Description

Version

Platform

Subsystem

What steps will reproduce the bug?

How often does it reproduce? Is there a required condition?

What is the expected behavior?

What do you see instead?

Additional information

Activity

alanshaw commented on Feb 24, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions