All Questions
1,954
questions
0
votes
0
answers
67
views
NodeJS ECONNRESET error onStreamRead while downloading large csv
I'm trying to download a very large gzipped csv file, hosted on a S3 server. I want to insert each csv row to my database. To achieve that I get the encoded stream, unzip it, parse it and save it to ...
0
votes
2
answers
70
views
Array not resetting in JavaScript while reading from stream
I am trying to insert data coming from a remote CSV file in MongoDB (using Mongoose).
I would like to do so in bulk inserts of 100 items at a time.
Here's my code:
import csv from 'csv-parser'
import ...
- 565
0
votes
1
answer
20
views
In node, is a readable file stream just a buffered, asynchronous read "under the hood"?
I'm wondering, if I were to dive into the stream code, would I find deep down that for a readable file stream what is basically happening is that data is being asynchronously read into a buffer which ...
0
votes
0
answers
21
views
How can I resolve the out-of-memory error in aws lambda with lots of fetching data from Mongo that I don't want to increase the memory?
I have lambda on aws that fetches data from opensearch and saves it on s3 bucket data most of the time with small data, but I have users with big data, and then the lambda fails. Is it good to fetch ...
2
votes
1
answer
172
views
Re-streaming of ESP32 JPEG video stream
I'm building DIY home surviellance system with 30 ESP32 AI Thinker cams. To join and manage all of them I create a web-application of NodeJS - a dashboard to view all cams, add/edit/delete them. Each ...
1
vote
0
answers
62
views
Variables from buffer in nodejs
I am studying pure nodejs, no express, no fastify and other frameworks that's why I don't need npm mutler and file-upload.
I am sending 2-3 mp3 files and two txt files from a client side to a nodejs ...
0
votes
1
answer
33
views
Node.js await .pipe()
I'm trying to read a lot of files and combine the information from them into one. Here is the implementation that I got. I want to rewrite it to .pipe(), but it only turns out to overwrite the file at ...
1
vote
1
answer
49
views
Express.Multer.File to ReadStream
I want to upload my video but the wrapper for Bunny.Net (https://github.com/dan-online/bunnycdn-stream/tree/main/src) in createAndUploadVideo method requires the ReadStream type while Express.Multer....
- 67
0
votes
0
answers
37
views
How to properly interrupt a Readable stream before end using async iterators
Node.js v20
I have a Readable stream obtained from a call to a database. I am creating an async iterator from this stream, but I don't want to iterate all the data. If a certain condition is true, I ...
- 4,567
0
votes
1
answer
29
views
In NodeJS data is not receiving as chunks (stream) in consumer
I having a two services producer and consumer.
Producer has big json file in the server. I want to serve over the network through rest api and I used nodejs stream technique to load bytes in memory ...
0
votes
0
answers
14
views
how to handle the return value of the write() of stream in nodejs
I have a question about the below Node.js documents.
First,as far as I tested, stream writer can controll the draining of it's buffer automatically based on the return value of write() and there is no ...
- 11
0
votes
1
answer
36
views
Incrementally upload zip data from memory to a single file on AWS S3 bucket using Node.js streams
I have a ReadableStream of uncompressed text data that I need to store on a single zip-compressed file on S3.
What I don't want to do:
Load all the data into memory.
Write the data on local disk.
...
- 4,567
0
votes
0
answers
34
views
Streamming file with @Command() in Nestjs
I'm using **@Command()** to **stream files** into the **gg cloud**. If you call the API to stream files, it works fine; the cloud contains files. But if I run a command, after the command is done, ...
- 1
0
votes
0
answers
46
views
Why's my lambda function exiting before stream finishes?
I have a lambda function which is reading from an S3 file triggered from an S3 ObjectCreate event. The file contains about 80,000 rows of csv which I'm trying to save to DynamoDB using batchWrite. It ...
- 2,666
0
votes
1
answer
56
views
Navigator not defined
I am working on a streaming project using Reactjs and Nodejs. I use peerjs and Webrtc for the streaming function and constantly having issue with: Navigator is not defined.
I am not sure how to fix it....
