nodejs formidable not triggering any events

Question

I am on express3.x and using formidable

app.post "/up",(req,res)->
formidable = require('formidable')
form = new formidable.IncomingForm()
form.uploadDir = __dirname + "/uploads"
form.encoding = 'binary'


form.addListener 'file',(name,file) ->
    #write file
    console.log('file uploaded')
    return
console.log('$$$$$$$$$$$')
form.addListener 'end', ->
    console.log('end')
    res.end()
    return
console.log('@@@@@$$$$')
form.parse req,(err,fields,files)->
    console.log('parse')
    console.log(err) if(err)
    return
console.log('######')
return

and the upload form is

block content
:coffeescript
    $ ->
        formData = new FormData()
        xhr = new XMLHttpRequest()
        onProgress = (e)->
            per_complete = (e.loaded/e.total) * 100 if e.lengthComputable

        onReady = (e)->
            alert("Ready")

        onError = (err)->
            alert("Error Loading "+err)

        $('#upload').click (e)->
            formData.append('img',document.getElementById('img').files[0])
            xhr.open('post','/up',true)
            xhr.addEventListener('error',onError,false)
            xhr.addEventListener('progress',onProgress,false)
            xhr.send(formData)
            xhr.addEventListener('readystatechange',onReady,false)

h1 hello world
form
    |select Image: 
    input(type='file',name='img', id='img')
    input(type='button',value='button', id='upload')

none of events are getting triggered...Not sure what am I my missing..

Answer 1

if you use app.use(express.bodyParser()) is equivalent to:

app.use(express.json());
app.use(express.urlencoded());
app.use(express.multipart());

you can remove app.use(express.multipart()); then you can use formidable object.

Answer 2

express is using formidable internally, so your formidable doesn't get any events, if you would like to use formidable on your own you have to remove multipart/form-data from the bodyParser

i am still on express 2.x but that should also do the trick on 3.x, in your app.configure function try

app.configure(function(){
  delete express.bodyParser.parse['multipart/form-data']
})

now you can use formidable on your own.

Answer 3

Is there a specific reason why you are using formidable and not the built in bodyParser? It uses the multipart middleware and is based on formidable. Therefore most of the options from formidable can also be applied to the bodyParser. For example:

app.use(connect.multipart({ uploadDir: path }));

To answer your question, try the following code:

app.js

app.configure(function(){
  app.set('port', process.env.PORT || 3000);
  app.set('views', __dirname + '/views');
  app.set('view engine', 'jade');
  app.use(express.favicon());
  app.use(express.logger('dev'));
  app.use(express.bodyParser());
  app.use(express.methodOverride());
  app.use(app.router);
  app.use(express.static(path.join(__dirname, 'public')));
});

app.post("/", function(req, res) {
  console.log(req.files.img)
  res.end()
})

index.jade

form
  input(type="file", name="img", id="img")
  input(type="button", value="button", id="upload")
script
  var formdata = new FormData()
  var xhr = new XMLHttpRequest()

  $("#upload").on("click", function(e) {            
    xhr.upload.onprogress = function(evt) {
      console.log("While sending and loading data.")
      if (evt.lengthComputable) {
        console.log (evt.loaded / evt.total * 100);
      }
    }

    xhr.onloadend = function(evt) {
      console.log("When the request has completed (either in success or failure).")
    }

    xhr.onload = function(evt) {
      console.log("When the request has successfully completed.")
    }

    var image = document.getElementById('img').files[0]
    formdata.append("img", image)
    xhr.open("POST", "/", true)
    xhr.send(formdata)
  })

Have a look at the W3C Specifications for more events.

Answer 4

I had the same issue in which the file was uploaded completly before My controller got arround to proccessing it. The events were fired but I wasn't listening yet. To counter act this I wrote a middleware to capture file uploads and keep them around for controller to access later.

https://gist.github.com/3813103

you can implement it with

var fileHandler = require('./middleware/fileHandler');
app.use(fileHandler());

Answer 5

Nodejs / Express Setup

// https://www.npmjs.com/package/express-formidable
npm install express-formidable

Quick Demo

const express = require('express');
const formidableMiddleware = require('express-formidable');

const app = express();

// Express-Formidable attaches your multi-part form to the request object
app.use('/formdata/with-fields-and-files', formidableMiddleware(), (req, res) => {
   console.log(req.fields, 'fields')
   console.log(req.files, 'files')
}

app.listen(3000)

Answer 6

Similar issue happened today.
Wasted 4+ hours of time.
Was expecting an answer here, but din't find it here, so writing it.

Problem

1. formidable - fileBegin event is     fired  
2. formidable - file      event is     fired - for small ~3MB/~4MB files
3. formidable - file      event is not fired - for large files

on local machine - nodejs/libraries will work fine
on remote machine - behind nginx it above issue happens

Libraries used:
- micro - https://github.com/zeit/micro - main culprit due to too easy programming model
- formidable - https://www.npmjs.com/package/formidable - smaller culprit does not give example of async/await

Solution

1. increase the VM memory size 2/3 times the file upload size (libraries eat a lot of memory)
2. increase both proxy & http timeout/ size in the nginx to the required
    - example, 10 mins timeout & 2G size
3. change the programming model for large files
    a. on local machine - nodejs/libraries will work fine
    b. but behind the nginx proxy, it optimizes and kills the on going process asap
        to solve this proxy issue, use more promises/ async-awaits, delay the reply to nginx

Hope that helps.