javascript file uploader

[Shelwien]

Here's a simple filehosting service
http://nishi.dreamhosters.com:8888/
Accepting suggestions and bugreports.

[osmanturan]

Hi Shelwien,
It's too long since we talked I was searching a much better uploader solution for our content management system which is based .NET platform. I was using like yours: single file upload with standard HTML element (I also did iframe improvement for customizing uploader interface). Anyway, it begans to demand upload progress, multi-file selection etc. So, I gave a try for SWFUpload 2 weeks ago. And I realized how an uploader can be buggy like that (not to mention how go nuts before finding a working solution for lost session data on uploads!!!). Anyway, I finally found a much better tool: Plupload. It's free and comes with some PHP examples. You can use Google Gears, Silverlight, Flash, Yahoo Browser Plus, HTML 5 and of course HTML 4 to upload files. So, it almost guarantees a client uses most advanced upload method which his browser supports. Maybe you want to give a try for multi-file upload.

A side note: I was thinking to add client side compression Do you know that a pure javascript library exists which applies deflate compression/decompression?

[Shelwien]

> It's too long since we talked.

Yeah. I guess having a job makes you asocial :)

> I also did iframe improvement for customizing uploader interface

I don't get this, but if you have a method to simulate input.type=file
without making the ugly default one visible, then please tell.
I tried an idea with calling input's .click() and it worked well
in chrome/safari, but not anywhere else.
Then there's the common zero opacity/button overlay idea, but I don't
want to use anything that troublesome.

> Anyway, it begans to demand upload progress, multi-file selection etc.

I just written a standalone web-service in C++... so it already shows
the upload progress and accepts large files, and supporting multiple
files is a matter of some more javascript.
I don't have much experience with html/css/javascript though, so that
side could use some improvements, but the basic approach seems okay to me...
there's only 17k of source in the whole thing, and it works.

> So, I gave a try for SWFUpload 2 weeks ago. And I realized
> how an uploader can be buggy like that

Yeah... that's why I'm trying to avoid complicated composite solutions,
like writing a specific handler for each browser, or using more
different languages.

> Anyway, I finally found a much better tool: Plupload.
[...]
> Maybe you want to give a try for multi-file upload.

Uh, thanks for the link, but I'd implement it myself,
to avoid going crazy :)

> A side note: I was thinking to add client side compression
> Do you know that a pure javascript library exists which
> applies deflate compression/decompression?

I'm not sure if its a good idea. Sure javascript is a good
language and you can even implement a paq model in it if you want :)
But what about file access and speed, especially in older browsers?
Also, I'd use BWT there, if anything (well, ST4 maybe).

[osmanturan]

> It's too long since we talked.

Yeah. I guess having a job makes you asocial

Well...I guess, you are right But, seems you have forgotten x2 jobs + master education

> I also did iframe improvement for customizing uploader interface

I don't get this, but if you have a method to simulate input.type=file
without making the ugly default one visible, then please tell.
I tried an idea with calling input's .click() and it worked well
in chrome/safari, but not anywhere else.
Then there's the common zero opacity/button overlay idea, but I don't
want to use anything that troublesome.

Well...At first try, I've used ugly interface. Then I used iframe method which is a bit tricky. Actually, it's not that complicated. Instead of sending whole page, i have used iframe and placed input-file in it. And at host page, I've linked a button with it via javascript. And also another button for iframe submission. So, for user point-of-view, user clicks a good-looking button then selects a file and then clicks another button to start upload. after file uploaded (btw, page is not reloaded during this operation), the user is confirmed about the stats.

Zero-opacity method is the most used method so far for HTML4. Because you have no choice to start local file interaction via javascript. It's kind of so-called security restriction.

> Anyway, it begans to demand upload progress, multi-file selection etc.

I just written a standalone web-service in C++... so it already shows
the upload progress and accepts large files, and supporting multiple
files is a matter of some more javascript.
I don't have much experience with html/css/javascript though, so that
side could use some improvements, but the basic approach seems okay to me...
there's only 17k of source in the whole thing, and it works.

Seems you have requesting another url for updating progress of the file. This is well used method and works ok for HTML4. Most of people at .NET side, writes a http module to report progress of a file via json. and frequently, they queries the file which have been uploading via javascript. It works well under most circumstances. But, it lacks of multi-file select. So, you have to repress a button every time to send -say 10- files. Look at one of the most advanced thing which is using this method: http://www.telerik.com/products/aspnet-ajax/upload.aspx Note that, this company has very well-known customers: such as Coca-Cola, BMW and even Microsoft.

> So, I gave a try for SWFUpload 2 weeks ago. And I realized
> how an uploader can be buggy like that

Yeah... that's why I'm trying to avoid complicated composite solutions,
like writing a specific handler for each browser, or using more
different languages.

Composite solutions are just work-arounds. Via flash hack, you can show progress of current upload without querying any url. Also, not to mentioned about multi-select file support. So, you can send more than one file at once. The best solution so far seems Google Gears. It even support drag-and-drop. And HTML5 also offers very good things. But, the best solution seems "use whichever is available".

> Anyway, I finally found a much better tool: Plupload.
[...]
> Maybe you want to give a try for multi-file upload.

Uh, thanks for the link, but I'd implement it myself,
to avoid going crazy

It would be really troublesome Really. I believe keeping cross-browser compability is one of the most hardest programming task in the universe I'm continuing to integrate to our CMS. It could take sometime because I've changed some internals prior of this task.

> A side note: I was thinking to add client side compression
> Do you know that a pure javascript library exists which
> applies deflate compression/decompression?

I'm not sure if its a good idea. Sure javascript is a good
language and you can even implement a paq model in it if you want
But what about file access and speed, especially in older browsers?
Also, I'd use BWT there, if anything (well, ST4 maybe).

As you guess, it's dead slow (speed king currently Chrome - it uses JIT for javascript). And it's designed to deliver data in Base64 via ajax callbacks then decompress and replace some stuff in the page. But, guess what, it's sometimes much bigger than the original data because of Base64 coding %)

If we could access local files directly via javascript, we could implement something neat. But, it's even impossible with brand-new HTML5 (well, only webkit dissallow to access raw file data). So, client side compression is only useful when you are directly talking with your server. And because of base64 coding it's again troublesome. Maybe you can give a try your modified MARS (?) approach to expand coding alphabet

BTW, Yahoo did something quite new (of course if we could omit plugin part), look at this client side compression via LZMA
http://browserplus.org/demos/ (Look at Bit Squeezr).

[osmanturan]

(Becomes out of topic... Sorry)
Here is some interesting javascript libraries for doing compression/decompression.

- Very clean deflate decompressor written in javascript
http://www.codeproject.com/KB/script...binaryenc.aspx (I've also attached for avoding login to CodeProject for just download).

- Zlib clone (the code looks ugly) of deflate compressor / decompressor
http://github.com/dankogai/js-deflate

- LZW compressor/decompressor implementation among Adobe's Base85 coding
http://jsbin.com/egemu3/edit

- Very limited PNG reader with deflate decompression
http://blog.calyptus.eu/seb/2009/05/...in-javascript/

Seems your "alphabet restricted" coding technique is a must for taking advantage of client-side compression Where was the link?

[Shelwien]

> Well...At first try, I've used ugly interface. Then I used
> iframe method which is a bit tricky. Actually, it's not
> that complicated. Instead of sending whole page, i have
> used iframe and placed input-file in it.

As to that, I did something (hopefully) interesting in
my version. There're no frames as you can see, but it
still remains on the same page after sending (while
normally browser would move to the page returned by POST).
I did it like this: server-side upload handler doesn't
close the connection after receiving the file and just waits.
While XHR handler signals to browser that file is received,
and the script there "stops" the page, thus terminating
the upload. So whole file is uploaded via a common
html form, but we stay on the same page.

> And at host page,
> I've linked a button with it via javascript. And also
> another button for iframe submission. So, for user
> point-of-view, user clicks a good-looking button then
> selects a file and then clicks another button to start
> upload. after file uploaded (btw, page is not reloaded
> during this operation), the user is confirmed about the
> stats.

I still don't understand whether there's any relation
between "good-looking button" and iframe.
Does that "good-looking button" work by zero-opacity trick,
or is there a portable way to customize file selection
via iframe?

> Zero-opacity method is the most used method so far for
> HTML4. Because you have no choice to start local file
> interaction via javascript. It's kind of so-called
> security restriction.

I know, but fileinput element has a different layout
in different browsers. Webkit browsers have the button
to the left, and others to the right of filename field.
So is there an easy way to determine the specific position
and size of fileinput button? I really don't want to
use much browser-specific code there...

> Most of people at .NET side, writes a http module to
> report progress of a file via json. and frequently, they
> queries the file which have been uploading via javascript.

In my case, I made a custom webserver with explicit support
for that feature. I think it would be much lighter than
common solutions - there's no dynamic memory allocation and
no file access (except for uploaded data).

> It works well under most circumstances. But, it lacks of
> multi-file select. So, you have to repress a button every
> time to send -say 10- files.

Well, the requirement to press the button is due to that
security restriction, but its only a matter of javascript
for me to support uploading multiple files in parallel.
For example, I can dynamically add iframes with the same form.

> Composite solutions are just work-arounds. Via flash hack,
> you can show progress of current upload without querying
> any url.

Yeah, but fortunately there's another effect from receiving
that info from server. There the amount of actual _received_
data is known, while browser only knows how much it sent,
while actual data might be still in proxy buffers or somewhere,
and would never reach the server due to some error.

> > Uh, thanks for the link, but I'd implement it myself
>
> It would be really troublesome Really. I believe keeping
> cross-browser compability is one of the most hardest
> programming task in the universe I'm continuing to
> integrate to our CMS. It could take sometime because I've
> changed some internals prior of this task.

I only talked about selecting and sending more files, while
previous files are still uploading.
Do you mean that, or selecting multiple files with a single
dialog?

> Seems your "alphabet restricted" coding technique is a
> must for taking advantage of client-side compression
> Where was the link?

Started a new thread about that.

[osmanturan]

> Well...At first try, I've used ugly interface. Then I used
> iframe method which is a bit tricky. Actually, it's not
> that complicated. Instead of sending whole page, i have
> used iframe and placed input-file in it.

As to that, I did something (hopefully) interesting in
my version. There're no frames as you can see, but it
still remains on the same page after sending (while
normally browser would move to the page returned by POST).
I did it like this: server-side upload handler doesn't
close the connection after receiving the file and just waits.
While XHR handler signals to browser that file is received,
and the script there "stops" the page, thus terminating
the upload. So whole file is uploaded via a common
html form, but we stay on the same page.

Yeah. I've looked at your code. Very tiny and effective. But, it still stops you to interact with page (page looks like loading while all uploading progress). But, in my case, the user still can interact with the page via ajax requests (unless he didn't jump another page).

> And at host page,
> I've linked a button with it via javascript. And also
> another button for iframe submission. So, for user
> point-of-view, user clicks a good-looking button then
> selects a file and then clicks another button to start
> upload. after file uploaded (btw, page is not reloaded
> during this operation), the user is confirmed about the
> stats.

I still don't understand whether there's any relation
between "good-looking button" and iframe.
Does that "good-looking button" work by zero-opacity trick,
or is there a portable way to customize file selection
via iframe?

In short, you can access an element which is located in iframe from host page via javascript. by using iframe, input-file is hidden + page still remains for interaction (unless jump another page...). As I said before, in host page there are two buttons: 'Select File' and 'Start Upload'. And there is a hidden iframe which include a input-file element. When you click 'Select File', it fires input-file element via javascript over iframe and pops up the dialog. after selecting file, again via javascript host page is informed and user can submit iframe content via clicking 'Start Upload'. And this is again done with javascript over iframe.

> Zero-opacity method is the most used method so far for
> HTML4. Because you have no choice to start local file
> interaction via javascript. It's kind of so-called
> security restriction.

I know, but fileinput element has a different layout
in different browsers. Webkit browsers have the button
to the left, and others to the right of filename field.
So is there an easy way to determine the specific position
and size of fileinput button? I really don't want to
use much browser-specific code there...

There is no way to 'stylize' the file-input (even with CSS or javascript). That's why most people are using zero-opacity method for pure HTML4 solution.

> Most of people at .NET side, writes a http module to
> report progress of a file via json. and frequently, they
> queries the file which have been uploading via javascript.

In my case, I made a custom webserver with explicit support
for that feature. I think it would be much lighter than
common solutions - there's no dynamic memory allocation and
no file access (except for uploaded data).

That's not a case for most people (especially including me). Because, hosting companies usually have strict rules (I haven't seen any company which allow to run an executable). For example, some hosting companies restrict even some .NET features such as reflection which I use frequently for accessing runtime type information and manipulation.

> It works well under most circumstances. But, it lacks of
> multi-file select. So, you have to repress a button every
> time to send -say 10- files.

Well, the requirement to press the button is due to that
security restriction, but its only a matter of javascript
for me to support uploading multiple files in parallel.
For example, I can dynamically add iframes with the same form.

I think, we should make it clear something. There are two way to upload multiple files at once (with single post). If you want to stick HTML4 solutions, you can dynamically create a input-file element and force the user to select another file at a time (which is very common method. i.e. Telerik RadUpload uses same thing). But, if you want to select multiple files with single dialog, you have to use composite solutions such as flash, silverlight or gears. There is no any other way.

> Composite solutions are just work-arounds. Via flash hack,
> you can show progress of current upload without querying
> any url.

Yeah, but fortunately there's another effect from receiving
that info from server. There the amount of actual _received_
data is known, while browser only knows how much it sent,
while actual data might be still in proxy buffers or somewhere,
and would never reach the server due to some error.

Yes. You are right. That's a problem. But, it has a single benefit over your method. It eliminates overhead for querying progress. But, again you are right, it's not much reliable.

> > Uh, thanks for the link, but I'd implement it myself
>
> It would be really troublesome Really. I believe keeping
> cross-browser compability is one of the most hardest
> programming task in the universe I'm continuing to
> integrate to our CMS. It could take sometime because I've
> changed some internals prior of this task.

I only talked about selecting and sending more files, while
previous files are still uploading.
Do you mean that, or selecting multiple files with a single
dialog?

With multiple "file-input in iframe"s, it's possible to upload multiple files while the others are already uploading. My problem is selecting multiple files with a single dialog. Because, people are lazy. And they have dozens of photos which are supposed to be shown in their pages. So, "multiple files with a single dialog" is a must in my case. Prior of this, I even think about to allow ZIP uploads for multiple files upload at once (server would decompress ZIP content after a successful upload).

> Seems your "alphabet restricted" coding technique is a
> must for taking advantage of client-side compression
> Where was the link?

Started a new thread about that.

Thanks!

[Shelwien]

> But, it still stops you to interact with page (page looks
> like loading while all uploading progress).

Erm, the upload progress doesn't work for you or what?
I don't think that there's a problem with page interaction,
and as to page loading... well, that's what it actually does.
Again, I can probably fix that by putting the form into a frame.

> In short, you can access an element which is located in
> iframe from host page via javascript. by using iframe,

Yeah, and I still don't understand how it's supposed to work
I think I already mentioned that I had a previous implementation
which worked kinda like what you describe - the button with
handler calling click() for fileinput.
But that didn't work in most browsers.
Maybe you can post a sample?

> That's not a case for most people (especially including
> me). Because, hosting companies usually have strict rules
> (I haven't seen any company which allow to run an
> executable).

Dreamhost does allow that.
Also you'd need a dedicated server (or at least some cloud VMs)
for any site with high load anyway.

Although there is a chance that I'd have to modify my
service to immediately reupload the received files
somewhere else.

> But, if you want to select multiple files with single
> dialog, you have to use composite solutions such as flash,
> silverlight or gears. There is no any other way.

Sure, and I think that's ok.
Dunno about others, but I surely don't want people trying
to select all their files and send, for a test

> It eliminates overhead for querying progress.

In fact atm I even have more overhead than necessary there...
it opens a new connection for each XHR request... probably
because the service returns HTTP/1.0 in status.
But anyway my extra kb per second is nothing comparing to
usual solutions which load a megabyte of javascript in advance

> I even think about to allow ZIP uploads for multiple files
> upload at once (server would decompress ZIP content after
> a successful upload).

Yeah, I understand, but I'd leave flash etc to somebody else
I surely can adapt my service to any kind of frontend.
But the original idea in my case was that an archiver would
be the main frontend, so I don't think that I'd have to care
about multiple file selection at all.