Re: File upload progress

Lance wrote: 

 I've been doing a fair bit of work on out of the box dwr support for file upload progress and cancel but wanted to run the new API past the user's list to get some feedback.

That's great!

The basic approach is this:

<javascript>

dwr.engine.setUploadProgressPollInterval(...);

Remote.uploadFile(

   dwr.util.byId("myFile"),

   callback: ...,

   progressCallback: function(percentageComplete) {

      updateSomeWidget(percentageComplete);

   },

   id: "myUpload" // this allows a developer to provide a handle for the upload, dwr generates it's own invocationId to use internally

);

I guess you forgot the object brackets for the options object in your example? With them it would look like this I think:

Remote.uploadFile(
    dwr.util.byId("myFile"),

    {
        callback: ...,
        progressCallback: function(percentageComplete) {
            updateSomeWidget(percentageComplete);
        },
        id: "myUpload"

    }
);

So, to my comments:

 

1) Settings

I can see two new configuration options in the example:

• progressCallback
• uploadProgressPollInterval

I think they should have a consistent naming and both should be available to set in the options object. We might have global defaults for these options (as you suggest with dwr.engine.setUploadProgressPollInterval) but we could also do without that?

dwr.engine.cancelUpload("myUpload");

</javascript>

engine.js generates an invocationId which is passed to the server which can be used to cancel an upload or poll it's current progress.

2) Generic progress and cancellation

As you mentioned in an earlier mail, it could be interesting to provide a generic progress/cancellation mechanism for all calls, not just for file uploads. I then think it would be natural to return an object from any async call following the Deferred/Future pattern. This object could be used for asking about progress, completion status or to trigger cancellation, and would replace (but possibly contain) the invocationId in your example.

If providing progress for all kinds of requests, then we can not depend on always getting a percentage to the progress callback, as we typically don't know the size of outbound data. The parameters could instead be designed this way:

function(transferredBytes, totalBytes)

and the totalBytes would only be set in the cases when we do have that information (and could easily be converted to a percentage). When totalBytes is undefined, a progress dialog could still show the running transferredBytes count.

 

3) Invocation id

After thinking a fair bit on this I'm a little worried about us taking the responsibility to create a unique id in client code. Joe had the same dilemma for the script session ids and decided to go with server-side generation. Also, we already have the scriptSessionId that is such a unique number that identifies a loaded page in the browser.

The invocationId needs to uniquely identify each call to the server and basing off the scriptSessionId we would just have to add a sequential call index number that we can keep locally in the client layer.

What I just recommended doesn't work if we disable script sessions, and this is a configuration I am personally interested in. Though, if disabling script sessions I think it is ok to require the user code to invent a good enough invocationId. So, by default I think we should use the scriptSessionId algorithm, and then allow it to be overridden in the options object.

 

Summing these suggestions up, this is how I would expect them to work:

 

Specifying no progress callback means no progress handling will be done. The returned Deferred object allows us to manually check progress or cancel upload:

var deferredInvocation = Remote.uploadFile(
    dwr.util.byId("myFile"),

    {
        callback: ...

    }
);

dwr.engine.getProgress(deferredInvocation) // sends progress check request

dwr.engine.isCompleted(deferredInvocation) // internal flag, no request needed

dwr.engine.cancel(deferredInvocation)

[Some more thought could be put into these method names]

Specifying a progress callback means progress handling will be active (in this case with scriptSessionId algorithm):

var deferredInvocation = Remote.uploadFile(
    dwr.util.byId("myFile"),

    {
        callback: ...,
        progressCallback: function(...) {},

        progressInterval: 500 // msec

    }
);

And finally, also specifying an invocationId means that we override the scriptSessionId algorithm with our own:

var deferredInvocation = Remote.uploadFile(
    dwr.util.byId("myFile"),

    {
        callback: ...,
        progressCallback: function(...) {},

        progressInterval: 500, // msec

        invocationId: "my Internet-wide unique string"

    }
);

On the serverside there is a (pluggable) UploadManager with the following interface:

public interface UploadManager {

    public void updateProgress(String invocationId, long bytesRead, long contentLength, int currentItemIndex);

    public double getPercentageComplete(String invocationId);

    public void cancelUpload(String invocationId);

}

Nice.

When/if going into a generic progress/cancellation feature, there would also be functionality like this in DWR's normal communication stack and the progress/cancellation calls would at some point branch to either be handled by the file uploader or the DWR stack. I guess we would then like to have them implement the same interface so maybe this interface would look a little different, I'm not sure.

I have implemented 2 upload managers. 1 stores on the session (similar to the current approach).  

If the scriptSessionId suggestion above works out, then I guess the default upload manager should store progress on the script session. A session-oriented upload manager would only be needed if it is desirable to track the progress from a different page than the one doing the upload, but then some kind of "list current uploads on session" function would also be needed.

Another (default) stores to a local map. 

I'm not sure what you mean with "local map"?

There is also a (pluggable) FileUpload (default CommonsFileUpload) that is capable of parsing a multipart request and informing the UploadManager of the current progress for the invocationId.

Nice too :-)

So... my main questions to the group are:

1. How many people use the current file upload progress implementation (storing on session)

2. Does anyone currently support cancel of a file upload

3. Do we agree on the UploadManager approach (you'll need a good argument here!) 

Interface could be affected by "generic support" choice discussed above.

 4. Does anyone have arguments for better naming conventions

  a) dwr.engine.setUploadProgressPollInterval(...)

  b) progressCallback: function(percentageComplete) { ... }

  c) id: "myUpload" 

Replacing id with Deferred return value.

  d) dwr.engine.cancelUpload("myUpload");

  e) invocationId

5. Any other suggestions / improvements? 

I remember from a previous discussion that it was suggested that progress could be delivered by Reverse Ajax, if active on the page. I'm not sure how much that would complicate things.

 

Best regards

Mike