Form Serialization

By Garrett Smith, August 25, 2007

Problem

No standard, reliable way to get FORM data to an Ajax request.

Use Case

A web site has a form containing multiple form fields.
This data needs to be sent to the server to be verified.
However, the author does not want to affect history state.
After submitting the form, a message appears on the page displaying customized details from the server.
From the user's perspective, he is still on the same page.

Current Usage

The need for Form Serialization has manifested itself strongly in all of today's popular Ajax libraries, however:

  1. nearly all of them get it wrong.
  2. none of them serialize a file

The need for File serialization is formally recognized, is being standardized in the File and FileList specifications [File Upload], and is implemented in Mozilla Firefox [bug 371432]. File serialization now makes it possible to send a file using XMLHttpRequest, although the functionality has not gotten mainstream acceptance (too early).

Most JavaScript libraries will create and then target a hidden IFRAME when a FORM to be serialized for an Ajax request contains a file input. This has some drawbacks, as we will see.

Lets examine some other possibilities.

The current (partial) technological solutions and possible new solutions:

  1. serialize the form by hand and send the data in an XMLHttpRequest
  2. target an IFRAME
  3. use standard form submission (not Ajax)
  4. Support native Form Serialization methods (new proposal)
  5. XMLHttpRequest method setForm( ) (contrasting new idea)
  6. add new features to IFRAME (idea to solve problems with hidden iframe loader/proxy)
  7. ignore this case

Proposal

Add to HTMLFormElement interface, the following methods:

  toJSONString
  getDataSetString

Both methods will return a serialized representation of the FORM's successful controls.

Background:

Many Ajax apps use forms to provide a UI to model data. There are some benefits to this approach:

The current (partial) technological solutions and possible new solutions:

Approach #1: Serialize the form by hand, using JavaScript and use an XMLHttpRequest

* Successful attempts by some libraries to get coordinates off an image input include the following strategy:

YUI and jQuery do all this just to get the seldom used coordinates off an image submit, and the MUST do this in order to conform to the HTML FORM specification.

Conclusion: FORM Serialization by hand is not completely possible. What is possible to do is complicated and authors often make honest mistakes in attempts to serialize the HTML FORM's successful controls.

Approach #2: Target a hidden IFRAME

Conclusion: Using an IFRAME to accommodate this use case will work, but has certain implications that make it less desirable.

Approach #3: Use standard FORM submission

Conclusion: Does not correctly satisfy use case.

Support native FORM Serialization (new proposal):

  toJSONString
  getDataSetString

File serialization has been recently recognized, is being standardized [File Upload], and is implemented in Mozilla Firefox [bug 371432]

The HTMLFormElement interface

interface HTMLFormElement {
  string getDataSetString(); raises FileException
  string toJSONString(); raises FileException
};

toJSONString returns a JSON string with keys as names of successful controls and values as an array containing values of their associated key. For successful conrols that have the same name (such as OPTION or checkbox), the value is an array, so to keep the API consistent, we use an array for every value, even when the array contains only one item.

getDataSetString returns a URI string defined by HTML FORM specification.

Contrast Web Forms 2.0 data:

Widespread. JSON and URI strings are more commonly used.

Different use case. The use case for WF 2.0 data: action is debugging. The use case for Form Serialization is submitting form data to an XHR.

This might be more suitable to have the JSON separated from the data serialization.

—Asbj¿rn Ulsberg
Pros and cons of each sub-feature:
  1. getDataSetString
    • pro: most common use case to submit a serialized form
    • pro: Data Set strings are standardized [FORM] and supported in all server side APIs
    • con: adds a new method to API, permanently changing the API
    • con: URI encoding greatly increases file size.
  2. toJSONString
    • pro: wide support in a all server-side languages for unmarshalling JSON objects.
    • pro: useful for storage in a browser-side cache — a common idiom
    • pro: could also be useful for offline storage [Use Cases]
    • pro: toJSONString is already going to be part of Object.prototype in ES4. This is planned to be supported in Firefox 3 and IE8, via a plugin distributed with the browser, but may be included in Mozilla sooner [Bug 340987].
    • pro: easier to read and parse than URIs
    • pro: does not need URI Encoding/Decoding (unless sent as GET)
    • con: seeding a FORM's data is specified in many formats, but not in JSON format. This proposal begs a change to form data: seeding, as JSON would not be round-trip, but only a one way API (we (need a way to set form data with a JSON object using the data: scheme in Web Forms 2.0).

Note: There is reasonable speculation in the JSONRequest whitepaper that JSON will be used to transfer data between sites and will be more widely used in the future JSONRequest

Conclusion: Can be used to satisfy use case.

XMLHttpRequest method setForm( form ) (contrasting new proposal)

Add new features to IFRAME to make it act like an XMLHttpRequest

Add the ability to get Progress events 1.0, cancel loads, and prevent rendering on the window object.

Modify the HTML specification to state that IFRAME can also be used as a transport mechanism.

HTMLIframeElement {
  attribute useHistory boolean
  attribute render boolean
  attribute response, object
useHistory boolean, default value is true
when false, the history will not be included in IFRAME loads.
render, boolean, default value is true
when false, the IFRAME does not render its contentDocument
response, object,
response entity body as described in XMLHttpRequest.

Conclusion: Can be used to satisfy use case.

Ignore this case

This is not really an option, as current web applications make heavy use of this with existing technological methodologies mentioned earlier.

Conclusion

References:

File Upload
File Upload, Robin Berjon, October 18, 2006
bug 371432
Need an API to store <input type=file> data offline and access when online, by Mark Finkle (reporter), Feb 2007, FIXED by Jonas Sicking
WCAG
HTML Techniques for Web Content Accessibility Guidelines 1.0, Wendy Chisholm, Gregg Vanderheiden, Ian Jacobs, November 6, 2000
xml-http-performance-and-caching
XML HTTP Performance and Caching, Emil A Eklund, November 2006
Form Serialization comparison
JavaScript Form Serialization Comparison: jQuery • dojo • Prototype • YUI • MochiKit, Mike Alsup
Progress events
Progress events 1.0, Charles McCathieNevile, July 31 2007
JSONRequest
JSONRequest, Douglas Crockford, November 2006
Use Cases
Comment #6: Use Cases for File Serialization, by Vladimir Vukicevic (:vlad), 2007
FRAMES
Introduction to frames, Dave Raggett Arnaud Le Hors, Ian Jacobs, December 24, 1999
FORMS
Forms in HTML Documents: Form Submission, Dave Raggett Arnaud Le Hors, Ian Jacobs, December 24, 1999
XHR
W3C: The XMLHttpRequest Object, Anne van Kesteren, June 18, 2007
Bug 340987
Bugzilla: Implement JSON extensions to the JS language., Erik Arvidsson
Web Forms 2.0
Web Forms 2.0, Ian Hickson, October 12, 2006
JSON
Introducing JSON

Acknowledgements

I would like to thank: