8. Server-Side Communications

Learning Objectives

By the end of the session you should:

• know how to use XMLHttpRequest calls to access the server
• be able to encode arbitrary Javascript objects into strings in JSON format
• be able to use the UCLServer scripts for storing and retrieving experimental data
• be able to use the Simple Statistics library for analyzing experimental data

Topics

Server-side requests

We tend to think of web pages as static HTML-coded information delivered from a web server. However with Javascript, web applications can act as fully-fledged network applications, retrieving data from the server and sending data back to the server while they operate. This communication with the server can be performed without the current page being replaced or reloaded.

At the heart of communication with the server is a Javascript object with the awkward name of XMLHttpRequest. Methods of this object allow you to request the server to supply some data to your application, or to request the server to take some data and store it at the server end.

Note: for security, XMLHttpRequest can only communicate with the web server that was the original source of the web application. This also means that you can only use XMLHttpRequest calls when the page has been delivered by a server - it does not work for applications running from local file storage, for example.

The use of XMLHttpRequest follows these stages:

1. Create a new XMLHttpRequest object
2. Send a request for a resource to the server
3. Wait for the request to be completed
4. Check that the request was satisfied
5. Process any data that was returned from the server

In this code example (taken from demo8-1.html), we show how XMLHttpRequest can be used to obtain a file of plain text from the server and display it in a textarea element:

function button1click() {
   var req=new XMLHttpRequest();
   req.open("GET","mydata.txt");
   req.onreadystatechange=function () {
      if ((req.readyState==4)&&(req.status==200))
         processData(req.responseText);
   };
   req.send();
}
function processData(txt) {
   document.getElementById("response").innerHTML=txt;
}
...
<button onclick="button1click()">Get Text</button>
<textarea id="response" rows="25" cols="80">
</textarea>

In this example you should see how we create a new XMLHttpRequest object, call its open() method with the name of the resource on the server, specify a function which is to be called when the request is processed, and then send the request. As the request is processed, the onreadystatechange function will be called multiple times. When and only when the readyState property has the value 4 and the status property has the value 200 will the request have completed satisfactorily. In this situation we send the response text to our own function to display it.

In the following example (taken from demo8-2.html) we upload information to the server. Let's assume there is a server application written in PHP that accepts the following pieces of data and stores them:

Name	Description
subject	Some identifying code for a subject
score	Some number representing the subject's performance

The way in which such a script would be called from a browser would be to create a URL formatted like this:

http://host/myapp/storeinfo.php?subject=23&score=75

Here "storeinfo.php" is the PHP server-side script. The '?' separates the script parameters from its name, and the parameters are supplied as "name=value" pairs separated by '&'.

We can achieve the same result from within a web application using XMLHttpRequest. We point the request object at the server-side script and encode the parameters in a similar way:

function saveResult(subject,score) {
   var req=new XMLHttpRequest();
   req.open("POST","storeinfo.php");
   req.setRequestHeader("Content-Type",
                     "application/x-www-form-urlencoded");
   var msg="subject="+subject+"&";
   msg += "score="+score;
   req.send(msg);
}

In this example we use the POST method to send the data values to the server side script. We tell the XMLHttpRequest object that these values will be encoded in the standard format used for web forms. We then create a text string in which the parameter-value pairs are separated by "&". Note that in this example we do not have any function that is called when the process runs, so we cannot check whether the request completed satisfactorily.

One final note: when sending information by the POST method, you must ensure that illegal characters are "escaped"; for example parameter values may not contain spaces, quotes or the ampersand character. If there is a risk that your parameter values contain illegal characters you should first encode the strings using the JavaScript function encodeURIComponent, for example:

   var msg="subject="+encodeURIComponent(subject)+"&";
   msg += "score="+encodeURIComponent(score);
   req.send(msg);

JSON coding

The kind of information we want to exchange with the server may be of various or mixed types. For example we may want to store and retrieve numbers, strings, booleans, arrays or structured objects. However the XMLHttpRequest mechanism works best when everything communicated to the server is coded as strings of characters.

Fortunately, Javascript provides two functions which will encode an arbitrary Javascript expression into a string value, and decode the resulting string value back to a copy of the original data.

The encoding and decoding routines are based on a string formatting notation called 'Javascript Object Notation' or JSON.

To encode a Javascript object or array into a JSON string, call the global function JSON.stringify(). To decode a JSON string back to an object, call JSON.parse(). For example:

var obj={ x:1, y:[ false, "hello" ] };
var jstr=JSON.stringify(obj);
var newobj=JSON.parse(jstr);

Here, jstr would contain "{x:1,y:[false,'hello']}", and newobj would contain a deep copy of obj.

You can use the JSON functions to convert any complex data into a string for storage server-side, and to convert a string returned from the server back into a Javascript object.

Saving and Retrieving results on the UCL server

For the purpose of constructing web experiments on this course, two server-side scripts have been written to facilitate the saving and retrieving of experimental data. In addition, two JavaScript functions (UCLSave() and UCLLoad()) have been written that package the XMLHttpRequest calls needed to send and load data.

To save information to the server, the UCLSave() function has this signature:

UCLSave(user,expt,subj,cond,stim,resp,callback)

Parameter	Description
user	A string identifying the student
expt	A string identifying the experiment
subj	A string identifying the subject
cond	A string identifying the condition
stim	A string identifying the stimulus
resp	A javascript object holding the response
callback	The name of a function to be called when the operation completes

The fields user, expt, subj, cond and stim must be supplied as (non-empty) text strings which identify a particular piece of data. The resp field is an arbitrary JavaScript object, where you can store any information. This will be encoded using JSON.stringify() before it is put in the database.

Here is an example call (adapted from demo8-3.html):

<script src="include/uclserver.js"></script>
<script>
function report(val) {
  console.log("UCLSave() returns " + val);
}
function recordResponse(rtype) {
   var rt=performance.now()-stimtime;
   UCLSave("Mark","Demonstration","1",stimtab[stimpos].condition,
      stimtab[stimpos].stimulus,{ "rtype":rtype, "rt":rt },report);
}
</script>

In this example, we assume that the current condition and stimulus are stored in some stimulus table (stimtab) at some position (stimpos). Also that we recorded the value of the performance timer when the stimulus was presented (stimtime). When the subject responds, the recordResponse function is called and stores away the type of response and the reaction time in the database. Here, the callback function is only used to put a debug message on the console.

To retrieve information from the database into your own program use the function UCLLoad() which has this signature:

UCLLoad(user,expt,subj,cond,stim,callback)

Parameter	Description
user	A string identifying the student
expt	A string identifying the experiment
subj	A string identifying the subject, or "" for all subjects
cond	A string identifying the condition, or "" for all conditions
stim	A string identifying the stimulus, or "" for all stimuli
callback	The name of a function to be called with the array of responses

The function queries the database for the specified experimental results and passes any matching values to the callback function as an array. Each row of the returned array is a structure with fields: id, date, user, expt, subj, cond, stim and resp. The resp property will be the output of JSON.parse applied to the string data retrieved from the database (i.e. it should match the object structure you sent in the resp parameter of UCLSave). Note that you must supply both the user and the expt fields in the query (i.e. you cannot query across users or across experiments).

Here is an example call:

<script src="include/uclserver.js"></script>
<script>
function process(data) {
   var txt="";
   if (data==null) {
      txt = "Request failed";
   }
   else {
      var sum=0,cnt=0;
      for (var i=0;i<data.length;i++) {
         sum = sum + data[i].resp.rt;
         cnt++;
      }
      txt = "Mean reaction time = " + (sum/cnt).ToFixed(1) + "ms";
   }
   document.getElementById("performance").innerHTML=txt;
}
function button2click() {
  UCLLoad("Mark","Demonstration","","1","",process);
}
</script>

In this example the UCLLoad() function is called to return all results from all subjects for condition 1 in Mark's Demonstration experiment and to call the function process with the returned data. Within the process function, the values of the rt property of the resp data field are summed and the mean value calculated and displayed.

A generic UCLoad script exists which you can use to examine what information has been saved on the server. Visit:

http://www.phon.ucl.ac.uk/courses/spsci/webprog/UCLTest.html

Simple Statistics

After we have obtained an array of subject responses back from the server we may want to calculate some statistics, such as a mean, standard deviation, inter-quartile range or t-value.

There is a freely-available library called Simple Statistics that you can use for this purpose.

Here are some of the methods available in Simple Statistics:

Method	Example	Description
sum	ss.sum(x)	Sum of a single-dimensional array of numbers.
mean	ss.mean(x)	Mean of a single-dimensional array of numbers.
mode	ss.mode(x)	Returns the number that appears most frequently in a single-dimensional array of numbers. If there are multiple modes, the one that appears last is returned.
median	ss.median(x)	Median of a single-dimensional array of numbers.
min	ss.min(x)	Finds the minimum of a single-dimensional array of numbers.
max	ss.max(x)	Finds the maximum of a single-dimensional array of numbers.
quantile	ss.quantile(sample, p)	Calculates one or more quantiles of a single-dimensional array of numbers. p must be a fraction between 0 and 1 or an array of fractions. If an array is given, an array of results will be returned instead of a single number.
variance	ss.variance(x)	Variance of a single-dimensional Array of numbers.
standardDeviation	ss.standardDeviation(x)	Standard deviation of a single-dimensional array of numbers.
iqr	ss.iqr(sample)	Calculates the Interquartile range of a sample - the difference between the upper and lower quartiles. Useful as a measure of dispersion.
tTest	ss.tTest(sample, x)	Calculates Students t-value from a sample of numbers and expected value of the mean under the null hypothesis. The output will need to be checked against a table of t-vales to obtain a significance value.
tTestTwoSample	ss.tTestTwoSample (samp_x,samp_y)	Calculates Students t-value from two independent samples of numbers on the assumption that the expected value of the difference in means is zero under the null hypothesis. The output will need to be checked against a table of t-vales to obtain a significance value.
sampleCorrelation	ss.sampleCorrelation(a, b)	Produces sample correlation of two single-dimensional arrays of numbers.
shuffleInPlace	ss.shuffleInPlace(sample)	Given a sample array (with any type of contents), randomly shuffle the order of the array using the Fisher-Yates shuffle algorithm.

Here is some example code:

<script src="include/simple-statistics.js"></script>
<script>
var sample=[];
for (var i=0;i<100;i++) sample.push(10*Math.random());
console.log("Mean="+ss.mean(sample));
console.log("Median="+ss.median(sample));
console.log("Stddev="+ss.standardDeviation(sample));
console.log("IQR="+ss.iqr(sample));
</script>

Try it out here:

Perform Sample Statistics

Resources

• Javascript Object Notation home page.
• Simple Statistics home page.
• The example server scripts can be found at:
  • http://www.phon.ucl.ac.uk/courses/spsci/webprog/storeinfo.php
  • http://www.phon.ucl.ac.uk/courses/spsci/webprog/loadinfo.php
• The UCLServer javascript routines can be found at:
  • http://www.phon.ucl.ac.uk/courses/spsci/webprog/include/uclserver.js
• The simple statistics routines can be found at:
  • http://www.phon.ucl.ac.uk/courses/spsci/webprog/include/simple-statistics.js

Exercises

Please note that to use the UCL server-side scripts in exercises 8.2 & 8.3, your programs must be served from the departmental web server. You will need to use your departmental web space account for this.

1. Implement "XMLHttpRequest" demonstration (demo8-1.html). Note that this demonstration will only work when the page is delivered by a web server. You will also need to create the file mydata.txt. Study the code to understand how the request is sent to the server and how the information is sent back.
2. Implement "Simple Server-side storage" demonstration (demo8-2.html). This example calls the following server-side scripts which you can also test out directly from a web browser:

http://www.phon.ucl.ac.uk/courses/spsci/webprog/storeinfo.php?subject=name&score=100

http://www.phon.ucl.ac.uk/courses/spsci/webprog/loadinfo.php

In the processData function, add code that will find and report the subject with the highest score (ex8-2.html). You can use the string methods split("\n") to divide the response string into lines, indexOf to find the ",", and substring to extract the score.

3. Implement "UCL Server-side storage" demonstration (demo8-3.html). This example demonstrates the course-specific server-side scripts for storing/retrieving experimental data. The experimental task is a simple Lexical Decision task. Modify the code (ex8-3.html) to calculate the mean and standard deviation of the reaction time for each condition.

Homework

1. Complete the exercises you did not finish in class. Upload your answers to your course web space.
2. Work out for your web experiment what information you need to store on the server. Then write a simple test program that uploads some sample experimental data using the UCLSave() function. Note that your test program must be run from your departmental web space account to use the UCL server-side scripts. You can test whether the information has been uploaded correctly using the web page:

http://www.phon.ucl.ac.uk/courses/spsci/webprog/UCLTest.html

3. Next write a simple test program that uses UCLLoad() to download and display all the experimental data you saved with your test program in 2.

Word count: . Last modified: 16:11 23-Nov-2017.