Overview

Backends connect Recline Datasets to data from a specific ‘Backend’ data source.

They provide methods for loading and saving Datasets and individuals Documents as well as for bulk loading via a query API and doing bulk transforms on the backend.

Backends come in 2 flavours:

Instantiation and Use

You can use a backend directly e.g.

var backend = recline.Backend.ElasticSearch.fetch({url: ...});

But more usually the backend will be created or loaded for you by Recline and all you need is provide the identifier for that Backend e.g.

var dataset = recline.Model.Dataset({
  backend: 'backend-identifier'
});

Backend identifiers How do you know the backend identifier for a given Backend? It's just the name of the 'class' in recline.Backend module (but case-insensitive). E.g. recline.Backend.ElasticSearch can be identified as 'ElasticSearch' or 'elasticsearch'.

What Backends are available from Recline?

Backend you’d like to see not available? It’s easy to write your own – see the Backend reference docs for details of the required API.

Preparing your app

This is as per the quickstart but the set of files is much more limited if you are just using a Backend. Specifically:

<!-- 3rd party dependencies -->
<script type="text/javascript" src="vendor/jquery/1.7.1/jquery.js"></script>
<script type="text/javascript" src="vendor/underscore/1.1.6/underscore.js"></script>
<script type="text/javascript" src="vendor/backbone/0.5.1/backbone.js"></script>

<!-- include the backend code you need e.g. here for csv -->
<script type="text/javascript" src="http://okfnlabs.org/csv.js/csv.js"></script>

<!-- Or you can just include all of recline. -->
<script type="text/javascript" src="dist/recline.js"></script>

Loading Data from Google Docs

We will be using the following Google Doc. For Recline to be able to access a Google Spreadsheet it must have been ‘Published to the Web’ (enabled via File -> Publish to the Web menu).

// include the Recline backend for Google Docs
<script type="text/javascript" src="http://okfnlabs.org/recline.backend.gdocs/backend.gdocs.js"></script>
// Create a dataset with a Google Docs backend and a url to the Google Doc
var dataset = new recline.Model.Dataset({
  url: 'https://docs.google.com/spreadsheet/ccc?key=0Aon3JiuouxLUdGZPaUZsMjBxeGhfOWRlWm85MmV0UUE#gid=0',
  backend: 'gdocs'
});

// Optional - display the results in a grid
// Note how we can set this up before any data has arrived
// Views will listen for query completion and update themselves automatically
var grid = new recline.View.Grid({
  model: dataset
});
$('#my-gdocs').append(grid.el);

// Now do the query to the backend to load data
dataset.fetch().done(function(dataset) {
  if (console) {
    console.log(dataset.records);
  }
});

Result

 

Loading Data from ElasticSearch

Recline supports ElasticSearch as a full read/write/query backend via the ElasticSearch.js library. See the library for examples.

Loading data from CSV files

For loading data from CSV files there are 3 cases:

  1. CSV is online but on same domain or supporting CORS (S3 and Google Storage support CORS!) – we can then load using AJAX (as no problems with same origin policy)
  2. CSV is on local disk – if your browser supports HTML5 File API we can load the CSV file off disk
  3. CSV is online but not on same domain – use DataProxy (see below)

In all cases we’ll need to have loaded the Recline CSV backend (for your own app you’ll probably want this locally):

<script type="text/javascript" src="http://okfnlabs.org/csv.js/csv.js"></script>

Local online CSV file

Let’s start with first case: loading a “local” online CSV file. We’ll be using this example file.

// Create the dataset in the usual way
// Note the additional options you can specify for parsing the CSV file
var dataset = new recline.Model.Dataset({
  url: '../demos/data/sample.csv',
  backend: 'csv',
  // delimiter: ',',
  // quotechar: '"',
  // encoding: 'utf8'
});

// remember this is async so if you want to do something you need to call it in done method e.g.
// dataset.fetch.done(function(dataset) { console.log(dataset.recordCount)});
dataset.fetch();

// show the data for illustrations sake
var grid = new recline.View.SlickGrid({
  model: dataset,
  el:  $('#my-online-csv')
});
grid.visible = true;

Result

 

CSV file on disk

This requires your browser to support the HTML5 file API. Suppose we have a file input like:

Then we can load the file into a Recline Dataset as follows:

// the file input
var $file = $('.my-file-input')[0];

// listen for the file to be submitted
$($file).change(function(e) {
  // create the dataset in the usual way but specifying file attribute
  var dataset = new recline.Model.Dataset({
    file: $file.files[0],
    backend: 'csv'
  });

  // now load - note that this is again async (HTML5 File API is async)
  // dataset.fetch().done(function() { console.log('here'); });
  dataset.fetch();

  // For demonstrations purposes display the data in a grid
  var grid = new recline.View.Grid({
    model: dataset
  });
  $('#my-csv-disk').append(grid.el);
});

Try it out!

Try it out by clicking on the file input above, selecting a CSV file and seeing what happens.

 

Loading data from CSV and Excel files online using DataProxy

The DataProxy is a web-service run by the Open Knowledge Foundation that converts CSV and Excel files to JSON. It has a convenient JSON-p-able API which means we can use it to load data from online CSV and Excel into Recline Datasets.

Recline ships with a simple DataProxy “backend” that takes care of fetching data from the DataProxy source.

The main limitation of the DataProxy is that it can only handle Excel files up to a certain size (5mb) and that as we must use JSONP to access it error information can be limited.

var dataset = new recline.Model.Dataset({
  url: 'http://data.london.gov.uk/datafiles/transport/tfl_passengers.csv',
  // optional rows parameter specifies how many rows to retrieve - default is a 1000
  // rows: 5000
  backend: 'dataproxy'
});

// async again as we fetch via AJAX behind the scenes
// once data is fetched it will be stored in a local MemoryStore so further querying will not involve the DataProxy
dataset.fetch();

// For demonstrations purposes display the data in a grid
var grid = new recline.View.Grid({
  model: dataset
});
$('#my-dataproxy').append(grid.el);

Result

 

Customizing the timeout

As we must use JSONP in this backend we have the problem that if DataProxy errors (e.g. 500) this won’t be picked up. To deal with this and prevent the case where the request never finishes We have a timeout on the request after which the Backend sends back an error stating that request timed out.

You can customize the length of this timeout by setting the following constant:

// Customize the timeout (in milliseconds) - default is 5000
recline.Backend.DataProxy.timeout = 10000;