Posts

Adventures with Javascript Graphing Libraries

/0 Comments/in , /by

I was looking for a Javascript library that would display a graph – in the mathematical sense of a node-arc network such as the following:

I might require curved lines and maybe arrows, and I need it to be zoomable and pannable.  I also need node labels and click events on both nodes and edges.

Remembering a few links someone sent around showing off impressive applications of libraries such as d3.js and jQuery Sparklines I had high hopes of finding something I could use that was polished and snazzy with superb documentation and an enthusiastic community.

The first thing I noticed is that it seems like everyone is in love with Force-Directed Layouts these days – you know those graphs where the nodes appear “springy” and the spatial layout is “discovered” by the physics algorithm, such as this example.  By their nature, however, these graphs have a dynamic layout – you generally can’t specify exact positions for your nodes.

The libraries I looked at included the following.  A discussion of SVG vs Canvas is here.

d3.js – this library is SVG-based (as opposed to say, HTML5 canvas-based).  With a cursory look I couldn’t see how to zoom (you should be able to though, right – it’s scalable vector graphics…!), and the docs are pretty large and initially confusing.  The consensus seems to be that it is a power users tool.

Sigma.js has some nice-looking examples and appears to scale to hundreds of nodes and arcs nicely. It can use GEXF graph format. However, it lacks good documentation and doesn’t seem to have much momentum – the last source code change was 9 months ago.

jsPlumb has demos that do not appear to be zoomable, though the library seems to be still actively developed and has good docs.  It can render by canvas/SVG/VML using jQuery/MooTools etc.  However it seems more aimed at connecting elements with drag and drop than showing graphs per se.

Raphael seems nice enough, is SVG based, has intriguing demos and reasonable docs, the last change on github was 10 months ago, but again the demos had no zoom.

The Javascript InfoVis Toolkit strikes the right balance of functionality with simplicity for me – there’s enough simplicity that I can actually delve into the source (only one file needed – jit.js) and pretty easily see what’s going on.  It’s canvas-based, but appears to mousewheel zoom (very smoothly) using canvas’ scale function.  The project was also involved in a Google Summer of Code.  Although there was no static graph example, I found it quite easy to adapt the force-directed example code to show a nice zoomable graph with node coordinates that I explictly specified (and has node and edge click events).  (In doing this, I also discovered something interesting – the force-directed layout example gives you a totally new layout every time you refresh – you can see this by refreshing the demo page here multiple times.  This randomised behaviour seems somewhat less than useful. Also interesting is the drag and drop of the nodes in the force directed example).

So I did not find anything that fit squarely with my requirements, though I’m going with the Infovis Toolit for now.  (Of course there is always the possibility to handcode it myself, which for reasonably simple requirements is always an option, though less preferred particularly as you might encounter issues such as cross-browser bugs that have been addressed in the libraries and frameworks.)

On the plus side I truly learnt what Bezier curves actually are (and how they differ from quadratic Bezier curves), thanks to some help from the Wikipedia page and this neat interactive page where you can drag the control points around to see the effect on the curve.

Some time after doing the above (admittedly rather cursory) research, I then came across another cluster of libraries with a slightly different focus.  These libraries are more geared towards interactive elements on canvas (e.g. drag and drop) and persisting the state of the elements when they are changed – a good starting discussion is at the Stack Overflow page here.  The “master list” of these libraries I found in a Google doc here.  These libraries provide more of a “scene graph” implementation which would be useful if you need a framework for proper tracking of the elements being drawn (especially if they are to be animated, e.g. a particle simulator).

js/css resource serving in python apps with Fanstatic

/0 Comments/in , , /by

I’ve just been checking out Fanstatic, a resource publishing/static file serving solution for wsgi python apps. I’ve been contemplating something like this as our javascript and css dependencies are getting more complex. It would also be useful to have some form of automatic cache invalidation so users don’t have to do a special browser refresh when we update our applications.

It’s easy to set up with CherryPy

from fanstatic import Fanstatic

if __name__ == "__main__":

    app = cherrypy.Application(Root())
    app.wsgiapp.pipeline.append(('repoze.who', setup_auth))
    app.wsgiapp.pipeline.append(('beaker', setup_session_storage))
    app.wsgiapp.pipeline.append(('fanstatic', Fanstatic))
    cherrypy.quickstart(app, config='workbench.conf')

after that is done, you can jquery.need() in the widget/template that needs jquery, and similar for our other dependencies. Has anyone else used fantastic? What are other solutions to dependencies and serving js/css? Is there an easier and better solution? Wrapping new libraries for fanstatic looks like a bit of effort but I haven’t explored it much yet.

Loki

Biari Workbench Technology Stack

/0 Comments/in , , /by

Over the course of developing our Workbench solution we’ve adopted a powerful set of interconnecting components. It’s worth mentioning what these are and how they fit together.

Almost all the components of the stack are free and/or open source. We want to be as platform independent as possible and not get too locked in to one technology paradigm. This means that as much as possible, parts should be as “hot swappable” as possible – which also helps encourage strong componentisation. Using components with mature and open/standardised interfaces is very necessary when you’re crossing language boundaries (most notably, Javascript-Python and Python-C++) and client/server boundaries; otherwise you risk re-inventing the wheel. Ideally each component we use should also still be in active development (in the IT world – with the odd highly venerable exception – if software is not growing and evolving, it’s usually either dying, already in it’s death throes, or extinct).

There’s an art to using the right tool for the job, and we’ve made mistakes. We over-used Mako (see Loki’s blog post) and also originally used a slightly inferior lib for the C++ xmlrpc back end; both these mis-steps were fairly easily rectified. Arguably, we probably still use too much C++ and not enough Python – the C++ line count dwarfs the Python line count by a considerable margin. One last interesting point is that, at the moment, we’re still eschewing use of an ORM (Object Relational Mapping layer – such as SQL Alchemy) – time will tell whether that is a good idea or not.

Client:

JavaScript
 – client-side browser language

jQuery
 – JavaScript library for event handling and more

Cloudmade
 – OSM map provider/server

 

Data Interchange:

JSON – JavaScript Object Notation

XML – eXtensible Markup Language

Mathematical Engines:

Mostly in C++ using STL

CppUnit
– C++ library for unit testing

OGR
 – map data library, part of GDAL – used to read map data

Libxmlrpc-c
 – C++ back end for XMLRPC – used by a running process to communicate with the front end via Python

 

Server:

Python
– language

CherryPy
 – Python-based web app framework

PostgreSQL
 – open source RDBMS

PsycoPg2
 – database adaptor for PostgreSQL/Python

XmlRPCLib
 – XML RPC (used to communicate with some of the engines)

Mako
 – Python template library

Repoze
 – Zope/WSGI Python middleware (for authentication)

 

 

Javascript: It’s actually pretty decent.

/0 Comments/in , /by

I admit it, the ‘Java’ in the word JavaScript has previously put me off. However, it turns out it really is rather nice and with some searching you can find some good documentation!

In our midst we have a language with more in common with scheme and lisp than Java or VB! And, it’s wonderful features are eloquently explained by Douglas Crockford covering how to use encapsulation and a great series of slides by John Resig explaining closures and prototype based inheritance.

Douglas Crockford mentions a JavaScript gotcha that bit me – “this” which you use like self in python but is implicit, is set incorrectly for inner functions. It’s simple to get around though, just set a var, i.e “that” to it’s value before your inner functions.

I’m planning, and I admit this plan may be impeded by other more pressing concerns, to write a series of posts explaining my recent investigation of python (and some non-python) web frameworks. I also want to discuss my quest to write a application framework based on the principles of simplicity, elegance and power, for users and developers.

Despite JavaScript’s positives the idea of a python to JS complier like Pyjamas does seem enticing. I’d be really interested in hearing comments from anyone who’s used it. Python to me is still far more elegant. 🙂

Loki