Extending GWT to support Canvas and Multi-touch

Medrium has built a document management system about 8 years ago. This allows users to scan/upload documents as well as the ability to annotate - circle, add text, markup .. the images. Our goal has always been to be platform and browser agnostic. When we built this imaging/document management system, browsers had no ability to "annotate/markup" images. We also wanted to use TIFF formats since this is the most common format used by Scanners. We started off using Java applets and some open source java code .. for reading TIFF images. We later switched to JAI - Java Advanced Imaging. The document management system has served us well, but with advances in browsers/HTML5 we wanted to move away from this dependence on Java Applets.

I wanted to use HTML5 Canvas for drawing images/annotations. Obvioulsy we figured using GWT would be the right choice for this. GWT however does not currently support Canvas element. There are some GWT canvas libraries GWTCanvas and others that provide GWT widgets. Given the nature of our application and the need for support on devices like ipad, iphone etc where the input mechanisms like multi-touch is needed, I figured a better model would be to actually implement Canvas as a CanvasElement. Widgets can then be built as needed. So I took the approach GWT Quake Port has taken and defined CanvasElement, CanvasRenderingContext2D etc. As mentioned earlier our goal was also to mimick the same functionality we had in JAI based applets - annotation of images on devices like Ipad .. using multi-touch inputs. I ended up using the same model Alex Moffet described for supporting mult-touch. Ended up adding gesture events as well to multi-touch events.

It took me just 2 weeks to get an initial implementation up and running. Worked great on Safari/Mac, Firefox and Chrome. The multi-touch functionality worked great on ipad as well. I also realized that touch provides a different and better interface so it does not make sense to simulate the same "mouse down/drag ..." or an icon to zoom but use natural touch events like pinch for zooming ... Here again GWT's power shows off. Ended up creating seperate widgets - one for touch devices like ipad and one for PCs/Macs .. mouse based devices and used GWT's deferred binding to have the right widget instantiated. Deferred binding also came in handy to overcome some of the issues we had with Safari on ipad. Altough Canvas is supposed to be supported fully - drawing text using Canvas does not work - or rather works incorrectly. So ended up using Label Widget and positioning them for ipad and used Canvas draw text for annotating images with text.

My experience with "extending" GWT showed how easy and powerful GWT framework is. Converting from Java Applet to "native" browser Canvas implementation using GWT was fairly straightforward. Turns out Canvas implementation is a big performance boost as compared to Java Applet implementation and also fewer issue with compatibility of Java version/JAI install issues not to mention that we will be able to support our document management on ipad, iphone ..etc. There is no way I would have been able to get a initial implementation working within 2 weeks without GWT. Also benefitted tremendously from all the users experiences and suggestions folks had on using GWT.

We will be releasing the "Canvas" version of document management next month. Canvas currently is not supported in IE8 but will be supported in IE9 - available in IE9 preview. Until IE9 is out, we will have our users who want to use IE using Google Chrome Frame. There are silverlight and excanvas implemenation to add on Canvas tag to IE but all those seem to be buggy/limited. Drawing text, complex arcs .. is neither supported or buggy - at least that's what my experience has been. I could not get my application - annotation part working using excanvas.js. The silverlight version is fast but still has issues. Google Chrome Frame came to the rescue. Works like a charm :-).

Thanks to Google GWT team for building such a powerful framework.

Integrating GWT Services with Mirth/HL7

In my previous blog I talked about exposing GWT services to other protocol and languages. This has paid off big time for Medrium. In healthcare the predominant protocols used are HL7 which IMHO is hard to understand. HL7 makes simple problems very complex and convoluted to program to. Mirth is an open source project that has been around for quite a while and has built a robust and well tested platform for interoperability in healthcare. Mirth supports HL7 and provides a platform and tools that are well tested and used. Mirth has good tools built in to map HL7 to other formats like XML and vice-versa. Medrium already built functionality to expose GWT interfaces to XML which we ended up mapping - GWT XML interfaces to HL7 using mirth tools. Medrium used Mirth to map XML to HL7 and vice versa to provide connectivity to SureScripts for electronic prescriptions/messages functionality as well as importing Lab reports. Electronic prescriptions and Lab reports both use HL7 formats/versions and Medrium never had to program/understand HL7 formats but just understand enough to map the HL7 formats to GWT XML formats.

All our developers wrote GWT RPC services in Java and the XMLRPC functionality allowed Medrium to map the GWT Service method returns/calls to HL7 messages thru Mirth. Medrium will be using the same mechanism to import/export CCDs into and out of Medrium just using GWT RPC services. This shows that the model medrium chose - defining IDLs to expose interfaces and then mapping these interfaces to GWT RPC services was the right model. Exposing GWT services to XML not only facilitated our mapping of XML to HL7 using Mirth but has also provided a framework to open up interfaces to other vendors who can provide additional services to our clients.