03/10/12

GWT Augmented Reality - HOWTO - step 0

This post is the first of a short (we hope) series describing the steps required to use NyARToolkit in a GWT/Elemental project.





The sample is at http://www.jooink.com/experiments/ARtest1/ (use chrome !!)


In this step we do not need really Elemental !

Using Elemental in a GWT project is quite straightforward:
  1. donwload GWT RC1 (well, RC2 is out today but I have not tested with it) and setup a new project;
  2. add to the build path gwt-elemental.jar (is in the unpacked gwt 2.5 archive);
  3. add to the gwt.xml file the line  <inherits name='elemental.Elemental'/>
    ....
and you're ready.
Unfortunately DevMode, at least in our setup, do not works well when you use Elemental so you need to compile to test your app ... and that, during development of this part, is certainly not amusing.

Note:
Chrome has an issue in loading local files that do not let you play with the (any) webapp from the local filesystem thus you have to use ff or start devmode but when loading in the browser use the url without  ?gwt.codesvr=127.0.0.1:9997
in my case
   http://127.0.0.1:8888/TestWGLdrawrectangles.html
instead of
   http://127.0.0.1:8888/TestWGLdrawrectangles.html?gwt.codesvr=127.0.0.1:9997

So let's keep hands on GWT 2.4 for now.

Download a fresh copy of artoolkit (we used 4.0.3, the current) from the official web page:
 http://nyatla.jp/nyartoolkit/wp/?page_id=315.

Project Layout

Inside NyARToolkit-4.0.3.zip file you find a lib folder, inside lib there are two subfolders you will need: src and src.markersystem, you should import both.
We merged both folders in a single eclipse java project, maybe not the ideal solution but this was an early decision we have to overcome in the future. After a few adjustments to paths and build paths you should end up with a couple of java projects without errors.

To proceed with the GWTization you must provide a module.gwt.xml file, we put it in the package
jp.nyatla.

 Nyatla.gwt.xml:
<module>
  <source path="nyartoolkit"/>
</module>

Now we need a GWT project to start testing (we're still far from the result).
Create a standard GWT 2.4 project (without AppEngine), add to the Build Path the Projects create in the previous step and add to the (current) module.gwt.xml file the following inherit line

<inherits name='jp.nyatla.Nyatla'/>

You have imported the toolkit so now you can, for example, add in your EnrtyPoint class

                    NyARSensor i_sensor = new NyARSensor(new NyARIntSize(320,240));

to create a sensor :)

Unfortunately if you try to run the project you get a lot of errors, something like:

[ERROR] [test] - Line 59: No source code is available for type java.io.InputStream; did you forget to inherit a required module?

Actually NyARToolkit (that is a java library intended for java projects) uses classes of the JRE that are not part of the GWT's JRE (https://developers.google.com/web-toolkit/doc/latest/RefJreEmulation).
We do not want to use InputStream but we need to provide some implementation to the compiler; well, you can invest some time in dropping inside NyARToolkit any reference to unsupported classes or, you can just provide no-nothing-code for the unsupported classes that we do not need through super-source tag.

super-source

As you can find in the documentation
https://developers.google.com/web-toolkit/doc/latest/DevGuideOrganizingProjects?hl=en,

"The <super-source> tag instructs the compiler to "re-root" a source path. This is useful for cases where you want to be re-use an existing Java API for a GWT project, but the original source is not available or not translatable. A common reason for this is to emulate part of the JRE not implemented by GWT."

So super-source is exactly what we need.

We created a jre directory in the GWT project and put implementation for:

java.io.FileInputStream
java.io.InputStream
java.io.InputStreamReader
java.io.StreamTokenizer
java.lang.reflect.Array
java.nio.ByteBuffer
java.nio.ByteOrder

All but the Array (java.lang.reflect) are really unused so you just need dumb implementations,
our FileInputStream reads as follow:

package java.io;
import java.io.InputStream;
import com.google.gwt.user.client.Window;

public  class FileInputStream  extends InputStream {
    public FileInputStream(String filename) {
        Window.alert("WARNING, FileInputStreamCreated with filename:" + filename );
    }   
    @Override
    public int read() {
        return 0;
    }
}


The Window.alert statement in the constructor is a warning useful during development so we will be alerted in case of usage of the dumb-inputstream (note, in developement mode the 'real' java jre is always used, read carefully the documentation).

java.lang.reflect.Array is actually used by the code we need so a not-completely-dumb implementation is provided,  this is our code:

package java.lang.reflect;
import jp.nyatla.nyartoolkit.core.labeling.rlelabeling.NyARRleLabelFragmentInfo;
import jp.nyatla.nyartoolkit.markersystem.utils.SquareStack;
import com.google.gwt.user.client.Window;
public class Array {
   
    public static <T> Object newInstance(Class<T> c, int n) {
   
        if( NyARRleLabelFragmentInfo.class.equals(c))
            return new NyARRleLabelFragmentInfo[n];
        else if(SquareStack.Item.class.equals(c))
            return new SquareStack.Item[n];
        else
            Window.alert("Creating array of size " + n  + " of " + c.toString());
        return null;
    }
}

That's all, place     <super-source path="jre"/> in the .gwt.xml module file and now you can safely create a NyARSensor. :)


In the next  post, loading markers (without InputStream) and providing data to NyARSensor... ASAP.

Ciao,
   Alberto.

The content of this post is licensed under the Creative Commons Attribution 3.0 License, and code samples are licensed under the Apache 2.0 License.