Discussion Group

There is a new discussion group open for MiniIM project. Thanks go to google-group-team for such wonderful support. Vision, action and community support mean success.

Serializable kdom rocks

The kxml project means a lot for j2me. I have done some modification to make a document serializable in Recordstore. Here is the code. You can follow the usage examples in the code to make it work for you.

To use this code you just need to get the code from trunk. The obfuscator will skip the classes you do not use.

Browser features explained

Earlier I talked about my vision. In that application there are pages like "More", "About" they are rendered from markup files(more.xml, about.xml, ..) found here. They are following markup.dtd rules. Surely you can create documents on fly and work on DOM.

Note that it is svn-branch, the trunk is always containing the updated features.

And there are application-switches making it possible to interact with user. Here are the responsible code doing the magic.

The markup is not rich in features. It was good to follow an existing markup rule. I am hoping to do that in future. And I am keeping eye on w3.org for that.

XMPP client with browser-like features

I am back to j2me again. In my earlier attempt I was trying to render a WML content. Now I am here with a little advanced browser like feature to show web content. Note that I have dropped the idea of writing WML renderer. I have added a markup renderer. To be specific it can render a list of markup items. Here we can show on line content. It is stylable on compile-time.


Motivation
When I want to add some feature to my sweet application, it just gets bigger in size. And bigger size prevent it to work in low end mobile phones. So the idea is to write little j2me code and add most of the features in the servers. And here I am sharing the implementation of the idea in a hope that it gets richer and useful to you.

Similar efforts
There is something similar available in ZK project. And there is also a browser feature in j2mepolish project. I prefer mine, because I believe that it has very small footprint, it is light-weight(reuses objects) and I am willing to process any feature request that will suite this. Mobile Ajax is another interesting project.

Input fields
It has textbox, selection(multiple) box, checkbox, radio input ideas implemented. I am hoping to add some multimedia feature as well. Like an input box for image, sound etc and also playback of the media content.

Benchmark
It is tested on WTk monitor output that shows low object instantiation.

Caveats
It is using document parser from kxml project. The document and elements there uses vectors and it makes the parsing little slower. I am looking for some alternative to this.

Thinking forward
I think this is a step forward to social features. The next thing to do is to write web application to represent information from social sites. I am now looking for any kind of reusable component for this. I mean I am not trying to reinvent. And it is hard to maintain too. If you think you have nice idea or have something to say please feel free post comment or email me directly.

Additionally it has a IM framework to develop end to end application too.

Please consider testing MiniIM in your mobile.

Pidgin as instant messaging gateway

Pidgin is the most powerful IM client I have ever seen . This is because of the big community support and also because I am big fan of Mark Spencer :-P .. The libpurple (Pidgin/Finch core) library can be used as IM gateway . I implemented such gateway for event notification . Something like a bot .



It keeps libpurple connected to the IM servers of different protocol and takes message request from UNIX socket . For example the following code will send a message to Chitrangoda ..


#!/usr/bin/perl -w

use strict;
use Socket;

my $sock_file = $ARGV[1] || "./im_socket_file";
my $msg;
die "provide a socket file address" unless $sock_file;

socket(SOCK, PF_UNIX, SOCK_STREAM, 0) || die "socket: $!";
print "connecting ..\n";
connect(SOCK, sockaddr_un($sock_file)) || die "connect: $!";

print "writing ..\n";
$msg = "chitrangoda\@example.com MSN This is sent to Chitrangoda via MSN server.".chr(0x1A);
send(SOCK,$msg,0);
recv(SOCK,$msg,100,0);
print "done\n";
close SOCK;



And here is the code with examples pidgin_send_im (please let me know if the link is broken).It works with Pidgin version 2.4.2 and upper (I am not sure about the lower version) ..

Again, there are more than one way to do it. You can use the perl/php bindings or plugins .. I expect you show those howtos in the comments.

MiniIM in Action

I am really enjoying the development, Here is how it looks currently.

KXML skip SubTree skip Depth

KXML is just the name of XML parsing in J2ME. As I am using it in MiniIM, parsing XMPP and even special markup, I found it useful to skip upto a specific depth of XML tree. Note, the skipSubTree() method in KXMLParser / XMLPullParser API is useful. But it needs that you call it at tag start otherwise it will throw exception. So I introduce skipDepth(depth) idea. You just set your depth upto which you want to skip. It will just eat data upto that.

For example here is a small part of stream,

<?xml version='1.0'?>

<stream:stream xmlns="jabber:client" xmlns:stream="http://etherx.jabber.org/streams" id="1523376107" from="jabber.org" version="1.0" xml:lang="en">

...
...
 <stream:features>
  <mechanisms xmlns="urn:ietf:params:xml:ns:xmpp-sasl">
   <mechanism>DIGEST-MD5</mechanism>
   <mechanism>PLAIN</mechanism>
  </mechanisms>
  <register xmlns="http://jabber.org/features/iq-register"/>
 </stream:features>

...
...

Here is the simple code to parse the "stream:features",

final void readFeatures() throws IOException {
  String tagName;
  String text;
  parser.nextTag();
  parser.require(KXmlParser.START_TAG, null, PREFIX_FEATURES);
  
  for (; parser.nextTag() != KXmlParser.END_TAG; parser.skipDepth(3)) { /* The features tag is placed at depth 1 */
   /* we are parsing things inside this feature tag */
   tagName = parser.getName();
   if (tagName == null) {
    throw new IllegalStateException("feature is null");
   } else if (tagName.equals(STARTTLS)) {
    /* set a starttls read command */
    SimpleLogger.debug(this, "<" + STARTTLS);
    featuresTLS = true;
   } else if (tagName.equals(MECHANISMS)) {
    /* traverse the mechanisms inside */
    SimpleLogger.info(this, "readFeatures()\t\tparsing mechanisms");
    /* so while inside the mechanisms */
    for (; parser.nextTag() != KXmlParser.END_TAG;parser.skipDepth(5)) {
     tagName = parser.getName();
     if (tagName.equals(MECHANISM)) {
      /* see if the mechanism is PLAIN */
      text = expectText();
      if (text == null) {
       
      } else if (text.equals(PLAIN)) {
       /* say that plain mechanism is found */
       SimpleLogger.info(this, "<" + PLAIN);
       featuresPlain = true;
      } else if(text.equals(DIGEST_MD5)) {
       SimpleLogger.info(this, "<" + DIGEST_MD5);
       featuresDigestMd5 = true;
      }
      /* do not break, go on parsing mechanisms */
     }
    }
   } else if (tagName.equals(BIND)) {
    SimpleLogger.debug(this, "<" + BIND);
    featuresBind = true;
   } else if(tagName.equals(COMPRESSION)) {
    // QUOTE "the ZLIB compression algorithm is mandatory-to-implement"
    featuresZLIB = true;
    /* traverse the mechanisms inside */
    SimpleLogger.info(this, "readFeatures()\t\tparsing compression");
    for (; parser.nextTag() != KXmlParser.END_TAG;parser.skipDepth(5)) {
     tagName = parser.getName();
     if (tagName.equals(METHOD)) {
      String compressionMethod = expectText();
      if(compressionMethod != null && compressionMethod.equals(ZLIB)) {
       featuresZLIB = true;
      }
     }
    }
   } else {
    SimpleLogger.debug(this, "<!" + tagName);
    /* whatever it is just finish the tag */
   }
   SimpleLogger.debug(this, "< next");
  }
  parser.skipDepth(2);  
 }
}

And here is the small method I have added in KXMLParser.

public void skipDepth(int dep) throws IOException,XmlPullParserException {
 int level = getDepth() - dep;
 int eventType = getEventType();
 
 if(eventType != KXmlParser.END_TAG) {
  level++;
 }
 
 while(level > 0) {
  eventType = next();
     if (eventType == KXmlParser.END_TAG) {
  --level;
     }
     else if (eventType == KXmlParser.START_TAG) {
  ++level;
     }
 }
}

I hope it helps :) ..

Shrink wurfl mobile database

WURFL is nice mobile database. The j2mepolish database is good too. Anyway I was trying to use wurfl.xml file(you can download it from wurfl homepage) while releasing my j2me application. But as the wurfl.xml file is too big(4.3M), I have removed some unused attributes and tags from it. And still I am able to use the parsers (I tested only the perl parser :-P ). Finally the size is reduced to (1.4M) .

My primary goal is to resize some of my background images appropriate to the the screen resolution for a specific mobile. So I filtered the resolution_width and resolution_height information.

Here is the small xslt code used to shrink the wurfl.xml .

<?xml version="1.0" ?>

<!--

This file part of MiniIM.

Copyright (C) 2007  Kamanashis Roy

MiniIM is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

MiniIM is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with MiniIM.  If not, see <http://www.gnu.org/licenses/>.

-->


<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">

<xsl:output method="xml" indent="yes"/> 

<!-- select the device tag -->
<xsl:template match="/wurfl">

       <wurfl>
  <xsl:copy-of select="version"/>
  <devices>
   <xsl:apply-templates select="devices/device"/>

  </devices>
 </wurfl>
</xsl:template>

<!-- process devices -->
<xsl:template match="device">

        <device fall_back="{@fall_back}" id="{@id}" user_agent="{@user_agent}">
  <!-- keep only display group -->

           <xsl:apply-templates select="group[@id='display']"/>
        </device>
</xsl:template>

<xsl:template match="group">

 <!-- render non empty group -->
 <xsl:if test="count(capability[@name='resolution_width'])=1">
   <group id="{@id}">

    <xsl:apply-templates select="capability[@name='resolution_width']"/>
    <xsl:apply-templates select="capability[@name='resolution_height']"/>
   </group>

 </xsl:if>
</xsl:template>

<xsl:template match="capability">
 <capability name="{@name}" value="{@value}"/>

</xsl:template>

</xsl:stylesheet>

Now let us do the transform, I have used xsltproc for the transform here, you may want to get xsltproc binary for your os or get source here. And you can apply transform like this,

shell> xsltproc your_shrinker.xslt wurfl.xml > wurfl_shrinked.xml

You can do some appropriate modifications to this code to keep some other attributes too. I prefer to write separate xsl code for each application. Note there are other alternatives, like database
usage available in wurfl site. So please read those docs before trying this :) .

Finally I used the cool perl module form imagemagick to resize my images :) .

Enjoy !

GPS device emulation how to

I do not have a GPS device. Though I dare to write applications for it. Somehow I have managed a way to emulate a GPS device.

So, what is the source of sample GPS data ? I use public gpsd servers as NMEA data source ..
You can do google to find a gpsd server.

How can we emulate ? I use blueproxy to proxy the gpsd server connection over bluetooth. So you just need to download and install blueproxy.

Finally it is simple, start your server with the following command,

shell #> blueproxy -v -c 11 -h gpsd_server.address -p gpsd_port

Please replace the "gpsd_server.address" and "gpsd_port" with something appropriate. Note that gpsd servers needs to receive R before it can start to broadcast NMEA messages. So just send 'R\r\n' before you wait for data.

As I am midlet developer, I tried the idea of proxy in midlet too .. Finally I was able to test the whole thing in WTK emulator.

Here is the code,(HTML generated by highlight 2.6.9, http://www.andre-simon.de/)

package blueProxy;

import java.io.IOException;

import javax.microedition.io.Connector;
import javax.microedition.io.StreamConnection;
import javax.microedition.io.StreamConnectionNotifier;
import javax.microedition.lcdui.Command;
import javax.microedition.lcdui.CommandListener;
import javax.microedition.lcdui.Display;
import javax.microedition.lcdui.Displayable;
import javax.microedition.lcdui.Form;
import javax.microedition.lcdui.TextField;
import javax.microedition.lcdui.StringItem;
import javax.microedition.midlet.MIDlet;
import java.io.OutputStream;
import java.io.InputStream;
import javax.bluetooth.BluetoothStateException;
import javax.bluetooth.LocalDevice;
import javax.bluetooth.DiscoveryAgent;

public class BlueProxy extends MIDlet implements CommandListener {
        // UI Specific
        private final TextField url = new TextField("URL", "socket://gpsd.mainframe.cx:2947", 200, TextField.ANY);
        private final StringItem log = new StringItem("Log", "no clients are connected ..");
        private final Command quit = new Command("Quit", Command.EXIT, 1);
        private final Command start = new Command("Start", Command.OK, 1);

        public void startApp() {
                // test ui
                Form myForm = new Form("Blue Proxy");
                myForm.append(url);
                myForm.append(log);
                myForm.addCommand(quit);
                myForm.addCommand(start);
                myForm.setCommandListener(this);
                Display.getDisplay(this).setCurrent(myForm);
        }

        public void pauseApp() {
        }

        /**
    * Destroy must cleanup everything. The thread is signaled to stop and no
    * result is produced.
    */
        public void destroyApp(boolean unconditional) {
                notifyDestroyed();
        }

        public void commandAction(Command cmd, Displayable arg1) {
                if(cmd == start) {
                        ((Form)arg1).removeCommand(start);
                        new Thread() {
                                        public void run() {
                                                System.out.println("Started thread#1");
                                                LocalDevice ld = null;
                                                try {
                                                        ld = LocalDevice.getLocalDevice();
                                                        ld.setDiscoverable(DiscoveryAgent.GIAC);
                                                } catch( BluetoothStateException e) {
                                                        e.printStackTrace();
                                                        return;
                                                }
                                                try {
                                                        // open server socket ..
                                                        log.setText("opening connection notifier ..");
                                                        StreamConnectionNotifier connn = (StreamConnectionNotifier) Connector.open("btspp://localhost:1;encrypt=false;authorize=false;authenticate=false");
                                                        while(true) {
                                                                log.setText("waiting for a new client ..");
                                                                try {
                                                                        final StreamConnection conn = (StreamConnection)connn.acceptAndOpen();
                                                                        // open client socket ..
                                                                        log.setText("opening socket to " + url.getString());
                                                                        final StreamConnection client = (StreamConnection) Connector.open(url.getString());
                                                                        // peering ..
                                                                        new Thread() {
                                                                                public void run() {
                                                                                        System.out.println("Started thread#2");
                                                                                        try {
                                                                                                InputStream is = conn.openInputStream();
                                                                                                OutputStream os = client.openOutputStream();
                                                                                                byte data[] = new byte[200];
                                                                                                int res = 0;
                                                                                                while((res = is.read(data)) != 0) {
                                                                                                        System.out.println("bt > " + new String(data, 0, res));
                                                                                                        os.write(data, 0, res);
                                                                                                }
                                                                                        } catch(IOException e) {
                                                                                                e.printStackTrace();
                                                                                        }
                                                                                }
                                                                        }.start();
                                                                        InputStream is = client.openInputStream();
                                                                        OutputStream os = conn.openOutputStream();
                                                                        byte data[] = new byte[200];
                                                                        int res = 0;
                                                                        while((res = is.read(data)) != 0) {
                                                                                System.out.println("bt < " + new String(data, 0, res));
                                                                                os.write(data, 0, res);
                                                                        }
                                                                } catch(IOException e) {
                                                                        e.printStackTrace();
                                                                }
                                                        }
                                                } catch (IOException e) {
                                                        e.printStackTrace();
                                                }
                                }
                        }.start();
                } else {
                        destroyApp(true);
                }


        }
}

Image thumbnail in an optimized way for J2Me platform

Trying to show thumbnail view after taking a big snapshot ? Trying to show a list view of images ? Or in any other case you try to resize or scale an image ?

I was searching for an optimized code for thumbnail creation. The main issues here are

- You need to be careful about the memory you use. After taking a big image in memory you may have a little left for allocation.
- You do not have floating point in most of the mobiles. So your image may not keep the image ratio right after scaling.
- Your need very time efficient code , you can keep the image quality apart ..

I did some google. And found the following discussions useful.

http://saloon.javaranch.com/cgi-bin/ubb/ultimatebb.cgi?ubb=get_topic&f=41&t=002988
http://developer.sonyericsson.com/thread/27493

I tested some of those examples. Some of them cause OutOfMemory Exception. Some take a long time to resize image. But I thought I could do better. Finally I wrote the following code.

    /**
     * Gets the thumbnail that fit with given screen width, height and padding ..
     *
     * @param image The source image
     * @param padding padding to the screen
     * @return scaled image
     */
    private final Image getThumbnailWrapper(Image image, int expectedWidth, int expectedHeight, int padding) {
     final int sourceWidth = image.getWidth();
  final int sourceHeight = image.getHeight();
     int thumbWidth = -1;
  int thumbHeight = -1;

  // big width
  if(sourceWidth >= sourceHeight) {
   thumbWidth = expectedWidth - padding;
   thumbHeight = thumbWidth * sourceHeight / sourceWidth;
   // fits to height ?
   if(thumbHeight > (expectedHeight - padding)) {
    thumbHeight = expectedHeight - padding;
    thumbWidth = thumbHeight * sourceWidth / sourceHeight;
   }
  } else {
   // big height
   thumbHeight = expectedHeight - padding;
   thumbWidth = thumbHeight * sourceWidth / sourceHeight;
   // fits to width ?
   if(thumbWidth > (expectedWidth - padding)) {
    thumbWidth = expectedWidth - padding;
    thumbHeight = thumbWidth * sourceHeight / sourceWidth;
   }
  }

  // XXX As we do not have floating point, sometimes the thumbnail resolution gets bigger ...
  // we are trying hard to avoid that ..
  thumbHeight = (sourceHeight < thumbHeight) ? sourceHeight : thumbHeight;
  thumbWidth = (sourceWidth < thumbWidth) ? sourceWidth : thumbWidth;

  return getThumbnail(image, thumbWidth, thumbHeight);
    }

    /**
     * Gets thumbnail with a height and width specified ..
     * @param image
     * @param thumbWidth
     * @param thumbHeight
     * @return scaled image
     */
    private final Image getThumbnail(Image image, int thumbWidth, int thumbHeight) {
  int x, y, pos, tmp, z = 0;
  final int sourceWidth = image.getWidth();
  final int sourceHeight = image.getHeight();

  // integer ratio ..
  final int ratio = sourceWidth / thumbWidth;

  // buffer where we read in data from image source
  final int[] in = new int[sourceWidth];

  // buffer of output thumbnail image
  final int[] out = new int[thumbWidth*thumbHeight];

  final int[] cols = new int[thumbWidth];

  // pre-calculate columns we need to access from source image
  for (x = 0,pos = 0; x < thumbWidth; x++) {
   cols[x] = pos;

   // increase the value without fraction calculation
   pos += ratio;
   tmp = pos + (thumbWidth - x) * ratio;
   if(tmp > sourceWidth) {
    pos--;
   } else if(tmp < sourceWidth - ratio) {
    pos++;
   }
  }

  // read through the rows ..
  for (y = 0, pos = 0, z = 0; y < thumbHeight; y++) {

   // read a single row ..
   image.getRGB(in, 0, sourceWidth, 0, pos, sourceWidth, 1);

   for (x = 0; x < thumbWidth; x++, z++) {
    // write this row to thumbnail
    out[z] = in[cols[x]];
   }

   pos += ratio;
   tmp = pos + (thumbHeight - y) * ratio;
   if(tmp > sourceHeight) {
    pos--;
   } else if(tmp < sourceHeight - ratio) {
    pos++;
   }
  }
  return Image.createRGBImage(out, thumbWidth, thumbHeight, false);
}

This code has low memory need as it only reads one row from the source image each time. And it is super fast (I mean unbelievable). If anyone argue at this point or think there is a way to make it more optimized then welcome :) ..