Summary of Algorithms

This page is moved to [github](https://github.com/kamanashisroy/algosnippet/blob/master/algorithms_summary.md). There I am doing bug-fixes and adding with new things that I learn.

JavaScript will evolve faster as a language than a compile target

Author:Kamanashis Roy
Note: I wrote this blog on request

JavaScript survived and thrived to be one of the most popular language for web development. There was concern over the best-practice and performance issue in JavaScript. Some people went further to say JavaScript language is Buggy. But it served it’s purpose as client-side scripting.

The main criticisms against JavaScript were,
- JavaScript has fragmented implementation( because of the lack of standardization and the browser-war.)
- Standardization that resulted JavaScript to be compiler target rather than a language itself.
- JavaScript is slow.

The criticism against the JavaScript gave rise to compilers that target it. These source-to-source compilers generate acceptable JavaScript code. One of them being Google Web Toolkit. It generates JavaScript from Java. The idea is to impose the best practices. Again after standardization and wide-spread acceptance of JavaScript there have been development in source-to-source compilers that target JavaScript. But JavaScript eventually is more popular for web-development than those compiled language. Let’s see how JavaScript got away with the criticisms.

Fragmented implementation and lack of standardization

The browser-war gave rise to reversed-engineered implementations of JavaScript. And individual programmers could not effort time to write JavaScript for each browser. So some people preferred to generate automated code for different browsers using compilers.

To solve the problem Netscape allowed it to be standardized as ECMAScript. Now there are multiple open-source implementations of JavaScript. And the most popular of them uses Just-in-time compilation. So it provides better opportunity to write uniform code for the developers. Again there are frameworks like jquery that wrap the platform depended code to give an uniform API. Finally the fragmentation is not a problem anymore.

Standardization that resulted javaScript to be compiler target

Again the cross-browser compatibility of javaScript gave the rise of source-to-source compilers (such as Emscripten) that target asm.js and NaCl. The asm.js is precompiled(ahead-of-time compilation) javaScript that allow the browser to run asm.js based application much faster.

JavaScript is slow

JavaScript performance is affected for varied reason. One of them being the support for dynamic type. Another being the dynamic binding. There are ways to get away with this things. The V8 engine actually uses JIT compiler that compiles the javaScript into native code on runtime, instead of interpreting. And the V8 developers compare the JavaScript performance with C/C++. They have some best practice suggestions about JavaScript. The idea boil down to ,

• not to use undefined rather use static types,
• to avoid arrays of varying types,
• finally to use less constructors and initialize all the variables in the constructors.

The poor coding can be identified by some static analysis. And these tools can be integrated in the IDE and browser. This way the programmers get notified of the performance issues.

Frameworks are the right way

Now there are some emerging trends of development regarding JavaScript. It is done by some frameworks and runtimes. Some of them are,

• jquery(It is more like library than a framework with easy-to-use API with a lot of open-source modules),
• prototypejs,
• threeJS,
• reactJS,
• backboneJS
• nodeJS(Not a framework but a runtime for server-side-javascript and desktop application, it features a lot of packages/modules as npm),

Some of them are domain specific and some are for general use. These frameworks feature popular programming patterns for object based language. They allow us to use those well-written patterns. It makes the user code small. It is like “write less do more”. Less code gives less bugs and may be less performance hazards. In that sense less is good. Again these frameworks allow us to write browser independent code. These frameworks also take advantage of the ahead of time compilation. This is why they are immensely popular.

A note on nodeJS

NodeJS is the JavaScript runtime that works on the vast range of platforms. It uses event-driven approach of concurrent programming which is closely relates to observer-pattern that is popular in Object-Oriented paradigm. There were a lot of issue dealing the concurrency and scalalibity in server-side scripting. NodeJS was keen to solve the problem in event-driven way. And as JavaScript is already popular in the web-development arena, NodeJS got huge success. It is supported by big companies. And there is an package-ecosystem, npm, based on nodeJS.

Again there are a lot of frameworks that work on top of NodeJS. Some of them being Express.js, Socket.io and Koa.js. And there is popular Atom IDE(runs over ioJS/(nodeJS variant) runtime) that support development based on NodeJS and JavasCript.

Future

As a C developer, I want to share that I like C so much. But I also knew that it contains a lot of blotted code. It makes things harder because it does not have a memory manager. But one intriguing thing about C is the portability as the ability to compile it in wide range of platforms, in the same way JavaScript has portability in the browser.

I worked on those things and I wrote a compiler that eventually generates C code out of Object-Oriented garbage collected code. And I grew some framework to support server-side development. I targeted C the same way that the source-to-source compilers target JavaScript.

But sometimes I get back to C for some reason. I discovered that my framework written in C also support doing cool things with small dependable code. I think my compiled-framework gave me greater understanding of cool stuff in C.

I think the compilers targeting the JavaScript are also giving new directions for JavaScript based development. JavaScript is now getting developer attractions by featuring runtimes, frameworks and IDEs. In fact there are a lot of packages available on these frameworks. Now they are also being used as desktop tools(GUI), command-line application and server-side scripting.

Memory profiler for shotodol framework

C programming language does not come up with garbage collector. It is not object oriented. But C is inevitable. C programs are portable into different platform. A lot of libraries and server applications are written in C. The Aroop compiler gives the developer flexibility to write object oriented Vala code that can generate C code. Likewise it is also possible to use existing C library in Vala code.

So Aroop is the source-to-source compiler. It compiles Vala code into C. The Aroop compiler is a fork of original Vala compiler. The Vala compiler generates code that uses the GObject library to represent the Object data. On the contrary the Aroop uses object pools.

Shotodol is an application framework based on Aroop compiler. It has instrumentations for memory profiling, dependency injection and more. In this blog we will disect the memory profiling feature.

The profiler can give the memory usage of specific module, it can dump existing string allocations and more.

help prof
Executing:help prof
<            help> -----------------------------------------------------------------
SYNOPSIS:
    profiler
         -heap          <none>  Show all the memory allocated in all the factories
       -string          <none>  Dump the string buffers
       -object          <none>  Show the objects
       -module          <text>  Select/filter out a module
<      Successful> -----------------------------------------------------------------

For example, if we investigate the memory usage of “core/console” module, we get the following.

prof -module core/console
Executing:prof -module core/console
<        profiler> -----------------------------------------------------------------
Source               Line  Module     -- pools      allocated  used       objects    slots      bitstring  pool size 
vsrc/ConsoleCommand.  2116 core/conso --          0          0          0          0          0         56         16
vsrc/ConsoleCommand.   954 core/conso --          0          0          0          0          0         56         16
vsrc/ConsoleCommand.   804 core/conso --          1      13792       8560         10         10         56         16
13792 bytes total, 8560 bytes used
<      Successful> -----------------------------------------------------------------

The table above says the line number where the memory pools are created. “vsrc/ConsoleCommand.c” is the generated C source. The pool allocates 16(it is set when the pool is created) slots at a time. Each slot size is assigned when the pool is created. The first two rows say that the pool is empty. The last row says that it contains 10 objects taking total 8KB space while the pool has 13KB(16 slots) allocated. And it says there is only one pool.

The profiling helps to set the optimal pool size. In optimal condition the pool does not waste too much unused memory and there is less system-call for malloc for pool creation.

It also identifies the memory leak or circular refernce that can invalidate garbage collection. This feature is particularly useful in the server application. In that scenario memory-leak is not desirable at all.

I hope to describe other shotodol features in the future. I just like to explain what the cool things it does.

Line of code

It is nice to have some numbers associated with my identity. I like the numbers and I like to define myself. As a part of that I did a project of know “which language you are”.

I admire the cloc application. It calculates the line of code out of an archive. Line of code gives a number that measure the amount of work. In that sense my work summary is as below.

File	files	blank	comment	code
libsync.txt	241	8649	9998	56467
aroop.txt	256	3561	2506	22042
miniim.txt	88	1757	3824	8572
shotodol.txt	134	574	867	6564
roopkotha.txt	66	737	990	4903
opp_factory.txt	23	517	308	3875
roopkotha_vela.txt	56	302	638	2689
hciplus.txt	29	207	141	2089
barrel.txt	47	270	135	1864
shotodol_net.txt	20	115	161	1454
shotodol_web.txt	20	116	189	1151
shotodol_db.txt	12	74	122	689
shotodol_script.txt	8	34	39	354
shotodol_media.txt	2	12	20	100
SUM:	1002	16925	19938	112813

That is glorious. 112K lines of code is big work. Needless to say, it only calculates major works.

Language	files	blank	comment	code
C	127	8070	7770	58476
Vala	424	3483	3279	27305
C/C++ Header	171	1662	3710	6752
Java	30	830	2867	3868
Vala Header	19	283	245	3387
C++	16	448	589	2959
PHP	46	272	173	1907
CSS	19	372	138	1789
XHTML	26	325	62	1463
JavaScript	13	238	727	1438
make	66	571	50	1423
Lua	25	198	110	1252
Bourne Shell	7	51	93	221
Ant	2	22	28	155
m4	2	28	17	110
Perl	1	32	36	107
Assembly	2	14	41	84
HTML	2	6	0	41
XML	1	0	3	27
Bourne Again Shell	1	9	0	24
NAnt script	1	9	0	23
DOS Batch	1	2	0	2
SUM:	1002	16925	19938	112813

I also wrote small script to generate pie chart out of this data.

The dart experiment

Something to measure, the objective study

The darts game is very simple. But it takes a lot of attention or sagacity or something to perform better. Additionally it has a way to get numbers for measuring. I was trying to measure the something in effect of some activity.

Our world is a chaos

The relation of cause and effect is very hard to associate when our practical world is full of chaos. For example, eating something or doing something may or may not affect the something straight-forward. There are so many things interacting each other.

But there are measurements

I did the experiment for fun. Though I have a vogue intention to find the causality. I kept the records of the events and how they affect the performance of the darts game. The events are given below,

1. Eating rice, cauliflower, soya nugget, olive oil, spinach, orange dish.
2. Eating areca nut.
3. Cognitive task and swimming.
4. Meditation.
5. Rosemary tea.
6. Tea.
7. Super-food.
8. Coding.
9. Song and dance.
10. Late night. (The idea was to find if I am night owl or morning person.)

The result

The result shows that my performance peaked after song and dance. I do not know if the result is something to do with song and dance. I do not know if the effect is reproducible. But I know that there is an effect called the mozart-effect. I assume that did the trick.

Life is not darts game but a guided missile

The something we measure here may or may not have great effect in life. One important thing to note that life is more like guided missile than a simple dart. Once born we have inner power to change our path. And there are a lot more skills than just throwing the dart.

More

The results are affected by the causes that have immediate effects. But the things that take hours, months or more cannot be traced in this way of experiment. It may or may not demonstrate the effects of stimulants like tea or coffee or songs in short-term. I hope to come with more experiments on darts game, especially with the reproducibility of the effects. It is very interesting.

Drupal plugin

Drupal is a PHP content-management framework for web development. The popularity of drupal is evident in the vast community and contributed modules. The popularity comes from the ease of customization and simplicity of design. In this article the plugins of drupal aka the `modules` are analyzed. This is a continuous effort to investigate plugins in different projects.

Drupal modules are grouped into a folder/directory named after that module. The directory contains the following files,

• .info file. The content of the .info file enable us to know about the module without loading it or installing it. These informations are kept in key-value pairs. They contain the name, description, version, package and testing information etc.
• .module file. This is a PHP script containing primarily the hooks that needs to be loaded when the page is hit. To write a drupal module one must write the .module file and write at least one hook function.
• .install file. This file is used only for installation and uninstallation. They may create database or do other tasks that are done at the time of installations. It contain the _install and _uninstall hooks.
• .inc files. These files are loaded on demand. They are PHP sources containing instructions which is not needed in every page hit.

There are other files for testing and for command line operations. But those are not covered here. The cardinal part of the module is in the .module file. It contains function definitions. The function names are identified by the hooks using the PHP reflection feature. For example, a module can add menu by writing the following functions.

function mymodule_menu() {
    $items['abc/def'] = array(
        'page callback' => 'mymodule_abc_view',
    );
    return $items;
}
function mymodule_abc_view() {
    return "Here is the abc content";
}

The above code is for module named mymodule. So the menu hook is the function named mymodule_menu. It is derived by prefixing the module name before _menu. And the return value of the hook is the array containing the menu items and the functions to be called on the menu hit. The ‘abc/def’ is the menu path identifying a page. It appears on the address bar of the browser.

All the _menu hooks can be called and executed by module_invoke_all function. It is possible to call user defined hooks to be populated by other modules.

function module_invoke_all($hook) {
  $args = func_get_args();
  // Remove $hook from the arguments.
  unset($args[0]);
  $return = array();
  foreach (module_implements($hook) as $module) {
    $function = $module . '_' . $hook;
    if (function_exists($function)) {
      $result = call_user_func_array($function, $args);
      if (isset($result) && is_array($result)) {
        $return = array_merge_recursive($return, $result);
      }
      elseif (isset($result)) {
        $return[] = $result;
      }
    }
  }

  return $return;
}

In the code above the function name is created by code $function = $module . '_' . $hook;. So it is conjunction of the module name and the hook name. The function above is responsible to call all the hooks of the loaded modules.

Now let us investigate the blog module.

The purpose of this module

The blog.info file contains the description of the module. It enables blogging for drupal users.

Loading and unloading

The blog.install contains blog_install and blog_uninstall functions which are the _install and _uninstall hooks.

The blog_install() alters the blog node and adds body field for it. The node is the parent object and it has it’s own defined way to save, load and render data. The blog is kind of subclass of the node. Anyway the abstraction of node is not covered here.

The blog_uninstall() just deletes unnecessary variable.

Note that the blog_install() function is called while installing and blog_uninstall() is called while uninstalling. But they are not called in each page hit. While in each page hit the blog.module file is parsed but none of them are executed.

Extension points

The blog_menu function defined in blog.module file is the _menu hook. Here _menu is the extension point. This hook is called when the module is enabled for use. Once called the total menu tree is built and kept in the database. It is not executed in per page hit.

/**
 * Implements hook_menu().
 */
function blog_menu() {
  $items['blog'] = array(
    'title' => 'Blogs',
    'page callback' => 'blog_page_last',
    'access arguments' => array('access content'),
    'type' => MENU_SUGGESTED_ITEM,
    'file' => 'blog.pages.inc',
  );
  $items['blog/%user_uid_optional'] = array(
    'title' => 'My blog',
    'page callback' => 'blog_page_user',
    'page arguments' => array(1),
    'access callback' => 'blog_page_user_access',
    'access arguments' => array(1),
    'file' => 'blog.pages.inc',
  );
  $items['blog/%user/feed'] = array(
    'title' => 'Blogs',
    'page callback' => 'blog_feed_user',
    'page arguments' => array(1),
    'access callback' => 'blog_page_user_access',
    'access arguments' => array(1),
    'type' => MENU_CALLBACK,
    'file' => 'blog.pages.inc',
  );
  $items['blog/feed'] = array(
    'title' => 'Blogs',
    'page callback' => 'blog_feed_last',
    'access arguments' => array('access content'),
    'type' => MENU_CALLBACK,
    'file' => 'blog.pages.inc',
  );

  return $items;
}

Note that the menu has it’s own plugin-system and the extension-points. The blog, blog/%user_uid_optional, blog/%user/feed and blog/feed are the extension-points aka paths of menu-plugin-system. And blog_page_last, blog_page_user, blog_feed_user and blog_feed_last are the hooks registered in those extension-points respectively. This is an added twist in drupal architecture. It decouples the menu-extension-points from the core extension-points. This makes it flexible and separable.

And with added benefits these menu related hooks and callbacks are defined in separate file named blog.pages.inc to make the blog.module slimmer. The blog.pages.inc file is loaded when the related pages are loaded. Thus it reduces the processing time too.

Summary

Drupal project is based on functional programming in PHP. It still affords complex architecture and development needs. The reflection property of PHP function makes the hooking effortless. The analytic minds are tuned to find dumb instruments in the code. Drupal is very compulsive in that sense.

Avoid distractions, practice internet fasting

Civilization is built in free time

Technology is the capstone of human civilization. At the advent of civilization we learned to use fire. We learned to cook. And cooking made the nutritions available to our body in less time and effort. This gave us time to think about other things, like art, culture and technology. We can now save a lot of our time using the technology. We can shop online, we can go places in cars or by trains. These all save our time and give us more time to think about us and to think about other ideas.

Now distractions are eating our time

On the other hand humans build habits, they are prone to classical conditioning. Today we are doped with the connecting technology. In the same way the over-nutrition caused us to become overweight and predisposed to some diseases; the overuse of social networking sites and internet made us occupied with social thoughts. We certainly lost our ability to wait and the ability to pursue a goal for a long time. Our personal tasks are overshadowed by the thoughts of society, politics and news-channels. The internet has made it easy to create and publish more information. Our desire for more online information caused information anxiety. And provided that anxiety we used up a lot of time gathering those informations. This resulted in reduced productive time and increased procrastinations.

Time is short and attention is limited resource

Life is short. This makes our attention limited. So we need to choose what we care most. We need to have a plan. And we need a sense of control. There are people who use the laws of economy to understand attention. And they call it attention economy. I find it very interesting.

Know thyself

In the earlier blog I talked about self profiling. We can use that profiling to measure the outcome of a month. In this way we can find what we are lacking and what we are overusing.

Formulating the solution

As I figured out that I spend a lot of time online, I formulated a simple solution. Too many of things are bad. Like in old days saints abstained themselves from food, we need to practice austerity using internet. It is like internet-fasting. We need to practice a ‘no-internet-day’ to edify our ability to think of ourselves.