Dec 03

Javascript Domless events

Using jQuery or nodejs makes you addicted to pubsub (publish/subscribe) model.
Nodejs has nattive support for event emitters but javascript doesn’t have. Events are only binded to a DOM, using which doesn’t make sense and its also adds to performance cost.

To solve the problem i wrote a custom class which implements `on` and `emit` method, Yes some methods like `once` are missing.

var tikEvent = function () {
    this.listeners = {};

};
tikEvent.prototype.on = function (event, callback) {
    if (!this.listeners.hasOwnProperty(event)) {
        this.listeners[event] = [];
    }
    this.listeners[event].push(callback);
};

tikEvent.prototype.emit = function (event) {
    if (this.listeners.hasOwnProperty(event)) {
        var args = Array.prototype.slice.call(arguments);
        args.shift();

        for (var i = 0; i < this.listeners[event].length; ++i) {
            try {
                this.listeners[event][i].apply(null, args);
            } catch (e) {
               console.error(e);
            }
        }
    }
};


//Some test code.

vat test = new tikEvent();
test.on('emitted',function(d){

console.log(d);
}

test.emit('emitted',"test");

Nov 23

Converting Opensource Dictionary to JSON

Folddoc provide free online computer dictionary but they don’t have any standard tool or format to import dictionary in database or so.

Dictionary can be downloaded form http://foldoc.org/Dictionary.txt

Run the following nodejs code it will dump dictionary to OUTPUT.json and don’t forget to remove initial lines of disclaimer or they will add to your dictionary.

var fs  = require("fs");
var words=[];
var word,meaning="";
fs.readFileSync('./Dictionary.txt').toString().split('\n').forEach(function (line) { 
	if(line.length==0){
		meaning+="\n";
		return;
	}
    if(line.indexOf("\t")==0)
		meaning+=line.trim()+"\n";
	else{
		if(meaning.length>0)
			words.push({title:word,definition:meaning.trim()});
		word = line.trim();
		meaning = "";
	}
    
});

fs.appendFileSync("./output.json", JSON.stringify(words));

if you want to import to mongo use `mongoimport –db mad –collection foldoc –file foldoc.json –jsonArray`

At time of writing this article it had 15093 terms but importing resulted in 15110 documents, this might be because of some false positive.

Nov 18

Importing large CSV file to Mongo.

Someday you will need to import a large file to mongo or some other DBMS ,
You start writing code for achieving same using NodeJS, but when you run memory usage start increasing and either core of your CPU is at 100%.

Continue reading

Oct 30

Getting whatsapp statistics – WhatsappWeb

There are no tool available for whatsapp which give us how many messages a user has contributed to group, Sometimes you need.

Whatsapp web exposes whatsapp to JS, whiche makes things more easy,
I wrote a simple code which logs Number of messages sent by each user of group, obviously you need full conversation open in web, You can use Home Key to do that,

The code was written in few minutes so it doesn’t have proper validations or automation,

But is good for purpose.
Open whatsappWeb and after opening complete conversation of group execute the following code in Google Chrome – Developer Console

var v = document.querySelectorAll('.msg.msg-group .text-clickable');
var t, count ={};
for(var i=0,l=v.length;i<l;++i){
t = v[i].textContent.trim();
 if(typeof count[t]=== 'undefined')
	count[t]=0;
 ++count[t];
}

console.log(count);

Oct 10

Redis Vs MongoDB

Redis and Mongodb are considered to solve a common problem that “There is something wrong with RDBMS”
So what is difference ?

  1. Redis is stored in memory and swapped with disk, So data is not lost and you can enjoy performance of your RAM. (Not to be confused with Memcached it is memory only database) whereas Mongodb is disk only database.
  2. Redis can store many type of object strings, hashes, lists, sets, sorted sets whereas Mongo can only store key value pair in form of document (mongo is schemaless).
  3. Redis is hard to learn whereas mongo is easier to understand and get started.
  4. Redis has no support for clustering (Clustering support has been added since version 3.0) whereas mongo has inbuilt clustering support.

Oct 01

Why not nodejs v4.1.1

Recently i updated nodejs installation to v4.1.1 using 

curl -sL https://deb.nodesource.com/setup_4.x | sudo -E bash -
sudo apt-get install -y nodejs

and i ended up fixing many bug in other modules, but i couldn’t move far in this decision.

Nodejs has a history of breaking compatibility with older code in every new release, so if you have project with multiple dependencies don’t upgrade you nodejs version to latest, Instead use NVM to test your code for compatibility in different versions.

Sep 28

Backup Server Configurations to Git

When you have multiple servers it pain to remember every configuration and it may take hours to configure servers again incase you need.
It is also impractical to copy each files.

There are few tools available but they come with overhead attached moreover its fun to write custom solutions 🙂

1. Create directory
2. cd into it.
3. Initialize git
4. Add a remote
5. Create branch specifically for that server
6. Check out branch
7. Add shell script
8. Make a commit and push to server.

mkdir backup
cd backup/  
git init
git remote add [email protected]:tikaj/ConfigurationBackups.git
git branch $(hostname)
git checkout $(hostname)

#This file contains space delimited configuration file name and path relative to git.
touch ConfigurationFiles.list

vi backup.sh

Content of backup.sh

#!/bin/bash
BASEDIR=$(dirname $0)

cd $BASEDIR

while read line; do 
 IFS=' ' read -a A <<< "$line"
 cp ${A[0]} $BASEDIR/${A[1]} 
done < ConfigurationFiles.list


#Add new files and commit changes
git add --all :/
git commit -m "Update configuration files."
git push origin  $(hostname)

Then, Don't forget to generate SSH key adding key to your git server. 

git push -u origin  $(hostname)

Now i can either setup a cron job or add command to daily backup script to update configuration files.
using 

ssh [email protected] "bash /root/backup/backup.sh"

from remote.

Sep 12

Cloning Server to New Hardware

You can find many ways of cloning old server to new hardware, create diskimage and move to new hardware and if you have identical hardware use `dd` or similar tool to copy disks block by block, but if you have new hardware and want to move server without downtime and with no configuration change there is nearly no solution.

I used vestacp, ejabberd in my servers so i had to make changes even after moving some files, as IP changes every where but it is still far better solution than installing and configuring each package manually.

We used rsync to copy whole server to new server except following files
because on new network you will never wish to propagate old network configuration,device specific configurations, installation specific configs.

/boot/
/lib/modules
/etc/fstab
/etc/mtab
/etc/modules
/proc
/dev
/etc/network/interfaces
/root

Create a file with following directories and then execute `rsync` with `exclude-from`.

rsync will Synchronize all your files from remote to local.

Sep 10

Adding ellipsis to long text

When you have some long division and you don’t want to wrap text or extend size of div, But instead you want to use add few `…` at end of text using CSS

.wrapped{
overflow:hidden;
white-space: nowrap;
text-overflow: ellipsis;
width: 300px;
}

Example :

Text without text-overflow :Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla id sem vehicula, faucibus leo id.

Text with text-overflow : Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla id sem vehicula, faucibus leo id.

Sep 06

Monitoring Website for Downtime

Sometimes our in house server go down, But at production level we cannot take risk of 1min downtime, so we were supposed to monitor our server for downtime.
Although there are many cheap services available but i decided to do it my own way, I can use this script in my RPI to monitor even my internal services.

Below is python code, Which sends email if a server goes down and when it comes back online. Obviously you can add code to get more statistics like Logging to file, average response time, Monitoring page changes, Downtime etc.

import httplib
import time
from smtplib import SMTP

def send_email(to,subject,body,SERVER="smtp.gmail.com",PORT=587,USERNAME="[email protected]",PASSWORD="YourPassword"):
	smtp = SMTP(SERVER, PORT)
	smtp.ehlo()
	smtp.starttls()
	smtp.login(USERNAME, PASSWORD)
	headers = "\r\n".join(["from: " + USERNAME,
                       "subject: " + subject,
                       "to: " + to,
                       "mime-version: 1.0",
                       "content-type: text/html"])
                    
	content = headers + "\r\n\r\n" + body
	smtp.sendmail(USERNAME, to, content)



def get_status_code(host,path="/"):
	try:
		conn = httplib.HTTPConnection(host,timeout=10)
		conn.request("HEAD",path)
		return conn.getresponse().status
	except StandardError:
		return 0


def get_site_status(host):
	start = time.time()
	status = False
	code = get_status_code(host)
	#if found, temp redirect, permanent redirect.
	if code in (200,302,301):
		status = True
	return status, time.time()-start,code


def check_internet():
	return get_site_status("google.com")

def check_status():
	toMonitor = ["tikaj.com"]
	down = []
	email = ""
	while True: 
		for value in toMonitor:
			st = "Checking %s..." %value

			#give enough space so that status code can be idented.
			while(len(st)<40):
				st += " "
			print st,

			status,rtime,code = get_site_status(value)
			print "[%s]" %code,

			if status==False :
				if check_internet()[0] == True :
					print "[Server down]"
					
					#if not present in Server down list. Add for email and add to down list.
					if value not in down:
						down.append(value)
						email += st + "<b>[SERVER_DOWN]</b><br/>"
				else:
					print "[Internet Fail]"
			else: 
				# if in down list compose mail that server is up.
				if value in down:
					email += st + "<b>[%fs]</b>" %rtime
					down.remove(value)
				print "[%fs]" %rtime


		if(len(email)>10):

			print "Sending alert email..."
			send_email("[email protected]","ServerDown!","<pre>%s</pre>" %email)
			email = ""

		time.sleep(60) #sleep for 60s

if __name__ == '__main__':
	check_status()

After saving code to server use cron job to start run script on every boot.

Edit crontab using `sudo crontab -e` and then add following line to file.

`@reboot screen -dmS monitor python /root/monitor.py &> /dev/null`

Above command starts a `screen` session named monitor and forwards error of screen to null.
Incase you want to see output of monitor.py use `screen -r monitor`.
You can also add it simply without screen like
`@reboot python /root/monitor.py &`

If you have some suggestions comment.