IOT Push Button (Like Amazon Dash)

Amazon Dash button is an incredible piece of hardware and another example of “applied engineering” in Amazon, just like Kindle.

The inspiration for this project isn’t directly derived from amazon dash, I wasn’t aware of it until I started digging the internet but at end, I had to set the performance goal to Amazon dash, because no other piece of hardware was up to mark. A detailed teardown of dash button can be found here.

So, Since “decades” I wanted to build some piece of hardware which I could dynamically program for any functionality like playing next youtube video, unlocking door, rebooting a machine, minimizing all open tabs or whatever with a push of a button. The button should be portable and independent of device for operation.

To meet my requirements I had to remove BLE or any other radio-based technology which needed a receiver or additional unit to operate. ESP-12F is power intensive and somewhat large if compared to esp-01.  The end prototype looked like this and worked as expected.

Below is list of components I used.

  • 1x LiPo Battery – 150mAh
  • 1x ESP8266-01
  • 1x 1k Register
  • 1x Tactile Switch Button

You might require a soldering iron with fine tip, basic desoldering skills, access to 3d printer, FTDI board or similar setup.

The schematics of setup would look like below:

The functioning would be like – We turn on ESP using push button, but it would take time to log in to wifi, authenticate – so we need to keep it on for pretty long time – which could be done by using programming output pins, As soon as ESP turn on GPIO2 can be set to HIGH, Once operation is finished it can be set to low. Few challenges which I found on way (with fixes):

  • Power is very limited – So we need to remove LED’s from the esp, this would save a lot of power and would increase battery life by almost a fold!
  • You should use diode and transistor to limit current, since I am not an electronic nerd I would not comment on it.
  • You should set a timeout period, in case you are writing custom logic else battery will drain and you will never come to know why!

After soldering – You have to upload following code which need to be tweaked according to need, but it has basic logic code –

The code can set in hotspot mode if couldn’t connect to wifi and if could connect it would make a connection to mqtt server and make an announcement then shutdown.

I hope you enjoyed the article, it wasn’t a detailed writeup and I dropped many details – but if you have any questions you can comment or drop mail – I will surely help.

Adding a display over network !

You are doomed if your laptop has only one HDMI Port & you are running Linux in that box.

Unlike windows – where there are tons of easy to run solution, & there are still tons of solutions if you have big numbers in your pocket.

Certainly – I neither wish to spend money on external VGA/HDMI extender or docking station nor to change OS. So after googling a lot I discovered a solution which utilized a different machine to act as the streaming client. You can use RPI or an old p4 machine.

Below is how it works.

  1. Adds a virtual display on your machine, usually all graphics card support at least one virtual display.  You do all this using xrandr
  2. Create a VNC Server to stream that display – but since you cannot each time run two commands just to connect display – we are running VNCViewer in listen mode
  3. Connect to vnc viewer & keep running it in the background.

You Laptop —[Display Data]—> Network —-> VNCViewer

As suggested above you data is being streamed over the network  – you cannot run 4k data. But if you have good Lan speed you won’t face any problem.

I have also optimized settings for best experience – so far I can use terminal, watch videos – the only drawback is you feel the lag when you use keyboard or mouse for realtime feedback.

In Client Machine, i.e. your laptop.

You will need to install x11vnc & screen

In server Machine (LAN address 10.0.0.2)

You will need to install VNC Viewer in client side.

Kindle Universal Dashboard

Kindle is Super awesome !

Because of its e-ink display. I wanted a display to present data which has – Least power consumption, Not painful to eyes & obviously one which doesn’t emit blue light.
E-Ink displays fit perfectly to my requirement – acquiring a display which can be driven using Raspberry Pi or Arduino is hard, size/cost ratio is much higher.

On googling I found some display modules which were more or near to 70-80$ – even smaller display – which impulsed me to get a Kindle Touch (8th Generation) at around 74$ approx.

Kindle - MadhurendraKindle comes with an experimental browser but it is narrowed version of WebKit, which is pretty useful if you want to display static content or just want to make a control panel, it can easily render normal websites which use js/CSS & works pretty well. But support for HTML5 is almost absent – so you can’t use WebSockets to push data, using long polling/fast polling is only solutions so far.

Moreover, there was another problem which I had to fix – Kindle has fixed timeout which sends it to sleep mode – for mine it was 10min, after digging I found you can use ~ds or similar but for me, nothing worked.

We can only hope that support to remove timeout or change timeout period will be added in future releases. I think old kindle supports.

If you can’t change timeout or you want to use few other features I suggest you to jailbreak. Follow steps mentioned here http://www.mobileread.com/forums/showthread.php?t=275877 , Don’t jump, It works for kindle touch 8th generation. Tried, tested, working !  For KT3 you will need to install MRPI along with KUAL. Once done your kindle is officially out of warranty 😀 . Post that you need to install USBNet – it will  allow you to ssh to your kindle.  All this will allow you to execute “lipc-set-prop com.lab126.powerd preventScreenSaver 1″ this on the kindle. It will simply disable screensaver. 🙂

Once you have your kindle whose screen doesn’t lock you can simply go & execute  a simple nodejs script to push data.

Note : Kindle doesn’t support WebSocket & none of transport methods in socket.io except “polling”. 

Below is nodejs server code :

Below is code of client.html

Voila – It works !

Below is video of working  in case you want to see demo before getting hands dirty 🙂

Warning & Update : This method might consume more power than expected, as experimental browser has loading status – which continuously refreshes a section of the screen. To overcome this problem I will be polling server with long interval difference – which will be adjustable by the server.

Note for nerds :  Since this method uses browser – it’s more flexible – but if you are possessive about power consumption & screen space – You can use JAVA to develop kindlet application. Lightweight pub/sub protocols like MQTT should help you in the way.


photo_20161030_124622Designing a wall holder : 
You can google for covers or design own or use some double sided foam tape. Since i had access to 3d printer i got two of http://www.thingiverse.com/thing:832273 printed & hung it on the wall – it just helped me in reading few books apart from using it only as display. SWAP!    

Use it as :

  • Scoreboard
  • Notification
  • Weather system
  • Wallpaper slideshow
  • News/RSS feeds display
  • Home automation control
  • Anything
  • Book reader 🙂

At the end, even if you place it behind your monitor it won’t hurt or push the new data to your eyes & spoil the code castle you were building.

Offline Wikipedia with Elasticsearch and MediaWiki

Wikipedia is Awesome ! It’s open, its free – Yea. & its huge in size, millions of articles  but as developer how to exploit the free knowledge.

I started digging internet just to find ways to exploit my fresh 9+ GB of XML Gzipped archive which seemed to me of no use as even a simple text editor can’t open it. (Just out of excitement what’s inside, how its structured, Schema ! )

Luckily people have already imported it. Elasticsearch is fast, reliable & its good for searching, so https://github.com/andrewvc/wikiparse was a saver.

  • Installed elastic search
  • Ran command to import

it took almost 48 hour in an i5, with 8gb ram – where mistake was i used same harddisk for data storage & database. Time might vary.

Data was imported but its still of no use ! Why ? Its in text/wiki format, parses is needed.

After doing search only solution i found was using mediawiki api, which is in PHP there were lots of things missing as its only for mediawiki but not for parsing plain text. (Though i didn’t spend much time in learning internal API)

I quickly downloaded mediawiki, ran nginx with php, installed it & used API.php.
it was good to see my offline API too, but still many things were missing, confusing, API has hard to modify structure. So i created a parse.php

So all steps were :

Maintaining JS Libs – Changing bower path

If you are web developer it becomes hard to manage all js libraries if you aren’t using any package manager.
Few folks created bower to make the job easier. (As far as functionality is concerned, it works same as npm, if you have used.)

Steps for installations are just for linux guys – windows people – DIY.

First you need to install npm sudo apt-get install npm
You can install it by sudo npm install -g bower

Once you have installed bower, You can install a web package, just by executing following command in the directory of project –

bower install jquery

You can even use your own git repository as source or any other url

bower install https://github.com/jquery/jquery.git#2.0.3

Above command will install jquery for your project.

Once you have installed it successfully things will be kept in
bower_components/jquery
you can refer it by
bower_components/jquery/dist/jquery.js. Simple. That’t it ?

No.

if you are like me – possessive about your directory structure – You want everything inside something like assets/lib/jquery/dist/jquery.js, or simply want it inside assets/lib directory instead of too many file/directory in root.

You can create a file in root of project .bowerrc, put following content inside the file
{
"directory" : "assets/lib"
}

Since filename begins with . it will be hidden in file browsers, clean.

Again – do you find these long assets path annoying ? – Grunt to rescue – check http://bower.io/docs/tools/#grunt

Optimal way of scheduling long job – nodejs/js

If you are using nodejs for some backend stuff to schedule some-work, probably you will hate scheduling because of two factors

  1. You can’t set long timeout
  2. Timers are removed when process restart

You would probably favor cronjobs for scheduling – or may be polling database for regular interval. Loading nodejs script every 5min is a costly job – developers understand, so you can’t follow php/short interval scheduling.

First problem can be solved by of setting a recursive call to a function, with time out. Many solutions on web.
But there lies a problem – You can’t cancel the timeout if you needed to after a recursive call. To solve the recursive call problem i wrote few lines of code.

Above code behaves same as setTimeout if you don’t need timer object updates, else you can register for that.

Second problem – Timers are removed when process are restarted. – That shouldn’t be hard – You can create a function to execute atomic functions,
Note atomic – atomic because if your setTimeout code will depend on variables state you will need to load those variables from database which will make job harder – better way is to schedule something like – send email to [email protected] instead of send message to currently loggedin users.

Solution of second problem really depends on your problem – but if you analyze your scheduled jobs closely you will find that they are really atomic, you made them non-atomic !.

if you really need that your jobs are never lost from memory better way is to run a stable nodeserver & use IPC. – but that would be practically hard to maintain.

Javascript Domless events

Using jQuery or nodejs makes you addicted to pubsub (publish/subscribe) model.
Nodejs has nattive support for event emitters but javascript doesn’t have. Events are only binded to a DOM, using which doesn’t make sense and its also adds to performance cost.

To solve the problem i wrote a custom class which implements on and emit method, Yes some methods like once are missing.

Converting Opensource Dictionary to JSON

Folddoc provide free online computer dictionary but they don’t have any standard tool or format to import dictionary in database or so.

Dictionary can be downloaded form http://foldoc.org/Dictionary.txt

Run the following nodejs code it will dump dictionary to OUTPUT.json and don’t forget to remove initial lines of disclaimer or they will add to your dictionary.

if you want to import to mongo use mongoimport --db mad --collection foldoc --file foldoc.json --jsonArray

At time of writing this article it had 15093 terms but importing resulted in 15110 documents, this might be because of some false positive.

Backup Server Configurations to Git

When you have multiple servers it pain to remember every configuration and it may take hours to configure servers again incase you need.
It is also impractical to copy each files.

There are few tools available but they come with overhead attached moreover its fun to write custom solutions 🙂

1. Create directory
2. cd into it.
3. Initialize git
4. Add a remote
5. Create branch specifically for that server
6. Check out branch
7. Add shell script
8. Make a commit and push to server.

Content of backup.sh

Then, Don’t forget to generate SSH key adding key to your git server.

Now i can either setup a cron job or add command to daily backup script to update configuration files.
using

from remote.