linux Archives - MAD Blog

IOT Push Button (Like Amazon Dash)

Amazon Dash button is an incredible piece of hardware and another example of “applied engineering” in Amazon, just like Kindle.

The inspiration for this project isn’t directly derived from amazon dash, I wasn’t aware of it until I started digging the internet but at end, I had to set the performance goal to Amazon dash, because no other piece of hardware was up to mark. A detailed teardown of dash button can be found here.

So, Since “decades” I wanted to build some piece of hardware which I could dynamically program for any functionality like playing next youtube video, unlocking door, rebooting a machine, minimizing all open tabs or whatever with a push of a button. The button should be portable and independent of device for operation.

To meet my requirements I had to remove BLE or any other radio-based technology which needed a receiver or additional unit to operate. ESP-12F is power intensive and somewhat large if compared to esp-01. The end prototype looked like this and worked as expected.

Below is list of components I used.

1x LiPo Battery – 150mAh
1x ESP8266-01
1x 1k Register
1x Tactile Switch Button

You might require a soldering iron with fine tip, basic desoldering skills, access to 3d printer, FTDI board or similar setup.

The schematics of setup would look like below:

The functioning would be like – We turn on ESP using push button, but it would take time to log in to wifi, authenticate – so we need to keep it on for pretty long time – which could be done by using programming output pins, As soon as ESP turn on GPIO2 can be set to HIGH, Once operation is finished it can be set to low. Few challenges which I found on way (with fixes):

Power is very limited – So we need to remove LED’s from the esp, this would save a lot of power and would increase battery life by almost a fold!
You should use diode and transistor to limit current, since I am not an electronic nerd I would not comment on it.
You should set a timeout period, in case you are writing custom logic else battery will drain and you will never come to know why!

After soldering – You have to upload following code which need to be tweaked according to need, but it has basic logic code –

The code can set in hotspot mode if couldn’t connect to wifi and if could connect it would make a connection to mqtt server and make an announcement then shutdown.

#define DEVICE_ID         "YOUR_USERNAME"
#define DEVICE_NAME       "iot-" DEVICE_ID
#define WIFI_SSID         "mad-" DEVICE_ID
#define WIFI_PASSWORD     DEVICE_ID
#define MQTT_SERVER       "YOUR_SERVER"
#define MQTT_USERNAME     DEVICE_ID
#define MQTT_PASSWORD     "YOUR_PASSWORD"
#define MQTT_SEND_CHANNEL "pushbutton"
#define MQTT_RECV_CHANNEL DEVICE_NAME




#define BUTTON_MODE 1
#define USE_SSL 1
#define GPIO2 2
#define BUTTON_TIMEOUT 30000


#include <ESP8266WiFi.h>
#include <WiFiClient.h>
#include <ESP8266WebServer.h>
#include <ESP8266mDNS.h>
#include <PubSubClient.h>
#include <DNSServer.h>
#include <WiFiManager.h>
#include <ArduinoJson.h>
#include <stdarg.h>


#if USE_SSL
#   include <WiFiClientSecure.h>
//    WiFiClientSecure client;
#   define PORT 443
#else
// WiFiClient client;
#   define PORT 80
#endif


void reconnect();
void sendMessage(String action, String data, char* num = "", ...);
char* stringToChar(String str);

WiFiClient espClient;
PubSubClient client(espClient);
WiFiManager wifiManager;


long lastMsg = 0;
char msg[50];
int value = 0;
String DEVICE_FEATURES = "";

void setup() {

  DEVICE_FEATURES += "pushbutton;";
  pinMode(GPIO2, OUTPUT);
  digitalWrite(GPIO2, HIGH);


  //Serial.begin(9600);

  //  wifiManager.resetSettings();
  if (!wifiManager.autoConnect(WIFI_SSID, WIFI_PASSWORD)) {
    //Serial.println("failed to connect, we should reset as see if it connects");
    delay(1000);
    ESP.reset();
    delay(1000);
  }
  client.setServer(MQTT_SERVER, 1883);

}
void loop() {

  //if button mode, & time since start is greater than TIMEOUT close the time.
  if (millis() > BUTTON_TIMEOUT)
    digitalWrite(GPIO2, LOW);
  //if MQTT Client not connect connect it back.
  if (!client.connected())
    reconnect();
  client.loop();

}



/**
   Send message to server
   example: sendMessage("hello", DEVICE_NAME,"ksks","mac",getMacAddress(),"localIP",WiFi.localIP().toString());
   k for key
   d : double, f : float, s : string, l : long,
   No char type
*/
void sendMessage(String action, String data, char *types, ...) {
  StaticJsonBuffer<200> jsonBuffer;
  JsonObject& root = jsonBuffer.createObject();
  root["action"] = action;
  root["data"] = data;

  //find length
  int count = 0;
  while (types[count++] != '\0');
  --count;

  //if not even args
  if (count % 2 != 0)
    return;

  //for argument parsing.
  va_list arguments;
  va_start ( arguments, types );

  String key;

  //even length validated already
  for (int i = 0; types[i] != '\0'; i += 2) {
    //this is intentionally done, default key as type k can be used, but developer might mistake, causing lot of debugging
    if (types[i] != 'k')
      continue;
    key =  va_arg ( arguments, char *  );

    switch (types[i + 1])
    {
      case 'd': root[key] = (va_arg(arguments, int));
        break;
      case 'l': root[key] = (va_arg(arguments, long));
        break;
      case 'f': root[key] = (va_arg(arguments, double));
        break;
      case 's': root[key] = (va_arg(arguments, char *));
        break;
      default:  ;
    };
  }

  va_end ( arguments );                  // Cleans up the list

  char tmp[root.measureLength() + 2];
  root.printTo(tmp, sizeof(tmp));
  client.publish(MQTT_SEND_CHANNEL, tmp);
}



void reconnect() {
  // Loop until we're reconnected
  while (!client.connected()) {
    //Serial.print("Attempting MQTT connection...");
    // Attempt to connect
    if (client.connect(DEVICE_NAME, MQTT_USERNAME, MQTT_PASSWORD)) {
      //Serial.println("connected");
      // Once connected, publish an announcement...
      sendMessage("hello", DEVICE_NAME, "ksks", "features", (DEVICE_FEATURES).c_str() , "localIP", (WiFi.localIP().toString().c_str()));
      digitalWrite(GPIO2, LOW);
    } else {
      //Serial.print("failed, rc=");
      //Serial.print(client.state());
      //Serial.println(" try again in 5 seconds");
      // Wait 5 seconds before retrying
      delay(2000);
    }
  }
}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

#define DEVICE_ID "YOUR_USERNAME"

#define DEVICE_NAME "iot-" DEVICE_ID

#define WIFI_SSID "mad-" DEVICE_ID

#define WIFI_PASSWORD DEVICE_ID

#define MQTT_SERVER "YOUR_SERVER"

#define MQTT_USERNAME DEVICE_ID

#define MQTT_PASSWORD "YOUR_PASSWORD"

#define MQTT_SEND_CHANNEL "pushbutton"

#define MQTT_RECV_CHANNEL DEVICE_NAME

#define BUTTON_MODE 1

#define USE_SSL 1

#define GPIO2 2

#define BUTTON_TIMEOUT 30000

#include <ESP8266WiFi.h>

#include <WiFiClient.h>

#include <ESP8266WebServer.h>

#include <ESP8266mDNS.h>

#include <PubSubClient.h>

#include <DNSServer.h>

#include <WiFiManager.h>

#include <ArduinoJson.h>

#include <stdarg.h>

#if USE_SSL

# include <WiFiClientSecure.h>

// WiFiClientSecure client;

# define PORT 443

#else

// WiFiClient client;

# define PORT 80

#endif

void reconnect();

void sendMessage(String action, String data, char* num = "", ...);

char* stringToChar(String str);

WiFiClient espClient;

PubSubClient client(espClient);

WiFiManager wifiManager;

long lastMsg = 0;

char msg[50];

int value = 0;

String DEVICE_FEATURES = "";

void setup() {

DEVICE_FEATURES += "pushbutton;";

pinMode(GPIO2, OUTPUT);

digitalWrite(GPIO2, HIGH);

//Serial.begin(9600);

// wifiManager.resetSettings();

if (!wifiManager.autoConnect(WIFI_SSID, WIFI_PASSWORD)) {

//Serial.println("failed to connect, we should reset as see if it connects");

delay(1000);

ESP.reset();

delay(1000);

}

client.setServer(MQTT_SERVER, 1883);

}

void loop() {

//if button mode, & time since start is greater than TIMEOUT close the time.

if (millis() > BUTTON_TIMEOUT)

digitalWrite(GPIO2, LOW);

//if MQTT Client not connect connect it back.

if (!client.connected())

reconnect();

client.loop();

}

/**

Send message to server

example: sendMessage("hello", DEVICE_NAME,"ksks","mac",getMacAddress(),"localIP",WiFi.localIP().toString());

k for key

d : double, f : float, s : string, l : long,

No char type

void sendMessage(String action, String data, char *types, ...) {

StaticJsonBuffer<200> jsonBuffer;

JsonObject& root = jsonBuffer.createObject();

root["action"] = action;

root["data"] = data;

//find length

int count = 0;

while (types[count++] != '\0');

--count;

//if not even args

if (count % 2 != 0)

return;

//for argument parsing.

va_list arguments;

va_start ( arguments, types );

String key;

//even length validated already

for (int i = 0; types[i] != '\0'; i += 2) {

//this is intentionally done, default key as type k can be used, but developer might mistake, causing lot of debugging

if (types[i] != 'k')

continue;

key = va_arg ( arguments, char * );

switch (types[i + 1])

{

case 'd': root[key] = (va_arg(arguments, int));

break;

case 'l': root[key] = (va_arg(arguments, long));

break;

case 'f': root[key] = (va_arg(arguments, double));

break;

case 's': root[key] = (va_arg(arguments, char *));

break;

default: ;

};

}

va_end ( arguments ); // Cleans up the list

char tmp[root.measureLength() + 2];

root.printTo(tmp, sizeof(tmp));

client.publish(MQTT_SEND_CHANNEL, tmp);

}

void reconnect() {

// Loop until we're reconnected

while (!client.connected()) {

//Serial.print("Attempting MQTT connection...");

// Attempt to connect

if (client.connect(DEVICE_NAME, MQTT_USERNAME, MQTT_PASSWORD)) {

//Serial.println("connected");

// Once connected, publish an announcement...

sendMessage("hello", DEVICE_NAME, "ksks", "features", (DEVICE_FEATURES).c_str() , "localIP", (WiFi.localIP().toString().c_str()));

digitalWrite(GPIO2, LOW);

} else {

//Serial.print("failed, rc=");

//Serial.print(client.state());

//Serial.println(" try again in 5 seconds");

// Wait 5 seconds before retrying

delay(2000);

}

I hope you enjoyed the article, it wasn’t a detailed writeup and I dropped many details – but if you have any questions you can comment or drop mail – I will surely help.

Cheap BLE : JDY-08

Disclaimer : This post is based on my experience which is completely subjective, in no way I am promoting any application or product 😉

I have been trying to get my hands into BLE4.0 modules since long. As always I was looking for small, cheap solution, JDY-08 was one of solution I came across – it seemed just Perfect !

It is based on TI-CC2541 SOC, Low power consumption – long range Voila!

Below is what is being advertised by most vendors:

maximum transmit power of 0db, the maximum transmission distance of 80 meters.
support serial allowing users to modify the device name through AT commands service UUID, transmit power, pairing password instructions, convenient to use and flexible.
Ultra-low standby power consumption 400uA ~ 800uA
Ultra-long connection distance 100 ft / 60 m
Fast reaction speed 0.4 secondsFor Apple, AndroidReceive no byte limit, up to 6K Bytes/sec.

I quickly ordered 10pcs without a second thought.

Below is what you should keep in mind:

Datasheet, documentation – everything is in Chinese! Cool !
Luckily, Few folks did translation – https://docs.google.com/document/d/14mHWT3GhELCj-6yxsam0k5bjzazq8nnPoz4B_gYh04k
Also, you can check http://forum.arduino.cc/index.php?topic=432074.0 because this is the only helpful resource in the web.
Firmware is broken – at least for me, some other user confirmed same in above Arduino thread.
No other support.
Non-standard board pitch, you cannot use header pins

But it is good in many ways:

Small size, thin, light, small – cool!
If you use HC-05 or HC-06 for transparent communication, you can use GATT terminal for transparent communication.
Very low power consumption – from my tests, I found 0.5ma consumption.
Long range – upto 30meters without optimization.
iBeacon mode – works fine – but power consumption was 0.5ma

I was unable to test as I could only use GATT write & notify methods, AT commands to change mode & device name, even AT+RST returned ERR.

You can use following tools to test :

Linux users can use this tuto -> https://github.com/pcborenstein/bluezDoc/wiki/hcitool-and-gatttool-example
Android – for iBeacon – “Beacon Scanner“
Android – GATT test – “BleTerm” or “BLE Monitor”

Below is schematic I followed:

Adding a display over network !

You are doomed if your laptop has only one HDMI Port & you are running Linux in that box.

Unlike windows – where there are tons of easy to run solution, & there are still tons of solutions if you have big numbers in your pocket.

Certainly – I neither wish to spend money on external VGA/HDMI extender or docking station nor to change OS. So after googling a lot I discovered a solution which utilized a different machine to act as the streaming client. You can use RPI or an old p4 machine.

Below is how it works.

Adds a virtual display on your machine, usually all graphics card support at least one virtual display. You do all this using xrandr
Create a VNC Server to stream that display – but since you cannot each time run two commands just to connect display – we are running VNCViewer in listen mode
Connect to vnc viewer & keep running it in the background.

You Laptop —[Display Data]—> Network —-> VNCViewer

As suggested above you data is being streamed over the network – you cannot run 4k data. But if you have good Lan speed you won’t face any problem.

I have also optimized settings for best experience – so far I can use terminal, watch videos – the only drawback is you feel the lag when you use keyboard or mouse for realtime feedback.

In Client Machine, i.e. your laptop.

You will need to install x11vnc & screen

#!/bin/bash
#Run VNC server in remote device. 10.0.0.2 is my machine which has monitor connected.
ssh  root@10.0.0.2 "nohup /root/vncserver.sh > /dev/null 2>&1 &"
xrandr --addmode VIRTUAL1 1920x1080
xrandr --output VIRTUAL1 --mode 1920x1080 --right-of eDP1 # find your main display
screen -X -S vnc quit
killall x11vnc
# adjust postion 1920x1080+3511+180, i.e. 3511+180 accordingly
# i have different resultion display so had to move.
screen -dmS vnc  x11vnc -connect 10.0.0.2:5500  -display :0 -clip 1920x1080+3511+180 -wirecopyrect  -viewonly --nossl -ncache 10 -ncache_cr

#!/bin/bash

#Run VNC server in remote device. 10.0.0.2 is my machine which has monitor connected.

ssh root@10.0.0.2 "nohup /root/vncserver.sh > /dev/null 2>&1 &"

xrandr --addmode VIRTUAL1 1920x1080

xrandr --output VIRTUAL1 --mode 1920x1080 --right-of eDP1 # find your main display

screen -X -S vnc quit

killall x11vnc

# adjust postion 1920x1080+3511+180, i.e. 3511+180 accordingly

# i have different resultion display so had to move.

screen -dmS vnc x11vnc -connect 10.0.0.2:5500 -display :0 -clip 1920x1080+3511+180 -wirecopyrect -viewonly --nossl -ncache 10 -ncache_cr

In server Machine (LAN address 10.0.0.2)

if [[ $(ps -ef | grep -c vncviewer)  -ne 1 ]];
then
    echo "Running"
else
   export DISPLAY=:0.0;
    vncviewer -listen 0 -fullscreen -owncmap -viewonly;
fi

if [[ $(ps -ef | grep -c vncviewer) -ne 1 ]];

then

echo "Running"

else

export DISPLAY=:0.0;

vncviewer -listen 0 -fullscreen -owncmap -viewonly;

You will need to install VNC Viewer in client side.

Just Another DIY “Smart” Watch

Note : This article was written after project was partially completed, so i don’t have better images.

There are hundred’s of projects for building “Smart watch using arduino”, I have curated a list of better projects at end of this article. I didn’t plan to copy any of given projects but at end it turned out that world is a big place. So this blog post is my effort in developing “My smartwatch”.

Hardware :

Arduino Pro mini
Bluetooth module (HC-05)
OLED Module (SSD1306)
LiPo Battery (110 mAh)
Thin copper wire, single strand if possible.
FTDI Breakout board or any way to program Arduino pro mini.
Micro USB female pins
Membrane Tape

Tools :

Soldering iron – along with soldering wire, flux, etc.
3d Printer/Good Crafting Skills to make a case
Working brain ! (Recommended)

Software :

Android Phone (BTTerm) /Linux System(minicom, rfcomm), Windows Phone, Windows *, Mac OS Users are on their own but you can search for “Bluetooth serial terminal”
Arduino IDE
Common Sense ! (Recommended)

The guide here is definitely not a step by step guide, it just tells the flow (an algorithm not code).

Desolder pins in your OLED display. Yea, all header pins.. for noobs like me it’s a tough task.
Desolder pins of HC-05/HC06 if it has, if you got module without header, skip
Connect all GND, VCC pins of arduino, hc-05, oled Easy ?
Solder oled pins SCL -> A5, SDA -> A4 to arduino.
Solder hc-05 pins TX -> 10, RX -> 11 to pro mini.
Then Solder VCC with a switch (Any you prefer) & to battery.
Upload the code
Connect to HC-05 using phone & send any data.
HC-05 Soldered

Click here to watch working video.

Below is arduino code :

#include "U8glib.h"

U8GLIB_SSD1306_128X64 u8g(U8G_I2C_OPT_NO_ACK);	// Display which does not send AC

#include <SoftwareSerial.h>
// software serial #1: RX = digital pin 10, TX = digital pin 11
SoftwareSerial bluetooth(10, 11);
#define buflen 10
 char str[buflen+1];
 int strCount=0;


// setup input buffer
#define LINE_MAX 30 
uint8_t line_buf[LINE_MAX] = "Booting...";
uint8_t line_pos = 0;

// setup a text screen to support scrolling
#define ROW_MAX 12


uint8_t screen[ROW_MAX][LINE_MAX];
uint8_t rows, cols;

// line height, which matches the selected font (5x7)
#define LINE_PIXEL_HEIGHT 7

// clear entire screen, called during setup
void clear_screen(void) {
  uint8_t i, j;
  for( i = 0; i < ROW_MAX; i++ )
    for( j = 0; j < LINE_MAX; j++ )
      screen[i][j] = 0;  
}

// append a line to the screen, scroll up
void add_line_to_screen(void) {
  uint8_t i, j;
  for( j = 0; j < LINE_MAX; j++ )
    for( i = 0; i < rows-1; i++ )
      screen[i][j] = screen[i+1][j];
  
  for( j = 0; j < LINE_MAX; j++ )
    screen[rows-1][j] = line_buf[j];
}

// U8GLIB draw procedure: output the screen
void draw(void) {
  uint8_t i, y;
  // graphic commands to redraw the complete screen are placed here    
  y = 0;       // reference is the top left -1 position of the string
  y--;           // correct the -1 position of the drawStr 
  for( i = 0; i < rows; i++ )
  {
    u8g.drawStr( 0, y, (char *)(screen[i]));
    y += u8g.getFontLineSpacing();
  }
}

void exec_line(void) {
  // echo line to the serial monitor
  Serial.println((const char *)line_buf);
  
  // add the line to the screen
  add_line_to_screen();
  
  // U8GLIB picture loop
  u8g.firstPage();  
  do {
    draw();
  } while( u8g.nextPage() );
}

// clear current input buffer
void reset_line(void) { 
      line_pos = 0;
      line_buf[line_pos] = '\0';  
}

// add a single character to the input buffer 
void char_to_line(uint8_t c) {
      line_buf[line_pos] = c;
      line_pos++;
      line_buf[line_pos] = '\0';  
}


void setup(void) {
  Serial.begin(9600);
  bluetooth.begin(9600);
  Serial.print("Hello");
//  bluetooth.println("Hi");
    
//     u8g.firstPage();  
//     u8g.drawStr( 0, 0, "Booting..");
// set font for the console window
  u8g.setFont(u8g_font_5x7);
  //u8g.setFont(u8g_font_9x15);
  
  // set upper left position for the string draw procedure
  u8g.setFontPosTop();
  
  // calculate the number of rows for the display
  rows = u8g.getHeight() / u8g.getFontLineSpacing();
  if ( rows > ROW_MAX )
    rows = ROW_MAX; 
  
  // estimate the number of columns for the display
  cols = u8g.getWidth() / u8g.getStrWidth("m");
  if ( cols > LINE_MAX-1 )
    cols = LINE_MAX-1; 
  
  clear_screen();               // clear screen
  delay(1000);                  // do some delay
  exec_line();                    // place the input buffer into the screen
  reset_line();                   // clear input buffer
}


void loop(void) {
  
   // Keep reading from HC-05 and send to Arduino Serial Monitor
  if (bluetooth.available()){
    char c = bluetooth.read();
    Serial.write(c);
//    uint8_t c;
    
    if ( line_pos >= cols-1 ) {
      exec_line();
      reset_line();
      char_to_line(c);
    } 
    else if ( c == '\n' ) {
      // ignore '\n' 
    }
    else if ( c == '\r' ) {
      exec_line();
      reset_line();
    }
    else {
      char_to_line(c);
    }
}
  if (Serial.available())
    bluetooth.write(Serial.read());

}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

#include "U8glib.h"

U8GLIB_SSD1306_128X64 u8g(U8G_I2C_OPT_NO_ACK); // Display which does not send AC

#include <SoftwareSerial.h>

// software serial #1: RX = digital pin 10, TX = digital pin 11

SoftwareSerial bluetooth(10, 11);

#define buflen 10

char str[buflen+1];

int strCount=0;

// setup input buffer

#define LINE_MAX 30

uint8_t line_buf[LINE_MAX] = "Booting...";

uint8_t line_pos = 0;

// setup a text screen to support scrolling

#define ROW_MAX 12

uint8_t screen[ROW_MAX][LINE_MAX];

uint8_t rows, cols;

// line height, which matches the selected font (5x7)

#define LINE_PIXEL_HEIGHT 7

// clear entire screen, called during setup

void clear_screen(void) {

uint8_t i, j;

for( i = 0; i < ROW_MAX; i++ )

for( j = 0; j < LINE_MAX; j++ )

screen[i][j] = 0;

}

// append a line to the screen, scroll up

void add_line_to_screen(void) {

uint8_t i, j;

for( j = 0; j < LINE_MAX; j++ )

for( i = 0; i < rows-1; i++ )

screen[i][j] = screen[i+1][j];

for( j = 0; j < LINE_MAX; j++ )

screen[rows-1][j] = line_buf[j];

}

// U8GLIB draw procedure: output the screen

void draw(void) {

uint8_t i, y;

// graphic commands to redraw the complete screen are placed here

y = 0; // reference is the top left -1 position of the string

y--; // correct the -1 position of the drawStr

for( i = 0; i < rows; i++ )

{

u8g.drawStr( 0, y, (char *)(screen[i]));

y += u8g.getFontLineSpacing();

}

void exec_line(void) {

// echo line to the serial monitor

Serial.println((const char *)line_buf);

// add the line to the screen

add_line_to_screen();

// U8GLIB picture loop

u8g.firstPage();

do {

draw();

} while( u8g.nextPage() );

}

// clear current input buffer

void reset_line(void) {

line_pos = 0;

line_buf[line_pos] = '\0';

}

// add a single character to the input buffer

void char_to_line(uint8_t c) {

line_buf[line_pos] = c;

line_pos++;

line_buf[line_pos] = '\0';

}

void setup(void) {

Serial.begin(9600);

bluetooth.begin(9600);

Serial.print("Hello");

// bluetooth.println("Hi");

// u8g.firstPage();

// u8g.drawStr( 0, 0, "Booting..");

// set font for the console window

u8g.setFont(u8g_font_5x7);

//u8g.setFont(u8g_font_9x15);

// set upper left position for the string draw procedure

u8g.setFontPosTop();

// calculate the number of rows for the display

rows = u8g.getHeight() / u8g.getFontLineSpacing();

if ( rows > ROW_MAX )

rows = ROW_MAX;

// estimate the number of columns for the display

cols = u8g.getWidth() / u8g.getStrWidth("m");

if ( cols > LINE_MAX-1 )

cols = LINE_MAX-1;

clear_screen(); // clear screen

delay(1000); // do some delay

exec_line(); // place the input buffer into the screen

reset_line(); // clear input buffer

}

void loop(void) {

// Keep reading from HC-05 and send to Arduino Serial Monitor

if (bluetooth.available()){

char c = bluetooth.read();

Serial.write(c);

// uint8_t c;

if ( line_pos >= cols-1 ) {

exec_line();

reset_line();

char_to_line(c);

}

else if ( c == '\n' ) {

// ignore '\n'

}

else if ( c == '\r' ) {

exec_line();

reset_line();

}

else {

char_to_line(c);

}

if (Serial.available())

bluetooth.write(Serial.read());

}

As told initially my build is not the best one, there are people with better electronics & better output check these :

http://www.instructables.com/id/Make-your-own-smart-watch/?ALLSTEP

http://makezine.com/projects/make-43/open-source-smartwatch/

http://www.tinkernut.com/portfolio/make-smartwatch-old-cell-phone-part-1/

http://oswatch.org/mkII_build_page_1.php

https://hackaday.io/project/12876-chronio

http://blog.zakkemble.co.uk/diy-digital-wristwatch/

Kindle Universal Dashboard

Kindle is Super awesome !

Because of its e-ink display. I wanted a display to present data which has – Least power consumption, Not painful to eyes & obviously one which doesn’t emit blue light.
E-Ink displays fit perfectly to my requirement – acquiring a display which can be driven using Raspberry Pi or Arduino is hard, size/cost ratio is much higher.

On googling I found some display modules which were more or near to 70-80$ – even smaller display – which impulsed me to get a Kindle Touch (8th Generation) at around 74$ approx.

Kindle comes with an experimental browser but it is narrowed version of WebKit, which is pretty useful if you want to display static content or just want to make a control panel, it can easily render normal websites which use js/CSS & works pretty well. But support for HTML5 is almost absent – so you can’t use WebSockets to push data, using long polling/fast polling is only solutions so far.

Moreover, there was another problem which I had to fix – Kindle has fixed timeout which sends it to sleep mode – for mine it was 10min, after digging I found you can use ~ds or similar but for me, nothing worked.

We can only hope that support to remove timeout or change timeout period will be added in future releases. I think old kindle supports.

If you can’t change timeout or you want to use few other features I suggest you to jailbreak. Follow steps mentioned here http://www.mobileread.com/forums/showthread.php?t=275877 , Don’t jump, It works for kindle touch 8th generation. Tried, tested, working ! For KT3 you will need to install MRPI along with KUAL. Once done your kindle is officially out of warranty 😀 . Post that you need to install USBNet – it will allow you to ssh to your kindle. All this will allow you to execute “lipc-set-prop com.lab126.powerd preventScreenSaver 1″ this on the kindle. It will simply disable screensaver. 🙂

Once you have your kindle whose screen doesn’t lock you can simply go & execute a simple nodejs script to push data.

Note : Kindle doesn’t support WebSocket & none of transport methods in socket.io except “polling”.

Below is nodejs server code :

var http = require('http'),
	fs = require('fs');
	//sanitize = require('validator').sanitize;
	
var app = http.createServer(function (request, response) {
	fs.readFile("client.html", 'utf-8', function (error, data) {
		response.writeHead(200, {'Content-Type': 'text/html'});
		response.write(data);
		response.end();
	});
}).listen(1337);

var io = require('socket.io')({
    "polling duration": 60,
    'log level' :1
}).listen(app);

io.set('transports', [ 'polling']);

io.sockets.on('connection', function(socket) { 
	socket.on('message_to_server', function(data) { 
		var escaped_message = (data["message"]);
		io.sockets.emit("message_to_client",{ message: escaped_message }); 
	});
});

var http = require('http'),

fs = require('fs');

//sanitize = require('validator').sanitize;

var app = http.createServer(function (request, response) {

fs.readFile("client.html", 'utf-8', function (error, data) {

response.writeHead(200, {'Content-Type': 'text/html'});

response.write(data);

response.end();

});

}).listen(1337);

var io = require('socket.io')({

"polling duration": 60,

'log level' :1

}).listen(app);

io.set('transports', [ 'polling']);

io.sockets.on('connection', function(socket) {

socket.on('message_to_server', function(data) {

var escaped_message = (data["message"]);

io.sockets.emit("message_to_client",{ message: escaped_message });

});

Below is code of client.html

<!DOCTYPE html>
<html>
	<head>
		<script src="/socket.io/socket.io.js"></script>
		<script type="text/javascript">
			var socketio = io.connect();

			socketio.on("message_to_client", function(data) {
				document.getElementById("chatlog").innerHTML = ("<hr/>" + data['message'] + document.getElementById("chatlog").innerHTML);
			});

			function sendMessage() {
				var msg = document.getElementById("message_input").value;
				socketio.emit("message_to_server", { message : msg});
			}
		</script>
	</head>
	<body>
		<input type="text" id="message_input"/>
		<button onclick="sendMessage()">send</button>
		<div id="chatlog"></div>
	</body>
</html>

<!DOCTYPE html>

<html>

<head>

var socketio = io.connect();

socketio.on("message_to_client", function(data) {

document.getElementById("chatlog").innerHTML = ("<hr/>" + data['message'] + document.getElementById("chatlog").innerHTML);

});

function sendMessage() {

var msg = document.getElementById("message_input").value;

socketio.emit("message_to_server", { message : msg});

}

</script>

</head>

<body>

</body>

</html>

Voila – It works !

Below is video of working in case you want to see demo before getting hands dirty 🙂

Warning & Update : This method might consume more power than expected, as experimental browser has loading status – which continuously refreshes a section of the screen. To overcome this problem I will be polling server with long interval difference – which will be adjustable by the server.

Note for nerds : Since this method uses browser – it’s more flexible – but if you are possessive about power consumption & screen space – You can use JAVA to develop kindlet application. Lightweight pub/sub protocols like MQTT should help you in the way.

Designing a wall holder : You can google for covers or design own or use some double sided foam tape. Since i had access to 3d printer i got two of http://www.thingiverse.com/thing:832273 printed & hung it on the wall – it just helped me in reading few books apart from using it only as display. SWAP!

Use it as :

Scoreboard
Notification
Weather system
Wallpaper slideshow
News/RSS feeds display
Home automation control
Anything
Book reader 🙂

At the end, even if you place it behind your monitor it won’t hurt or push the new data to your eyes & spoil the code castle you were building.

Custom Nginx with PageSpeed

Pulling nginx from source, customizing & auto updating across servers is a pain, We use pagespeed & custom header modules across all our servers, it autominifies resources & increases page performance without giving pain.

Below is script we use :

Continue reading Custom Nginx with PageSpeed

Offline Wikipedia with Elasticsearch and MediaWiki

Wikipedia is Awesome ! It’s open, its free – Yea. & its huge in size, millions of articles but as developer how to exploit the free knowledge.

I started digging internet just to find ways to exploit my fresh 9+ GB of XML Gzipped archive which seemed to me of no use as even a simple text editor can’t open it. (Just out of excitement what’s inside, how its structured, Schema ! )

Luckily people have already imported it. Elasticsearch is fast, reliable & its good for searching, so https://github.com/andrewvc/wikiparse was a saver.

Installed elastic search
Ran command to import

it took almost 48 hour in an i5, with 8gb ram – where mistake was i used same harddisk for data storage & database. Time might vary.

Data was imported but its still of no use ! Why ? Its in text/wiki format, parses is needed.

After doing search only solution i found was using mediawiki api, which is in PHP there were lots of things missing as its only for mediawiki but not for parsing plain text. (Though i didn’t spend much time in learning internal API)

I quickly downloaded mediawiki, ran nginx with php, installed it & used API.php.
it was good to see my offline API too, but still many things were missing, confusing, API has hard to modify structure. So i created a parse.php

<?PHP
header('Access-Control-Allow-Origin: *');
header('Access-Control-Allow-Methods: GET, POST');  
header('Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept, Key');

function fixLink ($this, $nt, $section, $tooltip, &$result)  {
		$linkName = "editSection-$section"; // the span around the link
		$anchorName = "editSectionAnchor-$section"; // the actual link anchor, generated in Linker::makeHeadline
		$chromeName = "editSectionChrome-$section"; // the chrome that surrounds the anchor	
		$result = preg_replace('/(\D+)( title=)(\D+)/', '${1} class=\'editSectionLinkInactive\' id=\'' . $anchorName . '\' onmouseover="editSectionHighlightOn(' . $section . ')" onmouseout="editSectionHighlightOff(' . $section . ')" title=$3', $result);
		$result = preg_replace('/<\/span>/', '<span class="editSectionChromeInactive" id=\'' . $chromeName . '\'>&#8690;</span></span>', $result);
		// while resourceloader loads this extension's css pretty late, it's still
		// overriden by skins/common/shared.css.  to get around that, insert an explicit style here.
		// i'd welcome a better way to do this.
		$result ='';
		return true;
	}


require(dirname(__FILE__) . '/includes/WebStart.php');

$wgHooks['DoEditSectionLink'][] = 'fixLink';

//include("api.php");

$output = $wgParser->parse(
    $_POST['text'],
    Title::newFromText('Some page title'),
    new ParserOptions());
   
echo json_encode(array('html'=>$output->getText()));

<?PHP

header('Access-Control-Allow-Origin: *');

header('Access-Control-Allow-Methods: GET, POST');

header('Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept, Key');

function fixLink ($this, $nt, $section, $tooltip, &$result) {

$linkName = "editSection-$section"; // the span around the link

$anchorName = "editSectionAnchor-$section"; // the actual link anchor, generated in Linker::makeHeadline

$chromeName = "editSectionChrome-$section"; // the chrome that surrounds the anchor

$result = preg_replace('/(\D+)( title=)(\D+)/', '${1} class=\'editSectionLinkInactive\' id=\'' . $anchorName . '\' onmouseover="editSectionHighlightOn(' . $section . ')" onmouseout="editSectionHighlightOff(' . $section . ')" title=$3', $result);

$result = preg_replace('/<\/span>/', '<span class="editSectionChromeInactive" id=\'' . $chromeName . '\'>⇲</span></span>', $result);

// while resourceloader loads this extension's css pretty late, it's still

// overriden by skins/common/shared.css. to get around that, insert an explicit style here.

// i'd welcome a better way to do this.

$result ='';

return true;

}

require(dirname(__FILE__) . '/includes/WebStart.php');

$wgHooks['DoEditSectionLink'][] = 'fixLink';

//include("api.php");

$output = $wgParser->parse(

$_POST['text'],

Title::newFromText('Some page title'),

new ParserOptions());

echo json_encode(array('html'=>$output->getText()));

So all steps were :

Install elasticsearch – https://www.elastic.co/
Download Wikipedia – https://en.wikipedia.org/wiki/Wikipedia:Database_download, https://dumps.wikimedia.org/enwiki/latest/
Import it using existing method – https://github.com/andrewvc/wikiparse
Parse Using WikiMedia API – https://github.com/wikimedia/mediawiki

Backup Server Configurations to Git

When you have multiple servers it pain to remember every configuration and it may take hours to configure servers again incase you need.
It is also impractical to copy each files.

There are few tools available but they come with overhead attached moreover its fun to write custom solutions 🙂

1. Create directory
2. cd into it.
3. Initialize git
4. Add a remote
5. Create branch specifically for that server
6. Check out branch
7. Add shell script
8. Make a commit and push to server.

mkdir backup
cd backup/  
git init
git remote add git@tikaj.com:tikaj/ConfigurationBackups.git
git branch $(hostname)
git checkout $(hostname)

#This file contains space delimited configuration file name and path relative to git.
touch ConfigurationFiles.list

vi backup.sh

mkdir backup

cd backup/

git init

git remote add git@tikaj.com:tikaj/ConfigurationBackups.git

git branch $(hostname)

git checkout $(hostname)

#This file contains space delimited configuration file name and path relative to git.

touch ConfigurationFiles.list

vi backup.sh

Content of backup.sh

#!/bin/bash
BASEDIR=$(dirname $0)

cd $BASEDIR

while read line; do 
 IFS=' ' read -a A <<< "$line"
 cp ${A[0]} $BASEDIR/${A[1]} 
done < ConfigurationFiles.list


#Add new files and commit changes
git add --all :/
git commit -m "Update configuration files."
git push origin  $(hostname)

#!/bin/bash

BASEDIR=$(dirname $0)

cd $BASEDIR

while read line; do

IFS=' ' read -a A <<< "$line"

cp ${A[0]} $BASEDIR/${A[1]}

done < ConfigurationFiles.list

#Add new files and commit changes

git add --all :/

git commit -m "Update configuration files."

git push origin $(hostname)

Then, Don’t forget to generate SSH key adding key to your git server.

git push -u origin  $(hostname)

1	git push -u origin $(hostname)

Now i can either setup a cron job or add command to daily backup script to update configuration files.
using

ssh mad@tikaj.com "bash /root/backup/backup.sh"

1	ssh mad@tikaj.com "bash /root/backup/backup.sh"

from remote.

Cloning Server to New Hardware

You can find many ways of cloning old server to new hardware, create diskimage and move to new hardware and if you have identical hardware use dd or similar tool to copy disks block by block, but if you have new hardware and want to move server without downtime and with no configuration change there is nearly no solution.

I used vestacp, ejabberd in my servers so i had to make changes even after moving some files, as IP changes every where but it is still far better solution than installing and configuring each package manually.

We used rsync to copy whole server to new server except following files
because on new network you will never wish to propagate old network configuration,device specific configurations, installation specific configs.
/boot/ /lib/modules /etc/fstab /etc/mtab /etc/modules /proc /dev /etc/network/interfaces /root

Create a file with following directories and then execute rsync with exclude-from.

rsync will Synchronize all your files from remote to local.

Monitoring Website for Downtime

Sometimes our in house server go down, But at production level we cannot take risk of 1min downtime, so we were supposed to monitor our server for downtime.
Although there are many cheap services available but i decided to do it my own way, I can use this script in my RPI to monitor even my internal services.

Below is python code, Which sends email if a server goes down and when it comes back online. Obviously you can add code to get more statistics like Logging to file, average response time, Monitoring page changes, Downtime etc.

import httplib
import time
from smtplib import SMTP

def send_email(to,subject,body,SERVER="smtp.gmail.com",PORT=587,USERNAME="something@gmail.com",PASSWORD="YourPassword"):
	smtp = SMTP(SERVER, PORT)
	smtp.ehlo()
	smtp.starttls()
	smtp.login(USERNAME, PASSWORD)
	headers = "\r\n".join(["from: " + USERNAME,
                       "subject: " + subject,
                       "to: " + to,
                       "mime-version: 1.0",
                       "content-type: text/html"])
                    
	content = headers + "\r\n\r\n" + body
	smtp.sendmail(USERNAME, to, content)



def get_status_code(host,path="/"):
	try:
		conn = httplib.HTTPConnection(host,timeout=10)
		conn.request("HEAD",path)
		return conn.getresponse().status
	except StandardError:
		return 0


def get_site_status(host):
	start = time.time()
	status = False
	code = get_status_code(host)
	#if found, temp redirect, permanent redirect.
	if code in (200,302,301):
		status = True
	return status, time.time()-start,code


def check_internet():
	return get_site_status("google.com")

def check_status():
	toMonitor = ["tikaj.com"]
	down = []
	email = ""
	while True: 
		for value in toMonitor:
			st = "Checking %s..." %value

			#give enough space so that status code can be idented.
			while(len(st)<40):
				st += " "
			print st,

			status,rtime,code = get_site_status(value)
			print "[%s]" %code,

			if status==False :
				if check_internet()[0] == True :
					print "[Server down]"
					
					#if not present in Server down list. Add for email and add to down list.
					if value not in down:
						down.append(value)
						email += st + "<b>[SERVER_DOWN]</b><br/>"
				else:
					print "[Internet Fail]"
			else: 
				# if in down list compose mail that server is up.
				if value in down:
					email += st + "<b>[%fs]</b>" %rtime
					down.remove(value)
				print "[%fs]" %rtime


		if(len(email)>10):

			print "Sending alert email..."
			send_email("myemail@gmail.com","ServerDown!","<pre>%s</pre>" %email)
			email = ""

		time.sleep(60) #sleep for 60s

if __name__ == '__main__':
	check_status()

import httplib

import time

from smtplib import SMTP

def send_email(to,subject,body,SERVER="smtp.gmail.com",PORT=587,USERNAME="[email protected]",PASSWORD="YourPassword"):

smtp = SMTP(SERVER, PORT)

smtp.ehlo()

smtp.starttls()

smtp.login(USERNAME, PASSWORD)

headers = "\r\n".join(["from: " + USERNAME,

"subject: " + subject,

"to: " + to,

"mime-version: 1.0",

"content-type: text/html"])

content = headers + "\r\n\r\n" + body

smtp.sendmail(USERNAME, to, content)

def get_status_code(host,path="/"):

try:

conn = httplib.HTTPConnection(host,timeout=10)

conn.request("HEAD",path)

return conn.getresponse().status

except StandardError:

return 0

def get_site_status(host):

start = time.time()

status = False

code = get_status_code(host)

#if found, temp redirect, permanent redirect.

if code in (200,302,301):

status = True

return status, time.time()-start,code

def check_internet():

return get_site_status("google.com")

def check_status():

toMonitor = ["tikaj.com"]

down = []

email = ""

while True:

for value in toMonitor:

st = "Checking %s..." %value

#give enough space so that status code can be idented.

while(len(st)<40):

st += " "

print st,

status,rtime,code = get_site_status(value)

print "[%s]" %code,

if status==False :

if check_internet()[0] == True :

print "[Server down]"

#if not present in Server down list. Add for email and add to down list.

if value not in down:

down.append(value)

email += st + "<b>[SERVER_DOWN]</b><br/>"

else:

print "[Internet Fail]"

else:

# if in down list compose mail that server is up.

if value in down:

email += st + "<b>[%fs]</b>" %rtime

down.remove(value)

print "[%fs]" %rtime

if(len(email)>10):

print "Sending alert email..."

send_email("[email protected]","ServerDown!","<pre>%s</pre>" %email)

email = ""

time.sleep(60) #sleep for 60s

if __name__ == '__main__':

check_status()

After saving code to server use cron job to start run script on every boot.

Edit crontab using sudo crontab -e and then add following line to file.

@reboot screen -dmS monitor python /root/monitor.py &> /dev/null

Above command starts a screen session named monitor and forwards error of screen to null.
Incase you want to see output of monitor.py use screen -r monitor.
You can also add it simply without screen like
@reboot python /root/monitor.py &

If you have some suggestions comment.