Centralise shell script logs using log.jod.li

log.jod.li

It often happens that we need to gather logs from several sources. Centralising them in one simple server place is the purpose of the log.jod.li initiative. The logs are then visible almost instantaneously on the log.jod.li page or can be retrieved in a format similar to CSV (raw logs). For more details, please visit https://log.jod.li .

Shell script situation

This page aims at showing how to use log.jod.li logs centralisation solution using Shell script code.

Let’s say you have some shell script code that may not always run as expected. You want to track the unwanted outcomes and centralise their reading on log.jod.li . There is no “try…catch…” mechanism in shell scripts but it is possible to know when there is an undesired result. In that case, you can add a call to log the situation. This is a simple GET call to a URL containing:

  • The server name to send the log to the right location. This will be en.log.jod.li .
  • The user name you created on log.jod.li .
  • The user key you received in your email box when you created your user. Make sure nobody else gets access to your key.
  • The log message that you want to record on the log.jod.li server. The maximum length of the message is 1700 characters. It must be passed url encoded so that the string is interpreted correctly by the server.
  • Optionally the application name so that you can filter your logs more easily. Max 32 characters.
  • Optionally the client guid so that you can filter on a given client. Max 32 characters.

Here is an example of URL:

https://en.log.jod.li/?api=a&u=toto&k=1234567890123456&a=BestApp&c=123456789&m=testing

Here, the user is “toto”, the key is “1234567890123456”, the application name is “BestApp”, the client guid is “123456789” and the log message is “testing”.

Let’s consider the following shell script:

#!/bin/sh

if cat /tmp/test.txt ; then
        echo "Command succeeded"
else
        echo "Command failed"
fi

In case the file /tmp/test.txt doesn’t exist, the script will display “Command failed” but the information will not be visible outside the console.

Send log with curl

What we can do to make use of log.jod.li in this case is using curl for a call to a URL similar to the one mentioned above. Here is the result:

#!/bin/sh

if cat /tmp/test.txt ; then
        echo "Command succeeded"
else
        echo "Command failed"
        curl -G --data-urlencode "api=a" --data-urlencode "u=toto" --data-urlencode "k=1234567890123456" --data-urlencode "a=ShellScript" --data-urlencode "c=`hostname|cut -c1-32`" --data-url
encode "m=/tmp/test.txt file doesn't seem to exist!" https://en.log.jod.li
fi

As the key must be present in the code, the script cannot be readable by anyone except yourself.

Send log with wget

You may prefer using wget over curl to call a GET url:

#!/bin/sh

if cat /tmp/test.txt ; then
        echo "Command succeeded"
else
        echo "Command failed"
        wget "https://en.log.jod.li/?api=a&u=toto&k=1234567890123456&a=ShellScriptWget&c=`host
name|cut -c1-32`&m=/tmp/test.txt file doesn't seem to exist!" 
fi

The outcome on the log.jod.li page is something like:

2018-09-12 05:54:31	MyHostName	ShellScriptWget	/tmp/test.txt file doesn't seem to exist!

Here is the result on the raw CSV log from log.jod.li :

2018-09-12 05:54:31,Mes-MacBook-Pro.fritz.box,ShellScriptWget,/tmp/test.txt file doesn't seem to exist!

Conclusion

This is a demonstration that you can log your messages by simply calling a url. You can then follow your logs from the https://log.jod.li page.