Shinobi - - a ZoneMinder replacement using Node.js and mysql

Configuration wizards, scanners, remote desktop, etc.
Post Reply
Message
Author
User avatar
rockedge
Posts: 1864
Joined: Wed 11 Apr 2012, 13:32
Location: Connecticut, United States
Contact:

Shinobi - - a ZoneMinder replacement using Node.js and mysql

#1 Post by rockedge »

A new project called Shinobi is under way to build an open-source solution for a network recording device. A possible replacement for ZoneMinder, using Node.js and mysql, addresses some of the short comings of ZoneMinder.

Using ffmpeg-release-32bit-static.tar.xz which runs nicely on tahr 6.0.5. Node.js 7.3 the mysql installed from the PPM and NPM manager for node.js and pm2 to daemonize Shinobi.

If anyone is interested I could throw out some tips to get it going.
Last edited by rockedge on Thu 05 Jan 2017, 00:21, edited 1 time in total.

User avatar
drunkjedi
Posts: 882
Joined: Mon 25 May 2015, 02:50

#2 Post by drunkjedi »

Kind of off topic, but Shinobi means a ninja warrior in Japanese, right?

User avatar
rockedge
Posts: 1864
Joined: Wed 11 Apr 2012, 13:32
Location: Connecticut, United States
Contact:

#3 Post by rockedge »

It must be.... the login page background is a ninja warrior in Anime style....

User avatar
rockedge
Posts: 1864
Joined: Wed 11 Apr 2012, 13:32
Location: Connecticut, United States
Contact:

#4 Post by rockedge »

there is a plan to add a feature to make it possible to record desktops across a LAN...... in the spirit of a complete Network video recorder.

User avatar
rockedge
Posts: 1864
Joined: Wed 11 Apr 2012, 13:32
Location: Connecticut, United States
Contact:

#5 Post by rockedge »

One report:
"4 cameras each doing 2048x1536@25fps recording h264 on copy
old machine with 4 cores, roughly sitting at 30% cpu
using about 1GB ram"

I am running 3 cams at 320x240 with one local webcam on a dual core 4 gigs of RAM...my results show 5% cpu 40 megs of memory in recording mode of all 4 cameras.

Set up with Tahr 6.0.5 updated and running Node 7.3 MySQL 5.5.53-0ubuntu0.14.04.1 (Ubuntu) with NPM and pm2 installed via PPM.
Node.js is fairly simple to set up. I downloaded node-v7.3.0-linux-x86.tar.xz and extracted to /root/my-applications then symlinked node and npm to /root/my-applications/bin
Using PPM find npm and install.
download the Shinobi tarball or zip from github extracted it to /root/my-applications and followed the install instructions to configure the mysql database then used pm2 to daemonize the camera.js

User avatar
rockedge
Posts: 1864
Joined: Wed 11 Apr 2012, 13:32
Location: Connecticut, United States
Contact:

#6 Post by rockedge »

Shinobi is currently being equipped with software face and license plate recognition. As a efficient Network video recorder it is already a strong competitor to ZoneMinder and many commercial CCTV systems

the Node.js version 7.4 and MySQL installed via the PPM work well and also make a nice Node.js server on Tahrpup.

https://github.com/moeiscool/Shinobi

Image

User avatar
Max Headroom
Posts: 421
Joined: Wed 28 Jun 2006, 07:17
Location: GodZone Kiwi
Contact:

#7 Post by Max Headroom »

Yes Please Rockedge, I'm Very interested in this Shinobi CCTV Security System. I'm considering Running it on a VIA Epia EN Mini - itx small Form Factor PC which is / was previously a VadaXchange Phone System. if by any chance yoU've come across such hardware, I Welcome any Comments about Fitness 4 Purpose or eXperience. Any Updates? Thanx ;-)K
PC is Not Political Correctness :P , it's a Personal Computer! Boycott the Evil M$ & Google!

User avatar
rockedge
Posts: 1864
Joined: Wed 11 Apr 2012, 13:32
Location: Connecticut, United States
Contact:

#8 Post by rockedge »

There has been a request to clarify a working recipe for installing Shinobi.

The recipe has been tested on Bionic64-v8 stock version with devx loaded.
once the system is up and running I believe one can unload the devx.

Get the devx_bionicpup64_8.0.sfs and load it using Menu->Setup->SFS-Load-on-the-fly

The devx sfs contains tools needed for compiling and has git included plus more of the PERL language modules.

Open a terminal ->

Code: Select all

adduser mysql
finish the prompts.

https://nodejs.org/en/download/
download nodejs LTS (in my case node-v10.16.0-linux-x64)

open it with xarchive (click on it)

extract to /root/my-applications

open /root/my-applications/node-v10.16.0-linux-x64/bin

sym-link the these files, open a termnal ->

Code: Select all

 ln -s /root/my-applications/node-v10.16.0-linux-x64/bin/node /root/my-applications/bin/node
 ln -s /root/my-applications/node-v10.16.0-linux-x64/bin/npm /root/my-applications/bin/npm
 ln -s /root/my-applications/node-v10.16.0-linux-x64/bin/npx /root/my-applications/bin/npx
open a terminal ->

Code: Select all

node -v
you should see a response like :

Code: Select all

v10.16.0
Open the PPM

search for "mariadb" and select -> mariadb-server_10.1.40
Let PPM install it
Open a terminal and type ->

Code: Select all

mysql_install_db
open a 2nd terminal ->

Code: Select all

mkdir /var/log/mysql
chown -R mysql /var/log/mysql
mysqld_safe
Return to the 1st terminal an type ->

Code: Select all

/usr/bin/mysql_secure_installation
reply Yes for all the prompts and fill in a password for the root user

close the terminals
mysql is running and the nodejs server is almost ready now.

open a terminal ->

Code: Select all

ffmpeg
note the version which on this Bionic64 is

Code: Select all

ffmpeg version 3.4.4-0ubuntu0.18.04.1
you may need this command as well...doesn't hurt so go ahead and run it

Code: Select all

npm install ffbinaries
Following from this page of the Shinobi docs -> https://shinobi.video/docs/start
Application Install

Download Shinobi with git (comes in the devx_bionicpup64_8.0.sfs)

Open Terminal in /root/my-applications

git clone https://gitlab.com/Shinobi-Systems/Shinobi.git Shinobi


Set permissions on the shinobi directory. Where camera.js is located.

chmod -R 755 /root/my-applications/Shinobi

Open Shinobi directory.

cd /root/my-applications/Shinobi

Setup SQL. Go to sql and install the SQL files in your database.

cd sql

Access MariaDB SQL Database from Terminal. The password will have been set during the installation of MySQL.

mysql -u root -p

OPTIONAL : Create New SQL User with privileges. If you choose to use your own pre-defined credentials skip this step.

source ./user.sql

or create your own

CREATE USER 'majesticflame'@'127.0.0.1' IDENTIFIED BY '';
GRANT ALL PRIVILEGES ON * . * TO 'majesticflame'@'127.0.0.1';
FLUSH PRIVILEGES;

while still in the SQL client. Install the Shinobi database. It will create a database called ccio.

source ./framework.sql

OPTIONAL : default_data.sql contains a demo user and a demo jpeg input monitor.

source ./default_data.sql

After importing the data. Exit the sql client.

exit

Go up one directory to enter the main directory. Where camera.js is located.

cd ..

Copy conf.sample.json as another file named conf.json.

cp conf.sample.json conf.json

Edit conf.json to reflect your sql credentials. Puppy Linux users can use geany or leafpad to modify any of the config files.

geany conf.json

Install Libraries

Run npm install while in the main directory. This will install the libraries Shinobi needs. PM2 is needed to use UPDATE.sh and to Daeomonize the process

npm install&&npm install pm2 -g

or simplify with
npm install
npm install pm2 -g

Copy super.sample.json as another file named super.json.

cp super.sample.json super.json

Launch Shinobi

then to start :

pm2 start camera.js
pm2 start cron.js

Run pm2 logs to see the console for any errors.

forever is another program to daemonize, but i've had more success with pm2.

To get your IP you can run the following command.

ifconfig

Open up http://YOUR_IP:8080 in your browser. The Default Login was originally set by adding default_data.sql.

Username : ccio@m03.ca
Password : password






REMEMBER:

all the passwords used are default and you can modify them as you wish.
once you have seen how it works one can place the components in any directories desired.
Last edited by rockedge on Sat 15 Jun 2019, 14:59, edited 6 times in total.

User avatar
rockedge
Posts: 1864
Joined: Wed 11 Apr 2012, 13:32
Location: Connecticut, United States
Contact:

#9 Post by rockedge »

examples of Shinobi running

1 & 2 are results of using pm2
Attachments
Screenshot(1).png
(25.96 KiB) Downloaded 446 times
screenshot-1.png
(98.54 KiB) Downloaded 472 times
screenshot-2.png
(102.18 KiB) Downloaded 402 times
Screenshot.png
(249.32 KiB) Downloaded 449 times

lysion
Posts: 1
Joined: Sat 08 Jun 2019, 05:54

Thnx rockedge

#10 Post by lysion »

Thank you very much rockedge for making the above instructions. It was really frustrating to get it to work because of a lot of obstacles for a new Puppy user like me. Like for example,

1.) I didnt know I can just download and mount devx_bionicpup64_8.0.sfs to be able to use

Code: Select all

git clone
Instead of downloading the large 133MB git-all package from PPM when i only needed git clone.
2.) I was searching for 'MySQL server' in PPM and couldnt find it. Thats because its 'MariaDB' I should be searching per your instructions.
3.) Installing MySQL client was also a pain since theres a lot of mysql packages in PPM but installing it was as easy as inputting,

Code: Select all

mysql_install_db
which I assume is because of devx_bionicpup64_8.0.sfs
4.) I didnt need to download ffmpeg from shinobi doc page since devx_bionicpup64_8.0.sfs also seems to carry it
5.) Following the Shinobi doc instructions for Ubuntu is also causing me problems because apparently Puppy doesnt have a fully functioning

Code: Select all

apt-get
I just want to add a couple of additions to your recipe. Since you quoted the Shinobi doc instructions for Ubuntu, the command

Code: Select all

nano
doesnt exist for Puppy so instead I downloaded 'nano-tiny' from PPM and I was able to run

Code: Select all

nano-tiny conf.json
I dont know why Shinobi dev included this instruction since 'conf.json' can be edited by a text editor anyway so installing 'nano-tiny' is optional.
The ffmpeg from devx_bionicpup64_8.0.sfs doesnt have ffbinaries according to pm2 log and it suggested the command

Code: Select all

npm install ffbinaries
which fixed the problem.
The Shinobi doc instructions for Ubuntu also forgot to mention to execute command

Code: Select all

pm2 start cron.js
and not just

Code: Select all

pm2 start camera.js
I ran into a lot of "Cannot read property 'forEach' of undefined...." errors in pm2 logs without cron.js running.
Must also execute command

Code: Select all

pm2 startup
and

Code: Select all

pm2 save
after starting camera.js and cron.js.
Theres also an instruction, To test MariaDB daemon,

Code: Select all

mysql-test-run.pl cd '/usr/mysql-test' ; perl mysql-test-run.pl
which doesnt work because the MariaDB from PPM doesnt have 'perl mysql-test-run.pl' file nor is there '/usr/mysql-test' directory. I still dont have a proper way to test the MariaDB daemon but if i can execute

Code: Select all

pm2 list
and it shows camera.js and cron.js online and i can login to [xxx.xxx.xxx.xxx:8080] then I assume MariaDB is running.

I now have Shinobi running and currently testing it. Thnx again.

User avatar
rockedge
Posts: 1864
Joined: Wed 11 Apr 2012, 13:32
Location: Connecticut, United States
Contact:

#11 Post by rockedge »

Thank you very much for the info...I can edit and improve the instructions to include the obstacles you faced. As well as the small details.

the mysql_install_db is from the mariadb package and is the easiest way to set up the mysql.

Yes the nano or nano-tiny are great to have around but not necessary for Puppy Linux users in this set up since Geany or leafpad will do just fine

If you are connecting via the browser you have Shinobi running.

use Menu->System->htop
to see what is running

Post Reply