Install and Configure Monit web interface

Install Monit

Monit is easiest to install through apt-get:

$ sudo apt-get install monit

Once monit downloads, you can add programs and processes to the configuration file:

$ sudo nano /etc/monit/monitrc

Monit can be started up with a command that then keeps it running in the background

$ monit

Typing monit status displays monit’s details:

The Monit daemon 5.3.2 uptime: 1h 25m 

System 'myhost.mydomain.tld'
  status                            Running
  monitoring status                 Monitored
  load average                      [0.03] [0.14] [0.20]
  cpu                               3.5%us 5.9%sy 0.0%wa
  memory usage                      26100 kB [10.4%]
  swap usage                        0 kB [0.0%]
  data collected                    Thu, 30 Aug 2012 18:35:00

Configure Monit

Monit is very easy to use nearly out of the box. By default, it is set up to check that services are running every 2 minutes and stores its log file in “/var/log/monit.log”.

These settings can be altered at the beginning of the configuration file in the set daemon and set logfile lines respectively.

Web Service

Monit comes with it’s own web server running on port 2812. To configure the web interface, find and uncomment the section that begins with set httpd port 2812. Once the section is uncommented, write in your server’s IP or domain name as the address, allow anyone to connect, and then create a monit user and password

    set httpd port 2812
    use address 12.34.56.789  # only accept connection from localhost
    allow 0.0.0.0/0.0.0.0     # allow localhost to connect to the server and
    allow admin:monit         # require user 'admin' with password 'monit'

Once this is configured, monit should reload and reread the configuration file, and the web interface will be available:

$ monit reload

When you are done, try

http://12.34.56.789:2812

You will see a dialog box appear in the browser.

General Monit Scripts

Sidekiq

  # /etc/monit/conf-available/sidekiq
  check process sidekiq_thepact_staging0 with pidfile "/home/deployer/www/staging/shared/tmp/pids/sidekiq.pid"  
  start program = "/bin/su - deployer -c 'cd /home/deployer/www/staging/current && /usr/local/rvm/bin/rvm default do bundle exec sidekiq --config /home/deployer/www/staging/current/config/sidekiq.yml --index 0 -e staging -d'" with timeout 30 seconds  
  stop program  = "/bin/su - deployer -c 'cd /home/deployer/www/staging/current && /usr/local/rvm/bin/rvm default do bundle exec sidekiqctl stop /home/deployer/www/staging/shared/tmp/pids/sidekiq.pid'" with timeout 110 seconds  group thepact-sidekiq-0

PostGreSQL

 # /etc/monit/conf-available/pg
 check process postgres with pidfile /var/run/postgresql/9.5-main.pid
 group database
 start program = "/etc/init.d/postgresql start"
 stop program  = "/etc/init.d/postgresql stop"
 if failed unixsocket /var/run/postgresql/.s.PGSQL.5432 protocol pgsql
 then restart

Puma

 # /etc/monit/conf-available/puma
 check process puma with pidfile /var/www/myapp/production/shared/tmp/pids/puma.pid
 group www

 start program   = "/bin/su - john -c '~/.rvm/bin/rvm default do bundle exec puma -C /var/www/myapp/production/shared/puma.rb --daemon'"
 stop program    = "/bin/su - john -c 'pumactl stop -P /var/www/myapp/production/shared/tmp/pids/puma.pid'"
 restart program = "/bin/su - john -c 'pumactl restart -P /var/www/myapp/production/shared/tmp/pids/puma.pid'"

or

 # /etc/monit/conf-available/puma
 check process puma with pidfile /var/www/myapp/production/shared/tmp/pids/puma.pid
 group www
 start program   = "/bin/su - john -c '~/.rvm/bin/rvm default do bundle exec puma -C /var/www/myapp/production/shared/puma.rb --daemon'"
 stop program    = "/bin/su - john -c '~/.rvm/bin/rvm default do bundle exec pumactl -S /var/www/myapp/production/shared/tmp/pids/puma.state stop'"
 restart program = "/bin/su - john -c '~/.rvm/bin/rvm default do bundle exec pumactl -S /var/www/myapp/production/shared/tmp/pids/puma.state restart'"

Using the scripts

By default Monit does not load conf-scripts in conf-available. You have to copy the scripts to conf-enabled to have them executed by Monit. So, we are going to link those files that folder instead of copying/duplicating them. Its a good practice.

$ sudo ln /etc/monit/conf-available/puma /etc/monit/conf-enabled/puma

This will link the file there, same file will act as if it is there too.

Sources

https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-monit

monit_banner

Some useful monit commands and configs

Config for Sidekiq

# Monit configuration for Sidekiq : myAPP

check process sidekiq_thepact_qa0
 with pidfile "/home/deployer/www/qa/shared/tmp/pids/sidekiq.pid"
 start program = "/bin/su - deployer -c 'cd /home/deployer/www/qa/current && /usr/local/rvm/bin/rvm default do bundle exec sidekiq --config /home/deployer/www/qa/current/config/sidekiq.yml --index 0 -e qa -d'" with timeout 30 seconds
stop program = "/bin/su - deployer -c 'cd /home/deployer/www/qa/current && /usr/local/rvm/bin/rvm default do bundle exec sidekiqctl stop /home/deployer/www/qa/shared/tmp/pids/sidekiq.pid'" with timeout 110 seconds
 group myapp-sidekiq-qa

Commands

reload configuration

$ sudo monit reload

start monit

$ sudo service monit start

 

catch2

Sidekiq : Production :running on older deployment codes

I have been deploying for may days; there were latest release directories in the production/staging environment. But my background jobs were failing unexpectedly. However, it was running fine in development environment.

I tailed the log file

$ tail log/sidekiq.log -f -n 400

and saw

/home/deployer/www/staging/releases/20160217062712/app/workers/notification_async.rb:40:in `perform'
/

# but there was no such directory
~/www/staging/releases $ ls
20160327044951 20160327050311 20160327050746 20160327051344 20160327052411

the `20160217062712` was outdated one.

Reason:

I had the following configuration in my monit config file.

Start program ‘/bin/su – deployer -c cd /home/deployer/www/staging/current && /usr/local/rvm/bin/rvm default do bundle exec sidekiq –config /home/deployer/www/staging/current/config/sidekiq.yml –index 0 –pidfile /home/deployer/www/staging/shared/tmp/pids/sidekiq-0.pid –environment staging –logfile /home/deployer/www/staging/shared/log/sidekiq.log -d’ timeout 30 second(s)
Stop program ‘/bin/su – deployer -c cd /home/deployer/www/staging/current && /usr/local/rvm/bin/rvm default do bundle exec sidekiqctl stop /home/deployer/www/staging/shared/tmp/pids/sidekiq-0.pid’ timeout 110 second(s)

means; monit was pointed to false pid file, so monit was not able to stop the old running process of sidekiq.

therefore my background processes were sometimes working fine; because the newly created processes started with new code base might have processed the tasks.

Sidekiq : Redis : Namespace

You need to add the gem ‘redis-namespace’ to namespace the redis keys via sidekiq

From the command line:

$ gem install redis-namespace

Or in your Gemfile:

gem 'redis-namespace'

When you might need this?

If you have multiple server environment or multiple applications hosted using the same redis server then you must namespace your data. Otherwise sidekiq will try to execute tasks from one application with another application’s codebase.

Continue reading

Rails : Sidekiq 4.0 : no :symbol keys supported when hash is passed as argument

 

NotificationAsync.perform_later({data:{pact_id: 1}, context: OrdersSignup::Notification::WelcomeChild.to_s})

I was sending Hash as a parameter/argument to my worker but it was now able to access the keys/values using :symbol as a key name

So Explicitly use ActiveSupport’s `.with_indifferent_access`
Continue reading