nest is the name for the system that runs riseup.net email and user management. it consists of:

  1. a central database to store all user data
  2. a web frontend to manage the database (ie user.riseup.net)
  3. a bunch of backend nest daemons (currently called taskrunner, but eventually renamed nestd).

installing the web frontend

The web front-end to nest is a rails application. We are running an apache2 with mongrel cluster setup to serve out the pages.

# apt-get install rubygems mongrel=1.1.3-2~bpo40+1 libbluecloth-ruby libnet-ssh-ruby libnet-sftp-ruby libneedle-ruby libdaemons-ruby libbuilder-ruby libmysql-ruby rake libapache2-mod-removeip gsfonts libopenssl-ruby
# get update --system
# gem install SyslogLogger scruffy rails hoe fastthread capistrano builder activesupport activerecord actionwebservice actionpack actionmailer (some of these might not be needed? and hopefully some of these will eventually become debian packages)
(note: docs say to install librmagick-ruby1.8 and gem tzinfo but they aren't installed on our system)
# mkdir -p /usr/apps/nest
# adduser nest 
# chown nest:nest /usr/apps/nest
# su nest

Then get the code from svn:

$ cd /usr/apps/nest
$ svn export https://code.riseup.net/svn/nest/trunk .
$ mkdir log
$ mkdir tmp tmp/cache tmp/session .....

Set up the database

Copy the config/database.yml.example to config/database.yml and then edit database.yml to suit your site’s needs. See the doc/README_FOR_APP for the necessary mysql GRANTs.

rake migrate RAILS_ENV=production
or
rake db:schema:load RAILS_ENV=production
rake db:fixtures:load RAILS_ENV=production

Setup postfix

For the ticket system, you need to plug-in the MTA, make an alias like this:

nest: "|/usr/app/nest/current/script/runner 'TicketMailer.receive(STDIN.read)'"

starting/stopping/restarting

As user nest:

using capistrano to manage things

$ cap --apply-to ~/dev/nest/trunk Nest

>>  set :repository, "svn+ssh:://svn.riseup.net/var/lib/svn/riseup/nest/trunk
>>  set :deploy_to, "/usr/apps/nest"
>>  set :user, "nest"

User ‘nest’ will be the user on the deployed servers. You must be able to ssh
into this account from your home computer. You will obviously need to change the svn URI.

installing the back-end nestd

The nest daemon (formerly known as taskrunner) is daemon which must be running on the back-end servers. It handles tasks, like ‘create a maildir’, issued by the web app. Nestd is a ruby deamon that polls the nest database every ten seconds. It then runs all the open tasks matching the hostname of the server where the nestd is currently running. If the task was successfully run, the task is marked as completed in the database. If there was an exception thrown, then the task error will be put in the database.

See also the nest console for other possible uses.

Taskrunner may be installed as part of a full nest installation (requiring rails) or as
a stand alone daemon (requiring activerecord only).

These are the instructions for installing as a stand alone. See the nest docs for installing
with nest.

# apt-get install ruby ruby1.8 rubygems rake libyaml-ruby libmysql-ruby1.8
# gem update --system
# gem install rails -y --no-rdoc --no-ri
# mkdir -p /usr/apps/nestd
# adduser nest --home /usr/apps/nestd
# chown nest /usr/apps/nestd
# su nest

get taskrunner code from svn:

$ cd /usr/apps/nestd
$ svn export https://code.riseup.net/svn/nest/trunk/taskrunner . --force

note: i think this does not actually work, since it might require a login

# cp /usr/apps/nestd/init-taskrunner /etc/init.d/nestd

configure nestd

edit /usr/apps/nestd/config/database.yml

running nestd

from the server:

  # /etc/init.d/nestd start

It can be run as root or as an non-privileged user, like ‘nest’. It will try to save the nestd.pid
file in the first of these locations it has write permission to:

1) /var/run/nestd.pid 2) /usr/apps/nest/current/taskrunner/nestd.pid (or wherever the executable lives) 3) $HOME/nestd.pid

nestd logs using logger -t nest. so the entries will show up in /var/log/user.log

More information

Further information can be found in taskrunner/README