gecko-dev/webtools/mozbot/INSTALL.UNIX

                       _           _ 
        m o z i l l a |.| o r g   | | 
    _ __ ___   ___ ___| |__   ___ | |_ 
   | '_ ` _ \ / _ \_  / '_ \ / _ \| __|
   | | | | | | (_) / /| |_) | (_) | |_ 
   |_| |_| |_|\___/___|_.__/ \___/ \__|
   =======================-  2 . 0  -==


INTRODUCTION
------------

This was written as a living document. I (the author of mozbot 2.0)
tried (successfully!) to set up mozbot in a secure environment,
chrooted and setuided. This requires much more than a usual
installation. So, without further ado, over to myself in the field:


GETTING STARTED
---------------

I will first be trying to install mozbot 2.0 on a SPARC machine
running Sun Solaris. These instructions will probably work for any
sane UNIX system. If you use Windows, see the INSTALL.WIN32 file.

   <ianh:~> mkdir mozbot
   <ianh:~> cd mozbot
   <ianh:~/mozbot> version
   Machine hardware:   sun4u
   OS version:         5.7
   Processor type:     sparc
   Hardware:           SUNW,Ultra-60

I already had Emacs 20.7 installed on the machine, for which I must
thank Pavlov. You may, of course, use any editor of your choosing when
doing this, although if you use vi or one of its siblings then don't
even _think_ about asking me for help. (If you can understand vi I
figure mozbot should no problem.)

   <ianh:~> mkdir mozbot
   <ianh:~> cd mozbot

I also had several gigabytes of free disk space. You'll probably need
several hundred megabytes to do all of this (including scratch space).
(I believe the end result was around 30 megs for everything in the
chroot jail directory.)


PERL
----

The first thing on my list was to install Perl.

   <ianh:~/mozbot> mkdir resources
   <ianh:~/mozbot> cd resources
   <ianh:~/mozbot/resources> wget http://www.perl.com/CPAN/src/stable.tar.gz
   <ianh:~/mozbot/resources> tar xvfz stable.tar.gz

Next I read the README and INSTALL files:

   <ianh:~/mozbot/resources> cd perl-5.6.0/
   <ianh:~/mozbot/resources/perl-5.6.0> emacs-20.7 README INSTALL

This told me how to do the next few bits.

   <ianh:~/mozbot/resources/perl-5.6.0> rm -f config.sh Policy.sh
   <ianh:~/mozbot/resources/perl-5.6.0> sh Configure -Dprefix=/u/ianh/mozbot

By providing a prefix, the default installation directory for a lot of
modules I am about to install is automatically set up correctly. So if
you don't install Perl yourself, remember to take this into account!

Note: I didn't change any of the build options, so threads, debugging
and the like are all disabled (or at their defaults). The only things
I changed were that I answered 'n' to the question 'Binary
compatibility with Perl 5.005?', which defaulted to 'y', and I told it
not to install into '/usr/bin/perl'.

   <ianh:~/mozbot/resources/perl-5.6.0> make
   <ianh:~/mozbot/resources/perl-5.6.0> make test
   <ianh:~/mozbot/resources/perl-5.6.0> make install
   <ianh:~/mozbot/resources/perl-5.6.0> cd ..

At this point I had Perl installed correctly in my mozbot directory.


WGET
----

The next thing to install was wget.

   <ianh:~/mozbot/resources> wget ftp://ftp.gnu.org/pub/gnu/wget/wget-1.6.tar.gz
   <ianh:~/mozbot/resources> tar xvfz wget-1.6.tar.gz
   <ianh:~/mozbot/resources> cd wget-1.6
   <ianh:~/mozbot/resources/wget-1.6> emacs-20.7 README INSTALL
   <ianh:~/mozbot/resources/wget-1.6> ./configure --prefix=/u/ianh/mozbot
   <ianh:~/mozbot/resources/wget-1.6> make
   <ianh:~/mozbot/resources/wget-1.6> make install
   <ianh:~/mozbot/resources/wget-1.6> cd ..

No problems, no difficulties.


MOZBOT
------

Now, before going on any further with installing the required modules,
I needed to find what those were. Ergo, the next thing to install was
mozbot. Presumably you already have the relevant files, or know where
to get them, since you are reading a file that comes with the source.

   <ianh:~/mozbot/resources> wget http://www.damowmow.com/mozilla/mozbot/mozbot.tar.gz

There is no configuration, makefile or install script for mozbot,
since there is nothing to compile or particularly install. So, I just
extracted the mozbot tarball directly inside what would be the root of
the file system when I eventually chroot()ed.

   <ianh:~/mozbot/resources> cd ../..
   <ianh:~> tar xvfz mozbot/resources/mozbot.tar.gz

Like all shell scripts, one thing to change about it is the location
of the Perl executable in the shebang.

   <ianh:~> cd mozbot
   <ianh:~/mozbot> emacs-20.7 mozbot.pl

Since I'll be running it from the version of Perl I just installed, I
changed the first line to read:

   #!./bin/perl -wT

Note that this requires me to run mozbot from the mozbot directory. If
you've read the README file, you'll know that this is a prerequisite
of running mozbot anyway.


Net::IRC
--------

If you tried running mozbot now, you'd find it was missing
Net::IRC. So, guess what I installed next? ;-)

   <ianh:~/mozbot> cd resources
   <ianh:~/mozbot/resources> wget http://www.cpan.org/authors/id/FIMM/Net-IRC-0.70.tar.gz
   <ianh:~/mozbot/resources> tar xvfz Net-IRC-0.70.tar.gz
   <ianh:~/mozbot/resources> cd Net-IRC-0.70
   <ianh:~/mozbot/resources/Net-IRC-0.70> emacs-20.7 README
   <ianh:~/mozbot/resources/Net-IRC-0.70> ../../bin/perl Makefile.PL
   <ianh:~/mozbot/resources/Net-IRC-0.70> make
   <ianh:~/mozbot/resources/Net-IRC-0.70> make install
   <ianh:~/mozbot/resources/Net-IRC-0.70> cd ..

It is important to use the Perl we just installed and not any other
Perl on the system, otherwise you'll get incorrect prefixes and
stuff. (I didn't bother to use the wget I just installed...)


Net::SMTP
---------

Yup, you guessed it, Net::SMTP is next.

   <ianh:~/mozbot/resources> wget http://www.cpan.org/authors/id/GBARR/libnet-1.0703.tar.gz
   <ianh:~/mozbot/resources> tar xvfz libnet-1.0703.tar.gz
   <ianh:~/mozbot/resources> cd libnet-1.0703
   <ianh:~/mozbot/resources/libnet-1.0703> emacs-20.7 README
   <ianh:~/mozbot/resources/libnet-1.0703> ../../bin/perl Makefile.PL

I answered 'y' to the question 'Do you want to modify/update your
configuration (y|n) ? [no]', which was asked because the system
had already had libnet installed once.

I kept the defaults for all the options though.

   <ianh:~/mozbot/resources/libnet-1.0703> make
   <ianh:~/mozbot/resources/libnet-1.0703> make test
   <ianh:~/mozbot/resources/libnet-1.0703> make install
   <ianh:~/mozbot/resources/libnet-1.0703> cd ..

This also installed Net::FTP, which is required by some of the modules
(in particular, the FTP module!).


INITIAL CONFIGURATION
---------------------

Now I needed to set up the environment for mozbot. The only real thing
that needs setting up is the PATH variable. So:

   <ianh:~/mozbot/resources> cd ..
   <ianh:~/mozbot> emacs-20.7 run-mozbot-chrooted

Here are the contents of my run-mozbot-chrooted script:

   export PATH=/u/ianh/mozbot/bin
   ./mozbot.pl

It is absolutely imperative that the path not contain '::' or '.'
anywhere, as this will be treated as the current directory, which will
then result in perl exiting with taint errors.

Now we make it executable:

   <ianh:~/mozbot> chmod +x run-mozbot-chrooted

(Note. a sample run-mozbot-chrooted script is shipped with mozbot --
it still requires you to follow all these steps though.)


INITIAL RUN
-----------

At this point, mozbot is runnable... so I ran it!

   <ianh:~/mozbot> ./run-mozbot-chrooted

Note that I'm running it via my script and not directly. If you were
not intending to run mozbot in a chroot() jail environment, then
'./mozbot.pl' would be sufficient.

It prompted me for various things, like servers and so on. Then it
connected without problems but with no modules set up, as I expected.

On IRC, I configured mozbot as I wanted it:

   /query mozbot
   mozbot auth admin password
   newuser Hixie newpass newpass
   bless Hixie
   auth Hixie newpass

I also played a bit with the configuration variables:

   vars Admin throttleTime '2.2'

This was all very well, but no modules makes mozbot a boring bot, so
the next thing was...


FILTERS
-------

I shut down mozbot ('shutdown please') and installed the filters
required by the 'Filters' BotModule.

   <ianh:~/mozbot> cd resources
   <ianh:~/mozbot/resources> wget ftp://ftp.debian.org/pub/mirrors/debian/dists/potato/main/source/games/filters_2.9.tar.gz
   <ianh:~/mozbot/resources> tar xvfz filters_2.9.tar.gz
   <ianh:~/mozbot/resources> cd filters
   <ianh:~/mozbot/resources/filters> emacs-20.7 README
   <ianh:~/mozbot/resources/filters> make

At this point, I edited the Makefile to change /usr/.../ so as to
point in the places we used for installing Perl.

   <ianh:~/mozbot/resources/filters> make install PREFIX=/u/ianh/mozbot
   <ianh:~/mozbot/resources/filters> cd ..
 
I should point out that this didn't go too well and I had to hack
about with the Makefile and my environment and so on, so good luck
(admittedly, Pavlov happened to install a new compiler at the same
time, and didn't bother to install a license for it, so I had a few
more problems than you should, but...).

You should also make sure that the shebang lines in the five relevant
perl scripts that you should make sure ended up in ~/mozbot/bin
actually point to the right perl executable. I had to edit the files
by hand.


Net::Telnet
-----------

In order to insult people, the Rude module needs to Telnet:

   <ianh:~/mozbot/resources> wget http://www.cpan.org/authors/id/JROGERS/Net-Telnet-3.02.tar.gz
   <ianh:~/mozbot/resources> tar xvfz Net-Telnet-3.02.tar.gz
   <ianh:~/mozbot/resources> cd Net-Telnet-3.02
   <ianh:~/mozbot/resources/Net-Telnet-3.02> emacs-20.7 README
   <ianh:~/mozbot/resources/Net-Telnet-3.02> ../../bin/perl Makefile.PL
   <ianh:~/mozbot/resources/Net-Telnet-3.02> make
   <ianh:~/mozbot/resources/Net-Telnet-3.02> make test
   <ianh:~/mozbot/resources/Net-Telnet-3.02> make install
   <ianh:~/mozbot/resources/Net-Telnet-3.02> cd ..

That went a lot smoother than the filters installation, let me tell
you! ;-)


WWW::Babelfish
--------------

The translation module requires a whole bunch of other modules, mainly
due to its dependency on WWW::Babelfish, which requires half of libwww
and also IO::String. libwww itself requires another half a dozen
modules, namely URI, MIME-Base64, HTML::Parser, libnet (which I
installed earlier, thankfully), and Digest::MD5. And HTML-Parser
requires HTML-Tagset!

I found these dependencies out by browsing CPAN reading README files.

   <ianh:~/mozbot/resources> lynx http://www.cpan.org/

Thankfully, they all installed rather smoothly. Here is the complete
list of commands I used to install WWW::Babelfish (starting in the
'resources' directory):

   wget http://www.cpan.org/authors/id/GAAS/MIME-Base64-2.12.tar.gz
   tar xvfz MIME-Base64-2.12.tar.gz
   cd MIME-Base64-2.12
   ../../bin/perl Makefile.PL
   make
   make test
   make install
   cd ..

   wget http://www.cpan.org/authors/id/GAAS/URI-1.11.tar.gz
   tar xvfz URI-1.11.tar.gz
   cd URI-1.11
   ../../bin/perl Makefile.PL
   make
   make test
   make install
   cd ..

   wget http://www.cpan.org/authors/id/S/SB/SBURKE/HTML-Tagset-3.03.tar.gz
   tar xvfz HTML-Tagset-3.03.tar.gz
   cd HTML-Tagset-3.03
   ../../bin/perl Makefile.PL
   make
   make test
   make install
   cd ..

   wget http://www.cpan.org/authors/id/GAAS/HTML-Parser-3.19_91.tar.gz
   tar xvfz HTML-Parser-3.19_91.tar.gz
   cd HTML-Parser-3.1991
   ../../bin/perl Makefile.PL
   make
   make test
   make install
   cd ..

   wget http://www.cpan.org/authors/id/GAAS/Digest-MD5-2.13.tar.gz
   tar xvfz Digest-MD5-2.13.tar.gz
   cd Digest-MD5-2.13
   ../../bin/perl Makefile.PL
   make
   make test
   make install
   cd ..

   wget http://www.cpan.org/authors/id/GAAS/libwww-perl-5.51.tar.gz
   tar xvfz libwww-perl-5.51.tar.gz
   cd libwww-perl-5.51
   ../../bin/perl Makefile.PL
   make
   make test
   make install
   cd ..

   wget http://www.cpan.org/authors/id/GAAS/IO-String-1.01.tar.gz
   tar xvfz IO-String-1.01.tar.gz
   cd IO-String-1.01
   ../../bin/perl Makefile.PL
   make
   make test
   make install
   cd ..

   wget http://www.cpan.org/authors/id/D/DU/DURIST/WWW-Babelfish-0.09.tar.gz
   tar xvfz WWW-Babelfish-0.09.tar.gz
   cd WWW-Babelfish-0.09/
   ../../bin/perl Makefile.PL
   make
   make test
   make install
   cd ..

Yes, this is surreal. I always knew languages were hard.


UUIDGEN
-------

The last module, the UUID generator, requires a program that you'll
find along with mozbot in CVS. You may have this already. If you
don't, then here's how I got my copy:

   <ianh:~/mozbot/resources> export CVSROOT=:pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot
   <ianh:~/mozbot/resources> cvs login

The password is 'anonymous'.

   <ianh:~/mozbot/resources> cvs checkout mozilla/webtools/mozbot/uuidgen
   <ianh:~/mozbot/resources> cd mozilla/webtools/mozbot/uuidgen/
   <ianh:~/mozbot/resources/mozilla/webtools/mozbot/uuidgen> make
   <ianh:~/mozbot/resources/mozilla/webtools/mozbot/uuidgen> cp uuidgen ../../../../../bin
   <ianh:~/mozbot/resources/mozilla/webtools/mozbot/uuidgen> cd ../../../../../

At this point I think I had all the required programs.


MORE THOROUGH CONFIGURATION
---------------------------

Now that I'm ready to run mozbot chroot()ed, it is time to make the
final preparations. Firts, I moved the resources directory out of the
way, since I had finished with it:

   <ianh:~/mozbot> mv resources ../installed-resources

Next I made sure all the rights were set to read-only for people other
than the user:

   <ianh:~/mozbot> chmod -R go-w .

At this point I wanted to make sure the bot started ok, so I ran the
run-mozbot-chrooted script:

   <ianh:~/mozbot> ./run-mozbot-chrooted

That worked. I changed the script to:

   export PATH=/bin
   ./mozbot.pl --chroot /config/default

What's this 'config' thing? Well, since we're about to chown() all the
files to root and then setuid the script to nobody, the bot wouldn't
be able to edit the config file if it was in the same directory as the
source -- so I created a new directory with no rights restrictions,
and moved the configuration file into it:

   <ianh:~/mozbot> mkdir config
   <ianh:~/mozbot> mv mozbot.pl.cfg config/default
   <ianh:~/mozbot> chmod ugo=rwx config
   <ianh:~/mozbot> chmod ugo=rw config/default

In order to not have to change all the perl scripts, I gave them a
fake 'mozbot' directory:

   <ianh:~/mozbot> mkdir u
   <ianh:~/mozbot> mkdir u/ianh
   <ianh:~/mozbot> cd u/ianh
   <ianh:~/mozbot/u/ianh> ln -s / mozbot
   <ianh:~/mozbot/u/ianh> cd ../../

At this point I ran 'su' to drop down to a root shell. Be careful!

I had to copy several library files to a usr/lib directory. To do
this, the 'truss' and 'ldd' tools came in very useful. In particular,
I used 'truss' to watch what calls mozbot was attempting, and 'ldd' to
find what modules dependencies Perl, wget, and the modules had.

Credit should be given to Pavlov for actually doing most of this for
me... I didn't even know 'ldd' existed until he showed me. ;-)

Here is the list of the modules I copied:

   usr/lib:
   ld.so.1          libdl.so.1       libgen.so.1      libmp.so.2 
   libresolv.so.1   libsec.so.1      nscd_nischeck    nss_files.so.1
   libc.so.1        libdoor.so.1     libld.so.2       libnsl.so.1       
   libresolv.so.2   libsocket.so.1   nss_compat.so.1  nss_nis.so.1
   libcrypt_i.so.1  libelf.so.1      liblddbg.so.4    libpthread.so.1   
   librtld.so.1     libthread.so.1   nss_dns.so.1     nss_nisplus.so.1

   usr/platform/SUNW,Ultra-60:
   libc_psr.so.1

You may not need all of these.

I also had to copy /dev/null, /dev/zero, /dev/tcp, /dev/ticotsord and
/dev/udp into a new dev/ directory (hint: use 'tar' to copy devices,
it won't work if you try to do it with 'cp'). I may not have needed
all of these (this was slightly complicated by the fact that on
Solaris the /dev devices are symlinks; I used 'tar' to copy the real
devices from /devices and renamed them when I extracted the tarball):

   total 4
   drwxrwxr-x   2 root     other        512 Mar 30 14:34 .
   drwxr-xr-x  16 root     staff        512 Mar 30 15:47 ..
   crw-rw-r--   1 root     sys       13,  2 Mar 30 14:25 null
   crw-rw-rw-   1 root     sys       11, 42 Jun  6  2000 tcp
   crw-rw-rw-   1 root     sys      105,  1 Jun  6  2000 ticotsord
   crw-rw-rw-   1 root     sys       11, 41 Jun  6  2000 udp
   crw-rw-r--   1 root     sys       13, 12 Jun  6  2000 zero

I had to copy several files from /etc into a new 'etc' directory, in
particular:

   etc:
   group            hosts            netconfig       nsswitch.conf
   passwd           protocols        resolv.conf     wgetrc

You may wish to sanitize your 'passwd' file. For the nsswitch.conf
file you should use the 'nsswitch.dns' file (if you have one) -- make
sure the DNS line is 'dns files' and not 'files dns'. (Profuse thanks
go to rfm from Sun who helped me with this.)

Now I used 'chown' to make every file in /u/ianh/mozbot/ be owned by
root, except the config directory. I also edited 'mozbot.pl' to ensure
that the correct arguments were passed to 'setuid' and 'setgid' --
search for 'setuid' in the source to find the right place.

With that all set up, I finally could run the bot safe in the
knowledge that it was relatively secure:

   <root:/u/ianh/mozbot> ./run-mozbot-chrooted

I hope this has helped you in some way!!!

-- end --