Create virtual machine via VMware's "Easy Install" easy install option. Use the username "bootstrap" with the password "bootstrap". (We'll create other accounts via chef.)

$ sudo apt-get update && sudo apt-get upgrade

Install openssh:

$ sudo apt-get install openssh-server

Change hostname:

$ sudoedit /etc/hostname # must be unique across chef-managed nodes!
$ sudoedit /etc/hosts # change "ubuntu" to whatever the hostname is

Get IP address:

$ ifconfig

Reboot:

$ sudo shutdown -r now

Create deploy account on git server:

$ sudo useradd -m deploy
$ sudo -u deploy ssh-keygen -N '' -f /home/deploy/.ssh/id_rsa
$ sudo -u deploy cp /home/deploy/.ssh/id_rsa.pub /home/deploy/.ssh/authorized_keys
$ sudo cat /home/deploy/.ssh/id_rsa
# copy the public key into the deploy_to key of the apps/beebo
# data bag via 'knife data bag edit apps beebo'
# ... or into /var/www/beebo.org/id_deploy directly

Note that the deploy account needs read-only access to the deploy@grace.beebo.org:/var/lib/git/beebo.org repository.

Bootstrap from host:

$ knife bootstrap 172.16.152.131 -x bootstrap -Pbootstrap -r 'role[base],role[beebo]' --distro ubuntu10.04-gems --sudo

(Enter "bootstrap" again as the sudo password when prompted.)

Getting Beebo (and other sites) running on Ubuntu

What follows is mostly instructions for my own use that tell me how to get beebo.org running on Ubuntu, with separate instructions for both a virtual machine running under VMware Fusion, and on gandi's virtual server. However, they may be somewhat helpful to anyone trying any of these applications on their own Ubuntu server.

Prerequisites

You must have knife installed, and be able to run knife node list before configuring a server to run Beebo. To do this, you need to do two things: install chef, and download and install the configuration files associated with the beebo chef organisation.

Install Chef

To install chef on OS X or Ubuntu, follow Opscode's Quick Start Instructions.

On other platforms, follow Opscode's Verbose Instructions.

You probably want to install the net-ssh-multi rubygem at the same time as installing chef itself; this is needed for knife bootstrap.

Download Config Files

The config files are stored in git. Simply clone the $GITROOT/chef repository into ~/.chef:

$ cd ~
$ git clone $GITROOT/chef .chef

At this point knife node list should show you a list of nodes connect to the beebo organisation:

$ knife node list
[
  "babyboom.localdomain",
  "syd.beebo.org"
]

Initial System Setup

That is, what you need to do, under different environments, to get the system to a point where chef can be bootstrapped via knife bootstrap.

In the following sections, it is assumed that the name of the regular user account is mjs.

VMware Fusion 3.1

Use the "Easy Install" process, and create the account mjs.
Log in to the virtual machine via VMware's terminal using the username and password you supplied in the "Easy Install" dialog.
Install the openssh-server package via
```
$ sudo apt-get install openssh-server
```
Set the name of the server:
1. Change /etc/hostname to the hostname of the machine. (e.g. babyboom--this shouldn't include a dot.)
2. Modify /etc/hosts replacing every instance of the string ubuntu with the hostname. (e.g. babyboom.)
Once the ssh server is running, modify /etc/hosts on the host and point the following hostnames at the IP address of the virtual machine:

beebo.local www.beebo.local static.beebo.local scratch.local

(Use ifconfig on the VM to find the IP address.) It should now be possible to ssh mjs@beebo.local.
Enable shared folders, and make ~/workspace on the host available as /mnt/hgfs/workspace on the VM. (If this doesn't "just work", you'll probably need to recompile and reinstall the HGFS kernel modules. Do this by running the following commands in order:
- sudo apt-get update && sudo apt-get -u upgrade
- sudo apt-get install build-essential linux-headers-$(uname -r)
- sudo vmware-config-tools.pl
Ensure that all source code dependencies (e.g. Zend Framework) have been downloaded and installed in the expected location. (See README.md in the root directory of the beebo repository.)

VirtualBox 3.x

(Note: I don't use VirtualBox anymore; this hasn't been tested in a while.)

Download Ubuntu Server Edition from http://www.ubuntu.com/getubuntu/download. We're going to install the JeOS edition (minimal install designed for Virtual Machines), but this ISO is now combined with the standard server ISO.
Follow the standard installation instructions to boot and install Ubuntu from the *.iso.
1. The virtual machine will probably boot into the graphical Ubuntu installer. Choose your language, then hit F4 (Fn-F4 on Macs), choose "Install a minimal virtual machine", and then choose "Install Ubuntu Server." (This selects the JeOS edition on Ubuntu Server Edition 8.10 or later. If you have Ubuntu 8.04 you need to download the (separate) 8.04 ISO.)
2. Continue through the (text mode) installer as usual. Note that if you want to be offered United Kingdom time zones you need to choose "United Kingdom" for your language.

Update the package list, and install any updates:

$ sudo apt-get update && sudo apt-get -u upgrade

Install/reinstall "Linux Guest Additions".
1. Install various packages necessary for compiling the drivers (if reinstalling the guest additions after an upgrade, check that these are actually installed, especially the kernel headers):
```
$ sudo apt-get install dkms # "Dynamic Kernel Module Support Framework"
$ sudo apt-get install build-essential # compiler, compiler tools
$ sudo apt-get install linux-headers-$(uname -r)
```
2. From the "Devices" menu, select "Install Guest Additions...". This does the equivalent of putting a CDROM in disk drive.
3. Mount this CDROM:
```
$ mount /dev/cdrom
```
4. Run the installer:
```
$ cd /cdrom && sudo sh ./VBoxLinuxAdditions-x86.run
```
5. Recompile the guest kernel modules (is this actually necessary?):
```
$ sudo /etc/init.d/vboxadd setup
```
Install the ssh server:
```
$ sudo apt-get install openssh-server
```
Shutdown so that we can configure networking:
```
$ sudo shutdown -h now
```
Configure networking.
1. Even though the networking is running at this point (you should be able to ping google.com from your virtual machine, for example), for some reason VirtualBox arranges networking so that you can't ssh in to your virtual machine from the host. You need to configure VirtualBox for forward ports on the host to ports on the guest. You do this by editing the virtual machine's configuration file, which is probably stored in a location like ~/Library/VirtualBox/Machines/Ubuntu JeOS 8.04/Ubuntu JeOS 8.04.xml. In the <ExtraData> element (should be near the top), add the following lines:
```

<ExtraDataItem name="VBoxInternal/Devices/pcnet/0/LUN#0/Config/ssh/HostPort" value="8022"/>
<ExtraDataItem name="VBoxInternal/Devices/pcnet/0/LUN#0/Config/ssh/GuestPort" value="22"/>
<ExtraDataItem name="VBoxInternal/Devices/pcnet/0/LUN#0/Config/ssh/Protocol" value="TCP"/>
<ExtraDataItem name="VBoxInternal/Devices/pcnet/0/LUN#0/Config/apache/HostPort" value="8080"/>
<ExtraDataItem name="VBoxInternal/Devices/pcnet/0/LUN#0/Config/apache/GuestPort" value="80"/>
<ExtraDataItem name="VBoxInternal/Devices/pcnet/0/LUN#0/Config/apache/Protocol" value="TCP"/>
```
  This maps localhost:8080 to port 80 on the virtual machine (for http) and localhost:8022 to port 22 on the virtual machine (for ssh).
2. Restart the virtual machine. You should now be able to ssh in to the guest with:
```
$ ssh -p 8022 mjs@localhost
```
Configure shared folders.
1. In the VirtualBox GUI on the host, configure "Shared Folders", so that /User/mjs/workspace is made available under the name workspace. (You may need to shut the guest down first.)
2. At this point, it should be possible to mount the shared folder to the directory /mnt via:
```
$ sudo mount -t vboxsf workspace /mnt
```
3. To arrange for this to happen automatically at boot, add the following line to /etc/fstab:
```
workspace /mnt vboxsf uid=1000,gid=1000
```
  This will mount the drive as if it is owned by the user with user id 1000, and group id 1000; check to see what ids your user has with id.
4. Reboot, or apply the mounts manually:
```
$ sudo mount -a
```

For more information see the in-application help. (A PDF file.)

Gandi

In the VPS management interface, create an Ubuntu 10.10 64-bit virtual machine, with ssh access via the username mjs.
Allow mjs to execute sudo:
```
$ su - # become root
$ echo "mjs ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers
```
NOTE: If editing this file by hand, use sudoedit not sudo $EDITOR ....
Edit /etc/default/gandi and change HOSTNAME_TYPE from short to full:
```
$ vim /etc/default/gandi
```
(This sets /etc/hostname to the full hostname, and ensures that hostname -f returns the fully qualified domain name--these are needed for chef.)
Reboot, and then check that hostname -f does in fact return the full domain name:
```
$ sudo shutdown -r now
# ...
$ hostname -f
syd.beebo.org
```

Install Chef

To install everything else needed to get the site running, use knife bootstrap: this installs chef-client on the server, and assigns it a particular role.

Before doing this, check that the value of the node[:beebo][:servers] attribute for the host you're bootstrapping is a list of the virtual hosts that the server is supposed to provide. Do this via e.g.

$ knife node edit babywhimsy.local

For example, to install chef-client on the virtual machine, and get it to "converge" to the vm role, run the following command on the host:

$ knife bootstrap $HOSTNAME -x $USERNAME -P$PASSWORD -r 'role[vm]' --distro ubuntu10.04-gems --sudo
# e.g.
$ knife bootstrap beebo.local -x mjs -PXXX -r 'role[vm]' --distro ubuntu10.04-gems --sudo
# (remember to specify the password!)

See the chef documentation for more information, including the distributions that are supported. (10.04 works with 10.10.)

(We're using the gems distro instead of apt because otherwise we seem to get 64-bit binaries installed on 32-bit systems.)

Note that the settings that will be applied are a function of the role (passed in the knife bootstrap command line) and the attributes assigned to the node (i.e. hostname). In particular, if you're using a completely new hostname, this probably won't do what you expect until you set the attributes as appropriate. For example, the location of Apache's document root is configured via the node's node[:beebo][:servers] attribute. This must be set correctly for the server to be configured correctly.

Install password, source, data, symlinks

At this point, the server is configured to run beebo.org, but the source code, databases, and a few other things are missing.

Password

So that the bulk of the source, including config files, can be placed on a public server, the passwords stored within the config are encrypted. The decryption procedure reads the file config/password to decrypt them; add the current password to this file.

Source code and symlinks

The location of the source code is configured via the node[:beebo][:servers] attribute in chef. (See chef/cookbooks/beebo/attributes/default.rb.) To arrange for the source code to be in the correct location, the following commands may be useful.
```
$ git clone --branch dev ../git/beebo.org dev.beebo.org
```
This clones the live branch of the git repository in ../git/beebo.org into dev.beebo.org, using hardlinks to save space and increase performance.

Currently, the source code is cloned into /srv/slammer/www/beebo.org on syd, and a post-receive script in /srv/slammer/git/beebo.org/hooks does:
```
#!/bin/bash

unset GIT_DIR

pushd /var/www/dev.beebo.org >/dev/null
git pull
cd lately
jekyll --no-auto
popd

pushd /var/www/beebo.org >/dev/null
git pull
cd lately
jekyll --no-auto
popd
```
(The unset GIT_DIR is very important! (Why?) Also, make sure the script is executable.)

Check out the latest version of Zend Framework:

# In /srv/slammer/www
$ svn co http://framework.zend.com/svn/framework/standard/tags/release-1.11.2/library/Zend

Then symlink this into the source at lib/Zend.

Similarly, create a symlink from data/quotes to misc/quotes.
And finally, create a symlink from web/stuff/tech to misc/tech.

MongoDB

MongoDB is expecting to find its databases in /var/lib/mongodb (see /etc/init/mongodb). (Though you probably want this to be a symlink to a larger drive.) To get the database loaded into MongoDB, the following commands may be useful.

Dump existing database to backup directory in current directory:
```
$ mongodump -o backup
```
Import contents of backup directory into running MongoDB instance:
```
$ mongorestore backup
```
Stop MongoDB:
```
$ sudo stop mongodb
```
Start MongoDB:
```
$ sudo start mongodb
```

SQLite

The location of the databases is store in config files (e.g. in config/beebo.local/config.ini), but it's recommended that they actually go into /var/lib/sqlite (which again, can be a symlink).

To move existing SQLite databases, the following commands may be useful.

Export SQLite database:

$ sqlite3 tracker.db .dump | gzip tracker.sql.gz

Import SQLite database:

$ gunzip -c tracker.sql.gz | sqlite3 tracker.db

FAQs and HOWTOs

HOWTO: Back Up

Use the script called backup in beebo/bin to back the various databases up into /srv/backup.
Copy the backups onto an offsite machine with a command like rsync -CLvax mel.beebo.org:/srv/backup .

HOWTO: Restore

You'll have to do it manually, I'm afraid.

HOWTO: Create SQLite database files from `*.sql.bz2` dumps

export SRC=~/Documents/Backup/beebo/2010-06-24/sqlite
export DST=~/workspace/beebo/db
for f in $SRC/*.bz2 ; do bzip2 -dc $f | sqlite3 $DST/$(basename $f .sql.bz2).db ; done

HOWTO: Encrypt passwords

From the root directory, run something like:

$ DOCUMENT_ROOT=web php bin/encrypt jjjj
encrypted:v2P+7KCtn+3zojavwjbpnw==

HOWTO: Decrypt passwords

The config contains a number of encrypted passwords. To decrypt these you need a password file that is not contained in the repository. It needs to go in config/password, should simply be the password itself, as text:

$ echo "ajaafsadsoj" > config/password

FAQ: Why did the shared folders stop working?

Perhaps because the kernel has been upgraded--to fix it, run

$ sudo /usr/bin/vmware-config-tools.pl

to reinstall VMware's kernel modules. If it complains about missing headers, add the appropriate headers with

$ sudo apt-get install linux-headers-`uname -r`

HOWTO: Copy SQLite databases from live to another machine

Do something like this:

function copy-sqlite-from-live {

    if [ "$1" = "" ]; then
        echo "usage: $FUNCNAME database.db"
        return
    fi

    rm -f /var/lib/sqlite/$1
    ssh -C mjs@grace.beebo.org sqlite3 /var/lib/sqlite/$1 .dump | sqlite3 /var/lib/sqlite/$1
    # Make sure apache can read/write the database
    chgrp www-data /var/lib/sqlite/$1
    chmod 664 /var/lib/sqlite/$1

}

copy-sqlite-from-live beebo.db           # creates /var/lib/sqlite/beebo.db
copy-sqlite-from-live tracker-current.db # creates /var/lib/sqlite/tracker-current.db

Restart Apache after copying databases! Some (and only some) Apache processes might have a connection open to the old database.

BEEBO - Tech Notes

Hi There

Around the web

Getting Beebo (and other sites) running on Ubuntu

Prerequisites

Install Chef

Download Config Files

Initial System Setup

VMware Fusion 3.1

VirtualBox 3.x

Gandi

Install Chef

Install password, source, data, symlinks

Password

Source code and symlinks

MongoDB

SQLite

FAQs and HOWTOs

HOWTO: Back Up

HOWTO: Restore

HOWTO: Create SQLite database files from `*.sql.bz2` dumps

HOWTO: Encrypt passwords

HOWTO: Decrypt passwords

FAQ: Why did the shared folders stop working?

HOWTO: Copy SQLite databases from live to another machine

BEEBO - Tech Notes

Hi There

Around the web

Getting Beebo (and other sites) running on Ubuntu

Prerequisites

Install Chef

Download Config Files

Initial System Setup

VMware Fusion 3.1

VirtualBox 3.x

Gandi

Install Chef

Install password, source, data, symlinks

Password

Source code and symlinks

MongoDB

SQLite

FAQs and HOWTOs

HOWTO: Back Up

HOWTO: Restore

HOWTO: Create SQLite database files from *.sql.bz2 dumps

HOWTO: Encrypt passwords

HOWTO: Decrypt passwords

FAQ: Why did the shared folders stop working?

HOWTO: Copy SQLite databases from live to another machine

HOWTO: Create SQLite database files from `*.sql.bz2` dumps