Setting up a web-server

The CMR web-pages

To get started from scratch:

$ cd ~
$ mkdir sphinx
$ cd sphinx
$ git clone https://gitlab.com/camd/cmr.git
$ git clone https://gitlab.com/ase/ase.git
$ mkdir downloads

Put db-files and images for the front-page in the downloads folder. Then run build_web_page.py from your crontab:

MAILTO=...
PP=$HOME/sphinx/ase:$PYTHONPATH
15 3 * * * cd ~/sphinx && PYTHONPATH=$PP python cmr/build_web_page.py

This will create a new compressed tar-file cmr-web-page.tar.gz when there are changes in the cmr or ase Git-repositories. The tar-file contains static HTML and can be copied to a web-server and unpacked there.

The CMR database server

The following should be done as the cmr user on the machine running the PostgreSQL server.

Get the code:

$ cd
$ git clone https://gitlab.com/ase/ase.git
$ git clone https://gitlab.com/camd/cmr.git

and install ASE:

$ cd ase
$ python3.6 -m pip install --editable . --user

Create these folders:

$ mkdir data uwsgi

Install psycopg2 and uwsgi:

$ python3.6 -m pip install --user psycopg2-binary uwsgi

Create a secret password and add the following to ~/.bash_profile:

export ASE_DB_APP_CONFIG=~/cmr/cmrconf.py
export PYTHONPATH=~/cmr/
export CMR_PASSWORD=password
export PATH=$PATH:$HOME/.local/bin

Log in as the postgres user and run the psql command (or do sudo -u postgres psql). Create the ase user and the databases:

$ psql
postgres=# create user ase login password 'password';
postgres=# create database c2db;
postgres=# create database oqmd12;
...
...

See the ~/cmr/cmrconf.py script for how to do this for all the databases.

From Niflheim, copy all the SQLite database files and static files (stored in /home/camd/jensj/CMR-backup/) to the PostgreSQL server:

$ scp *.db *.tar.gz cmr@fysik-cmr01:data/

On the PostgreSQL server, insert the SQLite databases into the PostgreSQL database:

$ cd data
$ ase -T db c2db.db -i postgresql://ase:password@localhost:5432/c2db
...
...

Again, see the ~/cmr/cmrconf.py script for how to automate this.

Now, unpack all the static files:

$ cd ~/uwsgi
$ tar xf ../data/static-c2db.tar.gz
$ tar xf ../data/static-oqmd12.tar.gz
...
...

Install the jsmol code:

$ cd
$ tar xf data/jsmol.tar.gz
$ cd ase/ase/db/static
$ ln -s ~/jsmol

Add this line to crontab:

UWSGI="uwsgi /home/cmr/cmr/uwsgi.ini --daemonize=/home/cmr/uwsgi/uwsgi.log"
@reboot . ~/.bashrc && $UWSGI && wget localhost:8000

and reboot.

Checking that things are OK

Check that things are OK with something like this in a crontab:

def check_cmr_db():
    print('checking CMR database')
    with urlopen('https://cmrdb.fysik.dtu.dk/catapp/json/6') as fd:
        json = fd.read().decode()
    atoms = ase.db.connect(io.StringIO(json), 'json').get_atoms()
    formula = atoms.get_chemical_formula()
    assert formula == 'Ag3Hg'

Updating the databases

As the postgres user, drop and create the database:

$ psql
postgres=# drop database c2db;
postgres=# create database c2db;

Insert new data:

$ cd ~/data
$ ase -T db c2db.db -i postgresql://ase:password@localhost:5432/c2db

and unpack new static files:

$ cd ~/uwsgi
$ rm static/c2db-*
$ tar xf ../data/static-c2db.tar.gz

Update ASE and CMR code:

$ cd ~/ase
$ git pull
$ cd ~/cmr
$ git pull

Then restart:

$ killall uwsgi
$ uwsgi ~/cmr/uwsgi.ini --daemonize=$HOME/uwsgi/uwsgi.log
$ wget localhost:8000