Difference between revisions of "Backup of geonode"

Latest revision as of 11:20, 10 December 2014

>> return to Cedeus IDE

Below I describe the backup procedure taken from: http://docs.geonode.org/en/latest/deploy/backup.html

For migrating from one GeoNode version to another, in particular 1.2 to 2.0 check this document: http://docs.geonode.org/en/latest/tutorials/admin/migrate/index.html#migrate

For moving to another server, the general advise is:

• backup GeoNode - Django project + DB, backup GeoServer (DataDirectory)
• update/upgrade the current server to the latest version
• create a new server + move all the project files
• see this link for a GeoNode style template: http://github.com/GeoNode/geonode-project

Note, it looks like the current GeoNode version package is stored in /usr/share/geonode/ where one can find a zip file.

Data to be backed up

GeoNode has the following types of data

1. uploaded Geodata
   • vector data (shp files, ?kml?) are stored in PostGIS DB, managed via GeoServer
   • raster data (GeoTiff) are stored locally in GeoServer as its own "Datastore" in /usr/share/geoserver/data/data/default
2. uploaded Document data (pdf, txt, doc, xls, zip) and images (jpg , png, tif) are stored locally in GeoNodes www upload folders: /var/www/geonode/uploaded/documents/
3. user account data are stored in local PostgreSQL DB, managed via Django
4. layer and document metadata - stored in local PostgreSQL DB
5. webpage content (in different languages) - stored in django - geonode project folder : /usr/local/lib/python2.7/dist-packages/geonode/locale/
6. webpage - stored in /var/www/geonode/

Backup GeoNode Server (with GeoNode 2.0c8):

Note, below I am using "sudo" in the commands. However, this may not be necessary for creating tar archives, but it is necessary to start/stop services etc.

1. Backup Geonode users DB (psql needs to be running)
   • sudo -u postgres -i pg_dump -c -Fc geonode > /home/ssteinig/geonodedb.backup
2. Stop services
   • sudo service apache2 stop
   • sudo service tomcat7 stop
   • sudo postgresql-8.4 stop o: sudo service postgresql stop
     (I am actually running PSQL 9.1.11 on cedeusgis1)
3. Backup Geonode config & GeoServer data directory:
   • sudo tar -cvzf /home/ssteinig/geonodeConfigBackup.tgz /etc/geonode
     Note, doing this step for backup is good, but when archiving for moving to a different server, it is better to edit the important local_settings.py file manually.
   • sudo tar -cvzf /home/ssteinig/geonodeDataBackup.tgz /var/lib/geoserver/geonode-data/
     Note, this step may not be very useful as the GeoServer data seems to be in a different folder (see next step). So before backing up the folder, check if there is some data inside. But ...
   • it looks like GeoServer data are here: /usr/share/geoserver/data/. In particular the raster data are to be found under /usr/share/geoserver/data/data/default/. So we make a backup of it:
     • sudo tar -cvzf /home/ssteinig/geoserverDataBackup.tgz /usr/share/geoserver/data/
4. In addition, any templates, design changes, uploaded documents, and CSS files will also need to be captured.
   • e.g. my modified css and logo have been in /etc/geonode/media/geonode/img/ ... so they should be backed-up within the config tgz file.
   • but my text modifications to the start pages (i18n strings) are in a different location (...but see as well below where I save the whole Django GeoNode package)
     • sudo tar -cvzf /home/ssteinig/geonodei18nBackup.tgz /usr/local/lib/python2.7/dist-packages/geonode/locale/
   • just to be save I may also do a backup of the GeoNode web folder where I have the CSS, the thumbnails, and uploaded documents (pdf + xls +doc, etc):
     • sudo tar -cvzf /home/ssteinig/geonodeWWWBackup.tgz /var/www/geonode/
   • and a copy of the Django GeoNode folder and the egg-info folder that I needed later:
     • sudo tar -cvzf /home/ssteinig/geonodePyDjangoBackup.tgz /usr/local/lib/python2.7/dist-packages/geonode/
     • sudo tar -cvzf /home/ssteinig/geonodePyDjangoEggBackup.tgz /usr/local/lib/python2.7/dist-packages/GeoNode-2.0-py2.7.egg-info/
5. re-start the previously stopped services: Apache2, Tomcat7, PSQL

Backup Spatial Data DB

Note, the spatial data DB can be run on the same machine as GeoNode or extra. I have installed an extra DB, PostgreSQL/PostGIS that runs on the PostGIS VM (9022 on Lautaro).

To backup the DB we use pg_dump (info : here) as follows (note, I am not sure if the -Fc option is best, to be able to re-store the db on different machines/psqls) :

• sudo -u postgres -i pg_dump -c -Fc geonodegisdb93 > /home/ssteinig/geonodegisdb93.backup

Options mean: "-c" = clean, "-Fc" = custom format + compressed

The data management for this DB is done by GeoServer (see folder /usr/share/geoserver/data/workspaces/geonode/datastore => file datastore.xml). Hence, it makes sense to backup the GeoServer documents on the GeoNode Server as well:

• perhaps stop apache2/tomcat
• sudo tar -cvzf /home/ssteinig/geoserverDataBackup.tgz /usr/share/geoserver/data/
• perhaps start apache2/tomcat

Backup GeoNode using Cron

see http://perspectiveunspoken.wordpress.com/2012/09/09/backup-geonode-using-cron/

Restore GeoNode Server

1. stop all services: apache2, tomcat7, psql
2. restore GeoNode config and GeoServer
   • sudo tar -C / -xvzf /home/ssteinig/geonodeConfigBackup.tgz
     Note it may actually best to not do this step, but to manually edit local_settings.py. The CEDEUS logo image and the CSS that is currently backed up in that archive is also not super important and gets placed in it position when I restore from geonodeWWWBackup.tgz that contains /var/www/geonode. For copyig files do chmod 777 on /etc/geonode/media/ before and then set it back to 755
   • sudo tar -C / -xvzf /home/ssteinig/geonodeDataBackup.tgz
     This step may also not be very useful, as the file does not contain any data since the folder /var/lib/geoserver/geonode-data/ is empty. However, GeoServer data seems to be /usr/share/geoserver/data/ instead)
   • when moving to a different server, I may actually run the following commands:
     • sudo geonode-updateip 10.0.2.15 to get the same db settings
     • modify the file local_settings.py
       • change the geonode db password to the one from the new machine
       • set the Allowed_Hosts variable to "*"
     • sudo geonode collectstatic -v0 to copy the styles/logo into /var/www/geonode/static/geonode
3. restore the template and design files
   • restore the static web page part
     • sudo tar -C / -xvzf /home/ssteinig/geonodeWWWBackup.tgz
   • when moving to another server with a different GeoNoder version (2.0 final instead of 2.0c8), then I got a DB error, ala: "DatabaseError at / [...] <column base_resourcebase.license_id does not exist [...] LINE 1: ...ype_id", "base_resourcebase"."constraints_other", "base_reso... " with the location /usr/local/lib/python2.7/dist-packages/geonode/search/views.py in search_api, line 160
     • hence, I deleted the django folders of GeoNode 2.0 final:
       • /usr/local/lib/python2.7/dist-packages/geonode/ and
       • sudo rm /usr/local/lib/python2.7/dist-packages/GeoNode-2.0.egg-info/
     • and restored the whole django-geonode package backup-ed from 2.0c8 before using
       • sudo tar -C / -xvzf /home/ssteinig/geonodePyDjangoBackup.tgz
       • sudo tar -C / -xvzf /home/ssteinig/geonodePyDjangoEggBackup.tgz
       • This will also restore my i18n strings, i.e. the text on the GeoNode startpage.
4. If not done already (see case above) do restore the i18n strings
   • sudo tar -C / -xvzf /home/ssteinig/geonodei18nBackup.tgz
5. Restore the GeoNode database
   • sudo service postgresql start (or: sudo /etc/init.d/postgresql-8.4 start)
   • sudo -u postgres psql
   • drop database geonode;
   • create database geonode;
   • \q
   • sudo -u postgres -i pg_restore -Fc -d geonode /home/ssteinig/geonodedb.backup
6. When switching Server to a new GeoNode version, then the problem is that the new GeoServer does not know the data yet. Hence, if I want to display a layer in GeoExplorer, then nothing is show. Looking into GeoServer, there are also no layers. Hence, do
   • sudo tar -C / -xvzf /home/ssteinig/geoserverDataBackup.tgz
   • this step does not seem to be necessary (!): sync GeoNode and GeoServer again using geonode syncdb --noinput --all
7. start services: apache2, tomcat7
8. Now, test if existing layers are displayed in GeoExplorer and if new spatial data and documents can be uploaded.
   For me it turned out that GeoExplorer could not load the layers. When choosing the GeoServer view from the GeoNode menu (logged in as admin) I realised that it connected to the local VM IP http://10.0.2.15 instead of http://146.155.17.113:14080/geoserver/web/ . Hence I changed:
   • in GeoServer > Global Settings > Field: Proxy Base URL : http://146.155.17.113:14080/geoserver/ as there was still the cedeusgis1 IP
   • adjusted the SITEURL variable in local_settings.py to http://146.155.17.113:14080/
   • + restart Apache
9. Change Layer info in GeoServer
   • After transferring, the Download Metadata and "Download Layer " section for each layer in GeoNode as well as in GeoServer (Layer>"Metadata links") contained the old IP value for the CSW catalogue (on the GeoNode/GeoServer machine), so this should be adjusted. But maybe it is not a problem if a domain name would be used instead of an IP?
   • so doing geonode updatelayers --ignore-errors solved this issue for me.

Restore DB

First check that PostgreSQL and PostGIS versions are the same to avoid problems. At the moment I am having PostgreSQL v 9.3.2 with PostGIS 2.1.1 on the PostGIS VM. The GeoNode DB uses PostgreSQL 9.1.11. (The postgis version can be received using "SELECT PostGIS_full_version();", but this command works only if one is connected to the spatially enabled DB i.e. geonodegisdb93 and not the default postgres DB)

1. Restore the GIS DB
   • sudo -u postgres psql
   • drop database geonodegisdb93;
   • create database geonodegisdb93;
   • \q
   • sudo -u postgres -i pg_restore -Fc -d geonodegisdb93 /home/ssteinig/geonodegisdb93.backup
2. Restore the GeoNode Data directory on GeoServer (if not done already)
   • sudo tar -C / -xvzf /home/ssteinig/geoserverDataBackup.tgz

In the case of moving the DB to a different machine:

• if not existing: install PostgreSQL with PostGIS
• check PostgreSQL DB access (i.e. modify files pg_hba.conf, postgresql.conf => access for specific IPs or Users)
• create the previously used db user
• create a the same (empty) DB as used previously
• activate the extensions
• do the steps above
• stop services
• modify the DB connection information in GeoServer (click in left side menu on "Stores" and select store with store name "datastore" in workspace "geonode")
• modify the DB connection information in GeoNode (i.e. local_settings.py must be adapted) => why actually twice, in GeoServer and GeoNode?
• restart services