Setting up geonode

From stgo
Jump to: navigation, search

>> return Cedeus IDE


Basic Install for GeoNode v. 2.0

This is the fast install from GeoNode.org Quickstart. Full install instructions, including installation of PostGIS, can be found in this Admin tutorial

  1. Install from PPA in Ubuntu 12.04 -> see http://docs.geonode.org/en/latest/intro/install.html
    1. if Tomcat7 is not installed, do: sudo apt-get install tomcat7 (but see below in step 1.5)
    2. sudo add-apt-repository ppa:geonode/release . Note, if add-apt does not work (command not found) then do first sudo apt-get update && sudo apt-get install python-software-properties
    3. sudo apt-get update
    4. check what geonode versions are available use: apt-cache showpkg geonode. When I looked there was only one version, i.e. 2.0.0+thefinal5
    5. sudo apt-get install geonode (before I could run this I had do to do sudo apt-get install libtomcat7-java so that Tomcat6 will be removed and then I could install tomcat7 - which was needed to for GeoServer - used by GeoNode. Another time I installed with Ubuntu Server as default solo OpenSSH and LAMPS (no tomcat no psql), so I could just do sudo apt-get install tomcat7)
    6. Note that the packages are stored in /var/cache/apt/archives. So they can be copied from there to another machine.
  2. Set the IP address and create a superuser (also described here: http://docs.geonode.org/en/latest/deploy/production.html)
    • sudo geonode-updateip 127.0.0.1 (=> Use the real IP if you have one, and for the VM I actually used the one given by ifconfig: 10.0.2.15)
    • geonode createsuperuser
    • set GeoServer ProxyURL as described in the geonode install instructions. Use user admin with geoserver.
    • If starting up Apache results in the following message: "Could not reliably determine the server’s fully qualified domain name, using 127.0.1.1 for ServerName.", Then do this:
      • sudo vim /etc/apache2/httpd.conf
      • add ServerName localhost to the blank file (the word ServerName needs to stay as such)
      • sudo service apache2 restart
  3. When using a VM with -nat I got at this point "There was a problem loading this page - Please contact your GeoNode administrator (he may have received an email automatically if he configured it properly).". I checked what is going on with tail -f /var/log/apache2/error.log. The log showed me that I should add the bridged IP 146.155.17.113:14080 to allowed hosts in /etc/geonode/local_settings.py. Hence I did changed the corresponding line in local_settings.py to ALLOWED_HOSTS=["*"]
  4. Import data

Read the admin docs and also OSGeo Live DVD Quickstart tutorial

Note, for assigning geoserver tomcat memory, do it for tomcat7 as described for tomcat6 (see OSGeo VM setup) using setenv.sh but in /usr/share/tomcat7/bin : 
#!/bin/sh 
JAVA_OPTS="-Xms512m -Xmx4g -XX:MaxPermSize=270m -server -Djava.awt.headless=true -Djava.util.prefs.systemRoot=$CATALINA_HOME/content/thredds/javaUtilPrefs" 
export JAVA_OPTS

Notes on tuning GeoServer can be found here: http://boundlessgeo.com/whitepaper/geoserver-production-2/

Notes on installing PostGIS

Basic PostgreSQL + PostGIS install

First check if PostgreSQL exists and what version exists. Use which psql which should return the path. Then with the path I did /usr/bin/psql --version to retrieve the version installed. As it was not installed on my brand new PostGIS VMs I installed Postgres 9.3 with PostGIS 2.1 as described here:

  • http://trac.osgeo.org/postgis/wiki/UsersWikiPostGIS21UbuntuPGSQL93Apt :
    • add the repository
      1. sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ precise-pgdg main" >> /etc/apt/sources.list'
      2. wget --quiet -O - http://apt.postgresql.org/pub/repos/apt/ACCC4CF8.asc | sudo apt-key add -
      3. sudo apt-get update
    • install: sudo apt-get install Postgresql-9.3-postgis pgadmin3 postgresql-contrib
    • enable admin pack:
      1. sudo -u postgres psql
      2. CREATE EXTENSION adminpack;
      3. leave psql \q
    • create the users
      1. sudo su - postgres (logs in as postgres user)
      2. createuser -d -E -i -l -P -r -s yourUserName => create user for own login, and a "geonode" user
      3. logout as postgres user exit
    • adjust remote connection permission in postgresql.conf and pg_hba.conf, for details see below (Note, the VM setting also needs to allow connections, i.e. "nat" rules)
    • enable the extensions "postgis" and "postgis_topology" for the particularDB (i.e. geonodegisdb93 > via PgAdmin: Expand the database tree in PGAdmin, and reveal the extensions node > Right-click the extensions node, and click new extension)

However, one can also install from source: see: http://trac.osgeo.org/postgis/wiki/UsersWikiPostGIS20Ubuntu1204src

PostGIS install comments with respect to my first GeoNode install

Here now some more install logs/info with respect to GeoNode:

  • So, Postgres 9.1 was installed, but when I executed "psql" I got this: psql: FATAL: role "ssteinig" does not exist
  • Hence, I first had to creating my account as a user using sudo -u postgres createuser -s $USER (got this from here).
  • Now I could create a db using createdb mygeodb, and connect via commandline with psql -d mygeodb
  • Finally I spatially enabled the db with CREATE EXTENSION postgis; and CREATE EXTENSION postgis_topology;
  • then add a Postgres/postgis port forwarding rule for port 5432

GeoNode with external PostgreSQL DB

  • modify settings in "/etc/geonode/local_settings.py" (before first use of db)
  • the changes may require to run geonode syncdb --noinput --all (and perhaps a tomcat7/apache2 restart?) to sync
  • if the db contains data/layers already (e.g. geonodegisb93 > schemata > public > tables), and those should be visible again from GeoNode, then
    • each table/layer needs to be published (re-loaded) in GeoServer and
    • geonode updatelayers command needs to be run.

Accessing GeoNode - PostgreSQL

Some PostgreSQL commands

  • quit psql: \q
  • change postgres user password do
    • sudo su postgres
    • psql and then:
    • ALTER USER postgres with encrypted password 'your_password';
  • connecting: psql -U geonode93 -d geonodegisdb93 -h localhost -p 5432
  • listing of dbs: psql -l -p 5432 or \l(including spec of port, as the PostGIS VM has postgres 9.1 (:9434) and 9.3 (:9432) installed)
    • or alternatively: SELECT datname FROM pg_database WHERE datistemplate = false;
  • adding / removing users createuser and dropuser e.g.: dropuser geonode -p 5434 but needs to be done as "sudo su - postgres"
  • creating /removing dbs createdb and dropdb e.g.: createdb geonodegisdb93
  • listing of tables: \dt : but this will only list the table from the database currently connected to
    • to see at least something (if nothing added yet) it can also be done using: SELECT table_schema,table_name FROM information_schema.tables ORDER BY table_schema,table_name;
  • showing a tables content, e.g. all layers: SELECT * FROM layers_layer;

Testing GeoExplorer with remote WMS clients

  • a list of running GeoNode instances can be found in this email thread: http://groups.google.com/d/topic/geonode-users/icuYPeL3rIg/discussion
  • a server to test is http://demo.geonode.org/geoserver/wms , or the server from the World Food Program: http://geonode.wfp.org/geoserver/wms (= http://74.3.255.211/geoserver/wms)
  • if errors occur to check what is going on run: tail -f /var/log/apache2/error.log
  • if a 403 error occurs, then the list of allowed hosts needs to be modified in the /etc/geonode/local_settings.py file (perhaps a "sudo service apache2 restart" is needed)
  • if a 500 error occurs, then it could be that the mapping domain => IP does not work and the DNS has to be added to the computer settings (i.e. adding the DNS to /etc/resolv.conf - but see ask-ubuntu : i edited now /etc/network/interface and ifdown/ifup eth0 => but this made me loosing the network connection !!! ...and I had to use VNC to connect to the VM again and do sudo ifup eth0 in a terminal)

Changing the GeoNode logo (GeoNode 2.0)

The original documention on how to do this is here: http://docs.geonode.org/en/latest/deploy/customize.html. However, some things have changed for GeoNode v 2.0. So we have to do the following:

  • The static web documents are in /var/www/geonode/static/geonode. And CSS and images are in the subfolders /css and /img.
  • Create a new logo file. The original logo file is called logo.png and has a dimension of 127x29 px.
  • Create a subfolder in /etc/geonode/media/ => so we have now the folder /etc/geonode/media/geonode/img/
  • Make this img subfolder accessible for ftp (e.g. using chmod 777)
  • copy the new logo into the etc/.../img subfolder
  • run sudo geonode collectstatic -v0
  • this should copy the new logo image into /var/geonode/static/geonode/img/.

Changing the GeoNode Theme (GeoNode 2.0)

Change of GeoNode web page colors

See comments above... for simple change of the CSS. I actually took the bootstrap? created base.css (see also base.less) and changed the color/backgroundcolor for web page elements (see personal notes in my log-book).

Apart from that: Everything else is more complicated and requires lots of others software (bootstrap, etc):

Translation + Change of GeoNode Welcome text

GeoNode is a Django Project - well, according to Daniel they put it into package for distribution (see here). So for that reason the GeoNode Django "project" can be found under the following path /usr/local/lib/python2.7/dist-packages/geonode/locale.

To edit the text of the GeoNode welcome page, one can edit the English, Spanish, etc, locales stored under /usr/local/lib/python2.7/dist-packages/geonode/locale. However, I need to check if they should get replaced when an upgrade is performed. The translation is in .po files. Information on how to compile these files can be found here: http://docs.djangoproject.com/en/1.6/topics/i18n/translation/#how-to-create-language-files

However, what did I do to change the german translation for the front page.

  • copy (backup first) and open /locale/de/LC_MESSAGES/django.po
  • check out in the file /templates/index.html what strings are used
  • modify the text for the corresponding language keys in django.po
  • save and cope the file to the server (ensure that you have a backup of the original!!! and use chmod to enable copying)
  • compile the changed language file using: django-admin.py compilemessages -l de or: sudo geonode compilemessages
  • perhaps do a sudo service apache2 restart?
  • now check if the german text changed.

Note, for a new language file, e.g. Chile Spanish one would create the template po files using: django-admin.py makemessages -l es_cl or: sudo geonode makemessages -l es_cl first. (I think so ;)

Adding own Tiles to GeoExplorer

Own background tiles can be created with TileMill and then distributed with TileStream. Daniel has done that on Lautaro, and the tiles can be access from here: http://lautaro.ing.puc.cl/map/v2/Fondef/{z}/{x}/{y}.png . The max zoom level seems to be "16".

How to enable showing of my own tiles served by TileStream can be found in the OpenQuake bugfix for "hack the MapBoxSource gxp plugin to allow for our tilestream data". The Git pull request can be found here. According to this, the following changes are necessary:

  1. check the js file by Ben Wyss TileStreamSource.js which is based on MapBoxSource.js plugin. The MapBoxSource.js is part of /var/www/geonode/static/geoexplorer/script/GeoExplorer.js in minified version, but can be seen raw in GeoExplorer-debug.js. Insert this code into GeoExplorer.js and replace with the IP of my private TileStream server.
  2. modifying local_settings.py by adding gxp_tilestreamsource equally as gxp_mapboxsource is added (+ restart Apache2 service)

Note: When I activate in local_settings.py a TileStream server that is not accessible (i.e. timeout because my personal TileStream server is behind the PUC - San Joaquin wall), then the GeoExplorer.js does not get loaded completely (I guess) and I can't compose maps. In that case the entry in local_settings.py has to be deactivated/commented out to be able to compose maps in GeoExplorer again. For that reason, at the moment I leave access to my TileStream VM on Lautaro commented out, but for demonstration purposes I added the OpenQuake.org TileStream source that has some nice maps.

PS: some more info is perhaps found in the tutorial: http://docs.geonode.org/en/latest/reference/developers/settings.html

Working fill-in code for OpenQuake tile layers for GeoExplorer.js

  • The OpenQuake TileStream server can be accessed via: http://tilestream.openquake.org (this link allows to browse the tile sets as well using Leaflet)
  • interestingly the following code executes first a request for a tile server description (a json document) to get info about all existing tile sets. This json metadata file is obtained from: http://tilestream.openquake.org/api/v1/Tileset .Lautaro doesn't seem to have installed api/v1/Tileset (at least I could not found it in /srv/www/.
  • Tiles from OpenQuake server are then accessed using this scheme: http://tilestream.openquake.org/v2/" + tileStreamLayerName + "/${z}/${x}/${y}.png 

  var tilestreamPlugin = {
     ptype: "gxp_tilestreamsource",
     title: "TileStream Layers",
  
   /** api: method[createStore]
    *
    *  Creates a store of layer records.  Fires "ready" when store is loaded.
    */
   createStore: function() {  
 
       var options = {
           sphericalMercator: true,
           wrapDateLine: true,
           //numZoomLevels: 7
           numZoomLevels: 16
       };  
       var layers = new Array();
       $.getJSON(
       	'http://tilestream.openquake.org/api/v1/Tileset',
       	function(json) {
           	   for (var i=0; i < json.length; i++) {
               	// Get the tile name and zoom level from the tilestream API
       		var tileStreamLayerName = json[i].id;
               	var tileMaxZoom = json[i].maxzoom;    
               	// Build the list of layers
                       var newLayer = new OpenLayers.Layer.XYZ(
                                       tileStreamLayerName,
                   	                ["http://tilestream.openquake.org/v2/" + tileStreamLayerName + "/${z}/${x}/${y}.png"],  
                   	                OpenLayers.Util.applyDefaults(
                                             {
                       	                   layername: tileStreamLayerName,
                       	                   numZoomLevels: tileMaxZoom
                   	                      }, 
                                            options)
               		        );
                                      layers.push(newLayer);
           	   }
           	   newLayerStore(layers);
       	}
       ); 
       var plugin = this;
       var newLayerStore = function(layers) {
           plugin.store = new GeoExt.data.LayerStore({
               layers: layers,
           	fields: [
                   {name: "source", type: "string"},
                   {name: "name", type: "string", mapping: "layername"},
                   //{name: "abstract", type: "string"},
                   {name: "group", type: "string"},
                   {name: "fixed", type: "boolean"},
                   {name: "selected", type: "boolean"}
               ]
           });
           plugin.fireEvent("ready", this);
       };
   },
   /** api: method[createLayerRecord]
    *  :arg config:  ``Object``  The application config for this layer.
    *  :returns: ``GeoExt.data.LayerRecord``
    *
    *  Create a layer record given the config.
    */
   createLayerRecord: function(config) {
       var record;
       var index = this.store.findExact("name", config.name);
       if (index > -1) {
           record = this.store.getAt(index).copy(Ext.data.Record.id({}));
           var layer = record.getLayer().clone();
           // set layer title from config
           if (config.title) {
               /**
                * Because the layer title data is duplicated, we have
                * to set it in both places.  After records have been
                * added to the store, the store handles this
                * synchronization.
                */
               layer.setName(config.title);
               record.set("title", config.title);
           }
           // set visibility from config
           if ("visibility" in config) {
               layer.visibility = config.visibility;
           }
           record.set("selected", config.selected || false);
           record.set("source", config.source);
           record.set("name", config.name);
           if ("group" in config) {
               record.set("group", config.group);
           }
           record.data.layer = layer;
           record.commit();
       }
       return record;
   }
 };
 gxp.plugins.TileStreamSource = Ext.extend(gxp.plugins.LayerSource, tilestreamPlugin);
 Ext.preg(gxp.plugins.TileStreamSource.prototype.ptype, gxp.plugins.TileStreamSource);
 Ext.ns("gxp.plugins");

Simple working fill-in code for Lautaro - Fondef tile layers for GeoExplorer.js

Link to zipped version of the GeoExplorer.js minified version. 

var tilestreamPlugin = {
   
   /** api: ptype = gxp_mapboxsource */
   ptype: "gxp_tilestreamsource",
   
   /** api: config[title]
    *  ``String``
    *  A descriptive title for this layer source (i18n).
    */
   title: "CEDEUS TileStream Layers",
   
   /** api: method[createStore]
    *  Creates a store of layer records.  Fires "ready" when store is loaded.
    *  sstein: in my case it creates only one layer "Fondef" that is part of the store
    *  I removed the original loop to gather layers from /api/v1/Tileset json
    */
   createStore: function() {
       
       var options = {
           sphericalMercator: true,
           wrapDateLine: true,
           //numZoomLevels: 7
           numZoomLevels: 16
       };  
       
       var layers = new Array();      
       var tileStreamLayerName = "Fondef";
       var tileMaxZoom = 17;
 
       var newLayer = new OpenLayers.Layer.XYZ( 
                               tileStreamLayerName,
                   		["http://lautaro.ing.puc.cl/map/v2/" + tileStreamLayerName + "/${z}/${x}/${y}.png"],  
                   		OpenLayers.Util.applyDefaults(
                   			{
                       			layername: tileStreamLayerName,
                       			numZoomLevels: tileMaxZoom
                   		        }, 
                   			options)
               		);
       layers.push(newLayer);  
		 		
       var plugin = this;
       var newLayerStore = function(layers) {
           plugin.store = new GeoExt.data.LayerStore({
               layers: layers,
           	fields: [
                   {name: "source", type: "string"},
                   {name: "name", type: "string", mapping: "layername"},
                   //{name: "abstract", type: "string"},
                   {name: "group", type: "string"},
                   {name: "fixed", type: "boolean"},
                   {name: "selected", type: "boolean"}
               ]
           });
           plugin.fireEvent("ready", this);
       };
        newLayerStore(layers);
   },
   
   /** api: method[createLayerRecord]
    *  :arg config:  ``Object``  The application config for this layer.
    *  :returns: ``GeoExt.data.LayerRecord``
    *
    *  Create a layer record given the config.
    */
   createLayerRecord: function(config) {
       var record;
       var index = this.store.findExact("name", config.name);
       if (index > -1) {

           record = this.store.getAt(index).copy(Ext.data.Record.id({}));
           var layer = record.getLayer().clone();

           // set layer title from config
           if (config.title) {
               /**
                * Because the layer title data is duplicated, we have
                * to set it in both places.  After records have been
                * added to the store, the store handles this
                * synchronization.
                */
               layer.setName(config.title);
               record.set("title", config.title);
           }

           // set visibility from config
           if ("visibility" in config) {
               layer.visibility = config.visibility;
           }
           
           record.set("selected", config.selected || false);
           record.set("source", config.source);
           record.set("name", config.name);
           if ("group" in config) {
               record.set("group", config.group);
           }
           record.data.layer = layer;
           record.commit();
       }
       return record;
   }
};
 
gxp.plugins.TileStreamSource = Ext.extend(gxp.plugins.LayerSource, tilestreamPlugin);
Ext.preg(gxp.plugins.TileStreamSource.prototype.ptype, gxp.plugins.TileStreamSource);
Ext.ns("gxp.plugins");

GeoNode Settings

ToDo: check for settings that still need to be adjusted for a running system:
see http://docs.geonode.org/en/latest/reference/developers/settings.html or for GeoNode 2.0 production setting: http://docs.geonode.org/en/2.0/tutorials/admin/production.html?

Print settings

http://docs.geonode.org/en/2.0/tutorials/admin/production.html?highlight=printing

For printing the MapFish printing module for GeoServer is used.

the print config file is under /usr/share/geoserver/data/printing/config.yaml

if not already added automagically (was done already for the servers IP address) change here: 

hosts:
 - !dnsMatch
   host: YOUR_SERVER_IP_ADDRESS
   port: 80

Perhaps remove the port. Then restart GeoServer / Tomcat.

However, for my GeoNode VM this did not work. So to check why it may not work inspect GeoServers log file: tail -f /usr/share/geoserver/data/logs/geoserver.log. Here I saw that there were no permissions for the VM, i.e. cedeusdata.geosteiniger.cl: 

2015-01-20 12:53:03,558 ERROR [servlet.BaseMapServlet] - Error while generating PDF
DocumentException: 
org.mapfish.print.InvalidJsonValueException: spec.layers[1].baseURL has an invalid value: http://cedeusdata.geosteiniger.cl/geoserver/wms?SERVICE=WMS& (spec.layers[1].baseURL has an invalid value: http://cedeusdata.geosteiniger.cl/geoserver/wms?SERVICE=WMS&)

So, I added to the config.yaml file the following lines: 

 - !dnsMatch
   host: cedeusdata.geosteiniger.cl

Now printing to pdf is working after Tomcat restart :)

Debugging GeoNode

(i) Case: dataset is not shown in GeoExplorer. Options to check what is going on:

  • GeoExplorer could not be loaded correctly (due to my mods + changes in local_settings.py). In this case, there should be an error message in the Firefox Web-Dev console.
    • e.g. in one case I did apt-get upgrade and GeoNode was upgraded. With that a new GeoExplorer.js file was installed and my local_settings.py advised to load the OpenQuake tileset plugin, which did not exist => solution, replace new with old/modified GeoExplore.js file.
  • No connection to DB?
    • the error should be visible with tail -f /var/lib/tomcat7/logs/catalina.out => check also what happens when doing a restart of Tomcat7.
    • connect with PgAdmin to see if the DB is accessible
  • Something with GeoServer? as above check also catalina.out logs. Switch to the GeoServer admin interface in GeoNode. Check if datasets/layers are loaded. Check if they are displayed in the Layer Preview > OpenLayers.

GeoNODE Authentication

GeoServer needs to know from GeoNode which users can see certain layers. So GeoServer needs to get the authentiaction data from GeoNode. How GeoServer can contact GeoNode is set in a config file.

for more general reading on authentication see here: http://en.wikipedia.org/wiki/Basic_access_authentication

Set GeoNode VM Server Name cedeusdata.geosteiniger.cl

The GeoNode VM Server Name has been set within the file /etc/apache2/sites-enabled/geonode

The content of the "sites-enabled/geonode" file is as follows: 

WSGIDaemonProcess geonode user=www-data threads=15 processes=2
<VirtualHost *:80 146.155.17.19:13080>
   ServerName geonode.cedeusgeonode.ing.puc.cl
   ServerAlias cedeusdata.geosteiniger.cl
   ServerAdmin webmaster@localhost

   ErrorLog /var/log/apache2/error.log
   LogLevel warn
   CustomLog /var/log/apache2/access.log combined

   WSGIProcessGroup geonode
   WSGIPassAuthorization On
   WSGIScriptAlias / /var/www/geonode/wsgi/geonode.wsgi

   <Directory "/var/www/geonode/">
      Order allow,deny
       Options Indexes FollowSymLinks
       Allow from all
       IndexOptions FancyIndexing
   </Directory>

   Alias /static/ /var/www/geonode/static/
   Alias /uploaded/ /var/www/geonode/uploaded/
   Alias /robots.txt /var/www/geonode/robots.txt

   <Directory "/var/www/geonode/uploaded/documents/">
      Order allow,deny
      Deny from all
   </Directory>

   <Proxy *>
     Order allow,deny
     Allow from all
   </Proxy>

   ProxyPreserveHost On
   ProxyPass /geoserver http://localhost:8080/geoserver
   ProxyPassReverse /geoserver http://localhost:8080/geoserver
</VirtualHost>

To have this VM accessible under http://cedeusdata.geosteiniger.cl I also had to use C-panel > Simple DNS Zone editor > "Add an A Record" in the GeoSteiniger domain admin, and to create a proxy forward in on cedeusgeonode machine: i.e. in the file /etc/apache2/sites-available/000-default.conf I added the following entry: 

#CedeusData geosteiniger
<VirtualHost *:80>
   ServerName cedeusdata.geosteiniger.cl
   ProxyRequests Off
   ProxyPreserveHost On
   ProxyPass / http://146.155.17.19:13080/
   ProxyPassReverse / http://146.155.17.19:13080/
</VirtualHost>
Retrieved from "http://wiki.geosteiniger.cl/mediawiki-1.22.7/index.php?title=Setting_up_geonode&oldid=2232"