Difference between revisions of "Nominatim for Traffic Accident Database"

Revision as of 14:17, 4 March 2015

>> return Cedeus_IDE

Objective

Develop a script for automated geo-referencing of Chiles transit accident database.

Tools

• Nominatim
• MS-Access / LibreOffice : need to load the MS Access DB from Carabineros de Chile in OpenOffice/LibreOffice: http://askubuntu.com/questions/187389/is-it-possible-to-open-an-access-2010-database-file-without-using-wine-or-virtua/519571#519571
• iPython + Notebook

Installing Nominatim

• Nominatim installation manual: https://wiki.openstreetmap.org/wiki/Nominatim/Installation

Setup of Nominatim VM

1. copying basicubunutu1404.vdi file and renaming to nominatim.vdi
2. creating the nominatim VM on CedeusDB (ip.18):
   • VBoxManage createvm --name nominatim --ostype Ubuntu_64 --register
   • VBoxManage modifyvm nominatim --memory 4096
   • VBoxManage modifyvm nominatim --cpus 4
   • VBoxManage modifyvm nominatim --nic1 nat
   • VBoxManage storagectl nominatim --name "SATA Controller" --add sata --controller IntelAhci
   • assign the (old) disk image a new uuid

     VBoxManage internalcommands sethduuid nominatim.vdi

   • attach the (old) disk image:

     VBoxManage storageattach "nominatim" --storagectl "SATA Controller" --port 0 --device 0 --type hdd --medium nominatim.vdi

   • VBoxManage storagectl nominatim --name "IDE Controller" --add ide --controller PIIX4
   • set the nat rules (ports):

     VBoxManage modifyvm nominatim --natpf1 "ssh,tcp,,20022,,22"

     VBoxManage modifyvm nominatim --natpf1 "apache,tcp,,20080,,80"

3. optional - set VRDE port:
   • VBoxManage modifyvm nominatim --vrdeport 7761 (Note: 3389 is the default port anyway)
4. check all settings with VBoxManage showvminfo nominatim
5. start the VM

   VBoxHeadless -s nominatim --vrde on &

   the VM will listen on port 7761 - connect with GTKvncViewer to check

   • check if ssh connection works
6. change the computers name in /etc/hosts and /etc/hostname
7. restarting Apache gives the error message: "apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1 ..."

   => solve this by adding ServerName nominatim as the last line in /etc/apache2/apache2.conf file

8. shut the VM down to resize the HDD if necessary
   • check the HDD size: VBoxManage showhdinfo nominatim.vdi returns:

UUID:           e148d758-00ac-4b60-a067-d5c97316069d
Parent UUID:    base</br>
State:          created
Type:           normal (base)
Location:       /home/ssteinig/nominatim.vdi
Storage format: VDI
Format variant: dynamic default
Capacity:       100000 MBytes
Size on disk:   6224 MBytes
In use by VMs:  nominatim (UUID: be795392-3786-4f3b-9714-a9445f91855d)

• so I resized the disk to 200GB as, according to Nominatim infos the disk size for a full planet is 700GB

  VBoxManage modifyhd nominatim.vdi --resize 200000

Setup of Nominatim itself

Install needed libraries

• install PHP 5
• install Postgres
• install PEAR
• install GCC, GIT, protbuf, wget, etc.
• install Osmosis

=> see https://wiki.openstreetmap.org/wiki/Nominatim/Installation#Software

sudo apt-get install build-essential libxml2-dev libgeos-dev libpq-dev libbz2-dev libtool automake libproj-dev
sudo apt-get install gcc proj-bin libgeos-c1 git osmosis libgeos++-dev
sudo apt-get install php5 php-pear php5-pgsql php5-json php-db
instead of: >> sudo apt-get install postgresql postgis postgresql-contrib postgresql-9.1-postgis postgresql-server-dev-9.1
  use: >> sudo apt-get install postgresql postgis postgresql-contrib postgresql-9.3-postgis-2.1 postgresql-server-dev-9.3
sudo apt-get install libprotobuf-c0-dev protobuf-c-compiler

and perhaps

sudo apt-get install git autoconf-archive

Edit PostreSQL settings

perhaps edit Postgres settings (before data import) sudo nano /etc/postgresql/9.3/main/postgresql.conf

=> see https://wiki.openstreetmap.org/wiki/Nominatim/Installation#Software

My machine has only 4GB, so I changed:

• shared_buffers = 128MB => 512MB
• work_mem = 50MB (previously uncommented, and set to 1MB)
• maintenance_work_mem = 2GB (previously commented, and set to 16MB) //=> This value should be reduced again afterwards to avoid swapping when autovacuum runs.
• effective_cache_size = 2GB (previously commented, and set to 128MB)
• synchronous_commit = off (previously commented, and set to on)
• checkpoint_segments = 100 (previously commented, and set to 3)
• checkpoint_timeout = 10min (previously commented, and set to 5min)
• checkpoint_completion_target = 0.9 (previously commented, and set to 0.5)

and for initial import only:

• fsync = off (previously commented, and set to 'on')
• full_page_writes = off (previously commented, and set to 'on')

=> switch both values back to 'on' after import, to avoid database corruption.

Note, I did a restart of the postgresql service after to have the settings applied.

First Nominatim Installation

Get the source code

I used the last stable release 2.3.1:

wget http://www.nominatim.org/release/Nominatim-2.3.1.tar.bz2
tar xvf Nominatim-2.3.1.tar.bz2

and rename the folder

mv Nominatim-2.3.1 Nominatim

Compiling

now compile the software

cd Nominatim
./configure
make

Note, the warning about missing lua libraries can be ignored. Nominatim does not make use of osm2pgsql's lua extension.

Configuration

create the configuration file local.php in the sub folder settings with the following content:

<?php
  // Paths
  @define('CONST_Postgresql_Version', '9.3');
  @define('CONST_Postgis_Version', '2.1');
  // Website settings
  //@define('CONST_Website_BaseURL', 'http://mysite/nominatim/');
  @define('CONST_Website_BaseURL', 'http://146.155.17.18:20080/nominatim/');

for further params see the file settings/settings.php.