Difference between revisions of "Accessing server infra"

Revision as of 18:07, 11 February 2016

>> return to Cedeus IDE

apt-get

an intro can be found here

• see what is installed dpkg -l
• info on a particular package apt-cache show PackageName
  • the version of ssl can be obtained using sudo openssl version -a
• upgrade apt-get upgrade PackageName or: sudo apt-get install libssl1.0.0
• if I get the message: "n packages can be updated", then do the following:
  • sudo apt-get update # Fetches the list of available updates
  • sudo apt-get upgrade # Strictly upgrades the current packages
  • sudo apt-get dist-upgrade # Installs updates (new ones)
  • perhaps reboot

git stuff

• git diff
• git pull
• git status
• git reset --hard HEAD
• git reset HEAD~1
• git reset HEAD opentripplanner-api-webapp/src/main/java/org/opentripplanner/api/ws/analyst/IsoChrone.java
• git log -p
• git add opentripplanner-api-webapp/src/main/java/org/opentripplanner/api/ws/analyst/IsoChrone.java
• git commit -m 'added debug code to be able to figure why some edges are traversed to fast (aside fromom u-shaped roads)'
• gitk
• git push origin master
• git show-ref
• git stash list
• git log origin/master..master
• git fetch upstream
• git remote add upstream https://github.com/openplans/OpenTripPlanner.git
• git remote show origin
• git remote set-url origin git://github.com/mentaer/OpenTripPlanner.git
• git remote set-url origin git@github.com:mentaer/OpenTripPlanner.git
• git diff > /Volumes/WINDOOF/walkingslopeeffects.patch
• patch -p1 -R < IsoChrone2.patch
• git apply --check /Volumes/WINDOOF/IsoChrone.patch

maven stuff

• mvn package -DskipTests

postgres stuff

• echo 'export PATH=/Library/PostgreSQL/9.2/bin:$PATH' >> ~/.bash_profile

• psql
• sudo mkdir /Library/PostgreSQL/9.2/data/
• sudo chown postgres /Library/PostgreSQL/9.2/data/
• sudo -u postgres initdb -D /Library/PostgreSQL/9.2/data/
• service postgresql start
• sudo -u postgres pg_ctl -D /Library/PostgreSQL/9.2/data -l logfile start
• sudo -u postgres postgres -D /Library/PostgreSQL/9.2/data &
• sudo -u postgres createdb santiagotest
• sudo -u postgres createuser -P -s -e mentaer
• psql santiagotest
• to access via terminal use for instance psql -U sstein -d geonode -h localhost

using VirtualBox

• a VM contains 3 files: .img + .vbox (an editable xml) + .vbox-prev
• " use ssh -X sstein..@xxx to enable a visual client
• vboxgtk (start visual client)

create and start a VM from comandline:

• VBoxManage createvm --name ubuntu --register
• VBoxManage modifyvm ubuntu --ostype ubuntu_64 --cpus 4
• VBoxManage modifyvm ubuntu --memory 8000
• VBoxManage startvm ubuntu

other:

• options of ip-addressing/web access: NAT (Network Address Translation) or Bridge (see also this blog entry)
• downloading .iso archives/vm from OSGeo (use commands curl or wget -c), e.g.:

wget -c "http://downloads.sourceforge.net/project/osgeo-live/7.0/osgeo-live-7.0.iso?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Fosgeo-live%2Ffiles%2F%2F7.0%2Fosgeo-live-7.0.iso%2Fdownload&ts=1384444616&use_mirror=ufpr"

• showing list of running VMs: VBoxManage list runningvms
• show list of VMs: VBoxManage list vms
• powerdown a VM: VBoxManage controlvm opensuse132 poweroff
• emergency power-off a VM that does not properly shutdown and when restarting there is a message such as:

VBoxHeadless: error: The machine 'opensuse132' is already locked for a session (or being unlocked)
VBoxHeadless: error: Details: code VBOX_E_INVALID_OBJECT_STATE (0x80bb0007), component Machine, interface IMachine, callee nsISupports
VBoxHeadless: error: Context: "LockMachine(session, LockType_VM)" at line 960 of file VBoxHeadless.cpp

=> solve with: VBoxManage startvm opensuse132 --type emergencystop

• add user to VBoxuser group to be able to install a VM: <ode>sudo usermod -a -G vboxusers usernane</code>

=> to check in what groups the user is: groups username

OpenTripPlanner

• start with "sh start-server.sh"; use "&" at the end to keep the process running
• make sure that winstone.jar is executable, with chmod 755
• define in the start-server.sh file also where the webapps folder is located (physical location), so winstone knows that
• define in the WEB-INF/classes/data-sources.xml file where the graph is stored (physical location) - the files is within /otp/webapps/opentripplanner-api-webapp.war (or: the deflated war folder)
• kill winstone server (or OTP) process: use

  ps aux | grep winstone

  to find the associated PID, and use kill pid

Unix / Linux / Ubuntu commands

http://www.computerhope.com/issues/ch000623.htm

assign static IP

• for Ubuntu see see http://www.howtogeek.com/howto/ubuntu/change-ubuntu-server-from-dhcp-to-a-static-ip-address/

connect to server

• ssh stefan@136.159.xxx.xxx
• ssh sstein@146.155.xxx.xxx.
• transfer a file to a remote computer: scp /home/username/ftw.txt user@example.com:/home/username/

see processes and find stuff

• pid
• top or: htop
• process

  commands to display/figure out processes: http://www.howtogeek.com/107217/

• (sudo) kill pid, e.g. kill 320

  kill process: http://www.cyberciti.biz/faq/kill-process-in-linux-or-terminate-a-process-in-unix-or-linux-systems/

• "sudo shutdown now -h" or "sudo reboot"
• seeing system info: cat /etc/*-release + htop (needs to be installed first)
• grep: search for files in the folder /var/.../static/ that contain the string nav-pills with: grep -r -i "nav-pills" /var/www/geonode/static/

  use of grep: http://www.cyberciti.biz/faq/howto-use-grep-command-in-linux-unix/

• find files/folders by name: find / -name game
• to find all ssh/terminal connections (in case one may freeze):
  • for a general overview use who
  • to get the pid use: ps aux | egrep "sshd: [a-zA-Z]+@"

user management commands

• create new users - from http://www.digitalocean.com/community/tutorials/how-to-add-and-delete-users-on-an-ubuntu-14-04-vps
  • add new user sudo adduser newuser
  • grant a user sudo privileges: type sudo visudo then search for the line root ALL=(ALL:ALL) ALL.

  => Now copy and paste this line below, and replace in this new line root by the user's name

• adding users to groups - from http://www.cyberciti.biz/faq/howto-linux-add-user-to-group/
  • check in what groups the user is: id username
  • add an existing user "myuser" to group "hallogroup": usermod -a -G hallogroup myuser
• switch to work as another user
  • switch to work as user tony: su - tony
  • switch to work as root: sudo su

memory related commands

• configuration info: http://www.cyberciti.biz/faq/linux-command-to-find-the-system-configuration-and-hardware-information/
• usage of disk space (free disc space): df -h
• usage of memory/ram : htop or free -m
• directory size: du -h /home/cedeusdbbackupuser/

other useful commands

• used ports: netstat -a
• monitor network traffic: iftop
• mac addresses: ifconfig -a
• creating an ISO file from a DVD: mount the DVD. If mounted under /dev/sr0 use: sudo cat /dev/sr0 > /home/ssteinig/example.iso
• what to do if I get "There are stopped jobs"? see http://unix.stackexchange.com/questions/116959/there-are-stopped-jobs-on-bash-exit
  • use jobs to check what jobs are still around
  • use fg + job# to get the job to be able to terminate it later

powerdown (sleep) and wakeup

• sudo rtcwake -m disk -s 120 & ...this worked with Camila
• for details see: http://www.howtogeek.com/121241/how-to-make-your-linux-pc-wake-from-sleep-automatically/

Installing lightweight XFCE Desktop GUI on Ubuntu Server

• use sudo apt-get install xubuntu-desktop to install XFCE client (see here).
• this will also install xauth, which allows remote window clients
• disable start of GUI-based login on system startup with Grub (taken from here):
  • Change (edit) in /etc/default/grub file: GRUB_CMDLINE_LINUX_DEFAULT="text"
  • then do sudo update-grub
• start the desktop GUI session using startx
• end the desktop GUI session using "logout" button

Enabling Remote Desktop for Windows

got the info from here

• install XRDP: sudo apt-get install xrdp
• connect via Windows Remote Desktop using the IP etc.
• if not needed, switch off the remote desktop server using: sudo service xrdp stop

Apache Access Log Analysis

• access to the servers is logged by Apache in the following 2 files (apart from error log):
  • /var/log/apache2/access.log This is for access to the server. --> for seeing it live use: tail -f /var/log/apache2/access.log
  • /var/log/apache2/other_vhosts_access.log This helps to see access to my VMs for instance. --> for seeing it live use: tail -f /var/log/apache2/other_vhosts_access.log
• I also installed on CedeusGeoNode a Top like open source log analyzer named GoAccess:
  • to run it simply type: goaccess -f /var/log/apache2/access.log -a (see http://goaccess.io/faq#howto-run)
  • and for the "other vhosts" goaccess -f /var/log/apache2/other_vhosts_access.log -a

other stuff

• gdalinfo cgydem_fromgrid.tif
• if computer crashs: boot from a LiveDVD and the run fsck (see http://www.thegeekstuff.com/2012/08/fsck-command-examples/)
• compress:
  • files by using: zip newzipfile.zip fileToZip.obj
  • whole folders by using: tar -cvzf tarfilename.tgz /folder/to/compress
  • using 7z
    • install 7zip: sudo apt-get install p7zip-full
    • on how to use 7zip: see here and here
    • e.g. to pack: 7z a archivefile.7z archivetocompress.vdi
• uncompress:
  • zip files, using: unzip filename.zip,
  • bz2 files with: bunzip2 filename.bz2
  • tar files: tar -C / -xvzf tarfilename.tgz . Attention !!! This will restore the data exactly in the same place on the computer, and overwrite existing folders and files.
• VBoxManage Windows 8 - 64bit createvm code : "Windows8_64"
• start PhpStorm on my machine: sh ./dev/PhpStorm-133.1777/bin/phpstorm.sh
• Installing New Harddisk in Ubuntu
• Solving Ubuntu GRUB issue
• check network bandwidth use: use iftop (sudo apt-get install iftop)