Difference between revisions of "Accessing server infra"

Latest revision as of 16:18, 11 April 2022

>> return to Cedeus IDE

apt-get

an intro can be found here

• see what is installed dpkg -l
• info on a particular package apt-cache show PackageName
  • the version of ssl can be obtained using sudo openssl version -a
• upgrade apt-get upgrade PackageName or: sudo apt-get install libssl1.0.0
• if I get the message: "n packages can be updated", then do the following:
  • sudo apt-get update # Fetches the list of available updates
  • sudo apt-get upgrade # Strictly upgrades the current packages
  • sudo apt-get dist-upgrade # Installs updates (new ones)
  • perhaps reboot

git stuff

• git diff
• git pull
• git status
• git reset --hard HEAD
• git reset HEAD~1
• git reset HEAD opentripplanner-api-webapp/src/main/java/org/opentripplanner/api/ws/analyst/IsoChrone.java
• git log -p
• git add opentripplanner-api-webapp/src/main/java/org/opentripplanner/api/ws/analyst/IsoChrone.java
• git commit -m 'added debug code to be able to figure why some edges are traversed to fast (aside fromom u-shaped roads)'
• gitk
• git push origin master
• git show-ref
• git stash list
• git log origin/master..master
• git fetch upstream
• git remote add upstream https://github.com/openplans/OpenTripPlanner.git
• git remote show origin
• git remote set-url origin git://github.com/mentaer/OpenTripPlanner.git
• git remote set-url origin git@github.com:mentaer/OpenTripPlanner.git
• git diff > /Volumes/WINDOOF/walkingslopeeffects.patch
• patch -p1 -R < IsoChrone2.patch
• git apply --check /Volumes/WINDOOF/IsoChrone.patch

maven stuff

• mvn package -DskipTests

postgres stuff

• echo 'export PATH=/Library/PostgreSQL/9.2/bin:$PATH' >> ~/.bash_profile

• psql
• sudo mkdir /Library/PostgreSQL/9.2/data/
• sudo chown postgres /Library/PostgreSQL/9.2/data/
• sudo -u postgres initdb -D /Library/PostgreSQL/9.2/data/
• service postgresql start
• sudo -u postgres pg_ctl -D /Library/PostgreSQL/9.2/data -l logfile start
• sudo -u postgres postgres -D /Library/PostgreSQL/9.2/data &
• sudo -u postgres createdb santiagotest
• sudo -u postgres createuser -P -s -e mentaer
• psql santiagotest
• to access via terminal use for instance psql -U sstein -d geonode -h localhost

using VirtualBox

• a VM contains 3 files: .img + .vbox (an editable xml) + .vbox-prev
• " use ssh -X sstein..@xxx to enable a visual client
• vboxgtk (start visual client)

create and start a VM from comandline:

• VBoxManage createvm --name ubuntu --register
• VBoxManage modifyvm ubuntu --ostype ubuntu_64 --cpus 4
• VBoxManage modifyvm ubuntu --memory 8000
• VBoxManage startvm ubuntu

other:

• options of ip-addressing/web access: NAT (Network Address Translation) or Bridge (see also this blog entry)
• downloading .iso archives/vm from OSGeo (use commands curl or wget -c), e.g.:

wget -c "http://downloads.sourceforge.net/project/osgeo-live/7.0/osgeo-live-7.0.iso?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Fosgeo-live%2Ffiles%2F%2F7.0%2Fosgeo-live-7.0.iso%2Fdownload&ts=1384444616&use_mirror=ufpr"

• showing list of running VMs: VBoxManage list runningvms
• show list of VMs: VBoxManage list vms
• powerdown a VM: VBoxManage controlvm opensuse132 poweroff
  • emergency power-off a VM that does not properly shutdown and when restarting there is a message such as:

VBoxHeadless: error: The machine 'opensuse132' is already locked for a session (or being unlocked)
VBoxHeadless: error: Details: code VBOX_E_INVALID_OBJECT_STATE (0x80bb0007), component Machine, interface IMachine, callee nsISupports
VBoxHeadless: error: Context: "LockMachine(session, LockType_VM)" at line 960 of file VBoxHeadless.cpp

=> solve with: VBoxManage startvm opensuse132 --type emergencystop

• start VM: VBoxHeadless -s opensuse132 --vrde on &
• add user to VBoxuser group to be able to install a VM: <ode>sudo usermod -a -G vboxusers usernane</code>

=> to check in what groups the user is: groups username

OpenTripPlanner (for older version: 0.9, not 0.20/1.0)

• start with "sh start-server.sh"; use "&" at the end to keep the process running
• make sure that winstone.jar is executable, with chmod 755
• define in the start-server.sh file also where the webapps folder is located (physical location), so winstone knows that
• define in the WEB-INF/classes/data-sources.xml file where the graph is stored (physical location) - the files is within /otp/webapps/opentripplanner-api-webapp.war (or: the deflated war folder)
• kill winstone server (or OTP) process: use

  ps aux | grep winstone

  to find the associated PID, and use kill pid

Unix / Linux / Ubuntu commands

http://www.computerhope.com/issues/ch000623.htm

assign static IP

• for Ubuntu see see http://www.howtogeek.com/howto/ubuntu/change-ubuntu-server-from-dhcp-to-a-static-ip-address/

connect to server

• ssh stefan@136.159.xxx.xxx
• ssh sstein@146.155.xxx.xxx.
• transfer a file to a remote computer: scp /home/username/ftw.txt user@example.com:/home/username/

see processes and find stuff

• pid
• top or: htop
• process

  commands to display/figure out processes: http://www.howtogeek.com/107217/

• (sudo) kill pid, e.g. kill 320

  kill process: http://www.cyberciti.biz/faq/kill-process-in-linux-or-terminate-a-process-in-unix-or-linux-systems/

• "sudo shutdown now -h" or "sudo reboot"
• seeing system info: cat /etc/*-release + htop (needs to be installed first)
• grep: search for files in the folder /var/.../static/ that contain the string nav-pills with: grep -r -i "nav-pills" /var/www/geonode/static/

  use of grep: http://www.cyberciti.biz/faq/howto-use-grep-command-in-linux-unix/

• find files/folders by name: find / -name game
• to find all ssh/terminal connections (in case one may freeze):
  • for a general overview use who
  • to get the pid use: ps aux | egrep "sshd: [a-zA-Z]+@"

user management commands

• create new users - from http://www.digitalocean.com/community/tutorials/how-to-add-and-delete-users-on-an-ubuntu-14-04-vps
  • add new user sudo adduser newuser
  • grant a user sudo privileges: type sudo visudo then search for the line root ALL=(ALL:ALL) ALL.

  => Now copy and paste this line below, and replace in this new line root by the user's name

• adding users to groups - from http://www.cyberciti.biz/faq/howto-linux-add-user-to-group/
  • check in what groups the user is: id username
  • add an existing user "myuser" to group "hallogroup": usermod -a -G hallogroup myuser
• switch to work as another user
  • switch to work as user tony: su - tony
  • switch to work as root: sudo su

memory related commands

• configuration info: http://www.cyberciti.biz/faq/linux-command-to-find-the-system-configuration-and-hardware-information/
• usage of disk space (free disc space): df -h
• usage of memory/ram : htop or free -m
• directory size: du -h /home/cedeusdbbackupuser/

other useful commands

• used ports: netstat -a
• monitor network traffic: iftop
• mac addresses: ifconfig -a
• creating an ISO file from a DVD: mount the DVD. If mounted under /dev/sr0 use: sudo cat /dev/sr0 > /home/ssteinig/example.iso
• what to do if I get "There are stopped jobs"? see http://unix.stackexchange.com/questions/116959/there-are-stopped-jobs-on-bash-exit
  • use jobs to check what jobs are still around
  • use fg + job# to get the job to be able to terminate it later

powerdown (sleep) and wakeup

• sudo rtcwake -m disk -s 120 & ...this worked with Camila and put her 120 secs to sleep
• the stuff below did not work as expected ("rtcwake" not found msg)
  • for a scheduled shutdown of the servers I put the command in the sudo crontab, like this:

  MAILTO=ssteiniger --- at --- uc.cl

  0 22 12 2 * rtcwake -m disk -s 108000 &

=> this shall send an email to me, and executes the rtcwake command at 12. Feb. 22:00 (each year, I guess)

• for rtcwake details see: http://www.howtogeek.com/121241/how-to-make-your-linux-pc-wake-from-sleep-automatically/
• for crontab details see: http://www.thegeekstuff.com/2009/06/15-practical-crontab-examples/

sending email from server

• install mailutils (apt-get)
• test: mail -s "test email at: `date +%m.%d.%Y.%H.%M`" myemail@email.de

=> I used this to send me an email on reboot from cedeusgeonode

Installing lightweight XFCE Desktop GUI on Ubuntu Server

• use sudo apt-get install xubuntu-desktop to install XFCE client (see here).
• this will also install xauth, which allows remote window clients
• disable start of GUI-based login on system startup with Grub (taken from here):
  • Change (edit) in /etc/default/grub file: GRUB_CMDLINE_LINUX_DEFAULT="text"
  • then do sudo update-grub
• start the desktop GUI session using startx
• end the desktop GUI session using "logout" button

Enabling Remote Desktop for Windows

got the info from here

• install XRDP: sudo apt-get install xrdp
• connect via Windows Remote Desktop using the IP etc.
• if not needed, switch off the remote desktop server using: sudo service xrdp stop

Apache Access Log Analysis

• access to the servers is logged by Apache in the following 2 files (apart from error log):
  • /var/log/apache2/access.log This is for access to the server. --> for seeing it live use: tail -f /var/log/apache2/access.log
  • /var/log/apache2/other_vhosts_access.log This helps to see access to my VMs for instance. --> for seeing it live use: tail -f /var/log/apache2/other_vhosts_access.log
• I also installed on CedeusGeoNode a Top like open source log analyzer named GoAccess:
  • to run it simply type: goaccess -f /var/log/apache2/access.log -a (see http://goaccess.io/faq#howto-run)
  • and for the "other vhosts" goaccess -f /var/log/apache2/other_vhosts_access.log -a

Setting Up a CEDEUS Subdomain

Its possible to create a CEDEUS subdomain using two methods, provided by Dreamhost

• under Domain entry create a (sub)domain redirect to the VM directly, i.e. inlcuding IP:port number
• under Domain clicking on DNS under the cedeus.cl entry, one can add an "A Record" specifying the the subdomain name (i.e. pubs.cedeus.cl) and the server IP, i.e. 146.155.17.18. Then on the server, the apache default sites file needs to be modified and a VirtualHost + ProxyPass entry added

sudo nano /etc/apache2/sites-available/000-default.conf

and then add a complete new entry (not modifying the main one)

#Omeka / Pubs.CEDEUS.cl
<VirtualHost *:80>
   ServerName pubs.cedeus.cl
   ProxyRequests Off
   ProxyPreserveHost On
   ProxyPass / http://146.155.17.18:20080/
   ProxyPassReverse / http://146.155.17.18:20080/
</VirtualHost>

Afterwards on should(???) also modify for the VM the Apache Site setting and define the ServerName (or ServerAlias), i.e. pubs.cedeus.cl for the example.

other stuff

• gdalinfo cgydem_fromgrid.tif
• if computer crashs: boot from a LiveDVD and the run fsck (see http://www.thegeekstuff.com/2012/08/fsck-command-examples/)
• compress:
  • files by using: zip newzipfile.zip fileToZip.obj
  • whole folders by using: tar -cvzf tarfilename.tgz /folder/to/compress
  • using 7z
    • install 7zip: sudo apt-get install p7zip-full
    • on how to use 7zip: see here and here
    • e.g. to pack: 7z a archivefile.7z archivetocompress.vdi
• uncompress:
  • zip files, using: unzip filename.zip,
  • bz2 files with: bunzip2 filename.bz2
  • tar files: tar -C / -xvzf tarfilename.tgz . Attention !!! This will restore the data exactly in the same place on the computer, and overwrite existing folders and files.
• VBoxManage Windows 8 - 64bit createvm code : "Windows8_64"
• start PhpStorm on my machine: sh ./dev/PhpStorm-133.1777/bin/phpstorm.sh
• Installing New Harddisk in Ubuntu
• Solving Ubuntu GRUB issue
• check network bandwidth use: use iftop (sudo apt-get install iftop)

resolve nginx 502 FIC

resolve error 502 for FIC web front

• cd /var/www/fic/
• pm2 start npm --name "fic" -- start (yes, with space before start)
• reload webpage

resolve nginx 502 Indicadores or Stride

resolve error 502 for Indicadores web front

• cd /home/csfuente/Indicadores => correcion, debe ser "indicadores_REST-API"
• source bin/activate
• gunicorn --bind 0.0.0.0:8000 indicadores.wsgi
• reload webpage

for Stride it would be instead:

• cd /home/csfuente/stride
• source bin/activate
• gunicorn --bind 0.0.0.0:8000 stride.wsgi
• reload webpage