shell

Executing Scripts

• Shebang: #!interpreter [optional-arg]
• A shebang is only used when a script has the execute permission
  • (e.g. chmod u+x script.sh).
• When a shell executes a script it will use the interpreter specified in the shebang

#!/bin/csh

commands:

# see jobs running 
jobs

# The fg command switches a job running in the background into the foreground
fg nano HYDR.py

# change permissions on all files in a directory (Required for HSP2 work):
jkleiner@deq3:/opt/model/HSPsquared$ sudo chgrp modelers HSP2/* -R
jkleiner@deq3:/opt/model/HSPsquared$ sudo chmod g+w HSP2/* -R

# change file owner and group information
chown

# Search shell command history
history | grep "STRING_TO_SEARCH_HERE'

ps -ef |grep nohup 
ps -ef
ps x

Ctrl + C # is used to send a SIGINT signal, which cancels or terminates the currently-running program

# run the river simulation
nohup run/mm/mm_run_basin $rscenario $rseg &
less +F nohup.out
# Ctrl + C to navigate around the log file
# F to view the log in real time again
# q to exit

cron:

# cron: https://www.cyberciti.biz/faq/how-do-i-add-jobs-to-cron-under-linux-or-unix-oses/#:~:text=Cron%20allows%20Linux%20and%20Unix,%2Ftmp%2F%20directories%20and%20more.
# the cron service (daemon) runs in the background and constantly checks the /etc/crontab file, and /etc/cron.*/ directories

ssh -p '311' 'jkleiner@deq1.bse.vt.edu'
sudo su -
cd /etc/cron.daily
cat deq-drought
# contains the shebang: #!/bin/sh

# run the cron script:
#./deq-drought
nohup ./deq-drought

# see contents of the nohup file:
tail -f nohup.out

cat /etc/crontab
# It runs all scripts in /etc/cron.daily/ once a day

Fixing git permissions on deq1:

Also, here is the command to make sure that we both can do stuff in there once you are back in sudo: 
sudo chgrp modelers .git
sudo chgrp modelers .git/* -R
sudo chmod g+w .git/* -R

commands:

pwd # "print working directory"
/ # on its own is the root directory.
ls # short for "listing"
ls /home/repl # shows you what's in your starting directory (usually called your home directory)
ls /home/repl/seasonal
# If it begins with /, it is absolute. If it does not begin with /, it is a relative path
cd # for "change directory"
cd .. # moves you up a directory 
~ # means "your home directory"
cp # short for "copy"
cp seasonal/autumn.csv seasonal/winter.csv backup # copies all of the files into that directory.
mv # moves it from one directory to another
mv autumn.csv winter.csv .. # This moves the files up one level
mv course.txt old-course.txt # mv can also be used to rename files
# mv works on directories as well
rm # for "remove"
rm thesis.txt backup/thesis-2017-08.txt
rmdir # for removing directories (directories must be empty first)
mkdir directory_name # to create a new (empty) directory
/tmp # is where people and programs often keep files they only need briefly. (Note that /tmp is immediately below the root directory /, not below your home directory.) 
cat # just prints the contents of files onto the screen. (Its name is short for "concatenate", meaning "to link things together", since it will print all the files whose names you give it, one after the other.)

got this far

When you less a file, one page is displayed at a time; you can press spacebar to page down or type q to quit
If you give less the names of several files, you can type :n (colon and a lower-case 'n') to move to the next file, :p to go back to the previous one, or :q to quit.
head. As its name suggests, it prints the first few lines of a file (where "a few" means 10)
head -n 3 seasonal/summer.csv
head will only display the first three lines of the file. If you run head -n 100, it will display the first 100 (assuming there are that many), and so on.
In order to see everything underneath a directory, no matter how deeply nested it is, you can give ls the flag -R (which means "recursive"). If you use ls -R. -F that prints a / after the name of every directory and a * after the name of every runnable program
ls -R -F ~
To find out what commands do, people used to use the man command (short for "manual"). For example, the command man head
If you want to select columns, you can use the command cut
cut -f 2-5,8 -d , values.csv
If you run some commands, you can then press the up-arrow key to cycle back through them
history will print a list of commands you have run recently
Each one is preceded by a serial number to make it easy to re-run particular commands: just type !55 to re-run the 55th command in your history
rep selects lines according to what they contain. In its simplest form, grep takes a piece of text followed by one or more filenames and prints all of the lines in those files that contain that text.
For example, grep bicuspid seasonal/winter.csv prints lines from winter.csv that contain "bicuspid".
some of grep's more common flags:
-c: print a count of matching lines rather than the lines themselves
-h: do not print the names of files when searching multiple files
-i: ignore case (e.g., treat "Regression" and "regression" as matches)
-l: print the names of files that contain matches, not the matches
-n: print line numbers for matching lines
-v: invert the match, i.e., only show lines that don't match
head -n 5 seasonal/summer.csv > top.csv
The greater-than sign > tells the shell to redirect head's output to a file
It isn't part of the head command; instead, it works with every shell command that produces output.
Suppose you want to get lines 3-5 from one of our data files
head -n 5 seasonal/winter.csv > top.csv
tail -n 3 top.csv
BETTER way to combine commands, using a pipe | 
The pipe symbol tells the shell to use the output of the command on the left as the input to the command on the right
Use cut to select all of the tooth names from column 2 of the comma delimited file seasonal/summer.csv, then pipe the result to grep, with an inverted match, to exclude the header line containing the word "Tooth"
cut -d , -f 2 seasonal/summer.csv | grep -v Tooth
combine many commands
cut -d , -f 1 seasonal/spring.csv | grep -v Date | head -n 10
1) select the first column from the spring data;
2) remove the header line containing the word "Date"; and
3) select the first 10 lines of actual data.
The command wc (short for "word count") prints the number of characters, words, and lines in a file. You can make it print only one of these using -c, -w, or -l respectively.
Count how many records in seasonal/spring.csv have dates in July 2017 (2017-07):
grep 2017-07 seasonal/spring.csv | wc -l
Most shell commands will work on multiple files if you give them multiple filenames. For example, you can get the first column from all of the seasonal data files at once like this:
shell allows you to use wildcards to specify a list of files with a single expression. The most common wildcard is *, which means "match zero or more characters"
Example: head -n 3 seasonal/s*.csv
The shell has other wildcards as well
? matches a single character, so 201?.txt will match 2017.txt or 2018.txt, but not 2017-01.txt.
[...] matches any one of the characters inside the square brackets, so 201[78].txt matches 2017.txt or 2018.txt, but not 2016.txt.
{...} matches any of the comma-separated patterns inside the curly brackets, so {*.txt, *.csv} matches any file whose name ends with .txt or .csv, but not files whose names end with .pdf.
sort: As its name suggests, sort puts data in order. By default it does this in ascending alphabetical order, but the flags -n and -r can be used to sort numerically and reverse the order of its output, while -b tells it to ignore leading blanks and -f tells it to fold case (i.e., be case-insensitive). Pipelines often use grep to get rid of unwanted records and then sort to put the remaining records in order.
 sort the names of the teeth in seasonal/winter.csvin descending alphabetical order
cut -d , -f 2 seasonal/winter.csv | grep -v Tooth | sort -r