diff --git a/examples/java-on-osg.md b/examples/java-on-osg.md index 9e8a3763..f5b5f750 100644 --- a/examples/java-on-osg.md +++ b/examples/java-on-osg.md @@ -39,6 +39,8 @@ like: # Add the unzipped JDK folder to the environment export PATH=$PWD/jdk-17.0.1/bin:$PATH export JAVA_HOME=$PWD/jdk-17.0.1 + # set TMPDIR variable + TMPDIR=$_CONDOR_SCRATCH_DIR # run your .jar file java -jar program.jar diff --git a/examples/julia-on-osg.md b/examples/julia-on-osg.md index 05847167..bfa106b2 100644 --- a/examples/julia-on-osg.md +++ b/examples/julia-on-osg.md @@ -44,6 +44,8 @@ the job's executable: # add Julia binary to PATH export PATH=$_CONDOR_SCRATCH_DIR/julia-#-#-#/bin:$PATH + # set TMPDIR variable + TMPDIR=$_CONDOR_SCRATCH_DIR # run Julia script julia my-script.jl @@ -163,6 +165,8 @@ the Julia Standard library) use the example script directly below. # add Julia binary to PATH export PATH=$_CONDOR_SCRATCH_DIR/julia-#.#.#/bin:$PATH + # set TMPDIR variable + TMPDIR=$_CONDOR_SCRATCH_DIR # run Julia script julia script.jl @@ -182,6 +186,8 @@ the Julia Standard library) use the example script directly below. export PATH=$_CONDOR_SCRATCH_DIR/julia-#.#.#/bin:$PATH # add Julia packages to DEPOT variable export JULIA_DEPOT_PATH=$_CONDOR_SCRATCH_DIR/my-project + # set TMPDIR variable + TMPDIR=$_CONDOR_SCRATCH_DIR # run Julia script julia --project=my-project script.jl diff --git a/examples/manage-python-packages.md b/examples/manage-python-packages.md index d97fa17b..a184e933 100644 --- a/examples/manage-python-packages.md +++ b/examples/manage-python-packages.md @@ -134,6 +134,8 @@ just created. That will look something like this: # Load Python # (should be the same version used to create the virtual environment) module load python/3.7.0 + # set TMPDIR variable + TMPDIR=$_CONDOR_SCRATCH_DIR # Unpack your envvironment (with your packages), and activate it tar -xzf my_env.tar.gz diff --git a/start/software/example-compilation.md b/start/software/example-compilation.md index d8e7124e..b7f299ac 100644 --- a/start/software/example-compilation.md +++ b/start/software/example-compilation.md @@ -476,6 +476,7 @@ to "untar" the Samtools and XZ Util tar archives, modify the `PATH` and # modify environment variables export LD_LIBRARY_PATH=$_CONDOR_SCRATCH_DIR/xz-5.2.5/lib:$LD_LIBRARY_PATH export PATH=$_CONDOR_SCRATCH_DIR/samtools-1.10/bin:$_CONDOR_SCRATCH_DIR/xz-5.2.5/bin:$PATH + export TMPDIR=$_CONDOR_SCRATCH_DIR # load bzip2 module module load bzip2/1.0.6 diff --git a/training/resources/frequently-asked-questions-faq-.md b/training/resources/frequently-asked-questions-faq-.md index bb6d89ef..e458469c 100644 --- a/training/resources/frequently-asked-questions-faq-.md +++ b/training/resources/frequently-asked-questions-faq-.md @@ -27,11 +27,6 @@ We have implemented modules within OSG Connect to manage the software that is av **Are there any restrictions on installing commercial softwares?** We can only *directly* support software that is freely distributable. At present, we do not have or support most commercial software due to licensing issues. (One exception is running [MATLAB standalone executables](https://support.opensciencegrid.org/support/solutions/articles/5000660751-basics-of-compiled-matlab-applications-hello-world-example) which have been compiled with the MATLAB Compiler Runtime). Software that is licensed to individual users (and not to be shared between users) can be staged within the user's /home directory with HTCondor transferring to jobs, but should not be staged in OSG's public data staging locations (see https://support.opensciencegrid.org/support/solutions/articles/12000002985-data-management-and-policies). Please get in touch with any questions about licensed software. - - -**Can I request for system wide installation of the open source software useful for my research?** - -Yes. Please contact . ## Running Jobs @@ -57,6 +52,27 @@ The number of jobs that are submitted to the queue by any one user should not ex This is the maximum number of jobs that you will have in the "Idle" or "Held" state for the submitted batch of jobs at any given time. Using a value of 2000 will ensure that your jobs continue to apply a constant pressure on the queue, but will not fill up the queue unnecessarily (which helps the scheduler to perform optimally). +**How do I solve a "disk quota exceeded" error?** + +If your jobs are returning errors about exceeding disk quota, there may be two different +problem: + +1. Check that you are requesting enough disk space for your job execution. You can find +this information in the job log file, near the end: + Partitionable Resources : Usage Request Allocated + Disk (KB) : 125 1000 53813 +You'll want to confirm that the number in the "Request" column is larger than the +number in the "Usage" column. + +1. If you ARE requesting enough disk space, your job may be trying to use disk space +outside of the main working directory. This most often happens if software expects to use +the temporary directory `/tmp` for intermediate files. These files should be contained +in the job's working directory instead; to make this happen, the `TMPDIR` environment +variable needs to be set to the working directory. In a shell script, this looks like this: + export TMPDI=$_CONDOR_SCRATCH_DIR +If you have questions about how to translate this into a different script format, +let us know. + ## Data Storage and Transfer **What is the best way to process large volume of data?** diff --git a/tutorials/tutorial-AutoDockVina b/tutorials/tutorial-AutoDockVina index c4b92832..27cea744 160000 --- a/tutorials/tutorial-AutoDockVina +++ b/tutorials/tutorial-AutoDockVina @@ -1 +1 @@ -Subproject commit c4b92832f5fd13543fced127fa6a5122aec087c2 +Subproject commit 27cea7446a917854e2f16ca39e5661af42c836c4 diff --git a/tutorials/tutorial-R b/tutorials/tutorial-R index cf37e228..dbc58f18 160000 --- a/tutorials/tutorial-R +++ b/tutorials/tutorial-R @@ -1 +1 @@ -Subproject commit cf37e22859e232af78cb6ad91ba71f19af10f192 +Subproject commit dbc58f18eb530820300811a526c8d97fa7d96d91 diff --git a/tutorials/tutorial-R-addlibSNA b/tutorials/tutorial-R-addlibSNA index 8513474d..60c601c8 160000 --- a/tutorials/tutorial-R-addlibSNA +++ b/tutorials/tutorial-R-addlibSNA @@ -1 +1 @@ -Subproject commit 8513474dc1397ebe9fb24646f061c0611af04e7d +Subproject commit 60c601c8bc682050944edbd44fa3001696e4e461 diff --git a/tutorials/tutorial-ScalingUp-Python b/tutorials/tutorial-ScalingUp-Python index d8689666..93511421 160000 --- a/tutorials/tutorial-ScalingUp-Python +++ b/tutorials/tutorial-ScalingUp-Python @@ -1 +1 @@ -Subproject commit d8689666249a9fdc309ec1d80b9e869d5119b14a +Subproject commit 9351142180c9f4efd478c4b1c6b7d4aba40184ae diff --git a/tutorials/tutorial-ScalingUp-R b/tutorials/tutorial-ScalingUp-R index ba94e2e5..2e0fbb36 160000 --- a/tutorials/tutorial-ScalingUp-R +++ b/tutorials/tutorial-ScalingUp-R @@ -1 +1 @@ -Subproject commit ba94e2e5aae61e62f5c7c6493ea95e18c39397d9 +Subproject commit 2e0fbb36ae23bb1737ffe995cfa02211c79b342d diff --git a/tutorials/tutorial-osg-locations b/tutorials/tutorial-osg-locations index c0109ecc..b79a2774 160000 --- a/tutorials/tutorial-osg-locations +++ b/tutorials/tutorial-osg-locations @@ -1 +1 @@ -Subproject commit c0109ecc3e662596dfcd0e477394dfab00221377 +Subproject commit b79a277430c4fe507759e1c13eb56d3e4dc9c247