How to run a job - Revision history

Paul Keekstra: /* First example */

2017-03-01T16:12:21Z

First example

← Older revision		Revision as of 18:12, 1 March 2017
Line 11:		Line 11:
	echo "Hello world!"		echo "Hello world!"

	This job does nothing for two minutes, after that it prints "Hello world!". The script starts with #PBS, which is a directive for Torque (the resource manager). In this example it means that you tell Torque to use one CPU (ppn:1) on one node (nodes=1) for this job. There can be ~~two or~~ more than one directives, they should always be at the start of the script. The remaing lines in this example are just commands that you could type on the command line.		This job does nothing for two minutes, after that it prints "Hello world!". The script starts with #PBS, which is a directive for Torque (the resource manager). In this example it means that you tell Torque to use one CPU (ppn:1) on one node (nodes=1) for this job. There can be more than one directives, they should always be at the start of the script. The remaing lines in this example are just commands that you could type on the command line.

	Assuming that the name of the script is <code>job1</code> it can be submitted like this:		Assuming that the name of the script is <code>job1</code> it can be submitted like this:

Paul Keekstra: /* Example with Matlab */

2017-02-03T13:43:26Z

Example with Matlab

← Older revision		Revision as of 15:43, 3 February 2017
Line 52:		Line 52:
	#/bin/sh		#/bin/sh
	#		#
	#PBS l nodes=1		#PBS -l nodes=1
	#		#
	module load matlab		module load matlab
	cd $PBS_O_WORKDIR		cd $PBS_O_WORKDIR
	matlab nosplash nodisplay ~~nojvm~~ singleCompThread < pseudoinv.m		matlab -nosplash -nodisplay -nojvm -singleCompThread < pseudoinv.m

Paul Keekstra: /* Example with Matlab */

2017-02-03T13:39:46Z

Example with Matlab

← Older revision		Revision as of 15:39, 3 February 2017
Line 47:		Line 47:

	The cluster is most effective used in batch mode. This requires disabling the default matlab GUI features and display capabilities.		The cluster is most effective used in batch mode. This requires disabling the default matlab GUI features and display capabilities.

	A simple example job for running a single core Matlab run in batch mode:		A simple example job for running a single core Matlab run in batch mode:

Paul Keekstra: /* First steps */

2017-02-03T13:39:16Z

First steps

← Older revision		Revision as of 15:39, 3 February 2017
Line 1:		Line 1:
	== First ~~steps~~ ==		== First example ==

	~~In order to make the worker nodes run~~ a ~~parallel job, you have to prepare~~ a job ~~script. This script tells the queue manager what you want to do, it has to be submitted with the <code>qsub</code> command~~. ~~A typical~~ job script looks like this:		We'll have a look at a detailed example of a very simple job. The job script for this example looks like this:

			#!/bin/sh
	#		#
	#PBS -l nodes=1:ppn=1		#PBS -l nodes=1:ppn=1
Line 10:		Line 11:
	echo "Hello world!"		echo "Hello world!"

	This job does nothing for two minutes, after that it prints "Hello world!". The script starts with #PBS, which is a directive for Torque (the resource manager). In this example it means that you tell Torque to use one CPU (ppn:1) on one node (nodes=1) for this job. There can be more than one ~~directive~~, they should always be at the start of the script. The remaing lines in this example are just commands that you could type on the command line.		This job does nothing for two minutes, after that it prints "Hello world!". The script starts with #PBS, which is a directive for Torque (the resource manager). In this example it means that you tell Torque to use one CPU (ppn:1) on one node (nodes=1) for this job. There can be two or more than one directives, they should always be at the start of the script. The remaing lines in this example are just commands that you could type on the command line.

	Assuming that the name of the script is <code>job1</code> it can be submitted like this:		Assuming that the name of the script is <code>job1</code> it can be submitted like this:
Line 40:		Line 41:

	As long as your job is running, there will be two temporary files in your home directory, like <code>24.hpc10.hpc.OU </code>and <code>24.hpc10.hpc.ER</code>, these names correspond with the jobd id. The output and error messages that would normally be printed on the command line, will be redirected to these temporary files. If your job is finished, these files will be renamed and moved to the directory where you were when the job was submitted. The new names will be the same as the name of your jobscript, followed by a dot and the letter o or e, plus the first part of the job id. The example job script above would, if executed correctly, leave two files; <code>job1.o24</code> which would contain the text 'Hello world!' and <code>job1.e24</code> which would be empty.		As long as your job is running, there will be two temporary files in your home directory, like <code>24.hpc10.hpc.OU </code>and <code>24.hpc10.hpc.ER</code>, these names correspond with the jobd id. The output and error messages that would normally be printed on the command line, will be redirected to these temporary files. If your job is finished, these files will be renamed and moved to the directory where you were when the job was submitted. The new names will be the same as the name of your jobscript, followed by a dot and the letter o or e, plus the first part of the job id. The example job script above would, if executed correctly, leave two files; <code>job1.o24</code> which would contain the text 'Hello world!' and <code>job1.e24</code> which would be empty.

			== Example with Matlab ==

			Matrix Laboratory. An interactive numerical white-board and scripting tool for prototyping algorithms and performing numerical analysis and visual representation of data.

			The cluster is most effective used in batch mode. This requires disabling the default matlab GUI features and display capabilities.
			A simple example job for running a single core Matlab run in batch mode:

			#/bin/sh
			#
			#PBS l nodes=1
			#
			module load matlab
			cd $PBS_O_WORKDIR
			matlab nosplash nodisplay nojvm singleCompThread < pseudoinv.m

Paul Keekstra: /* First steps */

2017-02-03T13:14:44Z

First steps

← Older revision		Revision as of 15:14, 3 February 2017
Line 36:		Line 36:

	qdel 24.hpc10.hpc		qdel 24.hpc10.hpc

			Of course, a job that is terminated prematurely will not give you the results you were expecting.

	As long as your job is running, there will be two temporary files in your home directory, like <code>24.hpc10.hpc.OU </code>and <code>24.hpc10.hpc.ER</code>, these names correspond with the jobd id. The output and error messages that would normally be printed on the command line, will be redirected to these temporary files. If your job is finished, these files will be renamed and moved to the directory where you were when the job was submitted. The new names will be the same as the name of your jobscript, followed by a dot and the letter o or e, plus the first part of the job id. The example job script above would, if executed correctly, leave two files; <code>job1.o24</code> which would contain the text 'Hello world!' and <code>job1.e24</code> which would be empty.		As long as your job is running, there will be two temporary files in your home directory, like <code>24.hpc10.hpc.OU </code>and <code>24.hpc10.hpc.ER</code>, these names correspond with the jobd id. The output and error messages that would normally be printed on the command line, will be redirected to these temporary files. If your job is finished, these files will be renamed and moved to the directory where you were when the job was submitted. The new names will be the same as the name of your jobscript, followed by a dot and the letter o or e, plus the first part of the job id. The example job script above would, if executed correctly, leave two files; <code>job1.o24</code> which would contain the text 'Hello world!' and <code>job1.e24</code> which would be empty.

Paul Keekstra: /* First steps */

2017-02-03T13:11:26Z

First steps

← Older revision		Revision as of 15:11, 3 February 2017
Line 6:		Line 6:
	#PBS -l nodes=1:ppn=1		#PBS -l nodes=1:ppn=1
	#		#
			cd $PBS_O_WORKDIR
	sleep 120		sleep 120
	echo "Hello world!"		echo "Hello world!"
Line 15:		Line 16:
	qsub job1		qsub job1

	The qsub command responds with a job id, wich looks like this: <code>24.hpc10.hpc</code>. The part before the first dot is system wide unique number that's increased by 1 for every new job that is submitted.		The qsub command responds with a job id, wich looks like this: <code>24.hpc10.hpc</code>. The part before the first dot is a system wide unique number that's increased by 1 for every new job that is submitted.

	You can check if your job is running with the <code>qstat</code> command. This command by itself gives a list of all running jobs, the job you just submitted will probably be one of the last down the list. For a wider list with slightly more information type <code>qstat -a</code>:		You can check if your job is running with the <code>qstat</code> command. This command by itself gives a list of all running jobs, the job you just submitted will probably be one of the last down the list. For a wider list with slightly more information type <code>qstat -a</code>:
Line 36:		Line 37:
	qdel 24.hpc10.hpc		qdel 24.hpc10.hpc

	As long as your job is running, there will be two temporary files in your home directory, like <code>24.hpc10.hpc.OU </code>and <code>24.hpc10.hpc.ER</code>, these names correspond with the jobd id. The output and error messages that would normally be printed on the command line, will be redirected to these temporary files. If your job is finished, these files will be renamed, the new names will be the same as the name of your jobscript, followed by a dot and the letter o or e, plus the first part of the job id. The example job script above would, if executed correctly, leave two files; <code>job1.o24</code> which would contain the text 'Hello world!' and <code>job1.e24</code> which would be empty.		As long as your job is running, there will be two temporary files in your home directory, like <code>24.hpc10.hpc.OU </code>and <code>24.hpc10.hpc.ER</code>, these names correspond with the jobd id. The output and error messages that would normally be printed on the command line, will be redirected to these temporary files. If your job is finished, these files will be renamed and moved to the directory where you were when the job was submitted. The new names will be the same as the name of your jobscript, followed by a dot and the letter o or e, plus the first part of the job id. The example job script above would, if executed correctly, leave two files; <code>job1.o24</code> which would contain the text 'Hello world!' and <code>job1.e24</code> which would be empty.

Paul Keekstra at 14:55, 21 December 2016

2016-12-21T14:55:21Z

← Older revision		Revision as of 16:55, 21 December 2016
Line 28:		Line 28:
	[jsmith@hpc10:~]$ </pre>		[jsmith@hpc10:~]$ </pre>

	The part that tells you if your job is running is the second to last column, if this is a capital Q, then your job is waiting for a free worker node, if it is a capital R, then your job is running.		The part that tells you if your job is running is the second to last column, if this is a capital Q, then your job is waiting for a free worker node, if it is a capital R, then your job is running. If you only want to see the job you just submitted, type <code>qstat -a</code> followed by the job id:

	~~On a large hpc cluster <code>qstat -a</code> shows all jobs of all users which can be a lot~~. If you only want to see the job you just submitted, type <code>qstat -a</code> followed by the job id:

	qstat -a 24.hpc10.hpc		qstat -a 24.hpc10.hpc

Paul Keekstra at 14:47, 21 December 2016

2016-12-21T14:47:44Z

← Older revision		Revision as of 16:47, 21 December 2016
Line 28:		Line 28:
	[jsmith@hpc10:~]$ </pre>		[jsmith@hpc10:~]$ </pre>

	The part that tells you if your job is running is the second to last column, if this is a capital Q, then your job is waiting for a free worker node, if it is a capital R, then your job is running. If you only want to see the job you just submitted, type <code>qstat -a</code> followed by the job id:		The part that tells you if your job is running is the second to last column, if this is a capital Q, then your job is waiting for a free worker node, if it is a capital R, then your job is running.

			On a large hpc cluster <code>qstat -a</code> shows all jobs of all users which can be a lot. If you only want to see the job you just submitted, type <code>qstat -a</code> followed by the job id:

	qstat -a 24.hpc10.hpc		qstat -a 24.hpc10.hpc

Paul Keekstra at 15:06, 15 September 2016

2016-09-15T15:06:33Z

← Older revision		Revision as of 17:06, 15 September 2016
Line 28:		Line 28:
	[jsmith@hpc10:~]$ </pre>		[jsmith@hpc10:~]$ </pre>

	The part that tells you if your job is running is the second to last column, if this is a capital Q, then ~~you~~ job is waiting for a free worker node, if it is a capital R, then your job is running. If you only want to see the job you just submitted, type <code>qstat -a</code> followed by the job id:		The part that tells you if your job is running is the second to last column, if this is a capital Q, then your job is waiting for a free worker node, if it is a capital R, then your job is running. If you only want to see the job you just submitted, type <code>qstat -a</code> followed by the job id:

	qstat -a 24.hpc10.hpc		qstat -a 24.hpc10.hpc

Paul Keekstra at 15:04, 15 September 2016

2016-09-15T15:04:19Z

← Older revision		Revision as of 17:04, 15 September 2016
Line 9:		Line 9:
	echo "Hello world!"		echo "Hello world!"

	This job does nothing for two minutes, after that it prints "Hello world!". The script starts with #PBS, which is a directive for Torque (the resource manager). In this example it means that you tell ~~the~~ Torque to use one CPU (ppn:1) on one node (nodes=1) for this job. There can be more than one directive, they should always be at the start of the script. The remaing lines in this example are just commands that you could type on the command line.		This job does nothing for two minutes, after that it prints "Hello world!". The script starts with #PBS, which is a directive for Torque (the resource manager). In this example it means that you tell Torque to use one CPU (ppn:1) on one node (nodes=1) for this job. There can be more than one directive, they should always be at the start of the script. The remaing lines in this example are just commands that you could type on the command line.

	Assuming that the name of the script is <code>job1</code> it can be submitted like this:		Assuming that the name of the script is <code>job1</code> it can be submitted like this: