<?xml version="1.0"?>
<feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en">
	<id>https://hpcwiki.tudelft.nl/api.php?action=feedcontributions&amp;feedformat=atom&amp;user=Paul+Keekstra</id>
	<title>hpcwiki - User contributions [en]</title>
	<link rel="self" type="application/atom+xml" href="https://hpcwiki.tudelft.nl/api.php?action=feedcontributions&amp;feedformat=atom&amp;user=Paul+Keekstra"/>
	<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Special:Contributions/Paul_Keekstra"/>
	<updated>2026-05-02T09:18:03Z</updated>
	<subtitle>User contributions</subtitle>
	<generator>MediaWiki 1.39.6</generator>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Introduction&amp;diff=303</id>
		<title>Introduction</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Introduction&amp;diff=303"/>
		<updated>2026-01-09T14:34:26Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* About the TU Delft hpc clusters */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== About the TU Delft hpc clusters ==&lt;br /&gt;
&lt;br /&gt;
Several research groups at the TU Delft have exclusive access to hpc (high performance computing) clusters that are managed and maintained by the ICT Department. Currently there are ten, they are typically named hpcXX.tudelft.net where the XX part is a number ranging from 03 to 27.&lt;br /&gt;
&lt;br /&gt;
{| class=&amp;quot;wikitable&amp;quot;&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Name&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Faculty&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Dept./group(s)&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| OS&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Nodes&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Total # CPU's&lt;br /&gt;
|-&lt;br /&gt;
| hpc05.tudelft.net || TNW || QN/BN || CentOS 7 || 28 || 624&lt;br /&gt;
|-&lt;br /&gt;
| hpc06.tudelft.net || 3ME || PME/MSE/DCSC/MTT || CentOS 7 || 37 || 2024&lt;br /&gt;
|-&lt;br /&gt;
| hpc07.tudelft.net || CiTG || ES-SCS || CentOS 7 || 8 || 256&lt;br /&gt;
|-&lt;br /&gt;
| hpc08.tudelft.net || CiTG || HE-EFM || CentOS 7 || 12 || 240&lt;br /&gt;
|-&lt;br /&gt;
| hpc11.tudelft.net || TNW || RST/RID/Cheme || CentOS 7 || 45 || 864&lt;br /&gt;
|-&lt;br /&gt;
| hpc12.tudelft.net || L&amp;amp;R || ASM/AWEP || CentOS 7 || 100 || 3972&lt;br /&gt;
|-&lt;br /&gt;
| hpc27.tudelft.net || CiTG || 3Md-AM || CentOS 7 || 20 || 400&lt;br /&gt;
|-			&lt;br /&gt;
|}&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are [https://en.wikipedia.org/wiki/Beowulf_cluster Beowulf] style clusters. The purpose of this wiki is to guide new users into using these clusters. Beowulf clusters are typically made of a bunch common and identical computers or servers linked together in a network. This can be used to perform parallel computing tasks on a much larger scale than would be possible on a single workstation or a heavy duty server.&lt;br /&gt;
&lt;br /&gt;
== About Linux ==&lt;br /&gt;
&lt;br /&gt;
The operating system on all our clusters is [http://www.centos.org/ CentOS 7], this is a well known server class Linux distribution. In order to use our hpc clusters you should at least have some basic knowledge of Linux on the command line. You will not find a Linux tutorial in this wiki, there are many tutorials on the internet that will teach you how to use Linux. Just use the phrase &amp;quot;Linux for beginners&amp;quot; in your favorite search engine and you'll find plenty of sources online. You can als opt to buy a book, [http://www.dummies.com/computers/operating-systems/linux/linux-for-dummies-9th-edition/ Linux for dummies] would be a good start, but there are many others that'll do just fine. If you have never used Linux on the command line before, please take the time to learn the basics, it will be well worth the effort.&lt;br /&gt;
&lt;br /&gt;
== More details ==&lt;br /&gt;
&lt;br /&gt;
Typically a Beowulf style cluster consists of a master node and several worker nodes. The master node is the machine where you log in to and where you prepare and manage your parallel jobs. A scheduler (Maui) and a resource manager (Torque) are both running on the master node. They both work together and provide a mechanism to submit jobs that will be run on the worker nodes where the actual parallel processing takes place. These nodes are stand alone computers connected through a local network to the master node and patiently waiting until the scheduler and the resource manager on the master node tells them to run a (part of a) parallel job.&lt;br /&gt;
&lt;br /&gt;
== About this wiki: contact and info ==&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are managed and maintained by the Linux systems department at the Shared Service Center ICT. You can contact us for any questions and comments about the hpc clusters by sending an email to servicepunt@tudelft.nl .&lt;br /&gt;
&lt;br /&gt;
You can also contact us if you find incorrect information on this wiki, if you are missing important information that you want added or if you have something to contribute to this wiki.&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Introduction&amp;diff=302</id>
		<title>Introduction</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Introduction&amp;diff=302"/>
		<updated>2026-01-09T14:32:18Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: &lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== About the TU Delft hpc clusters ==&lt;br /&gt;
&lt;br /&gt;
Several research groups at the TU Delft have exclusive access to hpc (high performance computing) clusters that are managed and maintained by the ICT Department. Currently there are ten, they are typically named hpcXX.tudelft.net where the XX part is a number ranging from 03 to 27.&lt;br /&gt;
&lt;br /&gt;
{| class=&amp;quot;wikitable&amp;quot;&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Name&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Faculty&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Dept./group(s)&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| OS&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Nodes&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Total # CPU's&lt;br /&gt;
|-&lt;br /&gt;
| hpc05.tudelft.net || TNW || QN/BN || CentOS 7 || 44 || 896&lt;br /&gt;
|-&lt;br /&gt;
| hpc06.tudelft.net || 3ME || PME/MSE/DCSC/MTT || CentOS 7 || 118 || 2084&lt;br /&gt;
|-&lt;br /&gt;
| hpc07.tudelft.net || CiTG || ES-SCS || CentOS 7 || 8 || 256&lt;br /&gt;
|-&lt;br /&gt;
| hpc08.tudelft.net || CiTG || HE-EFM || CentOS 7 || 12 || 240&lt;br /&gt;
|-&lt;br /&gt;
| hpc11.tudelft.net || TNW || RST/RID/Cheme || CentOS 7 || 45 || 1744&lt;br /&gt;
|-&lt;br /&gt;
| hpc12.tudelft.net || L&amp;amp;R || ASM/AWEP || CentOS 7 || 68 || 1536&lt;br /&gt;
|-&lt;br /&gt;
| hpc27.tudelft.net || CiTG || 3Md-AM || CentOS 7 || 20 || 400&lt;br /&gt;
|-			&lt;br /&gt;
|}&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are [https://en.wikipedia.org/wiki/Beowulf_cluster Beowulf] style clusters. The purpose of this wiki is to guide new users into using these clusters. Beowulf clusters are typically made of a bunch common and identical computers or servers linked together in a network. This can be used to perform parallel computing tasks on a much larger scale than would be possible on a single workstation or a heavy duty server.&lt;br /&gt;
&lt;br /&gt;
== About Linux ==&lt;br /&gt;
&lt;br /&gt;
The operating system on all our clusters is [http://www.centos.org/ CentOS 7], this is a well known server class Linux distribution. In order to use our hpc clusters you should at least have some basic knowledge of Linux on the command line. You will not find a Linux tutorial in this wiki, there are many tutorials on the internet that will teach you how to use Linux. Just use the phrase &amp;quot;Linux for beginners&amp;quot; in your favorite search engine and you'll find plenty of sources online. You can als opt to buy a book, [http://www.dummies.com/computers/operating-systems/linux/linux-for-dummies-9th-edition/ Linux for dummies] would be a good start, but there are many others that'll do just fine. If you have never used Linux on the command line before, please take the time to learn the basics, it will be well worth the effort.&lt;br /&gt;
&lt;br /&gt;
== More details ==&lt;br /&gt;
&lt;br /&gt;
Typically a Beowulf style cluster consists of a master node and several worker nodes. The master node is the machine where you log in to and where you prepare and manage your parallel jobs. A scheduler (Maui) and a resource manager (Torque) are both running on the master node. They both work together and provide a mechanism to submit jobs that will be run on the worker nodes where the actual parallel processing takes place. These nodes are stand alone computers connected through a local network to the master node and patiently waiting until the scheduler and the resource manager on the master node tells them to run a (part of a) parallel job.&lt;br /&gt;
&lt;br /&gt;
== About this wiki: contact and info ==&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are managed and maintained by the Linux systems department at the Shared Service Center ICT. You can contact us for any questions and comments about the hpc clusters by sending an email to servicepunt@tudelft.nl .&lt;br /&gt;
&lt;br /&gt;
You can also contact us if you find incorrect information on this wiki, if you are missing important information that you want added or if you have something to contribute to this wiki.&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=301</id>
		<title>How to log in</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=301"/>
		<updated>2025-01-21T12:42:07Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* Basics */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== Basics ==&lt;br /&gt;
&lt;br /&gt;
In order to log in you need a valid [https://www.tudelft.nl/en/student/ict/ict-facilities/netid-account/ NetID] that is enabled for access to one of the clusters. To have your NetID enabled, ask the contact person for the cluster you want access to (See the [[Introduction]] page) or ask the secretary of the department or research group where you work, he or she can tell you who the contact person is for an hpc cluster.&lt;br /&gt;
&lt;br /&gt;
If you want to collaborate with someone from outside the TU Delft and you want that person to access the same cluster as you, then you can request a guest account at the servicedesk. Once the guest account is created, it can be enabled for access to one of the clusters. '''Do not share your own NetID with others! Your NetID is for personal use only, sharing it with other people is forbidden!'''&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are only accessible via ssh (secure shell). To log in you need an ssh client. If you use Windows, you can download and use [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY], Macintosh and Linux computers already have an ssh client built in.&lt;br /&gt;
&lt;br /&gt;
You can only log in when you are on campus via a wireless (eduroam) or wired network. When you are off campus, you can log in using eduVPN (Institute Access). As of 16 Januari 2025 you can no longer use to the Linux bastion hosts when you are not on campus. There is also no need to use the Linux bastion hosts when you are on campus or using an active eduVPN connection.&lt;br /&gt;
&lt;br /&gt;
Every time you log in you'll see an motd (message of the day). Please pay attention to this, it may contain information about current events on the cluster. It also shows you an email address that you can use if there are problems or if you have questions. You can mail at any time you like, but you'll usually only get a reply during office hours, there is no 24/7 support on the hpc clusters.&lt;br /&gt;
&lt;br /&gt;
'''Please note:''' You have a local home directory on the cluster. No backups will be made of the home directories, if you make a mistake and loose a file, it is gone forever. '''Making backups of your important files on the hpc clusters is your own responsibility!'''&lt;br /&gt;
&lt;br /&gt;
== How to log in from Windows ==&lt;br /&gt;
&lt;br /&gt;
An often used ssh client for Windows is [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY]. If you use a Windows PC or a laptop that is provided and maintained by the TU Delft, it is already there, if you use your own Windows computer, you'll have to download and install it first from [http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html www.chiark.greenend.org.uk].&lt;br /&gt;
&lt;br /&gt;
Once you have started PuTTY you'll get a configuration window als shown.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can type the name of the cluster, for example 'hpc10.tudelft.net', in the field 'Host Name'. You can save the settings for a cluster by choosing a name for the session, enter it in the 'Saved Sessions' box and click 'Save'. The next time you open PuTTY the settings can be restored by clicking on the name and a click on 'Load'.&lt;br /&gt;
&lt;br /&gt;
By clicking 'Open' a connection is made to hpc10.tudelft.net. If this is the first time you are connecting to the cluster, PuTTY will warn you that you are connecting to a previously unknown host.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_02.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can confirm this connection by clicking 'yes' so PuTTY will store the host key in its database. You'll have to do this only once for every new server that you connect to.&lt;br /&gt;
&lt;br /&gt;
After you have accepted the connection, PuTTY opens a console and asks for your loginname. Once you have entered your NetID and your password, you are logged in to the cluster.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_03.png|center]]&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from Windows ==&lt;br /&gt;
&lt;br /&gt;
WinSCP is a freeware ftp client with support for secure file protocols sftp and scp. The login screen depicted below shows the settings for access within the University.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
After clicking on 'Login' a warning about host keys will appear, similar to PuTTY. If it is the first time, you can safely click on 'yes' to accept.&lt;br /&gt;
&lt;br /&gt;
Press 'Login'. If everything is setup correctly you are using the ssh tunnel the PuTTY bastion session has built and you are logged in on the cluster.&lt;br /&gt;
&lt;br /&gt;
When logged in, you will see the commander-like interface below.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_02.png|center|530px]]&lt;br /&gt;
&lt;br /&gt;
The left side displays the files in your home directory on the windows system. The right side displays the files on the cluster. You can drag and drop files between the two windows or select files with the mouse and press 'F5' for transfer to the other window.&lt;br /&gt;
&lt;br /&gt;
== How to log in from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
Mac OS X is in fact a full blown UNIX system under the hood. A terminal program comes standard with OS X, navigate with the finder to Applications -&amp;gt; Utilities to find it, the program is called 'Terminal'. There are many different flavours of Linux, each with their own desktop environments and customizations. Nearly every Linux system comes with a terminal program of some sort. Find the terminal program on your system and run it, once you get a command prompt type:&lt;br /&gt;
&lt;br /&gt;
 ssh &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
Where &amp;lt;netid&amp;gt; is of course your own netid.&lt;br /&gt;
&lt;br /&gt;
Upon first access, ssh requires you to check if the host key is correct. Since this is the first time you connect to the system, it's fine to accept the host key by pressing 'y'.&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh:&lt;br /&gt;
&lt;br /&gt;
 scp &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your remote home directory (folder) to the local home directory on your workstation or laptop. &lt;br /&gt;
&lt;br /&gt;
If you want to copy an entire directory with all its content, use the '-r' option, like this:&lt;br /&gt;
&lt;br /&gt;
 scp -r &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
This will copy the directory 'projects' from your remote home directory to the local home directory on your workstation or laptop.&lt;br /&gt;
&lt;br /&gt;
Of course, it is also possible to copy files and directories from your workstation or laptop to the remote machine:&lt;br /&gt;
&lt;br /&gt;
 scp -r projects.txt &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
 scp -r projects/ &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
And this will copy the directory 'projects' and all its content from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
&lt;br /&gt;
=== Windows ===&lt;br /&gt;
&lt;br /&gt;
There are several options for displaying graphical applications on a Windows system. A basic option is [http://www.straightrunning.com/XmingNotes/ Xming]. You can download it from: http://www.straightrunning.com/XmingNotes/ The site asks for a donation to use the latest version, but there is also a public domain version which can be used for free. Once you have installed it, you can find it in the usual place in the start menu. Before you log in to an hpc cluster and run graphical applications, you need to start Xming first. It will not show much, only a small X in the system tray once it's running. You also need to enable X forwarding in PuTTY. For that you need to click on the + next to SSH in the session panel (left part of the window), then click on X11. You'll see a line on the right side that says &amp;quot;Enable X11 forwarding&amp;quot;, click once on the white square next to that line and it will show a check mark. &lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_04.png|center]]&lt;br /&gt;
&lt;br /&gt;
Now you can continue logging in to the hpc cluster. To check if X forwarding works, type &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that will follow the mouse pointer.&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
[[File:Xeyes.png|center]]&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
An alternative option for displaying graphical applications is Exceed, this is a commercial package for which the TU Delft has a license. It is not available by default on the standard Windows systems, but it can be installed upon request if you contact the servicedesk.&lt;br /&gt;
&lt;br /&gt;
A second alternative is [https://mobaxterm.mobatek.net/ MobaXterm], this is an all-in-one solution containing an ssh client, X server and more. You can download a free version from: https://mobaxterm.mobatek.net/download.html The free version has some limitations, but it's still perfectly usable in our environment.&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
== How to access your Windows network shares ==&lt;br /&gt;
&lt;br /&gt;
=== Authenticate ===&lt;br /&gt;
&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
&lt;br /&gt;
 kinit&lt;br /&gt;
&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
=== Basic access ===&lt;br /&gt;
&lt;br /&gt;
Now just type:&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
&lt;br /&gt;
or&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-umbrella/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
&lt;br /&gt;
You can access your Windows home directory as well, the path is:&lt;br /&gt;
&lt;br /&gt;
 /tudelft/$USER/staff-homes/&lt;br /&gt;
&lt;br /&gt;
followed by the first letter of your surname and then your netid. For example, if your name would be John Smith and your netid jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
If you are a student, replace 'staff' with 'student' to access your files.&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
=== Important note ===&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your netid. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your netid yourself. If your netid would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well. Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own netid!&lt;br /&gt;
&lt;br /&gt;
'''If you use anthing other than your own netid you'll create a recipe for potential disaster!'''&lt;br /&gt;
&lt;br /&gt;
If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths. So &lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-group/&lt;br /&gt;
&lt;br /&gt;
only works if your netid is jsmith and&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/pbrown/staff-group/&lt;br /&gt;
&lt;br /&gt;
only works works if your netid is pbrown.&lt;br /&gt;
&lt;br /&gt;
If both netids have the same privileges for accessing the group volumes, then these commands give access to the same data.&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=300</id>
		<title>How to log in</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=300"/>
		<updated>2025-01-21T12:40:55Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: &lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== Basics ==&lt;br /&gt;
&lt;br /&gt;
In order to log in you need a valid [https://www.tudelft.nl/en/student/ict/ict-facilities/netid-account/ NetID] that is enabled for access to one of the clusters. To have your NetID enabled, ask the contact person for the cluster you want access to (See the [[Introduction]] page) or ask the secretary of the department or research group where you work, he or she can tell you who the contact person is for an hpc cluster.&lt;br /&gt;
&lt;br /&gt;
If you want to collaborate with someone from outside the TU Delft and you want that person to access the same cluster as you, then you can request a guest account at the servicedesk. Once the guest account is created, it can be enabled for access to one of the clusters. '''Do not share your own NetID with others! Your NetID is for personal use only, sharing it with other people is forbidden!'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are only accessible via ssh (secure shell). To log in you need an ssh client. If you use Windows, you can download and use [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY], Macintosh and Linux computers already have an ssh client built in.&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
'''You can only log in when you are on campus via a wireless (eduroam) or wired network. When you are off campus, you can log in using eduVPN (Institute Access). As of 16 Januari 2025 you can no longer use to the Linux bastion hosts when you are not on campus. There is also no need to use the Linux bastion hosts when you are on campus or using an active eduVPN connection.'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
Every time you log in you'll see an motd (message of the day). Please pay attention to this, it may contain information about current events on the cluster. It also shows you an email address that you can use if there are problems or if you have questions. You can mail at any time you like, but you'll usually only get a reply during office hours, there is no 24/7 support on the hpc clusters.&lt;br /&gt;
&lt;br /&gt;
'''Please note:''' You have a local home directory on the cluster. No backups will be made of the home directories, if you make a mistake and loose a file, it is gone forever. '''Making backups of your important files on the hpc clusters is your own responsibility!'''&lt;br /&gt;
&lt;br /&gt;
== How to log in from Windows ==&lt;br /&gt;
&lt;br /&gt;
An often used ssh client for Windows is [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY]. If you use a Windows PC or a laptop that is provided and maintained by the TU Delft, it is already there, if you use your own Windows computer, you'll have to download and install it first from [http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html www.chiark.greenend.org.uk].&lt;br /&gt;
&lt;br /&gt;
Once you have started PuTTY you'll get a configuration window als shown.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can type the name of the cluster, for example 'hpc10.tudelft.net', in the field 'Host Name'. You can save the settings for a cluster by choosing a name for the session, enter it in the 'Saved Sessions' box and click 'Save'. The next time you open PuTTY the settings can be restored by clicking on the name and a click on 'Load'.&lt;br /&gt;
&lt;br /&gt;
By clicking 'Open' a connection is made to hpc10.tudelft.net. If this is the first time you are connecting to the cluster, PuTTY will warn you that you are connecting to a previously unknown host.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_02.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can confirm this connection by clicking 'yes' so PuTTY will store the host key in its database. You'll have to do this only once for every new server that you connect to.&lt;br /&gt;
&lt;br /&gt;
After you have accepted the connection, PuTTY opens a console and asks for your loginname. Once you have entered your NetID and your password, you are logged in to the cluster.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_03.png|center]]&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from Windows ==&lt;br /&gt;
&lt;br /&gt;
WinSCP is a freeware ftp client with support for secure file protocols sftp and scp. The login screen depicted below shows the settings for access within the University.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
After clicking on 'Login' a warning about host keys will appear, similar to PuTTY. If it is the first time, you can safely click on 'yes' to accept.&lt;br /&gt;
&lt;br /&gt;
Press 'Login'. If everything is setup correctly you are using the ssh tunnel the PuTTY bastion session has built and you are logged in on the cluster.&lt;br /&gt;
&lt;br /&gt;
When logged in, you will see the commander-like interface below.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_02.png|center|530px]]&lt;br /&gt;
&lt;br /&gt;
The left side displays the files in your home directory on the windows system. The right side displays the files on the cluster. You can drag and drop files between the two windows or select files with the mouse and press 'F5' for transfer to the other window.&lt;br /&gt;
&lt;br /&gt;
== How to log in from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
Mac OS X is in fact a full blown UNIX system under the hood. A terminal program comes standard with OS X, navigate with the finder to Applications -&amp;gt; Utilities to find it, the program is called 'Terminal'. There are many different flavours of Linux, each with their own desktop environments and customizations. Nearly every Linux system comes with a terminal program of some sort. Find the terminal program on your system and run it, once you get a command prompt type:&lt;br /&gt;
&lt;br /&gt;
 ssh &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
Where &amp;lt;netid&amp;gt; is of course your own netid.&lt;br /&gt;
&lt;br /&gt;
Upon first access, ssh requires you to check if the host key is correct. Since this is the first time you connect to the system, it's fine to accept the host key by pressing 'y'.&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh:&lt;br /&gt;
&lt;br /&gt;
 scp &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your remote home directory (folder) to the local home directory on your workstation or laptop. &lt;br /&gt;
&lt;br /&gt;
If you want to copy an entire directory with all its content, use the '-r' option, like this:&lt;br /&gt;
&lt;br /&gt;
 scp -r &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
This will copy the directory 'projects' from your remote home directory to the local home directory on your workstation or laptop.&lt;br /&gt;
&lt;br /&gt;
Of course, it is also possible to copy files and directories from your workstation or laptop to the remote machine:&lt;br /&gt;
&lt;br /&gt;
 scp -r projects.txt &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
 scp -r projects/ &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
And this will copy the directory 'projects' and all its content from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
&lt;br /&gt;
=== Windows ===&lt;br /&gt;
&lt;br /&gt;
There are several options for displaying graphical applications on a Windows system. A basic option is [http://www.straightrunning.com/XmingNotes/ Xming]. You can download it from: http://www.straightrunning.com/XmingNotes/ The site asks for a donation to use the latest version, but there is also a public domain version which can be used for free. Once you have installed it, you can find it in the usual place in the start menu. Before you log in to an hpc cluster and run graphical applications, you need to start Xming first. It will not show much, only a small X in the system tray once it's running. You also need to enable X forwarding in PuTTY. For that you need to click on the + next to SSH in the session panel (left part of the window), then click on X11. You'll see a line on the right side that says &amp;quot;Enable X11 forwarding&amp;quot;, click once on the white square next to that line and it will show a check mark. &lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_04.png|center]]&lt;br /&gt;
&lt;br /&gt;
Now you can continue logging in to the hpc cluster. To check if X forwarding works, type &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that will follow the mouse pointer.&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
[[File:Xeyes.png|center]]&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
An alternative option for displaying graphical applications is Exceed, this is a commercial package for which the TU Delft has a license. It is not available by default on the standard Windows systems, but it can be installed upon request if you contact the servicedesk.&lt;br /&gt;
&lt;br /&gt;
A second alternative is [https://mobaxterm.mobatek.net/ MobaXterm], this is an all-in-one solution containing an ssh client, X server and more. You can download a free version from: https://mobaxterm.mobatek.net/download.html The free version has some limitations, but it's still perfectly usable in our environment.&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
== How to access your Windows network shares ==&lt;br /&gt;
&lt;br /&gt;
=== Authenticate ===&lt;br /&gt;
&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
&lt;br /&gt;
 kinit&lt;br /&gt;
&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
=== Basic access ===&lt;br /&gt;
&lt;br /&gt;
Now just type:&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
&lt;br /&gt;
or&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-umbrella/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
&lt;br /&gt;
You can access your Windows home directory as well, the path is:&lt;br /&gt;
&lt;br /&gt;
 /tudelft/$USER/staff-homes/&lt;br /&gt;
&lt;br /&gt;
followed by the first letter of your surname and then your netid. For example, if your name would be John Smith and your netid jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
If you are a student, replace 'staff' with 'student' to access your files.&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
=== Important note ===&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your netid. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your netid yourself. If your netid would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well. Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own netid!&lt;br /&gt;
&lt;br /&gt;
'''If you use anthing other than your own netid you'll create a recipe for potential disaster!'''&lt;br /&gt;
&lt;br /&gt;
If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths. So &lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-group/&lt;br /&gt;
&lt;br /&gt;
only works if your netid is jsmith and&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/pbrown/staff-group/&lt;br /&gt;
&lt;br /&gt;
only works works if your netid is pbrown.&lt;br /&gt;
&lt;br /&gt;
If both netids have the same privileges for accessing the group volumes, then these commands give access to the same data.&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=299</id>
		<title>Remote Access</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=299"/>
		<updated>2025-01-21T12:40:15Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: &lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;Access to the HPC-clusters is limited from outside the TU Delft network for security reasons.&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
'''You can only log in when you are on campus via a wireless (eduroam) or wired network. When you are off campus, you can log in using eduVPN (Institute Access). As of 16 Januari 2025 you can no longer use to the Linux bastion hosts when you are not on campus. There is also no need to use the Linux bastion hosts when you are on campus or using an active eduVPN connection'''&lt;br /&gt;
&lt;br /&gt;
The information below is for the most part outdated. Please refer to the [[How to log in|Access]] page for log in instructions: [[How to log in|Access (How to log in and how to copy files)]]&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
== Bastion ==&lt;br /&gt;
''(This information is outdated and can be safely ignored.)''&lt;br /&gt;
&lt;br /&gt;
The TU Delft offers two bastion servers. These servers can be used to tunnel your traffic to the HPC.&lt;br /&gt;
*linux-bastion.tudelft.nl for employees or guests.&lt;br /&gt;
*student-linux.tudelft.nl for students.&lt;br /&gt;
&lt;br /&gt;
== Access to the commandline ==&lt;br /&gt;
The explanations below are the simplest way to connect to a console on an HPC. [[#Advanced configs|Advanced configs]] describes more complex configurations which save time when regularly using the connection or when you want to use [[#Linux|X-forwarding]]&lt;br /&gt;
=== Command line ===&lt;br /&gt;
When your desktop/laptop supports ssh from the console:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
If your SSH-client doesn't support Jump hosts, you can first connect to the bastion server and then to the HPC:&lt;br /&gt;
&lt;br /&gt;
 [user@home]$ ssh &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
 [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt; with your NetID (also remove the &amp;lt;&amp;gt;)&lt;br /&gt;
*When your a student replace &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl with &amp;lt;NetID&amp;gt;@student-linux.tudelft.nl&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net with the HPC you are connecting to.&lt;br /&gt;
&lt;br /&gt;
=== Advanced configs ===&lt;br /&gt;
==== Making OpenSSH more user-friendly ====&lt;br /&gt;
The OpenSSH clients can be configured using a configuration file in ~/.ssh/config (Linux, macOS) or ~\.ssh\config (Windows).&lt;br /&gt;
&lt;br /&gt;
===== Define connections =====&lt;br /&gt;
For the OpenSSH clients you can predefine often used connections by placing a section like the following in the configuration file on your local computer. Authentication on the bastion with keys is not possible, so password is used.&amp;lt;br&amp;gt;&lt;br /&gt;
Employees:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
Students:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
You can then connect simply by ssh bastion. For example:&lt;br /&gt;
  [user@home]$ ssh bastion&lt;br /&gt;
    NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
&lt;br /&gt;
===== SSH Proxy Support =====&lt;br /&gt;
Since this configuration depends on [[#Define connections|Define connections]] configure that first.&amp;lt;br&amp;gt;&lt;br /&gt;
To connect directly from your machine to an HPC cluster login node (without connecting to the bastion server first), create a connection via a proxy by adding the following lines to the configuration file ~/.ssh/config on your local computer:&lt;br /&gt;
&lt;br /&gt;
  Host hpcXX&lt;br /&gt;
  Hostname hpcXX.tudelft.net&lt;br /&gt;
  ProxyCommand ssh -W %h:%p bastion&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
&lt;br /&gt;
You can then simply use ssh hpcXX (or scp -p '&amp;lt;local file&amp;gt;' &amp;quot;hpcXX:/scratch/&amp;lt;NetID&amp;gt;/&amp;quot;) from your local computer.&lt;br /&gt;
&lt;br /&gt;
===== Session multiplexing =====&lt;br /&gt;
OpenSSH clients support multiple sessions over a single connection. This means you have to enter your password only once, to make the connection, and can then open multiple sessions (SSH, SCP and SFTP) to that computer over the same connection. This is especially convenient when connecting to the bastion server, where you have to type your password always.&lt;br /&gt;
&lt;br /&gt;
Note: SSH public key logins won't work (reliably, (because Kerberos authentication is required to access your home directory).&lt;br /&gt;
&lt;br /&gt;
Add the following section to the end of the configuration file on your local computer and on the bastion server:&lt;br /&gt;
&lt;br /&gt;
  Host *&lt;br /&gt;
  ControlMaster auto&lt;br /&gt;
  ControlPath /tmp/ssh-%r@%h:%p&lt;br /&gt;
&lt;br /&gt;
Windows users may need to adapt the ControlPath location to match Windows.&lt;br /&gt;
&lt;br /&gt;
You can use the exact same commands as above.&lt;br /&gt;
=== PuTTY ===&lt;br /&gt;
When your desktop/laptop doesn't support ssh from the console, you can use [https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html PuTTY] (it may already be installed).&amp;lt;br&amp;gt;&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students), and select the SSH protocol (port 22):&amp;lt;br&amp;gt;[[File:PuTTY-Hostname.png]]&lt;br /&gt;
*OPTIONAL: Connection&amp;gt;SSH Settings&amp;lt;br&amp;gt;Select Share SSH connections if possible (this means that you will not have to authenticate when you start a second session):&amp;lt;br&amp;gt;[[File:PuTTY-ShareConnection.png]]&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;To store these session settings, go back to the session tab, and save the settings under a session name. To activate a stored session, double-click on the session name:&amp;lt;br&amp;gt;[[File:PuTTY-SaveSession.png]]&lt;br /&gt;
Once you're connected to the bastion server you can use the locally available ssh to connect to a HPC:&lt;br /&gt;
   [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
     NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
     Welcome&lt;br /&gt;
   [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
=== MobaXterm ===&lt;br /&gt;
MobaXterm is a very versatile application. It can be used to connect to the commandline on HPC, display graphical applications and transfer files. It also has built-in support for using SSH-tunneling to connect through a bastion. You can download it from [https://mobaxterm.mobatek.net/ https://mobaxterm.mobatek.net/]. The '''free version''' has enough features for most users. The '''Portable edition''' can be run without installing (just unzip).&lt;br /&gt;
*Click the '''Session''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSession.png]]&lt;br /&gt;
*Click the '''SSH''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSSH.png]]&lt;br /&gt;
*Enter '''Remote host'''&amp;lt;br&amp;gt;Check '''Specify username'''&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;[[File:MobaXterm_Hostname.png]]&lt;br /&gt;
*tab: Network settings&amp;lt;br&amp;gt;Click '''SSH gateway (jump host)'''&amp;lt;br&amp;gt;[[File:MobaXterm_NewJumphost.png]]&lt;br /&gt;
*Enter linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;Click '''OK'''&amp;lt;br&amp;gt;[[File:MobaXterm_Jumphost.png]]&lt;br /&gt;
*Click '''OK''' in the session settings to start connecting&amp;lt;br&amp;gt;You will be asked for your password&amp;lt;br&amp;gt;Enter your password on the console of hpcXX&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
== File Transfer ==&lt;br /&gt;
The best method for transfering files depends on where you want to store your files:&lt;br /&gt;
#Files in your home directory on HPC are only available within HPC&lt;br /&gt;
#Files on Network Shares are available on other locations (such as a TU Delft workplace, weblogin.tudelft.nl or other HPC clusters)&amp;lt;br&amp;gt;Copying files to these shares is best done using the manuals on [https://webdata.tudelft.nl/ webdata.tudelft.nl]&lt;br /&gt;
&lt;br /&gt;
When you want to copy files to the HPC you can copy them from Network Shares&lt;br /&gt;
=== (Windows) Network Shares ===&lt;br /&gt;
&lt;br /&gt;
==== Authenticate ====&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
 kinit&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
==== Basic access ====&lt;br /&gt;
&lt;br /&gt;
Now just type (depending on which share you want to use):&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
 cd /tudelft/$USER/student-group/&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
 cd /tudelft/$USER/staff-umbrella/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
You can access your Windows home directory as well. The subdirectories of the home directories are followed by '''the first letter of your surname''' and then your NetID.&amp;lt;br&amp;gt;&lt;br /&gt;
For example, if your name would be John Smith and your NetID jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
==== Important note ====&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your NetID. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your NetID yourself. If your NetID would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well. Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own NetID!&amp;lt;br&amp;gt;&lt;br /&gt;
'''If you use anthing other than your own NetID you'll create a recipe for potential disaster!'''&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths.&amp;lt;br&amp;gt;&lt;br /&gt;
So &amp;lt;code&amp;gt;/tudelft/jsmith/staff-group/&amp;lt;/code&amp;gt; only works if your NetID is jsmith and &amp;lt;code&amp;gt;/tudelft/pbrown/staff-group/&amp;lt;/code&amp;gt; only works works if your NetID is pbrown.&amp;lt;br&amp;gt;&lt;br /&gt;
If both NetIDs have the same privileges for accessing the group volumes, then these commands give access to the same data.&lt;br /&gt;
&lt;br /&gt;
=== Data on HPC ===&lt;br /&gt;
If you want to transfer data directly to the HPC you will need some extra configuration&lt;br /&gt;
==== Command line ====&lt;br /&gt;
Please first configure [[#SSH Proxy Support|SSH Proxy Support]].&amp;lt;br&amp;gt;&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your remote home directory on HPC to the local home directory on your desktop or laptop. &lt;br /&gt;
 scp &amp;lt;NetID&amp;gt;@hpcXX:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
*Copy an entire directory with all its content, use the '-r' option&amp;lt;br&amp;gt;This will copy the directory 'projects' from your remote home directory on HPC to the local home directory on your desktop or laptop:&lt;br /&gt;
 scp -r &amp;lt;NetID&amp;gt;@hpcXX:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your local home directory on your desktop or laptop to your remote home directory on HPC. &lt;br /&gt;
 scp projects.txt &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
*Copy the directory 'projects' and all its content from your current working directory on your desktop or laptop to your remote home directory on HPC.&lt;br /&gt;
 scp -r projects/ &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
==== WinSCP ====&lt;br /&gt;
WinSCP is a simple tool that has the internal option to create a tunnel through  the bastion server. It can be downloaded from [https://winscp.net/eng/download.php https://winscp.net/eng/download.php]&lt;br /&gt;
*Session &amp;gt; New Session&lt;br /&gt;
[[File:WinSCP01.png]]&lt;br /&gt;
*Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;Advanced&amp;quot; button&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Connection &amp;gt; Tunnel&amp;lt;br&amp;gt;Select &amp;quot;Connect through SSH tunnel&amp;quot;&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;OK&amp;quot; button&lt;br /&gt;
[[File:WinSCP03.png]]&lt;br /&gt;
*Click &amp;quot;Save&amp;quot;&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Save session as site&amp;lt;br&amp;gt;You can accept the suggested name&lt;br /&gt;
[[File:WinSCP04.png]]&lt;br /&gt;
*You can now &amp;quot;Login&amp;quot;&amp;lt;br&amp;gt;You will be asked to supply your password twice: first for the bastion, second for the HPC&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
==== MobaXterm ====&lt;br /&gt;
MobaXterm can also be used for file transfers. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*Left Window&amp;lt;br&amp;gt;Download files with '''arrow down'''&amp;lt;br&amp;gt;Upload files with '''arrow up'''&amp;lt;br&amp;gt;[[File:MobaXterm_SFTP.png]]&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
=== Windows ===&lt;br /&gt;
MobaXterm can also be used for displaying graphical applications. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*In the commandline windows (right)&amp;lt;br&amp;gt;Test X-forwarding (Graphics) by running &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt;&amp;lt;br&amp;gt;You should see these eyes (following the cursor)&amp;lt;br&amp;gt;[[File:MobaXterm_Xeyes.png]]&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in&amp;lt;br&amp;gt;&lt;br /&gt;
The easiest way is:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
When you have configured [[#SSH_Proxy_Support|SSH Proxy Support]] you can connect with:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=298</id>
		<title>Remote Access</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=298"/>
		<updated>2025-01-21T12:39:53Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: &lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;Access to the HPC-clusters is limited from outside the TU Delft network for security reasons.&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
'''You can only log in when you are on campus via a wireless (eduroam) or wired network. When you are off campus, you can log in using eduVPN (Institute Access). As of 16 Januari 2025 you can no longer use to the Linux bastion hosts when you are not on campus. There is also no need to use the Linux bastion hosts when you are on campus or using an active eduVPN connection'''&lt;br /&gt;
&lt;br /&gt;
'''The information below is for the most part outdated. Please refer to the [[How to log in|Access]] page for log in instructions:''' [[How to log in|Access (How to log in and how to copy files)]]&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
== Bastion ==&lt;br /&gt;
''(This information is outdated and can be safely ignored.)''&lt;br /&gt;
&lt;br /&gt;
The TU Delft offers two bastion servers. These servers can be used to tunnel your traffic to the HPC.&lt;br /&gt;
*linux-bastion.tudelft.nl for employees or guests.&lt;br /&gt;
*student-linux.tudelft.nl for students.&lt;br /&gt;
&lt;br /&gt;
== Access to the commandline ==&lt;br /&gt;
The explanations below are the simplest way to connect to a console on an HPC. [[#Advanced configs|Advanced configs]] describes more complex configurations which save time when regularly using the connection or when you want to use [[#Linux|X-forwarding]]&lt;br /&gt;
=== Command line ===&lt;br /&gt;
When your desktop/laptop supports ssh from the console:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
If your SSH-client doesn't support Jump hosts, you can first connect to the bastion server and then to the HPC:&lt;br /&gt;
&lt;br /&gt;
 [user@home]$ ssh &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
 [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt; with your NetID (also remove the &amp;lt;&amp;gt;)&lt;br /&gt;
*When your a student replace &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl with &amp;lt;NetID&amp;gt;@student-linux.tudelft.nl&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net with the HPC you are connecting to.&lt;br /&gt;
&lt;br /&gt;
=== Advanced configs ===&lt;br /&gt;
==== Making OpenSSH more user-friendly ====&lt;br /&gt;
The OpenSSH clients can be configured using a configuration file in ~/.ssh/config (Linux, macOS) or ~\.ssh\config (Windows).&lt;br /&gt;
&lt;br /&gt;
===== Define connections =====&lt;br /&gt;
For the OpenSSH clients you can predefine often used connections by placing a section like the following in the configuration file on your local computer. Authentication on the bastion with keys is not possible, so password is used.&amp;lt;br&amp;gt;&lt;br /&gt;
Employees:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
Students:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
You can then connect simply by ssh bastion. For example:&lt;br /&gt;
  [user@home]$ ssh bastion&lt;br /&gt;
    NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
&lt;br /&gt;
===== SSH Proxy Support =====&lt;br /&gt;
Since this configuration depends on [[#Define connections|Define connections]] configure that first.&amp;lt;br&amp;gt;&lt;br /&gt;
To connect directly from your machine to an HPC cluster login node (without connecting to the bastion server first), create a connection via a proxy by adding the following lines to the configuration file ~/.ssh/config on your local computer:&lt;br /&gt;
&lt;br /&gt;
  Host hpcXX&lt;br /&gt;
  Hostname hpcXX.tudelft.net&lt;br /&gt;
  ProxyCommand ssh -W %h:%p bastion&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
&lt;br /&gt;
You can then simply use ssh hpcXX (or scp -p '&amp;lt;local file&amp;gt;' &amp;quot;hpcXX:/scratch/&amp;lt;NetID&amp;gt;/&amp;quot;) from your local computer.&lt;br /&gt;
&lt;br /&gt;
===== Session multiplexing =====&lt;br /&gt;
OpenSSH clients support multiple sessions over a single connection. This means you have to enter your password only once, to make the connection, and can then open multiple sessions (SSH, SCP and SFTP) to that computer over the same connection. This is especially convenient when connecting to the bastion server, where you have to type your password always.&lt;br /&gt;
&lt;br /&gt;
Note: SSH public key logins won't work (reliably, (because Kerberos authentication is required to access your home directory).&lt;br /&gt;
&lt;br /&gt;
Add the following section to the end of the configuration file on your local computer and on the bastion server:&lt;br /&gt;
&lt;br /&gt;
  Host *&lt;br /&gt;
  ControlMaster auto&lt;br /&gt;
  ControlPath /tmp/ssh-%r@%h:%p&lt;br /&gt;
&lt;br /&gt;
Windows users may need to adapt the ControlPath location to match Windows.&lt;br /&gt;
&lt;br /&gt;
You can use the exact same commands as above.&lt;br /&gt;
=== PuTTY ===&lt;br /&gt;
When your desktop/laptop doesn't support ssh from the console, you can use [https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html PuTTY] (it may already be installed).&amp;lt;br&amp;gt;&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students), and select the SSH protocol (port 22):&amp;lt;br&amp;gt;[[File:PuTTY-Hostname.png]]&lt;br /&gt;
*OPTIONAL: Connection&amp;gt;SSH Settings&amp;lt;br&amp;gt;Select Share SSH connections if possible (this means that you will not have to authenticate when you start a second session):&amp;lt;br&amp;gt;[[File:PuTTY-ShareConnection.png]]&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;To store these session settings, go back to the session tab, and save the settings under a session name. To activate a stored session, double-click on the session name:&amp;lt;br&amp;gt;[[File:PuTTY-SaveSession.png]]&lt;br /&gt;
Once you're connected to the bastion server you can use the locally available ssh to connect to a HPC:&lt;br /&gt;
   [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
     NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
     Welcome&lt;br /&gt;
   [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
=== MobaXterm ===&lt;br /&gt;
MobaXterm is a very versatile application. It can be used to connect to the commandline on HPC, display graphical applications and transfer files. It also has built-in support for using SSH-tunneling to connect through a bastion. You can download it from [https://mobaxterm.mobatek.net/ https://mobaxterm.mobatek.net/]. The '''free version''' has enough features for most users. The '''Portable edition''' can be run without installing (just unzip).&lt;br /&gt;
*Click the '''Session''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSession.png]]&lt;br /&gt;
*Click the '''SSH''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSSH.png]]&lt;br /&gt;
*Enter '''Remote host'''&amp;lt;br&amp;gt;Check '''Specify username'''&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;[[File:MobaXterm_Hostname.png]]&lt;br /&gt;
*tab: Network settings&amp;lt;br&amp;gt;Click '''SSH gateway (jump host)'''&amp;lt;br&amp;gt;[[File:MobaXterm_NewJumphost.png]]&lt;br /&gt;
*Enter linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;Click '''OK'''&amp;lt;br&amp;gt;[[File:MobaXterm_Jumphost.png]]&lt;br /&gt;
*Click '''OK''' in the session settings to start connecting&amp;lt;br&amp;gt;You will be asked for your password&amp;lt;br&amp;gt;Enter your password on the console of hpcXX&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
== File Transfer ==&lt;br /&gt;
The best method for transfering files depends on where you want to store your files:&lt;br /&gt;
#Files in your home directory on HPC are only available within HPC&lt;br /&gt;
#Files on Network Shares are available on other locations (such as a TU Delft workplace, weblogin.tudelft.nl or other HPC clusters)&amp;lt;br&amp;gt;Copying files to these shares is best done using the manuals on [https://webdata.tudelft.nl/ webdata.tudelft.nl]&lt;br /&gt;
&lt;br /&gt;
When you want to copy files to the HPC you can copy them from Network Shares&lt;br /&gt;
=== (Windows) Network Shares ===&lt;br /&gt;
&lt;br /&gt;
==== Authenticate ====&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
 kinit&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
==== Basic access ====&lt;br /&gt;
&lt;br /&gt;
Now just type (depending on which share you want to use):&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
 cd /tudelft/$USER/student-group/&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
 cd /tudelft/$USER/staff-umbrella/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
You can access your Windows home directory as well. The subdirectories of the home directories are followed by '''the first letter of your surname''' and then your NetID.&amp;lt;br&amp;gt;&lt;br /&gt;
For example, if your name would be John Smith and your NetID jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
==== Important note ====&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your NetID. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your NetID yourself. If your NetID would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well. Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own NetID!&amp;lt;br&amp;gt;&lt;br /&gt;
'''If you use anthing other than your own NetID you'll create a recipe for potential disaster!'''&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths.&amp;lt;br&amp;gt;&lt;br /&gt;
So &amp;lt;code&amp;gt;/tudelft/jsmith/staff-group/&amp;lt;/code&amp;gt; only works if your NetID is jsmith and &amp;lt;code&amp;gt;/tudelft/pbrown/staff-group/&amp;lt;/code&amp;gt; only works works if your NetID is pbrown.&amp;lt;br&amp;gt;&lt;br /&gt;
If both NetIDs have the same privileges for accessing the group volumes, then these commands give access to the same data.&lt;br /&gt;
&lt;br /&gt;
=== Data on HPC ===&lt;br /&gt;
If you want to transfer data directly to the HPC you will need some extra configuration&lt;br /&gt;
==== Command line ====&lt;br /&gt;
Please first configure [[#SSH Proxy Support|SSH Proxy Support]].&amp;lt;br&amp;gt;&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your remote home directory on HPC to the local home directory on your desktop or laptop. &lt;br /&gt;
 scp &amp;lt;NetID&amp;gt;@hpcXX:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
*Copy an entire directory with all its content, use the '-r' option&amp;lt;br&amp;gt;This will copy the directory 'projects' from your remote home directory on HPC to the local home directory on your desktop or laptop:&lt;br /&gt;
 scp -r &amp;lt;NetID&amp;gt;@hpcXX:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your local home directory on your desktop or laptop to your remote home directory on HPC. &lt;br /&gt;
 scp projects.txt &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
*Copy the directory 'projects' and all its content from your current working directory on your desktop or laptop to your remote home directory on HPC.&lt;br /&gt;
 scp -r projects/ &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
==== WinSCP ====&lt;br /&gt;
WinSCP is a simple tool that has the internal option to create a tunnel through  the bastion server. It can be downloaded from [https://winscp.net/eng/download.php https://winscp.net/eng/download.php]&lt;br /&gt;
*Session &amp;gt; New Session&lt;br /&gt;
[[File:WinSCP01.png]]&lt;br /&gt;
*Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;Advanced&amp;quot; button&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Connection &amp;gt; Tunnel&amp;lt;br&amp;gt;Select &amp;quot;Connect through SSH tunnel&amp;quot;&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;OK&amp;quot; button&lt;br /&gt;
[[File:WinSCP03.png]]&lt;br /&gt;
*Click &amp;quot;Save&amp;quot;&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Save session as site&amp;lt;br&amp;gt;You can accept the suggested name&lt;br /&gt;
[[File:WinSCP04.png]]&lt;br /&gt;
*You can now &amp;quot;Login&amp;quot;&amp;lt;br&amp;gt;You will be asked to supply your password twice: first for the bastion, second for the HPC&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
==== MobaXterm ====&lt;br /&gt;
MobaXterm can also be used for file transfers. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*Left Window&amp;lt;br&amp;gt;Download files with '''arrow down'''&amp;lt;br&amp;gt;Upload files with '''arrow up'''&amp;lt;br&amp;gt;[[File:MobaXterm_SFTP.png]]&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
=== Windows ===&lt;br /&gt;
MobaXterm can also be used for displaying graphical applications. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*In the commandline windows (right)&amp;lt;br&amp;gt;Test X-forwarding (Graphics) by running &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt;&amp;lt;br&amp;gt;You should see these eyes (following the cursor)&amp;lt;br&amp;gt;[[File:MobaXterm_Xeyes.png]]&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in&amp;lt;br&amp;gt;&lt;br /&gt;
The easiest way is:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
When you have configured [[#SSH_Proxy_Support|SSH Proxy Support]] you can connect with:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=297</id>
		<title>Remote Access</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=297"/>
		<updated>2025-01-21T12:39:33Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: &lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;Access to the HPC-clusters is limited from outside the TU Delft network for security reasons.&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
'''You can only log in when you are on campus via a wireless (eduroam) or wired network. When you are off campus, you can log in using eduVPN (Institute Access). As of 16 Januari 2025 you can no longer use to the Linux bastion hosts when you are not on campus. There is also no need to use the Linux bastion hosts when you are on campus or using an active eduVPN connection&lt;br /&gt;
&lt;br /&gt;
The information below is for the most part outdated. Please refer to the [[How to log in|Access]] page for log in instructions:''' [[How to log in|Access (How to log in and how to copy files)]]&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
== Bastion ==&lt;br /&gt;
''(This information is outdated and can be safely ignored.)''&lt;br /&gt;
&lt;br /&gt;
The TU Delft offers two bastion servers. These servers can be used to tunnel your traffic to the HPC.&lt;br /&gt;
*linux-bastion.tudelft.nl for employees or guests.&lt;br /&gt;
*student-linux.tudelft.nl for students.&lt;br /&gt;
&lt;br /&gt;
== Access to the commandline ==&lt;br /&gt;
The explanations below are the simplest way to connect to a console on an HPC. [[#Advanced configs|Advanced configs]] describes more complex configurations which save time when regularly using the connection or when you want to use [[#Linux|X-forwarding]]&lt;br /&gt;
=== Command line ===&lt;br /&gt;
When your desktop/laptop supports ssh from the console:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
If your SSH-client doesn't support Jump hosts, you can first connect to the bastion server and then to the HPC:&lt;br /&gt;
&lt;br /&gt;
 [user@home]$ ssh &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
 [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt; with your NetID (also remove the &amp;lt;&amp;gt;)&lt;br /&gt;
*When your a student replace &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl with &amp;lt;NetID&amp;gt;@student-linux.tudelft.nl&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net with the HPC you are connecting to.&lt;br /&gt;
&lt;br /&gt;
=== Advanced configs ===&lt;br /&gt;
==== Making OpenSSH more user-friendly ====&lt;br /&gt;
The OpenSSH clients can be configured using a configuration file in ~/.ssh/config (Linux, macOS) or ~\.ssh\config (Windows).&lt;br /&gt;
&lt;br /&gt;
===== Define connections =====&lt;br /&gt;
For the OpenSSH clients you can predefine often used connections by placing a section like the following in the configuration file on your local computer. Authentication on the bastion with keys is not possible, so password is used.&amp;lt;br&amp;gt;&lt;br /&gt;
Employees:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
Students:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
You can then connect simply by ssh bastion. For example:&lt;br /&gt;
  [user@home]$ ssh bastion&lt;br /&gt;
    NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
&lt;br /&gt;
===== SSH Proxy Support =====&lt;br /&gt;
Since this configuration depends on [[#Define connections|Define connections]] configure that first.&amp;lt;br&amp;gt;&lt;br /&gt;
To connect directly from your machine to an HPC cluster login node (without connecting to the bastion server first), create a connection via a proxy by adding the following lines to the configuration file ~/.ssh/config on your local computer:&lt;br /&gt;
&lt;br /&gt;
  Host hpcXX&lt;br /&gt;
  Hostname hpcXX.tudelft.net&lt;br /&gt;
  ProxyCommand ssh -W %h:%p bastion&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
&lt;br /&gt;
You can then simply use ssh hpcXX (or scp -p '&amp;lt;local file&amp;gt;' &amp;quot;hpcXX:/scratch/&amp;lt;NetID&amp;gt;/&amp;quot;) from your local computer.&lt;br /&gt;
&lt;br /&gt;
===== Session multiplexing =====&lt;br /&gt;
OpenSSH clients support multiple sessions over a single connection. This means you have to enter your password only once, to make the connection, and can then open multiple sessions (SSH, SCP and SFTP) to that computer over the same connection. This is especially convenient when connecting to the bastion server, where you have to type your password always.&lt;br /&gt;
&lt;br /&gt;
Note: SSH public key logins won't work (reliably, (because Kerberos authentication is required to access your home directory).&lt;br /&gt;
&lt;br /&gt;
Add the following section to the end of the configuration file on your local computer and on the bastion server:&lt;br /&gt;
&lt;br /&gt;
  Host *&lt;br /&gt;
  ControlMaster auto&lt;br /&gt;
  ControlPath /tmp/ssh-%r@%h:%p&lt;br /&gt;
&lt;br /&gt;
Windows users may need to adapt the ControlPath location to match Windows.&lt;br /&gt;
&lt;br /&gt;
You can use the exact same commands as above.&lt;br /&gt;
=== PuTTY ===&lt;br /&gt;
When your desktop/laptop doesn't support ssh from the console, you can use [https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html PuTTY] (it may already be installed).&amp;lt;br&amp;gt;&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students), and select the SSH protocol (port 22):&amp;lt;br&amp;gt;[[File:PuTTY-Hostname.png]]&lt;br /&gt;
*OPTIONAL: Connection&amp;gt;SSH Settings&amp;lt;br&amp;gt;Select Share SSH connections if possible (this means that you will not have to authenticate when you start a second session):&amp;lt;br&amp;gt;[[File:PuTTY-ShareConnection.png]]&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;To store these session settings, go back to the session tab, and save the settings under a session name. To activate a stored session, double-click on the session name:&amp;lt;br&amp;gt;[[File:PuTTY-SaveSession.png]]&lt;br /&gt;
Once you're connected to the bastion server you can use the locally available ssh to connect to a HPC:&lt;br /&gt;
   [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
     NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
     Welcome&lt;br /&gt;
   [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
=== MobaXterm ===&lt;br /&gt;
MobaXterm is a very versatile application. It can be used to connect to the commandline on HPC, display graphical applications and transfer files. It also has built-in support for using SSH-tunneling to connect through a bastion. You can download it from [https://mobaxterm.mobatek.net/ https://mobaxterm.mobatek.net/]. The '''free version''' has enough features for most users. The '''Portable edition''' can be run without installing (just unzip).&lt;br /&gt;
*Click the '''Session''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSession.png]]&lt;br /&gt;
*Click the '''SSH''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSSH.png]]&lt;br /&gt;
*Enter '''Remote host'''&amp;lt;br&amp;gt;Check '''Specify username'''&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;[[File:MobaXterm_Hostname.png]]&lt;br /&gt;
*tab: Network settings&amp;lt;br&amp;gt;Click '''SSH gateway (jump host)'''&amp;lt;br&amp;gt;[[File:MobaXterm_NewJumphost.png]]&lt;br /&gt;
*Enter linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;Click '''OK'''&amp;lt;br&amp;gt;[[File:MobaXterm_Jumphost.png]]&lt;br /&gt;
*Click '''OK''' in the session settings to start connecting&amp;lt;br&amp;gt;You will be asked for your password&amp;lt;br&amp;gt;Enter your password on the console of hpcXX&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
== File Transfer ==&lt;br /&gt;
The best method for transfering files depends on where you want to store your files:&lt;br /&gt;
#Files in your home directory on HPC are only available within HPC&lt;br /&gt;
#Files on Network Shares are available on other locations (such as a TU Delft workplace, weblogin.tudelft.nl or other HPC clusters)&amp;lt;br&amp;gt;Copying files to these shares is best done using the manuals on [https://webdata.tudelft.nl/ webdata.tudelft.nl]&lt;br /&gt;
&lt;br /&gt;
When you want to copy files to the HPC you can copy them from Network Shares&lt;br /&gt;
=== (Windows) Network Shares ===&lt;br /&gt;
&lt;br /&gt;
==== Authenticate ====&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
 kinit&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
==== Basic access ====&lt;br /&gt;
&lt;br /&gt;
Now just type (depending on which share you want to use):&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
 cd /tudelft/$USER/student-group/&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
 cd /tudelft/$USER/staff-umbrella/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
You can access your Windows home directory as well. The subdirectories of the home directories are followed by '''the first letter of your surname''' and then your NetID.&amp;lt;br&amp;gt;&lt;br /&gt;
For example, if your name would be John Smith and your NetID jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
==== Important note ====&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your NetID. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your NetID yourself. If your NetID would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well. Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own NetID!&amp;lt;br&amp;gt;&lt;br /&gt;
'''If you use anthing other than your own NetID you'll create a recipe for potential disaster!'''&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths.&amp;lt;br&amp;gt;&lt;br /&gt;
So &amp;lt;code&amp;gt;/tudelft/jsmith/staff-group/&amp;lt;/code&amp;gt; only works if your NetID is jsmith and &amp;lt;code&amp;gt;/tudelft/pbrown/staff-group/&amp;lt;/code&amp;gt; only works works if your NetID is pbrown.&amp;lt;br&amp;gt;&lt;br /&gt;
If both NetIDs have the same privileges for accessing the group volumes, then these commands give access to the same data.&lt;br /&gt;
&lt;br /&gt;
=== Data on HPC ===&lt;br /&gt;
If you want to transfer data directly to the HPC you will need some extra configuration&lt;br /&gt;
==== Command line ====&lt;br /&gt;
Please first configure [[#SSH Proxy Support|SSH Proxy Support]].&amp;lt;br&amp;gt;&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your remote home directory on HPC to the local home directory on your desktop or laptop. &lt;br /&gt;
 scp &amp;lt;NetID&amp;gt;@hpcXX:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
*Copy an entire directory with all its content, use the '-r' option&amp;lt;br&amp;gt;This will copy the directory 'projects' from your remote home directory on HPC to the local home directory on your desktop or laptop:&lt;br /&gt;
 scp -r &amp;lt;NetID&amp;gt;@hpcXX:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your local home directory on your desktop or laptop to your remote home directory on HPC. &lt;br /&gt;
 scp projects.txt &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
*Copy the directory 'projects' and all its content from your current working directory on your desktop or laptop to your remote home directory on HPC.&lt;br /&gt;
 scp -r projects/ &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
==== WinSCP ====&lt;br /&gt;
WinSCP is a simple tool that has the internal option to create a tunnel through  the bastion server. It can be downloaded from [https://winscp.net/eng/download.php https://winscp.net/eng/download.php]&lt;br /&gt;
*Session &amp;gt; New Session&lt;br /&gt;
[[File:WinSCP01.png]]&lt;br /&gt;
*Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;Advanced&amp;quot; button&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Connection &amp;gt; Tunnel&amp;lt;br&amp;gt;Select &amp;quot;Connect through SSH tunnel&amp;quot;&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;OK&amp;quot; button&lt;br /&gt;
[[File:WinSCP03.png]]&lt;br /&gt;
*Click &amp;quot;Save&amp;quot;&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Save session as site&amp;lt;br&amp;gt;You can accept the suggested name&lt;br /&gt;
[[File:WinSCP04.png]]&lt;br /&gt;
*You can now &amp;quot;Login&amp;quot;&amp;lt;br&amp;gt;You will be asked to supply your password twice: first for the bastion, second for the HPC&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
==== MobaXterm ====&lt;br /&gt;
MobaXterm can also be used for file transfers. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*Left Window&amp;lt;br&amp;gt;Download files with '''arrow down'''&amp;lt;br&amp;gt;Upload files with '''arrow up'''&amp;lt;br&amp;gt;[[File:MobaXterm_SFTP.png]]&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
=== Windows ===&lt;br /&gt;
MobaXterm can also be used for displaying graphical applications. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*In the commandline windows (right)&amp;lt;br&amp;gt;Test X-forwarding (Graphics) by running &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt;&amp;lt;br&amp;gt;You should see these eyes (following the cursor)&amp;lt;br&amp;gt;[[File:MobaXterm_Xeyes.png]]&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in&amp;lt;br&amp;gt;&lt;br /&gt;
The easiest way is:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
When you have configured [[#SSH_Proxy_Support|SSH Proxy Support]] you can connect with:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=296</id>
		<title>Remote Access</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=296"/>
		<updated>2025-01-21T12:39:08Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: &lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;Access to the HPC-clusters is limited from outside the TU Delft network for security reasons.&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
'''You can only log in when you are on campus via a wireless (eduroam) or wired network. When you are off campus, you can log in using eduVPN (Institute Access). As of 16 Januari 2025 you can no longer use to the Linux bastion hosts when you are not on campus. There is also no need to use the Linux bastion hosts when you are on campus or using an active eduVPN connection. The information below is for the most part outdated. Please refer to the [[How to log in|Access]] page for log in instructions:''' [[How to log in|Access (How to log in and how to copy files)]]&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
== Bastion ==&lt;br /&gt;
''(This information is outdated and can be safely ignored.)''&lt;br /&gt;
&lt;br /&gt;
The TU Delft offers two bastion servers. These servers can be used to tunnel your traffic to the HPC.&lt;br /&gt;
*linux-bastion.tudelft.nl for employees or guests.&lt;br /&gt;
*student-linux.tudelft.nl for students.&lt;br /&gt;
&lt;br /&gt;
== Access to the commandline ==&lt;br /&gt;
The explanations below are the simplest way to connect to a console on an HPC. [[#Advanced configs|Advanced configs]] describes more complex configurations which save time when regularly using the connection or when you want to use [[#Linux|X-forwarding]]&lt;br /&gt;
=== Command line ===&lt;br /&gt;
When your desktop/laptop supports ssh from the console:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
If your SSH-client doesn't support Jump hosts, you can first connect to the bastion server and then to the HPC:&lt;br /&gt;
&lt;br /&gt;
 [user@home]$ ssh &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
 [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt; with your NetID (also remove the &amp;lt;&amp;gt;)&lt;br /&gt;
*When your a student replace &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl with &amp;lt;NetID&amp;gt;@student-linux.tudelft.nl&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net with the HPC you are connecting to.&lt;br /&gt;
&lt;br /&gt;
=== Advanced configs ===&lt;br /&gt;
==== Making OpenSSH more user-friendly ====&lt;br /&gt;
The OpenSSH clients can be configured using a configuration file in ~/.ssh/config (Linux, macOS) or ~\.ssh\config (Windows).&lt;br /&gt;
&lt;br /&gt;
===== Define connections =====&lt;br /&gt;
For the OpenSSH clients you can predefine often used connections by placing a section like the following in the configuration file on your local computer. Authentication on the bastion with keys is not possible, so password is used.&amp;lt;br&amp;gt;&lt;br /&gt;
Employees:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
Students:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
You can then connect simply by ssh bastion. For example:&lt;br /&gt;
  [user@home]$ ssh bastion&lt;br /&gt;
    NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
&lt;br /&gt;
===== SSH Proxy Support =====&lt;br /&gt;
Since this configuration depends on [[#Define connections|Define connections]] configure that first.&amp;lt;br&amp;gt;&lt;br /&gt;
To connect directly from your machine to an HPC cluster login node (without connecting to the bastion server first), create a connection via a proxy by adding the following lines to the configuration file ~/.ssh/config on your local computer:&lt;br /&gt;
&lt;br /&gt;
  Host hpcXX&lt;br /&gt;
  Hostname hpcXX.tudelft.net&lt;br /&gt;
  ProxyCommand ssh -W %h:%p bastion&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
&lt;br /&gt;
You can then simply use ssh hpcXX (or scp -p '&amp;lt;local file&amp;gt;' &amp;quot;hpcXX:/scratch/&amp;lt;NetID&amp;gt;/&amp;quot;) from your local computer.&lt;br /&gt;
&lt;br /&gt;
===== Session multiplexing =====&lt;br /&gt;
OpenSSH clients support multiple sessions over a single connection. This means you have to enter your password only once, to make the connection, and can then open multiple sessions (SSH, SCP and SFTP) to that computer over the same connection. This is especially convenient when connecting to the bastion server, where you have to type your password always.&lt;br /&gt;
&lt;br /&gt;
Note: SSH public key logins won't work (reliably, (because Kerberos authentication is required to access your home directory).&lt;br /&gt;
&lt;br /&gt;
Add the following section to the end of the configuration file on your local computer and on the bastion server:&lt;br /&gt;
&lt;br /&gt;
  Host *&lt;br /&gt;
  ControlMaster auto&lt;br /&gt;
  ControlPath /tmp/ssh-%r@%h:%p&lt;br /&gt;
&lt;br /&gt;
Windows users may need to adapt the ControlPath location to match Windows.&lt;br /&gt;
&lt;br /&gt;
You can use the exact same commands as above.&lt;br /&gt;
=== PuTTY ===&lt;br /&gt;
When your desktop/laptop doesn't support ssh from the console, you can use [https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html PuTTY] (it may already be installed).&amp;lt;br&amp;gt;&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students), and select the SSH protocol (port 22):&amp;lt;br&amp;gt;[[File:PuTTY-Hostname.png]]&lt;br /&gt;
*OPTIONAL: Connection&amp;gt;SSH Settings&amp;lt;br&amp;gt;Select Share SSH connections if possible (this means that you will not have to authenticate when you start a second session):&amp;lt;br&amp;gt;[[File:PuTTY-ShareConnection.png]]&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;To store these session settings, go back to the session tab, and save the settings under a session name. To activate a stored session, double-click on the session name:&amp;lt;br&amp;gt;[[File:PuTTY-SaveSession.png]]&lt;br /&gt;
Once you're connected to the bastion server you can use the locally available ssh to connect to a HPC:&lt;br /&gt;
   [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
     NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
     Welcome&lt;br /&gt;
   [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
=== MobaXterm ===&lt;br /&gt;
MobaXterm is a very versatile application. It can be used to connect to the commandline on HPC, display graphical applications and transfer files. It also has built-in support for using SSH-tunneling to connect through a bastion. You can download it from [https://mobaxterm.mobatek.net/ https://mobaxterm.mobatek.net/]. The '''free version''' has enough features for most users. The '''Portable edition''' can be run without installing (just unzip).&lt;br /&gt;
*Click the '''Session''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSession.png]]&lt;br /&gt;
*Click the '''SSH''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSSH.png]]&lt;br /&gt;
*Enter '''Remote host'''&amp;lt;br&amp;gt;Check '''Specify username'''&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;[[File:MobaXterm_Hostname.png]]&lt;br /&gt;
*tab: Network settings&amp;lt;br&amp;gt;Click '''SSH gateway (jump host)'''&amp;lt;br&amp;gt;[[File:MobaXterm_NewJumphost.png]]&lt;br /&gt;
*Enter linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;Click '''OK'''&amp;lt;br&amp;gt;[[File:MobaXterm_Jumphost.png]]&lt;br /&gt;
*Click '''OK''' in the session settings to start connecting&amp;lt;br&amp;gt;You will be asked for your password&amp;lt;br&amp;gt;Enter your password on the console of hpcXX&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
== File Transfer ==&lt;br /&gt;
The best method for transfering files depends on where you want to store your files:&lt;br /&gt;
#Files in your home directory on HPC are only available within HPC&lt;br /&gt;
#Files on Network Shares are available on other locations (such as a TU Delft workplace, weblogin.tudelft.nl or other HPC clusters)&amp;lt;br&amp;gt;Copying files to these shares is best done using the manuals on [https://webdata.tudelft.nl/ webdata.tudelft.nl]&lt;br /&gt;
&lt;br /&gt;
When you want to copy files to the HPC you can copy them from Network Shares&lt;br /&gt;
=== (Windows) Network Shares ===&lt;br /&gt;
&lt;br /&gt;
==== Authenticate ====&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
 kinit&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
==== Basic access ====&lt;br /&gt;
&lt;br /&gt;
Now just type (depending on which share you want to use):&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
 cd /tudelft/$USER/student-group/&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
 cd /tudelft/$USER/staff-umbrella/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
You can access your Windows home directory as well. The subdirectories of the home directories are followed by '''the first letter of your surname''' and then your NetID.&amp;lt;br&amp;gt;&lt;br /&gt;
For example, if your name would be John Smith and your NetID jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
==== Important note ====&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your NetID. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your NetID yourself. If your NetID would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well. Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own NetID!&amp;lt;br&amp;gt;&lt;br /&gt;
'''If you use anthing other than your own NetID you'll create a recipe for potential disaster!'''&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths.&amp;lt;br&amp;gt;&lt;br /&gt;
So &amp;lt;code&amp;gt;/tudelft/jsmith/staff-group/&amp;lt;/code&amp;gt; only works if your NetID is jsmith and &amp;lt;code&amp;gt;/tudelft/pbrown/staff-group/&amp;lt;/code&amp;gt; only works works if your NetID is pbrown.&amp;lt;br&amp;gt;&lt;br /&gt;
If both NetIDs have the same privileges for accessing the group volumes, then these commands give access to the same data.&lt;br /&gt;
&lt;br /&gt;
=== Data on HPC ===&lt;br /&gt;
If you want to transfer data directly to the HPC you will need some extra configuration&lt;br /&gt;
==== Command line ====&lt;br /&gt;
Please first configure [[#SSH Proxy Support|SSH Proxy Support]].&amp;lt;br&amp;gt;&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your remote home directory on HPC to the local home directory on your desktop or laptop. &lt;br /&gt;
 scp &amp;lt;NetID&amp;gt;@hpcXX:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
*Copy an entire directory with all its content, use the '-r' option&amp;lt;br&amp;gt;This will copy the directory 'projects' from your remote home directory on HPC to the local home directory on your desktop or laptop:&lt;br /&gt;
 scp -r &amp;lt;NetID&amp;gt;@hpcXX:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your local home directory on your desktop or laptop to your remote home directory on HPC. &lt;br /&gt;
 scp projects.txt &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
*Copy the directory 'projects' and all its content from your current working directory on your desktop or laptop to your remote home directory on HPC.&lt;br /&gt;
 scp -r projects/ &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
==== WinSCP ====&lt;br /&gt;
WinSCP is a simple tool that has the internal option to create a tunnel through  the bastion server. It can be downloaded from [https://winscp.net/eng/download.php https://winscp.net/eng/download.php]&lt;br /&gt;
*Session &amp;gt; New Session&lt;br /&gt;
[[File:WinSCP01.png]]&lt;br /&gt;
*Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;Advanced&amp;quot; button&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Connection &amp;gt; Tunnel&amp;lt;br&amp;gt;Select &amp;quot;Connect through SSH tunnel&amp;quot;&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;OK&amp;quot; button&lt;br /&gt;
[[File:WinSCP03.png]]&lt;br /&gt;
*Click &amp;quot;Save&amp;quot;&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Save session as site&amp;lt;br&amp;gt;You can accept the suggested name&lt;br /&gt;
[[File:WinSCP04.png]]&lt;br /&gt;
*You can now &amp;quot;Login&amp;quot;&amp;lt;br&amp;gt;You will be asked to supply your password twice: first for the bastion, second for the HPC&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
==== MobaXterm ====&lt;br /&gt;
MobaXterm can also be used for file transfers. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*Left Window&amp;lt;br&amp;gt;Download files with '''arrow down'''&amp;lt;br&amp;gt;Upload files with '''arrow up'''&amp;lt;br&amp;gt;[[File:MobaXterm_SFTP.png]]&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
=== Windows ===&lt;br /&gt;
MobaXterm can also be used for displaying graphical applications. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*In the commandline windows (right)&amp;lt;br&amp;gt;Test X-forwarding (Graphics) by running &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt;&amp;lt;br&amp;gt;You should see these eyes (following the cursor)&amp;lt;br&amp;gt;[[File:MobaXterm_Xeyes.png]]&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in&amp;lt;br&amp;gt;&lt;br /&gt;
The easiest way is:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
When you have configured [[#SSH_Proxy_Support|SSH Proxy Support]] you can connect with:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=295</id>
		<title>Remote Access</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=295"/>
		<updated>2025-01-21T12:38:40Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: &lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;Access to the HPC-clusters is limited from outside the TU Delft network for security reasons.&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
'''You can only log in when you are on campus via a wireless (eduroam) or wired network. When you are off campus, you can log in using eduVPN (Institute Access). As of 16 Januari you can no longer use to the Linux bastion hosts when you are not on campus. There is also no need to use the Linux bastion hosts when you are on campus or using an active eduVPN connection. The information below is for the most part outdated. Please refer to the [[How to log in|Access]] page for log in instructions:''' [[How to log in|Access (How to log in and how to copy files)]]&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
== Bastion ==&lt;br /&gt;
''(This information is outdated and can be safely ignored.)''&lt;br /&gt;
&lt;br /&gt;
The TU Delft offers two bastion servers. These servers can be used to tunnel your traffic to the HPC.&lt;br /&gt;
*linux-bastion.tudelft.nl for employees or guests.&lt;br /&gt;
*student-linux.tudelft.nl for students.&lt;br /&gt;
&lt;br /&gt;
== Access to the commandline ==&lt;br /&gt;
The explanations below are the simplest way to connect to a console on an HPC. [[#Advanced configs|Advanced configs]] describes more complex configurations which save time when regularly using the connection or when you want to use [[#Linux|X-forwarding]]&lt;br /&gt;
=== Command line ===&lt;br /&gt;
When your desktop/laptop supports ssh from the console:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
If your SSH-client doesn't support Jump hosts, you can first connect to the bastion server and then to the HPC:&lt;br /&gt;
&lt;br /&gt;
 [user@home]$ ssh &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
 [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt; with your NetID (also remove the &amp;lt;&amp;gt;)&lt;br /&gt;
*When your a student replace &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl with &amp;lt;NetID&amp;gt;@student-linux.tudelft.nl&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net with the HPC you are connecting to.&lt;br /&gt;
&lt;br /&gt;
=== Advanced configs ===&lt;br /&gt;
==== Making OpenSSH more user-friendly ====&lt;br /&gt;
The OpenSSH clients can be configured using a configuration file in ~/.ssh/config (Linux, macOS) or ~\.ssh\config (Windows).&lt;br /&gt;
&lt;br /&gt;
===== Define connections =====&lt;br /&gt;
For the OpenSSH clients you can predefine often used connections by placing a section like the following in the configuration file on your local computer. Authentication on the bastion with keys is not possible, so password is used.&amp;lt;br&amp;gt;&lt;br /&gt;
Employees:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
Students:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
You can then connect simply by ssh bastion. For example:&lt;br /&gt;
  [user@home]$ ssh bastion&lt;br /&gt;
    NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
&lt;br /&gt;
===== SSH Proxy Support =====&lt;br /&gt;
Since this configuration depends on [[#Define connections|Define connections]] configure that first.&amp;lt;br&amp;gt;&lt;br /&gt;
To connect directly from your machine to an HPC cluster login node (without connecting to the bastion server first), create a connection via a proxy by adding the following lines to the configuration file ~/.ssh/config on your local computer:&lt;br /&gt;
&lt;br /&gt;
  Host hpcXX&lt;br /&gt;
  Hostname hpcXX.tudelft.net&lt;br /&gt;
  ProxyCommand ssh -W %h:%p bastion&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
&lt;br /&gt;
You can then simply use ssh hpcXX (or scp -p '&amp;lt;local file&amp;gt;' &amp;quot;hpcXX:/scratch/&amp;lt;NetID&amp;gt;/&amp;quot;) from your local computer.&lt;br /&gt;
&lt;br /&gt;
===== Session multiplexing =====&lt;br /&gt;
OpenSSH clients support multiple sessions over a single connection. This means you have to enter your password only once, to make the connection, and can then open multiple sessions (SSH, SCP and SFTP) to that computer over the same connection. This is especially convenient when connecting to the bastion server, where you have to type your password always.&lt;br /&gt;
&lt;br /&gt;
Note: SSH public key logins won't work (reliably, (because Kerberos authentication is required to access your home directory).&lt;br /&gt;
&lt;br /&gt;
Add the following section to the end of the configuration file on your local computer and on the bastion server:&lt;br /&gt;
&lt;br /&gt;
  Host *&lt;br /&gt;
  ControlMaster auto&lt;br /&gt;
  ControlPath /tmp/ssh-%r@%h:%p&lt;br /&gt;
&lt;br /&gt;
Windows users may need to adapt the ControlPath location to match Windows.&lt;br /&gt;
&lt;br /&gt;
You can use the exact same commands as above.&lt;br /&gt;
=== PuTTY ===&lt;br /&gt;
When your desktop/laptop doesn't support ssh from the console, you can use [https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html PuTTY] (it may already be installed).&amp;lt;br&amp;gt;&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students), and select the SSH protocol (port 22):&amp;lt;br&amp;gt;[[File:PuTTY-Hostname.png]]&lt;br /&gt;
*OPTIONAL: Connection&amp;gt;SSH Settings&amp;lt;br&amp;gt;Select Share SSH connections if possible (this means that you will not have to authenticate when you start a second session):&amp;lt;br&amp;gt;[[File:PuTTY-ShareConnection.png]]&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;To store these session settings, go back to the session tab, and save the settings under a session name. To activate a stored session, double-click on the session name:&amp;lt;br&amp;gt;[[File:PuTTY-SaveSession.png]]&lt;br /&gt;
Once you're connected to the bastion server you can use the locally available ssh to connect to a HPC:&lt;br /&gt;
   [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
     NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
     Welcome&lt;br /&gt;
   [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
=== MobaXterm ===&lt;br /&gt;
MobaXterm is a very versatile application. It can be used to connect to the commandline on HPC, display graphical applications and transfer files. It also has built-in support for using SSH-tunneling to connect through a bastion. You can download it from [https://mobaxterm.mobatek.net/ https://mobaxterm.mobatek.net/]. The '''free version''' has enough features for most users. The '''Portable edition''' can be run without installing (just unzip).&lt;br /&gt;
*Click the '''Session''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSession.png]]&lt;br /&gt;
*Click the '''SSH''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSSH.png]]&lt;br /&gt;
*Enter '''Remote host'''&amp;lt;br&amp;gt;Check '''Specify username'''&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;[[File:MobaXterm_Hostname.png]]&lt;br /&gt;
*tab: Network settings&amp;lt;br&amp;gt;Click '''SSH gateway (jump host)'''&amp;lt;br&amp;gt;[[File:MobaXterm_NewJumphost.png]]&lt;br /&gt;
*Enter linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;Click '''OK'''&amp;lt;br&amp;gt;[[File:MobaXterm_Jumphost.png]]&lt;br /&gt;
*Click '''OK''' in the session settings to start connecting&amp;lt;br&amp;gt;You will be asked for your password&amp;lt;br&amp;gt;Enter your password on the console of hpcXX&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
== File Transfer ==&lt;br /&gt;
The best method for transfering files depends on where you want to store your files:&lt;br /&gt;
#Files in your home directory on HPC are only available within HPC&lt;br /&gt;
#Files on Network Shares are available on other locations (such as a TU Delft workplace, weblogin.tudelft.nl or other HPC clusters)&amp;lt;br&amp;gt;Copying files to these shares is best done using the manuals on [https://webdata.tudelft.nl/ webdata.tudelft.nl]&lt;br /&gt;
&lt;br /&gt;
When you want to copy files to the HPC you can copy them from Network Shares&lt;br /&gt;
=== (Windows) Network Shares ===&lt;br /&gt;
&lt;br /&gt;
==== Authenticate ====&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
 kinit&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
==== Basic access ====&lt;br /&gt;
&lt;br /&gt;
Now just type (depending on which share you want to use):&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
 cd /tudelft/$USER/student-group/&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
 cd /tudelft/$USER/staff-umbrella/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
You can access your Windows home directory as well. The subdirectories of the home directories are followed by '''the first letter of your surname''' and then your NetID.&amp;lt;br&amp;gt;&lt;br /&gt;
For example, if your name would be John Smith and your NetID jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
==== Important note ====&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your NetID. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your NetID yourself. If your NetID would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well. Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own NetID!&amp;lt;br&amp;gt;&lt;br /&gt;
'''If you use anthing other than your own NetID you'll create a recipe for potential disaster!'''&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths.&amp;lt;br&amp;gt;&lt;br /&gt;
So &amp;lt;code&amp;gt;/tudelft/jsmith/staff-group/&amp;lt;/code&amp;gt; only works if your NetID is jsmith and &amp;lt;code&amp;gt;/tudelft/pbrown/staff-group/&amp;lt;/code&amp;gt; only works works if your NetID is pbrown.&amp;lt;br&amp;gt;&lt;br /&gt;
If both NetIDs have the same privileges for accessing the group volumes, then these commands give access to the same data.&lt;br /&gt;
&lt;br /&gt;
=== Data on HPC ===&lt;br /&gt;
If you want to transfer data directly to the HPC you will need some extra configuration&lt;br /&gt;
==== Command line ====&lt;br /&gt;
Please first configure [[#SSH Proxy Support|SSH Proxy Support]].&amp;lt;br&amp;gt;&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your remote home directory on HPC to the local home directory on your desktop or laptop. &lt;br /&gt;
 scp &amp;lt;NetID&amp;gt;@hpcXX:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
*Copy an entire directory with all its content, use the '-r' option&amp;lt;br&amp;gt;This will copy the directory 'projects' from your remote home directory on HPC to the local home directory on your desktop or laptop:&lt;br /&gt;
 scp -r &amp;lt;NetID&amp;gt;@hpcXX:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your local home directory on your desktop or laptop to your remote home directory on HPC. &lt;br /&gt;
 scp projects.txt &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
*Copy the directory 'projects' and all its content from your current working directory on your desktop or laptop to your remote home directory on HPC.&lt;br /&gt;
 scp -r projects/ &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
==== WinSCP ====&lt;br /&gt;
WinSCP is a simple tool that has the internal option to create a tunnel through  the bastion server. It can be downloaded from [https://winscp.net/eng/download.php https://winscp.net/eng/download.php]&lt;br /&gt;
*Session &amp;gt; New Session&lt;br /&gt;
[[File:WinSCP01.png]]&lt;br /&gt;
*Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;Advanced&amp;quot; button&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Connection &amp;gt; Tunnel&amp;lt;br&amp;gt;Select &amp;quot;Connect through SSH tunnel&amp;quot;&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;OK&amp;quot; button&lt;br /&gt;
[[File:WinSCP03.png]]&lt;br /&gt;
*Click &amp;quot;Save&amp;quot;&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Save session as site&amp;lt;br&amp;gt;You can accept the suggested name&lt;br /&gt;
[[File:WinSCP04.png]]&lt;br /&gt;
*You can now &amp;quot;Login&amp;quot;&amp;lt;br&amp;gt;You will be asked to supply your password twice: first for the bastion, second for the HPC&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
==== MobaXterm ====&lt;br /&gt;
MobaXterm can also be used for file transfers. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*Left Window&amp;lt;br&amp;gt;Download files with '''arrow down'''&amp;lt;br&amp;gt;Upload files with '''arrow up'''&amp;lt;br&amp;gt;[[File:MobaXterm_SFTP.png]]&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
=== Windows ===&lt;br /&gt;
MobaXterm can also be used for displaying graphical applications. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*In the commandline windows (right)&amp;lt;br&amp;gt;Test X-forwarding (Graphics) by running &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt;&amp;lt;br&amp;gt;You should see these eyes (following the cursor)&amp;lt;br&amp;gt;[[File:MobaXterm_Xeyes.png]]&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in&amp;lt;br&amp;gt;&lt;br /&gt;
The easiest way is:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
When you have configured [[#SSH_Proxy_Support|SSH Proxy Support]] you can connect with:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=294</id>
		<title>Remote Access</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=294"/>
		<updated>2025-01-21T12:38:11Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: &lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;Access to the HPC-clusters is limited from outside the TU Delft network for security reasons.&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
'''You can only log in when you are on campus via a wireless (eduroam) or wired network. When you are off campus, you can log in using eduVPN (Institute Access). As of 16 Januari you can no longer use to the Linux bastion hosts when you are not on campus. There is also no need to use the Linux bastion hosts when you are on campus or using an active eduVPN connection. The information below is for the most part outdated. Please refer to the [[How to log in|Access]] page for log in instructions:''' [[How to log in|Access]] (How to log in and how to copy files) &lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
== Bastion ==&lt;br /&gt;
''(This information is outdated and can be safely ignored.)''&lt;br /&gt;
&lt;br /&gt;
The TU Delft offers two bastion servers. These servers can be used to tunnel your traffic to the HPC.&lt;br /&gt;
*linux-bastion.tudelft.nl for employees or guests.&lt;br /&gt;
*student-linux.tudelft.nl for students.&lt;br /&gt;
&lt;br /&gt;
== Access to the commandline ==&lt;br /&gt;
The explanations below are the simplest way to connect to a console on an HPC. [[#Advanced configs|Advanced configs]] describes more complex configurations which save time when regularly using the connection or when you want to use [[#Linux|X-forwarding]]&lt;br /&gt;
=== Command line ===&lt;br /&gt;
When your desktop/laptop supports ssh from the console:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
If your SSH-client doesn't support Jump hosts, you can first connect to the bastion server and then to the HPC:&lt;br /&gt;
&lt;br /&gt;
 [user@home]$ ssh &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
 [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt; with your NetID (also remove the &amp;lt;&amp;gt;)&lt;br /&gt;
*When your a student replace &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl with &amp;lt;NetID&amp;gt;@student-linux.tudelft.nl&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net with the HPC you are connecting to.&lt;br /&gt;
&lt;br /&gt;
=== Advanced configs ===&lt;br /&gt;
==== Making OpenSSH more user-friendly ====&lt;br /&gt;
The OpenSSH clients can be configured using a configuration file in ~/.ssh/config (Linux, macOS) or ~\.ssh\config (Windows).&lt;br /&gt;
&lt;br /&gt;
===== Define connections =====&lt;br /&gt;
For the OpenSSH clients you can predefine often used connections by placing a section like the following in the configuration file on your local computer. Authentication on the bastion with keys is not possible, so password is used.&amp;lt;br&amp;gt;&lt;br /&gt;
Employees:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
Students:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
You can then connect simply by ssh bastion. For example:&lt;br /&gt;
  [user@home]$ ssh bastion&lt;br /&gt;
    NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
&lt;br /&gt;
===== SSH Proxy Support =====&lt;br /&gt;
Since this configuration depends on [[#Define connections|Define connections]] configure that first.&amp;lt;br&amp;gt;&lt;br /&gt;
To connect directly from your machine to an HPC cluster login node (without connecting to the bastion server first), create a connection via a proxy by adding the following lines to the configuration file ~/.ssh/config on your local computer:&lt;br /&gt;
&lt;br /&gt;
  Host hpcXX&lt;br /&gt;
  Hostname hpcXX.tudelft.net&lt;br /&gt;
  ProxyCommand ssh -W %h:%p bastion&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
&lt;br /&gt;
You can then simply use ssh hpcXX (or scp -p '&amp;lt;local file&amp;gt;' &amp;quot;hpcXX:/scratch/&amp;lt;NetID&amp;gt;/&amp;quot;) from your local computer.&lt;br /&gt;
&lt;br /&gt;
===== Session multiplexing =====&lt;br /&gt;
OpenSSH clients support multiple sessions over a single connection. This means you have to enter your password only once, to make the connection, and can then open multiple sessions (SSH, SCP and SFTP) to that computer over the same connection. This is especially convenient when connecting to the bastion server, where you have to type your password always.&lt;br /&gt;
&lt;br /&gt;
Note: SSH public key logins won't work (reliably, (because Kerberos authentication is required to access your home directory).&lt;br /&gt;
&lt;br /&gt;
Add the following section to the end of the configuration file on your local computer and on the bastion server:&lt;br /&gt;
&lt;br /&gt;
  Host *&lt;br /&gt;
  ControlMaster auto&lt;br /&gt;
  ControlPath /tmp/ssh-%r@%h:%p&lt;br /&gt;
&lt;br /&gt;
Windows users may need to adapt the ControlPath location to match Windows.&lt;br /&gt;
&lt;br /&gt;
You can use the exact same commands as above.&lt;br /&gt;
=== PuTTY ===&lt;br /&gt;
When your desktop/laptop doesn't support ssh from the console, you can use [https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html PuTTY] (it may already be installed).&amp;lt;br&amp;gt;&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students), and select the SSH protocol (port 22):&amp;lt;br&amp;gt;[[File:PuTTY-Hostname.png]]&lt;br /&gt;
*OPTIONAL: Connection&amp;gt;SSH Settings&amp;lt;br&amp;gt;Select Share SSH connections if possible (this means that you will not have to authenticate when you start a second session):&amp;lt;br&amp;gt;[[File:PuTTY-ShareConnection.png]]&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;To store these session settings, go back to the session tab, and save the settings under a session name. To activate a stored session, double-click on the session name:&amp;lt;br&amp;gt;[[File:PuTTY-SaveSession.png]]&lt;br /&gt;
Once you're connected to the bastion server you can use the locally available ssh to connect to a HPC:&lt;br /&gt;
   [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
     NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
     Welcome&lt;br /&gt;
   [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
=== MobaXterm ===&lt;br /&gt;
MobaXterm is a very versatile application. It can be used to connect to the commandline on HPC, display graphical applications and transfer files. It also has built-in support for using SSH-tunneling to connect through a bastion. You can download it from [https://mobaxterm.mobatek.net/ https://mobaxterm.mobatek.net/]. The '''free version''' has enough features for most users. The '''Portable edition''' can be run without installing (just unzip).&lt;br /&gt;
*Click the '''Session''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSession.png]]&lt;br /&gt;
*Click the '''SSH''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSSH.png]]&lt;br /&gt;
*Enter '''Remote host'''&amp;lt;br&amp;gt;Check '''Specify username'''&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;[[File:MobaXterm_Hostname.png]]&lt;br /&gt;
*tab: Network settings&amp;lt;br&amp;gt;Click '''SSH gateway (jump host)'''&amp;lt;br&amp;gt;[[File:MobaXterm_NewJumphost.png]]&lt;br /&gt;
*Enter linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;Click '''OK'''&amp;lt;br&amp;gt;[[File:MobaXterm_Jumphost.png]]&lt;br /&gt;
*Click '''OK''' in the session settings to start connecting&amp;lt;br&amp;gt;You will be asked for your password&amp;lt;br&amp;gt;Enter your password on the console of hpcXX&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
== File Transfer ==&lt;br /&gt;
The best method for transfering files depends on where you want to store your files:&lt;br /&gt;
#Files in your home directory on HPC are only available within HPC&lt;br /&gt;
#Files on Network Shares are available on other locations (such as a TU Delft workplace, weblogin.tudelft.nl or other HPC clusters)&amp;lt;br&amp;gt;Copying files to these shares is best done using the manuals on [https://webdata.tudelft.nl/ webdata.tudelft.nl]&lt;br /&gt;
&lt;br /&gt;
When you want to copy files to the HPC you can copy them from Network Shares&lt;br /&gt;
=== (Windows) Network Shares ===&lt;br /&gt;
&lt;br /&gt;
==== Authenticate ====&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
 kinit&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
==== Basic access ====&lt;br /&gt;
&lt;br /&gt;
Now just type (depending on which share you want to use):&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
 cd /tudelft/$USER/student-group/&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
 cd /tudelft/$USER/staff-umbrella/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
You can access your Windows home directory as well. The subdirectories of the home directories are followed by '''the first letter of your surname''' and then your NetID.&amp;lt;br&amp;gt;&lt;br /&gt;
For example, if your name would be John Smith and your NetID jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
==== Important note ====&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your NetID. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your NetID yourself. If your NetID would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well. Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own NetID!&amp;lt;br&amp;gt;&lt;br /&gt;
'''If you use anthing other than your own NetID you'll create a recipe for potential disaster!'''&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths.&amp;lt;br&amp;gt;&lt;br /&gt;
So &amp;lt;code&amp;gt;/tudelft/jsmith/staff-group/&amp;lt;/code&amp;gt; only works if your NetID is jsmith and &amp;lt;code&amp;gt;/tudelft/pbrown/staff-group/&amp;lt;/code&amp;gt; only works works if your NetID is pbrown.&amp;lt;br&amp;gt;&lt;br /&gt;
If both NetIDs have the same privileges for accessing the group volumes, then these commands give access to the same data.&lt;br /&gt;
&lt;br /&gt;
=== Data on HPC ===&lt;br /&gt;
If you want to transfer data directly to the HPC you will need some extra configuration&lt;br /&gt;
==== Command line ====&lt;br /&gt;
Please first configure [[#SSH Proxy Support|SSH Proxy Support]].&amp;lt;br&amp;gt;&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your remote home directory on HPC to the local home directory on your desktop or laptop. &lt;br /&gt;
 scp &amp;lt;NetID&amp;gt;@hpcXX:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
*Copy an entire directory with all its content, use the '-r' option&amp;lt;br&amp;gt;This will copy the directory 'projects' from your remote home directory on HPC to the local home directory on your desktop or laptop:&lt;br /&gt;
 scp -r &amp;lt;NetID&amp;gt;@hpcXX:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your local home directory on your desktop or laptop to your remote home directory on HPC. &lt;br /&gt;
 scp projects.txt &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
*Copy the directory 'projects' and all its content from your current working directory on your desktop or laptop to your remote home directory on HPC.&lt;br /&gt;
 scp -r projects/ &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
==== WinSCP ====&lt;br /&gt;
WinSCP is a simple tool that has the internal option to create a tunnel through  the bastion server. It can be downloaded from [https://winscp.net/eng/download.php https://winscp.net/eng/download.php]&lt;br /&gt;
*Session &amp;gt; New Session&lt;br /&gt;
[[File:WinSCP01.png]]&lt;br /&gt;
*Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;Advanced&amp;quot; button&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Connection &amp;gt; Tunnel&amp;lt;br&amp;gt;Select &amp;quot;Connect through SSH tunnel&amp;quot;&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;OK&amp;quot; button&lt;br /&gt;
[[File:WinSCP03.png]]&lt;br /&gt;
*Click &amp;quot;Save&amp;quot;&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Save session as site&amp;lt;br&amp;gt;You can accept the suggested name&lt;br /&gt;
[[File:WinSCP04.png]]&lt;br /&gt;
*You can now &amp;quot;Login&amp;quot;&amp;lt;br&amp;gt;You will be asked to supply your password twice: first for the bastion, second for the HPC&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
==== MobaXterm ====&lt;br /&gt;
MobaXterm can also be used for file transfers. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*Left Window&amp;lt;br&amp;gt;Download files with '''arrow down'''&amp;lt;br&amp;gt;Upload files with '''arrow up'''&amp;lt;br&amp;gt;[[File:MobaXterm_SFTP.png]]&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
=== Windows ===&lt;br /&gt;
MobaXterm can also be used for displaying graphical applications. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*In the commandline windows (right)&amp;lt;br&amp;gt;Test X-forwarding (Graphics) by running &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt;&amp;lt;br&amp;gt;You should see these eyes (following the cursor)&amp;lt;br&amp;gt;[[File:MobaXterm_Xeyes.png]]&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in&amp;lt;br&amp;gt;&lt;br /&gt;
The easiest way is:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
When you have configured [[#SSH_Proxy_Support|SSH Proxy Support]] you can connect with:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=293</id>
		<title>Remote Access</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=293"/>
		<updated>2025-01-21T12:37:49Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: &lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;Access to the HPC-clusters is limited from outside the TU Delft network for security reasons.&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
'''You can only log in when you are on campus via a wireless (eduroam) or wired network. When you are off campus, you can log in using eduVPN (Institute Access). As of 16 Januari you can no longer use to the Linux bastion hosts when you are not on campus. There is also no need to use the Linux bastion hosts when you are on campus or using an active eduVPN connection. The information below is for the most part outdated. Please refer to the [[How to log in|Access]] page for log in instructions:''' [[How to log in|Access]]&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
== Bastion ==&lt;br /&gt;
''(This information is outdated and can be safely ignored.)''&lt;br /&gt;
&lt;br /&gt;
The TU Delft offers two bastion servers. These servers can be used to tunnel your traffic to the HPC.&lt;br /&gt;
*linux-bastion.tudelft.nl for employees or guests.&lt;br /&gt;
*student-linux.tudelft.nl for students.&lt;br /&gt;
&lt;br /&gt;
== Access to the commandline ==&lt;br /&gt;
The explanations below are the simplest way to connect to a console on an HPC. [[#Advanced configs|Advanced configs]] describes more complex configurations which save time when regularly using the connection or when you want to use [[#Linux|X-forwarding]]&lt;br /&gt;
=== Command line ===&lt;br /&gt;
When your desktop/laptop supports ssh from the console:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
If your SSH-client doesn't support Jump hosts, you can first connect to the bastion server and then to the HPC:&lt;br /&gt;
&lt;br /&gt;
 [user@home]$ ssh &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
 [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt; with your NetID (also remove the &amp;lt;&amp;gt;)&lt;br /&gt;
*When your a student replace &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl with &amp;lt;NetID&amp;gt;@student-linux.tudelft.nl&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net with the HPC you are connecting to.&lt;br /&gt;
&lt;br /&gt;
=== Advanced configs ===&lt;br /&gt;
==== Making OpenSSH more user-friendly ====&lt;br /&gt;
The OpenSSH clients can be configured using a configuration file in ~/.ssh/config (Linux, macOS) or ~\.ssh\config (Windows).&lt;br /&gt;
&lt;br /&gt;
===== Define connections =====&lt;br /&gt;
For the OpenSSH clients you can predefine often used connections by placing a section like the following in the configuration file on your local computer. Authentication on the bastion with keys is not possible, so password is used.&amp;lt;br&amp;gt;&lt;br /&gt;
Employees:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
Students:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
You can then connect simply by ssh bastion. For example:&lt;br /&gt;
  [user@home]$ ssh bastion&lt;br /&gt;
    NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
&lt;br /&gt;
===== SSH Proxy Support =====&lt;br /&gt;
Since this configuration depends on [[#Define connections|Define connections]] configure that first.&amp;lt;br&amp;gt;&lt;br /&gt;
To connect directly from your machine to an HPC cluster login node (without connecting to the bastion server first), create a connection via a proxy by adding the following lines to the configuration file ~/.ssh/config on your local computer:&lt;br /&gt;
&lt;br /&gt;
  Host hpcXX&lt;br /&gt;
  Hostname hpcXX.tudelft.net&lt;br /&gt;
  ProxyCommand ssh -W %h:%p bastion&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
&lt;br /&gt;
You can then simply use ssh hpcXX (or scp -p '&amp;lt;local file&amp;gt;' &amp;quot;hpcXX:/scratch/&amp;lt;NetID&amp;gt;/&amp;quot;) from your local computer.&lt;br /&gt;
&lt;br /&gt;
===== Session multiplexing =====&lt;br /&gt;
OpenSSH clients support multiple sessions over a single connection. This means you have to enter your password only once, to make the connection, and can then open multiple sessions (SSH, SCP and SFTP) to that computer over the same connection. This is especially convenient when connecting to the bastion server, where you have to type your password always.&lt;br /&gt;
&lt;br /&gt;
Note: SSH public key logins won't work (reliably, (because Kerberos authentication is required to access your home directory).&lt;br /&gt;
&lt;br /&gt;
Add the following section to the end of the configuration file on your local computer and on the bastion server:&lt;br /&gt;
&lt;br /&gt;
  Host *&lt;br /&gt;
  ControlMaster auto&lt;br /&gt;
  ControlPath /tmp/ssh-%r@%h:%p&lt;br /&gt;
&lt;br /&gt;
Windows users may need to adapt the ControlPath location to match Windows.&lt;br /&gt;
&lt;br /&gt;
You can use the exact same commands as above.&lt;br /&gt;
=== PuTTY ===&lt;br /&gt;
When your desktop/laptop doesn't support ssh from the console, you can use [https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html PuTTY] (it may already be installed).&amp;lt;br&amp;gt;&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students), and select the SSH protocol (port 22):&amp;lt;br&amp;gt;[[File:PuTTY-Hostname.png]]&lt;br /&gt;
*OPTIONAL: Connection&amp;gt;SSH Settings&amp;lt;br&amp;gt;Select Share SSH connections if possible (this means that you will not have to authenticate when you start a second session):&amp;lt;br&amp;gt;[[File:PuTTY-ShareConnection.png]]&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;To store these session settings, go back to the session tab, and save the settings under a session name. To activate a stored session, double-click on the session name:&amp;lt;br&amp;gt;[[File:PuTTY-SaveSession.png]]&lt;br /&gt;
Once you're connected to the bastion server you can use the locally available ssh to connect to a HPC:&lt;br /&gt;
   [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
     NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
     Welcome&lt;br /&gt;
   [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
=== MobaXterm ===&lt;br /&gt;
MobaXterm is a very versatile application. It can be used to connect to the commandline on HPC, display graphical applications and transfer files. It also has built-in support for using SSH-tunneling to connect through a bastion. You can download it from [https://mobaxterm.mobatek.net/ https://mobaxterm.mobatek.net/]. The '''free version''' has enough features for most users. The '''Portable edition''' can be run without installing (just unzip).&lt;br /&gt;
*Click the '''Session''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSession.png]]&lt;br /&gt;
*Click the '''SSH''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSSH.png]]&lt;br /&gt;
*Enter '''Remote host'''&amp;lt;br&amp;gt;Check '''Specify username'''&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;[[File:MobaXterm_Hostname.png]]&lt;br /&gt;
*tab: Network settings&amp;lt;br&amp;gt;Click '''SSH gateway (jump host)'''&amp;lt;br&amp;gt;[[File:MobaXterm_NewJumphost.png]]&lt;br /&gt;
*Enter linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;Click '''OK'''&amp;lt;br&amp;gt;[[File:MobaXterm_Jumphost.png]]&lt;br /&gt;
*Click '''OK''' in the session settings to start connecting&amp;lt;br&amp;gt;You will be asked for your password&amp;lt;br&amp;gt;Enter your password on the console of hpcXX&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
== File Transfer ==&lt;br /&gt;
The best method for transfering files depends on where you want to store your files:&lt;br /&gt;
#Files in your home directory on HPC are only available within HPC&lt;br /&gt;
#Files on Network Shares are available on other locations (such as a TU Delft workplace, weblogin.tudelft.nl or other HPC clusters)&amp;lt;br&amp;gt;Copying files to these shares is best done using the manuals on [https://webdata.tudelft.nl/ webdata.tudelft.nl]&lt;br /&gt;
&lt;br /&gt;
When you want to copy files to the HPC you can copy them from Network Shares&lt;br /&gt;
=== (Windows) Network Shares ===&lt;br /&gt;
&lt;br /&gt;
==== Authenticate ====&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
 kinit&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
==== Basic access ====&lt;br /&gt;
&lt;br /&gt;
Now just type (depending on which share you want to use):&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
 cd /tudelft/$USER/student-group/&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
 cd /tudelft/$USER/staff-umbrella/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
You can access your Windows home directory as well. The subdirectories of the home directories are followed by '''the first letter of your surname''' and then your NetID.&amp;lt;br&amp;gt;&lt;br /&gt;
For example, if your name would be John Smith and your NetID jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
==== Important note ====&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your NetID. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your NetID yourself. If your NetID would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well. Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own NetID!&amp;lt;br&amp;gt;&lt;br /&gt;
'''If you use anthing other than your own NetID you'll create a recipe for potential disaster!'''&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths.&amp;lt;br&amp;gt;&lt;br /&gt;
So &amp;lt;code&amp;gt;/tudelft/jsmith/staff-group/&amp;lt;/code&amp;gt; only works if your NetID is jsmith and &amp;lt;code&amp;gt;/tudelft/pbrown/staff-group/&amp;lt;/code&amp;gt; only works works if your NetID is pbrown.&amp;lt;br&amp;gt;&lt;br /&gt;
If both NetIDs have the same privileges for accessing the group volumes, then these commands give access to the same data.&lt;br /&gt;
&lt;br /&gt;
=== Data on HPC ===&lt;br /&gt;
If you want to transfer data directly to the HPC you will need some extra configuration&lt;br /&gt;
==== Command line ====&lt;br /&gt;
Please first configure [[#SSH Proxy Support|SSH Proxy Support]].&amp;lt;br&amp;gt;&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your remote home directory on HPC to the local home directory on your desktop or laptop. &lt;br /&gt;
 scp &amp;lt;NetID&amp;gt;@hpcXX:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
*Copy an entire directory with all its content, use the '-r' option&amp;lt;br&amp;gt;This will copy the directory 'projects' from your remote home directory on HPC to the local home directory on your desktop or laptop:&lt;br /&gt;
 scp -r &amp;lt;NetID&amp;gt;@hpcXX:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your local home directory on your desktop or laptop to your remote home directory on HPC. &lt;br /&gt;
 scp projects.txt &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
*Copy the directory 'projects' and all its content from your current working directory on your desktop or laptop to your remote home directory on HPC.&lt;br /&gt;
 scp -r projects/ &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
==== WinSCP ====&lt;br /&gt;
WinSCP is a simple tool that has the internal option to create a tunnel through  the bastion server. It can be downloaded from [https://winscp.net/eng/download.php https://winscp.net/eng/download.php]&lt;br /&gt;
*Session &amp;gt; New Session&lt;br /&gt;
[[File:WinSCP01.png]]&lt;br /&gt;
*Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;Advanced&amp;quot; button&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Connection &amp;gt; Tunnel&amp;lt;br&amp;gt;Select &amp;quot;Connect through SSH tunnel&amp;quot;&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;OK&amp;quot; button&lt;br /&gt;
[[File:WinSCP03.png]]&lt;br /&gt;
*Click &amp;quot;Save&amp;quot;&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Save session as site&amp;lt;br&amp;gt;You can accept the suggested name&lt;br /&gt;
[[File:WinSCP04.png]]&lt;br /&gt;
*You can now &amp;quot;Login&amp;quot;&amp;lt;br&amp;gt;You will be asked to supply your password twice: first for the bastion, second for the HPC&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
==== MobaXterm ====&lt;br /&gt;
MobaXterm can also be used for file transfers. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*Left Window&amp;lt;br&amp;gt;Download files with '''arrow down'''&amp;lt;br&amp;gt;Upload files with '''arrow up'''&amp;lt;br&amp;gt;[[File:MobaXterm_SFTP.png]]&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
=== Windows ===&lt;br /&gt;
MobaXterm can also be used for displaying graphical applications. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*In the commandline windows (right)&amp;lt;br&amp;gt;Test X-forwarding (Graphics) by running &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt;&amp;lt;br&amp;gt;You should see these eyes (following the cursor)&amp;lt;br&amp;gt;[[File:MobaXterm_Xeyes.png]]&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in&amp;lt;br&amp;gt;&lt;br /&gt;
The easiest way is:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
When you have configured [[#SSH_Proxy_Support|SSH Proxy Support]] you can connect with:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=292</id>
		<title>Remote Access</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=292"/>
		<updated>2025-01-21T12:35:11Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: &lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;Access to the HPC-clusters is limited from outside the TU Delft network for security reasons.&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
'''You can only log in when you are on campus via a wireless (eduroam) or wired network. When you are off campus, you can log in using eduVPN (Institute Access). As of 16 Januari you can no longer use to the Linux bastion hosts when you are not on campus. There is also no need to use the Linux bastion hosts when you are on campus or using an active eduVPN connection. The information below is for the most part outdated.'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
== Bastion ==&lt;br /&gt;
''(This information is outdated and can be safely ignored.)''&lt;br /&gt;
&lt;br /&gt;
The TU Delft offers two bastion servers. These servers can be used to tunnel your traffic to the HPC.&lt;br /&gt;
*linux-bastion.tudelft.nl for employees or guests.&lt;br /&gt;
*student-linux.tudelft.nl for students.&lt;br /&gt;
&lt;br /&gt;
== Access to the commandline ==&lt;br /&gt;
The explanations below are the simplest way to connect to a console on an HPC. [[#Advanced configs|Advanced configs]] describes more complex configurations which save time when regularly using the connection or when you want to use [[#Linux|X-forwarding]]&lt;br /&gt;
=== Command line ===&lt;br /&gt;
When your desktop/laptop supports ssh from the console:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
If your SSH-client doesn't support Jump hosts, you can first connect to the bastion server and then to the HPC:&lt;br /&gt;
&lt;br /&gt;
 [user@home]$ ssh &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
 [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt; with your NetID (also remove the &amp;lt;&amp;gt;)&lt;br /&gt;
*When your a student replace &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl with &amp;lt;NetID&amp;gt;@student-linux.tudelft.nl&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net with the HPC you are connecting to.&lt;br /&gt;
&lt;br /&gt;
=== Advanced configs ===&lt;br /&gt;
==== Making OpenSSH more user-friendly ====&lt;br /&gt;
The OpenSSH clients can be configured using a configuration file in ~/.ssh/config (Linux, macOS) or ~\.ssh\config (Windows).&lt;br /&gt;
&lt;br /&gt;
===== Define connections =====&lt;br /&gt;
For the OpenSSH clients you can predefine often used connections by placing a section like the following in the configuration file on your local computer. Authentication on the bastion with keys is not possible, so password is used.&amp;lt;br&amp;gt;&lt;br /&gt;
Employees:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
Students:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
You can then connect simply by ssh bastion. For example:&lt;br /&gt;
  [user@home]$ ssh bastion&lt;br /&gt;
    NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
&lt;br /&gt;
===== SSH Proxy Support =====&lt;br /&gt;
Since this configuration depends on [[#Define connections|Define connections]] configure that first.&amp;lt;br&amp;gt;&lt;br /&gt;
To connect directly from your machine to an HPC cluster login node (without connecting to the bastion server first), create a connection via a proxy by adding the following lines to the configuration file ~/.ssh/config on your local computer:&lt;br /&gt;
&lt;br /&gt;
  Host hpcXX&lt;br /&gt;
  Hostname hpcXX.tudelft.net&lt;br /&gt;
  ProxyCommand ssh -W %h:%p bastion&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
&lt;br /&gt;
You can then simply use ssh hpcXX (or scp -p '&amp;lt;local file&amp;gt;' &amp;quot;hpcXX:/scratch/&amp;lt;NetID&amp;gt;/&amp;quot;) from your local computer.&lt;br /&gt;
&lt;br /&gt;
===== Session multiplexing =====&lt;br /&gt;
OpenSSH clients support multiple sessions over a single connection. This means you have to enter your password only once, to make the connection, and can then open multiple sessions (SSH, SCP and SFTP) to that computer over the same connection. This is especially convenient when connecting to the bastion server, where you have to type your password always.&lt;br /&gt;
&lt;br /&gt;
Note: SSH public key logins won't work (reliably, (because Kerberos authentication is required to access your home directory).&lt;br /&gt;
&lt;br /&gt;
Add the following section to the end of the configuration file on your local computer and on the bastion server:&lt;br /&gt;
&lt;br /&gt;
  Host *&lt;br /&gt;
  ControlMaster auto&lt;br /&gt;
  ControlPath /tmp/ssh-%r@%h:%p&lt;br /&gt;
&lt;br /&gt;
Windows users may need to adapt the ControlPath location to match Windows.&lt;br /&gt;
&lt;br /&gt;
You can use the exact same commands as above.&lt;br /&gt;
=== PuTTY ===&lt;br /&gt;
When your desktop/laptop doesn't support ssh from the console, you can use [https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html PuTTY] (it may already be installed).&amp;lt;br&amp;gt;&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students), and select the SSH protocol (port 22):&amp;lt;br&amp;gt;[[File:PuTTY-Hostname.png]]&lt;br /&gt;
*OPTIONAL: Connection&amp;gt;SSH Settings&amp;lt;br&amp;gt;Select Share SSH connections if possible (this means that you will not have to authenticate when you start a second session):&amp;lt;br&amp;gt;[[File:PuTTY-ShareConnection.png]]&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;To store these session settings, go back to the session tab, and save the settings under a session name. To activate a stored session, double-click on the session name:&amp;lt;br&amp;gt;[[File:PuTTY-SaveSession.png]]&lt;br /&gt;
Once you're connected to the bastion server you can use the locally available ssh to connect to a HPC:&lt;br /&gt;
   [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
     NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
     Welcome&lt;br /&gt;
   [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
=== MobaXterm ===&lt;br /&gt;
MobaXterm is a very versatile application. It can be used to connect to the commandline on HPC, display graphical applications and transfer files. It also has built-in support for using SSH-tunneling to connect through a bastion. You can download it from [https://mobaxterm.mobatek.net/ https://mobaxterm.mobatek.net/]. The '''free version''' has enough features for most users. The '''Portable edition''' can be run without installing (just unzip).&lt;br /&gt;
*Click the '''Session''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSession.png]]&lt;br /&gt;
*Click the '''SSH''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSSH.png]]&lt;br /&gt;
*Enter '''Remote host'''&amp;lt;br&amp;gt;Check '''Specify username'''&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;[[File:MobaXterm_Hostname.png]]&lt;br /&gt;
*tab: Network settings&amp;lt;br&amp;gt;Click '''SSH gateway (jump host)'''&amp;lt;br&amp;gt;[[File:MobaXterm_NewJumphost.png]]&lt;br /&gt;
*Enter linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;Click '''OK'''&amp;lt;br&amp;gt;[[File:MobaXterm_Jumphost.png]]&lt;br /&gt;
*Click '''OK''' in the session settings to start connecting&amp;lt;br&amp;gt;You will be asked for your password&amp;lt;br&amp;gt;Enter your password on the console of hpcXX&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
== File Transfer ==&lt;br /&gt;
The best method for transfering files depends on where you want to store your files:&lt;br /&gt;
#Files in your home directory on HPC are only available within HPC&lt;br /&gt;
#Files on Network Shares are available on other locations (such as a TU Delft workplace, weblogin.tudelft.nl or other HPC clusters)&amp;lt;br&amp;gt;Copying files to these shares is best done using the manuals on [https://webdata.tudelft.nl/ webdata.tudelft.nl]&lt;br /&gt;
&lt;br /&gt;
When you want to copy files to the HPC you can copy them from Network Shares&lt;br /&gt;
=== (Windows) Network Shares ===&lt;br /&gt;
&lt;br /&gt;
==== Authenticate ====&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
 kinit&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
==== Basic access ====&lt;br /&gt;
&lt;br /&gt;
Now just type (depending on which share you want to use):&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
 cd /tudelft/$USER/student-group/&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
 cd /tudelft/$USER/staff-umbrella/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
You can access your Windows home directory as well. The subdirectories of the home directories are followed by '''the first letter of your surname''' and then your NetID.&amp;lt;br&amp;gt;&lt;br /&gt;
For example, if your name would be John Smith and your NetID jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
==== Important note ====&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your NetID. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your NetID yourself. If your NetID would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well. Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own NetID!&amp;lt;br&amp;gt;&lt;br /&gt;
'''If you use anthing other than your own NetID you'll create a recipe for potential disaster!'''&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths.&amp;lt;br&amp;gt;&lt;br /&gt;
So &amp;lt;code&amp;gt;/tudelft/jsmith/staff-group/&amp;lt;/code&amp;gt; only works if your NetID is jsmith and &amp;lt;code&amp;gt;/tudelft/pbrown/staff-group/&amp;lt;/code&amp;gt; only works works if your NetID is pbrown.&amp;lt;br&amp;gt;&lt;br /&gt;
If both NetIDs have the same privileges for accessing the group volumes, then these commands give access to the same data.&lt;br /&gt;
&lt;br /&gt;
=== Data on HPC ===&lt;br /&gt;
If you want to transfer data directly to the HPC you will need some extra configuration&lt;br /&gt;
==== Command line ====&lt;br /&gt;
Please first configure [[#SSH Proxy Support|SSH Proxy Support]].&amp;lt;br&amp;gt;&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your remote home directory on HPC to the local home directory on your desktop or laptop. &lt;br /&gt;
 scp &amp;lt;NetID&amp;gt;@hpcXX:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
*Copy an entire directory with all its content, use the '-r' option&amp;lt;br&amp;gt;This will copy the directory 'projects' from your remote home directory on HPC to the local home directory on your desktop or laptop:&lt;br /&gt;
 scp -r &amp;lt;NetID&amp;gt;@hpcXX:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your local home directory on your desktop or laptop to your remote home directory on HPC. &lt;br /&gt;
 scp projects.txt &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
*Copy the directory 'projects' and all its content from your current working directory on your desktop or laptop to your remote home directory on HPC.&lt;br /&gt;
 scp -r projects/ &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
==== WinSCP ====&lt;br /&gt;
WinSCP is a simple tool that has the internal option to create a tunnel through  the bastion server. It can be downloaded from [https://winscp.net/eng/download.php https://winscp.net/eng/download.php]&lt;br /&gt;
*Session &amp;gt; New Session&lt;br /&gt;
[[File:WinSCP01.png]]&lt;br /&gt;
*Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;Advanced&amp;quot; button&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Connection &amp;gt; Tunnel&amp;lt;br&amp;gt;Select &amp;quot;Connect through SSH tunnel&amp;quot;&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;OK&amp;quot; button&lt;br /&gt;
[[File:WinSCP03.png]]&lt;br /&gt;
*Click &amp;quot;Save&amp;quot;&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Save session as site&amp;lt;br&amp;gt;You can accept the suggested name&lt;br /&gt;
[[File:WinSCP04.png]]&lt;br /&gt;
*You can now &amp;quot;Login&amp;quot;&amp;lt;br&amp;gt;You will be asked to supply your password twice: first for the bastion, second for the HPC&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
==== MobaXterm ====&lt;br /&gt;
MobaXterm can also be used for file transfers. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*Left Window&amp;lt;br&amp;gt;Download files with '''arrow down'''&amp;lt;br&amp;gt;Upload files with '''arrow up'''&amp;lt;br&amp;gt;[[File:MobaXterm_SFTP.png]]&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
=== Windows ===&lt;br /&gt;
MobaXterm can also be used for displaying graphical applications. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*In the commandline windows (right)&amp;lt;br&amp;gt;Test X-forwarding (Graphics) by running &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt;&amp;lt;br&amp;gt;You should see these eyes (following the cursor)&amp;lt;br&amp;gt;[[File:MobaXterm_Xeyes.png]]&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in&amp;lt;br&amp;gt;&lt;br /&gt;
The easiest way is:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
When you have configured [[#SSH_Proxy_Support|SSH Proxy Support]] you can connect with:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=291</id>
		<title>Remote Access</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=291"/>
		<updated>2025-01-20T15:14:24Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* Bastion */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;Access to the HPC-clusters is limited from outside the TU Delft network for security reasons.&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
'''You can only log in when you are on campus via a wireless (eduroam) or wired network. When you are off campus, you can log in using eduVPN (Institute Access). As of 16 Januari you can no longer use to the Linux bastion hosts when you are not on campus. There is also no need to use the Linux bastion hosts when you are on campus or using an active eduVPN connection.'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
== Bastion ==&lt;br /&gt;
''(This information is outdated and can be safely ignored.)''&lt;br /&gt;
&lt;br /&gt;
The TU Delft offers two bastion servers. These servers can be used to tunnel your traffic to the HPC.&lt;br /&gt;
*linux-bastion.tudelft.nl for employees or guests.&lt;br /&gt;
*student-linux.tudelft.nl for students.&lt;br /&gt;
&lt;br /&gt;
== Access to the commandline ==&lt;br /&gt;
The explanations below are the simplest way to connect to a console on an HPC. [[#Advanced configs|Advanced configs]] describes more complex configurations which save time when regularly using the connection or when you want to use [[#Linux|X-forwarding]]&lt;br /&gt;
=== Command line ===&lt;br /&gt;
When your desktop/laptop supports ssh from the console:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
If your SSH-client doesn't support Jump hosts, you can first connect to the bastion server and then to the HPC:&lt;br /&gt;
&lt;br /&gt;
 [user@home]$ ssh &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
 [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt; with your NetID (also remove the &amp;lt;&amp;gt;)&lt;br /&gt;
*When your a student replace &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl with &amp;lt;NetID&amp;gt;@student-linux.tudelft.nl&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net with the HPC you are connecting to.&lt;br /&gt;
&lt;br /&gt;
=== Advanced configs ===&lt;br /&gt;
==== Making OpenSSH more user-friendly ====&lt;br /&gt;
The OpenSSH clients can be configured using a configuration file in ~/.ssh/config (Linux, macOS) or ~\.ssh\config (Windows).&lt;br /&gt;
&lt;br /&gt;
===== Define connections =====&lt;br /&gt;
For the OpenSSH clients you can predefine often used connections by placing a section like the following in the configuration file on your local computer. Authentication on the bastion with keys is not possible, so password is used.&amp;lt;br&amp;gt;&lt;br /&gt;
Employees:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
Students:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
You can then connect simply by ssh bastion. For example:&lt;br /&gt;
  [user@home]$ ssh bastion&lt;br /&gt;
    NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
&lt;br /&gt;
===== SSH Proxy Support =====&lt;br /&gt;
Since this configuration depends on [[#Define connections|Define connections]] configure that first.&amp;lt;br&amp;gt;&lt;br /&gt;
To connect directly from your machine to an HPC cluster login node (without connecting to the bastion server first), create a connection via a proxy by adding the following lines to the configuration file ~/.ssh/config on your local computer:&lt;br /&gt;
&lt;br /&gt;
  Host hpcXX&lt;br /&gt;
  Hostname hpcXX.tudelft.net&lt;br /&gt;
  ProxyCommand ssh -W %h:%p bastion&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
&lt;br /&gt;
You can then simply use ssh hpcXX (or scp -p '&amp;lt;local file&amp;gt;' &amp;quot;hpcXX:/scratch/&amp;lt;NetID&amp;gt;/&amp;quot;) from your local computer.&lt;br /&gt;
&lt;br /&gt;
===== Session multiplexing =====&lt;br /&gt;
OpenSSH clients support multiple sessions over a single connection. This means you have to enter your password only once, to make the connection, and can then open multiple sessions (SSH, SCP and SFTP) to that computer over the same connection. This is especially convenient when connecting to the bastion server, where you have to type your password always.&lt;br /&gt;
&lt;br /&gt;
Note: SSH public key logins won't work (reliably, (because Kerberos authentication is required to access your home directory).&lt;br /&gt;
&lt;br /&gt;
Add the following section to the end of the configuration file on your local computer and on the bastion server:&lt;br /&gt;
&lt;br /&gt;
  Host *&lt;br /&gt;
  ControlMaster auto&lt;br /&gt;
  ControlPath /tmp/ssh-%r@%h:%p&lt;br /&gt;
&lt;br /&gt;
Windows users may need to adapt the ControlPath location to match Windows.&lt;br /&gt;
&lt;br /&gt;
You can use the exact same commands as above.&lt;br /&gt;
=== PuTTY ===&lt;br /&gt;
When your desktop/laptop doesn't support ssh from the console, you can use [https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html PuTTY] (it may already be installed).&amp;lt;br&amp;gt;&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students), and select the SSH protocol (port 22):&amp;lt;br&amp;gt;[[File:PuTTY-Hostname.png]]&lt;br /&gt;
*OPTIONAL: Connection&amp;gt;SSH Settings&amp;lt;br&amp;gt;Select Share SSH connections if possible (this means that you will not have to authenticate when you start a second session):&amp;lt;br&amp;gt;[[File:PuTTY-ShareConnection.png]]&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;To store these session settings, go back to the session tab, and save the settings under a session name. To activate a stored session, double-click on the session name:&amp;lt;br&amp;gt;[[File:PuTTY-SaveSession.png]]&lt;br /&gt;
Once you're connected to the bastion server you can use the locally available ssh to connect to a HPC:&lt;br /&gt;
   [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
     NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
     Welcome&lt;br /&gt;
   [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
=== MobaXterm ===&lt;br /&gt;
MobaXterm is a very versatile application. It can be used to connect to the commandline on HPC, display graphical applications and transfer files. It also has built-in support for using SSH-tunneling to connect through a bastion. You can download it from [https://mobaxterm.mobatek.net/ https://mobaxterm.mobatek.net/]. The '''free version''' has enough features for most users. The '''Portable edition''' can be run without installing (just unzip).&lt;br /&gt;
*Click the '''Session''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSession.png]]&lt;br /&gt;
*Click the '''SSH''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSSH.png]]&lt;br /&gt;
*Enter '''Remote host'''&amp;lt;br&amp;gt;Check '''Specify username'''&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;[[File:MobaXterm_Hostname.png]]&lt;br /&gt;
*tab: Network settings&amp;lt;br&amp;gt;Click '''SSH gateway (jump host)'''&amp;lt;br&amp;gt;[[File:MobaXterm_NewJumphost.png]]&lt;br /&gt;
*Enter linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;Click '''OK'''&amp;lt;br&amp;gt;[[File:MobaXterm_Jumphost.png]]&lt;br /&gt;
*Click '''OK''' in the session settings to start connecting&amp;lt;br&amp;gt;You will be asked for your password&amp;lt;br&amp;gt;Enter your password on the console of hpcXX&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
== File Transfer ==&lt;br /&gt;
The best method for transfering files depends on where you want to store your files:&lt;br /&gt;
#Files in your home directory on HPC are only available within HPC&lt;br /&gt;
#Files on Network Shares are available on other locations (such as a TU Delft workplace, weblogin.tudelft.nl or other HPC clusters)&amp;lt;br&amp;gt;Copying files to these shares is best done using the manuals on [https://webdata.tudelft.nl/ webdata.tudelft.nl]&lt;br /&gt;
&lt;br /&gt;
When you want to copy files to the HPC you can copy them from Network Shares&lt;br /&gt;
=== (Windows) Network Shares ===&lt;br /&gt;
&lt;br /&gt;
==== Authenticate ====&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
 kinit&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
==== Basic access ====&lt;br /&gt;
&lt;br /&gt;
Now just type (depending on which share you want to use):&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
 cd /tudelft/$USER/student-group/&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
 cd /tudelft/$USER/staff-umbrella/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
You can access your Windows home directory as well. The subdirectories of the home directories are followed by '''the first letter of your surname''' and then your NetID.&amp;lt;br&amp;gt;&lt;br /&gt;
For example, if your name would be John Smith and your NetID jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
==== Important note ====&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your NetID. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your NetID yourself. If your NetID would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well. Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own NetID!&amp;lt;br&amp;gt;&lt;br /&gt;
'''If you use anthing other than your own NetID you'll create a recipe for potential disaster!'''&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths.&amp;lt;br&amp;gt;&lt;br /&gt;
So &amp;lt;code&amp;gt;/tudelft/jsmith/staff-group/&amp;lt;/code&amp;gt; only works if your NetID is jsmith and &amp;lt;code&amp;gt;/tudelft/pbrown/staff-group/&amp;lt;/code&amp;gt; only works works if your NetID is pbrown.&amp;lt;br&amp;gt;&lt;br /&gt;
If both NetIDs have the same privileges for accessing the group volumes, then these commands give access to the same data.&lt;br /&gt;
&lt;br /&gt;
=== Data on HPC ===&lt;br /&gt;
If you want to transfer data directly to the HPC you will need some extra configuration&lt;br /&gt;
==== Command line ====&lt;br /&gt;
Please first configure [[#SSH Proxy Support|SSH Proxy Support]].&amp;lt;br&amp;gt;&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your remote home directory on HPC to the local home directory on your desktop or laptop. &lt;br /&gt;
 scp &amp;lt;NetID&amp;gt;@hpcXX:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
*Copy an entire directory with all its content, use the '-r' option&amp;lt;br&amp;gt;This will copy the directory 'projects' from your remote home directory on HPC to the local home directory on your desktop or laptop:&lt;br /&gt;
 scp -r &amp;lt;NetID&amp;gt;@hpcXX:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your local home directory on your desktop or laptop to your remote home directory on HPC. &lt;br /&gt;
 scp projects.txt &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
*Copy the directory 'projects' and all its content from your current working directory on your desktop or laptop to your remote home directory on HPC.&lt;br /&gt;
 scp -r projects/ &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
==== WinSCP ====&lt;br /&gt;
WinSCP is a simple tool that has the internal option to create a tunnel through  the bastion server. It can be downloaded from [https://winscp.net/eng/download.php https://winscp.net/eng/download.php]&lt;br /&gt;
*Session &amp;gt; New Session&lt;br /&gt;
[[File:WinSCP01.png]]&lt;br /&gt;
*Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;Advanced&amp;quot; button&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Connection &amp;gt; Tunnel&amp;lt;br&amp;gt;Select &amp;quot;Connect through SSH tunnel&amp;quot;&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;OK&amp;quot; button&lt;br /&gt;
[[File:WinSCP03.png]]&lt;br /&gt;
*Click &amp;quot;Save&amp;quot;&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Save session as site&amp;lt;br&amp;gt;You can accept the suggested name&lt;br /&gt;
[[File:WinSCP04.png]]&lt;br /&gt;
*You can now &amp;quot;Login&amp;quot;&amp;lt;br&amp;gt;You will be asked to supply your password twice: first for the bastion, second for the HPC&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
==== MobaXterm ====&lt;br /&gt;
MobaXterm can also be used for file transfers. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*Left Window&amp;lt;br&amp;gt;Download files with '''arrow down'''&amp;lt;br&amp;gt;Upload files with '''arrow up'''&amp;lt;br&amp;gt;[[File:MobaXterm_SFTP.png]]&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
=== Windows ===&lt;br /&gt;
MobaXterm can also be used for displaying graphical applications. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*In the commandline windows (right)&amp;lt;br&amp;gt;Test X-forwarding (Graphics) by running &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt;&amp;lt;br&amp;gt;You should see these eyes (following the cursor)&amp;lt;br&amp;gt;[[File:MobaXterm_Xeyes.png]]&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in&amp;lt;br&amp;gt;&lt;br /&gt;
The easiest way is:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
When you have configured [[#SSH_Proxy_Support|SSH Proxy Support]] you can connect with:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=290</id>
		<title>Remote Access</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=290"/>
		<updated>2025-01-20T15:14:07Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* Bastion */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;Access to the HPC-clusters is limited from outside the TU Delft network for security reasons.&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
'''You can only log in when you are on campus via a wireless (eduroam) or wired network. When you are off campus, you can log in using eduVPN (Institute Access). As of 16 Januari you can no longer use to the Linux bastion hosts when you are not on campus. There is also no need to use the Linux bastion hosts when you are on campus or using an active eduVPN connection.'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
== Bastion ==&lt;br /&gt;
(This information is outdated and can be safely ignored.)&lt;br /&gt;
&lt;br /&gt;
The TU Delft offers two bastion servers. These servers can be used to tunnel your traffic to the HPC.&lt;br /&gt;
*linux-bastion.tudelft.nl for employees or guests.&lt;br /&gt;
*student-linux.tudelft.nl for students.&lt;br /&gt;
&lt;br /&gt;
== Access to the commandline ==&lt;br /&gt;
The explanations below are the simplest way to connect to a console on an HPC. [[#Advanced configs|Advanced configs]] describes more complex configurations which save time when regularly using the connection or when you want to use [[#Linux|X-forwarding]]&lt;br /&gt;
=== Command line ===&lt;br /&gt;
When your desktop/laptop supports ssh from the console:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
If your SSH-client doesn't support Jump hosts, you can first connect to the bastion server and then to the HPC:&lt;br /&gt;
&lt;br /&gt;
 [user@home]$ ssh &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
 [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt; with your NetID (also remove the &amp;lt;&amp;gt;)&lt;br /&gt;
*When your a student replace &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl with &amp;lt;NetID&amp;gt;@student-linux.tudelft.nl&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net with the HPC you are connecting to.&lt;br /&gt;
&lt;br /&gt;
=== Advanced configs ===&lt;br /&gt;
==== Making OpenSSH more user-friendly ====&lt;br /&gt;
The OpenSSH clients can be configured using a configuration file in ~/.ssh/config (Linux, macOS) or ~\.ssh\config (Windows).&lt;br /&gt;
&lt;br /&gt;
===== Define connections =====&lt;br /&gt;
For the OpenSSH clients you can predefine often used connections by placing a section like the following in the configuration file on your local computer. Authentication on the bastion with keys is not possible, so password is used.&amp;lt;br&amp;gt;&lt;br /&gt;
Employees:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
Students:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
You can then connect simply by ssh bastion. For example:&lt;br /&gt;
  [user@home]$ ssh bastion&lt;br /&gt;
    NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
&lt;br /&gt;
===== SSH Proxy Support =====&lt;br /&gt;
Since this configuration depends on [[#Define connections|Define connections]] configure that first.&amp;lt;br&amp;gt;&lt;br /&gt;
To connect directly from your machine to an HPC cluster login node (without connecting to the bastion server first), create a connection via a proxy by adding the following lines to the configuration file ~/.ssh/config on your local computer:&lt;br /&gt;
&lt;br /&gt;
  Host hpcXX&lt;br /&gt;
  Hostname hpcXX.tudelft.net&lt;br /&gt;
  ProxyCommand ssh -W %h:%p bastion&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
&lt;br /&gt;
You can then simply use ssh hpcXX (or scp -p '&amp;lt;local file&amp;gt;' &amp;quot;hpcXX:/scratch/&amp;lt;NetID&amp;gt;/&amp;quot;) from your local computer.&lt;br /&gt;
&lt;br /&gt;
===== Session multiplexing =====&lt;br /&gt;
OpenSSH clients support multiple sessions over a single connection. This means you have to enter your password only once, to make the connection, and can then open multiple sessions (SSH, SCP and SFTP) to that computer over the same connection. This is especially convenient when connecting to the bastion server, where you have to type your password always.&lt;br /&gt;
&lt;br /&gt;
Note: SSH public key logins won't work (reliably, (because Kerberos authentication is required to access your home directory).&lt;br /&gt;
&lt;br /&gt;
Add the following section to the end of the configuration file on your local computer and on the bastion server:&lt;br /&gt;
&lt;br /&gt;
  Host *&lt;br /&gt;
  ControlMaster auto&lt;br /&gt;
  ControlPath /tmp/ssh-%r@%h:%p&lt;br /&gt;
&lt;br /&gt;
Windows users may need to adapt the ControlPath location to match Windows.&lt;br /&gt;
&lt;br /&gt;
You can use the exact same commands as above.&lt;br /&gt;
=== PuTTY ===&lt;br /&gt;
When your desktop/laptop doesn't support ssh from the console, you can use [https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html PuTTY] (it may already be installed).&amp;lt;br&amp;gt;&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students), and select the SSH protocol (port 22):&amp;lt;br&amp;gt;[[File:PuTTY-Hostname.png]]&lt;br /&gt;
*OPTIONAL: Connection&amp;gt;SSH Settings&amp;lt;br&amp;gt;Select Share SSH connections if possible (this means that you will not have to authenticate when you start a second session):&amp;lt;br&amp;gt;[[File:PuTTY-ShareConnection.png]]&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;To store these session settings, go back to the session tab, and save the settings under a session name. To activate a stored session, double-click on the session name:&amp;lt;br&amp;gt;[[File:PuTTY-SaveSession.png]]&lt;br /&gt;
Once you're connected to the bastion server you can use the locally available ssh to connect to a HPC:&lt;br /&gt;
   [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
     NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
     Welcome&lt;br /&gt;
   [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
=== MobaXterm ===&lt;br /&gt;
MobaXterm is a very versatile application. It can be used to connect to the commandline on HPC, display graphical applications and transfer files. It also has built-in support for using SSH-tunneling to connect through a bastion. You can download it from [https://mobaxterm.mobatek.net/ https://mobaxterm.mobatek.net/]. The '''free version''' has enough features for most users. The '''Portable edition''' can be run without installing (just unzip).&lt;br /&gt;
*Click the '''Session''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSession.png]]&lt;br /&gt;
*Click the '''SSH''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSSH.png]]&lt;br /&gt;
*Enter '''Remote host'''&amp;lt;br&amp;gt;Check '''Specify username'''&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;[[File:MobaXterm_Hostname.png]]&lt;br /&gt;
*tab: Network settings&amp;lt;br&amp;gt;Click '''SSH gateway (jump host)'''&amp;lt;br&amp;gt;[[File:MobaXterm_NewJumphost.png]]&lt;br /&gt;
*Enter linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;Click '''OK'''&amp;lt;br&amp;gt;[[File:MobaXterm_Jumphost.png]]&lt;br /&gt;
*Click '''OK''' in the session settings to start connecting&amp;lt;br&amp;gt;You will be asked for your password&amp;lt;br&amp;gt;Enter your password on the console of hpcXX&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
== File Transfer ==&lt;br /&gt;
The best method for transfering files depends on where you want to store your files:&lt;br /&gt;
#Files in your home directory on HPC are only available within HPC&lt;br /&gt;
#Files on Network Shares are available on other locations (such as a TU Delft workplace, weblogin.tudelft.nl or other HPC clusters)&amp;lt;br&amp;gt;Copying files to these shares is best done using the manuals on [https://webdata.tudelft.nl/ webdata.tudelft.nl]&lt;br /&gt;
&lt;br /&gt;
When you want to copy files to the HPC you can copy them from Network Shares&lt;br /&gt;
=== (Windows) Network Shares ===&lt;br /&gt;
&lt;br /&gt;
==== Authenticate ====&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
 kinit&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
==== Basic access ====&lt;br /&gt;
&lt;br /&gt;
Now just type (depending on which share you want to use):&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
 cd /tudelft/$USER/student-group/&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
 cd /tudelft/$USER/staff-umbrella/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
You can access your Windows home directory as well. The subdirectories of the home directories are followed by '''the first letter of your surname''' and then your NetID.&amp;lt;br&amp;gt;&lt;br /&gt;
For example, if your name would be John Smith and your NetID jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
==== Important note ====&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your NetID. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your NetID yourself. If your NetID would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well. Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own NetID!&amp;lt;br&amp;gt;&lt;br /&gt;
'''If you use anthing other than your own NetID you'll create a recipe for potential disaster!'''&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths.&amp;lt;br&amp;gt;&lt;br /&gt;
So &amp;lt;code&amp;gt;/tudelft/jsmith/staff-group/&amp;lt;/code&amp;gt; only works if your NetID is jsmith and &amp;lt;code&amp;gt;/tudelft/pbrown/staff-group/&amp;lt;/code&amp;gt; only works works if your NetID is pbrown.&amp;lt;br&amp;gt;&lt;br /&gt;
If both NetIDs have the same privileges for accessing the group volumes, then these commands give access to the same data.&lt;br /&gt;
&lt;br /&gt;
=== Data on HPC ===&lt;br /&gt;
If you want to transfer data directly to the HPC you will need some extra configuration&lt;br /&gt;
==== Command line ====&lt;br /&gt;
Please first configure [[#SSH Proxy Support|SSH Proxy Support]].&amp;lt;br&amp;gt;&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your remote home directory on HPC to the local home directory on your desktop or laptop. &lt;br /&gt;
 scp &amp;lt;NetID&amp;gt;@hpcXX:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
*Copy an entire directory with all its content, use the '-r' option&amp;lt;br&amp;gt;This will copy the directory 'projects' from your remote home directory on HPC to the local home directory on your desktop or laptop:&lt;br /&gt;
 scp -r &amp;lt;NetID&amp;gt;@hpcXX:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your local home directory on your desktop or laptop to your remote home directory on HPC. &lt;br /&gt;
 scp projects.txt &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
*Copy the directory 'projects' and all its content from your current working directory on your desktop or laptop to your remote home directory on HPC.&lt;br /&gt;
 scp -r projects/ &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
==== WinSCP ====&lt;br /&gt;
WinSCP is a simple tool that has the internal option to create a tunnel through  the bastion server. It can be downloaded from [https://winscp.net/eng/download.php https://winscp.net/eng/download.php]&lt;br /&gt;
*Session &amp;gt; New Session&lt;br /&gt;
[[File:WinSCP01.png]]&lt;br /&gt;
*Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;Advanced&amp;quot; button&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Connection &amp;gt; Tunnel&amp;lt;br&amp;gt;Select &amp;quot;Connect through SSH tunnel&amp;quot;&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;OK&amp;quot; button&lt;br /&gt;
[[File:WinSCP03.png]]&lt;br /&gt;
*Click &amp;quot;Save&amp;quot;&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Save session as site&amp;lt;br&amp;gt;You can accept the suggested name&lt;br /&gt;
[[File:WinSCP04.png]]&lt;br /&gt;
*You can now &amp;quot;Login&amp;quot;&amp;lt;br&amp;gt;You will be asked to supply your password twice: first for the bastion, second for the HPC&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
==== MobaXterm ====&lt;br /&gt;
MobaXterm can also be used for file transfers. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*Left Window&amp;lt;br&amp;gt;Download files with '''arrow down'''&amp;lt;br&amp;gt;Upload files with '''arrow up'''&amp;lt;br&amp;gt;[[File:MobaXterm_SFTP.png]]&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
=== Windows ===&lt;br /&gt;
MobaXterm can also be used for displaying graphical applications. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*In the commandline windows (right)&amp;lt;br&amp;gt;Test X-forwarding (Graphics) by running &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt;&amp;lt;br&amp;gt;You should see these eyes (following the cursor)&amp;lt;br&amp;gt;[[File:MobaXterm_Xeyes.png]]&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in&amp;lt;br&amp;gt;&lt;br /&gt;
The easiest way is:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
When you have configured [[#SSH_Proxy_Support|SSH Proxy Support]] you can connect with:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=289</id>
		<title>Remote Access</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=289"/>
		<updated>2025-01-20T15:12:27Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* Bastion */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;Access to the HPC-clusters is limited from outside the TU Delft network for security reasons.&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
'''You can only log in when you are on campus via a wireless (eduroam) or wired network. When you are off campus, you can log in using eduVPN (Institute Access). As of 16 Januari you can no longer use to the Linux bastion hosts when you are not on campus. There is also no need to use the Linux bastion hosts when you are on campus or using an active eduVPN connection.'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
== Bastion ==&lt;br /&gt;
The TU Delft offers two bastion servers. These servers can be used to tunnel your traffic to the HPC.&lt;br /&gt;
*linux-bastion.tudelft.nl for employees or guests.&lt;br /&gt;
*student-linux.tudelft.nl for students.&lt;br /&gt;
&lt;br /&gt;
== Access to the commandline ==&lt;br /&gt;
The explanations below are the simplest way to connect to a console on an HPC. [[#Advanced configs|Advanced configs]] describes more complex configurations which save time when regularly using the connection or when you want to use [[#Linux|X-forwarding]]&lt;br /&gt;
=== Command line ===&lt;br /&gt;
When your desktop/laptop supports ssh from the console:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
If your SSH-client doesn't support Jump hosts, you can first connect to the bastion server and then to the HPC:&lt;br /&gt;
&lt;br /&gt;
 [user@home]$ ssh &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
 [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt; with your NetID (also remove the &amp;lt;&amp;gt;)&lt;br /&gt;
*When your a student replace &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl with &amp;lt;NetID&amp;gt;@student-linux.tudelft.nl&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net with the HPC you are connecting to.&lt;br /&gt;
&lt;br /&gt;
=== Advanced configs ===&lt;br /&gt;
==== Making OpenSSH more user-friendly ====&lt;br /&gt;
The OpenSSH clients can be configured using a configuration file in ~/.ssh/config (Linux, macOS) or ~\.ssh\config (Windows).&lt;br /&gt;
&lt;br /&gt;
===== Define connections =====&lt;br /&gt;
For the OpenSSH clients you can predefine often used connections by placing a section like the following in the configuration file on your local computer. Authentication on the bastion with keys is not possible, so password is used.&amp;lt;br&amp;gt;&lt;br /&gt;
Employees:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
Students:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
You can then connect simply by ssh bastion. For example:&lt;br /&gt;
  [user@home]$ ssh bastion&lt;br /&gt;
    NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
&lt;br /&gt;
===== SSH Proxy Support =====&lt;br /&gt;
Since this configuration depends on [[#Define connections|Define connections]] configure that first.&amp;lt;br&amp;gt;&lt;br /&gt;
To connect directly from your machine to an HPC cluster login node (without connecting to the bastion server first), create a connection via a proxy by adding the following lines to the configuration file ~/.ssh/config on your local computer:&lt;br /&gt;
&lt;br /&gt;
  Host hpcXX&lt;br /&gt;
  Hostname hpcXX.tudelft.net&lt;br /&gt;
  ProxyCommand ssh -W %h:%p bastion&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
&lt;br /&gt;
You can then simply use ssh hpcXX (or scp -p '&amp;lt;local file&amp;gt;' &amp;quot;hpcXX:/scratch/&amp;lt;NetID&amp;gt;/&amp;quot;) from your local computer.&lt;br /&gt;
&lt;br /&gt;
===== Session multiplexing =====&lt;br /&gt;
OpenSSH clients support multiple sessions over a single connection. This means you have to enter your password only once, to make the connection, and can then open multiple sessions (SSH, SCP and SFTP) to that computer over the same connection. This is especially convenient when connecting to the bastion server, where you have to type your password always.&lt;br /&gt;
&lt;br /&gt;
Note: SSH public key logins won't work (reliably, (because Kerberos authentication is required to access your home directory).&lt;br /&gt;
&lt;br /&gt;
Add the following section to the end of the configuration file on your local computer and on the bastion server:&lt;br /&gt;
&lt;br /&gt;
  Host *&lt;br /&gt;
  ControlMaster auto&lt;br /&gt;
  ControlPath /tmp/ssh-%r@%h:%p&lt;br /&gt;
&lt;br /&gt;
Windows users may need to adapt the ControlPath location to match Windows.&lt;br /&gt;
&lt;br /&gt;
You can use the exact same commands as above.&lt;br /&gt;
=== PuTTY ===&lt;br /&gt;
When your desktop/laptop doesn't support ssh from the console, you can use [https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html PuTTY] (it may already be installed).&amp;lt;br&amp;gt;&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students), and select the SSH protocol (port 22):&amp;lt;br&amp;gt;[[File:PuTTY-Hostname.png]]&lt;br /&gt;
*OPTIONAL: Connection&amp;gt;SSH Settings&amp;lt;br&amp;gt;Select Share SSH connections if possible (this means that you will not have to authenticate when you start a second session):&amp;lt;br&amp;gt;[[File:PuTTY-ShareConnection.png]]&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;To store these session settings, go back to the session tab, and save the settings under a session name. To activate a stored session, double-click on the session name:&amp;lt;br&amp;gt;[[File:PuTTY-SaveSession.png]]&lt;br /&gt;
Once you're connected to the bastion server you can use the locally available ssh to connect to a HPC:&lt;br /&gt;
   [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
     NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
     Welcome&lt;br /&gt;
   [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
=== MobaXterm ===&lt;br /&gt;
MobaXterm is a very versatile application. It can be used to connect to the commandline on HPC, display graphical applications and transfer files. It also has built-in support for using SSH-tunneling to connect through a bastion. You can download it from [https://mobaxterm.mobatek.net/ https://mobaxterm.mobatek.net/]. The '''free version''' has enough features for most users. The '''Portable edition''' can be run without installing (just unzip).&lt;br /&gt;
*Click the '''Session''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSession.png]]&lt;br /&gt;
*Click the '''SSH''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSSH.png]]&lt;br /&gt;
*Enter '''Remote host'''&amp;lt;br&amp;gt;Check '''Specify username'''&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;[[File:MobaXterm_Hostname.png]]&lt;br /&gt;
*tab: Network settings&amp;lt;br&amp;gt;Click '''SSH gateway (jump host)'''&amp;lt;br&amp;gt;[[File:MobaXterm_NewJumphost.png]]&lt;br /&gt;
*Enter linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;Click '''OK'''&amp;lt;br&amp;gt;[[File:MobaXterm_Jumphost.png]]&lt;br /&gt;
*Click '''OK''' in the session settings to start connecting&amp;lt;br&amp;gt;You will be asked for your password&amp;lt;br&amp;gt;Enter your password on the console of hpcXX&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
== File Transfer ==&lt;br /&gt;
The best method for transfering files depends on where you want to store your files:&lt;br /&gt;
#Files in your home directory on HPC are only available within HPC&lt;br /&gt;
#Files on Network Shares are available on other locations (such as a TU Delft workplace, weblogin.tudelft.nl or other HPC clusters)&amp;lt;br&amp;gt;Copying files to these shares is best done using the manuals on [https://webdata.tudelft.nl/ webdata.tudelft.nl]&lt;br /&gt;
&lt;br /&gt;
When you want to copy files to the HPC you can copy them from Network Shares&lt;br /&gt;
=== (Windows) Network Shares ===&lt;br /&gt;
&lt;br /&gt;
==== Authenticate ====&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
 kinit&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
==== Basic access ====&lt;br /&gt;
&lt;br /&gt;
Now just type (depending on which share you want to use):&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
 cd /tudelft/$USER/student-group/&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
 cd /tudelft/$USER/staff-umbrella/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
You can access your Windows home directory as well. The subdirectories of the home directories are followed by '''the first letter of your surname''' and then your NetID.&amp;lt;br&amp;gt;&lt;br /&gt;
For example, if your name would be John Smith and your NetID jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
==== Important note ====&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your NetID. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your NetID yourself. If your NetID would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well. Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own NetID!&amp;lt;br&amp;gt;&lt;br /&gt;
'''If you use anthing other than your own NetID you'll create a recipe for potential disaster!'''&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths.&amp;lt;br&amp;gt;&lt;br /&gt;
So &amp;lt;code&amp;gt;/tudelft/jsmith/staff-group/&amp;lt;/code&amp;gt; only works if your NetID is jsmith and &amp;lt;code&amp;gt;/tudelft/pbrown/staff-group/&amp;lt;/code&amp;gt; only works works if your NetID is pbrown.&amp;lt;br&amp;gt;&lt;br /&gt;
If both NetIDs have the same privileges for accessing the group volumes, then these commands give access to the same data.&lt;br /&gt;
&lt;br /&gt;
=== Data on HPC ===&lt;br /&gt;
If you want to transfer data directly to the HPC you will need some extra configuration&lt;br /&gt;
==== Command line ====&lt;br /&gt;
Please first configure [[#SSH Proxy Support|SSH Proxy Support]].&amp;lt;br&amp;gt;&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your remote home directory on HPC to the local home directory on your desktop or laptop. &lt;br /&gt;
 scp &amp;lt;NetID&amp;gt;@hpcXX:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
*Copy an entire directory with all its content, use the '-r' option&amp;lt;br&amp;gt;This will copy the directory 'projects' from your remote home directory on HPC to the local home directory on your desktop or laptop:&lt;br /&gt;
 scp -r &amp;lt;NetID&amp;gt;@hpcXX:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your local home directory on your desktop or laptop to your remote home directory on HPC. &lt;br /&gt;
 scp projects.txt &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
*Copy the directory 'projects' and all its content from your current working directory on your desktop or laptop to your remote home directory on HPC.&lt;br /&gt;
 scp -r projects/ &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
==== WinSCP ====&lt;br /&gt;
WinSCP is a simple tool that has the internal option to create a tunnel through  the bastion server. It can be downloaded from [https://winscp.net/eng/download.php https://winscp.net/eng/download.php]&lt;br /&gt;
*Session &amp;gt; New Session&lt;br /&gt;
[[File:WinSCP01.png]]&lt;br /&gt;
*Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;Advanced&amp;quot; button&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Connection &amp;gt; Tunnel&amp;lt;br&amp;gt;Select &amp;quot;Connect through SSH tunnel&amp;quot;&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;OK&amp;quot; button&lt;br /&gt;
[[File:WinSCP03.png]]&lt;br /&gt;
*Click &amp;quot;Save&amp;quot;&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Save session as site&amp;lt;br&amp;gt;You can accept the suggested name&lt;br /&gt;
[[File:WinSCP04.png]]&lt;br /&gt;
*You can now &amp;quot;Login&amp;quot;&amp;lt;br&amp;gt;You will be asked to supply your password twice: first for the bastion, second for the HPC&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
==== MobaXterm ====&lt;br /&gt;
MobaXterm can also be used for file transfers. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*Left Window&amp;lt;br&amp;gt;Download files with '''arrow down'''&amp;lt;br&amp;gt;Upload files with '''arrow up'''&amp;lt;br&amp;gt;[[File:MobaXterm_SFTP.png]]&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
=== Windows ===&lt;br /&gt;
MobaXterm can also be used for displaying graphical applications. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*In the commandline windows (right)&amp;lt;br&amp;gt;Test X-forwarding (Graphics) by running &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt;&amp;lt;br&amp;gt;You should see these eyes (following the cursor)&amp;lt;br&amp;gt;[[File:MobaXterm_Xeyes.png]]&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in&amp;lt;br&amp;gt;&lt;br /&gt;
The easiest way is:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
When you have configured [[#SSH_Proxy_Support|SSH Proxy Support]] you can connect with:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=288</id>
		<title>Remote Access</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=288"/>
		<updated>2025-01-20T15:12:08Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* Bastion */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;Access to the HPC-clusters is limited from outside the TU Delft network for security reasons.&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
'''You can only log in when you are on campus via a wireless (eduroam) or wired network. When you are off campus, you can log in using eduVPN (Institute Access). As of 16 Januari you can no longer use to the Linux bastion hosts when you are not on campus. There is also no need to use the Linux bastion hosts when you are on campus or using an active eduVPN connection.'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
== Bastion ==&lt;br /&gt;
&amp;lt;s&amp;gt;The TU Delft offers two bastion servers. These servers can be used to tunnel your traffic to the HPC.&amp;lt;/s&amp;gt;&lt;br /&gt;
*&amp;lt;s&amp;gt;linux-bastion.tudelft.nl for employees or guests.&amp;lt;/s&amp;gt;&lt;br /&gt;
*&amp;lt;s&amp;gt;student-linux.tudelft.nl for students.&amp;lt;/s&amp;gt;&lt;br /&gt;
&lt;br /&gt;
== Access to the commandline ==&lt;br /&gt;
The explanations below are the simplest way to connect to a console on an HPC. [[#Advanced configs|Advanced configs]] describes more complex configurations which save time when regularly using the connection or when you want to use [[#Linux|X-forwarding]]&lt;br /&gt;
=== Command line ===&lt;br /&gt;
When your desktop/laptop supports ssh from the console:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
If your SSH-client doesn't support Jump hosts, you can first connect to the bastion server and then to the HPC:&lt;br /&gt;
&lt;br /&gt;
 [user@home]$ ssh &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
 [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt; with your NetID (also remove the &amp;lt;&amp;gt;)&lt;br /&gt;
*When your a student replace &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl with &amp;lt;NetID&amp;gt;@student-linux.tudelft.nl&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net with the HPC you are connecting to.&lt;br /&gt;
&lt;br /&gt;
=== Advanced configs ===&lt;br /&gt;
==== Making OpenSSH more user-friendly ====&lt;br /&gt;
The OpenSSH clients can be configured using a configuration file in ~/.ssh/config (Linux, macOS) or ~\.ssh\config (Windows).&lt;br /&gt;
&lt;br /&gt;
===== Define connections =====&lt;br /&gt;
For the OpenSSH clients you can predefine often used connections by placing a section like the following in the configuration file on your local computer. Authentication on the bastion with keys is not possible, so password is used.&amp;lt;br&amp;gt;&lt;br /&gt;
Employees:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
Students:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
You can then connect simply by ssh bastion. For example:&lt;br /&gt;
  [user@home]$ ssh bastion&lt;br /&gt;
    NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
&lt;br /&gt;
===== SSH Proxy Support =====&lt;br /&gt;
Since this configuration depends on [[#Define connections|Define connections]] configure that first.&amp;lt;br&amp;gt;&lt;br /&gt;
To connect directly from your machine to an HPC cluster login node (without connecting to the bastion server first), create a connection via a proxy by adding the following lines to the configuration file ~/.ssh/config on your local computer:&lt;br /&gt;
&lt;br /&gt;
  Host hpcXX&lt;br /&gt;
  Hostname hpcXX.tudelft.net&lt;br /&gt;
  ProxyCommand ssh -W %h:%p bastion&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
&lt;br /&gt;
You can then simply use ssh hpcXX (or scp -p '&amp;lt;local file&amp;gt;' &amp;quot;hpcXX:/scratch/&amp;lt;NetID&amp;gt;/&amp;quot;) from your local computer.&lt;br /&gt;
&lt;br /&gt;
===== Session multiplexing =====&lt;br /&gt;
OpenSSH clients support multiple sessions over a single connection. This means you have to enter your password only once, to make the connection, and can then open multiple sessions (SSH, SCP and SFTP) to that computer over the same connection. This is especially convenient when connecting to the bastion server, where you have to type your password always.&lt;br /&gt;
&lt;br /&gt;
Note: SSH public key logins won't work (reliably, (because Kerberos authentication is required to access your home directory).&lt;br /&gt;
&lt;br /&gt;
Add the following section to the end of the configuration file on your local computer and on the bastion server:&lt;br /&gt;
&lt;br /&gt;
  Host *&lt;br /&gt;
  ControlMaster auto&lt;br /&gt;
  ControlPath /tmp/ssh-%r@%h:%p&lt;br /&gt;
&lt;br /&gt;
Windows users may need to adapt the ControlPath location to match Windows.&lt;br /&gt;
&lt;br /&gt;
You can use the exact same commands as above.&lt;br /&gt;
=== PuTTY ===&lt;br /&gt;
When your desktop/laptop doesn't support ssh from the console, you can use [https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html PuTTY] (it may already be installed).&amp;lt;br&amp;gt;&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students), and select the SSH protocol (port 22):&amp;lt;br&amp;gt;[[File:PuTTY-Hostname.png]]&lt;br /&gt;
*OPTIONAL: Connection&amp;gt;SSH Settings&amp;lt;br&amp;gt;Select Share SSH connections if possible (this means that you will not have to authenticate when you start a second session):&amp;lt;br&amp;gt;[[File:PuTTY-ShareConnection.png]]&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;To store these session settings, go back to the session tab, and save the settings under a session name. To activate a stored session, double-click on the session name:&amp;lt;br&amp;gt;[[File:PuTTY-SaveSession.png]]&lt;br /&gt;
Once you're connected to the bastion server you can use the locally available ssh to connect to a HPC:&lt;br /&gt;
   [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
     NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
     Welcome&lt;br /&gt;
   [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
=== MobaXterm ===&lt;br /&gt;
MobaXterm is a very versatile application. It can be used to connect to the commandline on HPC, display graphical applications and transfer files. It also has built-in support for using SSH-tunneling to connect through a bastion. You can download it from [https://mobaxterm.mobatek.net/ https://mobaxterm.mobatek.net/]. The '''free version''' has enough features for most users. The '''Portable edition''' can be run without installing (just unzip).&lt;br /&gt;
*Click the '''Session''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSession.png]]&lt;br /&gt;
*Click the '''SSH''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSSH.png]]&lt;br /&gt;
*Enter '''Remote host'''&amp;lt;br&amp;gt;Check '''Specify username'''&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;[[File:MobaXterm_Hostname.png]]&lt;br /&gt;
*tab: Network settings&amp;lt;br&amp;gt;Click '''SSH gateway (jump host)'''&amp;lt;br&amp;gt;[[File:MobaXterm_NewJumphost.png]]&lt;br /&gt;
*Enter linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;Click '''OK'''&amp;lt;br&amp;gt;[[File:MobaXterm_Jumphost.png]]&lt;br /&gt;
*Click '''OK''' in the session settings to start connecting&amp;lt;br&amp;gt;You will be asked for your password&amp;lt;br&amp;gt;Enter your password on the console of hpcXX&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
== File Transfer ==&lt;br /&gt;
The best method for transfering files depends on where you want to store your files:&lt;br /&gt;
#Files in your home directory on HPC are only available within HPC&lt;br /&gt;
#Files on Network Shares are available on other locations (such as a TU Delft workplace, weblogin.tudelft.nl or other HPC clusters)&amp;lt;br&amp;gt;Copying files to these shares is best done using the manuals on [https://webdata.tudelft.nl/ webdata.tudelft.nl]&lt;br /&gt;
&lt;br /&gt;
When you want to copy files to the HPC you can copy them from Network Shares&lt;br /&gt;
=== (Windows) Network Shares ===&lt;br /&gt;
&lt;br /&gt;
==== Authenticate ====&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
 kinit&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
==== Basic access ====&lt;br /&gt;
&lt;br /&gt;
Now just type (depending on which share you want to use):&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
 cd /tudelft/$USER/student-group/&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
 cd /tudelft/$USER/staff-umbrella/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
You can access your Windows home directory as well. The subdirectories of the home directories are followed by '''the first letter of your surname''' and then your NetID.&amp;lt;br&amp;gt;&lt;br /&gt;
For example, if your name would be John Smith and your NetID jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
==== Important note ====&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your NetID. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your NetID yourself. If your NetID would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well. Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own NetID!&amp;lt;br&amp;gt;&lt;br /&gt;
'''If you use anthing other than your own NetID you'll create a recipe for potential disaster!'''&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths.&amp;lt;br&amp;gt;&lt;br /&gt;
So &amp;lt;code&amp;gt;/tudelft/jsmith/staff-group/&amp;lt;/code&amp;gt; only works if your NetID is jsmith and &amp;lt;code&amp;gt;/tudelft/pbrown/staff-group/&amp;lt;/code&amp;gt; only works works if your NetID is pbrown.&amp;lt;br&amp;gt;&lt;br /&gt;
If both NetIDs have the same privileges for accessing the group volumes, then these commands give access to the same data.&lt;br /&gt;
&lt;br /&gt;
=== Data on HPC ===&lt;br /&gt;
If you want to transfer data directly to the HPC you will need some extra configuration&lt;br /&gt;
==== Command line ====&lt;br /&gt;
Please first configure [[#SSH Proxy Support|SSH Proxy Support]].&amp;lt;br&amp;gt;&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your remote home directory on HPC to the local home directory on your desktop or laptop. &lt;br /&gt;
 scp &amp;lt;NetID&amp;gt;@hpcXX:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
*Copy an entire directory with all its content, use the '-r' option&amp;lt;br&amp;gt;This will copy the directory 'projects' from your remote home directory on HPC to the local home directory on your desktop or laptop:&lt;br /&gt;
 scp -r &amp;lt;NetID&amp;gt;@hpcXX:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your local home directory on your desktop or laptop to your remote home directory on HPC. &lt;br /&gt;
 scp projects.txt &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
*Copy the directory 'projects' and all its content from your current working directory on your desktop or laptop to your remote home directory on HPC.&lt;br /&gt;
 scp -r projects/ &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
==== WinSCP ====&lt;br /&gt;
WinSCP is a simple tool that has the internal option to create a tunnel through  the bastion server. It can be downloaded from [https://winscp.net/eng/download.php https://winscp.net/eng/download.php]&lt;br /&gt;
*Session &amp;gt; New Session&lt;br /&gt;
[[File:WinSCP01.png]]&lt;br /&gt;
*Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;Advanced&amp;quot; button&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Connection &amp;gt; Tunnel&amp;lt;br&amp;gt;Select &amp;quot;Connect through SSH tunnel&amp;quot;&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;OK&amp;quot; button&lt;br /&gt;
[[File:WinSCP03.png]]&lt;br /&gt;
*Click &amp;quot;Save&amp;quot;&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Save session as site&amp;lt;br&amp;gt;You can accept the suggested name&lt;br /&gt;
[[File:WinSCP04.png]]&lt;br /&gt;
*You can now &amp;quot;Login&amp;quot;&amp;lt;br&amp;gt;You will be asked to supply your password twice: first for the bastion, second for the HPC&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
==== MobaXterm ====&lt;br /&gt;
MobaXterm can also be used for file transfers. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*Left Window&amp;lt;br&amp;gt;Download files with '''arrow down'''&amp;lt;br&amp;gt;Upload files with '''arrow up'''&amp;lt;br&amp;gt;[[File:MobaXterm_SFTP.png]]&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
=== Windows ===&lt;br /&gt;
MobaXterm can also be used for displaying graphical applications. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*In the commandline windows (right)&amp;lt;br&amp;gt;Test X-forwarding (Graphics) by running &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt;&amp;lt;br&amp;gt;You should see these eyes (following the cursor)&amp;lt;br&amp;gt;[[File:MobaXterm_Xeyes.png]]&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in&amp;lt;br&amp;gt;&lt;br /&gt;
The easiest way is:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
When you have configured [[#SSH_Proxy_Support|SSH Proxy Support]] you can connect with:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=287</id>
		<title>How to log in</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=287"/>
		<updated>2025-01-20T15:11:33Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* Basics */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== Basics ==&lt;br /&gt;
&lt;br /&gt;
In order to log in you need a valid [https://www.tudelft.nl/en/student/ict/ict-facilities/netid-account/ NetID] that is enabled for access to one of the clusters. To have your NetID enabled, ask the contact person for the cluster you want access to (See the [[Introduction]] page) or ask the secretary of the department or research group where you work, he or she can tell you who the contact person is for an hpc cluster.&lt;br /&gt;
&lt;br /&gt;
If you want to collaborate with someone from outside the TU Delft and you want that person to access the same cluster as you, then you can request a guest account at the servicedesk. Once the guest account is created, it can be enabled for access to one of the clusters. '''Do not share your own NetID with others! Your NetID is for personal use only, sharing it with other people is forbidden!'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are only accessible via ssh (secure shell). To log in you need an ssh client. If you use Windows, you can download and use [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY], Macintosh and Linux computers already have an ssh client built in.&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
'''You can only log in when you are on campus via a wireless (eduroam) or wired network. When you are off campus, you can log in using eduVPN (Institute Access). As of 16 Januari you can no longer use to the Linux bastion hosts when you are not on campus. There is also no need to use the Linux bastion hosts when you are on campus or using an active eduVPN connection.'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
Every time you log in you'll see an motd (message of the day). Please pay attention to this, it may contain information about current events on the cluster. It also shows you an email address that you can use if there are problems or if you have questions. You can mail at any time you like, but you'll usually only get a reply during office hours, there is no 24/7 support on the hpc clusters.&lt;br /&gt;
&lt;br /&gt;
'''Please note:''' You have a local home directory on the cluster. No backups will be made of the home directories, if you make a mistake and loose a file, it is gone forever. '''Making backups of your important files on the hpc clusters is your own responsibility!'''&lt;br /&gt;
&lt;br /&gt;
== How to log in from Windows ==&lt;br /&gt;
&lt;br /&gt;
An often used ssh client for Windows is [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY]. If you use a Windows PC or a laptop that is provided and maintained by the TU Delft, it is already there, if you use your own Windows computer, you'll have to download and install it first from [http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html www.chiark.greenend.org.uk].&lt;br /&gt;
&lt;br /&gt;
Once you have started PuTTY you'll get a configuration window als shown.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can type the name of the cluster, for example 'hpc10.tudelft.net', in the field 'Host Name'. You can save the settings for a cluster by choosing a name for the session, enter it in the 'Saved Sessions' box and click 'Save'. The next time you open PuTTY the settings can be restored by clicking on the name and a click on 'Load'.&lt;br /&gt;
&lt;br /&gt;
By clicking 'Open' a connection is made to hpc10.tudelft.net. If this is the first time you are connecting to the cluster, PuTTY will warn you that you are connecting to a previously unknown host.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_02.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can confirm this connection by clicking 'yes' so PuTTY will store the host key in its database. You'll have to do this only once for every new server that you connect to.&lt;br /&gt;
&lt;br /&gt;
After you have accepted the connection, PuTTY opens a console and asks for your loginname. Once you have entered your NetID and your password, you are logged in to the cluster.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_03.png|center]]&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from Windows ==&lt;br /&gt;
&lt;br /&gt;
WinSCP is a freeware ftp client with support for secure file protocols sftp and scp. The login screen depicted below shows the settings for access within the University.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
After clicking on 'Login' a warning about host keys will appear, similar to PuTTY. If it is the first time, you can safely click on 'yes' to accept.&lt;br /&gt;
&lt;br /&gt;
Press 'Login'. If everything is setup correctly you are using the ssh tunnel the PuTTY bastion session has built and you are logged in on the cluster.&lt;br /&gt;
&lt;br /&gt;
When logged in, you will see the commander-like interface below.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_02.png|center|530px]]&lt;br /&gt;
&lt;br /&gt;
The left side displays the files in your home directory on the windows system. The right side displays the files on the cluster. You can drag and drop files between the two windows or select files with the mouse and press 'F5' for transfer to the other window.&lt;br /&gt;
&lt;br /&gt;
== How to log in from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
Mac OS X is in fact a full blown UNIX system under the hood. A terminal program comes standard with OS X, navigate with the finder to Applications -&amp;gt; Utilities to find it, the program is called 'Terminal'. There are many different flavours of Linux, each with their own desktop environments and customizations. Nearly every Linux system comes with a terminal program of some sort. Find the terminal program on your system and run it, once you get a command prompt type:&lt;br /&gt;
&lt;br /&gt;
 ssh &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
Where &amp;lt;netid&amp;gt; is of course your own netid.&lt;br /&gt;
&lt;br /&gt;
Upon first access, ssh requires you to check if the host key is correct. Since this is the first time you connect to the system, it's fine to accept the host key by pressing 'y'.&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh:&lt;br /&gt;
&lt;br /&gt;
 scp &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your remote home directory (folder) to the local home directory on your workstation or laptop. &lt;br /&gt;
&lt;br /&gt;
If you want to copy an entire directory with all its content, use the '-r' option, like this:&lt;br /&gt;
&lt;br /&gt;
 scp -r &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
This will copy the directory 'projects' from your remote home directory to the local home directory on your workstation or laptop.&lt;br /&gt;
&lt;br /&gt;
Of course, it is also possible to copy files and directories from your workstation or laptop to the remote machine:&lt;br /&gt;
&lt;br /&gt;
 scp -r projects.txt &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
 scp -r projects/ &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
And this will copy the directory 'projects' and all its content from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
&lt;br /&gt;
=== Windows ===&lt;br /&gt;
&lt;br /&gt;
There are several options for displaying graphical applications on a Windows system. A basic option is [http://www.straightrunning.com/XmingNotes/ Xming]. You can download it from: http://www.straightrunning.com/XmingNotes/ The site asks for a donation to use the latest version, but there is also a public domain version which can be used for free. Once you have installed it, you can find it in the usual place in the start menu. Before you log in to an hpc cluster and run graphical applications, you need to start Xming first. It will not show much, only a small X in the system tray once it's running. You also need to enable X forwarding in PuTTY. For that you need to click on the + next to SSH in the session panel (left part of the window), then click on X11. You'll see a line on the right side that says &amp;quot;Enable X11 forwarding&amp;quot;, click once on the white square next to that line and it will show a check mark. &lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_04.png|center]]&lt;br /&gt;
&lt;br /&gt;
Now you can continue logging in to the hpc cluster. To check if X forwarding works, type &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that will follow the mouse pointer.&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
[[File:Xeyes.png|center]]&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
An alternative option for displaying graphical applications is Exceed, this is a commercial package for which the TU Delft has a license. It is not available by default on the standard Windows systems, but it can be installed upon request if you contact the servicedesk.&lt;br /&gt;
&lt;br /&gt;
A second alternative is [https://mobaxterm.mobatek.net/ MobaXterm], this is an all-in-one solution containing an ssh client, X server and more. You can download a free version from: https://mobaxterm.mobatek.net/download.html The free version has some limitations, but it's still perfectly usable in our environment.&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
== How to access your Windows network shares ==&lt;br /&gt;
&lt;br /&gt;
=== Authenticate ===&lt;br /&gt;
&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
&lt;br /&gt;
 kinit&lt;br /&gt;
&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
=== Basic access ===&lt;br /&gt;
&lt;br /&gt;
Now just type:&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
&lt;br /&gt;
or&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-umbrella/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
&lt;br /&gt;
You can access your Windows home directory as well, the path is:&lt;br /&gt;
&lt;br /&gt;
 /tudelft/$USER/staff-homes/&lt;br /&gt;
&lt;br /&gt;
followed by the first letter of your surname and then your netid. For example, if your name would be John Smith and your netid jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
If you are a student, replace 'staff' with 'student' to access your files.&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
=== Important note ===&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your netid. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your netid yourself. If your netid would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well. Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own netid!&lt;br /&gt;
&lt;br /&gt;
'''If you use anthing other than your own netid you'll create a recipe for potential disaster!'''&lt;br /&gt;
&lt;br /&gt;
If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths. So &lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-group/&lt;br /&gt;
&lt;br /&gt;
only works if your netid is jsmith and&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/pbrown/staff-group/&lt;br /&gt;
&lt;br /&gt;
only works works if your netid is pbrown.&lt;br /&gt;
&lt;br /&gt;
If both netids have the same privileges for accessing the group volumes, then these commands give access to the same data.&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=286</id>
		<title>Remote Access</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=286"/>
		<updated>2025-01-20T15:10:10Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: &lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;Access to the HPC-clusters is limited from outside the TU Delft network for security reasons.&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
'''You can only log in when you are on campus via a wireless (eduroam) or wired network. When you are off campus, you can log in using eduVPN (Institute Access). As of 16 Januari you can no longer use to the Linux bastion hosts when you are not on campus. There is also no need to use the Linux bastion hosts when you are on campus or using an active eduVPN connection.'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
== Bastion ==&lt;br /&gt;
The TU Delft offers two bastion servers. These servers can be used to tunnel your traffic to the HPC.&lt;br /&gt;
*linux-bastion.tudelft.nl for employees or guests.&lt;br /&gt;
*student-linux.tudelft.nl for students.&lt;br /&gt;
&lt;br /&gt;
== Access to the commandline ==&lt;br /&gt;
The explanations below are the simplest way to connect to a console on an HPC. [[#Advanced configs|Advanced configs]] describes more complex configurations which save time when regularly using the connection or when you want to use [[#Linux|X-forwarding]]&lt;br /&gt;
=== Command line ===&lt;br /&gt;
When your desktop/laptop supports ssh from the console:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
If your SSH-client doesn't support Jump hosts, you can first connect to the bastion server and then to the HPC:&lt;br /&gt;
&lt;br /&gt;
 [user@home]$ ssh &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
 [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt; with your NetID (also remove the &amp;lt;&amp;gt;)&lt;br /&gt;
*When your a student replace &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl with &amp;lt;NetID&amp;gt;@student-linux.tudelft.nl&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net with the HPC you are connecting to.&lt;br /&gt;
&lt;br /&gt;
=== Advanced configs ===&lt;br /&gt;
==== Making OpenSSH more user-friendly ====&lt;br /&gt;
The OpenSSH clients can be configured using a configuration file in ~/.ssh/config (Linux, macOS) or ~\.ssh\config (Windows).&lt;br /&gt;
&lt;br /&gt;
===== Define connections =====&lt;br /&gt;
For the OpenSSH clients you can predefine often used connections by placing a section like the following in the configuration file on your local computer. Authentication on the bastion with keys is not possible, so password is used.&amp;lt;br&amp;gt;&lt;br /&gt;
Employees:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
Students:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
You can then connect simply by ssh bastion. For example:&lt;br /&gt;
  [user@home]$ ssh bastion&lt;br /&gt;
    NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
&lt;br /&gt;
===== SSH Proxy Support =====&lt;br /&gt;
Since this configuration depends on [[#Define connections|Define connections]] configure that first.&amp;lt;br&amp;gt;&lt;br /&gt;
To connect directly from your machine to an HPC cluster login node (without connecting to the bastion server first), create a connection via a proxy by adding the following lines to the configuration file ~/.ssh/config on your local computer:&lt;br /&gt;
&lt;br /&gt;
  Host hpcXX&lt;br /&gt;
  Hostname hpcXX.tudelft.net&lt;br /&gt;
  ProxyCommand ssh -W %h:%p bastion&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
&lt;br /&gt;
You can then simply use ssh hpcXX (or scp -p '&amp;lt;local file&amp;gt;' &amp;quot;hpcXX:/scratch/&amp;lt;NetID&amp;gt;/&amp;quot;) from your local computer.&lt;br /&gt;
&lt;br /&gt;
===== Session multiplexing =====&lt;br /&gt;
OpenSSH clients support multiple sessions over a single connection. This means you have to enter your password only once, to make the connection, and can then open multiple sessions (SSH, SCP and SFTP) to that computer over the same connection. This is especially convenient when connecting to the bastion server, where you have to type your password always.&lt;br /&gt;
&lt;br /&gt;
Note: SSH public key logins won't work (reliably, (because Kerberos authentication is required to access your home directory).&lt;br /&gt;
&lt;br /&gt;
Add the following section to the end of the configuration file on your local computer and on the bastion server:&lt;br /&gt;
&lt;br /&gt;
  Host *&lt;br /&gt;
  ControlMaster auto&lt;br /&gt;
  ControlPath /tmp/ssh-%r@%h:%p&lt;br /&gt;
&lt;br /&gt;
Windows users may need to adapt the ControlPath location to match Windows.&lt;br /&gt;
&lt;br /&gt;
You can use the exact same commands as above.&lt;br /&gt;
=== PuTTY ===&lt;br /&gt;
When your desktop/laptop doesn't support ssh from the console, you can use [https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html PuTTY] (it may already be installed).&amp;lt;br&amp;gt;&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students), and select the SSH protocol (port 22):&amp;lt;br&amp;gt;[[File:PuTTY-Hostname.png]]&lt;br /&gt;
*OPTIONAL: Connection&amp;gt;SSH Settings&amp;lt;br&amp;gt;Select Share SSH connections if possible (this means that you will not have to authenticate when you start a second session):&amp;lt;br&amp;gt;[[File:PuTTY-ShareConnection.png]]&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;To store these session settings, go back to the session tab, and save the settings under a session name. To activate a stored session, double-click on the session name:&amp;lt;br&amp;gt;[[File:PuTTY-SaveSession.png]]&lt;br /&gt;
Once you're connected to the bastion server you can use the locally available ssh to connect to a HPC:&lt;br /&gt;
   [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
     NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
     Welcome&lt;br /&gt;
   [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
=== MobaXterm ===&lt;br /&gt;
MobaXterm is a very versatile application. It can be used to connect to the commandline on HPC, display graphical applications and transfer files. It also has built-in support for using SSH-tunneling to connect through a bastion. You can download it from [https://mobaxterm.mobatek.net/ https://mobaxterm.mobatek.net/]. The '''free version''' has enough features for most users. The '''Portable edition''' can be run without installing (just unzip).&lt;br /&gt;
*Click the '''Session''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSession.png]]&lt;br /&gt;
*Click the '''SSH''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSSH.png]]&lt;br /&gt;
*Enter '''Remote host'''&amp;lt;br&amp;gt;Check '''Specify username'''&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;[[File:MobaXterm_Hostname.png]]&lt;br /&gt;
*tab: Network settings&amp;lt;br&amp;gt;Click '''SSH gateway (jump host)'''&amp;lt;br&amp;gt;[[File:MobaXterm_NewJumphost.png]]&lt;br /&gt;
*Enter linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;Click '''OK'''&amp;lt;br&amp;gt;[[File:MobaXterm_Jumphost.png]]&lt;br /&gt;
*Click '''OK''' in the session settings to start connecting&amp;lt;br&amp;gt;You will be asked for your password&amp;lt;br&amp;gt;Enter your password on the console of hpcXX&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
== File Transfer ==&lt;br /&gt;
The best method for transfering files depends on where you want to store your files:&lt;br /&gt;
#Files in your home directory on HPC are only available within HPC&lt;br /&gt;
#Files on Network Shares are available on other locations (such as a TU Delft workplace, weblogin.tudelft.nl or other HPC clusters)&amp;lt;br&amp;gt;Copying files to these shares is best done using the manuals on [https://webdata.tudelft.nl/ webdata.tudelft.nl]&lt;br /&gt;
&lt;br /&gt;
When you want to copy files to the HPC you can copy them from Network Shares&lt;br /&gt;
=== (Windows) Network Shares ===&lt;br /&gt;
&lt;br /&gt;
==== Authenticate ====&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
 kinit&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
==== Basic access ====&lt;br /&gt;
&lt;br /&gt;
Now just type (depending on which share you want to use):&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
 cd /tudelft/$USER/student-group/&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
 cd /tudelft/$USER/staff-umbrella/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
You can access your Windows home directory as well. The subdirectories of the home directories are followed by '''the first letter of your surname''' and then your NetID.&amp;lt;br&amp;gt;&lt;br /&gt;
For example, if your name would be John Smith and your NetID jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
==== Important note ====&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your NetID. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your NetID yourself. If your NetID would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well. Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own NetID!&amp;lt;br&amp;gt;&lt;br /&gt;
'''If you use anthing other than your own NetID you'll create a recipe for potential disaster!'''&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths.&amp;lt;br&amp;gt;&lt;br /&gt;
So &amp;lt;code&amp;gt;/tudelft/jsmith/staff-group/&amp;lt;/code&amp;gt; only works if your NetID is jsmith and &amp;lt;code&amp;gt;/tudelft/pbrown/staff-group/&amp;lt;/code&amp;gt; only works works if your NetID is pbrown.&amp;lt;br&amp;gt;&lt;br /&gt;
If both NetIDs have the same privileges for accessing the group volumes, then these commands give access to the same data.&lt;br /&gt;
&lt;br /&gt;
=== Data on HPC ===&lt;br /&gt;
If you want to transfer data directly to the HPC you will need some extra configuration&lt;br /&gt;
==== Command line ====&lt;br /&gt;
Please first configure [[#SSH Proxy Support|SSH Proxy Support]].&amp;lt;br&amp;gt;&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your remote home directory on HPC to the local home directory on your desktop or laptop. &lt;br /&gt;
 scp &amp;lt;NetID&amp;gt;@hpcXX:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
*Copy an entire directory with all its content, use the '-r' option&amp;lt;br&amp;gt;This will copy the directory 'projects' from your remote home directory on HPC to the local home directory on your desktop or laptop:&lt;br /&gt;
 scp -r &amp;lt;NetID&amp;gt;@hpcXX:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your local home directory on your desktop or laptop to your remote home directory on HPC. &lt;br /&gt;
 scp projects.txt &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
*Copy the directory 'projects' and all its content from your current working directory on your desktop or laptop to your remote home directory on HPC.&lt;br /&gt;
 scp -r projects/ &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
==== WinSCP ====&lt;br /&gt;
WinSCP is a simple tool that has the internal option to create a tunnel through  the bastion server. It can be downloaded from [https://winscp.net/eng/download.php https://winscp.net/eng/download.php]&lt;br /&gt;
*Session &amp;gt; New Session&lt;br /&gt;
[[File:WinSCP01.png]]&lt;br /&gt;
*Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;Advanced&amp;quot; button&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Connection &amp;gt; Tunnel&amp;lt;br&amp;gt;Select &amp;quot;Connect through SSH tunnel&amp;quot;&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;OK&amp;quot; button&lt;br /&gt;
[[File:WinSCP03.png]]&lt;br /&gt;
*Click &amp;quot;Save&amp;quot;&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Save session as site&amp;lt;br&amp;gt;You can accept the suggested name&lt;br /&gt;
[[File:WinSCP04.png]]&lt;br /&gt;
*You can now &amp;quot;Login&amp;quot;&amp;lt;br&amp;gt;You will be asked to supply your password twice: first for the bastion, second for the HPC&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
==== MobaXterm ====&lt;br /&gt;
MobaXterm can also be used for file transfers. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*Left Window&amp;lt;br&amp;gt;Download files with '''arrow down'''&amp;lt;br&amp;gt;Upload files with '''arrow up'''&amp;lt;br&amp;gt;[[File:MobaXterm_SFTP.png]]&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
=== Windows ===&lt;br /&gt;
MobaXterm can also be used for displaying graphical applications. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*In the commandline windows (right)&amp;lt;br&amp;gt;Test X-forwarding (Graphics) by running &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt;&amp;lt;br&amp;gt;You should see these eyes (following the cursor)&amp;lt;br&amp;gt;[[File:MobaXterm_Xeyes.png]]&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in&amp;lt;br&amp;gt;&lt;br /&gt;
The easiest way is:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
When you have configured [[#SSH_Proxy_Support|SSH Proxy Support]] you can connect with:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=285</id>
		<title>Remote Access</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Remote_Access&amp;diff=285"/>
		<updated>2025-01-20T15:09:46Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: &lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;Access to the HPC-clusters is limited from outside the TU Delft network for security reasons.&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
'''You can only log in when you are on campus via wireless (eduroam) or wired network. When you are off campus, you can log in using eduVPN (Institute Access). As of 16 Januari you can no longer use to the Linux bastion hosts when you are not on campus. There is also no need to use the Linux bastion hosts when you are on campus or using an active eduVPN connection.'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
== Bastion ==&lt;br /&gt;
The TU Delft offers two bastion servers. These servers can be used to tunnel your traffic to the HPC.&lt;br /&gt;
*linux-bastion.tudelft.nl for employees or guests.&lt;br /&gt;
*student-linux.tudelft.nl for students.&lt;br /&gt;
&lt;br /&gt;
== Access to the commandline ==&lt;br /&gt;
The explanations below are the simplest way to connect to a console on an HPC. [[#Advanced configs|Advanced configs]] describes more complex configurations which save time when regularly using the connection or when you want to use [[#Linux|X-forwarding]]&lt;br /&gt;
=== Command line ===&lt;br /&gt;
When your desktop/laptop supports ssh from the console:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
If your SSH-client doesn't support Jump hosts, you can first connect to the bastion server and then to the HPC:&lt;br /&gt;
&lt;br /&gt;
 [user@home]$ ssh &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
 [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt; with your NetID (also remove the &amp;lt;&amp;gt;)&lt;br /&gt;
*When your a student replace &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl with &amp;lt;NetID&amp;gt;@student-linux.tudelft.nl&lt;br /&gt;
*Replace &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net with the HPC you are connecting to.&lt;br /&gt;
&lt;br /&gt;
=== Advanced configs ===&lt;br /&gt;
==== Making OpenSSH more user-friendly ====&lt;br /&gt;
The OpenSSH clients can be configured using a configuration file in ~/.ssh/config (Linux, macOS) or ~\.ssh\config (Windows).&lt;br /&gt;
&lt;br /&gt;
===== Define connections =====&lt;br /&gt;
For the OpenSSH clients you can predefine often used connections by placing a section like the following in the configuration file on your local computer. Authentication on the bastion with keys is not possible, so password is used.&amp;lt;br&amp;gt;&lt;br /&gt;
Employees:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
Students:&lt;br /&gt;
  Host bastion&lt;br /&gt;
  Hostname linux-bastion.tudelft.nl&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
  PreferredAuthentications password&lt;br /&gt;
&lt;br /&gt;
You can then connect simply by ssh bastion. For example:&lt;br /&gt;
  [user@home]$ ssh bastion&lt;br /&gt;
    NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
&lt;br /&gt;
===== SSH Proxy Support =====&lt;br /&gt;
Since this configuration depends on [[#Define connections|Define connections]] configure that first.&amp;lt;br&amp;gt;&lt;br /&gt;
To connect directly from your machine to an HPC cluster login node (without connecting to the bastion server first), create a connection via a proxy by adding the following lines to the configuration file ~/.ssh/config on your local computer:&lt;br /&gt;
&lt;br /&gt;
  Host hpcXX&lt;br /&gt;
  Hostname hpcXX.tudelft.net&lt;br /&gt;
  ProxyCommand ssh -W %h:%p bastion&lt;br /&gt;
  User &amp;lt;NetID&amp;gt;&lt;br /&gt;
&lt;br /&gt;
You can then simply use ssh hpcXX (or scp -p '&amp;lt;local file&amp;gt;' &amp;quot;hpcXX:/scratch/&amp;lt;NetID&amp;gt;/&amp;quot;) from your local computer.&lt;br /&gt;
&lt;br /&gt;
===== Session multiplexing =====&lt;br /&gt;
OpenSSH clients support multiple sessions over a single connection. This means you have to enter your password only once, to make the connection, and can then open multiple sessions (SSH, SCP and SFTP) to that computer over the same connection. This is especially convenient when connecting to the bastion server, where you have to type your password always.&lt;br /&gt;
&lt;br /&gt;
Note: SSH public key logins won't work (reliably, (because Kerberos authentication is required to access your home directory).&lt;br /&gt;
&lt;br /&gt;
Add the following section to the end of the configuration file on your local computer and on the bastion server:&lt;br /&gt;
&lt;br /&gt;
  Host *&lt;br /&gt;
  ControlMaster auto&lt;br /&gt;
  ControlPath /tmp/ssh-%r@%h:%p&lt;br /&gt;
&lt;br /&gt;
Windows users may need to adapt the ControlPath location to match Windows.&lt;br /&gt;
&lt;br /&gt;
You can use the exact same commands as above.&lt;br /&gt;
=== PuTTY ===&lt;br /&gt;
When your desktop/laptop doesn't support ssh from the console, you can use [https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html PuTTY] (it may already be installed).&amp;lt;br&amp;gt;&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students), and select the SSH protocol (port 22):&amp;lt;br&amp;gt;[[File:PuTTY-Hostname.png]]&lt;br /&gt;
*OPTIONAL: Connection&amp;gt;SSH Settings&amp;lt;br&amp;gt;Select Share SSH connections if possible (this means that you will not have to authenticate when you start a second session):&amp;lt;br&amp;gt;[[File:PuTTY-ShareConnection.png]]&lt;br /&gt;
*Session Settings&amp;lt;br&amp;gt;To store these session settings, go back to the session tab, and save the settings under a session name. To activate a stored session, double-click on the session name:&amp;lt;br&amp;gt;[[File:PuTTY-SaveSession.png]]&lt;br /&gt;
Once you're connected to the bastion server you can use the locally available ssh to connect to a HPC:&lt;br /&gt;
   [NetID@bastion]$ ssh &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
     NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
     Welcome&lt;br /&gt;
   [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
=== MobaXterm ===&lt;br /&gt;
MobaXterm is a very versatile application. It can be used to connect to the commandline on HPC, display graphical applications and transfer files. It also has built-in support for using SSH-tunneling to connect through a bastion. You can download it from [https://mobaxterm.mobatek.net/ https://mobaxterm.mobatek.net/]. The '''free version''' has enough features for most users. The '''Portable edition''' can be run without installing (just unzip).&lt;br /&gt;
*Click the '''Session''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSession.png]]&lt;br /&gt;
*Click the '''SSH''' icon&amp;lt;br&amp;gt;[[File:MobaXterm_NewSSH.png]]&lt;br /&gt;
*Enter '''Remote host'''&amp;lt;br&amp;gt;Check '''Specify username'''&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;[[File:MobaXterm_Hostname.png]]&lt;br /&gt;
*tab: Network settings&amp;lt;br&amp;gt;Click '''SSH gateway (jump host)'''&amp;lt;br&amp;gt;[[File:MobaXterm_NewJumphost.png]]&lt;br /&gt;
*Enter linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Enter your NetID&amp;lt;br&amp;gt;Click '''OK'''&amp;lt;br&amp;gt;[[File:MobaXterm_Jumphost.png]]&lt;br /&gt;
*Click '''OK''' in the session settings to start connecting&amp;lt;br&amp;gt;You will be asked for your password&amp;lt;br&amp;gt;Enter your password on the console of hpcXX&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
== File Transfer ==&lt;br /&gt;
The best method for transfering files depends on where you want to store your files:&lt;br /&gt;
#Files in your home directory on HPC are only available within HPC&lt;br /&gt;
#Files on Network Shares are available on other locations (such as a TU Delft workplace, weblogin.tudelft.nl or other HPC clusters)&amp;lt;br&amp;gt;Copying files to these shares is best done using the manuals on [https://webdata.tudelft.nl/ webdata.tudelft.nl]&lt;br /&gt;
&lt;br /&gt;
When you want to copy files to the HPC you can copy them from Network Shares&lt;br /&gt;
=== (Windows) Network Shares ===&lt;br /&gt;
&lt;br /&gt;
==== Authenticate ====&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
 kinit&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
==== Basic access ====&lt;br /&gt;
&lt;br /&gt;
Now just type (depending on which share you want to use):&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
 cd /tudelft/$USER/student-group/&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
 cd /tudelft/$USER/staff-umbrella/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
You can access your Windows home directory as well. The subdirectories of the home directories are followed by '''the first letter of your surname''' and then your NetID.&amp;lt;br&amp;gt;&lt;br /&gt;
For example, if your name would be John Smith and your NetID jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
==== Important note ====&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your NetID. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your NetID yourself. If your NetID would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well. Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own NetID!&amp;lt;br&amp;gt;&lt;br /&gt;
'''If you use anthing other than your own NetID you'll create a recipe for potential disaster!'''&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths.&amp;lt;br&amp;gt;&lt;br /&gt;
So &amp;lt;code&amp;gt;/tudelft/jsmith/staff-group/&amp;lt;/code&amp;gt; only works if your NetID is jsmith and &amp;lt;code&amp;gt;/tudelft/pbrown/staff-group/&amp;lt;/code&amp;gt; only works works if your NetID is pbrown.&amp;lt;br&amp;gt;&lt;br /&gt;
If both NetIDs have the same privileges for accessing the group volumes, then these commands give access to the same data.&lt;br /&gt;
&lt;br /&gt;
=== Data on HPC ===&lt;br /&gt;
If you want to transfer data directly to the HPC you will need some extra configuration&lt;br /&gt;
==== Command line ====&lt;br /&gt;
Please first configure [[#SSH Proxy Support|SSH Proxy Support]].&amp;lt;br&amp;gt;&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh&amp;lt;br&amp;gt;&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your remote home directory on HPC to the local home directory on your desktop or laptop. &lt;br /&gt;
 scp &amp;lt;NetID&amp;gt;@hpcXX:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
*Copy an entire directory with all its content, use the '-r' option&amp;lt;br&amp;gt;This will copy the directory 'projects' from your remote home directory on HPC to the local home directory on your desktop or laptop:&lt;br /&gt;
 scp -r &amp;lt;NetID&amp;gt;@hpcXX:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
*Copy the file 'projects.txt' from your local home directory on your desktop or laptop to your remote home directory on HPC. &lt;br /&gt;
 scp projects.txt &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
*Copy the directory 'projects' and all its content from your current working directory on your desktop or laptop to your remote home directory on HPC.&lt;br /&gt;
 scp -r projects/ &amp;lt;NetID&amp;gt;@hpcXX:~/&lt;br /&gt;
&lt;br /&gt;
==== WinSCP ====&lt;br /&gt;
WinSCP is a simple tool that has the internal option to create a tunnel through  the bastion server. It can be downloaded from [https://winscp.net/eng/download.php https://winscp.net/eng/download.php]&lt;br /&gt;
*Session &amp;gt; New Session&lt;br /&gt;
[[File:WinSCP01.png]]&lt;br /&gt;
*Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;Advanced&amp;quot; button&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Connection &amp;gt; Tunnel&amp;lt;br&amp;gt;Select &amp;quot;Connect through SSH tunnel&amp;quot;&amp;lt;br&amp;gt;Fill in the name of the TU Delft SSH server linux-bastion.tudelft.nl (student-linux.tudelft.nl for students)&amp;lt;br&amp;gt;Fill in your NetID&amp;lt;br&amp;gt;Click the &amp;quot;OK&amp;quot; button&lt;br /&gt;
[[File:WinSCP03.png]]&lt;br /&gt;
*Click &amp;quot;Save&amp;quot;&lt;br /&gt;
[[File:WinSCP02.png]]&lt;br /&gt;
*Save session as site&amp;lt;br&amp;gt;You can accept the suggested name&lt;br /&gt;
[[File:WinSCP04.png]]&lt;br /&gt;
*You can now &amp;quot;Login&amp;quot;&amp;lt;br&amp;gt;You will be asked to supply your password twice: first for the bastion, second for the HPC&amp;lt;br&amp;gt;You may be presented with a question about accepting the server's host key (this is normal the first time)&lt;br /&gt;
&lt;br /&gt;
==== MobaXterm ====&lt;br /&gt;
MobaXterm can also be used for file transfers. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*Left Window&amp;lt;br&amp;gt;Download files with '''arrow down'''&amp;lt;br&amp;gt;Upload files with '''arrow up'''&amp;lt;br&amp;gt;[[File:MobaXterm_SFTP.png]]&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
=== Windows ===&lt;br /&gt;
MobaXterm can also be used for displaying graphical applications. Configuration is described in [[#MobaXterm|Access to the commandline -&amp;gt; MobaXterm]]&lt;br /&gt;
*In the commandline windows (right)&amp;lt;br&amp;gt;Test X-forwarding (Graphics) by running &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt;&amp;lt;br&amp;gt;You should see these eyes (following the cursor)&amp;lt;br&amp;gt;[[File:MobaXterm_Xeyes.png]]&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in&amp;lt;br&amp;gt;&lt;br /&gt;
The easiest way is:&lt;br /&gt;
 [user@home]$ ssh -J &amp;lt;NetID&amp;gt;@linux-bastion.tudelft.nl -X &amp;lt;NetID&amp;gt;@hpcXX.tudelft.net&lt;br /&gt;
   NetID@linux-bastion.tudelft.nl's password:&lt;br /&gt;
   NetID@hpcXX.tudelft.net's password: &lt;br /&gt;
   Welcome&lt;br /&gt;
 [NetID@hpcXX]$&lt;br /&gt;
&lt;br /&gt;
When you have configured [[#SSH_Proxy_Support|SSH Proxy Support]] you can connect with:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;NetID&amp;gt;@hpcXX&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;NetID&amp;gt;@hpcXX&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=284</id>
		<title>How to log in</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=284"/>
		<updated>2025-01-20T15:07:06Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* Basics */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== Basics ==&lt;br /&gt;
&lt;br /&gt;
In order to log in you need a valid [https://www.tudelft.nl/en/student/ict/ict-facilities/netid-account/ NetID] that is enabled for access to one of the clusters. To have your NetID enabled, ask the contact person for the cluster you want access to (See the [[Introduction]] page) or ask the secretary of the department or research group where you work, he or she can tell you who the contact person is for an hpc cluster.&lt;br /&gt;
&lt;br /&gt;
If you want to collaborate with someone from outside the TU Delft and you want that person to access the same cluster as you, then you can request a guest account at the servicedesk. Once the guest account is created, it can be enabled for access to one of the clusters. '''Do not share your own NetID with others! Your NetID is for personal use only, sharing it with other people is forbidden!'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are only accessible via ssh (secure shell). To log in you need an ssh client. If you use Windows, you can download and use [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY], Macintosh and Linux computers already have an ssh client built in.&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
'''You can only log in when you are on campus via wireless (eduroam) or wired network. When you are off campus, you can log in using eduVPN (Institute Access). As of 16 Januari you can no longer use to the Linux bastion hosts when you are not on campus. There is also no need to use the Linux bastion hosts when you are on campus or using an active eduVPN connection.'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
Every time you log in you'll see an motd (message of the day). Please pay attention to this, it may contain information about current events on the cluster. It also shows you an email address that you can use if there are problems or if you have questions. You can mail at any time you like, but you'll usually only get a reply during office hours, there is no 24/7 support on the hpc clusters.&lt;br /&gt;
&lt;br /&gt;
'''Please note:''' You have a local home directory on the cluster. No backups will be made of the home directories, if you make a mistake and loose a file, it is gone forever. '''Making backups of your important files on the hpc clusters is your own responsibility!'''&lt;br /&gt;
&lt;br /&gt;
== How to log in from Windows ==&lt;br /&gt;
&lt;br /&gt;
An often used ssh client for Windows is [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY]. If you use a Windows PC or a laptop that is provided and maintained by the TU Delft, it is already there, if you use your own Windows computer, you'll have to download and install it first from [http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html www.chiark.greenend.org.uk].&lt;br /&gt;
&lt;br /&gt;
Once you have started PuTTY you'll get a configuration window als shown.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can type the name of the cluster, for example 'hpc10.tudelft.net', in the field 'Host Name'. You can save the settings for a cluster by choosing a name for the session, enter it in the 'Saved Sessions' box and click 'Save'. The next time you open PuTTY the settings can be restored by clicking on the name and a click on 'Load'.&lt;br /&gt;
&lt;br /&gt;
By clicking 'Open' a connection is made to hpc10.tudelft.net. If this is the first time you are connecting to the cluster, PuTTY will warn you that you are connecting to a previously unknown host.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_02.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can confirm this connection by clicking 'yes' so PuTTY will store the host key in its database. You'll have to do this only once for every new server that you connect to.&lt;br /&gt;
&lt;br /&gt;
After you have accepted the connection, PuTTY opens a console and asks for your loginname. Once you have entered your NetID and your password, you are logged in to the cluster.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_03.png|center]]&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from Windows ==&lt;br /&gt;
&lt;br /&gt;
WinSCP is a freeware ftp client with support for secure file protocols sftp and scp. The login screen depicted below shows the settings for access within the University.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
After clicking on 'Login' a warning about host keys will appear, similar to PuTTY. If it is the first time, you can safely click on 'yes' to accept.&lt;br /&gt;
&lt;br /&gt;
Press 'Login'. If everything is setup correctly you are using the ssh tunnel the PuTTY bastion session has built and you are logged in on the cluster.&lt;br /&gt;
&lt;br /&gt;
When logged in, you will see the commander-like interface below.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_02.png|center|530px]]&lt;br /&gt;
&lt;br /&gt;
The left side displays the files in your home directory on the windows system. The right side displays the files on the cluster. You can drag and drop files between the two windows or select files with the mouse and press 'F5' for transfer to the other window.&lt;br /&gt;
&lt;br /&gt;
== How to log in from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
Mac OS X is in fact a full blown UNIX system under the hood. A terminal program comes standard with OS X, navigate with the finder to Applications -&amp;gt; Utilities to find it, the program is called 'Terminal'. There are many different flavours of Linux, each with their own desktop environments and customizations. Nearly every Linux system comes with a terminal program of some sort. Find the terminal program on your system and run it, once you get a command prompt type:&lt;br /&gt;
&lt;br /&gt;
 ssh &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
Where &amp;lt;netid&amp;gt; is of course your own netid.&lt;br /&gt;
&lt;br /&gt;
Upon first access, ssh requires you to check if the host key is correct. Since this is the first time you connect to the system, it's fine to accept the host key by pressing 'y'.&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh:&lt;br /&gt;
&lt;br /&gt;
 scp &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your remote home directory (folder) to the local home directory on your workstation or laptop. &lt;br /&gt;
&lt;br /&gt;
If you want to copy an entire directory with all its content, use the '-r' option, like this:&lt;br /&gt;
&lt;br /&gt;
 scp -r &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
This will copy the directory 'projects' from your remote home directory to the local home directory on your workstation or laptop.&lt;br /&gt;
&lt;br /&gt;
Of course, it is also possible to copy files and directories from your workstation or laptop to the remote machine:&lt;br /&gt;
&lt;br /&gt;
 scp -r projects.txt &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
 scp -r projects/ &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
And this will copy the directory 'projects' and all its content from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
&lt;br /&gt;
=== Windows ===&lt;br /&gt;
&lt;br /&gt;
There are several options for displaying graphical applications on a Windows system. A basic option is [http://www.straightrunning.com/XmingNotes/ Xming]. You can download it from: http://www.straightrunning.com/XmingNotes/ The site asks for a donation to use the latest version, but there is also a public domain version which can be used for free. Once you have installed it, you can find it in the usual place in the start menu. Before you log in to an hpc cluster and run graphical applications, you need to start Xming first. It will not show much, only a small X in the system tray once it's running. You also need to enable X forwarding in PuTTY. For that you need to click on the + next to SSH in the session panel (left part of the window), then click on X11. You'll see a line on the right side that says &amp;quot;Enable X11 forwarding&amp;quot;, click once on the white square next to that line and it will show a check mark. &lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_04.png|center]]&lt;br /&gt;
&lt;br /&gt;
Now you can continue logging in to the hpc cluster. To check if X forwarding works, type &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that will follow the mouse pointer.&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
[[File:Xeyes.png|center]]&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
An alternative option for displaying graphical applications is Exceed, this is a commercial package for which the TU Delft has a license. It is not available by default on the standard Windows systems, but it can be installed upon request if you contact the servicedesk.&lt;br /&gt;
&lt;br /&gt;
A second alternative is [https://mobaxterm.mobatek.net/ MobaXterm], this is an all-in-one solution containing an ssh client, X server and more. You can download a free version from: https://mobaxterm.mobatek.net/download.html The free version has some limitations, but it's still perfectly usable in our environment.&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
== How to access your Windows network shares ==&lt;br /&gt;
&lt;br /&gt;
=== Authenticate ===&lt;br /&gt;
&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
&lt;br /&gt;
 kinit&lt;br /&gt;
&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
=== Basic access ===&lt;br /&gt;
&lt;br /&gt;
Now just type:&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
&lt;br /&gt;
or&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-umbrella/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
&lt;br /&gt;
You can access your Windows home directory as well, the path is:&lt;br /&gt;
&lt;br /&gt;
 /tudelft/$USER/staff-homes/&lt;br /&gt;
&lt;br /&gt;
followed by the first letter of your surname and then your netid. For example, if your name would be John Smith and your netid jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
If you are a student, replace 'staff' with 'student' to access your files.&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
=== Important note ===&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your netid. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your netid yourself. If your netid would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well. Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own netid!&lt;br /&gt;
&lt;br /&gt;
'''If you use anthing other than your own netid you'll create a recipe for potential disaster!'''&lt;br /&gt;
&lt;br /&gt;
If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths. So &lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-group/&lt;br /&gt;
&lt;br /&gt;
only works if your netid is jsmith and&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/pbrown/staff-group/&lt;br /&gt;
&lt;br /&gt;
only works works if your netid is pbrown.&lt;br /&gt;
&lt;br /&gt;
If both netids have the same privileges for accessing the group volumes, then these commands give access to the same data.&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=283</id>
		<title>How to log in</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=283"/>
		<updated>2025-01-20T14:53:39Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* Basics */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== Basics ==&lt;br /&gt;
&lt;br /&gt;
In order to log in you need a valid [https://www.tudelft.nl/en/student/ict/ict-facilities/netid-account/ NetID] that is enabled for access to one of the clusters. To have your NetID enabled, ask the contact person for the cluster you want access to (See the [[Introduction]] page) or ask the secretary of the department or research group where you work, he or she can tell you who the contact person is for an hpc cluster.&lt;br /&gt;
&lt;br /&gt;
If you want to collaborate with someone from outside the TU Delft and you want that person to access the same cluster as you, then you can request a guest account at the servicedesk. Once the guest account is created, it can be enabled for access to one of the clusters. '''Do not share your own NetID with others! Your NetID is for personal use only, sharing it with other people is forbidden!'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are only accessible via ssh (secure shell). To log in you need an ssh client. If you use Windows, you can download and use [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY], Macintosh and Linux computers already have an ssh client built in.&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
'''You can only log in when you are on campus via wireless (eduroam) or wired network. When you are off campus, you can log in using eduVPN (Institute Access). As of 16 Januari you can no longer use to the Linux bastion hosts when you are not on campus.'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
Every time you log in you'll see an motd (message of the day). Please pay attention to this, it may contain information about current events on the cluster. It also shows you an email address that you can use if there are problems or if you have questions. You can mail at any time you like, but you'll usually only get a reply during office hours, there is no 24/7 support on the hpc clusters.&lt;br /&gt;
&lt;br /&gt;
'''Please note:''' You have a local home directory on the cluster. No backups will be made of the home directories, if you make a mistake and loose a file, it is gone forever. '''Making backups of your important files on the hpc clusters is your own responsibility!'''&lt;br /&gt;
&lt;br /&gt;
== How to log in from Windows ==&lt;br /&gt;
&lt;br /&gt;
An often used ssh client for Windows is [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY]. If you use a Windows PC or a laptop that is provided and maintained by the TU Delft, it is already there, if you use your own Windows computer, you'll have to download and install it first from [http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html www.chiark.greenend.org.uk].&lt;br /&gt;
&lt;br /&gt;
Once you have started PuTTY you'll get a configuration window als shown.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can type the name of the cluster, for example 'hpc10.tudelft.net', in the field 'Host Name'. You can save the settings for a cluster by choosing a name for the session, enter it in the 'Saved Sessions' box and click 'Save'. The next time you open PuTTY the settings can be restored by clicking on the name and a click on 'Load'.&lt;br /&gt;
&lt;br /&gt;
By clicking 'Open' a connection is made to hpc10.tudelft.net. If this is the first time you are connecting to the cluster, PuTTY will warn you that you are connecting to a previously unknown host.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_02.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can confirm this connection by clicking 'yes' so PuTTY will store the host key in its database. You'll have to do this only once for every new server that you connect to.&lt;br /&gt;
&lt;br /&gt;
After you have accepted the connection, PuTTY opens a console and asks for your loginname. Once you have entered your NetID and your password, you are logged in to the cluster.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_03.png|center]]&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from Windows ==&lt;br /&gt;
&lt;br /&gt;
WinSCP is a freeware ftp client with support for secure file protocols sftp and scp. The login screen depicted below shows the settings for access within the University.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
After clicking on 'Login' a warning about host keys will appear, similar to PuTTY. If it is the first time, you can safely click on 'yes' to accept.&lt;br /&gt;
&lt;br /&gt;
Press 'Login'. If everything is setup correctly you are using the ssh tunnel the PuTTY bastion session has built and you are logged in on the cluster.&lt;br /&gt;
&lt;br /&gt;
When logged in, you will see the commander-like interface below.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_02.png|center|530px]]&lt;br /&gt;
&lt;br /&gt;
The left side displays the files in your home directory on the windows system. The right side displays the files on the cluster. You can drag and drop files between the two windows or select files with the mouse and press 'F5' for transfer to the other window.&lt;br /&gt;
&lt;br /&gt;
== How to log in from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
Mac OS X is in fact a full blown UNIX system under the hood. A terminal program comes standard with OS X, navigate with the finder to Applications -&amp;gt; Utilities to find it, the program is called 'Terminal'. There are many different flavours of Linux, each with their own desktop environments and customizations. Nearly every Linux system comes with a terminal program of some sort. Find the terminal program on your system and run it, once you get a command prompt type:&lt;br /&gt;
&lt;br /&gt;
 ssh &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
Where &amp;lt;netid&amp;gt; is of course your own netid.&lt;br /&gt;
&lt;br /&gt;
Upon first access, ssh requires you to check if the host key is correct. Since this is the first time you connect to the system, it's fine to accept the host key by pressing 'y'.&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh:&lt;br /&gt;
&lt;br /&gt;
 scp &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your remote home directory (folder) to the local home directory on your workstation or laptop. &lt;br /&gt;
&lt;br /&gt;
If you want to copy an entire directory with all its content, use the '-r' option, like this:&lt;br /&gt;
&lt;br /&gt;
 scp -r &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
This will copy the directory 'projects' from your remote home directory to the local home directory on your workstation or laptop.&lt;br /&gt;
&lt;br /&gt;
Of course, it is also possible to copy files and directories from your workstation or laptop to the remote machine:&lt;br /&gt;
&lt;br /&gt;
 scp -r projects.txt &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
 scp -r projects/ &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
And this will copy the directory 'projects' and all its content from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
&lt;br /&gt;
=== Windows ===&lt;br /&gt;
&lt;br /&gt;
There are several options for displaying graphical applications on a Windows system. A basic option is [http://www.straightrunning.com/XmingNotes/ Xming]. You can download it from: http://www.straightrunning.com/XmingNotes/ The site asks for a donation to use the latest version, but there is also a public domain version which can be used for free. Once you have installed it, you can find it in the usual place in the start menu. Before you log in to an hpc cluster and run graphical applications, you need to start Xming first. It will not show much, only a small X in the system tray once it's running. You also need to enable X forwarding in PuTTY. For that you need to click on the + next to SSH in the session panel (left part of the window), then click on X11. You'll see a line on the right side that says &amp;quot;Enable X11 forwarding&amp;quot;, click once on the white square next to that line and it will show a check mark. &lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_04.png|center]]&lt;br /&gt;
&lt;br /&gt;
Now you can continue logging in to the hpc cluster. To check if X forwarding works, type &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that will follow the mouse pointer.&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
[[File:Xeyes.png|center]]&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
An alternative option for displaying graphical applications is Exceed, this is a commercial package for which the TU Delft has a license. It is not available by default on the standard Windows systems, but it can be installed upon request if you contact the servicedesk.&lt;br /&gt;
&lt;br /&gt;
A second alternative is [https://mobaxterm.mobatek.net/ MobaXterm], this is an all-in-one solution containing an ssh client, X server and more. You can download a free version from: https://mobaxterm.mobatek.net/download.html The free version has some limitations, but it's still perfectly usable in our environment.&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
== How to access your Windows network shares ==&lt;br /&gt;
&lt;br /&gt;
=== Authenticate ===&lt;br /&gt;
&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
&lt;br /&gt;
 kinit&lt;br /&gt;
&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
=== Basic access ===&lt;br /&gt;
&lt;br /&gt;
Now just type:&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
&lt;br /&gt;
or&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-umbrella/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
&lt;br /&gt;
You can access your Windows home directory as well, the path is:&lt;br /&gt;
&lt;br /&gt;
 /tudelft/$USER/staff-homes/&lt;br /&gt;
&lt;br /&gt;
followed by the first letter of your surname and then your netid. For example, if your name would be John Smith and your netid jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
If you are a student, replace 'staff' with 'student' to access your files.&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
=== Important note ===&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your netid. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your netid yourself. If your netid would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well. Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own netid!&lt;br /&gt;
&lt;br /&gt;
'''If you use anthing other than your own netid you'll create a recipe for potential disaster!'''&lt;br /&gt;
&lt;br /&gt;
If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths. So &lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-group/&lt;br /&gt;
&lt;br /&gt;
only works if your netid is jsmith and&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/pbrown/staff-group/&lt;br /&gt;
&lt;br /&gt;
only works works if your netid is pbrown.&lt;br /&gt;
&lt;br /&gt;
If both netids have the same privileges for accessing the group volumes, then these commands give access to the same data.&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=282</id>
		<title>How to log in</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=282"/>
		<updated>2025-01-20T14:43:06Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* Basics */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== Basics ==&lt;br /&gt;
&lt;br /&gt;
In order to log in you need a valid [https://www.tudelft.nl/en/student/ict/ict-facilities/netid-account/ NetID] that is enabled for access to one of the clusters. To have your NetID enabled, ask the contact person for the cluster you want access to (See the [[Introduction]] page) or ask the secretary of the department or research group where you work, he or she can tell you who the contact person is for an hpc cluster.&lt;br /&gt;
&lt;br /&gt;
If you want to collaborate with someone from outside the TU Delft and you want that person to access the same cluster as you, then you can request a guest account at the servicedesk. Once the guest account is created, it can be enabled for access to one of the clusters. '''Do not share your own NetID with others! Your NetID is for personal use only, sharing it with other people is forbidden!'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are only accessible via ssh (secure shell). To log in you need an ssh client. If you use Windows, you can download and use [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY], Macintosh and Linux computers already have an ssh client built in.&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
'''You can only log in when you are on campus via eduroam or wired network. When you are off campus, you can log in using eduVPN (Institute Access). As of 16 Januari you can no longer use to the Linux bastion hosts when you are not on campus.'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
Every time you log in you'll see an motd (message of the day). Please pay attention to this, it may contain information about current events on the cluster. It also shows you an email address that you can use if there are problems or if you have questions. You can mail at any time you like, but you'll usually only get a reply during office hours, there is no 24/7 support on the hpc clusters.&lt;br /&gt;
&lt;br /&gt;
'''Please note:''' You have a local home directory on the cluster. No backups will be made of the home directories, if you make a mistake and loose a file, it is gone forever. '''Making backups of your important files on the hpc clusters is your own responsibility!'''&lt;br /&gt;
&lt;br /&gt;
== How to log in from Windows ==&lt;br /&gt;
&lt;br /&gt;
An often used ssh client for Windows is [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY]. If you use a Windows PC or a laptop that is provided and maintained by the TU Delft, it is already there, if you use your own Windows computer, you'll have to download and install it first from [http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html www.chiark.greenend.org.uk].&lt;br /&gt;
&lt;br /&gt;
Once you have started PuTTY you'll get a configuration window als shown.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can type the name of the cluster, for example 'hpc10.tudelft.net', in the field 'Host Name'. You can save the settings for a cluster by choosing a name for the session, enter it in the 'Saved Sessions' box and click 'Save'. The next time you open PuTTY the settings can be restored by clicking on the name and a click on 'Load'.&lt;br /&gt;
&lt;br /&gt;
By clicking 'Open' a connection is made to hpc10.tudelft.net. If this is the first time you are connecting to the cluster, PuTTY will warn you that you are connecting to a previously unknown host.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_02.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can confirm this connection by clicking 'yes' so PuTTY will store the host key in its database. You'll have to do this only once for every new server that you connect to.&lt;br /&gt;
&lt;br /&gt;
After you have accepted the connection, PuTTY opens a console and asks for your loginname. Once you have entered your NetID and your password, you are logged in to the cluster.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_03.png|center]]&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from Windows ==&lt;br /&gt;
&lt;br /&gt;
WinSCP is a freeware ftp client with support for secure file protocols sftp and scp. The login screen depicted below shows the settings for access within the University.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
After clicking on 'Login' a warning about host keys will appear, similar to PuTTY. If it is the first time, you can safely click on 'yes' to accept.&lt;br /&gt;
&lt;br /&gt;
Press 'Login'. If everything is setup correctly you are using the ssh tunnel the PuTTY bastion session has built and you are logged in on the cluster.&lt;br /&gt;
&lt;br /&gt;
When logged in, you will see the commander-like interface below.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_02.png|center|530px]]&lt;br /&gt;
&lt;br /&gt;
The left side displays the files in your home directory on the windows system. The right side displays the files on the cluster. You can drag and drop files between the two windows or select files with the mouse and press 'F5' for transfer to the other window.&lt;br /&gt;
&lt;br /&gt;
== How to log in from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
Mac OS X is in fact a full blown UNIX system under the hood. A terminal program comes standard with OS X, navigate with the finder to Applications -&amp;gt; Utilities to find it, the program is called 'Terminal'. There are many different flavours of Linux, each with their own desktop environments and customizations. Nearly every Linux system comes with a terminal program of some sort. Find the terminal program on your system and run it, once you get a command prompt type:&lt;br /&gt;
&lt;br /&gt;
 ssh &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
Where &amp;lt;netid&amp;gt; is of course your own netid.&lt;br /&gt;
&lt;br /&gt;
Upon first access, ssh requires you to check if the host key is correct. Since this is the first time you connect to the system, it's fine to accept the host key by pressing 'y'.&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh:&lt;br /&gt;
&lt;br /&gt;
 scp &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your remote home directory (folder) to the local home directory on your workstation or laptop. &lt;br /&gt;
&lt;br /&gt;
If you want to copy an entire directory with all its content, use the '-r' option, like this:&lt;br /&gt;
&lt;br /&gt;
 scp -r &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
This will copy the directory 'projects' from your remote home directory to the local home directory on your workstation or laptop.&lt;br /&gt;
&lt;br /&gt;
Of course, it is also possible to copy files and directories from your workstation or laptop to the remote machine:&lt;br /&gt;
&lt;br /&gt;
 scp -r projects.txt &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
 scp -r projects/ &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
And this will copy the directory 'projects' and all its content from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
&lt;br /&gt;
=== Windows ===&lt;br /&gt;
&lt;br /&gt;
There are several options for displaying graphical applications on a Windows system. A basic option is [http://www.straightrunning.com/XmingNotes/ Xming]. You can download it from: http://www.straightrunning.com/XmingNotes/ The site asks for a donation to use the latest version, but there is also a public domain version which can be used for free. Once you have installed it, you can find it in the usual place in the start menu. Before you log in to an hpc cluster and run graphical applications, you need to start Xming first. It will not show much, only a small X in the system tray once it's running. You also need to enable X forwarding in PuTTY. For that you need to click on the + next to SSH in the session panel (left part of the window), then click on X11. You'll see a line on the right side that says &amp;quot;Enable X11 forwarding&amp;quot;, click once on the white square next to that line and it will show a check mark. &lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_04.png|center]]&lt;br /&gt;
&lt;br /&gt;
Now you can continue logging in to the hpc cluster. To check if X forwarding works, type &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that will follow the mouse pointer.&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
[[File:Xeyes.png|center]]&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
An alternative option for displaying graphical applications is Exceed, this is a commercial package for which the TU Delft has a license. It is not available by default on the standard Windows systems, but it can be installed upon request if you contact the servicedesk.&lt;br /&gt;
&lt;br /&gt;
A second alternative is [https://mobaxterm.mobatek.net/ MobaXterm], this is an all-in-one solution containing an ssh client, X server and more. You can download a free version from: https://mobaxterm.mobatek.net/download.html The free version has some limitations, but it's still perfectly usable in our environment.&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
== How to access your Windows network shares ==&lt;br /&gt;
&lt;br /&gt;
=== Authenticate ===&lt;br /&gt;
&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
&lt;br /&gt;
 kinit&lt;br /&gt;
&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
=== Basic access ===&lt;br /&gt;
&lt;br /&gt;
Now just type:&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
&lt;br /&gt;
or&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-umbrella/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
&lt;br /&gt;
You can access your Windows home directory as well, the path is:&lt;br /&gt;
&lt;br /&gt;
 /tudelft/$USER/staff-homes/&lt;br /&gt;
&lt;br /&gt;
followed by the first letter of your surname and then your netid. For example, if your name would be John Smith and your netid jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
If you are a student, replace 'staff' with 'student' to access your files.&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
=== Important note ===&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your netid. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your netid yourself. If your netid would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well. Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own netid!&lt;br /&gt;
&lt;br /&gt;
'''If you use anthing other than your own netid you'll create a recipe for potential disaster!'''&lt;br /&gt;
&lt;br /&gt;
If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths. So &lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-group/&lt;br /&gt;
&lt;br /&gt;
only works if your netid is jsmith and&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/pbrown/staff-group/&lt;br /&gt;
&lt;br /&gt;
only works works if your netid is pbrown.&lt;br /&gt;
&lt;br /&gt;
If both netids have the same privileges for accessing the group volumes, then these commands give access to the same data.&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=281</id>
		<title>How to log in</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=281"/>
		<updated>2025-01-20T14:42:36Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* Basics */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== Basics ==&lt;br /&gt;
&lt;br /&gt;
In order to log in you need a valid [https://www.tudelft.nl/en/student/ict/ict-facilities/netid-account/ NetID] that is enabled for access to one of the clusters. To have your NetID enabled, ask the contact person for the cluster you want access to (See the [[Introduction]] page) or ask the secretary of the department or research group where you work, he or she can tell you who the contact person is for an hpc cluster.&lt;br /&gt;
&lt;br /&gt;
If you want to collaborate with someone from outside the TU Delft and you want that person to access the same cluster as you, then you can request a guest account at the servicedesk. Once the guest account is created, it can be enabled for access to one of the clusters. '''Do not share your own NetID with others! Your NetID is for personal use only, sharing it with other people is forbidden!'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are only accessible via ssh (secure shell). To log in you need an ssh client. If you use Windows, you can download and use [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY], Macintosh and Linux computers already have an ssh client built in.&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
'''You can only log in when you are on campus via eduroam or wired network. When you are off campus, you can log in using eduVPN (Institute Access). As of 16 Januari you can no longer log in to the Linux bastion hosts when you are not on campus.'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
Every time you log in you'll see an motd (message of the day). Please pay attention to this, it may contain information about current events on the cluster. It also shows you an email address that you can use if there are problems or if you have questions. You can mail at any time you like, but you'll usually only get a reply during office hours, there is no 24/7 support on the hpc clusters.&lt;br /&gt;
&lt;br /&gt;
'''Please note:''' You have a local home directory on the cluster. No backups will be made of the home directories, if you make a mistake and loose a file, it is gone forever. '''Making backups of your important files on the hpc clusters is your own responsibility!'''&lt;br /&gt;
&lt;br /&gt;
== How to log in from Windows ==&lt;br /&gt;
&lt;br /&gt;
An often used ssh client for Windows is [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY]. If you use a Windows PC or a laptop that is provided and maintained by the TU Delft, it is already there, if you use your own Windows computer, you'll have to download and install it first from [http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html www.chiark.greenend.org.uk].&lt;br /&gt;
&lt;br /&gt;
Once you have started PuTTY you'll get a configuration window als shown.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can type the name of the cluster, for example 'hpc10.tudelft.net', in the field 'Host Name'. You can save the settings for a cluster by choosing a name for the session, enter it in the 'Saved Sessions' box and click 'Save'. The next time you open PuTTY the settings can be restored by clicking on the name and a click on 'Load'.&lt;br /&gt;
&lt;br /&gt;
By clicking 'Open' a connection is made to hpc10.tudelft.net. If this is the first time you are connecting to the cluster, PuTTY will warn you that you are connecting to a previously unknown host.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_02.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can confirm this connection by clicking 'yes' so PuTTY will store the host key in its database. You'll have to do this only once for every new server that you connect to.&lt;br /&gt;
&lt;br /&gt;
After you have accepted the connection, PuTTY opens a console and asks for your loginname. Once you have entered your NetID and your password, you are logged in to the cluster.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_03.png|center]]&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from Windows ==&lt;br /&gt;
&lt;br /&gt;
WinSCP is a freeware ftp client with support for secure file protocols sftp and scp. The login screen depicted below shows the settings for access within the University.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
After clicking on 'Login' a warning about host keys will appear, similar to PuTTY. If it is the first time, you can safely click on 'yes' to accept.&lt;br /&gt;
&lt;br /&gt;
Press 'Login'. If everything is setup correctly you are using the ssh tunnel the PuTTY bastion session has built and you are logged in on the cluster.&lt;br /&gt;
&lt;br /&gt;
When logged in, you will see the commander-like interface below.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_02.png|center|530px]]&lt;br /&gt;
&lt;br /&gt;
The left side displays the files in your home directory on the windows system. The right side displays the files on the cluster. You can drag and drop files between the two windows or select files with the mouse and press 'F5' for transfer to the other window.&lt;br /&gt;
&lt;br /&gt;
== How to log in from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
Mac OS X is in fact a full blown UNIX system under the hood. A terminal program comes standard with OS X, navigate with the finder to Applications -&amp;gt; Utilities to find it, the program is called 'Terminal'. There are many different flavours of Linux, each with their own desktop environments and customizations. Nearly every Linux system comes with a terminal program of some sort. Find the terminal program on your system and run it, once you get a command prompt type:&lt;br /&gt;
&lt;br /&gt;
 ssh &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
Where &amp;lt;netid&amp;gt; is of course your own netid.&lt;br /&gt;
&lt;br /&gt;
Upon first access, ssh requires you to check if the host key is correct. Since this is the first time you connect to the system, it's fine to accept the host key by pressing 'y'.&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh:&lt;br /&gt;
&lt;br /&gt;
 scp &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your remote home directory (folder) to the local home directory on your workstation or laptop. &lt;br /&gt;
&lt;br /&gt;
If you want to copy an entire directory with all its content, use the '-r' option, like this:&lt;br /&gt;
&lt;br /&gt;
 scp -r &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
This will copy the directory 'projects' from your remote home directory to the local home directory on your workstation or laptop.&lt;br /&gt;
&lt;br /&gt;
Of course, it is also possible to copy files and directories from your workstation or laptop to the remote machine:&lt;br /&gt;
&lt;br /&gt;
 scp -r projects.txt &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
 scp -r projects/ &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
And this will copy the directory 'projects' and all its content from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
&lt;br /&gt;
=== Windows ===&lt;br /&gt;
&lt;br /&gt;
There are several options for displaying graphical applications on a Windows system. A basic option is [http://www.straightrunning.com/XmingNotes/ Xming]. You can download it from: http://www.straightrunning.com/XmingNotes/ The site asks for a donation to use the latest version, but there is also a public domain version which can be used for free. Once you have installed it, you can find it in the usual place in the start menu. Before you log in to an hpc cluster and run graphical applications, you need to start Xming first. It will not show much, only a small X in the system tray once it's running. You also need to enable X forwarding in PuTTY. For that you need to click on the + next to SSH in the session panel (left part of the window), then click on X11. You'll see a line on the right side that says &amp;quot;Enable X11 forwarding&amp;quot;, click once on the white square next to that line and it will show a check mark. &lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_04.png|center]]&lt;br /&gt;
&lt;br /&gt;
Now you can continue logging in to the hpc cluster. To check if X forwarding works, type &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that will follow the mouse pointer.&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
[[File:Xeyes.png|center]]&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
An alternative option for displaying graphical applications is Exceed, this is a commercial package for which the TU Delft has a license. It is not available by default on the standard Windows systems, but it can be installed upon request if you contact the servicedesk.&lt;br /&gt;
&lt;br /&gt;
A second alternative is [https://mobaxterm.mobatek.net/ MobaXterm], this is an all-in-one solution containing an ssh client, X server and more. You can download a free version from: https://mobaxterm.mobatek.net/download.html The free version has some limitations, but it's still perfectly usable in our environment.&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
== How to access your Windows network shares ==&lt;br /&gt;
&lt;br /&gt;
=== Authenticate ===&lt;br /&gt;
&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
&lt;br /&gt;
 kinit&lt;br /&gt;
&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
=== Basic access ===&lt;br /&gt;
&lt;br /&gt;
Now just type:&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
&lt;br /&gt;
or&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-umbrella/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
&lt;br /&gt;
You can access your Windows home directory as well, the path is:&lt;br /&gt;
&lt;br /&gt;
 /tudelft/$USER/staff-homes/&lt;br /&gt;
&lt;br /&gt;
followed by the first letter of your surname and then your netid. For example, if your name would be John Smith and your netid jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
If you are a student, replace 'staff' with 'student' to access your files.&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
=== Important note ===&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your netid. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your netid yourself. If your netid would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well. Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own netid!&lt;br /&gt;
&lt;br /&gt;
'''If you use anthing other than your own netid you'll create a recipe for potential disaster!'''&lt;br /&gt;
&lt;br /&gt;
If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths. So &lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-group/&lt;br /&gt;
&lt;br /&gt;
only works if your netid is jsmith and&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/pbrown/staff-group/&lt;br /&gt;
&lt;br /&gt;
only works works if your netid is pbrown.&lt;br /&gt;
&lt;br /&gt;
If both netids have the same privileges for accessing the group volumes, then these commands give access to the same data.&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=280</id>
		<title>How to log in</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=280"/>
		<updated>2025-01-20T14:41:30Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* Basics */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== Basics ==&lt;br /&gt;
&lt;br /&gt;
In order to log in you need a valid [https://www.tudelft.nl/en/student/ict/ict-facilities/netid-account/ NetID] that is enabled for access to one of the clusters. To have your NetID enabled, ask the contact person for the cluster you want access to (See the [[Introduction]] page) or ask the secretary of the department or research group where you work, he or she can tell you who the contact person is for an hpc cluster.&lt;br /&gt;
&lt;br /&gt;
If you want to collaborate with someone from outside the TU Delft and you want that person to access the same cluster as you, then you can request a guest account at the servicedesk. Once the guest account is created, it can be enabled for access to one of the clusters. '''Do not share your own NetID with others! Your NetID is for personal use only, sharing it with other people is forbidden!'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are only accessible via ssh (secure shell). To log in you need an ssh client. If you use Windows, you can download and use [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY], Macintosh and Linux computers already have an ssh client built in.&lt;br /&gt;
&lt;br /&gt;
'''You can only log in when you are on campus via eduroam or wired network. When you are off campus, you can log in using eduVPN (Institute Access). As of 16 Januari you can no longer log in to the Linux bastion hosts when you are not on campus.'''&lt;br /&gt;
&lt;br /&gt;
Every time you log in you'll see an motd (message of the day). Please pay attention to this, it may contain information about current events on the cluster. It also shows you an email address that you can use if there are problems or if you have questions. You can mail at any time you like, but you'll usually only get a reply during office hours, there is no 24/7 support on the hpc clusters.&lt;br /&gt;
&lt;br /&gt;
'''Please note:''' You have a local home directory on the cluster. No backups will be made of the home directories, if you make a mistake and loose a file, it is gone forever. '''Making backups of your important files on the hpc clusters is your own responsibility!'''&lt;br /&gt;
&lt;br /&gt;
== How to log in from Windows ==&lt;br /&gt;
&lt;br /&gt;
An often used ssh client for Windows is [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY]. If you use a Windows PC or a laptop that is provided and maintained by the TU Delft, it is already there, if you use your own Windows computer, you'll have to download and install it first from [http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html www.chiark.greenend.org.uk].&lt;br /&gt;
&lt;br /&gt;
Once you have started PuTTY you'll get a configuration window als shown.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can type the name of the cluster, for example 'hpc10.tudelft.net', in the field 'Host Name'. You can save the settings for a cluster by choosing a name for the session, enter it in the 'Saved Sessions' box and click 'Save'. The next time you open PuTTY the settings can be restored by clicking on the name and a click on 'Load'.&lt;br /&gt;
&lt;br /&gt;
By clicking 'Open' a connection is made to hpc10.tudelft.net. If this is the first time you are connecting to the cluster, PuTTY will warn you that you are connecting to a previously unknown host.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_02.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can confirm this connection by clicking 'yes' so PuTTY will store the host key in its database. You'll have to do this only once for every new server that you connect to.&lt;br /&gt;
&lt;br /&gt;
After you have accepted the connection, PuTTY opens a console and asks for your loginname. Once you have entered your NetID and your password, you are logged in to the cluster.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_03.png|center]]&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from Windows ==&lt;br /&gt;
&lt;br /&gt;
WinSCP is a freeware ftp client with support for secure file protocols sftp and scp. The login screen depicted below shows the settings for access within the University.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
After clicking on 'Login' a warning about host keys will appear, similar to PuTTY. If it is the first time, you can safely click on 'yes' to accept.&lt;br /&gt;
&lt;br /&gt;
Press 'Login'. If everything is setup correctly you are using the ssh tunnel the PuTTY bastion session has built and you are logged in on the cluster.&lt;br /&gt;
&lt;br /&gt;
When logged in, you will see the commander-like interface below.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_02.png|center|530px]]&lt;br /&gt;
&lt;br /&gt;
The left side displays the files in your home directory on the windows system. The right side displays the files on the cluster. You can drag and drop files between the two windows or select files with the mouse and press 'F5' for transfer to the other window.&lt;br /&gt;
&lt;br /&gt;
== How to log in from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
Mac OS X is in fact a full blown UNIX system under the hood. A terminal program comes standard with OS X, navigate with the finder to Applications -&amp;gt; Utilities to find it, the program is called 'Terminal'. There are many different flavours of Linux, each with their own desktop environments and customizations. Nearly every Linux system comes with a terminal program of some sort. Find the terminal program on your system and run it, once you get a command prompt type:&lt;br /&gt;
&lt;br /&gt;
 ssh &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
Where &amp;lt;netid&amp;gt; is of course your own netid.&lt;br /&gt;
&lt;br /&gt;
Upon first access, ssh requires you to check if the host key is correct. Since this is the first time you connect to the system, it's fine to accept the host key by pressing 'y'.&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh:&lt;br /&gt;
&lt;br /&gt;
 scp &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your remote home directory (folder) to the local home directory on your workstation or laptop. &lt;br /&gt;
&lt;br /&gt;
If you want to copy an entire directory with all its content, use the '-r' option, like this:&lt;br /&gt;
&lt;br /&gt;
 scp -r &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
This will copy the directory 'projects' from your remote home directory to the local home directory on your workstation or laptop.&lt;br /&gt;
&lt;br /&gt;
Of course, it is also possible to copy files and directories from your workstation or laptop to the remote machine:&lt;br /&gt;
&lt;br /&gt;
 scp -r projects.txt &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
 scp -r projects/ &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
And this will copy the directory 'projects' and all its content from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
&lt;br /&gt;
=== Windows ===&lt;br /&gt;
&lt;br /&gt;
There are several options for displaying graphical applications on a Windows system. A basic option is [http://www.straightrunning.com/XmingNotes/ Xming]. You can download it from: http://www.straightrunning.com/XmingNotes/ The site asks for a donation to use the latest version, but there is also a public domain version which can be used for free. Once you have installed it, you can find it in the usual place in the start menu. Before you log in to an hpc cluster and run graphical applications, you need to start Xming first. It will not show much, only a small X in the system tray once it's running. You also need to enable X forwarding in PuTTY. For that you need to click on the + next to SSH in the session panel (left part of the window), then click on X11. You'll see a line on the right side that says &amp;quot;Enable X11 forwarding&amp;quot;, click once on the white square next to that line and it will show a check mark. &lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_04.png|center]]&lt;br /&gt;
&lt;br /&gt;
Now you can continue logging in to the hpc cluster. To check if X forwarding works, type &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that will follow the mouse pointer.&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
[[File:Xeyes.png|center]]&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
An alternative option for displaying graphical applications is Exceed, this is a commercial package for which the TU Delft has a license. It is not available by default on the standard Windows systems, but it can be installed upon request if you contact the servicedesk.&lt;br /&gt;
&lt;br /&gt;
A second alternative is [https://mobaxterm.mobatek.net/ MobaXterm], this is an all-in-one solution containing an ssh client, X server and more. You can download a free version from: https://mobaxterm.mobatek.net/download.html The free version has some limitations, but it's still perfectly usable in our environment.&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
== How to access your Windows network shares ==&lt;br /&gt;
&lt;br /&gt;
=== Authenticate ===&lt;br /&gt;
&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
&lt;br /&gt;
 kinit&lt;br /&gt;
&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
=== Basic access ===&lt;br /&gt;
&lt;br /&gt;
Now just type:&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
&lt;br /&gt;
or&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-umbrella/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
&lt;br /&gt;
You can access your Windows home directory as well, the path is:&lt;br /&gt;
&lt;br /&gt;
 /tudelft/$USER/staff-homes/&lt;br /&gt;
&lt;br /&gt;
followed by the first letter of your surname and then your netid. For example, if your name would be John Smith and your netid jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
If you are a student, replace 'staff' with 'student' to access your files.&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
=== Important note ===&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your netid. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your netid yourself. If your netid would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well. Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own netid!&lt;br /&gt;
&lt;br /&gt;
'''If you use anthing other than your own netid you'll create a recipe for potential disaster!'''&lt;br /&gt;
&lt;br /&gt;
If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths. So &lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-group/&lt;br /&gt;
&lt;br /&gt;
only works if your netid is jsmith and&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/pbrown/staff-group/&lt;br /&gt;
&lt;br /&gt;
only works works if your netid is pbrown.&lt;br /&gt;
&lt;br /&gt;
If both netids have the same privileges for accessing the group volumes, then these commands give access to the same data.&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=279</id>
		<title>How to log in</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=279"/>
		<updated>2025-01-20T14:40:49Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* Basics */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== Basics ==&lt;br /&gt;
&lt;br /&gt;
In order to log in you need a valid [https://www.tudelft.nl/en/student/ict/ict-facilities/netid-account/ NetID] that is enabled for access to one of the clusters. To have your NetID enabled, ask the contact person for the cluster you want access to (See the [[Introduction]] page) or ask the secretary of the department or research group where you work, he or she can tell you who the contact person is for an hpc cluster.&lt;br /&gt;
&lt;br /&gt;
If you want to collaborate with someone from outside the TU Delft and you want that person to access the same cluster as you, then you can request a guest account at the servicedesk. Once the guest account is created, it can be enabled for access to one of the clusters. '''Do not share your own NetID with others! Your NetID is for personal use only, sharing it with other people is forbidden!'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are only accessible via ssh (secure shell). To log in you need an ssh client. If you use Windows, you can download and use [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY], Macintosh and Linux computers already have an ssh client built in.&lt;br /&gt;
&lt;br /&gt;
You can only log in when you are on campus via eduroam or wired network. When you are off campus, you can log in using eduVPN (Institute Access). As of 16 Januari you can no longer log in to the Linux bastion hosts when you are not on campus.&lt;br /&gt;
&lt;br /&gt;
Every time you log in you'll see an motd (message of the day). Please pay attention to this, it may contain information about current events on the cluster. It also shows you an email address that you can use if there are problems or if you have questions. You can mail at any time you like, but you'll usually only get a reply during office hours, there is no 24/7 support on the hpc clusters.&lt;br /&gt;
&lt;br /&gt;
'''Please note:''' You have a local home directory on the cluster. No backups will be made of the home directories, if you make a mistake and loose a file, it is gone forever. '''Making backups of your important files on the hpc clusters is your own responsibility!'''&lt;br /&gt;
&lt;br /&gt;
== How to log in from Windows ==&lt;br /&gt;
&lt;br /&gt;
An often used ssh client for Windows is [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY]. If you use a Windows PC or a laptop that is provided and maintained by the TU Delft, it is already there, if you use your own Windows computer, you'll have to download and install it first from [http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html www.chiark.greenend.org.uk].&lt;br /&gt;
&lt;br /&gt;
Once you have started PuTTY you'll get a configuration window als shown.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can type the name of the cluster, for example 'hpc10.tudelft.net', in the field 'Host Name'. You can save the settings for a cluster by choosing a name for the session, enter it in the 'Saved Sessions' box and click 'Save'. The next time you open PuTTY the settings can be restored by clicking on the name and a click on 'Load'.&lt;br /&gt;
&lt;br /&gt;
By clicking 'Open' a connection is made to hpc10.tudelft.net. If this is the first time you are connecting to the cluster, PuTTY will warn you that you are connecting to a previously unknown host.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_02.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can confirm this connection by clicking 'yes' so PuTTY will store the host key in its database. You'll have to do this only once for every new server that you connect to.&lt;br /&gt;
&lt;br /&gt;
After you have accepted the connection, PuTTY opens a console and asks for your loginname. Once you have entered your NetID and your password, you are logged in to the cluster.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_03.png|center]]&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from Windows ==&lt;br /&gt;
&lt;br /&gt;
WinSCP is a freeware ftp client with support for secure file protocols sftp and scp. The login screen depicted below shows the settings for access within the University.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
After clicking on 'Login' a warning about host keys will appear, similar to PuTTY. If it is the first time, you can safely click on 'yes' to accept.&lt;br /&gt;
&lt;br /&gt;
Press 'Login'. If everything is setup correctly you are using the ssh tunnel the PuTTY bastion session has built and you are logged in on the cluster.&lt;br /&gt;
&lt;br /&gt;
When logged in, you will see the commander-like interface below.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_02.png|center|530px]]&lt;br /&gt;
&lt;br /&gt;
The left side displays the files in your home directory on the windows system. The right side displays the files on the cluster. You can drag and drop files between the two windows or select files with the mouse and press 'F5' for transfer to the other window.&lt;br /&gt;
&lt;br /&gt;
== How to log in from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
Mac OS X is in fact a full blown UNIX system under the hood. A terminal program comes standard with OS X, navigate with the finder to Applications -&amp;gt; Utilities to find it, the program is called 'Terminal'. There are many different flavours of Linux, each with their own desktop environments and customizations. Nearly every Linux system comes with a terminal program of some sort. Find the terminal program on your system and run it, once you get a command prompt type:&lt;br /&gt;
&lt;br /&gt;
 ssh &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
Where &amp;lt;netid&amp;gt; is of course your own netid.&lt;br /&gt;
&lt;br /&gt;
Upon first access, ssh requires you to check if the host key is correct. Since this is the first time you connect to the system, it's fine to accept the host key by pressing 'y'.&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh:&lt;br /&gt;
&lt;br /&gt;
 scp &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your remote home directory (folder) to the local home directory on your workstation or laptop. &lt;br /&gt;
&lt;br /&gt;
If you want to copy an entire directory with all its content, use the '-r' option, like this:&lt;br /&gt;
&lt;br /&gt;
 scp -r &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
This will copy the directory 'projects' from your remote home directory to the local home directory on your workstation or laptop.&lt;br /&gt;
&lt;br /&gt;
Of course, it is also possible to copy files and directories from your workstation or laptop to the remote machine:&lt;br /&gt;
&lt;br /&gt;
 scp -r projects.txt &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
 scp -r projects/ &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
And this will copy the directory 'projects' and all its content from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
&lt;br /&gt;
=== Windows ===&lt;br /&gt;
&lt;br /&gt;
There are several options for displaying graphical applications on a Windows system. A basic option is [http://www.straightrunning.com/XmingNotes/ Xming]. You can download it from: http://www.straightrunning.com/XmingNotes/ The site asks for a donation to use the latest version, but there is also a public domain version which can be used for free. Once you have installed it, you can find it in the usual place in the start menu. Before you log in to an hpc cluster and run graphical applications, you need to start Xming first. It will not show much, only a small X in the system tray once it's running. You also need to enable X forwarding in PuTTY. For that you need to click on the + next to SSH in the session panel (left part of the window), then click on X11. You'll see a line on the right side that says &amp;quot;Enable X11 forwarding&amp;quot;, click once on the white square next to that line and it will show a check mark. &lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_04.png|center]]&lt;br /&gt;
&lt;br /&gt;
Now you can continue logging in to the hpc cluster. To check if X forwarding works, type &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that will follow the mouse pointer.&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
[[File:Xeyes.png|center]]&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
An alternative option for displaying graphical applications is Exceed, this is a commercial package for which the TU Delft has a license. It is not available by default on the standard Windows systems, but it can be installed upon request if you contact the servicedesk.&lt;br /&gt;
&lt;br /&gt;
A second alternative is [https://mobaxterm.mobatek.net/ MobaXterm], this is an all-in-one solution containing an ssh client, X server and more. You can download a free version from: https://mobaxterm.mobatek.net/download.html The free version has some limitations, but it's still perfectly usable in our environment.&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
== How to access your Windows network shares ==&lt;br /&gt;
&lt;br /&gt;
=== Authenticate ===&lt;br /&gt;
&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
&lt;br /&gt;
 kinit&lt;br /&gt;
&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
=== Basic access ===&lt;br /&gt;
&lt;br /&gt;
Now just type:&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
&lt;br /&gt;
or&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-umbrella/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
&lt;br /&gt;
You can access your Windows home directory as well, the path is:&lt;br /&gt;
&lt;br /&gt;
 /tudelft/$USER/staff-homes/&lt;br /&gt;
&lt;br /&gt;
followed by the first letter of your surname and then your netid. For example, if your name would be John Smith and your netid jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
If you are a student, replace 'staff' with 'student' to access your files.&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
=== Important note ===&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your netid. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your netid yourself. If your netid would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well. Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own netid!&lt;br /&gt;
&lt;br /&gt;
'''If you use anthing other than your own netid you'll create a recipe for potential disaster!'''&lt;br /&gt;
&lt;br /&gt;
If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths. So &lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-group/&lt;br /&gt;
&lt;br /&gt;
only works if your netid is jsmith and&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/pbrown/staff-group/&lt;br /&gt;
&lt;br /&gt;
only works works if your netid is pbrown.&lt;br /&gt;
&lt;br /&gt;
If both netids have the same privileges for accessing the group volumes, then these commands give access to the same data.&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Introduction&amp;diff=278</id>
		<title>Introduction</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Introduction&amp;diff=278"/>
		<updated>2024-03-22T13:20:11Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* About the TU Delft hpc clusters */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== About the TU Delft hpc clusters ==&lt;br /&gt;
&lt;br /&gt;
Several research groups at the TU Delft have exclusive access to hpc (high performance computing) clusters that are managed and maintained by the ICT Department. Currently there are ten, they are typically named hpcXX.tudelft.net where the XX part is a number ranging from 03 to 27.&lt;br /&gt;
&lt;br /&gt;
{| class=&amp;quot;wikitable&amp;quot;&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Name&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Faculty&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Dept./group(s)&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| OS&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Nodes&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Total # CPU's&lt;br /&gt;
|-&lt;br /&gt;
| hpc03.tudelft.net || CiTG || GRS-PSG/MGP || CentOS 7 || 17 || 264&lt;br /&gt;
|-&lt;br /&gt;
| hpc05.tudelft.net || TNW || QN/BN || CentOS 7 || 44 || 896&lt;br /&gt;
|-&lt;br /&gt;
| hpc06.tudelft.net || 3ME || PME/MSE/DCSC/MTT || CentOS 7 || 118 || 2084&lt;br /&gt;
|-&lt;br /&gt;
| hpc07.tudelft.net || CiTG || ES-SCS || CentOS 7 || 8 || 256&lt;br /&gt;
|-&lt;br /&gt;
| hpc08.tudelft.net || CiTG || HE-EFM || CentOS 7 || 12 || 240&lt;br /&gt;
|-&lt;br /&gt;
| hpc09.tudelft.net || CiTG || ES-PAVE || CentOS 7 || 12 || 192&lt;br /&gt;
|-&lt;br /&gt;
| hpc11.tudelft.net || TNW || RST/RID/Cheme || CentOS 7 || 45 || 1744&lt;br /&gt;
|-&lt;br /&gt;
| hpc12.tudelft.net || L&amp;amp;R || ASM/AWEP || CentOS 7 || 68 || 1536&lt;br /&gt;
|-&lt;br /&gt;
| hpc27.tudelft.net || CiTG || 3Md-AM || CentOS 7 || 20 || 400&lt;br /&gt;
|-			&lt;br /&gt;
|}&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are [https://en.wikipedia.org/wiki/Beowulf_cluster Beowulf] style clusters. The purpose of this wiki is to guide new users into using these clusters. Beowulf clusters are typically made of a bunch common and identical computers or servers linked together in a network. This can be used to perform parallel computing tasks on a much larger scale than would be possible on a single workstation or a heavy duty server.&lt;br /&gt;
&lt;br /&gt;
== About Linux ==&lt;br /&gt;
&lt;br /&gt;
The operating system on all our clusters is [http://www.centos.org/ CentOS 7], this is a well known server class Linux distribution. In order to use our hpc clusters you should at least have some basic knowledge of Linux on the command line. You will not find a Linux tutorial in this wiki, there are many tutorials on the internet that will teach you how to use Linux. Just use the phrase &amp;quot;Linux for beginners&amp;quot; in your favorite search engine and you'll find plenty of sources online. You can als opt to buy a book, [http://www.dummies.com/computers/operating-systems/linux/linux-for-dummies-9th-edition/ Linux for dummies] would be a good start, but there are many others that'll do just fine. If you have never used Linux on the command line before, please take the time to learn the basics, it will be well worth the effort.&lt;br /&gt;
&lt;br /&gt;
== More details ==&lt;br /&gt;
&lt;br /&gt;
Typically a Beowulf style cluster consists of a master node and several worker nodes. The master node is the machine where you log in to and where you prepare and manage your parallel jobs. A scheduler (Maui) and a resource manager (Torque) are both running on the master node. They both work together and provide a mechanism to submit jobs that will be run on the worker nodes where the actual parallel processing takes place. These nodes are stand alone computers connected through a local network to the master node and patiently waiting until the scheduler and the resource manager on the master node tells them to run a (part of a) parallel job.&lt;br /&gt;
&lt;br /&gt;
== About this wiki: contact and info ==&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are managed and maintained by the Linux systems department at the Shared Service Center ICT. You can contact us for any questions and comments about the hpc clusters by sending an email to servicepunt@tudelft.nl .&lt;br /&gt;
&lt;br /&gt;
You can also contact us if you find incorrect information on this wiki, if you are missing important information that you want added or if you have something to contribute to this wiki.&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Introduction&amp;diff=277</id>
		<title>Introduction</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Introduction&amp;diff=277"/>
		<updated>2024-03-22T13:19:33Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* About this wiki: contact and info */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== About the TU Delft hpc clusters ==&lt;br /&gt;
&lt;br /&gt;
Several research groups at the TU Delft have exclusive access to hpc (high performance computing) clusters that are managed and maintained by the ICT Department. Currently there are ten, they are typically named hpcXX.tudelft.net where the XX part is a number ranging from 03 to 27.&lt;br /&gt;
&lt;br /&gt;
{| class=&amp;quot;wikitable&amp;quot;&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Name&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Faculty&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Dept./group(s)&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| OS&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Nodes&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Total # CPU's&lt;br /&gt;
|-&lt;br /&gt;
| hpc03.tudelft.net || CiTG || GRS-PSG/MGP || CentOS 7 || 17 || 264&lt;br /&gt;
|-&lt;br /&gt;
| hpc05.tudelft.net || TNW || QN/BN || CentOS 7 || 44 || 896&lt;br /&gt;
|-&lt;br /&gt;
| hpc06.tudelft.net || 3ME || PME/MSE/DCSC/MTT || CentOS 7 || 118 || 2084&lt;br /&gt;
|-&lt;br /&gt;
| hpc07.tudelft.net || CiTG || ES-SCS || CentOS 7 || 8 || 256&lt;br /&gt;
|-&lt;br /&gt;
| hpc08.tudelft.net || CiTG || HE-EFM || CentOS 7 || 12 || 240&lt;br /&gt;
|-&lt;br /&gt;
| hpc09.tudelft.net || CiTG || ES-PAVE || CentOS 7 || 12 || 192&lt;br /&gt;
|-&lt;br /&gt;
| hpc10.tudelft.net || CiTG || Dream Teams || CentOS 7 || 8 || 112&lt;br /&gt;
|-&lt;br /&gt;
| hpc11.tudelft.net || TNW || RST/RID/Cheme || CentOS 7 || 45 || 1744&lt;br /&gt;
|-&lt;br /&gt;
| hpc12.tudelft.net || L&amp;amp;R || ASM/AWEP || CentOS 7 || 68 || 1536&lt;br /&gt;
|-&lt;br /&gt;
| hpc27.tudelft.net || CiTG || 3Md-AM || CentOS 7 || 20 || 400&lt;br /&gt;
|-			&lt;br /&gt;
|}&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are [https://en.wikipedia.org/wiki/Beowulf_cluster Beowulf] style clusters. The purpose of this wiki is to guide new users into using these clusters. Beowulf clusters are typically made of a bunch common and identical computers or servers linked together in a network. This can be used to perform parallel computing tasks on a much larger scale than would be possible on a single workstation or a heavy duty server.&lt;br /&gt;
&lt;br /&gt;
== About Linux ==&lt;br /&gt;
&lt;br /&gt;
The operating system on all our clusters is [http://www.centos.org/ CentOS 7], this is a well known server class Linux distribution. In order to use our hpc clusters you should at least have some basic knowledge of Linux on the command line. You will not find a Linux tutorial in this wiki, there are many tutorials on the internet that will teach you how to use Linux. Just use the phrase &amp;quot;Linux for beginners&amp;quot; in your favorite search engine and you'll find plenty of sources online. You can als opt to buy a book, [http://www.dummies.com/computers/operating-systems/linux/linux-for-dummies-9th-edition/ Linux for dummies] would be a good start, but there are many others that'll do just fine. If you have never used Linux on the command line before, please take the time to learn the basics, it will be well worth the effort.&lt;br /&gt;
&lt;br /&gt;
== More details ==&lt;br /&gt;
&lt;br /&gt;
Typically a Beowulf style cluster consists of a master node and several worker nodes. The master node is the machine where you log in to and where you prepare and manage your parallel jobs. A scheduler (Maui) and a resource manager (Torque) are both running on the master node. They both work together and provide a mechanism to submit jobs that will be run on the worker nodes where the actual parallel processing takes place. These nodes are stand alone computers connected through a local network to the master node and patiently waiting until the scheduler and the resource manager on the master node tells them to run a (part of a) parallel job.&lt;br /&gt;
&lt;br /&gt;
== About this wiki: contact and info ==&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are managed and maintained by the Linux systems department at the Shared Service Center ICT. You can contact us for any questions and comments about the hpc clusters by sending an email to servicepunt@tudelft.nl .&lt;br /&gt;
&lt;br /&gt;
You can also contact us if you find incorrect information on this wiki, if you are missing important information that you want added or if you have something to contribute to this wiki.&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Introduction&amp;diff=276</id>
		<title>Introduction</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Introduction&amp;diff=276"/>
		<updated>2024-03-22T13:18:56Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* More details */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== About the TU Delft hpc clusters ==&lt;br /&gt;
&lt;br /&gt;
Several research groups at the TU Delft have exclusive access to hpc (high performance computing) clusters that are managed and maintained by the ICT Department. Currently there are ten, they are typically named hpcXX.tudelft.net where the XX part is a number ranging from 03 to 27.&lt;br /&gt;
&lt;br /&gt;
{| class=&amp;quot;wikitable&amp;quot;&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Name&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Faculty&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Dept./group(s)&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| OS&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Nodes&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Total # CPU's&lt;br /&gt;
|-&lt;br /&gt;
| hpc03.tudelft.net || CiTG || GRS-PSG/MGP || CentOS 7 || 17 || 264&lt;br /&gt;
|-&lt;br /&gt;
| hpc05.tudelft.net || TNW || QN/BN || CentOS 7 || 44 || 896&lt;br /&gt;
|-&lt;br /&gt;
| hpc06.tudelft.net || 3ME || PME/MSE/DCSC/MTT || CentOS 7 || 118 || 2084&lt;br /&gt;
|-&lt;br /&gt;
| hpc07.tudelft.net || CiTG || ES-SCS || CentOS 7 || 8 || 256&lt;br /&gt;
|-&lt;br /&gt;
| hpc08.tudelft.net || CiTG || HE-EFM || CentOS 7 || 12 || 240&lt;br /&gt;
|-&lt;br /&gt;
| hpc09.tudelft.net || CiTG || ES-PAVE || CentOS 7 || 12 || 192&lt;br /&gt;
|-&lt;br /&gt;
| hpc10.tudelft.net || CiTG || Dream Teams || CentOS 7 || 8 || 112&lt;br /&gt;
|-&lt;br /&gt;
| hpc11.tudelft.net || TNW || RST/RID/Cheme || CentOS 7 || 45 || 1744&lt;br /&gt;
|-&lt;br /&gt;
| hpc12.tudelft.net || L&amp;amp;R || ASM/AWEP || CentOS 7 || 68 || 1536&lt;br /&gt;
|-&lt;br /&gt;
| hpc27.tudelft.net || CiTG || 3Md-AM || CentOS 7 || 20 || 400&lt;br /&gt;
|-			&lt;br /&gt;
|}&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are [https://en.wikipedia.org/wiki/Beowulf_cluster Beowulf] style clusters. The purpose of this wiki is to guide new users into using these clusters. Beowulf clusters are typically made of a bunch common and identical computers or servers linked together in a network. This can be used to perform parallel computing tasks on a much larger scale than would be possible on a single workstation or a heavy duty server.&lt;br /&gt;
&lt;br /&gt;
== About Linux ==&lt;br /&gt;
&lt;br /&gt;
The operating system on all our clusters is [http://www.centos.org/ CentOS 7], this is a well known server class Linux distribution. In order to use our hpc clusters you should at least have some basic knowledge of Linux on the command line. You will not find a Linux tutorial in this wiki, there are many tutorials on the internet that will teach you how to use Linux. Just use the phrase &amp;quot;Linux for beginners&amp;quot; in your favorite search engine and you'll find plenty of sources online. You can als opt to buy a book, [http://www.dummies.com/computers/operating-systems/linux/linux-for-dummies-9th-edition/ Linux for dummies] would be a good start, but there are many others that'll do just fine. If you have never used Linux on the command line before, please take the time to learn the basics, it will be well worth the effort.&lt;br /&gt;
&lt;br /&gt;
== More details ==&lt;br /&gt;
&lt;br /&gt;
Typically a Beowulf style cluster consists of a master node and several worker nodes. The master node is the machine where you log in to and where you prepare and manage your parallel jobs. A scheduler (Maui) and a resource manager (Torque) are both running on the master node. They both work together and provide a mechanism to submit jobs that will be run on the worker nodes where the actual parallel processing takes place. These nodes are stand alone computers connected through a local network to the master node and patiently waiting until the scheduler and the resource manager on the master node tells them to run a (part of a) parallel job.&lt;br /&gt;
&lt;br /&gt;
== About this wiki: contact and info ==&lt;br /&gt;
&lt;br /&gt;
TThe hpc clusters are managed and maintained by the Linux systems department at the Shared Service Center ICT. You can contact us for any questions and comments about the hpc clusters by sending an email to servicepunt@tudelft.nl .&lt;br /&gt;
&lt;br /&gt;
You can also contact us if you find incorrect information on this wiki, if you are missing important information that you want added or if you have something to contribute to this wiki.&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Further_reading&amp;diff=275</id>
		<title>Further reading</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Further_reading&amp;diff=275"/>
		<updated>2024-03-21T15:11:47Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: &lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;'''On an hpc cluster:''' &lt;br /&gt;
&lt;br /&gt;
Every time you log in you'll see an motd (message of the day). Please pay attention to this, it may contain information about current events on the cluster. &lt;br /&gt;
&lt;br /&gt;
The motd also shows you an '''email address''' that you can use if there are problems or if you have questions. Please note that you can mail at any time you like, but you'll usually only get a reply during office hours (local timezone in Delft), there is no 24/7 support on the hpc clusters. &lt;br /&gt;
&lt;br /&gt;
A brief instruction manual (plain text only) can be found here: &amp;lt;code&amp;gt;/opt/ud/LOCAL/doc/LOCALdoc.txt&amp;lt;/code&amp;gt;&lt;br /&gt;
&lt;br /&gt;
'''On the web:'''&lt;br /&gt;
&lt;br /&gt;
Wikipedia has a nice page about Beowulf style clusters: &lt;br /&gt;
&lt;br /&gt;
[https://en.wikipedia.org/wiki/Beowulf_cluster https://en.wikipedia.org/wiki/Beowulf_cluster]&lt;br /&gt;
&lt;br /&gt;
If you are unfamiliar with Linux and want to learn more, you may find this tutorial helpful: &lt;br /&gt;
&lt;br /&gt;
[https://ryanstutorials.net/linuxtutorial/ https://ryanstutorials.net/linuxtutorial/]&lt;br /&gt;
&lt;br /&gt;
Another wiki written by one of the users of hpc11: &lt;br /&gt;
&lt;br /&gt;
[http://wiki.tudelft.nl/bin/view/Sandbox/NewWikiWord44020 http://wiki.tudelft.nl/bin/view/Sandbox/NewWikiWord44020]&lt;br /&gt;
&lt;br /&gt;
Also a useful source of information is the cluster wiki of the Stellenbosch University in South Africa. This is a cluster with a queue system that differs from our hcp clusters, but the principles for creating and submitting jobs are the same. Be carefull though, do not just copy the scripts you find in the examples, just use them as inspiration for your own scripts!&lt;br /&gt;
&lt;br /&gt;
[https://www0.sun.ac.za/hpc/index.php?title=HOWTO_submit_jobs#Examples https://www0.sun.ac.za/hpc/index.php?title=HOWTO_submit_jobs#Examples]&lt;br /&gt;
&lt;br /&gt;
The TORQUE Administrator Guide has a chapter that covers job submission and management.&lt;br /&gt;
&lt;br /&gt;
[http://docs.adaptivecomputing.com/torque/4-2-10/help.htm#topics/2-jobs/submittingManagingJobs.htm%3FTocPath%3D2.0%2520Submitting%2520and%2520managing%2520jobs%7C_____0 Submitting and managing jobs]&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Further_reading&amp;diff=274</id>
		<title>Further reading</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Further_reading&amp;diff=274"/>
		<updated>2024-03-21T13:27:44Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: &lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;'''On an hpc cluster:''' &lt;br /&gt;
&lt;br /&gt;
Every time you log in you'll see an motd (message of the day). Please pay attention to this, it may contain information about current events on the cluster. &lt;br /&gt;
&lt;br /&gt;
The motd also shows you an '''email address''' that you can use if there are problems or if you have questions. Please note that you can mail at any time you like, but you'll usually only get a reply during office hours (local timezone in Delft), there is no 24/7 support on the hpc clusters. &lt;br /&gt;
&lt;br /&gt;
A brief instruction manual (plain text only) can be found here: &amp;lt;code&amp;gt;/opt/ud/LOCAL/doc/LOCALdoc.txt&amp;lt;/code&amp;gt;&lt;br /&gt;
&lt;br /&gt;
'''On the web:'''&lt;br /&gt;
&lt;br /&gt;
Wikipedia has a nice page about Beowulf style clusters: &lt;br /&gt;
&lt;br /&gt;
[https://en.wikipedia.org/wiki/Beowulf_cluster https://en.wikipedia.org/wiki/Beowulf_cluster]&lt;br /&gt;
&lt;br /&gt;
If you are unfamiliar with Linux and want to learn more, you may find this tutorial helpful: &lt;br /&gt;
&lt;br /&gt;
[http://www.ee.surrey.ac.uk/Teaching/Unix/ http://www.ee.surrey.ac.uk/Teaching/Unix/]&lt;br /&gt;
&lt;br /&gt;
Another wiki written by one of the users of hpc11: &lt;br /&gt;
&lt;br /&gt;
[http://wiki.tudelft.nl/bin/view/Sandbox/NewWikiWord44020 http://wiki.tudelft.nl/bin/view/Sandbox/NewWikiWord44020]&lt;br /&gt;
&lt;br /&gt;
Also a useful source of information is the cluster wiki of the Stellenbosch University in South Africa. This is a cluster with a queue system that differs from our hcp clusters, but the principles for creating and submitting jobs are the same. Be carefull though, do not just copy the scripts you find in the examples, just use them as inspiration for your own scripts!&lt;br /&gt;
&lt;br /&gt;
[https://www0.sun.ac.za/hpc/index.php?title=HOWTO_submit_jobs#Examples https://www0.sun.ac.za/hpc/index.php?title=HOWTO_submit_jobs#Examples]&lt;br /&gt;
&lt;br /&gt;
The TORQUE Administrator Guide has a chapter that covers job submission and management.&lt;br /&gt;
&lt;br /&gt;
[http://docs.adaptivecomputing.com/torque/4-2-10/help.htm#topics/2-jobs/submittingManagingJobs.htm%3FTocPath%3D2.0%2520Submitting%2520and%2520managing%2520jobs%7C_____0 Submitting and managing jobs]&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Further_reading&amp;diff=273</id>
		<title>Further reading</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Further_reading&amp;diff=273"/>
		<updated>2024-03-21T13:27:27Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: &lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;'''On an hpc cluster:''' &lt;br /&gt;
&lt;br /&gt;
Every time you log in you'll see an motd (message of the day). Please pay attention to this, it may contain information about current events on the cluster. &lt;br /&gt;
&lt;br /&gt;
The motd also shows you an '''email address''' that you can use if there are problems or if you have questions. Please note that you can mail at any time you like, but you'll usually only get a reply during office hours (local timezone in Delft), there is no 24/7 support on the hpc clusters. &lt;br /&gt;
&lt;br /&gt;
A brief instruction manual (plain text only) can be found here: &amp;lt;code&amp;gt;/opt/ud/LOCAL/doc/LOCALdoc.txt&amp;lt;/code&amp;gt;&lt;br /&gt;
&lt;br /&gt;
'''On the web:'''&lt;br /&gt;
&lt;br /&gt;
Wikipedia has a nice page about Beowulf style clusters: &lt;br /&gt;
&lt;br /&gt;
[https://en.wikipedia.org/wiki/Beowulf_cluster https://en.wikipedia.org/wiki/Beowulf_cluster]&lt;br /&gt;
&lt;br /&gt;
If you are unfamiliar with Linux and want to learn more, you may find this tutorial helpful: &lt;br /&gt;
&lt;br /&gt;
[http://www.ee.surrey.ac.uk/Teaching/Unix/ http://www.ee.surrey.ac.uk/Teaching/Unix/]&lt;br /&gt;
&lt;br /&gt;
Another wiki written by one of the users of hpc11: &lt;br /&gt;
&lt;br /&gt;
[http://wiki.tudelft.nl/bin/view/Sandbox/NewWikiWord44020 http://wiki.tudelft.nl/bin/view/Sandbox/NewWikiWord44020]&lt;br /&gt;
&lt;br /&gt;
Also a useful source of information is the cluster wiki of the Stellenbosch University in South Africa. This is a cluster with a queue system that differs from our hcp clusters, but the principles for creating and submitting jobs are the same. Be carefull though, do not just copy the scripts you find in the examples, just use them as inspiration for your own scripts!&lt;br /&gt;
&lt;br /&gt;
[https://www0.sun.ac.za/hpc/index.php?title=HOWTO_submit_jobs#Examples https://www0.sun.ac.za/hpc/index.php?title=HOWTO_submit_jobs#Examples]&lt;br /&gt;
&lt;br /&gt;
The TORQUE Administrator Guide has a chapter that covers job submission and management.&lt;br /&gt;
&lt;br /&gt;
[http://docs.adaptivecomputing.com/torque/4-2-10/help.htm#topics/2-jobs/submittingManagingJobs.htm%3FTocPath%3D2.0%2520Submitting%2520and%2520managing%2520jobs%7C_____0 Submitting and managing jobs]&lt;br /&gt;
&lt;br /&gt;
Testtest&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Introduction&amp;diff=268</id>
		<title>Introduction</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Introduction&amp;diff=268"/>
		<updated>2023-12-11T12:19:24Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* About Linux */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== About the TU Delft hpc clusters ==&lt;br /&gt;
&lt;br /&gt;
Several research groups at the TU Delft have exclusive access to hpc (high performance computing) clusters that are managed and maintained by the ICT Department. Currently there are ten, they are typically named hpcXX.tudelft.net where the XX part is a number ranging from 03 to 27.&lt;br /&gt;
&lt;br /&gt;
{| class=&amp;quot;wikitable&amp;quot;&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Name&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Faculty&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Dept./group(s)&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| OS&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Nodes&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Total # CPU's&lt;br /&gt;
|-&lt;br /&gt;
| hpc03.tudelft.net || CiTG || GRS-PSG/MGP || CentOS 7 || 17 || 264&lt;br /&gt;
|-&lt;br /&gt;
| hpc05.tudelft.net || TNW || QN/BN || CentOS 7 || 44 || 896&lt;br /&gt;
|-&lt;br /&gt;
| hpc06.tudelft.net || 3ME || PME/MSE/DCSC/MTT || CentOS 7 || 118 || 2084&lt;br /&gt;
|-&lt;br /&gt;
| hpc07.tudelft.net || CiTG || ES-SCS || CentOS 7 || 8 || 256&lt;br /&gt;
|-&lt;br /&gt;
| hpc08.tudelft.net || CiTG || HE-EFM || CentOS 7 || 12 || 240&lt;br /&gt;
|-&lt;br /&gt;
| hpc09.tudelft.net || CiTG || ES-PAVE || CentOS 7 || 12 || 192&lt;br /&gt;
|-&lt;br /&gt;
| hpc10.tudelft.net || CiTG || Dream Teams || CentOS 7 || 8 || 112&lt;br /&gt;
|-&lt;br /&gt;
| hpc11.tudelft.net || TNW || RST/RID/Cheme || CentOS 7 || 45 || 1744&lt;br /&gt;
|-&lt;br /&gt;
| hpc12.tudelft.net || L&amp;amp;R || ASM/AWEP || CentOS 7 || 68 || 1536&lt;br /&gt;
|-&lt;br /&gt;
| hpc27.tudelft.net || CiTG || 3Md-AM || CentOS 7 || 20 || 400&lt;br /&gt;
|-			&lt;br /&gt;
|}&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are [https://en.wikipedia.org/wiki/Beowulf_cluster Beowulf] style clusters. The purpose of this wiki is to guide new users into using these clusters. Beowulf clusters are typically made of a bunch common and identical computers or servers linked together in a network. This can be used to perform parallel computing tasks on a much larger scale than would be possible on a single workstation or a heavy duty server.&lt;br /&gt;
&lt;br /&gt;
== About Linux ==&lt;br /&gt;
&lt;br /&gt;
The operating system on all our clusters is [http://www.centos.org/ CentOS 7], this is a well known server class Linux distribution. In order to use our hpc clusters you should at least have some basic knowledge of Linux on the command line. You will not find a Linux tutorial in this wiki, there are many tutorials on the internet that will teach you how to use Linux. Just use the phrase &amp;quot;Linux for beginners&amp;quot; in your favorite search engine and you'll find plenty of sources online. You can als opt to buy a book, [http://www.dummies.com/computers/operating-systems/linux/linux-for-dummies-9th-edition/ Linux for dummies] would be a good start, but there are many others that'll do just fine. If you have never used Linux on the command line before, please take the time to learn the basics, it will be well worth the effort.&lt;br /&gt;
&lt;br /&gt;
== More details ==&lt;br /&gt;
&lt;br /&gt;
Typically a Beowulf style clusters consists of a master node and several worker nodes. The master node is the machine where you log in to and where you prepare and manage your parallel jobs. A scheduler (Maui) and a resource manager (Torque) are both running on the master node. They both work together and provide a mechanism to submit jobs that will be run on the worker nodes where the actual parallel processing takes place. These nodes are stand alone computers connected through a local network to the master node and patiently waiting until the scheduler and the resource manager on the master node tells them to run a (part of a) parallel job.&lt;br /&gt;
&lt;br /&gt;
== About this wiki: contact and info ==&lt;br /&gt;
&lt;br /&gt;
TThe hpc clusters are managed and maintained by the Linux systems department at the Shared Service Center ICT. You can contact us for any questions and comments about the hpc clusters by sending an email to servicepunt@tudelft.nl .&lt;br /&gt;
&lt;br /&gt;
You can also contact us if you find incorrect information on this wiki, if you are missing important information that you want added or if you have something to contribute to this wiki.&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=267</id>
		<title>How to log in</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=267"/>
		<updated>2023-09-19T16:25:11Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: &lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== Basics ==&lt;br /&gt;
&lt;br /&gt;
In order to log in you need a valid [https://www.tudelft.nl/en/student/ict/ict-facilities/netid-account/ NetID] that is enabled for access to one of the clusters. To have your NetID enabled, ask the contact person for the cluster you want access to (See the [[Introduction]] page) or ask the secretary of the department or research group where you work, he or she can tell you who the contact person is for an hpc cluster.&lt;br /&gt;
&lt;br /&gt;
If you want to collaborate with someone from outside the TU Delft and you want that person to access the same cluster as you, then you can request a guest account at the servicedesk. Once the guest account is created, it can be enabled for access to one of the clusters. '''Do not share your own NetID with others! Your NetID is for personal use only, sharing it with other people is forbidden!'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are only accessible via ssh (secure shell). To log in you need an ssh client. If you use Windows, you can download and use [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY], Macintosh and Linux computers already have an ssh client built in.&lt;br /&gt;
&lt;br /&gt;
Every time you log in you'll see an motd (message of the day). Please pay attention to this, it may contain information about current events on the cluster. It also shows you an email address that you can use if there are problems or if you have questions. You can mail at any time you like, but you'll usually only get a reply during office hours, there is no 24/7 support on the hpc clusters.&lt;br /&gt;
&lt;br /&gt;
'''Please note:''' You have a local home directory on the cluster. No backups will be made of the home directories, if you make a mistake and loose a file, it is gone forever. '''Making backups of your important files on the hpc clusters is your own responsibility!'''&lt;br /&gt;
&lt;br /&gt;
== How to log in from Windows ==&lt;br /&gt;
&lt;br /&gt;
An often used ssh client for Windows is [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY]. If you use a Windows PC or a laptop that is provided and maintained by the TU Delft, it is already there, if you use your own Windows computer, you'll have to download and install it first from [http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html www.chiark.greenend.org.uk].&lt;br /&gt;
&lt;br /&gt;
Once you have started PuTTY you'll get a configuration window als shown.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can type the name of the cluster, for example 'hpc10.tudelft.net', in the field 'Host Name'. You can save the settings for a cluster by choosing a name for the session, enter it in the 'Saved Sessions' box and click 'Save'. The next time you open PuTTY the settings can be restored by clicking on the name and a click on 'Load'.&lt;br /&gt;
&lt;br /&gt;
By clicking 'Open' a connection is made to hpc10.tudelft.net. If this is the first time you are connecting to the cluster, PuTTY will warn you that you are connecting to a previously unknown host.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_02.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can confirm this connection by clicking 'yes' so PuTTY will store the host key in its database. You'll have to do this only once for every new server that you connect to.&lt;br /&gt;
&lt;br /&gt;
After you have accepted the connection, PuTTY opens a console and asks for your loginname. Once you have entered your NetID and your password, you are logged in to the cluster.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_03.png|center]]&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from Windows ==&lt;br /&gt;
&lt;br /&gt;
WinSCP is a freeware ftp client with support for secure file protocols sftp and scp. The login screen depicted below shows the settings for access within the University.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
After clicking on 'Login' a warning about host keys will appear, similar to PuTTY. If it is the first time, you can safely click on 'yes' to accept.&lt;br /&gt;
&lt;br /&gt;
Press 'Login'. If everything is setup correctly you are using the ssh tunnel the PuTTY bastion session has built and you are logged in on the cluster.&lt;br /&gt;
&lt;br /&gt;
When logged in, you will see the commander-like interface below.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_02.png|center|530px]]&lt;br /&gt;
&lt;br /&gt;
The left side displays the files in your home directory on the windows system. The right side displays the files on the cluster. You can drag and drop files between the two windows or select files with the mouse and press 'F5' for transfer to the other window.&lt;br /&gt;
&lt;br /&gt;
== How to log in from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
Mac OS X is in fact a full blown UNIX system under the hood. A terminal program comes standard with OS X, navigate with the finder to Applications -&amp;gt; Utilities to find it, the program is called 'Terminal'. There are many different flavours of Linux, each with their own desktop environments and customizations. Nearly every Linux system comes with a terminal program of some sort. Find the terminal program on your system and run it, once you get a command prompt type:&lt;br /&gt;
&lt;br /&gt;
 ssh &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
Where &amp;lt;netid&amp;gt; is of course your own netid.&lt;br /&gt;
&lt;br /&gt;
Upon first access, ssh requires you to check if the host key is correct. Since this is the first time you connect to the system, it's fine to accept the host key by pressing 'y'.&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh:&lt;br /&gt;
&lt;br /&gt;
 scp &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your remote home directory (folder) to the local home directory on your workstation or laptop. &lt;br /&gt;
&lt;br /&gt;
If you want to copy an entire directory with all its content, use the '-r' option, like this:&lt;br /&gt;
&lt;br /&gt;
 scp -r &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
This will copy the directory 'projects' from your remote home directory to the local home directory on your workstation or laptop.&lt;br /&gt;
&lt;br /&gt;
Of course, it is also possible to copy files and directories from your workstation or laptop to the remote machine:&lt;br /&gt;
&lt;br /&gt;
 scp -r projects.txt &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
 scp -r projects/ &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
And this will copy the directory 'projects' and all its content from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
&lt;br /&gt;
=== Windows ===&lt;br /&gt;
&lt;br /&gt;
There are several options for displaying graphical applications on a Windows system. A basic option is [http://www.straightrunning.com/XmingNotes/ Xming]. You can download it from: http://www.straightrunning.com/XmingNotes/ The site asks for a donation to use the latest version, but there is also a public domain version which can be used for free. Once you have installed it, you can find it in the usual place in the start menu. Before you log in to an hpc cluster and run graphical applications, you need to start Xming first. It will not show much, only a small X in the system tray once it's running. You also need to enable X forwarding in PuTTY. For that you need to click on the + next to SSH in the session panel (left part of the window), then click on X11. You'll see a line on the right side that says &amp;quot;Enable X11 forwarding&amp;quot;, click once on the white square next to that line and it will show a check mark. &lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_04.png|center]]&lt;br /&gt;
&lt;br /&gt;
Now you can continue logging in to the hpc cluster. To check if X forwarding works, type &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that will follow the mouse pointer.&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
[[File:Xeyes.png|center]]&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
An alternative option for displaying graphical applications is Exceed, this is a commercial package for which the TU Delft has a license. It is not available by default on the standard Windows systems, but it can be installed upon request if you contact the servicedesk.&lt;br /&gt;
&lt;br /&gt;
A second alternative is [https://mobaxterm.mobatek.net/ MobaXterm], this is an all-in-one solution containing an ssh client, X server and more. You can download a free version from: https://mobaxterm.mobatek.net/download.html The free version has some limitations, but it's still perfectly usable in our environment.&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
== How to access your Windows network shares ==&lt;br /&gt;
&lt;br /&gt;
=== Authenticate ===&lt;br /&gt;
&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
&lt;br /&gt;
 kinit&lt;br /&gt;
&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
=== Basic access ===&lt;br /&gt;
&lt;br /&gt;
Now just type:&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
&lt;br /&gt;
or&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-umbrella/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
&lt;br /&gt;
You can access your Windows home directory as well, the path is:&lt;br /&gt;
&lt;br /&gt;
 /tudelft/$USER/staff-homes/&lt;br /&gt;
&lt;br /&gt;
followed by the first letter of your surname and then your netid. For example, if your name would be John Smith and your netid jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
If you are a student, replace 'staff' with 'student' to access your files.&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
=== Important note ===&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your netid. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your netid yourself. If your netid would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well. Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own netid!&lt;br /&gt;
&lt;br /&gt;
'''If you use anthing other than your own netid you'll create a recipe for potential disaster!'''&lt;br /&gt;
&lt;br /&gt;
If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths. So &lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-group/&lt;br /&gt;
&lt;br /&gt;
only works if your netid is jsmith and&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/pbrown/staff-group/&lt;br /&gt;
&lt;br /&gt;
only works works if your netid is pbrown.&lt;br /&gt;
&lt;br /&gt;
If both netids have the same privileges for accessing the group volumes, then these commands give access to the same data.&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Introduction&amp;diff=265</id>
		<title>Introduction</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Introduction&amp;diff=265"/>
		<updated>2023-05-02T15:09:13Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* About the TU Delft hpc clusters */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== About the TU Delft hpc clusters ==&lt;br /&gt;
&lt;br /&gt;
Several research groups at the TU Delft have exclusive access to hpc (high performance computing) clusters that are managed and maintained by the ICT Department. Currently there are ten, they are typically named hpcXX.tudelft.net where the XX part is a number ranging from 03 to 27.&lt;br /&gt;
&lt;br /&gt;
{| class=&amp;quot;wikitable&amp;quot;&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Name&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Faculty&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Dept./group(s)&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| OS&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Nodes&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Total # CPU's&lt;br /&gt;
|-&lt;br /&gt;
| hpc03.tudelft.net || CiTG || GRS-PSG/MGP || CentOS 7 || 17 || 264&lt;br /&gt;
|-&lt;br /&gt;
| hpc05.tudelft.net || TNW || QN/BN || CentOS 7 || 44 || 896&lt;br /&gt;
|-&lt;br /&gt;
| hpc06.tudelft.net || 3ME || PME/MSE/DCSC/MTT || CentOS 7 || 118 || 2084&lt;br /&gt;
|-&lt;br /&gt;
| hpc07.tudelft.net || CiTG || ES-SCS || CentOS 7 || 8 || 256&lt;br /&gt;
|-&lt;br /&gt;
| hpc08.tudelft.net || CiTG || HE-EFM || CentOS 7 || 12 || 240&lt;br /&gt;
|-&lt;br /&gt;
| hpc09.tudelft.net || CiTG || ES-PAVE || CentOS 7 || 12 || 192&lt;br /&gt;
|-&lt;br /&gt;
| hpc10.tudelft.net || CiTG || Dream Teams || CentOS 7 || 8 || 112&lt;br /&gt;
|-&lt;br /&gt;
| hpc11.tudelft.net || TNW || RST/RID/Cheme || CentOS 7 || 45 || 1744&lt;br /&gt;
|-&lt;br /&gt;
| hpc12.tudelft.net || L&amp;amp;R || ASM/AWEP || CentOS 7 || 68 || 1536&lt;br /&gt;
|-&lt;br /&gt;
| hpc27.tudelft.net || CiTG || 3Md-AM || CentOS 7 || 20 || 400&lt;br /&gt;
|-			&lt;br /&gt;
|}&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are [https://en.wikipedia.org/wiki/Beowulf_cluster Beowulf] style clusters. The purpose of this wiki is to guide new users into using these clusters. Beowulf clusters are typically made of a bunch common and identical computers or servers linked together in a network. This can be used to perform parallel computing tasks on a much larger scale than would be possible on a single workstation or a heavy duty server.&lt;br /&gt;
&lt;br /&gt;
== About Linux ==&lt;br /&gt;
&lt;br /&gt;
The operating system on all our clusters is [http://www.centos.org/ CentOS 7], this is a well known server class Linux distribution. In order to use our hpc clusters you should at least have some basic knowledge of Linux on the command line. You will not find a Linux tutorial in this wiki, there are many tutorials on the internet that will teach you how to use Linux. Just use the phrase &amp;quot;Linux for beginners&amp;quot; in your favorite search engine and you'll find plenty of sources online, for example the [http://www.ee.surrey.ac.uk/Teaching/Unix/ UNIX Tutorial for Beginners] from the [http://www.surrey.ac.uk/ University of Surrey]. You can als opt to buy a book, [http://www.dummies.com/computers/operating-systems/linux/linux-for-dummies-9th-edition/ Linux for dummies] would be a good start, but there are many others that'll do just fine. If you have never used Linux on the command line before, please take the time to learn the basics, it will be well worth the effort.&lt;br /&gt;
&lt;br /&gt;
== More details ==&lt;br /&gt;
&lt;br /&gt;
Typically a Beowulf style clusters consists of a master node and several worker nodes. The master node is the machine where you log in to and where you prepare and manage your parallel jobs. A scheduler (Maui) and a resource manager (Torque) are both running on the master node. They both work together and provide a mechanism to submit jobs that will be run on the worker nodes where the actual parallel processing takes place. These nodes are stand alone computers connected through a local network to the master node and patiently waiting until the scheduler and the resource manager on the master node tells them to run a (part of a) parallel job.&lt;br /&gt;
&lt;br /&gt;
== About this wiki: contact and info ==&lt;br /&gt;
&lt;br /&gt;
TThe hpc clusters are managed and maintained by the Linux systems department at the Shared Service Center ICT. You can contact us for any questions and comments about the hpc clusters by sending an email to servicepunt@tudelft.nl .&lt;br /&gt;
&lt;br /&gt;
You can also contact us if you find incorrect information on this wiki, if you are missing important information that you want added or if you have something to contribute to this wiki.&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Introduction&amp;diff=264</id>
		<title>Introduction</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Introduction&amp;diff=264"/>
		<updated>2023-05-02T15:08:24Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: &lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== About the TU Delft hpc clusters ==&lt;br /&gt;
&lt;br /&gt;
Several research groups at the TU Delft have exclusive access to hpc (high performance computing) clusters that are managed and maintained by the ICT Department. Currently there are ten, they are typically named hpcXX.tudelft.net where the XX part is a number ranging from 03 to 27.&lt;br /&gt;
&lt;br /&gt;
{| class=&amp;quot;wikitable&amp;quot;&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Name&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Faculty&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Dept./group(s)&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| OS&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Nodes&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Total # CPU's&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Contact&lt;br /&gt;
|-&lt;br /&gt;
| hpc03.tudelft.net || CiTG || GRS-PSG/MGP || CentOS 7 || 17 || 264 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc05.tudelft.net || TNW || QN/BN || CentOS 7 || 44 || 896 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc06.tudelft.net || 3ME || PME/MSE/DCSC/MTT || CentOS 7 || 118 || 2084 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc07.tudelft.net || CiTG || ES-SCS || CentOS 7 || 8 || 256 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc08.tudelft.net || CiTG || HE-EFM || CentOS 7 || 12 || 240 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc09.tudelft.net || CiTG || ES-PAVE || CentOS 7 || 12 || 192 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc10.tudelft.net || CiTG || Dream Teams || CentOS 7 || 8 || 112 || &lt;br /&gt;
|-&lt;br /&gt;
| hpc11.tudelft.net || TNW || RST/RID/Cheme || CentOS 7 || 45 || 1744 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc12.tudelft.net || L&amp;amp;R || ASM/AWEP || CentOS 7 || 68 || 1536 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc27.tudelft.net || CiTG || 3Md-AM || CentOS 7 || 20 || 400 ||&lt;br /&gt;
|-			&lt;br /&gt;
|}&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are [https://en.wikipedia.org/wiki/Beowulf_cluster Beowulf] style clusters. The purpose of this wiki is to guide new users into using these clusters. Beowulf clusters are typically made of a bunch common and identical computers or servers linked together in a network. This can be used to perform parallel computing tasks on a much larger scale than would be possible on a single workstation or a heavy duty server.&lt;br /&gt;
&lt;br /&gt;
== About Linux ==&lt;br /&gt;
&lt;br /&gt;
The operating system on all our clusters is [http://www.centos.org/ CentOS 7], this is a well known server class Linux distribution. In order to use our hpc clusters you should at least have some basic knowledge of Linux on the command line. You will not find a Linux tutorial in this wiki, there are many tutorials on the internet that will teach you how to use Linux. Just use the phrase &amp;quot;Linux for beginners&amp;quot; in your favorite search engine and you'll find plenty of sources online, for example the [http://www.ee.surrey.ac.uk/Teaching/Unix/ UNIX Tutorial for Beginners] from the [http://www.surrey.ac.uk/ University of Surrey]. You can als opt to buy a book, [http://www.dummies.com/computers/operating-systems/linux/linux-for-dummies-9th-edition/ Linux for dummies] would be a good start, but there are many others that'll do just fine. If you have never used Linux on the command line before, please take the time to learn the basics, it will be well worth the effort.&lt;br /&gt;
&lt;br /&gt;
== More details ==&lt;br /&gt;
&lt;br /&gt;
Typically a Beowulf style clusters consists of a master node and several worker nodes. The master node is the machine where you log in to and where you prepare and manage your parallel jobs. A scheduler (Maui) and a resource manager (Torque) are both running on the master node. They both work together and provide a mechanism to submit jobs that will be run on the worker nodes where the actual parallel processing takes place. These nodes are stand alone computers connected through a local network to the master node and patiently waiting until the scheduler and the resource manager on the master node tells them to run a (part of a) parallel job.&lt;br /&gt;
&lt;br /&gt;
== About this wiki: contact and info ==&lt;br /&gt;
&lt;br /&gt;
TThe hpc clusters are managed and maintained by the Linux systems department at the Shared Service Center ICT. You can contact us for any questions and comments about the hpc clusters by sending an email to servicepunt@tudelft.nl .&lt;br /&gt;
&lt;br /&gt;
You can also contact us if you find incorrect information on this wiki, if you are missing important information that you want added or if you have something to contribute to this wiki.&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Introduction&amp;diff=263</id>
		<title>Introduction</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Introduction&amp;diff=263"/>
		<updated>2023-05-02T15:07:42Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: &lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== About the TU Delft hpc clusters ==&lt;br /&gt;
&lt;br /&gt;
Several research groups at the TU Delft have exclusive access to hpc (high performance computing) clusters that are managed and maintained by the ICT Department. Currently there are ten, they are typically named hpcXX.tudelft.net where the XX part is a number ranging from 03 to 27.&lt;br /&gt;
&lt;br /&gt;
{| class=&amp;quot;wikitable&amp;quot;&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Name&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Faculty&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Dept./group(s)&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| OS&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Nodes&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Total # CPU's&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Contact&lt;br /&gt;
|-&lt;br /&gt;
| hpc03.tudelft.net || CiTG || GRS-PSG/MGP || CentOS 7 || 17 || 264 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc05.tudelft.net || TNW || QN/BN || CentOS 7 || 44 || 896 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc06.tudelft.net || 3ME || PME/MSE/DCSC/MTT || CentOS 7 || 118 || 2084 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc07.tudelft.net || CiTG || ES-SCS || CentOS 7 || 8 || 256 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc08.tudelft.net || CiTG || HE-EFM || CentOS 7 || 12 || 240 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc09.tudelft.net || CiTG || ES-PAVE || CentOS 7 || 12 || 192 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc10.tudelft.net || CiTG || Dream Teams || CentOS 7 || 8 || 112 || &lt;br /&gt;
|-&lt;br /&gt;
| hpc11.tudelft.net || TNW || RST/RID/Cheme || CentOS 7 || 45 || 1744 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc12.tudelft.net || L&amp;amp;R || ASM/AWEP || CentOS 7 || 68 || 1536 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc27.tudelft.net || CiTG || 3Md-AM || CentOS 7 || 20 || 400 ||&lt;br /&gt;
|-			&lt;br /&gt;
|}&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are [https://en.wikipedia.org/wiki/Beowulf_cluster Beowulf] style clusters. The purpose of this wiki is to guide new users into using these clusters. Beowulf clusters are typically made of a bunch common and identical computers or servers linked together in a network. This can be used to perform parallel computing tasks on a much larger scale than would be possible on a single workstation or a heavy duty server.&lt;br /&gt;
&lt;br /&gt;
== About Linux ==&lt;br /&gt;
&lt;br /&gt;
The operating system on all our clusters is [http://www.centos.org/ CentOS 7], this is a well known server class Linux distribution. In order to use our hpc clusters you should at least have some basic knowledge of Linux on the command line. You will not find a Linux tutorial in this wiki, there are many tutorials on the internet that will teach you how to use Linux. Just use the phrase &amp;quot;Linux for beginners&amp;quot; in your favorite search engine and you'll find plenty of sources online, for example the [http://www.ee.surrey.ac.uk/Teaching/Unix/ UNIX Tutorial for Beginners] from the [http://www.surrey.ac.uk/ University of Surrey]. You can als opt to buy a book, [http://www.dummies.com/computers/operating-systems/linux/linux-for-dummies-9th-edition/ Linux for dummies] would be a good start, but there are many others that'll do just fine. If you have never used Linux on the command line before, please take the time to learn the basics, it will be well worth the effort.&lt;br /&gt;
&lt;br /&gt;
== More details ==&lt;br /&gt;
&lt;br /&gt;
Typically a Beowulf style clusters consists of a master node and several worker nodes. The master node is the machine where you log in to and where you prepare and manage your parallel jobs. A scheduler (Maui) and a resource manager (Torque) are both running on the master node. They both work together and provide a mechanism to submit jobs that will be run on the worker nodes where the actual parallel processing takes place. These nodes are stand alone computers connected through a local network to the master node and patiently waiting until the scheduler and the resource manager on the master node tells them to run a (part of a) parallel job.&lt;br /&gt;
&lt;br /&gt;
== About this wiki: contact and info ==&lt;br /&gt;
&lt;br /&gt;
TThe hpc clusters are managed and maintained by the Linux systems department at the Shared Service Center ICT. You can contact us for any questions and comments about the hpc clusters at email address servicepunt@tudelft.nl&lt;br /&gt;
&lt;br /&gt;
You can also contact us if you find incorrect information on this wiki, if you are missing important information that you want added or if you have something to contribute to this wiki.&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Introduction&amp;diff=167</id>
		<title>Introduction</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Introduction&amp;diff=167"/>
		<updated>2020-09-30T09:17:02Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* About the TU Delft hpc clusters */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== About the TU Delft hpc clusters ==&lt;br /&gt;
&lt;br /&gt;
Several research groups at the TU Delft have exclusive access to hpc (high performance computing) clusters that are managed and maintained by the ICT Department. Currently there are ten, they are typically named hpcXX.tudelft.net where the XX part is a number ranging from 03 to 27.&lt;br /&gt;
&lt;br /&gt;
{| class=&amp;quot;wikitable&amp;quot;&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Name&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Faculty&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Dept./group(s)&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| OS&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Nodes&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Total # CPU's&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Contact&lt;br /&gt;
|-&lt;br /&gt;
| hpc03.tudelft.net || CiTG || GRS-PSG/MGP || CentOS 7 || 17 || 264 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc05.tudelft.net || TNW || QN/BN || CentOS 7 || 44 || 896 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc06.tudelft.net || 3ME || PME/MSE/DCSC/MTT || CentOS 7 || 118 || 2084 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc07.tudelft.net || CiTG || ES-SCS || CentOS 7 || 8 || 256 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc08.tudelft.net || CiTG || HE-EFM || CentOS 7 || 12 || 240 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc09.tudelft.net || CiTG || ES-PAVE || CentOS 7 || 12 || 192 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc10.tudelft.net || CiTG || Dream Teams || CentOS 7 || 8 || 112 || [[file:halbeheer.png|link=]]&lt;br /&gt;
|-&lt;br /&gt;
| hpc11.tudelft.net || TNW || RST/RID/Cheme || CentOS 7 || 45 || 1744 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc12.tudelft.net || L&amp;amp;R || ASM/AWEP || CentOS 7 || 68 || 1536 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc27.tudelft.net || CiTG || 3Md-AM || CentOS 7 || 20 || 400 ||&lt;br /&gt;
|-			&lt;br /&gt;
|}&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are [https://en.wikipedia.org/wiki/Beowulf_cluster Beowulf] style clusters. The purpose of this wiki is to guide new users into using these clusters. Beowulf clusters are typically made of a bunch common and identical computers or servers linked together in a network. This can be used to perform parallel computing tasks on a much larger scale than would be possible on a single workstation or a heavy duty server.&lt;br /&gt;
&lt;br /&gt;
== About Linux ==&lt;br /&gt;
&lt;br /&gt;
The operating system on all our clusters is [http://www.centos.org/ CentOS 7], this is a well known server class Linux distribution. In order to use our hpc clusters you should at least have some basic knowledge of Linux on the command line. You will not find a Linux tutorial in this wiki, there are many tutorials on the internet that will teach you how to use Linux. Just use the phrase &amp;quot;Linux for beginners&amp;quot; in your favorite search engine and you'll find plenty of sources online, for example the [http://www.ee.surrey.ac.uk/Teaching/Unix/ UNIX Tutorial for Beginners] from the [http://www.surrey.ac.uk/ University of Surrey]. You can als opt to buy a book, [http://www.dummies.com/computers/operating-systems/linux/linux-for-dummies-9th-edition/ Linux for dummies] would be a good start, but there are many others that'll do just fine. If you have never used Linux on the command line before, please take the time to learn the basics, it will be well worth the effort.&lt;br /&gt;
&lt;br /&gt;
== More details ==&lt;br /&gt;
&lt;br /&gt;
Typically a Beowulf style clusters consists of a master node and several worker nodes. The master node is the machine where you log in to and where you prepare and manage your parallel jobs. A scheduler (Maui) and a resource manager (Torque) are both running on the master node. They both work together and provide a mechanism to submit jobs that will be run on the worker nodes where the actual parallel processing takes place. These nodes are stand alone computers connected through a local network to the master node and patiently waiting until the scheduler and the resource manager on the master node tells them to run a (part of a) parallel job.&lt;br /&gt;
&lt;br /&gt;
== About this wiki: contact and info ==&lt;br /&gt;
&lt;br /&gt;
TThe hpc clusters are managed and maintained by the Linux systems department at the Shared Service Center ICT. You can contact us for any questions and comments about the hpc clusters at this address: &lt;br /&gt;
&lt;br /&gt;
[[file:beheer-o.png]] &lt;br /&gt;
&lt;br /&gt;
You can also contact us if you find incorrect information on this wiki, if you are missing important information that you want added or if you have something to contribute to this wiki.&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=More_about_queues_and_nodes&amp;diff=166</id>
		<title>More about queues and nodes</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=More_about_queues_and_nodes&amp;diff=166"/>
		<updated>2020-07-01T10:11:31Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* MPI jobs */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== The different queues ==&lt;br /&gt;
&lt;br /&gt;
The larger hpc clusters, most notably hpc03, hpc06, hpc11 and hpc12, are shared by two or more research groups. On those clusters every group has their own queue, sometimes even more than one. These queues give exclusive and full access to a specific set of nodes. &lt;br /&gt;
&lt;br /&gt;
There is also a guest queue on every hpc cluster that gives access to all nodes, but with some restrictions, you will not be able to run non-rerunable and interactive jobs. &lt;br /&gt;
&lt;br /&gt;
In most cases, access to one of the queues is based on group membership in the Active Directory. If your netid is not a member of the right group, you default to the guest queue if you submit a job. If you have access to the group and bulk network shares of your research group, you should also have access to the normal queue on the hpc cluster. If not, contact the secretary in your research group and let him/her arrange the group membership of your netid.&lt;br /&gt;
&lt;br /&gt;
You can check your default queue by submitting a small test job and then have a look at the list with jobs with the qstat command.&lt;br /&gt;
&lt;br /&gt;
 [jsmith@hpc10 ~]$ echo &amp;quot;sleep 60&amp;quot; | qsub &lt;br /&gt;
 [jsmith@hpc10 ~]$ qstat -u jsmith&lt;br /&gt;
&lt;br /&gt;
If you see anything other than guest in the third column, then you are all set.&lt;br /&gt;
&lt;br /&gt;
There are two ways to select the guest queue; &lt;br /&gt;
&lt;br /&gt;
With the -q switch on the commandline:&lt;br /&gt;
&lt;br /&gt;
 qsub -q guest job1&lt;br /&gt;
&lt;br /&gt;
Or with a directive at the start of your job script:&lt;br /&gt;
&lt;br /&gt;
 #PBS -q guest&lt;br /&gt;
&lt;br /&gt;
It is important to know that a job in the guest queue can be interrupted and resumed at any time. You should make sure that the application in your job saves the intermediate results at regular intervals and that it knows how to continue when your job is resumed. If you neglect this, your job in the guest queue will start all over again every time it is interrupted and resumed.&lt;br /&gt;
&lt;br /&gt;
== The different nodes ==&lt;br /&gt;
&lt;br /&gt;
On most hpc clusters you'll find that worker nodes are not all identical, different series of nodes exist which were purchased at different times and with different specifications. To distinguish between the different series of nodes, they are labelled with properties like typea, typeb, typec, etc. On some hpc clusters, nodes have extra properties showing to which queue they belong or showing additional features, like an infiniband network or extra memory compared to similar nodes.&lt;br /&gt;
&lt;br /&gt;
A useful command that shows all nodes and how they are utilized is &amp;lt;code&amp;gt;LOCALnodeload.pl&amp;lt;/code&amp;gt;. A typical output looks like this:&lt;br /&gt;
&lt;br /&gt;
&amp;lt;pre&amp;gt;&lt;br /&gt;
[jsmith@hpc10 ~]$ LOCALnodeload.pl&lt;br /&gt;
Node       Np State/jobs Load  Properties&lt;br /&gt;
---------- -- ---------- ----- ----------&lt;br /&gt;
n10-01     12 12         12.01 typea     &lt;br /&gt;
n10-02     12 free        0.00 typea     &lt;br /&gt;
n10-03     12 free        0.00 typea     &lt;br /&gt;
n10-04     12 free        0.00 typea     &lt;br /&gt;
n10-05     16 12         11.93 typeb     &lt;br /&gt;
n10-06     16 free        0.00 typeb     &lt;br /&gt;
n10-07     16 offline     0.00 typeb     &lt;br /&gt;
n10-08     16 down        0.00 typeb     &lt;br /&gt;
&amp;lt;/pre&amp;gt;&lt;br /&gt;
&lt;br /&gt;
The first column (Node) shows the names of the nodes. The second column (Np) shows the total number of processors. The third column (State/jobs) shows the number of processors currenly in use or the status of the node (free, offline or down). The forth colum (Load) shows the actual load on the nodes. In an ideal situation the load matches the number of processors in use. The last column (Properties) shows the properties as described above. As you can see in the example, typea nodes have 12 processors and typeb nodes have 16. Node n10-01 is fully occupied, node n10-05 is running one or more jobs but still has 4 processors free. Nodes n10-07 and n10-08 cannot be used.&lt;br /&gt;
&lt;br /&gt;
== Selecting nodes ==&lt;br /&gt;
&lt;br /&gt;
If you submit a job, the scheduler automatically selects a node to run it. By default a jobs gets one node and one processor. You can manually select the number of processors and nodes for your job by using the &amp;lt;code&amp;gt;-l&amp;lt;/code&amp;gt; switch with the &amp;lt;code&amp;gt;qsub&amp;lt;/code&amp;gt; command. You can also select nodes by property. the &amp;lt;code&amp;gt;-l&amp;lt;/code&amp;gt; switch works like this:&lt;br /&gt;
&lt;br /&gt;
 qsub -l nodes=&amp;lt;x&amp;gt;:ppn=&amp;lt;c&amp;gt;:&amp;lt;property&amp;gt;:&amp;lt;property&amp;gt;...&lt;br /&gt;
&lt;br /&gt;
* &amp;lt;x&amp;gt; is either an amount of nodes or the name(s) of the selected node(s)&lt;br /&gt;
* &amp;lt;c&amp;gt; is number of processors per node&lt;br /&gt;
* &amp;lt;property&amp;gt; is any of the properties you see in Properties column of the LOCALnodeload.pl command.&lt;br /&gt;
&lt;br /&gt;
Examples:&lt;br /&gt;
&lt;br /&gt;
{|&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;qsub -l nodes=4&amp;lt;/code&amp;gt; || Request 4 nodes of any type&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;qsub -l nodes=n10-07+n10-08&amp;lt;/code&amp;gt; || Request 2 specific nodes by hostname&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;qsub -l nodes=4:ppn=2&amp;lt;/code&amp;gt; || Request 2 processors on each of four nodes&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;qsub -l nodes=1:ppn=4&amp;lt;/code&amp;gt; || Request 4 processors on one node&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;qsub -l nodes=2:typea&amp;lt;/code&amp;gt; || Request 2 nodes with the &amp;lt;code&amp;gt;typea&amp;lt;/code&amp;gt; property&lt;br /&gt;
|-&lt;br /&gt;
|}&lt;br /&gt;
Instead of using the -l or the -q switches on the commandline when you submit your job with qsub, you can also add them as a directive to your job script. For instance, if you add&lt;br /&gt;
&lt;br /&gt;
 #PBS -l nodes=1:ppn=4&lt;br /&gt;
 #PBS -q guest&lt;br /&gt;
&lt;br /&gt;
at the start of your script, you can just use&lt;br /&gt;
&lt;br /&gt;
 qsub job.sh&lt;br /&gt;
&lt;br /&gt;
instead of&lt;br /&gt;
&lt;br /&gt;
 qsub -l nodes=1:ppn=4 -q guest job.sh&lt;br /&gt;
&lt;br /&gt;
== Avoid over- and underutilization ==&lt;br /&gt;
&lt;br /&gt;
An important thing to consider when you create your own job script is matching the number of processors that you request with the number of processors that the software in your script will actually use. It is possible that you request only one processor and that your program will use all processors available on the nodes. This is called overutilization and is not very efficient when other jobs are already running on the same node and using the same processors.&lt;br /&gt;
&lt;br /&gt;
It is also possible that you request several (or all) processors and that your program will only use one. This will leave the other processors you claimed unused (underutilization), which is also not very efficient because the unused processors you requested will not be used for other jobs.&lt;br /&gt;
&lt;br /&gt;
How to avoid over- and underutilization? Many programs have options that will let them use only one thread (utilization of only one processor) or a specific number of threads. &lt;br /&gt;
&lt;br /&gt;
For example, Ansys has the &amp;lt;code&amp;gt;-np&amp;lt;/code&amp;gt; switch:&lt;br /&gt;
&lt;br /&gt;
 ansys -np N&lt;br /&gt;
&lt;br /&gt;
and Fluent has the &amp;lt;code&amp;gt;-t&amp;lt;/code&amp;gt; switch &lt;br /&gt;
&lt;br /&gt;
 fluent -tN&lt;br /&gt;
&lt;br /&gt;
where N matches the number of processors that you request in your job. &lt;br /&gt;
&lt;br /&gt;
If your program does not have an option to limit the number of processor, you can try to add this line in your job script, just before the line where your progam starts:&lt;br /&gt;
&lt;br /&gt;
 export OMP_NUM_THREADS=N&lt;br /&gt;
&lt;br /&gt;
Of course, N must match the number of processors that you request in your job. Alternatively, you could also request an entire node (all processors) in your job and let your program use all available resources of that node.&lt;br /&gt;
&lt;br /&gt;
== Avoid excessive reads and writes on your homedir ==&lt;br /&gt;
&lt;br /&gt;
Some programs read and write a lot of data to and from your home directory. This is not very efficient, on the nodes your home directory is a network share, so access is relatively slow and it keeps the master node unnecessarily busy. If you expect that your job will do a lot of reading and writing to disk, you can use the local disk on the node instead, which is mounted on /var/tmp on all nodes. You can do this by adding a few extra lines to your job script, right before the line that starts the program in your job, for example: &lt;br /&gt;
&lt;br /&gt;
&amp;lt;pre&amp;gt;&lt;br /&gt;
TMP=/var/tmp/${PBS_JOBID}&lt;br /&gt;
mkdir -p ${TMP}&lt;br /&gt;
/usr/bin/rsync -vax &amp;quot;${PBS_O_WORKDIR}/&amp;quot; ${TMP}/&lt;br /&gt;
cd ${TMP}&lt;br /&gt;
&amp;lt;/pre&amp;gt;&lt;br /&gt;
&lt;br /&gt;
Once your program is done you can copy the results back to your home directory and clean up by adding these two lines at the end of your job script:&lt;br /&gt;
&lt;br /&gt;
&amp;lt;pre&amp;gt;&lt;br /&gt;
/usr/bin/rsync -vax ${TMP}/ &amp;quot;${PBS_O_WORKDIR}/&amp;quot;&lt;br /&gt;
[ $? -eq 0 ] &amp;amp;&amp;amp; /bin/rm -rf ${TMP}&lt;br /&gt;
&amp;lt;/pre&amp;gt;&lt;br /&gt;
&lt;br /&gt;
This usually works best if you create a seperate directory in your homedir, move the necessary files and the job script to it and run your job from there. Otherwise you would end up copying your entire home directory to the node for no good reason.&lt;br /&gt;
&lt;br /&gt;
== Access to nodes ==&lt;br /&gt;
&lt;br /&gt;
All nodes are independant Linux machines and you could be tempted to log in to one of the nodes and work from there. This is however forbidden, any attempt to log in to a node will fail. There is one exception, you can log in to a node if you have a job running on it, this way you can check on the progress of your job and see if things are still working as intended. To check which node runs your job, type:&lt;br /&gt;
&lt;br /&gt;
 qstat -u $USER -n1&lt;br /&gt;
&lt;br /&gt;
This will get you a list of all your jobs, in the last column you'll see the nodes in use. If you log in to a node, please do not run any additional CPU intensive programs to avoid overutilization.&lt;br /&gt;
&lt;br /&gt;
If you must log in to a node in order to run software that can not be run from a script, you can start an interactive job. This is done using the &amp;lt;code&amp;gt;-I&amp;lt;/code&amp;gt; switch with qsub, like this:&lt;br /&gt;
&lt;br /&gt;
 qsub -I&lt;br /&gt;
&lt;br /&gt;
As soon as a node is assigned to you (this may take a while), you'll get a new command line prompt, as if you just logged in with ssh. This will reserve only one processor, you should take care that if you start a CPU intensive program, it does not use more than one processor. If you need more processors or if you want to use a specific node, you can request this for your interactive job with the &amp;lt;code&amp;gt;-l&amp;lt;/code&amp;gt; switch, for example, if you want to request 8 processors on node n10-08:&lt;br /&gt;
&lt;br /&gt;
 qsub -I -l nodes=n10-08:ppn=8&lt;br /&gt;
&lt;br /&gt;
If you want to run a progam with a graphical interface on a node, you'll need to make sure that X forwarding works when logged in to the master node. Then you can use the &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; switch start your interactive job with X forwarding enabled:&lt;br /&gt;
&lt;br /&gt;
 qsub -I -X&lt;br /&gt;
&lt;br /&gt;
It is important to know that an interactive can only be run in the normal queues, '''you can not run an interactive job in the guest queue!'''&lt;br /&gt;
&lt;br /&gt;
== MPI jobs ==&lt;br /&gt;
&lt;br /&gt;
Some workloads need OpenMPI to run, typically on two or more nodes at once. For such a job your job script usually contains a line like this:&lt;br /&gt;
&lt;br /&gt;
 module load mpi/openmpi-1.8.8-gnu&lt;br /&gt;
&lt;br /&gt;
And your actual workload would start with &amp;lt;code&amp;gt;mpirun&amp;lt;/code&amp;gt; or &amp;lt;code&amp;gt;mpiexec&amp;lt;/code&amp;gt; like this:&lt;br /&gt;
&lt;br /&gt;
&amp;lt;pre&amp;gt;&lt;br /&gt;
#PBS -l nodes=2:ppn=20&lt;br /&gt;
module load mpi/openmpi-1.8.8-gnu&lt;br /&gt;
cd $PBS_O_WORKDIR&lt;br /&gt;
mpirun whatever_workload_there_is&lt;br /&gt;
&amp;lt;/pre&amp;gt;&lt;br /&gt;
&lt;br /&gt;
OpenMPI uses rsh or ssh under water to communicate between the assigned nodes, in some cases this leads to &amp;lt;code&amp;gt;Host key verification failed&amp;lt;/code&amp;gt; errors and a premature termination of your job. To prevent this, you need to prepare a few files in your home directory. You only have to do this once on the master node.&lt;br /&gt;
&lt;br /&gt;
First of all, if you have never done this before on the master node, generate an ssh private/public keypair:&lt;br /&gt;
&lt;br /&gt;
 ssh-keygen&lt;br /&gt;
&lt;br /&gt;
Do not enter a passphrase, just press the enter key three times.&lt;br /&gt;
&lt;br /&gt;
Next type (or copy/paste) these two commands:&lt;br /&gt;
&lt;br /&gt;
 cat ${HOME}/.ssh/id_rsa.pub &amp;gt;&amp;gt; ${HOME}/.ssh/authorized_keys&lt;br /&gt;
 chmod go-rwx ${HOME}/.ssh/authorized_keys&lt;br /&gt;
&lt;br /&gt;
And finally type (or copy/paste) these two lines:&lt;br /&gt;
&lt;br /&gt;
 HPC=$(hostname | cut -c 4-5) ; \&lt;br /&gt;
 printf &amp;quot;host n${HPC}-* hpc${HPC}*\n\tStrictHostKeyChecking no\n\tUserKnownHostsFile /dev/null\n\tLogLevel QUIET\n&amp;quot; &amp;gt;&amp;gt; ${HOME}/.ssh/config&lt;br /&gt;
&lt;br /&gt;
The first line will give you a temporary &amp;lt;code&amp;gt; &amp;gt; &amp;lt;/code&amp;gt; prompt, this is normal behaviour.&lt;br /&gt;
&lt;br /&gt;
== A generic example of a job script ==&lt;br /&gt;
&lt;br /&gt;
The script below can be used as a starting point to create your own jobs. Feel free to copy, paste and modify it to your needs. Lines starting with # will not be executed and contain useful information.&lt;br /&gt;
&lt;br /&gt;
&amp;lt;pre&amp;gt;&lt;br /&gt;
#!/bin/bash&lt;br /&gt;
#&lt;br /&gt;
# Torque directives (#PBS) must always be at the start of a job script!&lt;br /&gt;
#&lt;br /&gt;
# Request nodes and processors per node&lt;br /&gt;
#&lt;br /&gt;
#PBS -l nodes=1:ppn=1&lt;br /&gt;
#&lt;br /&gt;
#&lt;br /&gt;
# Set the name of the job&lt;br /&gt;
#&lt;br /&gt;
#PBS -N name_of_job&lt;br /&gt;
#&lt;br /&gt;
#&lt;br /&gt;
# Set the mail options (type 'man qsub' for more information)&lt;br /&gt;
#&lt;br /&gt;
#PBS -m bea&lt;br /&gt;
#&lt;br /&gt;
#&lt;br /&gt;
# Set the email address where you want notifications sent to&lt;br /&gt;
# By default mail will be sent to your TU Delft mailbox&lt;br /&gt;
#&lt;br /&gt;
#PBS -M $USER@mailboxcluster.tudelft.net&lt;br /&gt;
#&lt;br /&gt;
#&lt;br /&gt;
# Set the rerunable flag, 'n' is not rerunable, default is 'y'&lt;br /&gt;
#&lt;br /&gt;
#PBS -r y&lt;br /&gt;
&lt;br /&gt;
# Make sure I'm the only one that can read my output&lt;br /&gt;
umask 0077&lt;br /&gt;
&lt;br /&gt;
# create a temporary directory in /var/tmp&lt;br /&gt;
TMP=/var/tmp/${PBS_JOBID}&lt;br /&gt;
mkdir -p ${TMP}&lt;br /&gt;
echo &amp;quot;Temporary work dir: ${TMP}&amp;quot;&lt;br /&gt;
if [ ! -d &amp;quot;${TMP}&amp;quot; ]; then&lt;br /&gt;
    echo &amp;quot;Cannot create temporary directory. Disk probably full.&amp;quot;&lt;br /&gt;
    exit 1&lt;br /&gt;
fi&lt;br /&gt;
&lt;br /&gt;
# copy the input files to ${TMP}&lt;br /&gt;
echo &amp;quot;Copying from ${PBS_O_WORKDIR}/ to ${TMP}/&amp;quot;&lt;br /&gt;
/usr/bin/rsync -vax &amp;quot;${PBS_O_WORKDIR}/&amp;quot; ${TMP}/&lt;br /&gt;
&lt;br /&gt;
cd ${TMP}&lt;br /&gt;
&lt;br /&gt;
# &lt;br /&gt;
&lt;br /&gt;
module load application1&lt;br /&gt;
module load application2&lt;br /&gt;
&lt;br /&gt;
export OMP_NUM_THREADS=1&lt;br /&gt;
&lt;br /&gt;
# Here is where the application is started on the node&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
# job done, copy everything back&lt;br /&gt;
echo &amp;quot;Copying from ${TMP}/ to ${PBS_O_WORKDIR}/&amp;quot;&lt;br /&gt;
/usr/bin/rsync -vax ${TMP}/ &amp;quot;${PBS_O_WORKDIR}/&amp;quot;&lt;br /&gt;
&lt;br /&gt;
# delete my temporary files&lt;br /&gt;
[ $? -eq 0 ] &amp;amp;&amp;amp; /bin/rm -rf ${TMP}&lt;br /&gt;
&amp;lt;/pre&amp;gt;&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=More_about_queues_and_nodes&amp;diff=165</id>
		<title>More about queues and nodes</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=More_about_queues_and_nodes&amp;diff=165"/>
		<updated>2020-07-01T10:11:02Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* MPI jobs */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== The different queues ==&lt;br /&gt;
&lt;br /&gt;
The larger hpc clusters, most notably hpc03, hpc06, hpc11 and hpc12, are shared by two or more research groups. On those clusters every group has their own queue, sometimes even more than one. These queues give exclusive and full access to a specific set of nodes. &lt;br /&gt;
&lt;br /&gt;
There is also a guest queue on every hpc cluster that gives access to all nodes, but with some restrictions, you will not be able to run non-rerunable and interactive jobs. &lt;br /&gt;
&lt;br /&gt;
In most cases, access to one of the queues is based on group membership in the Active Directory. If your netid is not a member of the right group, you default to the guest queue if you submit a job. If you have access to the group and bulk network shares of your research group, you should also have access to the normal queue on the hpc cluster. If not, contact the secretary in your research group and let him/her arrange the group membership of your netid.&lt;br /&gt;
&lt;br /&gt;
You can check your default queue by submitting a small test job and then have a look at the list with jobs with the qstat command.&lt;br /&gt;
&lt;br /&gt;
 [jsmith@hpc10 ~]$ echo &amp;quot;sleep 60&amp;quot; | qsub &lt;br /&gt;
 [jsmith@hpc10 ~]$ qstat -u jsmith&lt;br /&gt;
&lt;br /&gt;
If you see anything other than guest in the third column, then you are all set.&lt;br /&gt;
&lt;br /&gt;
There are two ways to select the guest queue; &lt;br /&gt;
&lt;br /&gt;
With the -q switch on the commandline:&lt;br /&gt;
&lt;br /&gt;
 qsub -q guest job1&lt;br /&gt;
&lt;br /&gt;
Or with a directive at the start of your job script:&lt;br /&gt;
&lt;br /&gt;
 #PBS -q guest&lt;br /&gt;
&lt;br /&gt;
It is important to know that a job in the guest queue can be interrupted and resumed at any time. You should make sure that the application in your job saves the intermediate results at regular intervals and that it knows how to continue when your job is resumed. If you neglect this, your job in the guest queue will start all over again every time it is interrupted and resumed.&lt;br /&gt;
&lt;br /&gt;
== The different nodes ==&lt;br /&gt;
&lt;br /&gt;
On most hpc clusters you'll find that worker nodes are not all identical, different series of nodes exist which were purchased at different times and with different specifications. To distinguish between the different series of nodes, they are labelled with properties like typea, typeb, typec, etc. On some hpc clusters, nodes have extra properties showing to which queue they belong or showing additional features, like an infiniband network or extra memory compared to similar nodes.&lt;br /&gt;
&lt;br /&gt;
A useful command that shows all nodes and how they are utilized is &amp;lt;code&amp;gt;LOCALnodeload.pl&amp;lt;/code&amp;gt;. A typical output looks like this:&lt;br /&gt;
&lt;br /&gt;
&amp;lt;pre&amp;gt;&lt;br /&gt;
[jsmith@hpc10 ~]$ LOCALnodeload.pl&lt;br /&gt;
Node       Np State/jobs Load  Properties&lt;br /&gt;
---------- -- ---------- ----- ----------&lt;br /&gt;
n10-01     12 12         12.01 typea     &lt;br /&gt;
n10-02     12 free        0.00 typea     &lt;br /&gt;
n10-03     12 free        0.00 typea     &lt;br /&gt;
n10-04     12 free        0.00 typea     &lt;br /&gt;
n10-05     16 12         11.93 typeb     &lt;br /&gt;
n10-06     16 free        0.00 typeb     &lt;br /&gt;
n10-07     16 offline     0.00 typeb     &lt;br /&gt;
n10-08     16 down        0.00 typeb     &lt;br /&gt;
&amp;lt;/pre&amp;gt;&lt;br /&gt;
&lt;br /&gt;
The first column (Node) shows the names of the nodes. The second column (Np) shows the total number of processors. The third column (State/jobs) shows the number of processors currenly in use or the status of the node (free, offline or down). The forth colum (Load) shows the actual load on the nodes. In an ideal situation the load matches the number of processors in use. The last column (Properties) shows the properties as described above. As you can see in the example, typea nodes have 12 processors and typeb nodes have 16. Node n10-01 is fully occupied, node n10-05 is running one or more jobs but still has 4 processors free. Nodes n10-07 and n10-08 cannot be used.&lt;br /&gt;
&lt;br /&gt;
== Selecting nodes ==&lt;br /&gt;
&lt;br /&gt;
If you submit a job, the scheduler automatically selects a node to run it. By default a jobs gets one node and one processor. You can manually select the number of processors and nodes for your job by using the &amp;lt;code&amp;gt;-l&amp;lt;/code&amp;gt; switch with the &amp;lt;code&amp;gt;qsub&amp;lt;/code&amp;gt; command. You can also select nodes by property. the &amp;lt;code&amp;gt;-l&amp;lt;/code&amp;gt; switch works like this:&lt;br /&gt;
&lt;br /&gt;
 qsub -l nodes=&amp;lt;x&amp;gt;:ppn=&amp;lt;c&amp;gt;:&amp;lt;property&amp;gt;:&amp;lt;property&amp;gt;...&lt;br /&gt;
&lt;br /&gt;
* &amp;lt;x&amp;gt; is either an amount of nodes or the name(s) of the selected node(s)&lt;br /&gt;
* &amp;lt;c&amp;gt; is number of processors per node&lt;br /&gt;
* &amp;lt;property&amp;gt; is any of the properties you see in Properties column of the LOCALnodeload.pl command.&lt;br /&gt;
&lt;br /&gt;
Examples:&lt;br /&gt;
&lt;br /&gt;
{|&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;qsub -l nodes=4&amp;lt;/code&amp;gt; || Request 4 nodes of any type&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;qsub -l nodes=n10-07+n10-08&amp;lt;/code&amp;gt; || Request 2 specific nodes by hostname&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;qsub -l nodes=4:ppn=2&amp;lt;/code&amp;gt; || Request 2 processors on each of four nodes&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;qsub -l nodes=1:ppn=4&amp;lt;/code&amp;gt; || Request 4 processors on one node&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;qsub -l nodes=2:typea&amp;lt;/code&amp;gt; || Request 2 nodes with the &amp;lt;code&amp;gt;typea&amp;lt;/code&amp;gt; property&lt;br /&gt;
|-&lt;br /&gt;
|}&lt;br /&gt;
Instead of using the -l or the -q switches on the commandline when you submit your job with qsub, you can also add them as a directive to your job script. For instance, if you add&lt;br /&gt;
&lt;br /&gt;
 #PBS -l nodes=1:ppn=4&lt;br /&gt;
 #PBS -q guest&lt;br /&gt;
&lt;br /&gt;
at the start of your script, you can just use&lt;br /&gt;
&lt;br /&gt;
 qsub job.sh&lt;br /&gt;
&lt;br /&gt;
instead of&lt;br /&gt;
&lt;br /&gt;
 qsub -l nodes=1:ppn=4 -q guest job.sh&lt;br /&gt;
&lt;br /&gt;
== Avoid over- and underutilization ==&lt;br /&gt;
&lt;br /&gt;
An important thing to consider when you create your own job script is matching the number of processors that you request with the number of processors that the software in your script will actually use. It is possible that you request only one processor and that your program will use all processors available on the nodes. This is called overutilization and is not very efficient when other jobs are already running on the same node and using the same processors.&lt;br /&gt;
&lt;br /&gt;
It is also possible that you request several (or all) processors and that your program will only use one. This will leave the other processors you claimed unused (underutilization), which is also not very efficient because the unused processors you requested will not be used for other jobs.&lt;br /&gt;
&lt;br /&gt;
How to avoid over- and underutilization? Many programs have options that will let them use only one thread (utilization of only one processor) or a specific number of threads. &lt;br /&gt;
&lt;br /&gt;
For example, Ansys has the &amp;lt;code&amp;gt;-np&amp;lt;/code&amp;gt; switch:&lt;br /&gt;
&lt;br /&gt;
 ansys -np N&lt;br /&gt;
&lt;br /&gt;
and Fluent has the &amp;lt;code&amp;gt;-t&amp;lt;/code&amp;gt; switch &lt;br /&gt;
&lt;br /&gt;
 fluent -tN&lt;br /&gt;
&lt;br /&gt;
where N matches the number of processors that you request in your job. &lt;br /&gt;
&lt;br /&gt;
If your program does not have an option to limit the number of processor, you can try to add this line in your job script, just before the line where your progam starts:&lt;br /&gt;
&lt;br /&gt;
 export OMP_NUM_THREADS=N&lt;br /&gt;
&lt;br /&gt;
Of course, N must match the number of processors that you request in your job. Alternatively, you could also request an entire node (all processors) in your job and let your program use all available resources of that node.&lt;br /&gt;
&lt;br /&gt;
== Avoid excessive reads and writes on your homedir ==&lt;br /&gt;
&lt;br /&gt;
Some programs read and write a lot of data to and from your home directory. This is not very efficient, on the nodes your home directory is a network share, so access is relatively slow and it keeps the master node unnecessarily busy. If you expect that your job will do a lot of reading and writing to disk, you can use the local disk on the node instead, which is mounted on /var/tmp on all nodes. You can do this by adding a few extra lines to your job script, right before the line that starts the program in your job, for example: &lt;br /&gt;
&lt;br /&gt;
&amp;lt;pre&amp;gt;&lt;br /&gt;
TMP=/var/tmp/${PBS_JOBID}&lt;br /&gt;
mkdir -p ${TMP}&lt;br /&gt;
/usr/bin/rsync -vax &amp;quot;${PBS_O_WORKDIR}/&amp;quot; ${TMP}/&lt;br /&gt;
cd ${TMP}&lt;br /&gt;
&amp;lt;/pre&amp;gt;&lt;br /&gt;
&lt;br /&gt;
Once your program is done you can copy the results back to your home directory and clean up by adding these two lines at the end of your job script:&lt;br /&gt;
&lt;br /&gt;
&amp;lt;pre&amp;gt;&lt;br /&gt;
/usr/bin/rsync -vax ${TMP}/ &amp;quot;${PBS_O_WORKDIR}/&amp;quot;&lt;br /&gt;
[ $? -eq 0 ] &amp;amp;&amp;amp; /bin/rm -rf ${TMP}&lt;br /&gt;
&amp;lt;/pre&amp;gt;&lt;br /&gt;
&lt;br /&gt;
This usually works best if you create a seperate directory in your homedir, move the necessary files and the job script to it and run your job from there. Otherwise you would end up copying your entire home directory to the node for no good reason.&lt;br /&gt;
&lt;br /&gt;
== Access to nodes ==&lt;br /&gt;
&lt;br /&gt;
All nodes are independant Linux machines and you could be tempted to log in to one of the nodes and work from there. This is however forbidden, any attempt to log in to a node will fail. There is one exception, you can log in to a node if you have a job running on it, this way you can check on the progress of your job and see if things are still working as intended. To check which node runs your job, type:&lt;br /&gt;
&lt;br /&gt;
 qstat -u $USER -n1&lt;br /&gt;
&lt;br /&gt;
This will get you a list of all your jobs, in the last column you'll see the nodes in use. If you log in to a node, please do not run any additional CPU intensive programs to avoid overutilization.&lt;br /&gt;
&lt;br /&gt;
If you must log in to a node in order to run software that can not be run from a script, you can start an interactive job. This is done using the &amp;lt;code&amp;gt;-I&amp;lt;/code&amp;gt; switch with qsub, like this:&lt;br /&gt;
&lt;br /&gt;
 qsub -I&lt;br /&gt;
&lt;br /&gt;
As soon as a node is assigned to you (this may take a while), you'll get a new command line prompt, as if you just logged in with ssh. This will reserve only one processor, you should take care that if you start a CPU intensive program, it does not use more than one processor. If you need more processors or if you want to use a specific node, you can request this for your interactive job with the &amp;lt;code&amp;gt;-l&amp;lt;/code&amp;gt; switch, for example, if you want to request 8 processors on node n10-08:&lt;br /&gt;
&lt;br /&gt;
 qsub -I -l nodes=n10-08:ppn=8&lt;br /&gt;
&lt;br /&gt;
If you want to run a progam with a graphical interface on a node, you'll need to make sure that X forwarding works when logged in to the master node. Then you can use the &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; switch start your interactive job with X forwarding enabled:&lt;br /&gt;
&lt;br /&gt;
 qsub -I -X&lt;br /&gt;
&lt;br /&gt;
It is important to know that an interactive can only be run in the normal queues, '''you can not run an interactive job in the guest queue!'''&lt;br /&gt;
&lt;br /&gt;
== MPI jobs ==&lt;br /&gt;
&lt;br /&gt;
Some workloads need OpenMPI to run, typically on two or more nodes at once. For such a job your job script usually contains a line like this:&lt;br /&gt;
&lt;br /&gt;
 module load mpi/openmpi-1.8.8-gnu&lt;br /&gt;
&lt;br /&gt;
And your actual workload would start with &amp;lt;code&amp;gt;mpirun&amp;lt;/code&amp;gt; or &amp;lt;code&amp;gt;mpiexec&amp;lt;/code&amp;gt; like this:&lt;br /&gt;
&lt;br /&gt;
&amp;lt;pre&amp;gt;&lt;br /&gt;
#PBS -l nodes=2:ppn=20&lt;br /&gt;
module load mpi/openmpi-1.8.8-gnu&lt;br /&gt;
cd $PBS_O_WORKDIR&lt;br /&gt;
mpirun whatever_workload_there_is&lt;br /&gt;
&amp;lt;/pre&amp;gt;&lt;br /&gt;
&lt;br /&gt;
OpenMPI uses rsh or ssh under water to communicate between the assigned nodes, in some cases this leads to &amp;lt;code&amp;gt;Host key verification failed&amp;lt;/code&amp;gt; errors and a premature termination of your job. To prevent this, you need to prepare a few files in your home directory. You only have to do this once on the master node.&lt;br /&gt;
&lt;br /&gt;
First of all, if you have never done this before on the master node, generate an ssh private/public keypair:&lt;br /&gt;
&lt;br /&gt;
 ssh-keygen&lt;br /&gt;
&lt;br /&gt;
Do not enter a passphrase, just press the enter key three times.&lt;br /&gt;
&lt;br /&gt;
Next type these two commands:&lt;br /&gt;
&lt;br /&gt;
 cat ${HOME}/.ssh/id_rsa.pub &amp;gt;&amp;gt; ${HOME}/.ssh/authorized_keys&lt;br /&gt;
 chmod go-rwx ${HOME}/.ssh/authorized_keys&lt;br /&gt;
&lt;br /&gt;
And finally type (or copy/paste) these three lines:&lt;br /&gt;
&lt;br /&gt;
 HPC=$(hostname | cut -c 4-5) ; \&lt;br /&gt;
 printf &amp;quot;host n${HPC}-* hpc${HPC}*\n\tStrictHostKeyChecking no\n\tUserKnownHostsFile /dev/null\n\tLogLevel QUIET\n&amp;quot; &amp;gt;&amp;gt; ${HOME}/.ssh/config&lt;br /&gt;
&lt;br /&gt;
The first line will give you a temporary &amp;lt;code&amp;gt; &amp;gt; &amp;lt;/code&amp;gt; prompt, this is normal behaviour.&lt;br /&gt;
&lt;br /&gt;
== A generic example of a job script ==&lt;br /&gt;
&lt;br /&gt;
The script below can be used as a starting point to create your own jobs. Feel free to copy, paste and modify it to your needs. Lines starting with # will not be executed and contain useful information.&lt;br /&gt;
&lt;br /&gt;
&amp;lt;pre&amp;gt;&lt;br /&gt;
#!/bin/bash&lt;br /&gt;
#&lt;br /&gt;
# Torque directives (#PBS) must always be at the start of a job script!&lt;br /&gt;
#&lt;br /&gt;
# Request nodes and processors per node&lt;br /&gt;
#&lt;br /&gt;
#PBS -l nodes=1:ppn=1&lt;br /&gt;
#&lt;br /&gt;
#&lt;br /&gt;
# Set the name of the job&lt;br /&gt;
#&lt;br /&gt;
#PBS -N name_of_job&lt;br /&gt;
#&lt;br /&gt;
#&lt;br /&gt;
# Set the mail options (type 'man qsub' for more information)&lt;br /&gt;
#&lt;br /&gt;
#PBS -m bea&lt;br /&gt;
#&lt;br /&gt;
#&lt;br /&gt;
# Set the email address where you want notifications sent to&lt;br /&gt;
# By default mail will be sent to your TU Delft mailbox&lt;br /&gt;
#&lt;br /&gt;
#PBS -M $USER@mailboxcluster.tudelft.net&lt;br /&gt;
#&lt;br /&gt;
#&lt;br /&gt;
# Set the rerunable flag, 'n' is not rerunable, default is 'y'&lt;br /&gt;
#&lt;br /&gt;
#PBS -r y&lt;br /&gt;
&lt;br /&gt;
# Make sure I'm the only one that can read my output&lt;br /&gt;
umask 0077&lt;br /&gt;
&lt;br /&gt;
# create a temporary directory in /var/tmp&lt;br /&gt;
TMP=/var/tmp/${PBS_JOBID}&lt;br /&gt;
mkdir -p ${TMP}&lt;br /&gt;
echo &amp;quot;Temporary work dir: ${TMP}&amp;quot;&lt;br /&gt;
if [ ! -d &amp;quot;${TMP}&amp;quot; ]; then&lt;br /&gt;
    echo &amp;quot;Cannot create temporary directory. Disk probably full.&amp;quot;&lt;br /&gt;
    exit 1&lt;br /&gt;
fi&lt;br /&gt;
&lt;br /&gt;
# copy the input files to ${TMP}&lt;br /&gt;
echo &amp;quot;Copying from ${PBS_O_WORKDIR}/ to ${TMP}/&amp;quot;&lt;br /&gt;
/usr/bin/rsync -vax &amp;quot;${PBS_O_WORKDIR}/&amp;quot; ${TMP}/&lt;br /&gt;
&lt;br /&gt;
cd ${TMP}&lt;br /&gt;
&lt;br /&gt;
# &lt;br /&gt;
&lt;br /&gt;
module load application1&lt;br /&gt;
module load application2&lt;br /&gt;
&lt;br /&gt;
export OMP_NUM_THREADS=1&lt;br /&gt;
&lt;br /&gt;
# Here is where the application is started on the node&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
# job done, copy everything back&lt;br /&gt;
echo &amp;quot;Copying from ${TMP}/ to ${PBS_O_WORKDIR}/&amp;quot;&lt;br /&gt;
/usr/bin/rsync -vax ${TMP}/ &amp;quot;${PBS_O_WORKDIR}/&amp;quot;&lt;br /&gt;
&lt;br /&gt;
# delete my temporary files&lt;br /&gt;
[ $? -eq 0 ] &amp;amp;&amp;amp; /bin/rm -rf ${TMP}&lt;br /&gt;
&amp;lt;/pre&amp;gt;&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=More_about_queues_and_nodes&amp;diff=164</id>
		<title>More about queues and nodes</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=More_about_queues_and_nodes&amp;diff=164"/>
		<updated>2020-07-01T10:10:04Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* MPI jobs */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== The different queues ==&lt;br /&gt;
&lt;br /&gt;
The larger hpc clusters, most notably hpc03, hpc06, hpc11 and hpc12, are shared by two or more research groups. On those clusters every group has their own queue, sometimes even more than one. These queues give exclusive and full access to a specific set of nodes. &lt;br /&gt;
&lt;br /&gt;
There is also a guest queue on every hpc cluster that gives access to all nodes, but with some restrictions, you will not be able to run non-rerunable and interactive jobs. &lt;br /&gt;
&lt;br /&gt;
In most cases, access to one of the queues is based on group membership in the Active Directory. If your netid is not a member of the right group, you default to the guest queue if you submit a job. If you have access to the group and bulk network shares of your research group, you should also have access to the normal queue on the hpc cluster. If not, contact the secretary in your research group and let him/her arrange the group membership of your netid.&lt;br /&gt;
&lt;br /&gt;
You can check your default queue by submitting a small test job and then have a look at the list with jobs with the qstat command.&lt;br /&gt;
&lt;br /&gt;
 [jsmith@hpc10 ~]$ echo &amp;quot;sleep 60&amp;quot; | qsub &lt;br /&gt;
 [jsmith@hpc10 ~]$ qstat -u jsmith&lt;br /&gt;
&lt;br /&gt;
If you see anything other than guest in the third column, then you are all set.&lt;br /&gt;
&lt;br /&gt;
There are two ways to select the guest queue; &lt;br /&gt;
&lt;br /&gt;
With the -q switch on the commandline:&lt;br /&gt;
&lt;br /&gt;
 qsub -q guest job1&lt;br /&gt;
&lt;br /&gt;
Or with a directive at the start of your job script:&lt;br /&gt;
&lt;br /&gt;
 #PBS -q guest&lt;br /&gt;
&lt;br /&gt;
It is important to know that a job in the guest queue can be interrupted and resumed at any time. You should make sure that the application in your job saves the intermediate results at regular intervals and that it knows how to continue when your job is resumed. If you neglect this, your job in the guest queue will start all over again every time it is interrupted and resumed.&lt;br /&gt;
&lt;br /&gt;
== The different nodes ==&lt;br /&gt;
&lt;br /&gt;
On most hpc clusters you'll find that worker nodes are not all identical, different series of nodes exist which were purchased at different times and with different specifications. To distinguish between the different series of nodes, they are labelled with properties like typea, typeb, typec, etc. On some hpc clusters, nodes have extra properties showing to which queue they belong or showing additional features, like an infiniband network or extra memory compared to similar nodes.&lt;br /&gt;
&lt;br /&gt;
A useful command that shows all nodes and how they are utilized is &amp;lt;code&amp;gt;LOCALnodeload.pl&amp;lt;/code&amp;gt;. A typical output looks like this:&lt;br /&gt;
&lt;br /&gt;
&amp;lt;pre&amp;gt;&lt;br /&gt;
[jsmith@hpc10 ~]$ LOCALnodeload.pl&lt;br /&gt;
Node       Np State/jobs Load  Properties&lt;br /&gt;
---------- -- ---------- ----- ----------&lt;br /&gt;
n10-01     12 12         12.01 typea     &lt;br /&gt;
n10-02     12 free        0.00 typea     &lt;br /&gt;
n10-03     12 free        0.00 typea     &lt;br /&gt;
n10-04     12 free        0.00 typea     &lt;br /&gt;
n10-05     16 12         11.93 typeb     &lt;br /&gt;
n10-06     16 free        0.00 typeb     &lt;br /&gt;
n10-07     16 offline     0.00 typeb     &lt;br /&gt;
n10-08     16 down        0.00 typeb     &lt;br /&gt;
&amp;lt;/pre&amp;gt;&lt;br /&gt;
&lt;br /&gt;
The first column (Node) shows the names of the nodes. The second column (Np) shows the total number of processors. The third column (State/jobs) shows the number of processors currenly in use or the status of the node (free, offline or down). The forth colum (Load) shows the actual load on the nodes. In an ideal situation the load matches the number of processors in use. The last column (Properties) shows the properties as described above. As you can see in the example, typea nodes have 12 processors and typeb nodes have 16. Node n10-01 is fully occupied, node n10-05 is running one or more jobs but still has 4 processors free. Nodes n10-07 and n10-08 cannot be used.&lt;br /&gt;
&lt;br /&gt;
== Selecting nodes ==&lt;br /&gt;
&lt;br /&gt;
If you submit a job, the scheduler automatically selects a node to run it. By default a jobs gets one node and one processor. You can manually select the number of processors and nodes for your job by using the &amp;lt;code&amp;gt;-l&amp;lt;/code&amp;gt; switch with the &amp;lt;code&amp;gt;qsub&amp;lt;/code&amp;gt; command. You can also select nodes by property. the &amp;lt;code&amp;gt;-l&amp;lt;/code&amp;gt; switch works like this:&lt;br /&gt;
&lt;br /&gt;
 qsub -l nodes=&amp;lt;x&amp;gt;:ppn=&amp;lt;c&amp;gt;:&amp;lt;property&amp;gt;:&amp;lt;property&amp;gt;...&lt;br /&gt;
&lt;br /&gt;
* &amp;lt;x&amp;gt; is either an amount of nodes or the name(s) of the selected node(s)&lt;br /&gt;
* &amp;lt;c&amp;gt; is number of processors per node&lt;br /&gt;
* &amp;lt;property&amp;gt; is any of the properties you see in Properties column of the LOCALnodeload.pl command.&lt;br /&gt;
&lt;br /&gt;
Examples:&lt;br /&gt;
&lt;br /&gt;
{|&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;qsub -l nodes=4&amp;lt;/code&amp;gt; || Request 4 nodes of any type&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;qsub -l nodes=n10-07+n10-08&amp;lt;/code&amp;gt; || Request 2 specific nodes by hostname&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;qsub -l nodes=4:ppn=2&amp;lt;/code&amp;gt; || Request 2 processors on each of four nodes&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;qsub -l nodes=1:ppn=4&amp;lt;/code&amp;gt; || Request 4 processors on one node&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;qsub -l nodes=2:typea&amp;lt;/code&amp;gt; || Request 2 nodes with the &amp;lt;code&amp;gt;typea&amp;lt;/code&amp;gt; property&lt;br /&gt;
|-&lt;br /&gt;
|}&lt;br /&gt;
Instead of using the -l or the -q switches on the commandline when you submit your job with qsub, you can also add them as a directive to your job script. For instance, if you add&lt;br /&gt;
&lt;br /&gt;
 #PBS -l nodes=1:ppn=4&lt;br /&gt;
 #PBS -q guest&lt;br /&gt;
&lt;br /&gt;
at the start of your script, you can just use&lt;br /&gt;
&lt;br /&gt;
 qsub job.sh&lt;br /&gt;
&lt;br /&gt;
instead of&lt;br /&gt;
&lt;br /&gt;
 qsub -l nodes=1:ppn=4 -q guest job.sh&lt;br /&gt;
&lt;br /&gt;
== Avoid over- and underutilization ==&lt;br /&gt;
&lt;br /&gt;
An important thing to consider when you create your own job script is matching the number of processors that you request with the number of processors that the software in your script will actually use. It is possible that you request only one processor and that your program will use all processors available on the nodes. This is called overutilization and is not very efficient when other jobs are already running on the same node and using the same processors.&lt;br /&gt;
&lt;br /&gt;
It is also possible that you request several (or all) processors and that your program will only use one. This will leave the other processors you claimed unused (underutilization), which is also not very efficient because the unused processors you requested will not be used for other jobs.&lt;br /&gt;
&lt;br /&gt;
How to avoid over- and underutilization? Many programs have options that will let them use only one thread (utilization of only one processor) or a specific number of threads. &lt;br /&gt;
&lt;br /&gt;
For example, Ansys has the &amp;lt;code&amp;gt;-np&amp;lt;/code&amp;gt; switch:&lt;br /&gt;
&lt;br /&gt;
 ansys -np N&lt;br /&gt;
&lt;br /&gt;
and Fluent has the &amp;lt;code&amp;gt;-t&amp;lt;/code&amp;gt; switch &lt;br /&gt;
&lt;br /&gt;
 fluent -tN&lt;br /&gt;
&lt;br /&gt;
where N matches the number of processors that you request in your job. &lt;br /&gt;
&lt;br /&gt;
If your program does not have an option to limit the number of processor, you can try to add this line in your job script, just before the line where your progam starts:&lt;br /&gt;
&lt;br /&gt;
 export OMP_NUM_THREADS=N&lt;br /&gt;
&lt;br /&gt;
Of course, N must match the number of processors that you request in your job. Alternatively, you could also request an entire node (all processors) in your job and let your program use all available resources of that node.&lt;br /&gt;
&lt;br /&gt;
== Avoid excessive reads and writes on your homedir ==&lt;br /&gt;
&lt;br /&gt;
Some programs read and write a lot of data to and from your home directory. This is not very efficient, on the nodes your home directory is a network share, so access is relatively slow and it keeps the master node unnecessarily busy. If you expect that your job will do a lot of reading and writing to disk, you can use the local disk on the node instead, which is mounted on /var/tmp on all nodes. You can do this by adding a few extra lines to your job script, right before the line that starts the program in your job, for example: &lt;br /&gt;
&lt;br /&gt;
&amp;lt;pre&amp;gt;&lt;br /&gt;
TMP=/var/tmp/${PBS_JOBID}&lt;br /&gt;
mkdir -p ${TMP}&lt;br /&gt;
/usr/bin/rsync -vax &amp;quot;${PBS_O_WORKDIR}/&amp;quot; ${TMP}/&lt;br /&gt;
cd ${TMP}&lt;br /&gt;
&amp;lt;/pre&amp;gt;&lt;br /&gt;
&lt;br /&gt;
Once your program is done you can copy the results back to your home directory and clean up by adding these two lines at the end of your job script:&lt;br /&gt;
&lt;br /&gt;
&amp;lt;pre&amp;gt;&lt;br /&gt;
/usr/bin/rsync -vax ${TMP}/ &amp;quot;${PBS_O_WORKDIR}/&amp;quot;&lt;br /&gt;
[ $? -eq 0 ] &amp;amp;&amp;amp; /bin/rm -rf ${TMP}&lt;br /&gt;
&amp;lt;/pre&amp;gt;&lt;br /&gt;
&lt;br /&gt;
This usually works best if you create a seperate directory in your homedir, move the necessary files and the job script to it and run your job from there. Otherwise you would end up copying your entire home directory to the node for no good reason.&lt;br /&gt;
&lt;br /&gt;
== Access to nodes ==&lt;br /&gt;
&lt;br /&gt;
All nodes are independant Linux machines and you could be tempted to log in to one of the nodes and work from there. This is however forbidden, any attempt to log in to a node will fail. There is one exception, you can log in to a node if you have a job running on it, this way you can check on the progress of your job and see if things are still working as intended. To check which node runs your job, type:&lt;br /&gt;
&lt;br /&gt;
 qstat -u $USER -n1&lt;br /&gt;
&lt;br /&gt;
This will get you a list of all your jobs, in the last column you'll see the nodes in use. If you log in to a node, please do not run any additional CPU intensive programs to avoid overutilization.&lt;br /&gt;
&lt;br /&gt;
If you must log in to a node in order to run software that can not be run from a script, you can start an interactive job. This is done using the &amp;lt;code&amp;gt;-I&amp;lt;/code&amp;gt; switch with qsub, like this:&lt;br /&gt;
&lt;br /&gt;
 qsub -I&lt;br /&gt;
&lt;br /&gt;
As soon as a node is assigned to you (this may take a while), you'll get a new command line prompt, as if you just logged in with ssh. This will reserve only one processor, you should take care that if you start a CPU intensive program, it does not use more than one processor. If you need more processors or if you want to use a specific node, you can request this for your interactive job with the &amp;lt;code&amp;gt;-l&amp;lt;/code&amp;gt; switch, for example, if you want to request 8 processors on node n10-08:&lt;br /&gt;
&lt;br /&gt;
 qsub -I -l nodes=n10-08:ppn=8&lt;br /&gt;
&lt;br /&gt;
If you want to run a progam with a graphical interface on a node, you'll need to make sure that X forwarding works when logged in to the master node. Then you can use the &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; switch start your interactive job with X forwarding enabled:&lt;br /&gt;
&lt;br /&gt;
 qsub -I -X&lt;br /&gt;
&lt;br /&gt;
It is important to know that an interactive can only be run in the normal queues, '''you can not run an interactive job in the guest queue!'''&lt;br /&gt;
&lt;br /&gt;
== MPI jobs ==&lt;br /&gt;
&lt;br /&gt;
Some workloads need OpenMPI to run, typically on two or more nodes at once. For such a job your job script usually contains a line like this:&lt;br /&gt;
&lt;br /&gt;
 module load mpi/openmpi-1.8.8-gnu&lt;br /&gt;
&lt;br /&gt;
And your actual workload would start with &amp;lt;code&amp;gt;mpirun&amp;lt;/code&amp;gt; or &amp;lt;code&amp;gt;mpiexec&amp;lt;/code&amp;gt; like this:&lt;br /&gt;
&lt;br /&gt;
&amp;lt;pre&amp;gt;&lt;br /&gt;
#PBS -l nodes=2:ppn=20&lt;br /&gt;
module load mpi/openmpi-1.8.8-gnu&lt;br /&gt;
cd $PBS_O_WORKDIR&lt;br /&gt;
mpirun whatever_workload_there_is&lt;br /&gt;
&amp;lt;/pre&amp;gt;&lt;br /&gt;
&lt;br /&gt;
OpenMPI uses rsh or ssh under water to communicate between the assigned nodes, in some cases this leads to &amp;lt;code&amp;gt;Host key verification failed&amp;lt;/code&amp;gt; errors and a premature termination of your job. To prevent this, you need to prepare a few files in your home directory. You only have to do this once on the master node.&lt;br /&gt;
&lt;br /&gt;
First of all, if you have never done this before on the master node, generate an ssh private/public keypair:&lt;br /&gt;
&lt;br /&gt;
 ssh-keygen&lt;br /&gt;
&lt;br /&gt;
Do not enter a passphrase, just press the enter key three times. Next type these two commands:&lt;br /&gt;
&lt;br /&gt;
 cat ${HOME}/.ssh/id_rsa.pub &amp;gt;&amp;gt; ${HOME}/.ssh/authorized_keys&lt;br /&gt;
 chmod go-rwx ${HOME}/.ssh/authorized_keys&lt;br /&gt;
&lt;br /&gt;
And finally type (or copy/paste) these three lines:&lt;br /&gt;
&lt;br /&gt;
 HPC=$(hostname | cut -c 4-5) ; \&lt;br /&gt;
 printf &amp;quot;host n${HPC}-* hpc${HPC}*\n\tStrictHostKeyChecking no\n\tUserKnownHostsFile /dev/null\n\tLogLevel QUIET\n&amp;quot; &amp;gt;&amp;gt; ${HOME}/.ssh/config&lt;br /&gt;
&lt;br /&gt;
The first line will give you a temporary &amp;lt;code&amp;gt; &amp;gt; &amp;lt;/code&amp;gt; prompt, this is normal behaviour.&lt;br /&gt;
&lt;br /&gt;
== A generic example of a job script ==&lt;br /&gt;
&lt;br /&gt;
The script below can be used as a starting point to create your own jobs. Feel free to copy, paste and modify it to your needs. Lines starting with # will not be executed and contain useful information.&lt;br /&gt;
&lt;br /&gt;
&amp;lt;pre&amp;gt;&lt;br /&gt;
#!/bin/bash&lt;br /&gt;
#&lt;br /&gt;
# Torque directives (#PBS) must always be at the start of a job script!&lt;br /&gt;
#&lt;br /&gt;
# Request nodes and processors per node&lt;br /&gt;
#&lt;br /&gt;
#PBS -l nodes=1:ppn=1&lt;br /&gt;
#&lt;br /&gt;
#&lt;br /&gt;
# Set the name of the job&lt;br /&gt;
#&lt;br /&gt;
#PBS -N name_of_job&lt;br /&gt;
#&lt;br /&gt;
#&lt;br /&gt;
# Set the mail options (type 'man qsub' for more information)&lt;br /&gt;
#&lt;br /&gt;
#PBS -m bea&lt;br /&gt;
#&lt;br /&gt;
#&lt;br /&gt;
# Set the email address where you want notifications sent to&lt;br /&gt;
# By default mail will be sent to your TU Delft mailbox&lt;br /&gt;
#&lt;br /&gt;
#PBS -M $USER@mailboxcluster.tudelft.net&lt;br /&gt;
#&lt;br /&gt;
#&lt;br /&gt;
# Set the rerunable flag, 'n' is not rerunable, default is 'y'&lt;br /&gt;
#&lt;br /&gt;
#PBS -r y&lt;br /&gt;
&lt;br /&gt;
# Make sure I'm the only one that can read my output&lt;br /&gt;
umask 0077&lt;br /&gt;
&lt;br /&gt;
# create a temporary directory in /var/tmp&lt;br /&gt;
TMP=/var/tmp/${PBS_JOBID}&lt;br /&gt;
mkdir -p ${TMP}&lt;br /&gt;
echo &amp;quot;Temporary work dir: ${TMP}&amp;quot;&lt;br /&gt;
if [ ! -d &amp;quot;${TMP}&amp;quot; ]; then&lt;br /&gt;
    echo &amp;quot;Cannot create temporary directory. Disk probably full.&amp;quot;&lt;br /&gt;
    exit 1&lt;br /&gt;
fi&lt;br /&gt;
&lt;br /&gt;
# copy the input files to ${TMP}&lt;br /&gt;
echo &amp;quot;Copying from ${PBS_O_WORKDIR}/ to ${TMP}/&amp;quot;&lt;br /&gt;
/usr/bin/rsync -vax &amp;quot;${PBS_O_WORKDIR}/&amp;quot; ${TMP}/&lt;br /&gt;
&lt;br /&gt;
cd ${TMP}&lt;br /&gt;
&lt;br /&gt;
# &lt;br /&gt;
&lt;br /&gt;
module load application1&lt;br /&gt;
module load application2&lt;br /&gt;
&lt;br /&gt;
export OMP_NUM_THREADS=1&lt;br /&gt;
&lt;br /&gt;
# Here is where the application is started on the node&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
# job done, copy everything back&lt;br /&gt;
echo &amp;quot;Copying from ${TMP}/ to ${PBS_O_WORKDIR}/&amp;quot;&lt;br /&gt;
/usr/bin/rsync -vax ${TMP}/ &amp;quot;${PBS_O_WORKDIR}/&amp;quot;&lt;br /&gt;
&lt;br /&gt;
# delete my temporary files&lt;br /&gt;
[ $? -eq 0 ] &amp;amp;&amp;amp; /bin/rm -rf ${TMP}&lt;br /&gt;
&amp;lt;/pre&amp;gt;&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=More_about_queues_and_nodes&amp;diff=163</id>
		<title>More about queues and nodes</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=More_about_queues_and_nodes&amp;diff=163"/>
		<updated>2020-07-01T09:22:20Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* Access to nodes */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== The different queues ==&lt;br /&gt;
&lt;br /&gt;
The larger hpc clusters, most notably hpc03, hpc06, hpc11 and hpc12, are shared by two or more research groups. On those clusters every group has their own queue, sometimes even more than one. These queues give exclusive and full access to a specific set of nodes. &lt;br /&gt;
&lt;br /&gt;
There is also a guest queue on every hpc cluster that gives access to all nodes, but with some restrictions, you will not be able to run non-rerunable and interactive jobs. &lt;br /&gt;
&lt;br /&gt;
In most cases, access to one of the queues is based on group membership in the Active Directory. If your netid is not a member of the right group, you default to the guest queue if you submit a job. If you have access to the group and bulk network shares of your research group, you should also have access to the normal queue on the hpc cluster. If not, contact the secretary in your research group and let him/her arrange the group membership of your netid.&lt;br /&gt;
&lt;br /&gt;
You can check your default queue by submitting a small test job and then have a look at the list with jobs with the qstat command.&lt;br /&gt;
&lt;br /&gt;
 [jsmith@hpc10 ~]$ echo &amp;quot;sleep 60&amp;quot; | qsub &lt;br /&gt;
 [jsmith@hpc10 ~]$ qstat -u jsmith&lt;br /&gt;
&lt;br /&gt;
If you see anything other than guest in the third column, then you are all set.&lt;br /&gt;
&lt;br /&gt;
There are two ways to select the guest queue; &lt;br /&gt;
&lt;br /&gt;
With the -q switch on the commandline:&lt;br /&gt;
&lt;br /&gt;
 qsub -q guest job1&lt;br /&gt;
&lt;br /&gt;
Or with a directive at the start of your job script:&lt;br /&gt;
&lt;br /&gt;
 #PBS -q guest&lt;br /&gt;
&lt;br /&gt;
It is important to know that a job in the guest queue can be interrupted and resumed at any time. You should make sure that the application in your job saves the intermediate results at regular intervals and that it knows how to continue when your job is resumed. If you neglect this, your job in the guest queue will start all over again every time it is interrupted and resumed.&lt;br /&gt;
&lt;br /&gt;
== The different nodes ==&lt;br /&gt;
&lt;br /&gt;
On most hpc clusters you'll find that worker nodes are not all identical, different series of nodes exist which were purchased at different times and with different specifications. To distinguish between the different series of nodes, they are labelled with properties like typea, typeb, typec, etc. On some hpc clusters, nodes have extra properties showing to which queue they belong or showing additional features, like an infiniband network or extra memory compared to similar nodes.&lt;br /&gt;
&lt;br /&gt;
A useful command that shows all nodes and how they are utilized is &amp;lt;code&amp;gt;LOCALnodeload.pl&amp;lt;/code&amp;gt;. A typical output looks like this:&lt;br /&gt;
&lt;br /&gt;
&amp;lt;pre&amp;gt;&lt;br /&gt;
[jsmith@hpc10 ~]$ LOCALnodeload.pl&lt;br /&gt;
Node       Np State/jobs Load  Properties&lt;br /&gt;
---------- -- ---------- ----- ----------&lt;br /&gt;
n10-01     12 12         12.01 typea     &lt;br /&gt;
n10-02     12 free        0.00 typea     &lt;br /&gt;
n10-03     12 free        0.00 typea     &lt;br /&gt;
n10-04     12 free        0.00 typea     &lt;br /&gt;
n10-05     16 12         11.93 typeb     &lt;br /&gt;
n10-06     16 free        0.00 typeb     &lt;br /&gt;
n10-07     16 offline     0.00 typeb     &lt;br /&gt;
n10-08     16 down        0.00 typeb     &lt;br /&gt;
&amp;lt;/pre&amp;gt;&lt;br /&gt;
&lt;br /&gt;
The first column (Node) shows the names of the nodes. The second column (Np) shows the total number of processors. The third column (State/jobs) shows the number of processors currenly in use or the status of the node (free, offline or down). The forth colum (Load) shows the actual load on the nodes. In an ideal situation the load matches the number of processors in use. The last column (Properties) shows the properties as described above. As you can see in the example, typea nodes have 12 processors and typeb nodes have 16. Node n10-01 is fully occupied, node n10-05 is running one or more jobs but still has 4 processors free. Nodes n10-07 and n10-08 cannot be used.&lt;br /&gt;
&lt;br /&gt;
== Selecting nodes ==&lt;br /&gt;
&lt;br /&gt;
If you submit a job, the scheduler automatically selects a node to run it. By default a jobs gets one node and one processor. You can manually select the number of processors and nodes for your job by using the &amp;lt;code&amp;gt;-l&amp;lt;/code&amp;gt; switch with the &amp;lt;code&amp;gt;qsub&amp;lt;/code&amp;gt; command. You can also select nodes by property. the &amp;lt;code&amp;gt;-l&amp;lt;/code&amp;gt; switch works like this:&lt;br /&gt;
&lt;br /&gt;
 qsub -l nodes=&amp;lt;x&amp;gt;:ppn=&amp;lt;c&amp;gt;:&amp;lt;property&amp;gt;:&amp;lt;property&amp;gt;...&lt;br /&gt;
&lt;br /&gt;
* &amp;lt;x&amp;gt; is either an amount of nodes or the name(s) of the selected node(s)&lt;br /&gt;
* &amp;lt;c&amp;gt; is number of processors per node&lt;br /&gt;
* &amp;lt;property&amp;gt; is any of the properties you see in Properties column of the LOCALnodeload.pl command.&lt;br /&gt;
&lt;br /&gt;
Examples:&lt;br /&gt;
&lt;br /&gt;
{|&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;qsub -l nodes=4&amp;lt;/code&amp;gt; || Request 4 nodes of any type&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;qsub -l nodes=n10-07+n10-08&amp;lt;/code&amp;gt; || Request 2 specific nodes by hostname&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;qsub -l nodes=4:ppn=2&amp;lt;/code&amp;gt; || Request 2 processors on each of four nodes&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;qsub -l nodes=1:ppn=4&amp;lt;/code&amp;gt; || Request 4 processors on one node&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;qsub -l nodes=2:typea&amp;lt;/code&amp;gt; || Request 2 nodes with the &amp;lt;code&amp;gt;typea&amp;lt;/code&amp;gt; property&lt;br /&gt;
|-&lt;br /&gt;
|}&lt;br /&gt;
Instead of using the -l or the -q switches on the commandline when you submit your job with qsub, you can also add them as a directive to your job script. For instance, if you add&lt;br /&gt;
&lt;br /&gt;
 #PBS -l nodes=1:ppn=4&lt;br /&gt;
 #PBS -q guest&lt;br /&gt;
&lt;br /&gt;
at the start of your script, you can just use&lt;br /&gt;
&lt;br /&gt;
 qsub job.sh&lt;br /&gt;
&lt;br /&gt;
instead of&lt;br /&gt;
&lt;br /&gt;
 qsub -l nodes=1:ppn=4 -q guest job.sh&lt;br /&gt;
&lt;br /&gt;
== Avoid over- and underutilization ==&lt;br /&gt;
&lt;br /&gt;
An important thing to consider when you create your own job script is matching the number of processors that you request with the number of processors that the software in your script will actually use. It is possible that you request only one processor and that your program will use all processors available on the nodes. This is called overutilization and is not very efficient when other jobs are already running on the same node and using the same processors.&lt;br /&gt;
&lt;br /&gt;
It is also possible that you request several (or all) processors and that your program will only use one. This will leave the other processors you claimed unused (underutilization), which is also not very efficient because the unused processors you requested will not be used for other jobs.&lt;br /&gt;
&lt;br /&gt;
How to avoid over- and underutilization? Many programs have options that will let them use only one thread (utilization of only one processor) or a specific number of threads. &lt;br /&gt;
&lt;br /&gt;
For example, Ansys has the &amp;lt;code&amp;gt;-np&amp;lt;/code&amp;gt; switch:&lt;br /&gt;
&lt;br /&gt;
 ansys -np N&lt;br /&gt;
&lt;br /&gt;
and Fluent has the &amp;lt;code&amp;gt;-t&amp;lt;/code&amp;gt; switch &lt;br /&gt;
&lt;br /&gt;
 fluent -tN&lt;br /&gt;
&lt;br /&gt;
where N matches the number of processors that you request in your job. &lt;br /&gt;
&lt;br /&gt;
If your program does not have an option to limit the number of processor, you can try to add this line in your job script, just before the line where your progam starts:&lt;br /&gt;
&lt;br /&gt;
 export OMP_NUM_THREADS=N&lt;br /&gt;
&lt;br /&gt;
Of course, N must match the number of processors that you request in your job. Alternatively, you could also request an entire node (all processors) in your job and let your program use all available resources of that node.&lt;br /&gt;
&lt;br /&gt;
== Avoid excessive reads and writes on your homedir ==&lt;br /&gt;
&lt;br /&gt;
Some programs read and write a lot of data to and from your home directory. This is not very efficient, on the nodes your home directory is a network share, so access is relatively slow and it keeps the master node unnecessarily busy. If you expect that your job will do a lot of reading and writing to disk, you can use the local disk on the node instead, which is mounted on /var/tmp on all nodes. You can do this by adding a few extra lines to your job script, right before the line that starts the program in your job, for example: &lt;br /&gt;
&lt;br /&gt;
&amp;lt;pre&amp;gt;&lt;br /&gt;
TMP=/var/tmp/${PBS_JOBID}&lt;br /&gt;
mkdir -p ${TMP}&lt;br /&gt;
/usr/bin/rsync -vax &amp;quot;${PBS_O_WORKDIR}/&amp;quot; ${TMP}/&lt;br /&gt;
cd ${TMP}&lt;br /&gt;
&amp;lt;/pre&amp;gt;&lt;br /&gt;
&lt;br /&gt;
Once your program is done you can copy the results back to your home directory and clean up by adding these two lines at the end of your job script:&lt;br /&gt;
&lt;br /&gt;
&amp;lt;pre&amp;gt;&lt;br /&gt;
/usr/bin/rsync -vax ${TMP}/ &amp;quot;${PBS_O_WORKDIR}/&amp;quot;&lt;br /&gt;
[ $? -eq 0 ] &amp;amp;&amp;amp; /bin/rm -rf ${TMP}&lt;br /&gt;
&amp;lt;/pre&amp;gt;&lt;br /&gt;
&lt;br /&gt;
This usually works best if you create a seperate directory in your homedir, move the necessary files and the job script to it and run your job from there. Otherwise you would end up copying your entire home directory to the node for no good reason.&lt;br /&gt;
&lt;br /&gt;
== Access to nodes ==&lt;br /&gt;
&lt;br /&gt;
All nodes are independant Linux machines and you could be tempted to log in to one of the nodes and work from there. This is however forbidden, any attempt to log in to a node will fail. There is one exception, you can log in to a node if you have a job running on it, this way you can check on the progress of your job and see if things are still working as intended. To check which node runs your job, type:&lt;br /&gt;
&lt;br /&gt;
 qstat -u $USER -n1&lt;br /&gt;
&lt;br /&gt;
This will get you a list of all your jobs, in the last column you'll see the nodes in use. If you log in to a node, please do not run any additional CPU intensive programs to avoid overutilization.&lt;br /&gt;
&lt;br /&gt;
If you must log in to a node in order to run software that can not be run from a script, you can start an interactive job. This is done using the &amp;lt;code&amp;gt;-I&amp;lt;/code&amp;gt; switch with qsub, like this:&lt;br /&gt;
&lt;br /&gt;
 qsub -I&lt;br /&gt;
&lt;br /&gt;
As soon as a node is assigned to you (this may take a while), you'll get a new command line prompt, as if you just logged in with ssh. This will reserve only one processor, you should take care that if you start a CPU intensive program, it does not use more than one processor. If you need more processors or if you want to use a specific node, you can request this for your interactive job with the &amp;lt;code&amp;gt;-l&amp;lt;/code&amp;gt; switch, for example, if you want to request 8 processors on node n10-08:&lt;br /&gt;
&lt;br /&gt;
 qsub -I -l nodes=n10-08:ppn=8&lt;br /&gt;
&lt;br /&gt;
If you want to run a progam with a graphical interface on a node, you'll need to make sure that X forwarding works when logged in to the master node. Then you can use the &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; switch start your interactive job with X forwarding enabled:&lt;br /&gt;
&lt;br /&gt;
 qsub -I -X&lt;br /&gt;
&lt;br /&gt;
It is important to know that an interactive can only be run in the normal queues, '''you can not run an interactive job in the guest queue!'''&lt;br /&gt;
&lt;br /&gt;
== MPI jobs ==&lt;br /&gt;
&lt;br /&gt;
== A generic example of a job script ==&lt;br /&gt;
&lt;br /&gt;
The script below can be used as a starting point to create your own jobs. Feel free to copy, paste and modify it to your needs. Lines starting with # will not be executed and contain useful information.&lt;br /&gt;
&lt;br /&gt;
&amp;lt;pre&amp;gt;&lt;br /&gt;
#!/bin/bash&lt;br /&gt;
#&lt;br /&gt;
# Torque directives (#PBS) must always be at the start of a job script!&lt;br /&gt;
#&lt;br /&gt;
# Request nodes and processors per node&lt;br /&gt;
#&lt;br /&gt;
#PBS -l nodes=1:ppn=1&lt;br /&gt;
#&lt;br /&gt;
#&lt;br /&gt;
# Set the name of the job&lt;br /&gt;
#&lt;br /&gt;
#PBS -N name_of_job&lt;br /&gt;
#&lt;br /&gt;
#&lt;br /&gt;
# Set the mail options (type 'man qsub' for more information)&lt;br /&gt;
#&lt;br /&gt;
#PBS -m bea&lt;br /&gt;
#&lt;br /&gt;
#&lt;br /&gt;
# Set the email address where you want notifications sent to&lt;br /&gt;
# By default mail will be sent to your TU Delft mailbox&lt;br /&gt;
#&lt;br /&gt;
#PBS -M $USER@mailboxcluster.tudelft.net&lt;br /&gt;
#&lt;br /&gt;
#&lt;br /&gt;
# Set the rerunable flag, 'n' is not rerunable, default is 'y'&lt;br /&gt;
#&lt;br /&gt;
#PBS -r y&lt;br /&gt;
&lt;br /&gt;
# Make sure I'm the only one that can read my output&lt;br /&gt;
umask 0077&lt;br /&gt;
&lt;br /&gt;
# create a temporary directory in /var/tmp&lt;br /&gt;
TMP=/var/tmp/${PBS_JOBID}&lt;br /&gt;
mkdir -p ${TMP}&lt;br /&gt;
echo &amp;quot;Temporary work dir: ${TMP}&amp;quot;&lt;br /&gt;
if [ ! -d &amp;quot;${TMP}&amp;quot; ]; then&lt;br /&gt;
    echo &amp;quot;Cannot create temporary directory. Disk probably full.&amp;quot;&lt;br /&gt;
    exit 1&lt;br /&gt;
fi&lt;br /&gt;
&lt;br /&gt;
# copy the input files to ${TMP}&lt;br /&gt;
echo &amp;quot;Copying from ${PBS_O_WORKDIR}/ to ${TMP}/&amp;quot;&lt;br /&gt;
/usr/bin/rsync -vax &amp;quot;${PBS_O_WORKDIR}/&amp;quot; ${TMP}/&lt;br /&gt;
&lt;br /&gt;
cd ${TMP}&lt;br /&gt;
&lt;br /&gt;
# &lt;br /&gt;
&lt;br /&gt;
module load application1&lt;br /&gt;
module load application2&lt;br /&gt;
&lt;br /&gt;
export OMP_NUM_THREADS=1&lt;br /&gt;
&lt;br /&gt;
# Here is where the application is started on the node&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
# job done, copy everything back&lt;br /&gt;
echo &amp;quot;Copying from ${TMP}/ to ${PBS_O_WORKDIR}/&amp;quot;&lt;br /&gt;
/usr/bin/rsync -vax ${TMP}/ &amp;quot;${PBS_O_WORKDIR}/&amp;quot;&lt;br /&gt;
&lt;br /&gt;
# delete my temporary files&lt;br /&gt;
[ $? -eq 0 ] &amp;amp;&amp;amp; /bin/rm -rf ${TMP}&lt;br /&gt;
&amp;lt;/pre&amp;gt;&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Introduction&amp;diff=162</id>
		<title>Introduction</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Introduction&amp;diff=162"/>
		<updated>2020-04-28T14:54:54Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* About the TU Delft hpc clusters */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== About the TU Delft hpc clusters ==&lt;br /&gt;
&lt;br /&gt;
Several research groups at the TU Delft have exclusive access to hpc (high performance computing) clusters that are managed and maintained by the ICT Department. Currently there are ten, they are typically named hpcXX.tudelft.net where the XX part is a number ranging from 03 to 27.&lt;br /&gt;
&lt;br /&gt;
{| class=&amp;quot;wikitable&amp;quot;&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Name&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Faculty&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Dept./group(s)&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| OS&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Nodes&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Total # CPU's&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Contact&lt;br /&gt;
|-&lt;br /&gt;
| hpc03.tudelft.net || CiTG || GRS-PSG/MGP || CentOS 7 || 17 || 264 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc05.tudelft.net || TNW || QN/BN || CentOS 7 || 44 || 896 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc06.tudelft.net || 3ME || PME/MSE/DCSC/MTT || CentOS 7 || 118 || 2084 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc07.tudelft.net || CiTG || ES-SCS || CentOS 7 || 8 || 256 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc08.tudelft.net || CiTG || HE-EFM || CentOS 7 || 12 || 240 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc09.tudelft.net || CiTG || ES-PAVE || CentOS 7 || 12 || 192 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc10.tudelft.net || CiTG || Dream Teams || CentOS 7 || 8 || 112 || [[file:halbeheer.png|link=]]&lt;br /&gt;
|-&lt;br /&gt;
| hpc11.tudelft.net || TNW || RID/CHEME || CentOS 7 || 45 || 1744 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc12.tudelft.net || L&amp;amp;R || ASM/AWEP || CentOS 7 || 68 || 1536 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc27.tudelft.net || CiTG || 3Md-AM || CentOS 7 || 20 || 400 ||&lt;br /&gt;
|-			&lt;br /&gt;
|}&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are [https://en.wikipedia.org/wiki/Beowulf_cluster Beowulf] style clusters. The purpose of this wiki is to guide new users into using these clusters. Beowulf clusters are typically made of a bunch common and identical computers or servers linked together in a network. This can be used to perform parallel computing tasks on a much larger scale than would be possible on a single workstation or a heavy duty server.&lt;br /&gt;
&lt;br /&gt;
== About Linux ==&lt;br /&gt;
&lt;br /&gt;
The operating system on all our clusters is [http://www.centos.org/ CentOS 7], this is a well known server class Linux distribution. In order to use our hpc clusters you should at least have some basic knowledge of Linux on the command line. You will not find a Linux tutorial in this wiki, there are many tutorials on the internet that will teach you how to use Linux. Just use the phrase &amp;quot;Linux for beginners&amp;quot; in your favorite search engine and you'll find plenty of sources online, for example the [http://www.ee.surrey.ac.uk/Teaching/Unix/ UNIX Tutorial for Beginners] from the [http://www.surrey.ac.uk/ University of Surrey]. You can als opt to buy a book, [http://www.dummies.com/computers/operating-systems/linux/linux-for-dummies-9th-edition/ Linux for dummies] would be a good start, but there are many others that'll do just fine. If you have never used Linux on the command line before, please take the time to learn the basics, it will be well worth the effort.&lt;br /&gt;
&lt;br /&gt;
== More details ==&lt;br /&gt;
&lt;br /&gt;
Typically a Beowulf style clusters consists of a master node and several worker nodes. The master node is the machine where you log in to and where you prepare and manage your parallel jobs. A scheduler (Maui) and a resource manager (Torque) are both running on the master node. They both work together and provide a mechanism to submit jobs that will be run on the worker nodes where the actual parallel processing takes place. These nodes are stand alone computers connected through a local network to the master node and patiently waiting until the scheduler and the resource manager on the master node tells them to run a (part of a) parallel job.&lt;br /&gt;
&lt;br /&gt;
== About this wiki: contact and info ==&lt;br /&gt;
&lt;br /&gt;
TThe hpc clusters are managed and maintained by the Linux systems department at the Shared Service Center ICT. You can contact us for any questions and comments about the hpc clusters at this address: &lt;br /&gt;
&lt;br /&gt;
[[file:beheer-o.png]] &lt;br /&gt;
&lt;br /&gt;
You can also contact us if you find incorrect information on this wiki, if you are missing important information that you want added or if you have something to contribute to this wiki.&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=161</id>
		<title>How to log in</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=161"/>
		<updated>2020-03-20T09:41:59Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* Basic access */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== Basics ==&lt;br /&gt;
&lt;br /&gt;
In order to log in you need a valid [https://www.tudelft.nl/en/student/ict/ict-facilities/netid-account/ NetID] that is enabled for access to one of the clusters. To have your NetID enabled, ask the contact person for the cluster you want access to (See the [[Introduction]] page) or ask the secretary of the department or research group where you work, he or she can tell you who the contact person is for an hpc cluster.&lt;br /&gt;
&lt;br /&gt;
If you want to collaborate with someone from outside the TU Delft and you want that person to access the same cluster as you, then you can request a guest account at the servicedesk. Once the guest account is created, it can be enabled for access to one of the clusters. '''Do not share your own NetID with others! Your NetID is for personal use only, sharing it with other people is forbidden!'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are only accessible via ssh (secure shell). To log in you need an ssh client. If you use Windows, you can download and use [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY], Macintosh and Linux computers already have an ssh client built in. There is no need to log in to a bastion host first, you can log in from anywhere in the world.&lt;br /&gt;
&lt;br /&gt;
Every time you log in you'll see an motd (message of the day). Please pay attention to this, it may contain information about current events on the cluster. It also shows you an email address that you can use if there are problems or if you have questions. You can mail at any time you like, but you'll usually only get a reply during office hours, there is no 24/7 support on the hpc clusters.&lt;br /&gt;
&lt;br /&gt;
'''Please note:''' You have a local home directory on the cluster. No backups will be made of the home directories, if you make a mistake and loose a file, it is gone forever. '''Making backups of your important files on the hpc clusters is your own responsibility!'''&lt;br /&gt;
&lt;br /&gt;
== How to log in from Windows ==&lt;br /&gt;
&lt;br /&gt;
An often used ssh client for Windows is [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY]. If you use a Windows PC or a laptop that is provided and maintained by the TU Delft, it is already there, if you use your own Windows computer, you'll have to download and install it first from [http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html www.chiark.greenend.org.uk].&lt;br /&gt;
&lt;br /&gt;
Once you have started PuTTY you'll get a configuration window als shown.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can type the name of the cluster, for example 'hpc10.tudelft.net', in the field 'Host Name'. You can save the settings for a cluster by choosing a name for the session, enter it in the 'Saved Sessions' box and click 'Save'. The next time you open PuTTY the settings can be restored by clicking on the name and a click on 'Load'.&lt;br /&gt;
&lt;br /&gt;
By clicking 'Open' a connection is made to hpc10.tudelft.net. If this is the first time you are connecting to the cluster, PuTTY will warn you that you are connecting to a previously unknown host.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_02.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can confirm this connection by clicking 'yes' so PuTTY will store the host key in its database. You'll have to do this only once for every new server that you connect to.&lt;br /&gt;
&lt;br /&gt;
After you have accepted the connection, PuTTY opens a console and asks for your loginname. Once you have entered your NetID and your password, you are logged in to the cluster.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_03.png|center]]&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from Windows ==&lt;br /&gt;
&lt;br /&gt;
WinSCP is a freeware ftp client with support for secure file protocols sftp and scp. The login screen depicted below shows the settings for access within the University.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
After clicking on 'Login' a warning about host keys will appear, similar to PuTTY. If it is the first time, you can safely click on 'yes' to accept.&lt;br /&gt;
&lt;br /&gt;
Press 'Login'. If everything is setup correctly you are using the ssh tunnel the PuTTY bastion session has built and you are logged in on the cluster.&lt;br /&gt;
&lt;br /&gt;
When logged in, you will see the commander-like interface below.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_02.png|center|530px]]&lt;br /&gt;
&lt;br /&gt;
The left side displays the files in your home directory on the windows system. The right side displays the files on the cluster. You can drag and drop files between the two windows or select files with the mouse and press 'F5' for transfer to the other window.&lt;br /&gt;
&lt;br /&gt;
== How to log in from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
Mac OS X is in fact a full blown UNIX system under the hood. A terminal program comes standard with OS X, navigate with the finder to Applications -&amp;gt; Utilities to find it, the program is called 'Terminal'. There are many different flavours of Linux, each with their own desktop environments and customizations. Nearly every Linux system comes with a terminal program of some sort. Find the terminal program on your system and run it, once you get a command prompt type:&lt;br /&gt;
&lt;br /&gt;
 ssh &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
Where &amp;lt;netid&amp;gt; is of course your own netid.&lt;br /&gt;
&lt;br /&gt;
Upon first access, ssh requires you to check if the host key is correct. Since this is the first time you connect to the system, it's fine to accept the host key by pressing 'y'.&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh:&lt;br /&gt;
&lt;br /&gt;
 scp &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your remote home directory (folder) to the local home directory on your workstation or laptop. &lt;br /&gt;
&lt;br /&gt;
If you want to copy an entire directory with all its content, use the '-r' option, like this:&lt;br /&gt;
&lt;br /&gt;
 scp -r &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
This will copy the directory 'projects' from your remote home directory to the local home directory on your workstation or laptop.&lt;br /&gt;
&lt;br /&gt;
Of course, it is also possible to copy files and directories from your workstation or laptop to the remote machine:&lt;br /&gt;
&lt;br /&gt;
 scp -r projects.txt &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
 scp -r projects/ &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
And this will copy the directory 'projects' and all its content from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
&lt;br /&gt;
=== Windows ===&lt;br /&gt;
&lt;br /&gt;
There are several options for displaying graphical applications on a Windows system. A basic option is [http://www.straightrunning.com/XmingNotes/ Xming]. You can download it from: http://www.straightrunning.com/XmingNotes/ The site asks for a donation to use the latest version, but there is also a public domain version which can be used for free. Once you have installed it, you can find it in the usual place in the start menu. Before you log in to an hpc cluster and run graphical applications, you need to start Xming first. It will not show much, only a small X in the system tray once it's running. You also need to enable X forwarding in PuTTY. For that you need to click on the + next to SSH in the session panel (left part of the window), then click on X11. You'll see a line on the right side that says &amp;quot;Enable X11 forwarding&amp;quot;, click once on the white square next to that line and it will show a check mark. &lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_04.png|center]]&lt;br /&gt;
&lt;br /&gt;
Now you can continue logging in to the hpc cluster. To check if X forwarding works, type &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that will follow the mouse pointer.&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
[[File:Xeyes.png|center]]&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
An alternative option for displaying graphical applications is Exceed, this is a commercial package for which the TU Delft has a license. It is not available by default on the standard Windows systems, but it can be installed upon request if you contact the servicedesk.&lt;br /&gt;
&lt;br /&gt;
A second alternative is [https://mobaxterm.mobatek.net/ MobaXterm], this is an all-in-one solution containing an ssh client, X server and more. You can download a free version from: https://mobaxterm.mobatek.net/download.html The free version has some limitations, but it's still perfectly usable in our environment.&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
== How to access your Windows network shares ==&lt;br /&gt;
&lt;br /&gt;
=== Authenticate ===&lt;br /&gt;
&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
&lt;br /&gt;
 kinit&lt;br /&gt;
&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
=== Basic access ===&lt;br /&gt;
&lt;br /&gt;
Now just type:&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
&lt;br /&gt;
or&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-umbrella/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
&lt;br /&gt;
You can access your Windows home directory as well, the path is:&lt;br /&gt;
&lt;br /&gt;
 /tudelft/$USER/staff-homes/&lt;br /&gt;
&lt;br /&gt;
followed by the first letter of your surname and then your netid. For example, if your name would be John Smith and your netid jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
If you are a student, replace 'staff' with 'student' to access your files.&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
=== Important note ===&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your netid. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your netid yourself. If your netid would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well. Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own netid!&lt;br /&gt;
&lt;br /&gt;
'''If you use anthing other than your own netid you'll create a recipe for potential disaster!'''&lt;br /&gt;
&lt;br /&gt;
If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths. So &lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-group/&lt;br /&gt;
&lt;br /&gt;
only works if your netid is jsmith and&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/pbrown/staff-group/&lt;br /&gt;
&lt;br /&gt;
only works works if your netid is pbrown.&lt;br /&gt;
&lt;br /&gt;
If both netids have the same privileges for accessing the group volumes, then these commands give access to the same data.&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=160</id>
		<title>How to log in</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=160"/>
		<updated>2020-03-20T09:33:29Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* How to access your Windows network shares */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== Basics ==&lt;br /&gt;
&lt;br /&gt;
In order to log in you need a valid [https://www.tudelft.nl/en/student/ict/ict-facilities/netid-account/ NetID] that is enabled for access to one of the clusters. To have your NetID enabled, ask the contact person for the cluster you want access to (See the [[Introduction]] page) or ask the secretary of the department or research group where you work, he or she can tell you who the contact person is for an hpc cluster.&lt;br /&gt;
&lt;br /&gt;
If you want to collaborate with someone from outside the TU Delft and you want that person to access the same cluster as you, then you can request a guest account at the servicedesk. Once the guest account is created, it can be enabled for access to one of the clusters. '''Do not share your own NetID with others! Your NetID is for personal use only, sharing it with other people is forbidden!'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are only accessible via ssh (secure shell). To log in you need an ssh client. If you use Windows, you can download and use [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY], Macintosh and Linux computers already have an ssh client built in. There is no need to log in to a bastion host first, you can log in from anywhere in the world.&lt;br /&gt;
&lt;br /&gt;
Every time you log in you'll see an motd (message of the day). Please pay attention to this, it may contain information about current events on the cluster. It also shows you an email address that you can use if there are problems or if you have questions. You can mail at any time you like, but you'll usually only get a reply during office hours, there is no 24/7 support on the hpc clusters.&lt;br /&gt;
&lt;br /&gt;
'''Please note:''' You have a local home directory on the cluster. No backups will be made of the home directories, if you make a mistake and loose a file, it is gone forever. '''Making backups of your important files on the hpc clusters is your own responsibility!'''&lt;br /&gt;
&lt;br /&gt;
== How to log in from Windows ==&lt;br /&gt;
&lt;br /&gt;
An often used ssh client for Windows is [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY]. If you use a Windows PC or a laptop that is provided and maintained by the TU Delft, it is already there, if you use your own Windows computer, you'll have to download and install it first from [http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html www.chiark.greenend.org.uk].&lt;br /&gt;
&lt;br /&gt;
Once you have started PuTTY you'll get a configuration window als shown.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can type the name of the cluster, for example 'hpc10.tudelft.net', in the field 'Host Name'. You can save the settings for a cluster by choosing a name for the session, enter it in the 'Saved Sessions' box and click 'Save'. The next time you open PuTTY the settings can be restored by clicking on the name and a click on 'Load'.&lt;br /&gt;
&lt;br /&gt;
By clicking 'Open' a connection is made to hpc10.tudelft.net. If this is the first time you are connecting to the cluster, PuTTY will warn you that you are connecting to a previously unknown host.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_02.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can confirm this connection by clicking 'yes' so PuTTY will store the host key in its database. You'll have to do this only once for every new server that you connect to.&lt;br /&gt;
&lt;br /&gt;
After you have accepted the connection, PuTTY opens a console and asks for your loginname. Once you have entered your NetID and your password, you are logged in to the cluster.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_03.png|center]]&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from Windows ==&lt;br /&gt;
&lt;br /&gt;
WinSCP is a freeware ftp client with support for secure file protocols sftp and scp. The login screen depicted below shows the settings for access within the University.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
After clicking on 'Login' a warning about host keys will appear, similar to PuTTY. If it is the first time, you can safely click on 'yes' to accept.&lt;br /&gt;
&lt;br /&gt;
Press 'Login'. If everything is setup correctly you are using the ssh tunnel the PuTTY bastion session has built and you are logged in on the cluster.&lt;br /&gt;
&lt;br /&gt;
When logged in, you will see the commander-like interface below.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_02.png|center|530px]]&lt;br /&gt;
&lt;br /&gt;
The left side displays the files in your home directory on the windows system. The right side displays the files on the cluster. You can drag and drop files between the two windows or select files with the mouse and press 'F5' for transfer to the other window.&lt;br /&gt;
&lt;br /&gt;
== How to log in from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
Mac OS X is in fact a full blown UNIX system under the hood. A terminal program comes standard with OS X, navigate with the finder to Applications -&amp;gt; Utilities to find it, the program is called 'Terminal'. There are many different flavours of Linux, each with their own desktop environments and customizations. Nearly every Linux system comes with a terminal program of some sort. Find the terminal program on your system and run it, once you get a command prompt type:&lt;br /&gt;
&lt;br /&gt;
 ssh &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
Where &amp;lt;netid&amp;gt; is of course your own netid.&lt;br /&gt;
&lt;br /&gt;
Upon first access, ssh requires you to check if the host key is correct. Since this is the first time you connect to the system, it's fine to accept the host key by pressing 'y'.&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh:&lt;br /&gt;
&lt;br /&gt;
 scp &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your remote home directory (folder) to the local home directory on your workstation or laptop. &lt;br /&gt;
&lt;br /&gt;
If you want to copy an entire directory with all its content, use the '-r' option, like this:&lt;br /&gt;
&lt;br /&gt;
 scp -r &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
This will copy the directory 'projects' from your remote home directory to the local home directory on your workstation or laptop.&lt;br /&gt;
&lt;br /&gt;
Of course, it is also possible to copy files and directories from your workstation or laptop to the remote machine:&lt;br /&gt;
&lt;br /&gt;
 scp -r projects.txt &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
 scp -r projects/ &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
And this will copy the directory 'projects' and all its content from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
&lt;br /&gt;
=== Windows ===&lt;br /&gt;
&lt;br /&gt;
There are several options for displaying graphical applications on a Windows system. A basic option is [http://www.straightrunning.com/XmingNotes/ Xming]. You can download it from: http://www.straightrunning.com/XmingNotes/ The site asks for a donation to use the latest version, but there is also a public domain version which can be used for free. Once you have installed it, you can find it in the usual place in the start menu. Before you log in to an hpc cluster and run graphical applications, you need to start Xming first. It will not show much, only a small X in the system tray once it's running. You also need to enable X forwarding in PuTTY. For that you need to click on the + next to SSH in the session panel (left part of the window), then click on X11. You'll see a line on the right side that says &amp;quot;Enable X11 forwarding&amp;quot;, click once on the white square next to that line and it will show a check mark. &lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_04.png|center]]&lt;br /&gt;
&lt;br /&gt;
Now you can continue logging in to the hpc cluster. To check if X forwarding works, type &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that will follow the mouse pointer.&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
[[File:Xeyes.png|center]]&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
An alternative option for displaying graphical applications is Exceed, this is a commercial package for which the TU Delft has a license. It is not available by default on the standard Windows systems, but it can be installed upon request if you contact the servicedesk.&lt;br /&gt;
&lt;br /&gt;
A second alternative is [https://mobaxterm.mobatek.net/ MobaXterm], this is an all-in-one solution containing an ssh client, X server and more. You can download a free version from: https://mobaxterm.mobatek.net/download.html The free version has some limitations, but it's still perfectly usable in our environment.&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
== How to access your Windows network shares ==&lt;br /&gt;
&lt;br /&gt;
=== Authenticate ===&lt;br /&gt;
&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
&lt;br /&gt;
 kinit&lt;br /&gt;
&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
=== Basic access ===&lt;br /&gt;
&lt;br /&gt;
Now just type:&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
&lt;br /&gt;
or&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
&lt;br /&gt;
You can access your Windows home directory as well, the path is:&lt;br /&gt;
&lt;br /&gt;
 /tudelft/$USER/staff-homes/&lt;br /&gt;
&lt;br /&gt;
followed by the first letter of your surname and then your netid. For example, if your name would be John Smith and your netid jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
If you are a student, replace 'staff' with 'student' to access your files.&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
=== Important note ===&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your netid. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your netid yourself. If your netid would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well. Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own netid!&lt;br /&gt;
&lt;br /&gt;
'''If you use anthing other than your own netid you'll create a recipe for potential disaster!'''&lt;br /&gt;
&lt;br /&gt;
If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths. So &lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-group/&lt;br /&gt;
&lt;br /&gt;
only works if your netid is jsmith and&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/pbrown/staff-group/&lt;br /&gt;
&lt;br /&gt;
only works works if your netid is pbrown.&lt;br /&gt;
&lt;br /&gt;
If both netids have the same privileges for accessing the group volumes, then these commands give access to the same data.&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=159</id>
		<title>How to log in</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=159"/>
		<updated>2020-03-20T09:22:52Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* How to access your Windows network shares */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== Basics ==&lt;br /&gt;
&lt;br /&gt;
In order to log in you need a valid [https://www.tudelft.nl/en/student/ict/ict-facilities/netid-account/ NetID] that is enabled for access to one of the clusters. To have your NetID enabled, ask the contact person for the cluster you want access to (See the [[Introduction]] page) or ask the secretary of the department or research group where you work, he or she can tell you who the contact person is for an hpc cluster.&lt;br /&gt;
&lt;br /&gt;
If you want to collaborate with someone from outside the TU Delft and you want that person to access the same cluster as you, then you can request a guest account at the servicedesk. Once the guest account is created, it can be enabled for access to one of the clusters. '''Do not share your own NetID with others! Your NetID is for personal use only, sharing it with other people is forbidden!'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are only accessible via ssh (secure shell). To log in you need an ssh client. If you use Windows, you can download and use [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY], Macintosh and Linux computers already have an ssh client built in. There is no need to log in to a bastion host first, you can log in from anywhere in the world.&lt;br /&gt;
&lt;br /&gt;
Every time you log in you'll see an motd (message of the day). Please pay attention to this, it may contain information about current events on the cluster. It also shows you an email address that you can use if there are problems or if you have questions. You can mail at any time you like, but you'll usually only get a reply during office hours, there is no 24/7 support on the hpc clusters.&lt;br /&gt;
&lt;br /&gt;
'''Please note:''' You have a local home directory on the cluster. No backups will be made of the home directories, if you make a mistake and loose a file, it is gone forever. '''Making backups of your important files on the hpc clusters is your own responsibility!'''&lt;br /&gt;
&lt;br /&gt;
== How to log in from Windows ==&lt;br /&gt;
&lt;br /&gt;
An often used ssh client for Windows is [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY]. If you use a Windows PC or a laptop that is provided and maintained by the TU Delft, it is already there, if you use your own Windows computer, you'll have to download and install it first from [http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html www.chiark.greenend.org.uk].&lt;br /&gt;
&lt;br /&gt;
Once you have started PuTTY you'll get a configuration window als shown.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can type the name of the cluster, for example 'hpc10.tudelft.net', in the field 'Host Name'. You can save the settings for a cluster by choosing a name for the session, enter it in the 'Saved Sessions' box and click 'Save'. The next time you open PuTTY the settings can be restored by clicking on the name and a click on 'Load'.&lt;br /&gt;
&lt;br /&gt;
By clicking 'Open' a connection is made to hpc10.tudelft.net. If this is the first time you are connecting to the cluster, PuTTY will warn you that you are connecting to a previously unknown host.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_02.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can confirm this connection by clicking 'yes' so PuTTY will store the host key in its database. You'll have to do this only once for every new server that you connect to.&lt;br /&gt;
&lt;br /&gt;
After you have accepted the connection, PuTTY opens a console and asks for your loginname. Once you have entered your NetID and your password, you are logged in to the cluster.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_03.png|center]]&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from Windows ==&lt;br /&gt;
&lt;br /&gt;
WinSCP is a freeware ftp client with support for secure file protocols sftp and scp. The login screen depicted below shows the settings for access within the University.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
After clicking on 'Login' a warning about host keys will appear, similar to PuTTY. If it is the first time, you can safely click on 'yes' to accept.&lt;br /&gt;
&lt;br /&gt;
Press 'Login'. If everything is setup correctly you are using the ssh tunnel the PuTTY bastion session has built and you are logged in on the cluster.&lt;br /&gt;
&lt;br /&gt;
When logged in, you will see the commander-like interface below.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_02.png|center|530px]]&lt;br /&gt;
&lt;br /&gt;
The left side displays the files in your home directory on the windows system. The right side displays the files on the cluster. You can drag and drop files between the two windows or select files with the mouse and press 'F5' for transfer to the other window.&lt;br /&gt;
&lt;br /&gt;
== How to log in from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
Mac OS X is in fact a full blown UNIX system under the hood. A terminal program comes standard with OS X, navigate with the finder to Applications -&amp;gt; Utilities to find it, the program is called 'Terminal'. There are many different flavours of Linux, each with their own desktop environments and customizations. Nearly every Linux system comes with a terminal program of some sort. Find the terminal program on your system and run it, once you get a command prompt type:&lt;br /&gt;
&lt;br /&gt;
 ssh &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
Where &amp;lt;netid&amp;gt; is of course your own netid.&lt;br /&gt;
&lt;br /&gt;
Upon first access, ssh requires you to check if the host key is correct. Since this is the first time you connect to the system, it's fine to accept the host key by pressing 'y'.&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh:&lt;br /&gt;
&lt;br /&gt;
 scp &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your remote home directory (folder) to the local home directory on your workstation or laptop. &lt;br /&gt;
&lt;br /&gt;
If you want to copy an entire directory with all its content, use the '-r' option, like this:&lt;br /&gt;
&lt;br /&gt;
 scp -r &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
This will copy the directory 'projects' from your remote home directory to the local home directory on your workstation or laptop.&lt;br /&gt;
&lt;br /&gt;
Of course, it is also possible to copy files and directories from your workstation or laptop to the remote machine:&lt;br /&gt;
&lt;br /&gt;
 scp -r projects.txt &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
 scp -r projects/ &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
And this will copy the directory 'projects' and all its content from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
&lt;br /&gt;
=== Windows ===&lt;br /&gt;
&lt;br /&gt;
There are several options for displaying graphical applications on a Windows system. A basic option is [http://www.straightrunning.com/XmingNotes/ Xming]. You can download it from: http://www.straightrunning.com/XmingNotes/ The site asks for a donation to use the latest version, but there is also a public domain version which can be used for free. Once you have installed it, you can find it in the usual place in the start menu. Before you log in to an hpc cluster and run graphical applications, you need to start Xming first. It will not show much, only a small X in the system tray once it's running. You also need to enable X forwarding in PuTTY. For that you need to click on the + next to SSH in the session panel (left part of the window), then click on X11. You'll see a line on the right side that says &amp;quot;Enable X11 forwarding&amp;quot;, click once on the white square next to that line and it will show a check mark. &lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_04.png|center]]&lt;br /&gt;
&lt;br /&gt;
Now you can continue logging in to the hpc cluster. To check if X forwarding works, type &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that will follow the mouse pointer.&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
[[File:Xeyes.png|center]]&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
An alternative option for displaying graphical applications is Exceed, this is a commercial package for which the TU Delft has a license. It is not available by default on the standard Windows systems, but it can be installed upon request if you contact the servicedesk.&lt;br /&gt;
&lt;br /&gt;
A second alternative is [https://mobaxterm.mobatek.net/ MobaXterm], this is an all-in-one solution containing an ssh client, X server and more. You can download a free version from: https://mobaxterm.mobatek.net/download.html The free version has some limitations, but it's still perfectly usable in our environment.&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
== How to access your Windows network shares ==&lt;br /&gt;
&lt;br /&gt;
It is possible to access the group and bulk shares of your research group. It is required that you have logged in using your password. If you logged in with an ssh public/private keypair instead of your password, you need to authenticate first with:&lt;br /&gt;
&lt;br /&gt;
 kinit&lt;br /&gt;
&lt;br /&gt;
This will ask you to type your password. This authentication is valid for about eight hours, after that you have to authenticate again. &lt;br /&gt;
&lt;br /&gt;
Now just type:&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
&lt;br /&gt;
or&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
&lt;br /&gt;
You can access your Windows home directory as well, the path is:&lt;br /&gt;
&lt;br /&gt;
 /tudelft/$USER/staff-homes/&lt;br /&gt;
&lt;br /&gt;
followed by the first letter of your surname and then your netid. For example, if your name would be John Smith and your netid jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
If you are a student, replace 'staff' with 'student' to access your files.&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; part in the commands above is a shell variable that contains your netid. You can also substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; for your netid yourself. If your netid would be &amp;lt;code&amp;gt;jsmith&amp;lt;/code&amp;gt;, the command&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
would work just as well.&lt;br /&gt;
&lt;br /&gt;
Please do not substitute &amp;lt;code&amp;gt;$USER&amp;lt;/code&amp;gt; with anything else than your own netid! If you and a coworker need to access te same data on a group or bulk volume at the same time, you both use different paths. So &lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/jsmith/staff-group/&lt;br /&gt;
&lt;br /&gt;
only works if your netid is jsmith and&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/pbrown/staff-group/&lt;br /&gt;
&lt;br /&gt;
only works works if your netid is pbrown. If both netids have the same privileges for accessing the group volumes, then commands give access to the same data.&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Introduction&amp;diff=158</id>
		<title>Introduction</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Introduction&amp;diff=158"/>
		<updated>2019-12-31T14:24:29Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: &lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== About the TU Delft hpc clusters ==&lt;br /&gt;
&lt;br /&gt;
Several research groups at the TU Delft have exclusive access to hpc (high performance computing) clusters that are managed and maintained by the ICT Department. Currently there are ten, they are typically named hpcXX.tudelft.net where the XX part is a number ranging from 03 to 27.&lt;br /&gt;
&lt;br /&gt;
{| class=&amp;quot;wikitable&amp;quot;&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Name&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Faculty&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Dept./group(s)&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| OS&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Nodes&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Total # CPU's&lt;br /&gt;
! style=&amp;quot;text-align:left;&amp;quot;| Contact&lt;br /&gt;
|-&lt;br /&gt;
| hpc03.tudelft.net || CiTG || GRS-PSG/MGP || CentOS 7 || 17 || 264 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc05.tudelft.net || TNW || QN/BN || CentOS 7 || 44 || 896 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc06.tudelft.net || 3ME || PME/MSE/DCSC/MTT || CentOS 7 || 118 || 2084 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc07.tudelft.net || TBM || ESS-EI || CentOS 7 || 4 || 64 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc08.tudelft.net || CiTG || HE-EFM || CentOS 7 || 12 || 240 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc09.tudelft.net || CiTG || ES-PAVE || CentOS 7 || 12 || 192 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc10.tudelft.net || CiTG || Dream Teams || CentOS 7 || 8 || 112 || [[file:halbeheer.png|link=]]&lt;br /&gt;
|-&lt;br /&gt;
| hpc11.tudelft.net || TNW || RID/CHEME || CentOS 7 || 45 || 1744 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc12.tudelft.net || L&amp;amp;R || ASM/AWEP || CentOS 7 || 68 || 1536 ||&lt;br /&gt;
|-&lt;br /&gt;
| hpc27.tudelft.net || CiTG || 3Md-AM || CentOS 7 || 20 || 400 ||&lt;br /&gt;
|-			&lt;br /&gt;
|}&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are [https://en.wikipedia.org/wiki/Beowulf_cluster Beowulf] style clusters. The purpose of this wiki is to guide new users into using these clusters. Beowulf clusters are typically made of a bunch common and identical computers or servers linked together in a network. This can be used to perform parallel computing tasks on a much larger scale than would be possible on a single workstation or a heavy duty server.&lt;br /&gt;
&lt;br /&gt;
== About Linux ==&lt;br /&gt;
&lt;br /&gt;
The operating system on all our clusters is [http://www.centos.org/ CentOS 7], this is a well known server class Linux distribution. In order to use our hpc clusters you should at least have some basic knowledge of Linux on the command line. You will not find a Linux tutorial in this wiki, there are many tutorials on the internet that will teach you how to use Linux. Just use the phrase &amp;quot;Linux for beginners&amp;quot; in your favorite search engine and you'll find plenty of sources online, for example the [http://www.ee.surrey.ac.uk/Teaching/Unix/ UNIX Tutorial for Beginners] from the [http://www.surrey.ac.uk/ University of Surrey]. You can als opt to buy a book, [http://www.dummies.com/computers/operating-systems/linux/linux-for-dummies-9th-edition/ Linux for dummies] would be a good start, but there are many others that'll do just fine. If you have never used Linux on the command line before, please take the time to learn the basics, it will be well worth the effort.&lt;br /&gt;
&lt;br /&gt;
== More details ==&lt;br /&gt;
&lt;br /&gt;
Typically a Beowulf style clusters consists of a master node and several worker nodes. The master node is the machine where you log in to and where you prepare and manage your parallel jobs. A scheduler (Maui) and a resource manager (Torque) are both running on the master node. They both work together and provide a mechanism to submit jobs that will be run on the worker nodes where the actual parallel processing takes place. These nodes are stand alone computers connected through a local network to the master node and patiently waiting until the scheduler and the resource manager on the master node tells them to run a (part of a) parallel job.&lt;br /&gt;
&lt;br /&gt;
== About this wiki: contact and info ==&lt;br /&gt;
&lt;br /&gt;
TThe hpc clusters are managed and maintained by the Linux systems department at the Shared Service Center ICT. You can contact us for any questions and comments about the hpc clusters at this address: &lt;br /&gt;
&lt;br /&gt;
[[file:beheer-o.png]] &lt;br /&gt;
&lt;br /&gt;
You can also contact us if you find incorrect information on this wiki, if you are missing important information that you want added or if you have something to contribute to this wiki.&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=Main_Page&amp;diff=157</id>
		<title>Main Page</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=Main_Page&amp;diff=157"/>
		<updated>2019-12-31T14:24:06Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: &lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;&amp;lt;strong&amp;gt;Welcome to the hpcwiki!&amp;lt;/strong&amp;gt;&lt;br /&gt;
&lt;br /&gt;
This wiki is intended as a guide for the hpc clusters managed bij the ICT Department. If you have never used one of our clusters before, this is the place to start. Please note that this wiki is a work in progress, it may not yet be complete.&lt;br /&gt;
&lt;br /&gt;
[[Introduction]]&lt;br /&gt;
&lt;br /&gt;
[[How to log in|Access]] (How to log in and how to copy files)&lt;br /&gt;
&lt;br /&gt;
[[The queue system]]&lt;br /&gt;
&lt;br /&gt;
[[More about queues and nodes]]&lt;br /&gt;
&lt;br /&gt;
[[How to run a job]] (Examples)&lt;br /&gt;
&lt;br /&gt;
[[Applications]]&lt;br /&gt;
&lt;br /&gt;
[[Further reading]] (Where to find more information)&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=More_about_queues_and_nodes&amp;diff=156</id>
		<title>More about queues and nodes</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=More_about_queues_and_nodes&amp;diff=156"/>
		<updated>2019-07-16T19:01:30Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* Avoid over- and underutilization */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== The different queues ==&lt;br /&gt;
&lt;br /&gt;
The larger hpc clusters, most notably hpc03, hpc06, hpc11 and hpc12, are shared by two or more research groups. On those clusters every group has their own queue, sometimes even more than one. These queues give exclusive and full access to a specific set of nodes. &lt;br /&gt;
&lt;br /&gt;
There is also a guest queue on every hpc cluster that gives access to all nodes, but with some restrictions, you will not be able to run non-rerunable and interactive jobs. &lt;br /&gt;
&lt;br /&gt;
In most cases, access to one of the queues is based on group membership in the Active Directory. If your netid is not a member of the right group, you default to the guest queue if you submit a job. If you have access to the group and bulk network shares of your research group, you should also have access to the normal queue on the hpc cluster. If not, contact the secretary in your research group and let him/her arrange the group membership of your netid.&lt;br /&gt;
&lt;br /&gt;
You can check your default queue by submitting a small test job and then have a look at the list with jobs with the qstat command.&lt;br /&gt;
&lt;br /&gt;
 [jsmith@hpc10 ~]$ echo &amp;quot;sleep 60&amp;quot; | qsub &lt;br /&gt;
 [jsmith@hpc10 ~]$ qstat -u jsmith&lt;br /&gt;
&lt;br /&gt;
If you see anything other than guest in the third column, then you are all set.&lt;br /&gt;
&lt;br /&gt;
There are two ways to select the guest queue; &lt;br /&gt;
&lt;br /&gt;
With the -q switch on the commandline:&lt;br /&gt;
&lt;br /&gt;
 qsub -q guest job1&lt;br /&gt;
&lt;br /&gt;
Or with a directive at the start of your job script:&lt;br /&gt;
&lt;br /&gt;
 #PBS -q guest&lt;br /&gt;
&lt;br /&gt;
It is important to know that a job in the guest queue can be interrupted and resumed at any time. You should make sure that the application in your job saves the intermediate results at regular intervals and that it knows how to continue when your job is resumed. If you neglect this, your job in the guest queue will start all over again every time it is interrupted and resumed.&lt;br /&gt;
&lt;br /&gt;
== The different nodes ==&lt;br /&gt;
&lt;br /&gt;
On most hpc clusters you'll find that worker nodes are not all identical, different series of nodes exist which were purchased at different times and with different specifications. To distinguish between the different series of nodes, they are labelled with properties like typea, typeb, typec, etc. On some hpc clusters, nodes have extra properties showing to which queue they belong or showing additional features, like an infiniband network or extra memory compared to similar nodes.&lt;br /&gt;
&lt;br /&gt;
A useful command that shows all nodes and how they are utilized is &amp;lt;code&amp;gt;LOCALnodeload.pl&amp;lt;/code&amp;gt;. A typical output looks like this:&lt;br /&gt;
&lt;br /&gt;
&amp;lt;pre&amp;gt;&lt;br /&gt;
[jsmith@hpc10 ~]$ LOCALnodeload.pl&lt;br /&gt;
Node       Np State/jobs Load  Properties&lt;br /&gt;
---------- -- ---------- ----- ----------&lt;br /&gt;
n10-01     12 12         12.01 typea     &lt;br /&gt;
n10-02     12 free        0.00 typea     &lt;br /&gt;
n10-03     12 free        0.00 typea     &lt;br /&gt;
n10-04     12 free        0.00 typea     &lt;br /&gt;
n10-05     16 12         11.93 typeb     &lt;br /&gt;
n10-06     16 free        0.00 typeb     &lt;br /&gt;
n10-07     16 offline     0.00 typeb     &lt;br /&gt;
n10-08     16 down        0.00 typeb     &lt;br /&gt;
&amp;lt;/pre&amp;gt;&lt;br /&gt;
&lt;br /&gt;
The first column (Node) shows the names of the nodes. The second column (Np) shows the total number of processors. The third column (State/jobs) shows the number of processors currenly in use or the status of the node (free, offline or down). The forth colum (Load) shows the actual load on the nodes. In an ideal situation the load matches the number of processors in use. The last column (Properties) shows the properties as described above. As you can see in the example, typea nodes have 12 processors and typeb nodes have 16. Node n10-01 is fully occupied, node n10-05 is running one or more jobs but still has 4 processors free. Nodes n10-07 and n10-08 cannot be used.&lt;br /&gt;
&lt;br /&gt;
== Selecting nodes ==&lt;br /&gt;
&lt;br /&gt;
If you submit a job, the scheduler automatically selects a node to run it. By default a jobs gets one node and one processor. You can manually select the number of processors and nodes for your job by using the &amp;lt;code&amp;gt;-l&amp;lt;/code&amp;gt; switch with the &amp;lt;code&amp;gt;qsub&amp;lt;/code&amp;gt; command. You can also select nodes by property. the &amp;lt;code&amp;gt;-l&amp;lt;/code&amp;gt; switch works like this:&lt;br /&gt;
&lt;br /&gt;
 qsub -l nodes=&amp;lt;x&amp;gt;:ppn=&amp;lt;c&amp;gt;:&amp;lt;property&amp;gt;:&amp;lt;property&amp;gt;...&lt;br /&gt;
&lt;br /&gt;
* &amp;lt;x&amp;gt; is either an amount of nodes or the name(s) of the selected node(s)&lt;br /&gt;
* &amp;lt;c&amp;gt; is number of processors per node&lt;br /&gt;
* &amp;lt;property&amp;gt; is any of the properties you see in Properties column of the LOCALnodeload.pl command.&lt;br /&gt;
&lt;br /&gt;
Examples:&lt;br /&gt;
&lt;br /&gt;
{|&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;qsub -l nodes=4&amp;lt;/code&amp;gt; || Request 4 nodes of any type&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;qsub -l nodes=n10-07+n10-08&amp;lt;/code&amp;gt; || Request 2 specific nodes by hostname&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;qsub -l nodes=4:ppn=2&amp;lt;/code&amp;gt; || Request 2 processors on each of four nodes&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;qsub -l nodes=1:ppn=4&amp;lt;/code&amp;gt; || Request 4 processors on one node&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;qsub -l nodes=2:typea&amp;lt;/code&amp;gt; || Request 2 nodes with the &amp;lt;code&amp;gt;typea&amp;lt;/code&amp;gt; property&lt;br /&gt;
|-&lt;br /&gt;
|}&lt;br /&gt;
Instead of using the -l or the -q switches on the commandline when you submit your job with qsub, you can also add them as a directive to your job script. For instance, if you add&lt;br /&gt;
&lt;br /&gt;
 #PBS -l nodes=1:ppn=4&lt;br /&gt;
 #PBS -q guest&lt;br /&gt;
&lt;br /&gt;
at the start of your script, you can just use&lt;br /&gt;
&lt;br /&gt;
 qsub job.sh&lt;br /&gt;
&lt;br /&gt;
instead of&lt;br /&gt;
&lt;br /&gt;
 qsub -l nodes=1:ppn=4 -q guest job.sh&lt;br /&gt;
&lt;br /&gt;
== Avoid over- and underutilization ==&lt;br /&gt;
&lt;br /&gt;
An important thing to consider when you create your own job script is matching the number of processors that you request with the number of processors that the software in your script will actually use. It is possible that you request only one processor and that your program will use all processors available on the nodes. This is called overutilization and is not very efficient when other jobs are already running on the same node and using the same processors.&lt;br /&gt;
&lt;br /&gt;
It is also possible that you request several (or all) processors and that your program will only use one. This will leave the other processors you claimed unused (underutilization), which is also not very efficient because the unused processors you requested will not be used for other jobs.&lt;br /&gt;
&lt;br /&gt;
How to avoid over- and underutilization? Many programs have options that will let them use only one thread (utilization of only one processor) or a specific number of threads. &lt;br /&gt;
&lt;br /&gt;
For example, Ansys has the &amp;lt;code&amp;gt;-np&amp;lt;/code&amp;gt; switch:&lt;br /&gt;
&lt;br /&gt;
 ansys -np N&lt;br /&gt;
&lt;br /&gt;
and Fluent has the &amp;lt;code&amp;gt;-t&amp;lt;/code&amp;gt; switch &lt;br /&gt;
&lt;br /&gt;
 fluent -tN&lt;br /&gt;
&lt;br /&gt;
where N matches the number of processors that you request in your job. &lt;br /&gt;
&lt;br /&gt;
If your program does not have an option to limit the number of processor, you can try to add this line in your job script, just before the line where your progam starts:&lt;br /&gt;
&lt;br /&gt;
 export OMP_NUM_THREADS=N&lt;br /&gt;
&lt;br /&gt;
Of course, N must match the number of processors that you request in your job. Alternatively, you could also request an entire node (all processors) in your job and let your program use all available resources of that node.&lt;br /&gt;
&lt;br /&gt;
== Avoid excessive reads and writes on your homedir ==&lt;br /&gt;
&lt;br /&gt;
Some programs read and write a lot of data to and from your home directory. This is not very efficient, on the nodes your home directory is a network share, so access is relatively slow and it keeps the master node unnecessarily busy. If you expect that your job will do a lot of reading and writing to disk, you can use the local disk on the node instead, which is mounted on /var/tmp on all nodes. You can do this by adding a few extra lines to your job script, right before the line that starts the program in your job, for example: &lt;br /&gt;
&lt;br /&gt;
&amp;lt;pre&amp;gt;&lt;br /&gt;
TMP=/var/tmp/${PBS_JOBID}&lt;br /&gt;
mkdir -p ${TMP}&lt;br /&gt;
/usr/bin/rsync -vax &amp;quot;${PBS_O_WORKDIR}/&amp;quot; ${TMP}/&lt;br /&gt;
cd ${TMP}&lt;br /&gt;
&amp;lt;/pre&amp;gt;&lt;br /&gt;
&lt;br /&gt;
Once your program is done you can copy the results back to your home directory and clean up by adding these two lines at the end of your job script:&lt;br /&gt;
&lt;br /&gt;
&amp;lt;pre&amp;gt;&lt;br /&gt;
/usr/bin/rsync -vax ${TMP}/ &amp;quot;${PBS_O_WORKDIR}/&amp;quot;&lt;br /&gt;
[ $? -eq 0 ] &amp;amp;&amp;amp; /bin/rm -rf ${TMP}&lt;br /&gt;
&amp;lt;/pre&amp;gt;&lt;br /&gt;
&lt;br /&gt;
This usually works best if you create a seperate directory in your homedir, move the necessary files and the job script to it and run your job from there. Otherwise you would end up copying your entire home directory to the node for no good reason.&lt;br /&gt;
&lt;br /&gt;
== Access to nodes ==&lt;br /&gt;
&lt;br /&gt;
All nodes are independant Linux machines and you could be tempted to log in to one of the nodes and work from there. This is however forbidden, any attempt to log in to a node will fail. There is one exception, you can log in to a node if you have a job running on it, this way you can check on the progress of your job and see if things are still working as intended. To check which node runs your job, type:&lt;br /&gt;
&lt;br /&gt;
 qstat -u $USER -n1&lt;br /&gt;
&lt;br /&gt;
This will get you a list of all your jobs, in the last column you'll see the nodes in use. If you log in to a node, please do not run any additional CPU intensive programs to avoid overutilization.&lt;br /&gt;
&lt;br /&gt;
If you must log in to a node in order to run software that can not be run from a script, you can start an interactive job. This is done using the &amp;lt;code&amp;gt;-I&amp;lt;/code&amp;gt; switch with qsub, like this:&lt;br /&gt;
&lt;br /&gt;
 qsub -I&lt;br /&gt;
&lt;br /&gt;
As soon as a node is assigned to you (this may take a while), you'll get a new command line prompt, as if you just logged in with ssh. This will reserve only one processor, you should take care that if you start a CPU intensive program, it does not use more than one processor. If you need more processors or if you want to use a specific node, you can request this for your interactive job with the &amp;lt;code&amp;gt;-l&amp;lt;/code&amp;gt; switch, for example, if you want to request 8 processors on node n10-08:&lt;br /&gt;
&lt;br /&gt;
 qsub -I -l nodes=n10-08:ppn=8&lt;br /&gt;
&lt;br /&gt;
If you want to run a progam with a graphical interface on a node, you'll need to make sure that X forwarding works when logged in to the master node. Then you can use the &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; switch start your interactive job with X forwarding enabled:&lt;br /&gt;
&lt;br /&gt;
 qsub -I -X&lt;br /&gt;
&lt;br /&gt;
It is important to know that an interactive can only be run in the normal queues, '''you can not run an interactive job in the guest queue!'''&lt;br /&gt;
&lt;br /&gt;
== A generic example of a job script ==&lt;br /&gt;
&lt;br /&gt;
The script below can be used as a starting point to create your own jobs. Feel free to copy, paste and modify it to your needs. Lines starting with # will not be executed and contain useful information.&lt;br /&gt;
&lt;br /&gt;
&amp;lt;pre&amp;gt;&lt;br /&gt;
#!/bin/bash&lt;br /&gt;
#&lt;br /&gt;
# Torque directives (#PBS) must always be at the start of a job script!&lt;br /&gt;
#&lt;br /&gt;
# Request nodes and processors per node&lt;br /&gt;
#&lt;br /&gt;
#PBS -l nodes=1:ppn=1&lt;br /&gt;
#&lt;br /&gt;
#&lt;br /&gt;
# Set the name of the job&lt;br /&gt;
#&lt;br /&gt;
#PBS -N name_of_job&lt;br /&gt;
#&lt;br /&gt;
#&lt;br /&gt;
# Set the mail options (type 'man qsub' for more information)&lt;br /&gt;
#&lt;br /&gt;
#PBS -m bea&lt;br /&gt;
#&lt;br /&gt;
#&lt;br /&gt;
# Set the email address where you want notifications sent to&lt;br /&gt;
# By default mail will be sent to your TU Delft mailbox&lt;br /&gt;
#&lt;br /&gt;
#PBS -M $USER@mailboxcluster.tudelft.net&lt;br /&gt;
#&lt;br /&gt;
#&lt;br /&gt;
# Set the rerunable flag, 'n' is not rerunable, default is 'y'&lt;br /&gt;
#&lt;br /&gt;
#PBS -r y&lt;br /&gt;
&lt;br /&gt;
# Make sure I'm the only one that can read my output&lt;br /&gt;
umask 0077&lt;br /&gt;
&lt;br /&gt;
# create a temporary directory in /var/tmp&lt;br /&gt;
TMP=/var/tmp/${PBS_JOBID}&lt;br /&gt;
mkdir -p ${TMP}&lt;br /&gt;
echo &amp;quot;Temporary work dir: ${TMP}&amp;quot;&lt;br /&gt;
if [ ! -d &amp;quot;${TMP}&amp;quot; ]; then&lt;br /&gt;
    echo &amp;quot;Cannot create temporary directory. Disk probably full.&amp;quot;&lt;br /&gt;
    exit 1&lt;br /&gt;
fi&lt;br /&gt;
&lt;br /&gt;
# copy the input files to ${TMP}&lt;br /&gt;
echo &amp;quot;Copying from ${PBS_O_WORKDIR}/ to ${TMP}/&amp;quot;&lt;br /&gt;
/usr/bin/rsync -vax &amp;quot;${PBS_O_WORKDIR}/&amp;quot; ${TMP}/&lt;br /&gt;
&lt;br /&gt;
cd ${TMP}&lt;br /&gt;
&lt;br /&gt;
# &lt;br /&gt;
&lt;br /&gt;
module load application1&lt;br /&gt;
module load application2&lt;br /&gt;
&lt;br /&gt;
export OMP_NUM_THREADS=1&lt;br /&gt;
&lt;br /&gt;
# Here is where the application is started on the node&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
# job done, copy everything back&lt;br /&gt;
echo &amp;quot;Copying from ${TMP}/ to ${PBS_O_WORKDIR}/&amp;quot;&lt;br /&gt;
/usr/bin/rsync -vax ${TMP}/ &amp;quot;${PBS_O_WORKDIR}/&amp;quot;&lt;br /&gt;
&lt;br /&gt;
# delete my temporary files&lt;br /&gt;
[ $? -eq 0 ] &amp;amp;&amp;amp; /bin/rm -rf ${TMP}&lt;br /&gt;
&amp;lt;/pre&amp;gt;&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=155</id>
		<title>How to log in</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=155"/>
		<updated>2019-01-16T16:07:01Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* Basics */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== Basics ==&lt;br /&gt;
&lt;br /&gt;
In order to log in you need a valid [https://www.tudelft.nl/en/student/ict/ict-facilities/netid-account/ NetID] that is enabled for access to one of the clusters. To have your NetID enabled, ask the contact person for the cluster you want access to (See the [[Introduction]] page) or ask the secretary of the department or research group where you work, he or she can tell you who the contact person is for an hpc cluster.&lt;br /&gt;
&lt;br /&gt;
If you want to collaborate with someone from outside the TU Delft and you want that person to access the same cluster as you, then you can request a guest account at the servicedesk. Once the guest account is created, it can be enabled for access to one of the clusters. '''Do not share your own NetID with others! Your NetID is for personal use only, sharing it with other people is forbidden!'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are only accessible via ssh (secure shell). To log in you need an ssh client. If you use Windows, you can download and use [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY], Macintosh and Linux computers already have an ssh client built in. There is no need to log in to a bastion host first, you can log in from anywhere in the world.&lt;br /&gt;
&lt;br /&gt;
Every time you log in you'll see an motd (message of the day). Please pay attention to this, it may contain information about current events on the cluster. It also shows you an email address that you can use if there are problems or if you have questions. You can mail at any time you like, but you'll usually only get a reply during office hours, there is no 24/7 support on the hpc clusters.&lt;br /&gt;
&lt;br /&gt;
'''Please note:''' You have a local home directory on the cluster. No backups will be made of the home directories, if you make a mistake and loose a file, it is gone forever. '''Making backups of your important files on the hpc clusters is your own responsibility!'''&lt;br /&gt;
&lt;br /&gt;
== How to log in from Windows ==&lt;br /&gt;
&lt;br /&gt;
An often used ssh client for Windows is [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY]. If you use a Windows PC or a laptop that is provided and maintained by the TU Delft, it is already there, if you use your own Windows computer, you'll have to download and install it first from [http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html www.chiark.greenend.org.uk].&lt;br /&gt;
&lt;br /&gt;
Once you have started PuTTY you'll get a configuration window als shown.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can type the name of the cluster, for example 'hpc10.tudelft.net', in the field 'Host Name'. You can save the settings for a cluster by choosing a name for the session, enter it in the 'Saved Sessions' box and click 'Save'. The next time you open PuTTY the settings can be restored by clicking on the name and a click on 'Load'.&lt;br /&gt;
&lt;br /&gt;
By clicking 'Open' a connection is made to hpc10.tudelft.net. If this is the first time you are connecting to the cluster, PuTTY will warn you that you are connecting to a previously unknown host.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_02.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can confirm this connection by clicking 'yes' so PuTTY will store the host key in its database. You'll have to do this only once for every new server that you connect to.&lt;br /&gt;
&lt;br /&gt;
After you have accepted the connection, PuTTY opens a console and asks for your loginname. Once you have entered your NetID and your password, you are logged in to the cluster.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_03.png|center]]&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from Windows ==&lt;br /&gt;
&lt;br /&gt;
WinSCP is a freeware ftp client with support for secure file protocols sftp and scp. The login screen depicted below shows the settings for access within the University.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
After clicking on 'Login' a warning about host keys will appear, similar to PuTTY. If it is the first time, you can safely click on 'yes' to accept.&lt;br /&gt;
&lt;br /&gt;
Press 'Login'. If everything is setup correctly you are using the ssh tunnel the PuTTY bastion session has built and you are logged in on the cluster.&lt;br /&gt;
&lt;br /&gt;
When logged in, you will see the commander-like interface below.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_02.png|center|530px]]&lt;br /&gt;
&lt;br /&gt;
The left side displays the files in your home directory on the windows system. The right side displays the files on the cluster. You can drag and drop files between the two windows or select files with the mouse and press 'F5' for transfer to the other window.&lt;br /&gt;
&lt;br /&gt;
== How to log in from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
Mac OS X is in fact a full blown UNIX system under the hood. A terminal program comes standard with OS X, navigate with the finder to Applications -&amp;gt; Utilities to find it, the program is called 'Terminal'. There are many different flavours of Linux, each with their own desktop environments and customizations. Nearly every Linux system comes with a terminal program of some sort. Find the terminal program on your system and run it, once you get a command prompt type:&lt;br /&gt;
&lt;br /&gt;
 ssh &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
Where &amp;lt;netid&amp;gt; is of course your own netid.&lt;br /&gt;
&lt;br /&gt;
Upon first access, ssh requires you to check if the host key is correct. Since this is the first time you connect to the system, it's fine to accept the host key by pressing 'y'.&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh:&lt;br /&gt;
&lt;br /&gt;
 scp &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your remote home directory (folder) to the local home directory on your workstation or laptop. &lt;br /&gt;
&lt;br /&gt;
If you want to copy an entire directory with all its content, use the '-r' option, like this:&lt;br /&gt;
&lt;br /&gt;
 scp -r &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
This will copy the directory 'projects' from your remote home directory to the local home directory on your workstation or laptop.&lt;br /&gt;
&lt;br /&gt;
Of course, it is also possible to copy files and directories from your workstation or laptop to the remote machine:&lt;br /&gt;
&lt;br /&gt;
 scp -r projects.txt &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
 scp -r projects/ &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
And this will copy the directory 'projects' and all its content from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
&lt;br /&gt;
=== Windows ===&lt;br /&gt;
&lt;br /&gt;
There are several options for displaying graphical applications on a Windows system. A basic option is [http://www.straightrunning.com/XmingNotes/ Xming]. You can download it from: http://www.straightrunning.com/XmingNotes/ The site asks for a donation to use the latest version, but there is also a public domain version which can be used for free. Once you have installed it, you can find it in the usual place in the start menu. Before you log in to an hpc cluster and run graphical applications, you need to start Xming first. It will not show much, only a small X in the system tray once it's running. You also need to enable X forwarding in PuTTY. For that you need to click on the + next to SSH in the session panel (left part of the window), then click on X11. You'll see a line on the right side that says &amp;quot;Enable X11 forwarding&amp;quot;, click once on the white square next to that line and it will show a check mark. &lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_04.png|center]]&lt;br /&gt;
&lt;br /&gt;
Now you can continue logging in to the hpc cluster. To check if X forwarding works, type &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that will follow the mouse pointer.&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
[[File:Xeyes.png|center]]&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
An alternative option for displaying graphical applications is Exceed, this is a commercial package for which the TU Delft has a license. It is not available by default on the standard Windows systems, but it can be installed upon request if you contact the servicedesk.&lt;br /&gt;
&lt;br /&gt;
A second alternative is [https://mobaxterm.mobatek.net/ MobaXterm], this is an all-in-one solution containing an ssh client, X server and more. You can download a free version from: https://mobaxterm.mobatek.net/download.html The free version has some limitations, but it's still perfectly usable in our environment.&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
== How to access your Windows network shares ==&lt;br /&gt;
&lt;br /&gt;
It is possible to access the group and bulk shares of your research group, just type: &lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
&lt;br /&gt;
or &lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
&lt;br /&gt;
You can access your Windows home directory as well, the path is:&lt;br /&gt;
&lt;br /&gt;
 /tudelft/$USER/staff-homes/&lt;br /&gt;
&lt;br /&gt;
followed by the first letter of your surname and then your netid. For example, if your name would be John Smith and your netid jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
If you are a student, replace 'staff' with 'student' to access your files.&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
	<entry>
		<id>https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=154</id>
		<title>How to log in</title>
		<link rel="alternate" type="text/html" href="https://hpcwiki.tudelft.nl/index.php?title=How_to_log_in&amp;diff=154"/>
		<updated>2019-01-16T16:02:29Z</updated>

		<summary type="html">&lt;p&gt;Paul Keekstra: /* Basics */&lt;/p&gt;
&lt;hr /&gt;
&lt;div&gt;== Basics ==&lt;br /&gt;
&lt;br /&gt;
In order to log in you need a valid [https://www.tudelft.nl/en/student/ict/ict-facilities/netid-account/ NetID] that is enabled for access to one of the clusters. To have your NetID enabled, ask the contact person for the cluster you want access to (See the [[Introduction]] page) or contact the secretary of the department or research group where you work, he or she can tell you who the contact person is for an hpc cluster.&lt;br /&gt;
&lt;br /&gt;
If you want to collaborate with someone from outside the TU Delft and you want that person to access the same cluster as you, then you can request a guest account at the servicedesk. Once the guest account is created, it can be enabled for access to one of the clusters. '''Do not share your own NetID with others! Your NetID is for personal use only, sharing it with other people is forbidden!'''&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
The hpc clusters are only accessible via ssh (secure shell). To log in you need an ssh client. If you use Windows, you can download and use [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY], Macintosh and Linux computers already have an ssh client built in. There is no need to log in to a bastion host first, you can log in from anywhere in the world.&lt;br /&gt;
&lt;br /&gt;
Every time you log in you'll see an motd (message of the day). Please pay attention to this, it may contain information about current events on the cluster. It also shows you an email address that you can use if there are problems or if you have questions. You can mail at any time you like, but you'll usually only get a reply during office hours, there is no 24/7 support on the hpc clusters.&lt;br /&gt;
&lt;br /&gt;
'''Please note:''' You have a local home directory on the cluster. No backups will be made of the home directories, if you make a mistake and loose a file, it is gone forever. '''Making backups of your important files on the hpc clusters is your own responsibility!'''&lt;br /&gt;
&lt;br /&gt;
== How to log in from Windows ==&lt;br /&gt;
&lt;br /&gt;
An often used ssh client for Windows is [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY]. If you use a Windows PC or a laptop that is provided and maintained by the TU Delft, it is already there, if you use your own Windows computer, you'll have to download and install it first from [http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html www.chiark.greenend.org.uk].&lt;br /&gt;
&lt;br /&gt;
Once you have started PuTTY you'll get a configuration window als shown.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can type the name of the cluster, for example 'hpc10.tudelft.net', in the field 'Host Name'. You can save the settings for a cluster by choosing a name for the session, enter it in the 'Saved Sessions' box and click 'Save'. The next time you open PuTTY the settings can be restored by clicking on the name and a click on 'Load'.&lt;br /&gt;
&lt;br /&gt;
By clicking 'Open' a connection is made to hpc10.tudelft.net. If this is the first time you are connecting to the cluster, PuTTY will warn you that you are connecting to a previously unknown host.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_02.png|center]]&lt;br /&gt;
&lt;br /&gt;
You can confirm this connection by clicking 'yes' so PuTTY will store the host key in its database. You'll have to do this only once for every new server that you connect to.&lt;br /&gt;
&lt;br /&gt;
After you have accepted the connection, PuTTY opens a console and asks for your loginname. Once you have entered your NetID and your password, you are logged in to the cluster.&lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_03.png|center]]&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from Windows ==&lt;br /&gt;
&lt;br /&gt;
WinSCP is a freeware ftp client with support for secure file protocols sftp and scp. The login screen depicted below shows the settings for access within the University.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_01.png|center]]&lt;br /&gt;
&lt;br /&gt;
After clicking on 'Login' a warning about host keys will appear, similar to PuTTY. If it is the first time, you can safely click on 'yes' to accept.&lt;br /&gt;
&lt;br /&gt;
Press 'Login'. If everything is setup correctly you are using the ssh tunnel the PuTTY bastion session has built and you are logged in on the cluster.&lt;br /&gt;
&lt;br /&gt;
When logged in, you will see the commander-like interface below.&lt;br /&gt;
&lt;br /&gt;
[[File:WinSCP_02.png|center|530px]]&lt;br /&gt;
&lt;br /&gt;
The left side displays the files in your home directory on the windows system. The right side displays the files on the cluster. You can drag and drop files between the two windows or select files with the mouse and press 'F5' for transfer to the other window.&lt;br /&gt;
&lt;br /&gt;
== How to log in from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
Mac OS X is in fact a full blown UNIX system under the hood. A terminal program comes standard with OS X, navigate with the finder to Applications -&amp;gt; Utilities to find it, the program is called 'Terminal'. There are many different flavours of Linux, each with their own desktop environments and customizations. Nearly every Linux system comes with a terminal program of some sort. Find the terminal program on your system and run it, once you get a command prompt type:&lt;br /&gt;
&lt;br /&gt;
 ssh &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
Where &amp;lt;netid&amp;gt; is of course your own netid.&lt;br /&gt;
&lt;br /&gt;
Upon first access, ssh requires you to check if the host key is correct. Since this is the first time you connect to the system, it's fine to accept the host key by pressing 'y'.&lt;br /&gt;
&lt;br /&gt;
== How to transfer files to and from OS X or Linux ==&lt;br /&gt;
&lt;br /&gt;
To transfer files you can use scp, which is very similar to ssh:&lt;br /&gt;
&lt;br /&gt;
 scp &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects.txt ~&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your remote home directory (folder) to the local home directory on your workstation or laptop. &lt;br /&gt;
&lt;br /&gt;
If you want to copy an entire directory with all its content, use the '-r' option, like this:&lt;br /&gt;
&lt;br /&gt;
 scp -r &amp;lt;netid&amp;gt;@hpc10.tudelft.net:~/projects/ ~&lt;br /&gt;
&lt;br /&gt;
This will copy the directory 'projects' from your remote home directory to the local home directory on your workstation or laptop.&lt;br /&gt;
&lt;br /&gt;
Of course, it is also possible to copy files and directories from your workstation or laptop to the remote machine:&lt;br /&gt;
&lt;br /&gt;
 scp -r projects.txt &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
This will copy the file 'projects.txt' from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
 scp -r projects/ &amp;lt;netid&amp;gt;@hpc10:~/&lt;br /&gt;
&lt;br /&gt;
And this will copy the directory 'projects' and all its content from your current working directory to the home directory on the remote machine.&lt;br /&gt;
&lt;br /&gt;
== How to display graphical applications ==&lt;br /&gt;
&lt;br /&gt;
=== Windows ===&lt;br /&gt;
&lt;br /&gt;
There are several options for displaying graphical applications on a Windows system. A basic option is [http://www.straightrunning.com/XmingNotes/ Xming]. You can download it from: http://www.straightrunning.com/XmingNotes/ The site asks for a donation to use the latest version, but there is also a public domain version which can be used for free. Once you have installed it, you can find it in the usual place in the start menu. Before you log in to an hpc cluster and run graphical applications, you need to start Xming first. It will not show much, only a small X in the system tray once it's running. You also need to enable X forwarding in PuTTY. For that you need to click on the + next to SSH in the session panel (left part of the window), then click on X11. You'll see a line on the right side that says &amp;quot;Enable X11 forwarding&amp;quot;, click once on the white square next to that line and it will show a check mark. &lt;br /&gt;
&lt;br /&gt;
[[File:PuTTY_04.png|center]]&lt;br /&gt;
&lt;br /&gt;
Now you can continue logging in to the hpc cluster. To check if X forwarding works, type &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that will follow the mouse pointer.&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
[[File:Xeyes.png|center]]&lt;br /&gt;
&lt;br /&gt;
&lt;br /&gt;
An alternative option for displaying graphical applications is Exceed, this is a commercial package for which the TU Delft has a license. It is not available by default on the standard Windows systems, but it can be installed upon request if you contact the servicedesk.&lt;br /&gt;
&lt;br /&gt;
A second alternative is [https://mobaxterm.mobatek.net/ MobaXterm], this is an all-in-one solution containing an ssh client, X server and more. You can download a free version from: https://mobaxterm.mobatek.net/download.html The free version has some limitations, but it's still perfectly usable in our environment.&lt;br /&gt;
&lt;br /&gt;
=== Mac OS X ===&lt;br /&gt;
&lt;br /&gt;
For displaying graphical applications on a Mac OS X system you need a software package called [https://www.xquartz.org/ XQuartz] which can be downloaded from: [https://www.xquartz.org/ https://www.xquartz.org/] This will run on Mac OS X v10.6 and higher. Once it's installed, you end up with an application called XQuartz located in /Applications/Utilties. There is no need to start it first before you log in to an hpc cluster, it will start automatically as soon as it is needed. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
=== Linux ===&lt;br /&gt;
&lt;br /&gt;
Most Linux workstations come with a graphical environment. This works transparantly with the graphical applications on the hpc clusters. The only thing you need to do is add the option &amp;lt;code&amp;gt;-X&amp;lt;/code&amp;gt; to the &amp;lt;code&amp;gt;ssh&amp;lt;/code&amp;gt; command that you use to log in:&lt;br /&gt;
&lt;br /&gt;
 ssh -X &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
You can test if this works by typing &amp;lt;code&amp;gt;xeyes&amp;lt;/code&amp;gt; on the command line, this will show a small window on your screen with two eyes that follow the mouse pointer. If it doesn't work, try the &amp;lt;code&amp;gt;-Y&amp;lt;/code&amp;gt; option:&lt;br /&gt;
&lt;br /&gt;
 ssh -Y &amp;lt;netid&amp;gt;@hpc10.tudelft.net&lt;br /&gt;
&lt;br /&gt;
== How to access your Windows network shares ==&lt;br /&gt;
&lt;br /&gt;
It is possible to access the group and bulk shares of your research group, just type: &lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-group/&lt;br /&gt;
&lt;br /&gt;
or &lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-bulk/&lt;br /&gt;
&lt;br /&gt;
and navigate from there to the directories that you need.&lt;br /&gt;
&lt;br /&gt;
You can access your Windows home directory as well, the path is:&lt;br /&gt;
&lt;br /&gt;
 /tudelft/$USER/staff-homes/&lt;br /&gt;
&lt;br /&gt;
followed by the first letter of your surname and then your netid. For example, if your name would be John Smith and your netid jsmith;&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/staff-homes/s/jsmith&lt;br /&gt;
&lt;br /&gt;
If you are a student, replace 'staff' with 'student' to access your files.&lt;br /&gt;
&lt;br /&gt;
 cd /tudelft/$USER/student-homes/s/jsmith&lt;/div&gt;</summary>
		<author><name>Paul Keekstra</name></author>
	</entry>
</feed>