User Tools

Site Tools


documents:101103imagej_cluster_short

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
documents:101103imagej_cluster_short [2010/11/25 14:02]
kota
documents:101103imagej_cluster_short [2016/06/01 09:57] (current)
kota
Line 1: Line 1:
 ====== Using Cluster (Quick Version) ====== ====== Using Cluster (Quick Version) ======
  
-===== Processing by ImageJ from Command Line =====+**Note: This page is outdated (June 1, 2016)**. I will prepare a newer version. ​
  
-Command syntax ​is+Since most of the scripts for cluster use is located under a specific directory in the ALMF server, I recommend you to add the path to your .tcshrc as below.  
 +<code shell> 
 +set path=($path /​g/​almf/​software/​bin2) 
 +</​code>​ 
 +Otherwise, please add the path above in front of script name. 
  
-**/​g/​almf/​software/​ij/​doIJ.sh <​fullpath to macro file> <​fullpath to image file>**+===== Interactive Usage of Cluster CPU =====
  
 +Before submitting huge data with your script, you might want to test run the script interactively to see if it is indeed working. To do so, you could access cluster in interactive mode. 
 +
 +  - Log in to sub-master
 +  - then "qsub -I"
 +    - wait for some seconds, you will be assigned to a specific node where you could test around. ​
 +    - To enable multi-threaded computation,​ do "qsub -I -l select=ncpus=2:​mem=3gb"​ (in this case, uses 2 CPUs and 3Gb of memory).
 + 
 +
 +===== Processing by ImageJ from Command Line =====
 +
 +**Script name: doIJ**
 +
 +Syntax
 +<​code>​
 +sh doIJ <​fullpath to macro file> <​fullpath to image file>
 +</​code>​
 For example: For example:
-<​code ​bash+<​code>​ 
-sh /​g/​almf/​software/​ij/doIJ.sh /​g/​almf/​software/​ij/​getArgumentMacro.ijm /​g/​almf/​miura/​blobs.tif+sh /​g/​almf/​software/​bin2/doIJ.sh /​g/​almf/​software/​ij/​getArgumentMacro.ijm /​g/​almf/​miura/​blobs.tif
 </​code>​ </​code>​
 +
 +Use this script only for small tasks. Heavy calculation consumes the server resource where others are also using. ​
  
 ===== Processing Single Image file by ImageJ, throw the job to cluster ===== ===== Processing Single Image file by ImageJ, throw the job to cluster =====
  
-Above commands should be pasted ​in another shell script as follows:+**Script name: qsubIJ** 
 + 
 +To process image file /a/b/c.tif with an ImageJ macro file /d/e/f.ijm, like above, but with cluster  
 +<​quote>​ 
 + **sh qsubIJ /d/e/f.ijm /​a/​b/​c.tif** 
 +</​quote>​ 
 + 
 +Job will be submitted to the cluster, and what should be the output from ImageJ will be saved as a text file in your ALMF home directory:​\\  
 +**/​g/​almf/<​Your Unix Username>/​CLlogyyyymmdd_hhmm.txt**. \\ 
 +"​yyyymmdd_hhmm"​ should be the time of execution.  
 + 
 +==== a bit more on customization ==== 
 + 
 +To customize arguments and so on, shell commands should be written ​in another shell script as follows:
 <sxh shell> <sxh shell>
 #!/bin/sh #!/bin/sh
Line 35: Line 70:
 /​usr/​pbs/​bin/​qsub -q clusterng /​g/​almf/​software/​ij/​cl_doIJ.sh /​usr/​pbs/​bin/​qsub -q clusterng /​g/​almf/​software/​ij/​cl_doIJ.sh
 </​code>​ </​code>​
- 
---- 
-**UPGRADE (2010.11.25)**\\ 
-\\ 
-To process image file /a/b/c.tif with an ImageJ macro file /d/e/f.ijm, then  
-<​quote>​ 
- **sh /​g/​almf/​software/​bin2/​qsubIJ /d/e/f.ijm /​a/​b/​c.tif** 
-</​quote>​ 
- 
-Job will be submitter to the cluster, and what should be output from ImageJ will be saved as a text file at:\\  
-**/​g/​almf/<​Your Unix Username>/​CLlogyyyymmdd_hhmm.txt**. \\ 
-"​yyyymmdd_hhmm"​ should be the time of execution. ​ 
- 
- 
  
 ===== Processing All Image Files in a Folder ===== ===== Processing All Image Files in a Folder =====
  
-Syntax is  +**Script name: qsubIJbatch**
- +
-**sh /​g/​almf/​software/​ij/​clust5fullp.sh </​full/​path/​to/​dir>​ <File Name of Macro File>**+
  
 +Syntax is 
 +<​code>​
 +sh qsubIJbatch </​full/​path/​to/​dir>​ <File Name of Macro File>
 +</​code>​
 For example, if you want to process all images under ///​g/​almf/​miura/​testsmalls//​ using an ImageJ macro file //​headlesstest3.ijm//,​ then your command would be: For example, if you want to process all images under ///​g/​almf/​miura/​testsmalls//​ using an ImageJ macro file //​headlesstest3.ijm//,​ then your command would be:
 <code bash> <code bash>
-sh /​g/​almf/​software/​ij/​clust5fullp.sh ​/​g/​almf/​miura/​testsmalls headlesstest3.ijm+sh qsubIJbatch ​/​g/​almf/​miura/​testsmalls headlesstest3.ijm
 </​code>​ </​code>​
  
Line 67: Line 90:
 sh /​g/​almf/​software/​ij/​lifcon.sh /​g/​haering/​Pombescreen/​test.lif sh /​g/​almf/​software/​ij/​lifcon.sh /​g/​haering/​Pombescreen/​test.lif
 </​del>​\\ </​del>​\\
-**/​g/​almf/​software/​bin2/​lifconvert** /​g/​haering/​Pombescreen/​pb2/​100927ExpNo2.lif+**sh /​g/​almf/​software/​bin2/​lifconvert** /​g/​haering/​Pombescreen/​pb2/​100927ExpNo2.lif
  
 actual processing:​\\ actual processing:​\\
-**sh /g/almf/softwae/ij/clust5fullp.sh** /​g/​haering/​Pombescreen/​test.lif_tifStack headlesstest3.ijm+**sh /g/almf/software/bin2/qsubIJbatch** /​g/​haering/​Pombescreen/​test.lif_tifStack headlesstest3.ijm
documents/101103imagej_cluster_short.1290693738.txt.gz · Last modified: 2016/05/24 12:46 (external edit)