Differences

This shows you the differences between two versions of the page.

--- documents:100922imagej_cluster [2010/10/01 12:19] – synthax highlighter kota
+++ documents:100922imagej_cluster [2016/05/24 12:46] (current) – external edit 127.0.0.1
@@ Line 7: / Line 7: @@
   * headless.jar
     * This .jar file manages to work with ImageJ headlessly.
-    * download headless.jar file from [[http://sourceforge.net/projects/imageja|http://sourceforge.net/projects/imageja]]. Place it in the current directory /home/miura/test
+    * accessible by /g/almf/software/ij/headless.jar from within EMBL network.
+      * Otherwise, download headless.jar file from [[http://sourceforge.net/projects/imageja|http://sourceforge.net/projects/imageja]]. Place it somewhere you could access.
   * blobs.tif
-    * You could download this by Normal ImageJ (with-head, or GUI). [File -> Open Samples -> Blob]. Save it in the current directory /home/miura/test
+    * You could download this by Normal ImageJ (with-head, or GUI). [File -> Open Samples -> Blob]. Save it in the current directory /home/<your username>/test
   * headlesstest.ijm
-    * You must created this ImageJ macro file and save it in the current directory. See below.
+    * You must write an ImageJ macro file and save it in the current directory. See below.
   * testpbs.sh
-    * You must created this ImageJ macro file and save it in the current directory. See below.
+    * You must write a shell script and save it in the current directory. See below.
 **Write an ImageJ macro** that does the actual processing.
@@ Line 33: / Line 34: @@
 #PBS -N TestPBS
 #PBS -l walltime=1:00:00
-/usr/struct/bin/java -classpath headless.jar:/struct/software/linux/imagej-1.42/ij.jarr -Djava.awt.headless=true ij.ImageJ -batch headlesstest.ijm /home/miura/test/blobs.tif
+/usr/struct/bin/java -classpath /g/almf/software/ij/headless.jar:/g/almf/software/ij/ij-1.44h.jar -Djava.awt.headless=true ij.ImageJ -batch /home/<your user name>/test/headlesstest.ijm /home/<your user name>/test/blobs.tif
 </code>
 Some more explanation:
@@ Line 60: / Line 61: @@
 Using plugin is a bit tricky. In case of desktop ImageJ, what you need to install plugin is simply drug and drop .class or .jar files to plugins folder within imageJ folder. With headless mode, you need to do the following:
   * explicitly state where the plugins folder resides by setting -ijpath.
-  * all plugins must not be in jar. They all have to be class files.
+  * <del>all plugins must not be in jar. They all have to be class files.</del>
-    * if you only have jar file, then you should open it by zip archiver and extract all class files. File hierarchy should be maintained within the plugin folder.
+    * <del>if you only have jar file, then you should open it by zip archiver and extract all class files. File hierarchy should be maintained within the plugin folder.</del>
-... so the java command would look like this:
+... so the java command would look like this: Macro file "headlesstest2.ijm" is supposed to be at $HOME and images to be processed are under ~/test, and headlesstest2.ijm takes this path as an argument.
 <code>
-/usr/struct/bin/java -cp /home/miura/test/headless.jar:/home/miura/test/ij-1.44h.jar -Djava.awt.headless=true ij.ImageJ -ijpath /home/miura/test -batch headlesstest2.ijm /home/miura/test
+cd ~
+/usr/struct/bin/java -cp /g/almf/software/ij/headless.jar:/g/almf/software/ij/ij-1.44h.jar -Djava.awt.headless=true ij.ImageJ -ijpath /g/almf/software/ij -batch headlesstest2.ijm /home/miura/test
 </code>
-and the plugins folder resides within /home/miura/test.
+and the plugins folder resides within /g/almf/software/ij/ in the above case. If you need to add a plugin, please ask Kota.
 ===== Spreading Jobs with qsub =====
@@ Line 73: / Line 78: @@
 For example, if there are 100 files to be image-processed, job script for each file could be prepared with numerated job file name, and submitted to the  cluster. Jobs are then spread to vacant resources (uses vacant CPU).
-Here is a script for generating job scripts for all files in a directory (e.g. for 100 files, then 100 scripts + job array script is generated).
+Preparation of scripts could be automated. [[http://cmci.embl.de/documents/100922imagej_cluster#sample_imagej_macro_for_generating_job_array|I will paste an example ImageJ macro for doing this.]] The macro generates job scripts for all files in a directory (e.g. for 100 files, then 100 scripts + job array script is generated).
-<code>
+There are 7 arguments required to run this script.
-/* ImageJ macro for generating shell scripts to be used by qsub array
-	Kota miura (miura@embl.de) 2010
-	...better be written as a shell script in future.
+  * arg[0]: path to the folder cotaining images to be processed
+  * arg[1]: path to the output folder for processed images
+  * arg[2]: path to the folder where scripts will be saved
+  * arg[3]: prefix of job shell scrips. should be longer than 2 chars
+  * arg[4]: path to ij jars
+  * arg[5]: path to IJ macro
+  * arg[6]: IJ macro name
- to use this macro, adjust path variables BASEP, IJMACROFILE, IJP, SUN_JAVA
- you might probably need to change the name of imageJ jar file of var_ijjar
- requres three arguments separated by colon.
+After generating files, change the permission to all the script files so that they could be executed. Should be something like (if your job script prefix is job_):
- arg[0]: path to the folder cotaining images to be processed
+<code>
- arg[1]: path to the folder where scripts will be saved
- arg[2]: prefix of job shell scrips. should be longer than 2 chars
- Example command for execution
+chmod +x job_*
- /usr/struct/bin/java -cp headless.jar:ij-1.44h.jar -Djava.awt.headless=true ij.ImageJ -batch JCreate.ijm /home/miura/test/testsrc:/home/miura/test/testdest:jobbb
-*/
-/* path where batch file (IJ macro) is placed,*/
-var BASEP = "\/home\/miura\/test";
-/* file name of IJ macro*/
-var IJMACROFILE = "headlesstest3.ijm";
-/* path to folder where ij.jar and plugins folder is*/
-var IJP = "\/home\/miura\/test";
-/* path to SUN java*/
-var SUN_JAVA = "\/usr\/struct\/bin\/java";
-var var_headless =  "HEADLESS=" + IJP + "\/headless.jar\n";
-var var_ijjar =     "IJ_JAR=" + IJP + "\/ij-1.44h.jar\n";
-var var_ijpath =    "IJ_PATH=" + IJP + "\n";
-var var_batchfile = "BATCH_FILE=" + BASEP + "\/" + IJMACROFILE + "\n";
-var var_batcharg =  "BATCH_ARG=";
-var vars = var_headless
-	+ var_ijjar
-	+ var_ijpath
-	+ var_batchfile
-	+ var_batcharg;
-var jobfilePrefix = "job_";
-arg = getArgument();
-argA = split(arg, ":");
-srcdir = argA[0];
-destdir = argA[1];
-argjobprefix = argA[2];
-if (lengthOf(argjobprefix)>2){
-	jobfilePrefix = argjobprefix;
-}
-vars += srcdir + File.separator;
-print("input: " + srcdir);
-print("output: " + destdir);
-if (!File.isDirectory(srcdir)) {
-	print("Abort: source directory does not exist!");
-	exit();
-}
-if (!File.isDirectory(destdir)) {
-	File.makeDirectory(destdir);
-}
-filesA = getFileList(srcdir);
-jobfilename = "job_";
-for (i = 0; i< filesA.length; i++){
-	generateJobScript(filesA[i], i, destdir);
-}
-jobarraystring = "#!/bin/sh\n"
-		+ "\n"
-		+ "#PBS -J 1-" + filesA.length + "\n"
-		+ "#PBS -q clusterng\n"
-		+ destdir+  File.separator
-		+ jobfilePrefix + "$PBS_ARRAY_INDEX.sh";
-jobarrayfile_fullpath = destdir + File.separator + "jobarray.sh";
-File.saveString(jobarraystring, jobarrayfile_fullpath);
-function generateJobScript(filename, number, destdir){
-	header = "#!/bin/sh\n"
-	+ "#PBS -N TestPBS\n"
-	+ "#PBS -l walltime=1:00:00\n";
-	varslocal = vars +  filename;
-	command = SUN_JAVA + " "
-		+ "-cp $HEADLESS:$IJ_JAR -Djava.awt.headless=true "
-		+ "ij.ImageJ -ijpath $IJ_PATH -batch $BATCH_FILE $BATCH_ARG";
-	job = header + "\n\n" + varslocal + "\n" + command + "\n";
-	print(job);
-	jobname = jobfilePrefix+ (number+1) + ".sh";
-	fullpath = destdir + File.separator + jobname;
-	File.saveString(job, fullpath);
-}
 </code>
-Create a text file as above and save it to a directory where you could access (in the command below, the file is named JCreate.ijm). Then type the following command.
-**/usr/struct/bin/java -cp headless.jar:ij-1.44h.jar -Djava.awt.headless=true ij.ImageJ -batch JCreate.ijm /home/miura/test/testsrc:/home/miura/test/testdest
-**\\
-In above case, all the scripts will be generated under
-/home/miura/test/testdest
-for all the files in
-/home/miura/test/testsrc
-after generating files, change the permission to all the script files so that they could be executed. Should be something like
-<code>
-chmod +x job_*
-</code>
 ==== Example work flow ====
@@ Line 256: / Line 163: @@
 </sxh>
+===== Automated Script (Only from within EMBL network) =====
+Automated Script is available in the ALMF server (clust5fullp.sh). With this script, you only need to invoke the script with two arguments.
+  * arg[0]: full path to the directory containing image/stack files
+  * arg[1]: name of the imageJ macro you want to apply.
+<code>
+sh /g/almf/software/ij/clust5fullp.sh <path>/<to>/<ImageContainingFolder> <Name of ImageJ Macro>
+</code>
+argument <path>/<to>/<ImageContainingFolder> is full path to the folder where images / stacks are stored.
-===== Automated Script (Only from within EMBL network) =====
+argument <Name of ImageJ Macro> is the file name of ImageJ macro, and **this file should be placed under /g/almf/software/ij**.
-Automated Script is available, with which you need to change only two lines defining the ImageJ macro that you want to apply to images in a folder.
+For example, your command could be
-[[clust5fulllpath.sh]]
-lines to be changed are
 <code>
-  IJMACROPATH="/g/almf/miura/pub"
+sh /g/almf/software/ij/clust5fullp.sh /g/almf/miura/testsmalls headlesstest3.ijm
-...
-  IJMACRONAME="headlesstest3.ijm"
 </code>
-Path to the macro and name of the macro file should be replaced according to your situation.
-rename and save the script, and do
+This script will create two new folders in the directory same as where the image directory is (*_job and *_proc). Processed images or stacks will be saved under *_proc. \\
+Besides, so-called job reports (text files containing output messages) will appear in your current directory where you executed the command.
+The script could be viewed in the link below: [[http://cmci.embl.de/documents/100922imagej_cluster#jobarray_generator|clust5fulllpath.sh]]
+===== Example Scripts =====
+==== Cluster ImageJ processing script ====
+Sample Shell Script that does all.
+<sxh shell>
+#!/bin/sh
+#script for imageJ cluster calculation
+#clust5fullp.sh argument must be the full path to the folder containing source images/stacks.
+#
+imgdir=$1
+if test -d ${imgdir}
+then
+	echo ${imgdir}" ...input directory"
+else
+	echo ${imgdir}" ...no such directory."
+	exit 1
+fi
+outdir=${imgdir}"_proc"
+if test -d ${outdir}
+then
+	echo ${outdir}" ...output directory"
+else
+	mkdir ${outdir}
+	echo ${outdir}" ...output directory created"
+fi
+jobdir=${imgdir}"_jobs"
+if test -d ${jobdir}
+then
+	echo ${jobdir}" ...job scripts dir"
+else
+	mkdir ${jobdir}
+	echo ${jobdir}" ...job scripts directory created"
+fi
+#curdir=`pwd`
+# path to IJ jar file
+IJJARS="/g/almf/miura/pub"
+# path to images and stacks
+#SRCPATH="/home/miura/test/testsrc"
+#SRCPATH="/g/almf/miura/testsmalls"
+#SRCPATH=${curdir}"/"${imgdir}
+SRCPATH=${imgdir}
+#path to output directory
+#OUTPATH=${curdir}"/"${outdir}
+OUTPATH=${outdir}
+# path to job array generator macro
+#JOBGENPATH="/home/miura/test/ij"
+JOBGENPATH="/g/almf/miura/pub"
+# name of job array generator
+JOBGENNAME="JCreate3.ijm"
+# path to save job scripts and job array script
+#JOBPATH="/home/miura/test/job3"
+#JOBPATH=${curdir}"/"${jobdir}
+JOBPATH=${jobdir}
+# base name (prefix) of job script generated for each images/stacks
+#JOBPREF="job_"${imgdir}
+JOBPREF="job_"
+# path to image processing IJ macro
+#IJMACROPATH="/home/miura/test/ij"
+IJMACROPATH="/g/almf/miura/pub"
+# image processing IJ macro name
+IJMACRONAME="headlesstest3.ijm"
+echo "IJ full-path ${IJJARS}/headless.jar"
+#timer
+jobstart=$(date +%s)
+#jobstartN=$(date +%N)
+macroarg=${SRCPATH}:${OUTPATH}:${JOBPATH}:${JOBPREF}:${IJJARS}:${IJMACROPATH}:${IJMACRONAME}
+echo ${macroarg}
+/usr/struct/bin/java -cp ${IJJARS}/headless.jar:${IJJARS}/ij-1.44h.jar -Djava.awt.headless=true ij.ImageJ -batch ${JOBGENPATH}/${JOBGENNAME} ${macroarg}
+chmod +x ${JOBPATH}/${JOBPREF}*
+qsub ${JOBPATH}/jobarray.sh
+# timer
+jobend=$(date +%s)
+#jobendN=$(date +%N)
+echo "Time: $((jobend-jobstart)) secs."
+#echo "Time: $((jobendN-jobstartN)) nano-sec."
+</sxh>
+==== Sample ImageJ macro for Generating Job Array ====
+The ImageJ macro below is the actual content of the script above (called in line 75). it generates one script for one image/stack, and also meta-script called jobarray.sh for the submission to cluster.
 <code>
-sh <renamedFile>.sh <path>/<to>/<ImageContainingFolder>
+/* ImageJ macro for generating shell scripts to be used by qsub array
-</code>
+	Kota miura (miura@embl.de) 2010
-argument <path>/<to>/<ImageContainingFolder> is full path to the folder where images / stacks are stored.
+	...better be written as a shell script in future.
+ to use this macro, adjust path variable SUN_JAVA if required
+ you might probably need to change the name of imageJ jar file of var_ijjar
+ requres three arguments separated by colon.
+ arg[0]: path to the folder cotaining images to be processed
+ arg[1]: path to the output folder for processed images
+ arg[2]: path to the folder where scripts will be saved
+ arg[3]: prefix of job shell scrips. should be longer than 2 chars
+  arg[4]: path to ij jars
+  arg[5]: path to IJ macro
+  arg[6]: IJ macro name
+ Example command for execution
+ /usr/struct/bin/java -cp headless.jar:ij-1.44h.jar -Djava.awt.headless=true ij.ImageJ -batch JCreate.ijm /home/miura/test/testsrc:/home/miura/test/testdest:jobbb
+*/
+/* path where batch file (IJ macro) is placed,*/
+var BASEP = "\/home\/miura\/test\/ij";
+/* file name of IJ macro*/
+var IJMACROFILE = "headlesstest3.ijm";
+/* path to folder where ij.jar and plugins folder is*/
+var IJP = "\/home\/miura\/test\/ij";
+/* path to SUN java*/
+var SUN_JAVA = "\/usr\/struct\/bin\/java";
+var var_headless;
+var var_ijjar;
+var var_ijpath;
+var var_batchfile;
+var var_batcharg;
+var vars;
+var jobfilePrefix = "job_";
+arg = getArgument();
+print(arg);
+argA = split(arg, ":");
+//for (i=0; i<argA.length; i++) print(argA[i]);
+if (argA.length != 7){
+	print("Number of Arguments:" + argA.length);
+	print("Abort Generator: There should be exactly 7 arguments for running script generator.");
+	exit();
+}
+srcdir = argA[0];
+outdir = argA[1];
+destdir = argA[2];
+argjobprefix = argA[3];
+if (lengthOf(argjobprefix)>2){
+	jobfilePrefix = argjobprefix;
+}
+IJP = argA[4];
+BASEP = argA[5];
+IJMACROFILE = argA[6];
+print("variables taken from arguments");
+setVariables();
+vars += srcdir + File.separator;
+print("input: " + srcdir);
+print("output: " + outdir);
+print("jobs: " + destdir);
+print("ImageJ Path: " + IJP);
+print("ImageJ macro Path: " + BASEP);
+print("ImageJ macro Name: " + IJMACROFILE);
+if (!File.isDirectory(srcdir)) {
+	print("Abort: source directory does not exist!");
+	exit();
+}
+if (!File.isDirectory(outdir)) {
+	print("Abort: output directory does not exist!");
+	exit();
+}
+if (!File.isDirectory(destdir)) {
+	File.makeDirectory(destdir);
+}
+if (!File.isDirectory(IJP)) {
+	print("Abort: ImageJ directory does not exist!");
+	exit();
+}
+if (!File.isDirectory(BASEP)) {
+	print("Abort: ImageJ Image Processing Macro directory does not exist!");
+	exit();
+}
+if (!File.exists(BASEP + File.separator + IJMACROFILE)) {
+	print("Abort: Image Processing Macro File does not exist!");
+	exit();
+}
+filesA = getFileList(srcdir);
+jobfilename = "job_";
+for (i = 0; i< filesA.length; i++){
+	generateJobScript(filesA[i], i, destdir);
+}
+jobarraystring = "#!/bin/sh\n"
+		+ "\n"
+		+ "#PBS -J 1-" + filesA.length + "\n"
+		+ "#PBS -q clusterng\n"
+		+ destdir+  File.separator
+		+ jobfilePrefix + "$PBS_ARRAY_INDEX.sh";
+jobarrayfile_fullpath = destdir + File.separator + "jobarray.sh";
+File.saveString(jobarraystring, jobarrayfile_fullpath);
+function setVariables(){
+	var_headless =  "HEADLESS=" + IJP + "\/headless.jar\n";
+	var_ijjar =     "IJ_JAR=" + IJP + "\/ij-1.44h.jar\n";
+	var_ijpath =    "IJ_PATH=" + IJP + "\n";
+	var_batchfile = "BATCH_FILE=" + BASEP + "\/" + IJMACROFILE + "\n";
+	var_batcharg =  "BATCH_ARG=";
+	vars = var_headless
+		+ var_ijjar
+		+ var_ijpath
+		+ var_batchfile
+		+ var_batcharg;
+}
+function generateJobScript(filename, number, destdir){
+	header = "#!/bin/sh\n"
+	+ "#PBS -N TestPBS\n"
+	+ "#PBS -l walltime=1:00:00\n";
+	varslocal = vars +  filename;
+	command = SUN_JAVA + " "
+		+ "-cp $HEADLESS:$IJ_JAR -Djava.awt.headless=true "
+		+ "ij.ImageJ -ijpath $IJ_PATH -batch $BATCH_FILE $BATCH_ARG";
+	job = header + "\n\n" + varslocal + "\n" + command + "\n";
+	print(job);
+	jobname = jobfilePrefix+ (number+1) + ".sh";
+	fullpath = destdir + File.separator + jobname;
+	File.saveString(job, fullpath);
+}
+</code>
@@ Line 329: / Line 484: @@
 ===== Acknowledgements =====
-Thanks to Frank Thommen(SCB) and Andres Lindau (IT support) for their great help.
+Thanks to Frank Thommen(SCB), Andres Lindau (IT support) and Christian Tischer (ALMF) for their great help and suggestions.