User Tools

Site Tools


Sidebar

Top
Seminar
Courses -2016
Courses 2018-
Textbooks
Documents
Downloads (-2016)
Downloads (2016-)
Weblog
RSS aggregates
Discussions
Archives


EMBL BioImage Data Analysis

EuBIAS

NEUBIAS

—- Contact
CMCI Alumni
ALMF
EMBL Heidelberg
EMBL Intranet


Popularity Ranking


Timeline of @cmci_

cmci_ avatar

Yoshikatsu Sato, the head of microscope facility @NagoyaITbM presenting practicals #EMBOLivePlant https://t.co/SVrkfOKQCn
About 50 mins, 17 secs ago by: Kota Miura (@cmci_)

cmci_ avatar

Miyo Terao Morita showing analysis of gravitropism using centrifuge microscope they newly developed. I only know… https://t.co/Atj9E4QJAY
About 1 hour, 22 mins ago by: Kota Miura (@cmci_)

cmci_ avatar

@KeikoUTorii ... which then became the cover page of Development cell. convincing. https://t.co/XBgiqJpHog
About 2 hours, 19 mins ago by: Kota Miura (@cmci_)

cmci_ avatar

@KeikoUTorii model-based hypothesis lead to experiment results with single guard cell development or multiple >2 gu… https://t.co/oGhh4NJMPh
About 2 hours, 26 mins ago by: Kota Miura (@cmci_)

cmci_ avatar

now, @KeikoUTorii on stomata development - how very specific cell fate decisions are made, with proper spacing bet… https://t.co/XoR8tpB5WR
About 2 hours, 44 mins ago by: Kota Miura (@cmci_)

cmci_ avatar

@YvonJaillais talk on nanoclustering of ROP6, great analyses using various approaches to assess protein mobility… https://t.co/WHrzBsWTYJ
About 3 hours ago by: Kota Miura (@cmci_)
documents:101103imagej_cluster_short

Using Cluster (Quick Version)

Note: This page is outdated (June 1, 2016). I will prepare a newer version.

Since most of the scripts for cluster use is located under a specific directory in the ALMF server, I recommend you to add the path to your .tcshrc as below.

set path=($path /g/almf/software/bin2)

Otherwise, please add the path above in front of script name.

Interactive Usage of Cluster CPU

Before submitting huge data with your script, you might want to test run the script interactively to see if it is indeed working. To do so, you could access cluster in interactive mode.

  1. Log in to sub-master
  2. then “qsub -I”
    1. wait for some seconds, you will be assigned to a specific node where you could test around.
    2. To enable multi-threaded computation, do “qsub -I -l select=ncpus=2:mem=3gb” (in this case, uses 2 CPUs and 3Gb of memory).

Processing by ImageJ from Command Line

Script name: doIJ

Syntax

sh doIJ <fullpath to macro file> <fullpath to image file>

For example:

sh /g/almf/software/bin2/doIJ.sh /g/almf/software/ij/getArgumentMacro.ijm /g/almf/miura/blobs.tif

Use this script only for small tasks. Heavy calculation consumes the server resource where others are also using.

Processing Single Image file by ImageJ, throw the job to cluster

Script name: qsubIJ

To process image file /a/b/c.tif with an ImageJ macro file /d/e/f.ijm, like above, but with cluster

sh qsubIJ /d/e/f.ijm /a/b/c.tif

Job will be submitted to the cluster, and what should be the output from ImageJ will be saved as a text file in your ALMF home directory:
/g/almf/<Your Unix Username>/CLlogyyyymmdd_hhmm.txt.
“yyyymmdd_hhmm” should be the time of execution.

a bit more on customization

To customize arguments and so on, shell commands should be written in another shell script as follows:

#!/bin/sh
#PBS -N ImageJProcess
#PBS -l walltime=1:00:00
#
# *** Script for single image file procesing by ImageJ ***
#
#   two arguments for full paths to (2) an image file and (1) a macro file is required
#   assuming that macro file takes the fisrt argument as the 
#   Kota Miura (cmci.embl.de)
#   20101103
#   ****************************************************
#
sh /g/almf/software/ij/doIJ.sh /g/almf/software/ij/getArgumentMacro.ijm /g/almf/miura/blobs.tif

and then save the script as something like “cl_doIJ.sh”. Execute the job by following command in terminal

/usr/pbs/bin/qsub -q clusterng /g/almf/software/ij/cl_doIJ.sh

Processing All Image Files in a Folder

Script name: qsubIJbatch

Syntax is

sh qsubIJbatch </full/path/to/dir> <File Name of Macro File>

For example, if you want to process all images under /g/almf/miura/testsmalls using an ImageJ macro file headlesstest3.ijm, then your command would be:

sh qsubIJbatch /g/almf/miura/testsmalls headlesstest3.ijm

Example Commands for Bory and Christian

converting lif to tif stacks:
sh /g/almf/software/ij/lifcon.sh /g/haering/Pombescreen/test.lif
sh /g/almf/software/bin2/lifconvert /g/haering/Pombescreen/pb2/100927ExpNo2.lif

actual processing:
sh /g/almf/software/bin2/qsubIJbatch /g/haering/Pombescreen/test.lif_tifStack headlesstest3.ijm

documents/101103imagej_cluster_short.txt · Last modified: 2016/06/01 02:57 by kota