Rafael Saldaña (Ateneo de Manila University, Philippines)
Tuesday, April 8, 2008
Dr. Rafael Saldaña: EUAsia Grid Kick-of Meeting, 8 April 2008
Rafael Saldaña (Ateneo de Manila University, Philippines)
Dr. Alexander Voss: EU-Asia Grid Kick-off Meeting, 8 April 2008
Alexander Voss (NCeSS, United Kingdom)
Dr. Ludek Matyska: EUAsia Grid Kick-off Meeting, 8 April 2008
Ludek Matyska (CESNET, Czech Republic)
EUAsia Grid Kick-off Meeting, Taipei, Taiwan, 8 April 2008
EUAsia Grid Project Meeting
3F, Buiding for Humanities and Social Sciences
Academia Sinica, Taipei, Taiwan
13:30 - 17:40, Tuesday, 8 April 2008
13:30 - 18:00, Wednesday, 9 April 2008
PROGRAM
13:30 - 17:40, Tuesday, 8 April 2008
13:30 - 14:15 General Description (Marco Paganoni, Italy)
14:15 - 14:45 Administration (Ludek Matyska, Czech Republic)
14:45 - 14:55 INFN (Marco Paganoni, Italy)
14:55 - 15:05 CESNET (Ludek Matyska, Czech Republic)
15:05 - 15:15 NCeSS (Alexander Voss, Mark Birkin, UK)
15:15 - 15:40 Coffee Break
15:40 - 15:50 HealthGrid (Yannick Legre, France)
15:50 - 16:00 Ateneo de Manila University (Rafael Saldaña, Philippines)
16:00 - 16:10 ANU (George MacLaughlin, Australia) (on behalf of Markus Buckhorn)
16:10 - 16:20 ASGC (Simon Lin/ Eric Yen, Taiwan)
16:20 - 16:30 ASTI (Rey Vincent Babilonia, Philippines)
16:30 - 16:40 HAII (Royol Chitradon, Thailand)
16:40 - 16:50 NGO (Hing-Yen Lee, Singapore) (Remote)
16:50 - 17:00 IOT-HCM (Tran Van Lang/Do Van Long, Vietnam)
17:00 - 17:10 ITB (Basuki Suhardiman, Indonesia)
17:20 - 17:30 UPM (Suhaimi Napis, UPM, Malaysia)
17:30 - 17:40 MIMOS (Jingyuan Luke, Malaysia)
3F, Buiding for Humanities and Social Sciences
Academia Sinica, Taipei, Taiwan
13:30 - 17:40, Tuesday, 8 April 2008
13:30 - 18:00, Wednesday, 9 April 2008
PROGRAM
13:30 - 17:40, Tuesday, 8 April 2008
13:30 - 14:15 General Description (Marco Paganoni, Italy)
14:15 - 14:45 Administration (Ludek Matyska, Czech Republic)
14:45 - 14:55 INFN (Marco Paganoni, Italy)
14:55 - 15:05 CESNET (Ludek Matyska, Czech Republic)
15:05 - 15:15 NCeSS (Alexander Voss, Mark Birkin, UK)
15:15 - 15:40 Coffee Break
15:40 - 15:50 HealthGrid (Yannick Legre, France)
15:50 - 16:00 Ateneo de Manila University (Rafael Saldaña, Philippines)
16:00 - 16:10 ANU (George MacLaughlin, Australia) (on behalf of Markus Buckhorn)
16:10 - 16:20 ASGC (Simon Lin/ Eric Yen, Taiwan)
16:20 - 16:30 ASTI (Rey Vincent Babilonia, Philippines)
16:30 - 16:40 HAII (Royol Chitradon, Thailand)
16:40 - 16:50 NGO (Hing-Yen Lee, Singapore) (Remote)
16:50 - 17:00 IOT-HCM (Tran Van Lang/Do Van Long, Vietnam)
17:00 - 17:10 ITB (Basuki Suhardiman, Indonesia)
17:20 - 17:30 UPM (Suhaimi Napis, UPM, Malaysia)
17:30 - 17:40 MIMOS (Jingyuan Luke, Malaysia)
Monday, April 7, 2008
APGRID Meeting In ISGC 2008 (Taipei, Taiwan)
Links:
http://www.apgrid.org
http://www.apgridpma.org/meetings/index.html
4th APGrid PMA Meeting
Academia Sinica, Taipei, Taiwan
April 8, 2008
This F2F meeting is co-located with ISGC 2008 event.
ASGC TWGrid
Registration
The plenary session is open for everyone. The other sessions of the meeting is closed to IGTF members. Registration is required through the ISGC 2008 registation web site. All participants are strongly encouraged to participate in ISGC 2008 event.
Accommodation
Information of your accomodation is also available on the ISGC 2008 web site.
Transportation
Transportation services between the recommended hotels (Howard Plaza, etc.) and the meeting venue will be provided.
Visa information
If you need visa, you will be helped by ISGC 2008 organizer. You can request supporting documents for visa through the registration process.
Agenda (tentative)
Plenary Session (open for everyone)
09:00 - 09:10
Welcome address (Eric Yen, Academia Sinica)
09:10 - 09:20
Self Introduction of participants
09:20 - 09:50
Examination for accreditation of a CA (1)
PRAGMA-UCSD CA (Cindy Zheng)
CP/CPS
slides
09:50 - 10:10
Opening remarks including updates of APGridPMA (Yoshio Tanaka, AIST) [slides]
10:10 - 10:30
Updates of EUGrid PMA (David Groep, NIKHEF)
Coffee/Tea Break
10:30 - 11:00
Coffee/tea breakClosed Session (1)
11:00 - 11:20
Updates of TAGPMA (Doug Olson, LBL)
11:20 - 11:50
Examination for accreditation of a CA (2)
NGO/Netrust CA (Jon Lau and Foo Jong Ai)
CP/CPS slides
11:50 - 12:10
Next F2F meeting
chair election
ML
AOB Lunch Break
12:10 - 13:30
Lunch break, PGP and Thawte Key Signing Party
Closed Session (2)
13:30 - 14:00
Examination for accreditation of a new CA (3)
NCHC CA (Weicheng Whuang)
CP/CPS slides
14:00 - 14:30
Beief status reports from member CAs
APAC
CNIC
IHEP
KEK
New CP/CPS
New Profile
Changes on CP/CPS
New enrollment manual
KISTI
NAREGI
NECTEC
THAI
14:30 - 15:00
In-depth report of self-auditing (1)
AIST
slides
Coffee/Tea Break
15:00 - 15:30
Coffee/Tea breakClosed Session (3)
15:30 - 16:00
In-depth report of self-auditing (2)
ASGCC
New CP/CPS
slides
Audit Results
16:00 - 16:30
Distributed sponsorship based ID vetting (Doug Olson)
16:30 - 17:00
AOB, Summary and closing
http://www.apgrid.org
http://www.apgridpma.org/meetings/index.html
4th APGrid PMA Meeting
Academia Sinica, Taipei, Taiwan
April 8, 2008
This F2F meeting is co-located with ISGC 2008 event.
ASGC TWGrid
Registration
The plenary session is open for everyone. The other sessions of the meeting is closed to IGTF members. Registration is required through the ISGC 2008 registation web site. All participants are strongly encouraged to participate in ISGC 2008 event.
Accommodation
Information of your accomodation is also available on the ISGC 2008 web site.
Transportation
Transportation services between the recommended hotels (Howard Plaza, etc.) and the meeting venue will be provided.
Visa information
If you need visa, you will be helped by ISGC 2008 organizer. You can request supporting documents for visa through the registration process.
Agenda (tentative)
Plenary Session (open for everyone)
09:00 - 09:10
Welcome address (Eric Yen, Academia Sinica)
09:10 - 09:20
Self Introduction of participants
09:20 - 09:50
Examination for accreditation of a CA (1)
PRAGMA-UCSD CA (Cindy Zheng)
CP/CPS
slides
09:50 - 10:10
Opening remarks including updates of APGridPMA (Yoshio Tanaka, AIST) [slides]
10:10 - 10:30
Updates of EUGrid PMA (David Groep, NIKHEF)
Coffee/Tea Break
10:30 - 11:00
Coffee/tea breakClosed Session (1)
11:00 - 11:20
Updates of TAGPMA (Doug Olson, LBL)
11:20 - 11:50
Examination for accreditation of a CA (2)
NGO/Netrust CA (Jon Lau and Foo Jong Ai)
CP/CPS slides
11:50 - 12:10
Next F2F meeting
chair election
ML
AOB Lunch Break
12:10 - 13:30
Lunch break, PGP and Thawte Key Signing Party
Closed Session (2)
13:30 - 14:00
Examination for accreditation of a new CA (3)
NCHC CA (Weicheng Whuang)
CP/CPS slides
14:00 - 14:30
Beief status reports from member CAs
APAC
CNIC
IHEP
KEK
New CP/CPS
New Profile
Changes on CP/CPS
New enrollment manual
KISTI
NAREGI
NECTEC
THAI
14:30 - 15:00
In-depth report of self-auditing (1)
AIST
slides
Coffee/Tea Break
15:00 - 15:30
Coffee/Tea breakClosed Session (3)
15:30 - 16:00
In-depth report of self-auditing (2)
ASGCC
New CP/CPS
slides
Audit Results
16:00 - 16:30
Distributed sponsorship based ID vetting (Doug Olson)
16:30 - 17:00
AOB, Summary and closing
Sunday, April 6, 2008
YAIM Overview (Tutorial Session, International Symposium On Grid Computing 2008, Taipei, Taiwan)
NOTES on the EGEE Tutorial, 7 April 2008, Academia Sinica, Taipei, Taiwan.
http://quattor.grid.sinica.edu.tw/UI_practical.html
Topic: YAIM Introduction
This tutorial will guide you through the process of installing a User Interface (UI). In particular we will install a TAR UI which is installed into your own home directory and does not require root permission. The standard UI installation will require root permissions, but will allow all users to access UI commands. For this tutorial we will use the Yaim Tutorial Server. Please refer to the tutorial handout sheet for server and account details.
1. Installing the UI TAR
Install the UI_TAR packages
Configure the UI
2. Preparing your UI environment
Configure the Grid Environment
Install your user certificate
3. Test the Grid jobs submission by your UI
Query the Grid Computing Environment
Simple job submission
Submitting jobs by the Resource Broker
Query the job status
Get your job results
4. Test the Grid data stor by your UI
Query the Grid Storage Environment
Set the DPNS HOST
Create directory on diskpool
Put your file into diskpool
Down file from diskpool
Installing the UI TAR
1. Please make sure the following file is already into your home directory
[taipei01@t-yaim~]$ ls -1F certificates/gLite/glite-UI-3.1.8-0-external.tar.gzglite-UI-3.1.8-0.tar.gzjobs/site-info.def.isgc2008
You can also find the last packages by the following link:The relocatable UI
Notice: The PPS is the pre-production packages, that sometime have several new features, but might also have some bugs, If you are not certainty to use the new features, we recommend you to use the production packages.
To install the UI_TAR, you have only to untar the UI tar ball in your own directory.
2. Untar the tar balls
[taipei01@t-yaim~]$ tar -xzf glite-UI-3.1.8-0-external.tar.gz -C gLite [taipei01@t-yaim~]$ tar -xzf glite-UI-3.1.8-0.tar.gz -C gLite
3. Change the work directory and check the installed files[taipei01@t-yaim gLite]$ cd gLite[taipei01@t-yaim gLite]$ ls -Fclassads/ edg/ external/ globus/ lcg/d-cache/ etc/ glite/ gpt/ tmp.31683/[taipei01@t-yaim gLite]$
4. Run the yaim script to configure the UI_TAR. Here we use the -c "configure command" to configure the UI. The configuration has already been completed for you in site-info.def. Please take a look inside the file to get an idea of how to fill in the yaim parameters. This particular site-info.def allows you to configure a UI that uses the gilda grid infrastructure.[taipei01@t-yaim gLite]$./glite/yaim/bin/yaim -c -s ~/site-info.def.isgc2008 -n UI_TAR
You shouldn't get any error messages.
Notice:Please pay attention to the X509_CERT_DIR of site-info.def variable, here we used "/etc/grid-security/certificates/", if you have not root permission and your computer never installed any certificates, you should modify the X509_CERT_DIR to fit your own directory, the yaim script will go to install all the certificates into this directory.For example:If you have specifyX509_CERT_DIR=$HOME/gLite/external/etc/certificatesin your site-info.def,The yaim script will install all the certificates into $HOME/gLite/external/etc/certificatesHere we were already installed gilda certificates into /etc/grid-security/certificates/ for this tutorial, so please don't modify the X509_CERT_DIR in this practical anyway.
Preparing your UI environment
5. Setup the environment. UI tar package contains two environment scripts that must be sourced in order to use the UI client tools. Here we add these scripts to your startup script, that can help to auto configure the UI environment every time you login the shell.[taipei01@t-yaim gLite]$ echo 'source $HOME/gLite/external/etc/profile.d/grid-env.sh' >>~/.bashrc
Now, you can re-login to the shell or just running following command:[taipei01@t-yaim gLite]$ source ~/.bashrc
6. Put your user certificate and private key into ~/.globus. After you obtain your certificate from your CA, you will need to install them on an UI in the .globus directory.The commands below show you how this is done. Pay close attention to the permission of these certificates.[taipei01@t-yaim gLite]$cd ~[taipei01@t-yaim ~]$ mkdir .globus[taipei01@t-yaim ~]$ chmod 700 .globus[taipei01@t-yaim ~]$ cp certificates/TAIPEIxx-cert.pem .globus/usercert.pem[taipei01@t-yaim ~]$ cp certificates/TAIPEIxx-key.pem .globus/userkey.pem[taipei01@t-yaim ~]$ ls .globus/usercert.pem userkey.pem[taipei01@t-yaim ~]$ chmod 644 .globus/usercert.pem[taipei01@t-yaim ~]$ chmod 400 .globus/userkey.pem
The following step is only for this tutorial that will add gilda vo certificate into your own UI environment [taipei01@t-yaim ~]$ cp /opt/gilda-voms/gilda-voms.ct.infn.it ~/gLite/external/etc/grid-security/vomsdir/
7. Create a proxy certificate.
[taipei01@t-yaim ~]$ voms-proxy-init -voms gildaEnter GRID pass phrase:Your identity: /C=IT/O=GILDA/OU=Personal Certificate/L=TAIPEI/CN=TAIPEI01/Email=jennsu@ccweb.sinica.edu.twCreating temporary proxy ...................................... DoneContacting voms.ct.infn.it:15001 [/C=IT/O=INFN/OU=Host/L=Catania/CN=voms.ct.infn.it] "gilda" DoneCreating proxy ........................................................ Done Your proxy is valid until Fri Mar 26 21:56:48 2008
Congratulations, you have now configured and setup a Grid User Interface environment that is required to access the Grid resources. We will now test your environment by performing simple job submission and data management task.
Test the grid jobs submission by your UI
8. Check which CE are available on for gilda Virtual Organization with lcg-infosites:[taipei01@t-yaim ~]$ lcg-infosites --vo gilda ce
valor del bdii: grid004.ct.infn.it:2170
#CPU Free Total Jobs Running Waiting ComputingElement
----------------------------------------------------------
11 11 0 0 0 grid-ce.bio.dist.unige.it:2119/jobmanager-lcglsf-gilda
10 10 0 0 0 grid011f.cnaf.infn.it:2119/jobmanager-lcgpbs-gilda
34 34 0 0 0 grid010.ct.infn.it:2119/jobmanager-lcgpbs-long
34 34 0 0 0 grid010.ct.infn.it:2119/jobmanager-lcgpbs-short
34 34 0 0 0 grid010.ct.infn.it:2119/jobmanager-lcgpbs-infinite
56 46 2 2 0 iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-short
56 46 1 1 0 iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-long
56 46 0 0 0 iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-infinite
10 10 0 0 0 grid004.iucc.ac.il:2119/jobmanager-lcgpbs-short
10 10 0 0 0 grid004.iucc.ac.il:2119/jobmanager-lcgpbs-long
10 10 0 0 0 grid004.iucc.ac.il:2119/jobmanager-lcgpbs-infinite
16 16 0 0 0 gn0.hpcc.sztaki.hu:2119/jobmanager-lcgpbs-gilda
9.Test the CE by submitting a simple job that runs directly on the CE itself:[taipei01@t-yaim ~]$ globus-job-run grid011f.cnaf.infn.it /bin/hostname
grid011f.cnaf.infn.it
Make sure you have a correct output which is the hostname of the CE.
10. Test job submission through a resource broker:
Edit a jdl file first[taipei01@t-yaim ~]$ cd jobs/
[taipei01@t-yaim jobs]$ ls
edglog.log testJob.jdl testJob.sh
[taipei01@t-yaim jobs]$ vim testJob.jdl
* Please add following contents in to the testJob.jdl:
Executable = "testJob.sh";
StdOutput = "testJob.out";
StdError = "testJob.err";
InputSandbox = {"./testJob.sh"};
OutputSandbox = {"testJob.out","testJob.err"};
VirtualOrganisation = "gilda";
Feel free to modify the JDL and specify your job requirements.
Edit testJob.sh[taipei01@t-yaim jobs]$ vim testJob.sh
#!/bin/bash
date
hostname
echo "****************************************"
echo "env sort"
echo "****************************************"
env sort
echo "****************************************"
echo "mount"
echo "****************************************"
mount
date
11. Check which CEs match your JDL requirements.
usage: edg-job-list-match
[taipei01@t-yaim jobs]$ edg-job-list-match testJob.jdl
Selected Virtual Organisation name (from JDL): gilda
Connecting to host glite-rb.ct.infn.it, port 7772
***************************************************************************
COMPUTING ELEMENT IDs LIST
The following CE(s) matching your job requirements have been found:
*CEId*
ce-nano-37.to.infn.it:2119/jobmanager-lcgpbs-infinite
ce-nano-37.to.infn.it:2119/jobmanager-lcgpbs-long
ce-nano-37.to.infn.it:2119/jobmanager-lcgpbs-short
dgt01.ui.savba.sk:2119/jobmanager-lcgpbs-infinite
dgt01.ui.savba.sk:2119/jobmanager-lcgpbs-long
dgt01.ui.savba.sk:2119/jobmanager-lcgpbs-short
gilda-ce-01.pd.infn.it:2119/jobmanager-lcgpbs-infinite
gilda-ce-01.pd.infn.it:2119/jobmanager-lcgpbs-long
gilda-ce-01.pd.infn.it:2119/jobmanager-lcgpbs-short
gilda01.ihep.ac.cn:2119/jobmanager-lcgpbs-gilda
gn0.hpcc.sztaki.hu:2119/jobmanager-lcgpbs-gilda
grid010.ct.infn.it:2119/jobmanager-lcgpbs-infinite
grid010.ct.infn.it:2119/jobmanager-lcgpbs-long
grid010.ct.infn.it:2119/jobmanager-lcgpbs-short
iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-short
iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-infinite
iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-long
***************************************************************************
12. Submit the test job:
usage: edg-job-submit[taipei01@t-yaim jobs]$ edg-job-submit testJob.jdl
Selected Virtual Organisation name (from JDL): gilda
Connecting to host glite-rb.ct.infn.it, port 7772
Logging to host glite-rb.ct.infn.it, port 9002
*********************************************************************************************
JOB SUBMIT OUTCOME
The job has been successfully submitted to the Network Server.
Use edg-job-status command to check job current status. Your job identifier (edg_jobId) is:
- https://glite-rb.ct.infn.it:9000/js8FEQjvTiVDaIPwQhWqMA
*********************************************************************************************
* Notice, please remember your job id, in this case the job id is:
"https://glite-rb.ct.infn.it:9000/js8FEQjvTiVDaIPwQhWqMA"
Get the job status:
usage: edg-job-status[taipei01@t-yaim jobs]$ edg-job-status https://glite-rb.ct.infn.it:9000/js8FEQjvTiVDaIPwQhWqMA
*************************************************************
BOOKKEEPING INFORMATION:
Status info for the Job : https://glite-rb.ct.infn.it:9000/js8FEQjvTiVDaIPwQhWqMA
Current Status: Scheduled
Status Reason: Job successfully submitted to Globus
Destination: gilda-ce-01.pd.infn.it:2119/jobmanager-lcgpbs-infinite
reached on: Fri Oct 26 13:22:35 2007
*************************************************************
Check the status until the “Current Status” changes to "Done"[taipei01@t-yaim jobs]$ edg-job-status https://glite-rb.ct.infn.it:9000/js8FEQjvTiVDaIPwQhWqMA
*************************************************************
BOOKKEEPING INFORMATION:
Status info for the Job : https://glite-rb.ct.infn.it:9000/js8FEQjvTiVDaIPwQhWqMA
Current Status: Done (Success)
Exit code: 0
Status Reason: Job terminated successfully
Destination: gilda-ce-01.pd.infn.it:2119/jobmanager-lcgpbs-infinite
reached on: Fri Oct 26 13:28:08 2007
*************************************************************
12. Download the job results from the Resource Broker to your User Interface[taipei01@t-yaim jobs]$ edg-job-get-output https://glite-rb.ct.infn.it:9000/js8FEQjvTiVDaIPwQhWqMA
Retrieving files from host: glite-rb.ct.infn.it ( for https://glite-rb.ct.infn.it:9000/js8FEQjvTiVDaIPwQhWqMA )
*********************************************************************************
JOB GET OUTPUT OUTCOME
Output sandbox files for the job:
- https://glite-rb.ct.infn.it:9000/js8FEQjvTiVDaIPwQhWqMA
have been successfully retrieved and stored in the directory:
/home/taipei01/jobOutput/taipe01_js8FEQjvTiVDaIPwQhWqMA
*********************************************************************************
The result output path was define in site-info.def the OUTPUT_STORAGE variable, for this practical, the OUTPUT_STORAGE is $HOME/jobOutput/.
so the result will put into $HOME/jobOutput/taipei01_js8FEQjvTiVDaIPwQhWqMA
Check the result:[taipei01@t-yaim jobs]$ less ~/jobOutput/camp_js8FEQjvTiVDaIPwQhWqMA/testJob.out
Test the Grid data store by your UI
13. Get SE information:[taipei01@t-yaim gLite]$ lcg-infosites --vo gilda se
Avail Space(Kb) Used Space(Kb) Type SEs
----------------------------------------------------------
32350000 210590 n.a grid-se.bio.dist.unige.it
74980000 5153576 n.a egee016.cnaf.infn.it
321010000 50293604 n.a aliserv6.ct.infn.it
50660000 7340000 n.a iceage-se-01.ct.infn.it
884662792 27744624 n.a grid005.iucc.ac.il
100120000 30 n.a fn2.hpcc.sztaki.hu
14. Set your DPNS_HOST:[taipei01@t-yaim jobs]$ export DPNS_HOST=grid-se.bio.dist.unige.it
Here we use the "grid-se.bio.dist.unige.it" for dpns host, you can feel free to select any listed one, but please remember to modify the domain name in following exercise.For example, if you use "egee016.cnaf.infn.it" for DPNS_HOST, your dpm home should be "/dpm/cnaf.infn.it/home/gilda/"
15. Create your own directory:[taipei01@t-yaim jobs]$ dpns-mkdir /dpm/bio.dist.unige.it/home/gilda/taipeixx
[taipei01@t-yaim jobs]$ dpns-ls /dpm/bio.dist.unige.it/home/gilda/
taipeixx
generated
sh_copy
16. Generate a file:[taipei01@t-yaim jobs]$ echo 'hello!! just a test'> data
17. Put file to SE:[taipei01@t-yaim jobs]$ globus-url-copy file://$HOME/jobs/data gsiftp://grid-se.bio.dist.unige.it/dpm/bio.dist.unige.it/home/gilda/taipeixx/
18. Check file:[taipei01@t-yaim jobs]$ dpns-ls /dpm/bio.dist.unige.it/home/gilda/taipeixx/
data
19. Delete local file:[taipei01@t-yaim jobs]$ rm data
20. Download the file from the Storage Element back to the User Interface:[taipei01@t-yaim jobs]$ globus-url-copy gsiftp://grid-se.bio.dist.unige.it/dpm/bio.dist.unige.it/home/gilda/camp/data file://$HOME/jobs/
[taipei01@t-yaim jobs]$ cat data
hello!! just a test
21. Finish
http://quattor.grid.sinica.edu.tw/UI_practical.html
Topic: YAIM Introduction
- How to Begin
- Install a node
- Configure a node
YAIM = Yet Another Installation Manage
Developed by EGEE SA3 team
This tutorial will guide you through the process of installing a User Interface (UI). In particular we will install a TAR UI which is installed into your own home directory and does not require root permission. The standard UI installation will require root permissions, but will allow all users to access UI commands. For this tutorial we will use the Yaim Tutorial Server. Please refer to the tutorial handout sheet for server and account details.
1. Installing the UI TAR
Install the UI_TAR packages
Configure the UI
2. Preparing your UI environment
Configure the Grid Environment
Install your user certificate
3. Test the Grid jobs submission by your UI
Query the Grid Computing Environment
Simple job submission
Submitting jobs by the Resource Broker
Query the job status
Get your job results
4. Test the Grid data stor by your UI
Query the Grid Storage Environment
Set the DPNS HOST
Create directory on diskpool
Put your file into diskpool
Down file from diskpool
Installing the UI TAR
1. Please make sure the following file is already into your home directory
[taipei01@t-yaim~]$ ls -1F certificates/gLite/glite-UI-3.1.8-0-external.tar.gzglite-UI-3.1.8-0.tar.gzjobs/site-info.def.isgc2008
You can also find the last packages by the following link:The relocatable UI
Notice: The PPS is the pre-production packages, that sometime have several new features, but might also have some bugs, If you are not certainty to use the new features, we recommend you to use the production packages.
To install the UI_TAR, you have only to untar the UI tar ball in your own directory.
2. Untar the tar balls
[taipei01@t-yaim~]$ tar -xzf glite-UI-3.1.8-0-external.tar.gz -C gLite [taipei01@t-yaim~]$ tar -xzf glite-UI-3.1.8-0.tar.gz -C gLite
3. Change the work directory and check the installed files[taipei01@t-yaim gLite]$ cd gLite[taipei01@t-yaim gLite]$ ls -Fclassads/ edg/ external/ globus/ lcg/d-cache/ etc/ glite/ gpt/ tmp.31683/[taipei01@t-yaim gLite]$
4. Run the yaim script to configure the UI_TAR. Here we use the -c "configure command" to configure the UI. The configuration has already been completed for you in site-info.def. Please take a look inside the file to get an idea of how to fill in the yaim parameters. This particular site-info.def allows you to configure a UI that uses the gilda grid infrastructure.[taipei01@t-yaim gLite]$./glite/yaim/bin/yaim -c -s ~/site-info.def.isgc2008 -n UI_TAR
You shouldn't get any error messages.
Notice:Please pay attention to the X509_CERT_DIR of site-info.def variable, here we used "/etc/grid-security/certificates/", if you have not root permission and your computer never installed any certificates, you should modify the X509_CERT_DIR to fit your own directory, the yaim script will go to install all the certificates into this directory.For example:If you have specifyX509_CERT_DIR=$HOME/gLite/external/etc/certificatesin your site-info.def,The yaim script will install all the certificates into $HOME/gLite/external/etc/certificatesHere we were already installed gilda certificates into /etc/grid-security/certificates/ for this tutorial, so please don't modify the X509_CERT_DIR in this practical anyway.
Preparing your UI environment
5. Setup the environment. UI tar package contains two environment scripts that must be sourced in order to use the UI client tools. Here we add these scripts to your startup script, that can help to auto configure the UI environment every time you login the shell.[taipei01@t-yaim gLite]$ echo 'source $HOME/gLite/external/etc/profile.d/grid-env.sh' >>~/.bashrc
Now, you can re-login to the shell or just running following command:[taipei01@t-yaim gLite]$ source ~/.bashrc
6. Put your user certificate and private key into ~/.globus. After you obtain your certificate from your CA, you will need to install them on an UI in the .globus directory.The commands below show you how this is done. Pay close attention to the permission of these certificates.[taipei01@t-yaim gLite]$cd ~[taipei01@t-yaim ~]$ mkdir .globus[taipei01@t-yaim ~]$ chmod 700 .globus[taipei01@t-yaim ~]$ cp certificates/TAIPEIxx-cert.pem .globus/usercert.pem[taipei01@t-yaim ~]$ cp certificates/TAIPEIxx-key.pem .globus/userkey.pem[taipei01@t-yaim ~]$ ls .globus/usercert.pem userkey.pem[taipei01@t-yaim ~]$ chmod 644 .globus/usercert.pem[taipei01@t-yaim ~]$ chmod 400 .globus/userkey.pem
The following step is only for this tutorial that will add gilda vo certificate into your own UI environment [taipei01@t-yaim ~]$ cp /opt/gilda-voms/gilda-voms.ct.infn.it ~/gLite/external/etc/grid-security/vomsdir/
7. Create a proxy certificate.
[taipei01@t-yaim ~]$ voms-proxy-init -voms gildaEnter GRID pass phrase:
Congratulations, you have now configured and setup a Grid User Interface environment that is required to access the Grid resources. We will now test your environment by performing simple job submission and data management task.
Test the grid jobs submission by your UI
8. Check which CE are available on for gilda Virtual Organization with lcg-infosites:[taipei01@t-yaim ~]$ lcg-infosites --vo gilda ce
valor del bdii: grid004.ct.infn.it:2170
#CPU Free Total Jobs Running Waiting ComputingElement
----------------------------------------------------------
11 11 0 0 0 grid-ce.bio.dist.unige.it:2119/jobmanager-lcglsf-gilda
10 10 0 0 0 grid011f.cnaf.infn.it:2119/jobmanager-lcgpbs-gilda
34 34 0 0 0 grid010.ct.infn.it:2119/jobmanager-lcgpbs-long
34 34 0 0 0 grid010.ct.infn.it:2119/jobmanager-lcgpbs-short
34 34 0 0 0 grid010.ct.infn.it:2119/jobmanager-lcgpbs-infinite
56 46 2 2 0 iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-short
56 46 1 1 0 iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-long
56 46 0 0 0 iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-infinite
10 10 0 0 0 grid004.iucc.ac.il:2119/jobmanager-lcgpbs-short
10 10 0 0 0 grid004.iucc.ac.il:2119/jobmanager-lcgpbs-long
10 10 0 0 0 grid004.iucc.ac.il:2119/jobmanager-lcgpbs-infinite
16 16 0 0 0 gn0.hpcc.sztaki.hu:2119/jobmanager-lcgpbs-gilda
9.Test the CE by submitting a simple job that runs directly on the CE itself:[taipei01@t-yaim ~]$ globus-job-run grid011f.cnaf.infn.it /bin/hostname
grid011f.cnaf.infn.it
Make sure you have a correct output which is the hostname of the CE.
10. Test job submission through a resource broker:
Edit a jdl file first[taipei01@t-yaim ~]$ cd jobs/
[taipei01@t-yaim jobs]$ ls
edglog.log testJob.jdl testJob.sh
[taipei01@t-yaim jobs]$ vim testJob.jdl
* Please add following contents in to the testJob.jdl:
Executable = "testJob.sh";
StdOutput = "testJob.out";
StdError = "testJob.err";
InputSandbox = {"./testJob.sh"};
OutputSandbox = {"testJob.out","testJob.err"};
VirtualOrganisation = "gilda";
Feel free to modify the JDL and specify your job requirements.
Edit testJob.sh[taipei01@t-yaim jobs]$ vim testJob.sh
#!/bin/bash
date
hostname
echo "****************************************"
echo "env sort"
echo "****************************************"
env sort
echo "****************************************"
echo "mount"
echo "****************************************"
mount
date
11. Check which CEs match your JDL requirements.
usage: edg-job-list-match
[taipei01@t-yaim jobs]$ edg-job-list-match testJob.jdl
Selected Virtual Organisation name (from JDL): gilda
Connecting to host glite-rb.ct.infn.it, port 7772
***************************************************************************
COMPUTING ELEMENT IDs LIST
The following CE(s) matching your job requirements have been found:
*CEId*
ce-nano-37.to.infn.it:2119/jobmanager-lcgpbs-infinite
ce-nano-37.to.infn.it:2119/jobmanager-lcgpbs-long
ce-nano-37.to.infn.it:2119/jobmanager-lcgpbs-short
dgt01.ui.savba.sk:2119/jobmanager-lcgpbs-infinite
dgt01.ui.savba.sk:2119/jobmanager-lcgpbs-long
dgt01.ui.savba.sk:2119/jobmanager-lcgpbs-short
gilda-ce-01.pd.infn.it:2119/jobmanager-lcgpbs-infinite
gilda-ce-01.pd.infn.it:2119/jobmanager-lcgpbs-long
gilda-ce-01.pd.infn.it:2119/jobmanager-lcgpbs-short
gilda01.ihep.ac.cn:2119/jobmanager-lcgpbs-gilda
gn0.hpcc.sztaki.hu:2119/jobmanager-lcgpbs-gilda
grid010.ct.infn.it:2119/jobmanager-lcgpbs-infinite
grid010.ct.infn.it:2119/jobmanager-lcgpbs-long
grid010.ct.infn.it:2119/jobmanager-lcgpbs-short
iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-short
iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-infinite
iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-long
***************************************************************************
12. Submit the test job:
usage: edg-job-submit
Selected Virtual Organisation name (from JDL): gilda
Connecting to host glite-rb.ct.infn.it, port 7772
Logging to host glite-rb.ct.infn.it, port 9002
*********************************************************************************************
JOB SUBMIT OUTCOME
The job has been successfully submitted to the Network Server.
Use edg-job-status command to check job current status. Your job identifier (edg_jobId) is:
- https://glite-rb.ct.infn.it:9000/js8FEQjvTiVDaIPwQhWqMA
*********************************************************************************************
* Notice, please remember your job id, in this case the job id is:
"https://glite-rb.ct.infn.it:9000/js8FEQjvTiVDaIPwQhWqMA"
Get the job status:
usage: edg-job-status
*************************************************************
BOOKKEEPING INFORMATION:
Status info for the Job : https://glite-rb.ct.infn.it:9000/js8FEQjvTiVDaIPwQhWqMA
Current Status: Scheduled
Status Reason: Job successfully submitted to Globus
Destination: gilda-ce-01.pd.infn.it:2119/jobmanager-lcgpbs-infinite
reached on: Fri Oct 26 13:22:35 2007
*************************************************************
Check the status until the “Current Status” changes to "Done"[taipei01@t-yaim jobs]$ edg-job-status https://glite-rb.ct.infn.it:9000/js8FEQjvTiVDaIPwQhWqMA
*************************************************************
BOOKKEEPING INFORMATION:
Status info for the Job : https://glite-rb.ct.infn.it:9000/js8FEQjvTiVDaIPwQhWqMA
Current Status: Done (Success)
Exit code: 0
Status Reason: Job terminated successfully
Destination: gilda-ce-01.pd.infn.it:2119/jobmanager-lcgpbs-infinite
reached on: Fri Oct 26 13:28:08 2007
*************************************************************
12. Download the job results from the Resource Broker to your User Interface[taipei01@t-yaim jobs]$ edg-job-get-output https://glite-rb.ct.infn.it:9000/js8FEQjvTiVDaIPwQhWqMA
Retrieving files from host: glite-rb.ct.infn.it ( for https://glite-rb.ct.infn.it:9000/js8FEQjvTiVDaIPwQhWqMA )
*********************************************************************************
JOB GET OUTPUT OUTCOME
Output sandbox files for the job:
- https://glite-rb.ct.infn.it:9000/js8FEQjvTiVDaIPwQhWqMA
have been successfully retrieved and stored in the directory:
/home/taipei01/jobOutput/taipe01_js8FEQjvTiVDaIPwQhWqMA
*********************************************************************************
The result output path was define in site-info.def the OUTPUT_STORAGE variable, for this practical, the OUTPUT_STORAGE is $HOME/jobOutput/.
so the result will put into $HOME/jobOutput/taipei01_js8FEQjvTiVDaIPwQhWqMA
Check the result:[taipei01@t-yaim jobs]$ less ~/jobOutput/camp_js8FEQjvTiVDaIPwQhWqMA/testJob.out
Test the Grid data store by your UI
13. Get SE information:[taipei01@t-yaim gLite]$ lcg-infosites --vo gilda se
Avail Space(Kb) Used Space(Kb) Type SEs
----------------------------------------------------------
32350000 210590 n.a grid-se.bio.dist.unige.it
74980000 5153576 n.a egee016.cnaf.infn.it
321010000 50293604 n.a aliserv6.ct.infn.it
50660000 7340000 n.a iceage-se-01.ct.infn.it
884662792 27744624 n.a grid005.iucc.ac.il
100120000 30 n.a fn2.hpcc.sztaki.hu
14. Set your DPNS_HOST:[taipei01@t-yaim jobs]$ export DPNS_HOST=grid-se.bio.dist.unige.it
Here we use the "grid-se.bio.dist.unige.it" for dpns host, you can feel free to select any listed one, but please remember to modify the domain name in following exercise.For example, if you use "egee016.cnaf.infn.it" for DPNS_HOST, your dpm home should be "/dpm/cnaf.infn.it/home/gilda/"
15. Create your own directory:[taipei01@t-yaim jobs]$ dpns-mkdir /dpm/bio.dist.unige.it/home/gilda/taipeixx
[taipei01@t-yaim jobs]$ dpns-ls /dpm/bio.dist.unige.it/home/gilda/
taipeixx
generated
sh_copy
16. Generate a file:[taipei01@t-yaim jobs]$ echo 'hello!! just a test'> data
17. Put file to SE:[taipei01@t-yaim jobs]$ globus-url-copy file://$HOME/jobs/data gsiftp://grid-se.bio.dist.unige.it/dpm/bio.dist.unige.it/home/gilda/taipeixx/
18. Check file:[taipei01@t-yaim jobs]$ dpns-ls /dpm/bio.dist.unige.it/home/gilda/taipeixx/
data
19. Delete local file:[taipei01@t-yaim jobs]$ rm data
20. Download the file from the Storage Element back to the User Interface:[taipei01@t-yaim jobs]$ globus-url-copy gsiftp://grid-se.bio.dist.unige.it/dpm/bio.dist.unige.it/home/gilda/camp/data file://$HOME/jobs/
[taipei01@t-yaim jobs]$ cat data
hello!! just a test
21. Finish
International Symposium on Grid Computing 2008, Taipei, Taiwan: Tutorial on EGEE and gLite
http://homepages.nesc.ac.uk/~mjm/NA3ETF/Usage%20of%20lcg-infosites%20and%20lcg-info.htm
The aim of this practical is to gain an understanding of the Information System by exploring the GILDA grid. This will be done by the use of the "lcg-infosites" and "lcg-info" commands.
Instructions:
1. Use putty
2. Login to the Yaim Tutorial Server:
3. Enter User Name
4. Enter Password
5. Enter Certificate Password
LGC-Infosites
The "lcg-infosites" command is actually just a perl script wrapping a series of LDAP commands and was developed to allow the user to retrieve information on Grid resources for the most common cases.
Before beginning it is worth observing that "lcg-infosites" does not use your VOMS proxy certificate and hence all commands need to include the option "--vo gilda" (note that you can actually query the information for any VO supported by GILDA).
Start by getting a list of the compute elements available in the gilda VO and the state of the associated CPU's:
lcg-infosites --vo gilda ce
Note: Follow the details indicated in the tutorial page with the following link:
http://homepages.nesc.ac.uk/~mjm/NA3ETF/Usage%20of%20lcg-infosites%20and%20lcg-info.htm
Tutorial Page Link: http://indico.cern.ch/conferenceDisplay.py?confId=31719
The aim of this practical is to gain an understanding of the Information System by exploring the GILDA grid. This will be done by the use of the "lcg-infosites" and "lcg-info" commands.
Instructions:
1. Use putty
2. Login to the Yaim Tutorial Server:
3. Enter User Name
4. Enter Password
5. Enter Certificate Password
LGC-Infosites
The "lcg-infosites" command is actually just a perl script wrapping a series of LDAP commands and was developed to allow the user to retrieve information on Grid resources for the most common cases.
Before beginning it is worth observing that "lcg-infosites" does not use your VOMS proxy certificate and hence all commands need to include the option "--vo gilda" (note that you can actually query the information for any VO supported by GILDA).
Start by getting a list of the compute elements available in the gilda VO and the state of the associated CPU's:
lcg-infosites --vo gilda ce
Note: Follow the details indicated in the tutorial page with the following link:
http://homepages.nesc.ac.uk/~mjm/NA3ETF/Usage%20of%20lcg-infosites%20and%20lcg-info.htm
Tutorial Page Link: http://indico.cern.ch/conferenceDisplay.py?confId=31719
International Symposium on Grid Computing 2008, Day 1 (Tutorial), Taipei, Taiwan
April 7, 2008 (Monday)
From the Howard Plaza Hotel a shuttle service took us to the Academia Sinica. Travel time is around 1 hour due to morning rush hour traffic in Taipei.
The tutorial session started a bit late.
Sessions and Notes:
8:30 a.m. Registration
09:00 - 09:30 gLite Introduction & Security
09:30 - 10:00 Information System and Practice Session
10:00 - 10:45 Work Load Management & Simple Job Submission Practical
10:45 - 11:15 Coffee Break
11:15 - 12:00 Data Management
12:00 - 12:45 Yaim & (UI TAR installation) Practical
12:45 - 14:00 LUNCH (Lecture Hall)
14:00 - 14:30 Grid Application Introduction
14:30 - 15:50 Posting an application to the EGEE Grid Practical
15:50 - 16:20 Coffee Break
16:20 - 17:40 Data Grid Services/SRB/SRM & Practical
From the Howard Plaza Hotel a shuttle service took us to the Academia Sinica. Travel time is around 1 hour due to morning rush hour traffic in Taipei.
The tutorial session started a bit late.
Sessions and Notes:
8:30 a.m. Registration
09:00 - 09:30 gLite Introduction & Security
09:30 - 10:00 Information System and Practice Session
10:00 - 10:45 Work Load Management & Simple Job Submission Practical
10:45 - 11:15 Coffee Break
11:15 - 12:00 Data Management
12:00 - 12:45 Yaim & (UI TAR installation) Practical
12:45 - 14:00 LUNCH (Lecture Hall)
14:00 - 14:30 Grid Application Introduction
14:30 - 15:50 Posting an application to the EGEE Grid Practical
15:50 - 16:20 Coffee Break
16:20 - 17:40 Data Grid Services/SRB/SRM & Practical
Another Visit To Taiwan, Day 1
April 6, 2008 (Sunday)
Today I woke up early. My flight to Taipei is at 11:00 a.m. via Cebu Pacific.
There are two of us from Ateneo de Manila University who will participate in the International Symposium on Grid Computing hosted by the Academia Sinica Grid Computing Center in Taipei, Taiwan.
Allan and I arrived at the Ninoy Aquino International Airport (NAIA, Terminal 1) around 9:30 a.m.
The plane was scheduled to leave at 11:00 a.m. but it had a slight delay. We arrived in the Taipei International Airport around 1:30 p.m.
From the the Taipei airport we boarded an airport bus to Howard Plaza Hotel. The fare per person is 145 Taiwan Dollars (which is much cheaper than a taxi service from the airport -- around 2,000 Taiwan Dollars).
Related Links:
1. ISGC 2008: http://event.twgrid.org/isgc2008/
2. Holiday Plaza Hotel: http://www.hoteltravel.com/taiwan/taipei/howard_plaza_taipei.htm
Raffy
4/6/08
Today I woke up early. My flight to Taipei is at 11:00 a.m. via Cebu Pacific.
There are two of us from Ateneo de Manila University who will participate in the International Symposium on Grid Computing hosted by the Academia Sinica Grid Computing Center in Taipei, Taiwan.
Allan and I arrived at the Ninoy Aquino International Airport (NAIA, Terminal 1) around 9:30 a.m.
The plane was scheduled to leave at 11:00 a.m. but it had a slight delay. We arrived in the Taipei International Airport around 1:30 p.m.
From the the Taipei airport we boarded an airport bus to Howard Plaza Hotel. The fare per person is 145 Taiwan Dollars (which is much cheaper than a taxi service from the airport -- around 2,000 Taiwan Dollars).
Related Links:
1. ISGC 2008: http://event.twgrid.org/isgc2008/
2. Holiday Plaza Hotel: http://www.hoteltravel.com/taiwan/taipei/howard_plaza_taipei.htm
Raffy
4/6/08
Another Visit To Taiwan (April 6 - 14, 2008)
On April 6 to 14, 2008 I will be in Taiwan for the following meetings/conferences:
7 April - EGEE Tutorial
8 April - EUAsiaGrid Kick-off Meeting
9 April - EUAsiaGrid Kick-off Meeting
9-11 April - International Symposium on Grid Computing
Link to the ISGC 2008 Symposium: http://event.twgrid.org/isgc2008/
7 April - EGEE Tutorial
8 April - EUAsiaGrid Kick-off Meeting
9 April - EUAsiaGrid Kick-off Meeting
9-11 April - International Symposium on Grid Computing
Link to the ISGC 2008 Symposium: http://event.twgrid.org/isgc2008/
Wednesday, April 2, 2008
Announcement Re: Draft Resolution of PSHS Alumni Association Board of Directors
ANNOUNCEMENT:
Please be informed that the Board of Directors of the Philippine Science High School Alumni Association (PSHS AA) is now finalizing a resolution pertaining to a recent decision of the PSHS Board of Trustees.
What I posted previously in my blog (http://raffysaldana.blogspot.com) is just a DRAFT RESOLUTION, and I have deleted that particular entry in my blog already.
I am sorry for any inconvenience that it may have caused.
The message that I would like to convey is that the Board of Directors of PSHS AA is doing something about the issue.
Thank you very much.
Raffy Saldaña
PSHS AA/ PSHS NAA
Please be informed that the Board of Directors of the Philippine Science High School Alumni Association (PSHS AA) is now finalizing a resolution pertaining to a recent decision of the PSHS Board of Trustees.
What I posted previously in my blog (http://raffysaldana.blogspot.com) is just a DRAFT RESOLUTION, and I have deleted that particular entry in my blog already.
I am sorry for any inconvenience that it may have caused.
The message that I would like to convey is that the Board of Directors of PSHS AA is doing something about the issue.
Thank you very much.
Raffy Saldaña
PSHS AA/ PSHS NAA
Tuesday, April 1, 2008
Graduation Photos in Bacolod City
Note: I was the Graduation Speaker of IMPACT (Institute of Manpower and Computer Technology), a school owned by the Tolentino family, during the Commencement Exercises held last March 31, 2008 in Bacolod City, Negros Occidental.
Click on the following link to view more photos during my recent trip in Bacolod City:
Subscribe to:
Posts (Atom)
Posts
