Sunday, April 6, 2008

YAIM Overview (Tutorial Session, International Symposium On Grid Computing 2008, Taipei, Taiwan)

NOTES on the EGEE Tutorial, 7 April 2008, Academia Sinica, Taipei, Taiwan.

Topic: YAIM Introduction

  • How to Begin

  • Install a node

  • Configure a node

YAIM = Yet Another Installation Manage
Developed by EGEE SA3 team

This tutorial will guide you through the process of installing a User Interface (UI). In particular we will install a TAR UI which is installed into your own home directory and does not require root permission. The standard UI installation will require root permissions, but will allow all users to access UI commands. For this tutorial we will use the Yaim Tutorial Server. Please refer to the tutorial handout sheet for server and account details.

1. Installing the UI TAR
Install the UI_TAR packages
Configure the UI

2. Preparing your UI environment
Configure the Grid Environment
Install your user certificate

3. Test the Grid jobs submission by your UI
Query the Grid Computing Environment
Simple job submission
Submitting jobs by the Resource Broker
Query the job status
Get your job results

4. Test the Grid data stor by your UI
Query the Grid Storage Environment
Create directory on diskpool
Put your file into diskpool
Down file from diskpool

Installing the UI TAR

1. Please make sure the following file is already into your home directory
[taipei01@t-yaim~]$ ls -1F certificates/gLite/glite-UI-3.1.8-0-external.tar.gzglite-UI-3.1.8-0.tar.gzjobs/site-info.def.isgc2008
You can also find the last packages by the following link:The relocatable UI
Notice: The PPS is the pre-production packages, that sometime have several new features, but might also have some bugs, If you are not certainty to use the new features, we recommend you to use the production packages.
To install the UI_TAR, you have only to untar the UI tar ball in your own directory.
2. Untar the tar balls
[taipei01@t-yaim~]$ tar -xzf glite-UI-3.1.8-0-external.tar.gz -C gLite [taipei01@t-yaim~]$ tar -xzf glite-UI-3.1.8-0.tar.gz -C gLite
3. Change the work directory and check the installed files[taipei01@t-yaim gLite]$ cd gLite[taipei01@t-yaim gLite]$ ls -Fclassads/ edg/ external/ globus/ lcg/d-cache/ etc/ glite/ gpt/ tmp.31683/[taipei01@t-yaim gLite]$
4. Run the yaim script to configure the UI_TAR. Here we use the -c "configure command" to configure the UI. The configuration has already been completed for you in site-info.def. Please take a look inside the file to get an idea of how to fill in the yaim parameters. This particular site-info.def allows you to configure a UI that uses the gilda grid infrastructure.[taipei01@t-yaim gLite]$./glite/yaim/bin/yaim -c -s ~/site-info.def.isgc2008 -n UI_TAR
You shouldn't get any error messages.
Notice:Please pay attention to the X509_CERT_DIR of site-info.def variable, here we used "/etc/grid-security/certificates/", if you have not root permission and your computer never installed any certificates, you should modify the X509_CERT_DIR to fit your own directory, the yaim script will go to install all the certificates into this directory.For example:If you have specifyX509_CERT_DIR=$HOME/gLite/external/etc/certificatesin your site-info.def,The yaim script will install all the certificates into $HOME/gLite/external/etc/certificatesHere we were already installed gilda certificates into /etc/grid-security/certificates/ for this tutorial, so please don't modify the X509_CERT_DIR in this practical anyway.
Preparing your UI environment
5. Setup the environment. UI tar package contains two environment scripts that must be sourced in order to use the UI client tools. Here we add these scripts to your startup script, that can help to auto configure the UI environment every time you login the shell.[taipei01@t-yaim gLite]$ echo 'source $HOME/gLite/external/etc/profile.d/' >>~/.bashrc
Now, you can re-login to the shell or just running following command:[taipei01@t-yaim gLite]$ source ~/.bashrc
6. Put your user certificate and private key into ~/.globus. After you obtain your certificate from your CA, you will need to install them on an UI in the .globus directory.The commands below show you how this is done. Pay close attention to the permission of these certificates.[taipei01@t-yaim gLite]$cd ~[taipei01@t-yaim ~]$ mkdir .globus[taipei01@t-yaim ~]$ chmod 700 .globus[taipei01@t-yaim ~]$ cp certificates/TAIPEIxx-cert.pem .globus/usercert.pem[taipei01@t-yaim ~]$ cp certificates/TAIPEIxx-key.pem .globus/userkey.pem[taipei01@t-yaim ~]$ ls .globus/usercert.pem userkey.pem[taipei01@t-yaim ~]$ chmod 644 .globus/usercert.pem[taipei01@t-yaim ~]$ chmod 400 .globus/userkey.pem
The following step is only for this tutorial that will add gilda vo certificate into your own UI environment [taipei01@t-yaim ~]$ cp /opt/gilda-voms/ ~/gLite/external/etc/grid-security/vomsdir/
7. Create a proxy certificate.
[taipei01@t-yaim ~]$ voms-proxy-init -voms gildaEnter GRID pass phrase: Your identity: /C=IT/O=GILDA/OU=Personal Certificate/L=TAIPEI/CN=TAIPEI01/ temporary proxy ...................................... DoneContacting [/C=IT/O=INFN/OU=Host/L=Catania/] "gilda" DoneCreating proxy ........................................................ Done Your proxy is valid until Fri Mar 26 21:56:48 2008
Congratulations, you have now configured and setup a Grid User Interface environment that is required to access the Grid resources. We will now test your environment by performing simple job submission and data management task.
Test the grid jobs submission by your UI
8. Check which CE are available on for gilda Virtual Organization with lcg-infosites:[taipei01@t-yaim ~]$ lcg-infosites --vo gilda ce
valor del bdii:
#CPU Free Total Jobs Running Waiting ComputingElement
11 11 0 0 0
10 10 0 0 0
34 34 0 0 0
34 34 0 0 0
34 34 0 0 0
56 46 2 2 0
56 46 1 1 0
56 46 0 0 0
10 10 0 0 0
10 10 0 0 0
10 10 0 0 0
16 16 0 0 0
9.Test the CE by submitting a simple job that runs directly on the CE itself:[taipei01@t-yaim ~]$ globus-job-run /bin/hostname
Make sure you have a correct output which is the hostname of the CE.
10. Test job submission through a resource broker:
Edit a jdl file first[taipei01@t-yaim ~]$ cd jobs/
[taipei01@t-yaim jobs]$ ls
edglog.log testJob.jdl
[taipei01@t-yaim jobs]$ vim testJob.jdl
* Please add following contents in to the testJob.jdl:
Executable = "";
StdOutput = "testJob.out";
StdError = "testJob.err";
InputSandbox = {"./"};
OutputSandbox = {"testJob.out","testJob.err"};
VirtualOrganisation = "gilda";
Feel free to modify the JDL and specify your job requirements.
Edit[taipei01@t-yaim jobs]$ vim
echo "****************************************"
echo "env sort"
echo "****************************************"
env sort
echo "****************************************"
echo "mount"
echo "****************************************"
11. Check which CEs match your JDL requirements.
usage: edg-job-list-match
[taipei01@t-yaim jobs]$ edg-job-list-match testJob.jdl
Selected Virtual Organisation name (from JDL): gilda
Connecting to host, port 7772
The following CE(s) matching your job requirements have been found:
12. Submit the test job:
usage: edg-job-submit [taipei01@t-yaim jobs]$ edg-job-submit testJob.jdl
Selected Virtual Organisation name (from JDL): gilda
Connecting to host, port 7772
Logging to host, port 9002
The job has been successfully submitted to the Network Server.
Use edg-job-status command to check job current status. Your job identifier (edg_jobId) is:
* Notice, please remember your job id, in this case the job id is:
Get the job status:
usage: edg-job-status [taipei01@t-yaim jobs]$ edg-job-status
Status info for the Job :
Current Status: Scheduled
Status Reason: Job successfully submitted to Globus
reached on: Fri Oct 26 13:22:35 2007
Check the status until the “Current Status” changes to "Done"[taipei01@t-yaim jobs]$ edg-job-status
Status info for the Job :
Current Status: Done (Success)
Exit code: 0
Status Reason: Job terminated successfully
reached on: Fri Oct 26 13:28:08 2007
12. Download the job results from the Resource Broker to your User Interface[taipei01@t-yaim jobs]$ edg-job-get-output
Retrieving files from host: ( for )
Output sandbox files for the job:
have been successfully retrieved and stored in the directory:
The result output path was define in site-info.def the OUTPUT_STORAGE variable, for this practical, the OUTPUT_STORAGE is $HOME/jobOutput/.
so the result will put into $HOME/jobOutput/taipei01_js8FEQjvTiVDaIPwQhWqMA
Check the result:[taipei01@t-yaim jobs]$ less ~/jobOutput/camp_js8FEQjvTiVDaIPwQhWqMA/testJob.out
Test the Grid data store by your UI
13. Get SE information:[taipei01@t-yaim gLite]$ lcg-infosites --vo gilda se
Avail Space(Kb) Used Space(Kb) Type SEs
32350000 210590 n.a
74980000 5153576 n.a
321010000 50293604 n.a
50660000 7340000 n.a
884662792 27744624 n.a
100120000 30 n.a
14. Set your DPNS_HOST:[taipei01@t-yaim jobs]$ export
Here we use the "" for dpns host, you can feel free to select any listed one, but please remember to modify the domain name in following exercise.For example, if you use "" for DPNS_HOST, your dpm home should be "/dpm/"
15. Create your own directory:[taipei01@t-yaim jobs]$ dpns-mkdir /dpm/
[taipei01@t-yaim jobs]$ dpns-ls /dpm/
16. Generate a file:[taipei01@t-yaim jobs]$ echo 'hello!! just a test'> data
17. Put file to SE:[taipei01@t-yaim jobs]$ globus-url-copy file://$HOME/jobs/data gsi
18. Check file:[taipei01@t-yaim jobs]$ dpns-ls /dpm/
19. Delete local file:[taipei01@t-yaim jobs]$ rm data
20. Download the file from the Storage Element back to the User Interface:[taipei01@t-yaim jobs]$ globus-url-copy gsi file://$HOME/jobs/
[taipei01@t-yaim jobs]$ cat data
hello!! just a test
21. Finish

No comments: