Showing posts with label Tutorial. Show all posts
Showing posts with label Tutorial. Show all posts

Sunday, March 7, 2010

Social Simulation Tutorial at ISGC 2010 , Taiwan (March 2010)

Link to blog post: http://raffysaldana.blogspot.com/2010/03/social-simulation-tutorial-at-isgc-2010.html

Link: https://e-research.cs.st-andrews.ac.uk/site/bin/view/SocialSimulation/ISGC2010

TUTORIAL ON 'SOCIAL SIMULATIONS'
International Symposium on Grid Computing (ISGC 2010)
7 March 2010
Academia Sinica, Taipei, Taiwan

Organizers:

Alex Voss, School of Computer Science, University of St Andrews, UK
Rob Procter, Manchester e-Research Centre, University of Manchester, UK
Andy Turner, School of Geography, University of Leeds, UK

If you have any questions about this tutorial please send an email to Alex Voss at avoss@cs.st-andrews.ac.uk.


Introduction

Agent-Based Social Simulation is a relatively new social science research method. It allows social phenomena to be investigated by studying dynamic processes in computational models of populations. Social simulation complements existing social science research methods and builds on empirical findings and theories of social phenomena. Applications of social simulation range from the study of basic demographic processes to uses in a range of interdisciplinary research areas such as public health, urban studies or criminology, to name a few examples.

Social scientists as well as other professionals such as policy makers have used various methods in the past to model social phenomena and generate predictions about them. At its simplest, one can extrapolate from empirical data by fitting functions that approximate the data gathered. In the past, a key limitation for such extrapolations was that the underlying model was a static one, defined in terms of mathematical formulas that could be manipulated manually. The necessary simplifying assumptions often meant that the applicability and the predictive power of these models was limited. The advent of computers and their increasing capacity has made it possible to build models that consist of representations of social agents and their environment and to simulate their behaviour and interactions.

Such agent based models (ABMs) are still limited by the capacity and processing speed of today's computer systems. It is now possible to simulate relatively large populations with a reasonable number of variables even on a desktop computer. However, to study social phenomena it is often necessary to run multiple simulations that vary specific variables in order to study their impact. Such parameter studies can be relatively easily distributed across a number of computers using grid technologies. Each instance of a simulation run with a specific set of parameters can be sent to a different machine in a grid, with all the machines working in parallel to produce an overall result.

Aims

This tutorial aims to provide an introduction to the principles behind social simulation and to demonstrate these principles by teaching participants how to develop a social simulation model and running a simulation using the Repast Simphony Toolkit. Building and running this model will involve the creation of a model population on the basis of freely available social science datasets. In order to investigate hypotheses or to test the impact of policy interventions of real-world events, it is often necessary to run ensembles of simulation jobs. Running such ensembles is most effectively done on high throughput computing infrastructures such as grids or clouds. Participants will learn about grids and clouds and will start using them by submitting ensembles of the models on real-world infrastructures.

Program

Time Session
09:00 Registration
09:30 Welcome, What is social simulation? What is Agent-Based Modelling (ABM)?
10:00 The RePast Simphony Toolkit, an example model
10:30 Coffee break
11:00 Practical I -
Installing RePast and Running Example Models
Installing and Running the Tutorial Model
Adding New Charts
Running a Batch Job
12:00 Population Reconstruction (Andy Turner)
12:30 Lunch
14:00 Infrastructures for Social Simulation (Rob Procter)
14:30 Introduction to Grids and Cloud Computing
15:00 Coffee break
15:30 Practical II - running model ensembles on grids and clouds
Installing g-Eclipse
Running the Tutorial Model on the Grid
16:45 Closing remarks
17:00 End

For more information, contact:

Dr. Rafael P. Saldaña
Coordinator for ADMU
EUAsiaGrid Project
(http://www.euasiagrid.org)
School of Science and Engineering
Ateneo de Manila University
Philippines

E-mail: rsaldana@ateneo.edu




Sunday, April 6, 2008

YAIM Overview (Tutorial Session, International Symposium On Grid Computing 2008, Taipei, Taiwan)


NOTES on the EGEE Tutorial, 7 April 2008, Academia Sinica, Taipei, Taiwan.


http://quattor.grid.sinica.edu.tw/UI_practical.html

Topic: YAIM Introduction


  • How to Begin

  • Install a node

  • Configure a node


YAIM = Yet Another Installation Manage
Developed by EGEE SA3 team


This tutorial will guide you through the process of installing a User Interface (UI). In particular we will install a TAR UI which is installed into your own home directory and does not require root permission. The standard UI installation will require root permissions, but will allow all users to access UI commands. For this tutorial we will use the Yaim Tutorial Server. Please refer to the tutorial handout sheet for server and account details.


1. Installing the UI TAR
Install the UI_TAR packages
Configure the UI

2. Preparing your UI environment
Configure the Grid Environment
Install your user certificate

3. Test the Grid jobs submission by your UI
Query the Grid Computing Environment
Simple job submission
Submitting jobs by the Resource Broker
Query the job status
Get your job results

4. Test the Grid data stor by your UI
Query the Grid Storage Environment
Set the DPNS HOST
Create directory on diskpool
Put your file into diskpool
Down file from diskpool

Installing the UI TAR

1. Please make sure the following file is already into your home directory
[taipei01@t-yaim~]$ ls -1F certificates/gLite/glite-UI-3.1.8-0-external.tar.gzglite-UI-3.1.8-0.tar.gzjobs/site-info.def.isgc2008
You can also find the last packages by the following link:The relocatable UI
Notice: The PPS is the pre-production packages, that sometime have several new features, but might also have some bugs, If you are not certainty to use the new features, we recommend you to use the production packages.
To install the UI_TAR, you have only to untar the UI tar ball in your own directory.
2. Untar the tar balls
[taipei01@t-yaim~]$ tar -xzf glite-UI-3.1.8-0-external.tar.gz -C gLite [taipei01@t-yaim~]$ tar -xzf glite-UI-3.1.8-0.tar.gz -C gLite
3. Change the work directory and check the installed files[taipei01@t-yaim gLite]$ cd gLite[taipei01@t-yaim gLite]$ ls -Fclassads/ edg/ external/ globus/ lcg/d-cache/ etc/ glite/ gpt/ tmp.31683/[taipei01@t-yaim gLite]$
4. Run the yaim script to configure the UI_TAR. Here we use the -c "configure command" to configure the UI. The configuration has already been completed for you in site-info.def. Please take a look inside the file to get an idea of how to fill in the yaim parameters. This particular site-info.def allows you to configure a UI that uses the gilda grid infrastructure.[taipei01@t-yaim gLite]$./glite/yaim/bin/yaim -c -s ~/site-info.def.isgc2008 -n UI_TAR
You shouldn't get any error messages.
Notice:Please pay attention to the X509_CERT_DIR of site-info.def variable, here we used "/etc/grid-security/certificates/", if you have not root permission and your computer never installed any certificates, you should modify the X509_CERT_DIR to fit your own directory, the yaim script will go to install all the certificates into this directory.For example:If you have specifyX509_CERT_DIR=$HOME/gLite/external/etc/certificatesin your site-info.def,The yaim script will install all the certificates into $HOME/gLite/external/etc/certificatesHere we were already installed gilda certificates into /etc/grid-security/certificates/ for this tutorial, so please don't modify the X509_CERT_DIR in this practical anyway.
Preparing your UI environment
5. Setup the environment. UI tar package contains two environment scripts that must be sourced in order to use the UI client tools. Here we add these scripts to your startup script, that can help to auto configure the UI environment every time you login the shell.[taipei01@t-yaim gLite]$ echo 'source $HOME/gLite/external/etc/profile.d/grid-env.sh' >>~/.bashrc
Now, you can re-login to the shell or just running following command:[taipei01@t-yaim gLite]$ source ~/.bashrc
6. Put your user certificate and private key into ~/.globus. After you obtain your certificate from your CA, you will need to install them on an UI in the .globus directory.The commands below show you how this is done. Pay close attention to the permission of these certificates.[taipei01@t-yaim gLite]$cd ~[taipei01@t-yaim ~]$ mkdir .globus[taipei01@t-yaim ~]$ chmod 700 .globus[taipei01@t-yaim ~]$ cp certificates/TAIPEIxx-cert.pem .globus/usercert.pem[taipei01@t-yaim ~]$ cp certificates/TAIPEIxx-key.pem .globus/userkey.pem[taipei01@t-yaim ~]$ ls .globus/usercert.pem userkey.pem[taipei01@t-yaim ~]$ chmod 644 .globus/usercert.pem[taipei01@t-yaim ~]$ chmod 400 .globus/userkey.pem
The following step is only for this tutorial that will add gilda vo certificate into your own UI environment [taipei01@t-yaim ~]$ cp /opt/gilda-voms/gilda-voms.ct.infn.it ~/gLite/external/etc/grid-security/vomsdir/
7. Create a proxy certificate.
[taipei01@t-yaim ~]$ voms-proxy-init -voms gildaEnter GRID pass phrase: Your identity: /C=IT/O=GILDA/OU=Personal Certificate/L=TAIPEI/CN=TAIPEI01/Email=jennsu@ccweb.sinica.edu.twCreating temporary proxy ...................................... DoneContacting voms.ct.infn.it:15001 [/C=IT/O=INFN/OU=Host/L=Catania/CN=voms.ct.infn.it] "gilda" DoneCreating proxy ........................................................ Done Your proxy is valid until Fri Mar 26 21:56:48 2008
Congratulations, you have now configured and setup a Grid User Interface environment that is required to access the Grid resources. We will now test your environment by performing simple job submission and data management task.
Test the grid jobs submission by your UI
8. Check which CE are available on for gilda Virtual Organization with lcg-infosites:[taipei01@t-yaim ~]$ lcg-infosites --vo gilda ce
valor del bdii: grid004.ct.infn.it:2170
#CPU Free Total Jobs Running Waiting ComputingElement
----------------------------------------------------------
11 11 0 0 0 grid-ce.bio.dist.unige.it:2119/jobmanager-lcglsf-gilda
10 10 0 0 0 grid011f.cnaf.infn.it:2119/jobmanager-lcgpbs-gilda
34 34 0 0 0 grid010.ct.infn.it:2119/jobmanager-lcgpbs-long
34 34 0 0 0 grid010.ct.infn.it:2119/jobmanager-lcgpbs-short
34 34 0 0 0 grid010.ct.infn.it:2119/jobmanager-lcgpbs-infinite
56 46 2 2 0 iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-short
56 46 1 1 0 iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-long
56 46 0 0 0 iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-infinite
10 10 0 0 0 grid004.iucc.ac.il:2119/jobmanager-lcgpbs-short
10 10 0 0 0 grid004.iucc.ac.il:2119/jobmanager-lcgpbs-long
10 10 0 0 0 grid004.iucc.ac.il:2119/jobmanager-lcgpbs-infinite
16 16 0 0 0 gn0.hpcc.sztaki.hu:2119/jobmanager-lcgpbs-gilda
9.Test the CE by submitting a simple job that runs directly on the CE itself:[taipei01@t-yaim ~]$ globus-job-run grid011f.cnaf.infn.it /bin/hostname
grid011f.cnaf.infn.it
Make sure you have a correct output which is the hostname of the CE.
10. Test job submission through a resource broker:
Edit a jdl file first[taipei01@t-yaim ~]$ cd jobs/
[taipei01@t-yaim jobs]$ ls
edglog.log testJob.jdl testJob.sh
[taipei01@t-yaim jobs]$ vim testJob.jdl
* Please add following contents in to the testJob.jdl:
Executable = "testJob.sh";
StdOutput = "testJob.out";
StdError = "testJob.err";
InputSandbox = {"./testJob.sh"};
OutputSandbox = {"testJob.out","testJob.err"};
VirtualOrganisation = "gilda";
Feel free to modify the JDL and specify your job requirements.
Edit testJob.sh[taipei01@t-yaim jobs]$ vim testJob.sh
#!/bin/bash
date
hostname
echo "****************************************"
echo "env sort"
echo "****************************************"
env sort
echo "****************************************"
echo "mount"
echo "****************************************"
mount
date
11. Check which CEs match your JDL requirements.
usage: edg-job-list-match
[taipei01@t-yaim jobs]$ edg-job-list-match testJob.jdl
Selected Virtual Organisation name (from JDL): gilda
Connecting to host glite-rb.ct.infn.it, port 7772
***************************************************************************
COMPUTING ELEMENT IDs LIST
The following CE(s) matching your job requirements have been found:
*CEId*
ce-nano-37.to.infn.it:2119/jobmanager-lcgpbs-infinite
ce-nano-37.to.infn.it:2119/jobmanager-lcgpbs-long
ce-nano-37.to.infn.it:2119/jobmanager-lcgpbs-short
dgt01.ui.savba.sk:2119/jobmanager-lcgpbs-infinite
dgt01.ui.savba.sk:2119/jobmanager-lcgpbs-long
dgt01.ui.savba.sk:2119/jobmanager-lcgpbs-short
gilda-ce-01.pd.infn.it:2119/jobmanager-lcgpbs-infinite
gilda-ce-01.pd.infn.it:2119/jobmanager-lcgpbs-long
gilda-ce-01.pd.infn.it:2119/jobmanager-lcgpbs-short
gilda01.ihep.ac.cn:2119/jobmanager-lcgpbs-gilda
gn0.hpcc.sztaki.hu:2119/jobmanager-lcgpbs-gilda
grid010.ct.infn.it:2119/jobmanager-lcgpbs-infinite
grid010.ct.infn.it:2119/jobmanager-lcgpbs-long
grid010.ct.infn.it:2119/jobmanager-lcgpbs-short
iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-short
iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-infinite
iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-long
***************************************************************************
12. Submit the test job:
usage: edg-job-submit [taipei01@t-yaim jobs]$ edg-job-submit testJob.jdl
Selected Virtual Organisation name (from JDL): gilda
Connecting to host glite-rb.ct.infn.it, port 7772
Logging to host glite-rb.ct.infn.it, port 9002
*********************************************************************************************
JOB SUBMIT OUTCOME
The job has been successfully submitted to the Network Server.
Use edg-job-status command to check job current status. Your job identifier (edg_jobId) is:
- https://glite-rb.ct.infn.it:9000/js8FEQjvTiVDaIPwQhWqMA
*********************************************************************************************
* Notice, please remember your job id, in this case the job id is:
"https://glite-rb.ct.infn.it:9000/js8FEQjvTiVDaIPwQhWqMA"
Get the job status:
usage: edg-job-status [taipei01@t-yaim jobs]$ edg-job-status https://glite-rb.ct.infn.it:9000/js8FEQjvTiVDaIPwQhWqMA
*************************************************************
BOOKKEEPING INFORMATION:
Status info for the Job : https://glite-rb.ct.infn.it:9000/js8FEQjvTiVDaIPwQhWqMA
Current Status: Scheduled
Status Reason: Job successfully submitted to Globus
Destination: gilda-ce-01.pd.infn.it:2119/jobmanager-lcgpbs-infinite
reached on: Fri Oct 26 13:22:35 2007
*************************************************************
Check the status until the “Current Status” changes to "Done"[taipei01@t-yaim jobs]$ edg-job-status https://glite-rb.ct.infn.it:9000/js8FEQjvTiVDaIPwQhWqMA
*************************************************************
BOOKKEEPING INFORMATION:
Status info for the Job : https://glite-rb.ct.infn.it:9000/js8FEQjvTiVDaIPwQhWqMA
Current Status: Done (Success)
Exit code: 0
Status Reason: Job terminated successfully
Destination: gilda-ce-01.pd.infn.it:2119/jobmanager-lcgpbs-infinite
reached on: Fri Oct 26 13:28:08 2007
*************************************************************
12. Download the job results from the Resource Broker to your User Interface[taipei01@t-yaim jobs]$ edg-job-get-output https://glite-rb.ct.infn.it:9000/js8FEQjvTiVDaIPwQhWqMA
Retrieving files from host: glite-rb.ct.infn.it ( for https://glite-rb.ct.infn.it:9000/js8FEQjvTiVDaIPwQhWqMA )
*********************************************************************************
JOB GET OUTPUT OUTCOME
Output sandbox files for the job:
- https://glite-rb.ct.infn.it:9000/js8FEQjvTiVDaIPwQhWqMA
have been successfully retrieved and stored in the directory:
/home/taipei01/jobOutput/taipe01_js8FEQjvTiVDaIPwQhWqMA
*********************************************************************************
The result output path was define in site-info.def the OUTPUT_STORAGE variable, for this practical, the OUTPUT_STORAGE is $HOME/jobOutput/.
so the result will put into $HOME/jobOutput/taipei01_js8FEQjvTiVDaIPwQhWqMA
Check the result:[taipei01@t-yaim jobs]$ less ~/jobOutput/camp_js8FEQjvTiVDaIPwQhWqMA/testJob.out
Test the Grid data store by your UI
13. Get SE information:[taipei01@t-yaim gLite]$ lcg-infosites --vo gilda se
Avail Space(Kb) Used Space(Kb) Type SEs
----------------------------------------------------------
32350000 210590 n.a grid-se.bio.dist.unige.it
74980000 5153576 n.a egee016.cnaf.infn.it
321010000 50293604 n.a aliserv6.ct.infn.it
50660000 7340000 n.a iceage-se-01.ct.infn.it
884662792 27744624 n.a grid005.iucc.ac.il
100120000 30 n.a fn2.hpcc.sztaki.hu
14. Set your DPNS_HOST:[taipei01@t-yaim jobs]$ export DPNS_HOST=grid-se.bio.dist.unige.it
Here we use the "grid-se.bio.dist.unige.it" for dpns host, you can feel free to select any listed one, but please remember to modify the domain name in following exercise.For example, if you use "egee016.cnaf.infn.it" for DPNS_HOST, your dpm home should be "/dpm/cnaf.infn.it/home/gilda/"
15. Create your own directory:[taipei01@t-yaim jobs]$ dpns-mkdir /dpm/bio.dist.unige.it/home/gilda/taipeixx
[taipei01@t-yaim jobs]$ dpns-ls /dpm/bio.dist.unige.it/home/gilda/
taipeixx
generated
sh_copy
16. Generate a file:[taipei01@t-yaim jobs]$ echo 'hello!! just a test'> data
17. Put file to SE:[taipei01@t-yaim jobs]$ globus-url-copy file://$HOME/jobs/data gsiftp://grid-se.bio.dist.unige.it/dpm/bio.dist.unige.it/home/gilda/taipeixx/
18. Check file:[taipei01@t-yaim jobs]$ dpns-ls /dpm/bio.dist.unige.it/home/gilda/taipeixx/
data
19. Delete local file:[taipei01@t-yaim jobs]$ rm data
20. Download the file from the Storage Element back to the User Interface:[taipei01@t-yaim jobs]$ globus-url-copy gsiftp://grid-se.bio.dist.unige.it/dpm/bio.dist.unige.it/home/gilda/camp/data file://$HOME/jobs/
[taipei01@t-yaim jobs]$ cat data
hello!! just a test
21. Finish