OGSA Teleconference February 05, 2004 ===================================== * Participants Hariharan Balakrishnan (IBM) Dave Berry (NeSC) Michael Fraenkel (IBM) Andrew Grimshaw (UVa) Hiro Kishimoto (Fujitsu) Takashi Kojo (NEC) Fred Maciel (Hitachi) Hrabri Rajic (Intel) Andreas Savva (Fujitsu) Frank Siebenlist (ANL) Ravi Subramaniam (Intel) Jem Treadwell (HP) Jeffrin Von-Reich (HP) Apologies: Bill Horn, Dave Snelling Minutes: Andreas Savva * Summary Action: Hiro to update F2F agenda (incl. bridge info) and send out before F2F. -Done Action: Andrew and Ravi to draw up PE requirements for CDDLM group. Action: Andrew to make a revised PE figure for next week's F2F. Action: Kojo to draw up similar picture to the PE one for CDDLM services for the F2F. Conclusion: Keep Job and Job Wrapper i/f separate. PE Target for GGF10 (Berlin): - Discuss PE big picture. Also sketch out some i/fs and scenarios. - Do not go into details of how agreement is used or can be used. * Early discussion ** Teleconf minutes Feb 2 - approved. ** Feb. F2F meeting - Andrew suggested carpooling to SDSC since parking is complex. E.g., meet at the Sealodge 20 minutes before the session starts. Monday session starts 8:00 so meet by 7:40am. - Andrew will be at the Sealodge from Saturday night. - The conference room is the Auditorium on the main floor. - No phone bridge arrangement yet. Hiro is working with Ian to arrange it. - PE discussion - Currently allocated 2 hours. - Given past experience and the target (get agreement on basic model, tackle some new stuff) two hours may not be enough. - Andrew prefers 2 hours on Day 1, followed by 1 hour on Day 2. Action: Hiro to update F2F agenda (incl. bridge info) and send out before F2F. -DONE * Program Execution Discussion ** Documents used available in http://www-unix.gridforum.org/mail_archive/ogsa-wg/2004/02/msg00008.html ** Status review Using [Program Execution Services Figure] http://www-unix.gridforum.org/mail_archive/ogsa-wg/2004/02/pdf00000.pdf - PE work was done before the introduction of WSRF. - Current definitions should be thought of as porttype/interface definitions and not (necessarily) as entities that exist at separate physical locations. - Different compositions are possible. E.g., an Execution Planner Service may include its own Candidate Set Generator and Information Service. - Andrew added a couple of entities since the last PE discussion. - Service Factories: Used for instantiation of binaries; what is needed to run a service. - Monitoring services - (Ravi) It is not in the figure but there was also discussion on the job document aspect of a job - (Andrew) A job document set; describing a number of different aspects of the job; initial submission, how many times it has been restarted, etc... ** Job and Job wrapper - (Andrew) The difference between Job and Job Wrapper is not clear. Might want to merge the two at a later point. - Job wrapper is used for low level job control. It only becomes available after the job starts. It can be used to manipulate things like stdio, stdout, stderr, etc. - Job is available even before the job starts running. It can be used to query the status of the job, e.g., is it running or not. - (Ravi) Operations on the job get passed to the Job Wrapper if the Job Wrapper is instantiated. A Job Wrapper is mostly an internal i/f. But it may be visible in some implementations - (A longer name for it could be "job runtime management i/f") - Conclusion: Keep Job and Job Wrapper i/f separate. ** Container & Executable Services - (Andrew) A Container must be able to get an executable binary or know where it is located; e.g., 'grid loader' functionality, or deployment. - On the Figure these services are called "Executable Services" - (Hiro) Deployment Services is a better term. - (Ravi) Also require environment configuration. In a broader sense these could be Provisioning Services. - Provisioning implies a higher level activity. The "Executable Services" do the low level work of setting up containers. So they seem to map nicely to the CDDLM "Deployment and configuration Services" - Deployment Services find out what they need to deploy and configure by using other services. E.g., 'Information services' (in broad sense) or candidate set generator may be one option. (Or job definition; again the term is used broadly.) - Note: The diagram is a broad view of what is needed. OGSA does not have to do everything. Use it to drive requirements to other groups, e.g., CDDLM. - Container has porttypes for moving/staging data in and out. Action: Andrew to make a revised PE figure for next week's F2F. ** Open issues - Higher level services, e.g., (job) queue services have not been discussed yet. Also logging/monitoring/accounting services, fault detection/management, license management services. - The role of WS-Agreement has not been explored yet. - One scenario is the Job Manager and Execution Planning Services negotiating a plan to use a set of containers. Agreements are then reached between the Job Manager and the Reservation Services for future availability of containers. (After the agreement is reached the deployment/configuration services would get called at an appropriate time to set up the containers.) - Not decided how far to use agreement concept. - For example might even have agreement on Information Services, e.g., QoS agreement that the Information Service will provide fresh data, e.g., not older than 2 minutes. - (Andrew) Agreed but we should use WS-agreement for main-path only at this moment for simplicity. - PE Target for GGF10 (Berlin). - Discuss PE big picture. Also sketch out some i/f and scenarios. - Do not go into details of how agreement is used or can be used. ** Relation with other groups - In the past it was accepted that we do not have to be constrained to fitting our work into what people are already doing. We can draw up a proposal and take it to that group and discuss how to fit things together. *** CDDLM - (Kojo) CDDLM will also try and draw similar picture. (There is a somewhat similar one in last Monday's material, mostly p3.) http://www-unix.gridforum.org/mail_archive/ogsa-wg/2004/02/ppt00000.ppt Action: Kojo to draw up similar picture to the PE for CDDLM services using the PE terminology for the F2F. - Resource allocation: Probably corresponds to a number of entities on the PE figure (e.g., execution planning, reservation) - (Kojo/Hiro) Resource is similar to container. - (Ravi) Container is a configured resource. (It exists at a higher level (above) the resource in the CDDLM figure.) The PE figure probably does not have anything equivalent to resource. - (Andrew) In CDDLM figure deployment uses resources only. But it might also have to use to non-resource entities, e..g., virtualization of filesystems. - Resources can be used for more things than just job execution. - Also i/f for using resources and managing resources are different. Not necessariy the same as the CDDLM figure seems to imply. - CDDLM figure left hand side: Makes resource available (index service and reservation) - CDDLM figure right hand side: Sets up resource. - Deployment: Give it a description of what's needed for the job to run. - (Kojo) Negotiating resource allocations is done prior to deployment. Everything needed for deploy ment is available before deployment begins. No further negotiation is needed. - (Ravi) Agreement to get resource allocation; Agreement to arrange deployment. Or they could be one single agreement. This sums up to broader 'provisioning' functionality. - (Kojo) Deployment is narrower. - In broad terms provisioning is - How: This is covered by deployment. Deployment is told when and where and carries out the operation. - When/Where: Driven by other services, e.g., job manager or reservation services. - Action: Andrew and Ravi to draw up PE requirements for CDDLM group.