OGSA-WG Interim Meeting - 18-20 August 2004 ============================================ * Participants Peter Ziu (Northrop Grumman) Jay Unger (IBM) Jeffrin Von Reich (HP) Ravi Subramaniam (Intel) Chris Smith (Platform) Frank Siebenlist (ANL) Andreas Savva (Fujitsu) Takuya Mori (NEC/ANL) Fred Maciel (Hitachi) Allen Luniewski (IBM) Hiro Kishimoto (Fujitsu) Andrew Grimshaw (UVa) Keisuke Fukui (Fujitsu) Ian Foster (ANL) Michael Behrens (R2AD, LLC) ** On bridge: Latha Srinivasan (HP) Jem Treadwell (HP Susan Malaika (IBM) Steve Crumb (GGF) Dave Berry (NeSC) ** WSDM cross-WG session Alan Weissberger (NEC) William Vambenepe (HP) Igor Sedukhin (CA) Bryan Murray (HP) Heather Kreger (IBM) Mitsunori Satomi (Hitachi) Minutes: Andreas Savva * OGSA Marketing (bubble) diagram GGF wants some accompanying explanatory text for the bubble diagram. Consulted with Steve Crumb on what's needed. - An introductory paragraph on the architecture: use case driven; industry focused; and why architecture is needed (use Andrew+Dave's draft from some time back as a start) - One paragraph per capability (large bubble); not necessarily explaining or even mentioning all 'services' (smaller bubbles) - It's ok to go more than one page of text if necessary. The length will be adjusted later. - The smaller bubbles are not correct wrt OGSA v1. - Discussed and agreed that there is no need to fix the names; bubble diagram doesn't have to be consistent with OGSA v1 Action: Form a sub-group and produce explanatory text by Sep 1. - Volunteers: Andrew, Andreas, Jay, Ravi, Jem... - Use the draft that Andreas started. (Andreas to give copy to Andrew > Done) * Preparation for cross-WG session with WSDM (See WSDM cross-wg discussion further down for continuation/resolution.) - Discussion kicked off from management in OGSA v1.0 slide - A question was raised on whether the types of management are really as discrete as the figure (OGSA v1) seems to imply. - WSDM has start/stop. Is this basic functionality or following (the OGSI paradigm) something is there or not; and if it's there you can find things about it, e.g., how to start/stop it. [ - A side discussion on whether the mechanism of how things get created (factory) is out of OGSA scope or not? - What is returned is an EPR (i.e., that there is a representation of the 'thing' that can be used) ] - Need for a base manageability type that is pervasive - E.g., discover & (bring into existence and make it go away) - And it seems that WSDM are doing one layer above what OGSA is after. E.g., the state diagram (and start/stop) - Manageability aspect not as a base but a 'side' or one aspect - Andrew is planning to do a naming white paper for distributed systems (MS) - Identity and mapping to EPRs; concern that if done separately by adding some property it will be done in different (incompatible) ways. - Is identity the responsibility of the endpoint or not? - Assigning identity and authenticating are different aspects - Uniqueness (or not); typically not - Are they AKAs; or - Are they separate identities (aliases) - Identity is not (necessarily) address - And how does it relate to EPRs - There is a normative way of comparing references in the latest W3C submission of WS-Addressing. - http://www.w3.org/Submission/2004/SUBM-ws-addressing-20040810/ - 2.4 Endpoint reference comparison - But it is unclear if this is really helpful - Data design team also had naming requirements; they would like OGSA to do something about it. Action: Create a design team on naming/identity to determine requirements - Andrew to lead - Write up scope - Send invitation as first step and arrange a teleconference - Volunteers: Ian, Frank, Jay, Ravi, Jeffrin, ... - Ask Jeff Frey, Dave Snelling, Steve Graham - Ask WSDM to join the design team (they are working on naming too) > Heather, Igor, William - Ask for UK e-Science volunteers - Lots of schemes to review: DNS, JNDI, handle.net, UDDI, etc - Make a list of these as a parallel first step. - End result is a white paper by Sep 7 - Then have to decide how to proceed (WG, which standards body, etc) * OGSA v2 discussion and how to proceed - Go to more detail versus step back and try to do a refactoring - Might have to go one level down before we can do a refactoring - What's the next step? - As architecture group preference is to step back - But also need to go on to detailed work to show progress - Detailed work is to some extent already being done by other groups - Do not necessarily fit in OGSA in a natural way - Some WGs are willing to adapt (e.g., by considering OGSA requirements) - Spinning design teams out to WGs is an option - Doing so quickly may end up being a problem if the architecure is not reconciled first. - Is it important for a group doing OGSA related work to be 'OGSA sactioned' or not? - (Consensus towards not; but would prefer some involvement.) - Some WGs are working independently and we are saying we are using their work (WSRF, WSN, etc) - And they may have there own idea of the architecture they fit in Resolved: Get more input from other external sources - And who should participate to make effort successful - What do we need to do to get broader industry participation - This is more than just asking people to review OGSA v1; in a sense it is evangelizing. ** Unclear how to go about this; first step is to work out ** whom to contact and how to go about it. - CIOs (e.g.,pharmaceutical companies, (end users)) - Probably need a short introductory document for this purpose - Grid architects - Would be acceptable to give them a longer document; but may not need the more detailed explanation on services. - Chris and Andrew had ideas about who to contact. [ - Some side discussion on how far industry approval should also drive what is part of OGSA ] Resolved: Start on a refactoring of the architecture first (and do not start any new WGs at this point; pending on next proposal) ** Data and EMS should be the first ones to refactor ** - Prioritize refactoring: Start with EMS; Ravi to lead - Ravi also to look at taking things one level deeper with EMS. Resolved: Start new WGs (or coopt existing ones) to do work that is mature enough to be spun out of design teams ** Start off with spinning out just one WG ** - Topic, scope and timescale (something simple) - Need to be as concrete as possible when defining tasks (and how they fit in OGSA) - Successful spin out requires what? > It's a good idea to find out and "stack the deck" to > make sure that a spin out will be successful. - Choose a candidate probably from Data: GFS, DAIS, etc? - (EMS seems more complex than Data; defer for now.) - Above resolutions and implied actions are not mutually exclusive (but may end byt being so due to resource limitations) ** "OGSA Documents" document review Agreed that the content is ok, including the paragraph starting 'In due course we expect to add Recommendation documents ...' - Key phrase is "In due course..." - What does it mean to be OGSA compliant? - Can we talk about OGSA recommendation document? - OGSA is not just the constituent specifications - But compliance comes in bits (interoperable implementations per specification) - And also need the implementations to make sure that things will actually work (before going too far; shouldn't let specification get too far ahead of implementation; and vice versa presumably) - Will need to be careful what OGSA endorses; endorsing the 'wrong' spec might be more harmful than not endorsing a spec. - Include interactions or profiles; but comes down to interoperable implementations - Comment: recommendations need - 2 'interoperable' implementations; and - experience report to document interoperability Proposal: Make the "OGSA Documents" document an informational document (Postponed resolution for later) Proposal: The name of the document is confusing. Change the name of this document to "OGSA timeline" or "OGSA process" (Postponed resolution for later) Action: Andrew as Area director to look into getting an eighth question on the WG formation question list: "Relation to OGSA" * WSDM cross-WG session ** EMS presentation p7: "resources that model processing": resources that can contain jobs ('containers') p11: fault tolerant requirements are pervasive? - Some might not have any requirements. Those that do are unlikely have the same general requirements; different techniques depending on requirements. p13: BLAST is chosen as an example that is a simple enough p15: "job is a grid service": i.e., it is modeled as ws-resource. - Distinction between job and job document - Job document can be represented as (or is equivalent to saying that it is) a resource property - At the moment the ogsa specification describes what is required functionally; may not state things in correct WSRF terms. - (Also distinction between what is the state and what is the state that you want to expose?) - Job document is a logical entity but might also have a 'real' incarnation as an xml document - Job manager encapsulates all aspects of executing a job or a set of jobs, from start to finish. (e.g., a workflow etc) - Is there a model of what is required to manage a 'running' job (a job that is in the system)? - Have not defined as a formal model; it is high level design only - A number of states are mentioned in the document - DRMAA has a job state model; it can be viewed as a state model for a specific job type (so subset of what EMS will need.) - Any work on monitoring the agreement etc. - Not going to that level at the moment. ** Q&A session - Management types (Resource management table) - (See preparation discussion) - Table puts things more discretely than intended. Talking about the 'same' resources; wsdm defines a set of base porttypes that should be extended to address higher level capabilities of the resource. - Domain specific resources can be defined; WSDM does not dictate what can be a resource - Figure was also trying to highlight the gray area where it is not clear if something is a functional interface or a manageability i/f (e.g., a job manager) - The manageability i/f is or should be an extension of WSDM base interface(s) - The functional aspect is not necessarily an extension of WSDM interfaces. - The distinction between functional or management 'capability' is not always clear. - May not be necessary to make clear distinction; better to think of it instead as extending a porttype. - (OGSA) Desire to draw line at 'things' that if they exist they are always about management. - WSDM would be interested to know if there are 'capabilities' or requirements that arise from 'capabilities' that cannot be represented by WSDM. - WSDM base manageability is identity; only required functionality. - Resource ID - This is the identity of the resource not of the interface. - identity is a URI (not a UUID) - unique in space and time - could use it as abstract name - can translate from EPR to resource id - considering whether to also define a registry (to support the translation from resource id to EPR) - (and then related question of how to find an appropriate registry to ask) - (Question on usage of the 'infrastructure' term; it is an OGSA v1 term.) - Identity/Name/Address - Identity, naming and statelessness are orthogonal issues; just being stateless doesn't remove the need to have a name or an identity. - What are the attributes required; and what is required for naming or for identity. - WSDM identity can be tested for equality; if equal it is the same; if not, further tests on capabilities to check whether it is the same or not. - Manageable resource needs to have a manageability endpoint; endpoint has an EPR. - Agreement on need of being able to go from name to address (and back) - EPR contains an address that is a URI; it doesn't have to be resolvable by DNS for example. It could be thought of as an abstract name; but that is not the 'expected' or typical 'usage' - URIs if they are not using the 'normal' scheme are not resolvable [See earlier action on naming; and action to start naming design team with Andrew as lead.] - State diagram is optional; extensible; and multiple ones can be defined. - Event format - It is a message format; a wrapper - WSDm also defining a couple of specific events - Again extensible - (And also thinking of filtering based on message content as well as topics) - If a resource is emitting events then define what topics must be defined for receiving those events - Does OGSA-WG care about this in the timeframe of OGSA v2? - Consensus is yes. Should be engaged early to make sure that we can use the same thing. Action: Heather to send details on events. - Ravi and Andrew would like to be involved in this discussion. - Policy and negotiation - Too early to discuss this. * Design team status ** EMS - Status update by Andrew - Completed reduced version for v1 - Still have the more detailed version prior to v1 - Main members: Andrew, Ravi, Chris - More people/groups were involved from time to time. - Ravi taking over team - Expect more interaction with GGF SRM area - CSF - Relation to EMS? - CSF could be one implementation of OGSA-EMS for legacy jobs. - (To be) contributed to GT4 - Source is available on sourceforge - License is the same as the Globus license - Planning tutorial at GGF12; including on how to extend it to support other schedulers. - (Customers exist that might queue half a million jobs per submission; need to be able to support that kind of scenario before it can be used in production.) ** Data - Status update by Andrew - Had a F2F in July - Developed a set of use cases in addition to the ones in the document. - Topics discussed: - Files, directories, caches, replication services, info dissemination, data movement (how to reduce overhead for data movement, e.g, by not introducing unnecessary copies.) - A strawman for proposals - A document proposal for OGSA ("OGSA Documents") - A question on how far some existing document/effort can be 'taken over' and used to develop a standard. - Example was JNDI; seems to be quite close to naming requirements - IP rights is the main question; will the owner agree to the standards organization's IP policy - Service description and then how an API might look like - A number of issues for OGSA (more general than data) - Naming - Factories: no formalized way of instantiating things - Migration - Change management - multi-valued functions (return more than one result to possibly more than zero party) [ - Multi-valued functions - multiple results and the ability to send results to multiple services - can be done via proxy and composition (overhead) - Is this something that can be done with wsdl at the moment (and specialized interactions) ] - Where to find information on what OGSA has discussed and decided - Philosophy, design, (anything else that might be important) - Some people seem to have the impression that OGSA-WG decisions are 'opaque.' - Everything is made public. You can find almost anything in the minutes or on gridforge. - Not necessarily easy to find to find things on gridforge Proposal: A set of notes each detailing a decision and its rationale (Architecture (AR) notes) - Doesn't have to be very long - Must be done in a collaborative manner to spread the load - And how to disseminate these (make them easy to find) Action: Andrew to check with Data team if such a mechanism would be appropriate to solve this problem. ** Security - Status update by Frank - Teleconference - Couldn't get enough people to come to calls - BoF like session at GGF11 (focus on OTP) - Workshop at GGF12 - At the moment people seem more interested in implementing; and only afterwards to go to GGF for standardization - Also a lot of the work is not done in GGF (instead it is in OASIS, etc). (Impression that GGF is not the place to do security work.) - Approach: - Design team focus on higher level requirements more closely related to Grid, e.g., VO requirements, lifecycle management, use cases, naming and security - Also possibly later on - Security attribute service and security provisioning service - Tie in between languages that specify job or agreements and security - Current members: Frank, Takuya - Interested to join: Chris, (Andreas), Ravi ** Resource Management - Status update by Fred - AKA CMM - Short version of CMM doc in OGSA v1 - Long version to be submitted to GGF Editor soon (almost final version for GGF11) - Also working with WSDM and DMTF - Scope or definition of 'resource mgmt' - If everything is a resource would management also include things like migrate, reserve a resource, etc. - Resource itself might not be able to do such functions on its own; might have a manager or some manageability (interface?) that drives a native capability. - Think of these as more 'functional' - It's out of scope for the design team to define 'functional' interfaces like migration and reservation - Design team also has RM folder under OGSA-WG; to start uploading things there too. ** Information Services - No representative present (Fred gave short overview) - Seems fairly advanced but in paused state right now - Spawned Logging WG - Seems to suffer form similar problem with security (not enough members) Action: Frank to ask Ian for an MDS representative to join the team ** Self-management - Current members: Jeffrin, Andreas, Jeff - To join: Andrew (to get a member of his team), Jay,... - Initial work resulted in OGSA v1 text - Presentation at GGF11 - Not much activity since - For v2 need to refine and make the current text more concrete; think about what services/interfaces are involved - No user requirements; at the moment use cases - Probably initial interactions with EMS - Comment on v1 text: A service can have self-managing aspects and also be part of a larger self managing activity Action: Jeffrin to keep track of this (or upload to public comments) Action: Jeffrin to schedule a call and announce on the list * Prioritize sections/areas to cover in v2 [This was fairly short session without a lot of discussion; probably needs more work.] - Naming - Security model; simple model that clearly separates policy from mechanism - WSDM relation and how we use it - e.g., Events - How context is propagated - Clearer definition of data model - Commonality between EMS and Data (refactoring) * GGF12 session plan review - EMS session (Tuesday) - Conflict with GSA is ok - Conflict with JSDL not ok for Chris and Andreas - Tuesday is not good for Ravi - Agreed to use 90 minutes within this slot (09:00 - 10:30) and re-schedule the other 90 minutes at a different time; try to exchange with another design team. Action: Hiro and Ravi to talk with other design teams and arrange re-schedule. - Security Sep 22 3:30-5:00 session is ok - No phone bridge available; not a problem - Design team report on last day - Frank is leaving early; Takuya will give the security team report - Confirmed: - Data: Dave (& Andrew) - EMS: Ravi (& Chris) - Naming design team - Need to arrange for a new slot; decided to fit in existing ones - Naming is part of Information services (in OGSA v1); Naming also has relation to security - Agreed to put cover naming within Information design team (half session) - And security aspects of naming in security session (another half session) Action: Hiro to confirm with Information design team - CMM cross-wg session is ok. * ACS plans presentation - Presentation by Keisuke - Planning a BoF at GGF12 - Application Contents Service: - Described in OGSA v14 - Would like an ID to be able to compare what is installed with what is available in the repository - ID not shown but it exists - Archive may be very large; may cause problems if it has to be transfered as one unit (e.g., insufficient resources to stage in). Might want to provide an interface to transfer in parts. - E.g., logical operation is 'get'; an interface to implement that operation may be 'get parts' - This might be also be thought of as a protocol level issue; so not visible in the interface. - (Reference to Mike Lewis dissertation) - Can parts be updated separately? >yes - (Content compression or consistency(signatures)) - IBM Intelligent Application Orchestrator (TIO) has similar information (dependency between applications required) - There is a functionality spectrum: from just a package to deployable functionality - This is aimed towards the low end; interface to repository and package. - Some related work (Solution Installation Schema) by IBM & Novell & InstallShield & etc was recently submitted to W3C - It is an effort from IBM On Demand; doesn't take into account provisioning - (Ian) Also interested in this topic; but seems to be fairly complex topic; not sure if OGSA can take it on (within group) - (Peter) Also working on use cases related to this topic. - Might want to specify 3-tier systems in detail and exactly where you want things to be laid out; to make sure policy or security issues are addressed. - Spec might be simple if it is just 'carton and packing list'; if it covers more functionality then it might quickly get more complex. - Intention is to keep things as simple as possible initially - (Andrew) Might also want to think of the archive as a specialization of work done in data > (relation to refactoring work) - (Jeffrin) Seems related to CDDLM. Identify interactions and relations with CDDLM. - OGSA v14 text listed part of the expected interactions. Intention is not to overlap with CDDLM but to build on that work. [ - Doubt that service level provisioning it feasible - This is trying to address a simpler issue. ] Action: Andreas to upload presentation to gridforge > Done. * Roadmap discussion [See two related ppts on gridforge ] - Identify dependencies and priorities we have received by the community - Also to help people looking from the outside on what is OGSA and what is happening - List of what is required and in what order - Intended audience: need to identify in more detail. Resolved: Roadmap document per design team and a document that links them for OGSA. OGSA document to also describe dependencies between the design teams' roadmaps. - And a one page powerpoint slide that shows the overall map (and dates) - All the documents are internal to ogsa-wg - While the one page powerpoint slide is public (See also roadmap-process ppt) - Agreement that it is ok continue working on higher levels based on the expection of specs at lower level that will define some needed functionality; as long as there is a description of what can be expected of the lower spec (in terms of requirements). - Create an ordering of the specs - (and how easy it is to do things; which are the low hanging fruits) Proposal: That roadmap should be independent of the dependencies (but should include the dependencies) - (Essentially milestones; but need to do more work before that can be done.) - Agreed that relations and dependencies with other groups need to be explicitly spelled out: - What we will work with them or defer to them, priorities, etc. Document decisions and what they imply and who is doing what. - Need to communicate to them our requirements. This is not just liaison work; someone should 'own' that piece of work. - E.g., a joint work/document or some more explicit understanding - In the case of WSDM we have identitified: - Naming/identity - Events - EMS overlap (or EMS usage of WSDM) --- as an initial 'application' area - (Metadata --- query to data design team) - (IP policy issues; when work is done in different standards bodies) [ - One ppt was an attempt to map the area at the low end. Things that are needed for higher level work - Not added: Distributed state is one issue but cannot decide what model to use. - Discussed going up the stack or sorting the entries between those that fall in WS and those that don't. (Postponed for later) - Second ppt was an attempt to map the work and how it should be carried out. ] Action: Jay to talk with Ian and Hiro to form a core design team to work on dependencies at architecture level (and roadmap) * R2AD presentation - R2AD: Rapid Research Agile Development - Possibly not interoperable with grids but aim is to advocate/make grid & OGSA aware - Pending on funding starting October - grid.geden.org is the public site for this project - Really remote environment, e.g., ship - Interested and want to participate in ACS - Expect 5 years to first deployment - p7: Not sure how accurate it is at the moment. But format looks like what is needed for the one page roadmap - Goal is to use standard software & facilitate adoption of standard software - XML based registration and deployment of components and automation - Security is a big concern. Especially authentication/integrity of components. - Government gets use rights to work (but collaboration is ok) Action: Jay to convert p7 from graphic to powerpoint and send to Andreas - Then have to update for OGSA (WS specs); also show optional/required Action: Jay to check status of WSIL * Teleconference schedule - No call on Monday - Wednesday call to continue roadmap discussion and finalize GGF12 plan if possible.