Transactions BoF, Session 1, 8th June 2004

Andrew Martin (on behalf of Andrew Simpson): eDiamond

The eDiamond project provides a Grid infrastructure for sharing and processing mamographic data in a virtual organisation - typically across UK NHS trusts.

eDiamond has a ranges of sources for mamographic images and data. This data is captured in a standard format called DICOM, and associated scan image.

The large file (DICOM) goes to contract management system which is optimised for access to this type of media.

The parsed DICOM file produces a smaller XML document which is stored in a database to enable fast operations on metadata. 

The database entry and content management service entries are both keyed from the same GUID. 

In effect the content manager system is indexed by the DB.

The underlying requirement is that the two systems be updated consistently. There are two caveats:

1. Time to upload to content manager system may be long
2. No need for interleaved updates

Discussion:

Simple use case, can use a variety of means to solve it (transactions, messaging, application level stuff)

Q: There are WS standards out there - why not use them?
A: Probably will if the use cases are amenable to them. This use-case is almost certainly able to be solved with current work.

Dieter Gawlick

Messaging Environments and Transactions

Dieter presented a model for transactions in autonomous systems which avoids some of the performance penalties of traditional approaches. In particular he drew the group's attention to the notion of data being "committed but not recoverable."
See Dieter's slides for the model - they are more informative than these notes can be.

Discussion:

Does this work change our charter to be looking at new models? Two possibilities: current use cases easily solved, Dieter's use cases not easily solved by current techniques?
Current charter already allows for both possibilities.


General discussion:

Point raised:

1. That we as a group could have usefully have invited the various WS transactions spec authors. The chair responded that this option had been considered and would hoefully go ahead at a subsequenty GGF. There was broad acceptance that some educational input from practitioners in this space would be helpful.

2. Is it worthwhile to create a generic WS transaction model?

3. Is there support for "bulk file copy" scenario for long running transactions? 
(this was answered by way of explaining briefly some of the features in the popular WS transactions specs)

4. Some thought that some transactions HAVE to be ACID
(this was partially answered by the chair as an issue of trust and buy-in into particular models (e.g. WS-AT))

During the general discusson, an ad-hoc use case was given based on Job scheduling. 

In this case all processing nodes of a compute server have to be recoverable, including manager node. There may be a need transactional update of status from slave "processes" to master database.

Futher discussion tended towards is transactional message delivery a required use case (inspired by Dieter's presentation)?

Meeting adjourned.