From rhood@nas.nasa.gov Wed Mar 24 12:06:02 2004 Date: Mon, 15 Mar 2004 08:37:35 -0800 From: Robert T. Hood To: Rob Van der Wijngaart Subject: GB notes from GGF10 Grid Benchmarking Working Group 3/10/04 meeting in Berlin at GGF10 Rob Van der Wijngaart chairing Allan Snavely can't be here notes by R. Hood please join mailing list please sign in to list update: document status one doc on ALU intensive GB in pipeline 5 new proposals data cube DC data intensive GB data traffic DT comm intensive GB GRASP comm intensive grid probes sleep-0 proc intensive GB rapid fire RF proc intensive GB charter revisit: we propose a set of tasks that assess grid perf at the level of the user apps--specified as a paper-and-pencil benchmark not: characterize existing and emerging grid apps delivs: want: a small set of families of grid benchmarks not: a framework that specifies the requirements of constructing new data flow graphs to be used as GB's types of grid benchmarks dominated by computation (ALU-intensive) AIGB reference implementations doc w/ AIGB p-a-p specs (GGF10) GGF11: DIGB DC, CICB (DT GRASP) PIGB (RF, sleep-0) GGF12: ref impl's GGF13: final docs w/ p-a-p specs of all of above NOTE: This may be too ambitious given the amount of activity we've seen on the mailing list, etc GGF Informational document: ALU Intensive GB's tried to run ref benchmark: trouble compiling for Fortran on linux (RVW: fixed) Rosa CEPBA?: have implemented all in GridSuperscalar: problem is Size A has 9 nodes; need more nodes graphs of ED, HC, VP, and MB VP note: needs to be directed path from any node to report node MB: challenge to implementers (try to schedule) diagrams are for smallest size VP: +n "iterations" of 3 step chain; work w/in node grows too MB: add row & col for each increasing class size only go to class "C" because NPB class "D" runs for hours on a supercomputer try to grow graphs to roughly keep communication/computation ratio the same Andrea Note: there are two extra jobs; RVW: when NPB processes are connected they have different discretizations--different array sizes; need to do reduction & interpolation; this is work attached to computational nodes; (e.g. part of the MG); the "mesh filter" program; not necessary to do it the way the ref impl does it; the globus version groups pre--proc, proc, and post-proc in same graph node Andrea NOTE: needs to have an explanation in document error criterion: output data as data or ASCII; for heterogeneity; had been a problem with free formatted ASCII output; is not formatted representation: other plans besides p-a-p specs? e.g., use UML to spec them? role of GGF is to do stds docs; could derive other things; would want to see if there is a need in community; formulate useful extensions & put on mailing list are there companies providing benchmarks? none known; benchmarks usually done by groups of people (users) ------------------------------------------------------ switch to Andrea Casotto with proposal for PIGB test ability to handle jobs (many tiny jobs) expose & measure overhead of job mgmt system job: dispatch-able stoppable recoverable accountable commands that do nothing: sleep 0 echo $N /bin/true empty shell script empty C program is there a sleep 0 web-service Benchmark proposal measure time to submit & execute 1000 sleep-0's sleep-zero-ps (s0ps) measurements 25 CPUS t=29m 0.57 s0ps 200 25s 40 s0ps use benchmark to match grid w/ workload low s0ps: don't submit lots of small jobs; lump small comp'l units into 1 job high s0ps: could submit examples of high s0ps needs VLSI library characterization (10k+ sims, 20 secs ea) protein folding bad benchmark simple shell loop with sleep 0 validation of benchmark could do echo $N; sum results; check value future when can run 1000000 sleep0's in a reasonable amount of time when will we get 1000 s0ps Q: what about scheduling? Q: ?; possible scheduling error have families of benchmarks C: very easy to implement anyone know how to build a web service that does nothing? ----------------------------------- RVW: another PIGB proposal AIGB: class X uses NGB class X AIGB: class X>=A uses A PIGB: class X uses S premature numerical convergence; introduce time-dependent source term verification of many nodes (wide ED and MB grophs); verify arith avg of soln of all nodes that are connected to report node suggestions: why is VP 3xN? because of RVW: amount of work in PIGB class X is supposed to match work in AIGB class X