outputting the aggregate column

[Ethan Mallove]

[snip] ...


  <parameter id="count">
    <value>op</value>
  </parameter>

  <source id="id_test_name">
    <result>test_name</result>
  </source>

  <operator id="op" type="count">
     <input>id_test_name</input>
  </operator>

  <combiner id="comb" parameters="unique">
    <input>id_test_name</input>
  </combiner>

  <output>
    <input>comb</input>
    <input>id_test_name</input>
  </output>

</query>


The above query outputs:

allocmem
allocmem
allocmem

How can I make it output?

allocmem 3

-Ethan

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Joachim Worringen-4]

Ethan Mallove wrote:

> [snip] ...
>
>
>   <parameter id="count">
>     <value>op</value>
>   </parameter>
>
>   <source id="id_test_name">
>     <result>test_name</result>
>   </source>
>
>   <operator id="op" type="count">
>      <input>id_test_name</input>
>   </operator>
>
>   <combiner id="comb" parameters="unique">
>     <input>id_test_name</input>

       <input>op</input>

>   </combiner>
>
>   <output>
>     <input>comb</input>
remove this line

>   </output>
>
> </query>
>
>
> The above query outputs:
>
> allocmem
> allocmem
> allocmem
>
> How can I make it output?
>
> allocmem 3

I don't know the structure of your experiment, but you need to <include>
the operator somewhere! How should perfbase print the outcome of the
operator without being it referenced somewhere? And use the
'show="data"' attribute for the <parameter>.

Maybe it'll work with the above changes, but maybe you need to do more
make it work.

  Joachim

--
Joachim Worringen, Software Architect, Dolphin Interconnect Solutions
phone ++49/(0)228/324 08 17 - http://www.dolphinics.com

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Ethan Mallove]

This is what I have in the expermiment (it's the same experiment as before).
There are are about twenty or so of these <result>'s which are very similar to
the "test_name" (I appended the *full* experiment description, but do you need
all of it? I'm dealing only with the "test_name" <result>.).


[snip] ...

  <result>
    <name>test_name</name>
    <synopsis>name of test being run</synopsis>
    <datatype>string</datatype>
  </result>

[snip] ...


Joachim Worringen wrote On 07/27/06 03:21,:

> Ethan Mallove wrote:
>
>>[snip] ...
>>
>>
>>  <parameter id="count">
>>    <value>op</value>
>>  </parameter>
>>
>>  <source id="id_test_name">
>>    <result>test_name</result>
>>  </source>
>>
>>  <operator id="op" type="count">
>>     <input>id_test_name</input>
>>  </operator>
>>
>>  <combiner id="comb" parameters="unique">
>>    <input>id_test_name</input>
>
>
>        <input>op</input>
>

This isn't where the operator "op" is to be referenced? Looks like the analog of
the test/combiner example to me:

  <operator id="op.min" type="min">
    <input>src.latency</input>
  </operator>

... [snip] ...

  <combiner id="comb.minmaxavg">
    <input>op.min</input>

... [snip] ...

  </combiner>

>>  </combiner>
>>
>>  <output>
>>    <input>comb</input>
>
> remove this line
>
>>  </output>
>>
>></query>
>>
>>
>>The above query outputs:
>>
>>allocmem
>>allocmem
>>allocmem
>>
>>How can I make it output?
>>
>>allocmem 3
>
>
> I don't know the structure of your experiment, but you need to <include>
> the operator somewhere! How should perfbase print the outcome of the
> operator without being it referenced somewhere? And use the
> 'show="data"' attribute for the <parameter>.
>

show="all" didn't ouptut differently than show="auto".


> Maybe it'll work with the above changes, but maybe you need to do more
> make it work.
>
>   Joachim
>


--- full exp description ---

  <result occurrence="once">
    <name>hostname</name>
    <synopsis>host name</synopsis>
    <datatype>string</datatype>
  </result>

  <result occurrence="once">
    <name>os_name</name>
    <synopsis>operating system name</synopsis>
    <default>N/A</default>
    <datatype>string</datatype>
  </result>

  <result occurrence="once">
    <name>os_version</name>
    <synopsis>operating system version</synopsis>
    <default>N/A</default>
    <datatype>string</datatype>
  </result>

  <result occurrence="once">
    <name>platform_hardware</name>
    <synopsis>platform architecture</synopsis>
    <default>N/A</default>
    <datatype>string</datatype>
  </result>

  <result occurrence="once">
    <name>platform_type</name>
    <synopsis>platform os/architecture</synopsis>
    <default>N/A</default>
    <datatype>string</datatype>
  </result>

  <result occurrence="once">
    <name>platform_id</name>
    <synopsis>platform identifier</synopsis>
    <default>N/A</default>
    <datatype>string</datatype>
  </result>

  <result occurrence="once">
    <name>mpi_name</name>
    <synopsis>MPI name</synopsis>
    <datatype>string</datatype>
  </result>

  <result occurrence="once">
    <name>mpi_version</name>
    <synopsis>MPI version</synopsis>
    <datatype>string</datatype>
  </result>

  <result occurrence="once">
    <name>mpi_get_section_name</name>
    <synopsis>MPI get section name</synopsis>
    <default>N/A</default>
    <datatype>string</datatype>
  </result>

  <result occurrence="once">
    <name>mpi_install_section_name</name>
    <synopsis>MPI install section name</synopsis>
    <default>N/A</default>
    <datatype>string</datatype>
  </result>

  <result occurrence="once">
    <name>test_build_section_name</name>
    <synopsis>test build section name</synopsis>
    <default>N/A</default>
    <datatype>string</datatype>
  </result>

  <result occurrence="once">
    <name>test_run_section_name</name>
    <synopsis>test run section name</synopsis>
    <default>N/A</default>
    <datatype>string</datatype>
  </result>

  <result occurrence="once">
    <name>merge_stdout_stderr</name>
    <synopsis>true if stdout and stderr are merged</synopsis>
    <default>0</default>
    <datatype>integer</datatype>
  </result>

  <result occurrence="once">
    <name>environment</name>
    <synopsis>environment the run was performed in</synopsis>
    <default>N/A</default>
    <datatype>text</datatype>
  </result>

  <parameter occurrence="once">
    <name>submit_timestamp</name>
    <synopsis>date and time the run results were submitted</synopsis>
    <datatype>timestamp</datatype>
  </parameter>

  <!-- MULTIPLE-occurrence FIELDS -->

  <result>
    <name>test_command</name>
    <synopsis>full command line for this test</synopsis>
    <default>N/A</default>
    <datatype>string</datatype>
  </result>

  <result>
    <name>test_name</name>
    <synopsis>name of test being run</synopsis>
    <datatype>string</datatype>
  </result>

  <result>
    <name>test_np</name>
    <synopsis>number of processes used for test</synopsis>
    <datatype>integer</datatype>
  </result>

  <result>
    <name>test_message</name>
    <synopsis>test result message</synopsis>
    <default>N/A</default>
    <datatype>string</datatype>
  </result>

  <parameter>
    <name>test_pass</name>
    <synopsis>whether or not the test passed</synopsis>
    <datatype>integer</datatype>
  </parameter>

  <result>
    <name>stderr</name>
    <synopsis>stderr output from run</synopsis>
    <default>N/A</default>
    <datatype>text</datatype>
  </result>

  <result>
    <name>stdout</name>
    <synopsis>stdout output from run</synopsis>
    <default>N/A</default>
    <datatype>text</datatype>
  </result>

  <result>
    <name>start_timestamp</name>
    <synopsis>date and time this test was started</synopsis>
    <datatype>timestamp</datatype>
  </result>

  <result>
    <name>stop_timestamp</name>
    <synopsis>date and time this test was finished</synopsis>
    <datatype>timestamp</datatype>
  </result>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Ethan Mallove]

Much simpler example:


$ cat www/exp_rollup.xml
<?xml version="1.0"?>
<!DOCTYPE experiment SYSTEM "../../dtd/pb_experiment.dtd">
<experiment>
  <name>rollup</name>

  <info>
    <performed_by>
      <name>ethan</name>
      <organization>n/a</organization>
    </performed_by>
    <synopsis>n/a</synopsis>
    <description>simple rollup test</description>
  </info>

  <result>
    <name>test_name</name>
    <datatype>string</datatype>
  </result>
</experiment>

$ cat www/inp_rollup.xml
<?xml version="1.0"?>
<!DOCTYPE input SYSTEM "../../dtd/pb_input.dtd">
<input>
  <experiment>rollup</experiment>

  <named_location store_set="yes" len="0">
    <name>test_name</name>
    <match>test_name: </match>
  </named_location>

</input>

$ cat www/query_rollup.xml
<?xml version="1.0" standalone="no"?>
<!DOCTYPE query SYSTEM "../../dtd/pb_query.dtd">

<query id="rollup">

  <experiment>rollup</experiment>
  <description>rollup test</description>

  <source id="id_test_name">
    <result>test_name</result>
  </source>

  <combiner id="comb">
    <input>id_test_name</input>
  </combiner>

  <operator id="op" type="count">
    <input>id_test_name</input>
  </operator>

  <output>
    <input>id_test_name</input>
  </output>

</query>

$ cat www/query_rollup.xml | perfbase query --desc=- --exp=rollup
# test_name[]
fint8_f
fint8_f
fint8_f
fint8_f
fint8_f
fint8_f
freal16_f
freal16_f
freal16_f
spawn


Two questions:

1. In what order does perfbase process each element of the XML query? (My poor
guess from looking at pb_query.py is: source, combiner, operator, then output.)

2. How do I get the query to output:

fint8_f 6
freal16_f 3
spawn 1

-Ethan

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Joachim Worringen-4]

Ethan Mallove wrote:
> Joachim,
>
> It appears that pb_experiment.dtd requires that there be at least one <parameter> and one <result>. "perfbase setup" hasn't complained that I didn't have one - maybe I ought to validate my xml files? I would've thought the following would do just that, but it did not:
>
> $ cat ../www/exp_test_run_correctness.xml | xmllint --postvalid --dtdvalid ../perfbase/trunk/dtd/pb_experiment.dtd

I'm not familiar with xmllint, but will look into it.

> Do you mean it is impossible for me to obtain the above result if both test_name and n_error are a <result>? I'm assuming this must be, because the following changes to your queries generate this error (my xml's have -e appended to the name):
>
> #* ERROR: can not process query specification
>     "<operator> 'op.cnt_pass': no parameter specified which provides data
> vectors (use a <series>!)"
>
> $ diff query_rollup-e.xml query_rollup.xml
> 7a8,11
>>   <parameter id="p.name" show="all">
>>     <value>test_name</value>
>>   </parameter>
>>
> 11c15
> <     <result>test_name</result>
> ---
>>     <input>p.name</input>
>
> $ diff exp_rollup-e.xml exp_rollup.xml
> 15c15
> <   <result>
> ---
>>   <parameter>
> 18c18
> <   </result>
> ---
>>   </parameter>
>
>
> Or is there a workaround? I ask because I would like to be able to filter/aggregate on any field exactly the same way without having to treat one
> or the other differently on the querying side.

perfbase differentiates between results and parameters for some reason,
i.e. for the output of the data. This will not change, and I would
recommend to create a experiment structure that reflects this.

If you have no parameter that provides a data vector, you can add a
pseudo-parameter which is a <series> (that's what the error message
above says) within a source:

<source ...>
        ....
        <series id="s.vector">
                <name>n</name>
        </series>
</source>

> Lastly, this is how I understand (or misunderstand) your query:
>
> The <parameter> element aliases a database field (or another object of some
> kind) from the database.
>
> The <source> element takes a set of <parameter> and <result> objects.
>
> An <operator> takes a <source> object as an input, and outputs the result of
> performing the operation on the source object.
>
> That result from the <operator> is then passed to <combiner>.
>
> The combiner takes query objects as input and directs them to <output>.

At the bottom of every query is (at least) one source. It supplies
result and parameter content, possibly filtered by limits on the
parameters. At the top, there is (at least) one output. Between those,
you can arrange any number of operators and combiners, connected however
you like. Simple as that...

  Joachim

--
Joachim Worringen, Software Architect, Dolphin Interconnect Solutions
phone ++49/(0)228/324 08 17 - http://www.dolphinics.com

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...