Nabble - Esper User list

Re: Retrieving events prior an event

2009-12-10T05:53:25Z

Hello Thomas,

Thanks for the reply, but I still can't reach a good solution for this...

If I use the match_recognize, I end up with something like this:
select * from BaseEvent
match_recognize(
measures A as start, B as stop
pattern(A B)
define
B as instanceof(B, DerivedEvent)
)

This gives an error (Incorrect syntax near 'B'). Furthermore, I still can't access the fields specific to DerivedEvent (or I have to use the cast function and I will end up with something extremely complicated).

I tried also using the prev function and it could almost do what I'm looking for, except that the prev function selects only a field, and not the complette event. Is it possible to have something like this:
select prev(5, BaseEvent) from DerivedEvent(....) unidirectional, BaseEvent.win:length(5)

So, in the prev function I select the full BaseEvent (of course, this gives an error stating that BaseEvent is not valid in any streams).

I really would like to find an elegant solution to this problem but it seems there's none :(. The problem seemed to be very simple but trying to express that in Esper is much more complicated than I first thought.

Thanks for any additional feedback.
Cédric

At 14:14 10/12/2009, you wrote:

Hi Cedric,

match recognize can also work with variant streams, i.e. multiple types of events. Yes followed-by in a pattern is another way, and a where-clause against "prev" is also possible for example with length window.

Best regards
Tom

From: Cédric Moonen <cedric.moonen@...>
To: user@...
Sent: Wed, December 9, 2009 5:02:00 AM
Subject: Re: [esper-user] Retrieving events prior an event

Hello Thomas,

I've looked at the documentation of Esper 3.2 and this is a new feature (we were still using 3.1). It looks indeed very promising to use the match-recognize. I will have to experiment with it but I think what might be a problem (not sure for now) is that you can only work with one type of events. Luckily, all our events have the same common base class so there is probably a work around here if we are able to specify a more defined type in the define clause. Is that possible ?
By the way, I realized that I made a small mistake in my previous example: what should be retrieved is "A3 B1" and "A12 B2" (and not "A12 B1"), sorry for the confusion.

Out of curiosity, is there a way to achieve this without using the match-recognize ? Any way using the prev or prior function ?

There is a way but it is really ugly and I really would like to find an easier solution. Here is what we have right now:
select * from B as evtB unidirectional, pattern[every A[5] as evtA]

This seems to work fine but it is really complex. Any other better looking way :-) ?

Thanks
Cédric

At 19:24 8/12/2009, you wrote:

Hi Cedrik,

here is a possible match-recognize (the regular expression only):
- A3 B1: (A X X X X B), select A and B
- A12 B1: (B X X X X A), select A and B

or both at the same time "(A X X X X B) | (B X X X X A)"

Best regards
Tom

From: Cédric Moonen <cedric.moonen@...>
To: user@...
Sent: Tue, December 8, 2009 9:07:18 AM
Subject: Re: [esper-user] Retrieving events prior an event

Thanks for the reply,

But I don't think this will really solve my problem. Let me show you with an example, which will probably be clearer than my explanation. Suppose the following sequence of events:
A1 A2 A3 A4 A5 A6 A7 B1 A8 A9 A10 A11 A12 A13 A14 A15 A16 B2 A17 A18

What I want to retrieve is:
- A3 B1 (A3 being the fifth A event before B10)
- A12 B1

I don't think this is really feasible using a pattern, is it ?

At 14:51 8/12/2009, you wrote:

May be a pattern using repeat or repeat until would be a better way?

2009/12/8 Cédric Moonen <cedric.moonen@... >

Hello,
I'm a bit stuck with a problem when trying to retrieve events that happened before a specific event. My scenario is that I have events coming in and all those events have the same base event (let's call it BaseEvent). Then, there are some specific events that I am looking for (SpecificEvent). I need to retrieve all those specific events with the base event that occured 5th-prior that specific event.
The first thing I did was something like this:
select * from SpecificEvent as event unidirectional, BaseEvent.win:length(5).std:firstevent() as prevEvent
But this didn't work as I expected and looking at the documentation of the std:firstevent, it is explained there that I should use the prev function instead because the window is restarted each time. So, I tried then to use the prev function but there's no explanation in the doc on how I have to specify that I want the full event and not one of the properties (although, it is stated that it should be possible). I tried to write something like this:
select event, prev(5, *) as prevEvent from SpecificEvent as event unidirectional, BaseEvent.win:length(5)
But it complains about the syntax error for the '*' symbol.
So my questions are:
1) How to write suche a statement ?
2) If there are multiple streams, how will the prev function "know" for which stream it has to select the event ?
I'm a bit confused with this prev function.
Thanks for your feedback.
Cédric

---------------------------------------------------------------------
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email

Cédric Moonen
Projectingenieur

Flanders' MECHATRONICS Technology Centre (FMTC)
Celestijnenlaan 300D
B-3001 Leuven
Belgium

GSM: +32-498-91.94.14
Fax: +32-16-32.80.64
cedric.moonen@...

www.fmtc.be

Cédric Moonen
Projectingenieur

Flanders' MECHATRONICS Technology Centre (FMTC)
Celestijnenlaan 300D
B-3001 Leuven
Belgium

GSM: +32-498-91.94.14
Fax: +32-16-32.80.64
cedric.moonen@...

www.fmtc.be

Cédric Moonen
Projectingenieur

Flanders' MECHATRONICS Technology Centre (FMTC)
Celestijnenlaan 300D
B-3001 Leuven
Belgium

GSM: +32-498-91.94.14
Fax: +32-16-32.80.64
cedric.moonen@...

www.fmtc.be

Re: Retrieving events prior an event

2009-12-10T05:14:57Z

Hi Cedric,

match recognize can also work with variant streams, i.e. multiple types of events. Yes followed-by in a pattern is another way, and a where-clause against "prev" is also possible for example with length window.

Best regards
Tom

From: Cédric Moonen <cedric.moonen@...>
To: user@...
Sent: Wed, December 9, 2009 5:02:00 AM
Subject: Re: [esper-user] Retrieving events prior an event

Hello Thomas,

I've looked at the documentation of Esper 3.2 and this is a new feature (we were still using 3.1). It looks indeed very promising to use the match-recognize. I will have to experiment with it but I think what might be a problem (not sure for now) is that you can only work with one type of events. Luckily, all our events have the same common base class so there is probably a work around here if we are able to specify a more defined type in the define clause. Is that possible ?
By the way, I realized that I made a small mistake in my previous example: what should be retrieved is "A3 B1" and "A12 B2" (and not "A12 B1"), sorry for the confusion.

Out of curiosity, is there a way to achieve this without using the match-recognize ? Any way using the prev or prior function ?

There is a way but it is really ugly and I really would like to find an easier solution. Here is what we have right now:
select * from B as evtB unidirectional, pattern[every A[5] as evtA]

This seems to work fine but it is really complex. Any other better looking way :-) ?

Thanks
Cédric

At 19:24 8/12/2009, you wrote:

Hi Cedrik,

here is a possible match-recognize (the regular expression only):
- A3 B1: (A X X X X B), select A and B
- A12 B1: (B X X X X A), select A and B

or both at the same time "(A X X X X B) | (B X X X X A)"

Best regards
Tom

From: Cédric Moonen <cedric.moonen@...>
To: user@...
Sent: Tue, December 8, 2009 9:07:18 AM
Subject: Re: [esper-user] Retrieving events prior an event

Thanks for the reply,

But I don't think this will really solve my problem. Let me show you with an example, which will probably be clearer than my explanation. Suppose the following sequence of events:
A1 A2 A3 A4 A5 A6 A7 B1 A8 A9 A10 A11 A12 A13 A14 A15 A16 B2 A17 A18

What I want to retrieve is:
- A3 B1 (A3 being the fifth A event before B10)
- A12 B1

I don't think this is really feasible using a pattern, is it ?

At 14:51 8/12/2009, you wrote:

May be a pattern using repeat or repeat until would be a better way?

2009/12/8 Cédric Moonen <cedric.moonen@... >

Hello,

I'm a bit stuck with a problem when trying to retrieve events that happened before a specific event. My scenario is that I have events coming in and all those events have the same base event (let's call it BaseEvent). Then, there are some specific events that I am looking for (SpecificEvent). I need to retrieve all those specific events with the base event that occured 5th-prior that specific event.

The first thing I did was something like this:
select * from SpecificEvent as event unidirectional, BaseEvent.win:length(5).std:firstevent() as prevEvent

But this didn't work as I expected and looking at the documentation of the std:firstevent, it is explained there that I should use the prev function instead because the window is restarted each time. So, I tried then to use the prev function but there's no explanation in the doc on how I have to specify that I want the full event and not one of the properties (although, it is stated that it should be possible). I tried to write something like this:

select event, prev(5, *) as prevEvent from SpecificEvent as event unidirectional, BaseEvent.win:length(5)

But it complains about the syntax error for the '*' symbol.

So my questions are:
1) How to write suche a statement ?
2) If there are multiple streams, how will the prev function "know" for which stream it has to select the event ?

I'm a bit confused with this prev function.

Thanks for your feedback.
Cédric

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

Cédric Moonen
Projectingenieur

Flanders' MECHATRONICS Technology Centre (FMTC)
Celestijnenlaan 300D
B-3001 Leuven
Belgium

GSM: +32-498-91.94.14
Fax: +32-16-32.80.64
cedric.moonen@...

www.fmtc.be

Cédric Moonen
Projectingenieur

Flanders' MECHATRONICS Technology Centre (FMTC)
Celestijnenlaan 300D
B-3001 Leuven
Belgium

GSM: +32-498-91.94.14
Fax: +32-16-32.80.64
cedric.moonen@...

www.fmtc.be

Re: Stuck at NStreamQueryPlanBuilder.computeBestPath

2009-12-09T19:22:18Z

got it. thanks a lot!

yep, I just needs to keep a single value from the last event from many streams and convert them (kind of transposing) to a single event. The mechanism should be efficient even with ~ 500 events. The use case for this is that I want Esper to create a kind of vector snapshot from several streams.

Thomas Bernhardt wrote:

If your 15 subqueries are very similar, perhaps a single subquery and a custom plugin aggregation function would be another way to go, function to simply return all x events in an array

For example:

select
(select myaggregation(value) from Event.std:unique(value)), j.value
from Event(id=14).std.lastevent() as j

Best regards
Tom

________________________________
From: pedroteixeira <pedro.t@gmail.com>
To: user@esper.codehaus.org
Sent: Wed, December 9, 2009 2:04:42 PM
Subject: Re: [esper-user] Stuck at NStreamQueryPlanBuilder.computeBestPath

Thanks, Thomas. Will create a test case and then create the issue.

I'was reading the example in the docs:
// Fast version, second alternative
select
(select avg(value) from MyEvent.win:length(100)) as avgA,
(select avg(value) from MyEvent.win:length(50)) as avgB,
a.symbol
from MyEvent.std:lastevent() a, MyEvent.std:lastevent() b

For a 15-stream join, is this the subquery approach that you meant?

select
(select value from Event(id=1).std:lastevent()),
(select value from Event(id=2).std:lastevent()),
...
(select value from Event(id=13).std:lastevent()),
j.value
from Event(id=14).std.lastevent() as j

cheers,
Pedro

Thomas Bernhardt wrote:
>
> Hi,
> can you please create a JIRA and attach the test Java class.
> As a workaround to a 15-stream join consider subqueries.
>
>
> Best regards
> Tom
>
>
> ________________________________
> From: pedroteixeira <pedro.t@gmail.com>
> To: user@esper.codehaus.org
> Sent: Wed, December 9, 2009 8:10:18 AM
> Subject: Re: [esper-user] Stuck at NStreamQueryPlanBuilder.computeBestPath
>
>
>
> TYPO. The actual query is more like this:
>
> select a.value, b.value, c.value ... j.value from Stream(id =
> 1).std:lastevent() as a, Stream(id = 2).std:lastevent() as b, ...,
> Stream(id
> =10).std:lastevent() as j
>
>
>
> pedroteixeira wrote:
>>
>>
>> Hello,
>>
>> I've stumbled across a problem, but need some help to understand what it
>> is the intended behavior of the NStreamQueryPlanBuilder class. Perhaps,
>> someone has already had this problem, but I was not able to find in the
>> archive.
>>
>>
>> I have a query which joins 10 streams in the following manner:
>>
>> select a.value, b.value, c.value ... j.value from Stream(id = 1) as a,
>> Stream(id = 2) as b, ..., Stream(id =10) as j
>>
>>
>> The esper engine gets stuck in the "computeBestPath" method, consuming
>> one
>> of the CPUs allocated to the running thread. With yourkit, I extracted
>> the
>> following stack trace:
>>
>>
>> com.espertech.esper.collection.PermutationEnumeration.nextElement()
>> com.espertech.esper.collection.NumberSetPermutationEnumeration.nextElement()
>> com.espertech.esper.epl.join.plan.NStreamQueryPlanBuilder.computeBestPath(int,
>> QueryGraph, HistoricalDependencyGraph)
>> com.espertech.esper.epl.join.plan.NStreamQueryPlanBuilder.build(QueryGraph,
>> EventType[], boolean, boolean[], HistoricalDependencyGraph,
>> HistoricalStreamIndexList[])
>> com.espertech.esper.epl.join.plan.QueryPlanBuilder.getPlan(EventType[],
>> List, QueryGraph, String[], boolean, boolean[],
>> HistoricalDependencyGraph, HistoricalStreamIndexList[],
>> ExprEvaluatorContext)
>> com.espertech.esper.epl.join.JoinSetComposerFactoryImpl.makeComposer(List,
>> ExprNode, EventType[], String[], Viewable[],
>> SelectClauseStreamSelectorEnum, StreamJoinAnalysisResult,
>> ExprEvaluatorContext)
>> com.espertech.esper.core.EPStatementStartMethod.handleJoin(String[],
>> EventType[], Viewable[], ResultSetProcessor,
>> SelectClauseStreamSelectorEnum, StatementContext, List,
>> StreamJoinAnalysisResult)
>> com.espertech.esper.core.EPStatementStartMethod.startSelect(boolean)
>> com.espertech.esper.core.EPStatementStartMethod.start(boolean, boolean,
>> boolean)
>> com.espertech.esper.core.StatementLifecycleSvcImpl.startInternal(String,
>> StatementLifecycleSvcImpl$EPStatementDesc, boolean, boolean, boolean)
>> com.espertech.esper.core.StatementLifecycleSvcImpl.start(String,
>> StatementLifecycleSvcImpl$EPStatementDesc, boolean, boolean, boolean)
>> com.espertech.esper.core.StatementLifecycleSvcImpl.createAndStart(StatementSpecRaw,
>> String, boolean, String, String, Map, Object, EPIsolationUnitServices)
>> com.espertech.esper.core.StatementLifecycleSvcImpl.createAndStart(StatementSpecRaw,
>> String, boolean, String, Object, EPIsolationUnitServices)
>> com.espertech.esper.core.EPAdministratorImpl.create(EPStatementObjectModel,
>> String)
>> com.espertech.esper.core.EPAdministratorImpl.create(EPStatementObjectModel)
>>
>>
>>
>> There seems to be some sort of caveat in the "computeBestPath" method,
>> since it might have to try 15! combinations (in my case). Is that really
>> intented?
>>
>> Thanks a bunch,
>> Pedro
>>
>>
>>
>
> --
> View this message in context:
> http://old.nabble.com/Stuck-at-NStreamQueryPlanBuilder.computeBestPath-tp26709887p26709902.html
> Sent from the Esper User list mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
> http://xircles.codehaus.org/manage_email
>
>
>
>

--
View this message in context: http://old.nabble.com/Stuck-at-NStreamQueryPlanBuilder.computeBestPath-tp26709887p26715790.html
Sent from the Esper User list mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

Re: Retrieving events prior an event

2009-12-09T18:58:56Z

use "instanceof"?

Best regards
Tom

From: Cédric Moonen <cedric.moonen@...>
To: user@...
Sent: Wed, December 9, 2009 6:12:41 AM
Subject: Re: [esper-user] Retrieving events prior an event

I made a mistake in my previous reply. The statement that is working looks like this in fact:
select * from B as evtB unidirectional, pattern[every evtA=A -> A -> A -> A -> A]

Which is even more ugly...

I've been looking at the mathc_recognize but as I expected I am a bit stuck on detecting the type of the event in the define clause. What I have so far is:

select * from BaseEvent
match_recognize (
measures A as start, B as stop
pattern(A X X X X B)
define
B as (??? AND B.var1="Val1" AND B.var2="Val2" )
)

So, to summarize, both A and B inherit from BaseEvent. The "???" in the define should be replaced by something that specifies the type of the event B (event A can be of any type). var1 and var2 are both members of the more derived type which B is representing. Is there any way to do this ?

Thanks for any feedback.
Cédric

At 11:02 9/12/2009, you wrote:

Hello Thomas,

I've looked at the documentation of Esper 3.2 and this is a new feature (we were still using 3.1). It looks indeed very promising to use the match-recognize. I will have to experiment with it but I think what might be a problem (not sure for now) is that you can only work with one type of events. Luckily, all our events have the same common base class so there is probably a work around here if we are able to specify a more defined type in the define clause. Is that possible ?
By the way, I realized that I made a small mistake in my previous example: what should be retrieved is "A3 B1" and "A12 B2" (and not "A12 B1"), sorry for the confusion.

Out of curiosity, is there a way to achieve this without using the match-recognize ? Any way using the prev or prior function ?

There is a way but it is really ugly and I really would like to find an easier solution. Here is what we have right now:
select * from B as evtB unidirectional, pattern[every A[5] as evtA]

This seems to work fine but it is really complex. Any other better looking way :-) ?

Thanks
Cédric

At 19:24 8/12/2009, you wrote:

Hi Cedrik,

here is a possible match-recognize (the regular expression only):
- A3 B1: (A X X X X B), select A and B
- A12 B1: (B X X X X A), select A and B

or both at the same time "(A X X X X B) | (B X X X X A)"

Best regards
Tom

From: Cédric Moonen <cedric.moonen@...>
To: user@...
Sent: Tue, December 8, 2009 9:07:18 AM
Subject: Re: [esper-user] Retrieving events prior an event

Thanks for the reply,

But I don't think this will really solve my problem. Let me show you with an example, which will probably be clearer than my explanation. Suppose the following sequence of events:
A1 A2 A3 A4 A5 A6 A7 B1 A8 A9 A10 A11 A12 A13 A14 A15 A16 B2 A17 A18

What I want to retrieve is:
- A3 B1 (A3 being the fifth A event before B10)
- A12 B1

I don't think this is really feasible using a pattern, is it ?

At 14:51 8/12/2009, you wrote:

May be a pattern using repeat or repeat until would be a better way?

2009/12/8 Cédric Moonen <cedric.moonen@... >

Hello,
I'm a bit stuck with a problem when trying to retrieve events that happened before a specific event. My scenario is that I have events coming in and all those events have the same base event (let's call it BaseEvent). Then, there are some specific events that I am looking for (SpecificEvent). I need to retrieve all those specific events with the base event that occured 5th-prior that specific event.
The first thing I did was something like this:
select * from SpecificEvent as event unidirectional, BaseEvent.win:length(5).std:firstevent() as prevEvent
But this didn't work as I expected and looking at the documentation of the std:firstevent, it is explained there that I should use the prev function instead because the window is restarted each time. So, I tried then to use the prev function but there's no explanation in the doc on how I have to specify that I want the full event and not one of the properties (although, it is stated that it should be possible). I tried to write something like this:
select event, prev(5, *) as prevEvent from SpecificEvent as event unidirectional, BaseEvent.win:length(5)
But it complains about the syntax error for the '*' symbol.
So my questions are:
1) How to write suche a statement ?
2) If there are multiple streams, how will the prev function "know" for which stream it has to select the event ?
I'm a bit confused with this prev function.
Thanks for your feedback.
Cédric

---------------------------------------------------------------------
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email

Cédric Moonen
Projectingenieur

Flanders' MECHATRONICS Technology Centre (FMTC)
Celestijnenlaan 300D
B-3001 Leuven
Belgium

GSM: +32-498-91.94.14
Fax: +32-16-32.80.64
cedric.moonen@...

www.fmtc.be

Cédric Moonen
Projectingenieur

Flanders' MECHATRONICS Technology Centre (FMTC)
Celestijnenlaan 300D
B-3001 Leuven
Belgium

GSM: +32-498-91.94.14
Fax: +32-16-32.80.64
cedric.moonen@...

www.fmtc.be

Cédric Moonen
Projectingenieur

Flanders' MECHATRONICS Technology Centre (FMTC)
Celestijnenlaan 300D
B-3001 Leuven
Belgium

GSM: +32-498-91.94.14
Fax: +32-16-32.80.64
cedric.moonen@...

www.fmtc.be

Re: Stuck at NStreamQueryPlanBuilder.computeBestPath

2009-12-09T18:56:20Z

If your 15 subqueries are very similar, perhaps a single subquery and a custom plugin aggregation function would be another way to go, function to simply return all x events in an array

For example:

select
(select myaggregation(value) from Event.std:unique(value)), j.value
from Event(id=14).std.lastevent() as j

Best regards
Tom

From: pedroteixeira <pedro.t@...>
To: user@...
Sent: Wed, December 9, 2009 2:04:42 PM
Subject: Re: [esper-user] Stuck at NStreamQueryPlanBuilder.computeBestPath

Thanks, Thomas. Will create a test case and then create the issue.

I'was reading the example in the docs:
// Fast version, second alternative
select
(select avg(value) from MyEvent.win:length(100)) as avgA,
(select avg(value) from MyEvent.win:length(50)) as avgB,
a.symbol
from MyEvent.std:lastevent() a, MyEvent.std:lastevent() b

For a 15-stream join, is this the subquery approach that you meant?

select
(select value from Event(id=1).std:lastevent()),
(select value from Event(id=2).std:lastevent()),
...
(select value from Event(id=13).std:lastevent()),
j.value
from Event(id=14).std.lastevent() as j

cheers,
Pedro

Thomas Bernhardt wrote:

>
> Hi,
> can you please create a JIRA and attach the test Java class.
> As a workaround to a 15-stream join consider subqueries.
>
>
> Best regards
> Tom
>
>
> ________________________________
> From: pedroteixeira <pedro.t@...>
> To: user@...
> Sent: Wed, December 9, 2009 8:10:18 AM
> Subject: Re: [esper-user] Stuck at NStreamQueryPlanBuilder.computeBestPath
>
>
>
> TYPO. The actual query is more like this:
>
> select a.value, b.value, c.value ... j.value from Stream(id =
> 1).std:lastevent() as a, Stream(id = 2).std:lastevent() as b, ...,
> Stream(id
> =10).std:lastevent() as j
>
>
>
> pedroteixeira wrote:
>>
>>
>> Hello,
>>
>> I've stumbled across a problem, but need some help to understand what it
>> is the intended behavior of the NStreamQueryPlanBuilder class. Perhaps,
>> someone has already had this problem, but I was not able to find in the
>> archive.
>>
>>
>> I have a query which joins 10 streams in the following manner:
>>
>> select a.value, b.value, c.value ... j.value from Stream(id = 1) as a,
>> Stream(id = 2) as b, ..., Stream(id =10) as j
>>
>>
>> The esper engine gets stuck in the "computeBestPath" method, consuming
>> one
>> of the CPUs allocated to the running thread. With yourkit, I extracted
>> the
>> following stack trace:
>>
>>
>> com.espertech.esper.collection.PermutationEnumeration.nextElement()
>> com.espertech.esper.collection.NumberSetPermutationEnumeration.nextElement()
>> com.espertech.esper.epl.join.plan.NStreamQueryPlanBuilder.computeBestPath(int,
>> QueryGraph, HistoricalDependencyGraph)
>> com.espertech.esper.epl.join.plan.NStreamQueryPlanBuilder.build(QueryGraph,
>> EventType[], boolean, boolean[], HistoricalDependencyGraph,
>> HistoricalStreamIndexList[])
>> com.espertech.esper.epl.join.plan.QueryPlanBuilder.getPlan(EventType[],
>> List, QueryGraph, String[], boolean, boolean[],
>> HistoricalDependencyGraph, HistoricalStreamIndexList[],
>> ExprEvaluatorContext)
>> com.espertech.esper.epl.join.JoinSetComposerFactoryImpl.makeComposer(List,
>> ExprNode, EventType[], String[], Viewable[],
>> SelectClauseStreamSelectorEnum, StreamJoinAnalysisResult,
>> ExprEvaluatorContext)
>> com.espertech.esper.core.EPStatementStartMethod.handleJoin(String[],
>> EventType[], Viewable[], ResultSetProcessor,
>> SelectClauseStreamSelectorEnum, StatementContext, List,
>> StreamJoinAnalysisResult)
>> com.espertech.esper.core.EPStatementStartMethod.startSelect(boolean)
>> com.espertech.esper.core.EPStatementStartMethod.start(boolean, boolean,
>> boolean)
>> com.espertech.esper.core.StatementLifecycleSvcImpl.startInternal(String,
>> StatementLifecycleSvcImpl$EPStatementDesc, boolean, boolean, boolean)
>> com.espertech.esper.core.StatementLifecycleSvcImpl.start(String,
>> StatementLifecycleSvcImpl$EPStatementDesc, boolean, boolean, boolean)
>> com.espertech.esper.core.StatementLifecycleSvcImpl.createAndStart(StatementSpecRaw,
>> String, boolean, String, String, Map, Object, EPIsolationUnitServices)
>> com.espertech.esper.core.StatementLifecycleSvcImpl.createAndStart(StatementSpecRaw,
>> String, boolean, String, Object, EPIsolationUnitServices)
>> com.espertech.esper.core.EPAdministratorImpl.create(EPStatementObjectModel,
>> String)
>> com.espertech.esper.core.EPAdministratorImpl.create(EPStatementObjectModel)
>>
>>
>>
>> There seems to be some sort of caveat in the "computeBestPath" method,
>> since it might have to try 15! combinations (in my case). Is that really
>> intented?
>>
>> Thanks a bunch,
>> Pedro
>>
>>
>>
>
> --
> View this message in context:

> http://old.nabble.com/Stuck-at-NStreamQueryPlanBuilder.computeBestPath-tp26709887p26709902.html
> Sent from the Esper User list mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
> http://xircles.codehaus.org/manage_email
>
>
>
>

--
View this message in context: http://old.nabble.com/Stuck-at-NStreamQueryPlanBuilder.computeBestPath-tp26709887p26715790.html
Sent from the Esper User list mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

Re: Stuck at NStreamQueryPlanBuilder.computeBestPath

2009-12-09T11:04:41Z

Thanks, Thomas. Will create a test case and then create the issue.

I'was reading the example in the docs:
// Fast version, second alternative
select
(select avg(value) from MyEvent.win:length(100)) as avgA,
(select avg(value) from MyEvent.win:length(50)) as avgB,
a.symbol
from MyEvent.std:lastevent() a, MyEvent.std:lastevent() b

For a 15-stream join, is this the subquery approach that you meant?

select
(select value from Event(id=1).std:lastevent()),
(select value from Event(id=2).std:lastevent()),
...
(select value from Event(id=13).std:lastevent()),
j.value
from Event(id=14).std.lastevent() as j

cheers,
Pedro

Thomas Bernhardt wrote:

Hi,
can you please create a JIRA and attach the test Java class.
As a workaround to a 15-stream join consider subqueries.

Best regards
Tom

________________________________
From: pedroteixeira <pedro.t@gmail.com>
To: user@esper.codehaus.org
Sent: Wed, December 9, 2009 8:10:18 AM
Subject: Re: [esper-user] Stuck at NStreamQueryPlanBuilder.computeBestPath

TYPO. The actual query is more like this:

select a.value, b.value, c.value ... j.value from Stream(id =
1).std:lastevent() as a, Stream(id = 2).std:lastevent() as b, ..., Stream(id
=10).std:lastevent() as j

pedroteixeira wrote:
>
>
> Hello,
>
> I've stumbled across a problem, but need some help to understand what it
> is the intended behavior of the NStreamQueryPlanBuilder class. Perhaps,
> someone has already had this problem, but I was not able to find in the
> archive.
>
>
> I have a query which joins 10 streams in the following manner:
>
> select a.value, b.value, c.value ... j.value from Stream(id = 1) as a,
> Stream(id = 2) as b, ..., Stream(id =10) as j
>
>
> The esper engine gets stuck in the "computeBestPath" method, consuming one
> of the CPUs allocated to the running thread. With yourkit, I extracted the
> following stack trace:
>
>
> com.espertech.esper.collection.PermutationEnumeration.nextElement()
> com.espertech.esper.collection.NumberSetPermutationEnumeration.nextElement()
> com.espertech.esper.epl.join.plan.NStreamQueryPlanBuilder.computeBestPath(int,
> QueryGraph, HistoricalDependencyGraph)
> com.espertech.esper.epl.join.plan.NStreamQueryPlanBuilder.build(QueryGraph,
> EventType[], boolean, boolean[], HistoricalDependencyGraph,
> HistoricalStreamIndexList[])
> com.espertech.esper.epl.join.plan.QueryPlanBuilder.getPlan(EventType[],
> List, QueryGraph, String[], boolean, boolean[],
> HistoricalDependencyGraph, HistoricalStreamIndexList[],
> ExprEvaluatorContext)
> com.espertech.esper.epl.join.JoinSetComposerFactoryImpl.makeComposer(List,
> ExprNode, EventType[], String[], Viewable[],
> SelectClauseStreamSelectorEnum, StreamJoinAnalysisResult,
> ExprEvaluatorContext)
> com.espertech.esper.core.EPStatementStartMethod.handleJoin(String[],
> EventType[], Viewable[], ResultSetProcessor,
> SelectClauseStreamSelectorEnum, StatementContext, List,
> StreamJoinAnalysisResult)
> com.espertech.esper.core.EPStatementStartMethod.startSelect(boolean)
> com.espertech.esper.core.EPStatementStartMethod.start(boolean, boolean,
> boolean)
> com.espertech.esper.core.StatementLifecycleSvcImpl.startInternal(String,
> StatementLifecycleSvcImpl$EPStatementDesc, boolean, boolean, boolean)
> com.espertech.esper.core.StatementLifecycleSvcImpl.start(String,
> StatementLifecycleSvcImpl$EPStatementDesc, boolean, boolean, boolean)
> com.espertech.esper.core.StatementLifecycleSvcImpl.createAndStart(StatementSpecRaw,
> String, boolean, String, String, Map, Object, EPIsolationUnitServices)
> com.espertech.esper.core.StatementLifecycleSvcImpl.createAndStart(StatementSpecRaw,
> String, boolean, String, Object, EPIsolationUnitServices)
> com.espertech.esper.core.EPAdministratorImpl.create(EPStatementObjectModel,
> String)
> com.espertech.esper.core.EPAdministratorImpl.create(EPStatementObjectModel)
>
>
>
> There seems to be some sort of caveat in the "computeBestPath" method,
> since it might have to try 15! combinations (in my case). Is that really
> intented?
>
> Thanks a bunch,
> Pedro
>
>
>

--
View this message in context: http://old.nabble.com/Stuck-at-NStreamQueryPlanBuilder.computeBestPath-tp26709887p26709902.html
Sent from the Esper User list mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

Re: Stuck at NStreamQueryPlanBuilder.computeBestPath

2009-12-09T09:28:00Z

Hi,
can you please create a JIRA and attach the test Java class.
As a workaround to a 15-stream join consider subqueries.

Best regards
Tom

From: pedroteixeira <pedro.t@...>
To: user@...
Sent: Wed, December 9, 2009 8:10:18 AM
Subject: Re: [esper-user] Stuck at NStreamQueryPlanBuilder.computeBestPath

TYPO. The actual query is more like this:

select a.value, b.value, c.value ... j.value from Stream(id =
1).std:lastevent() as a, Stream(id = 2).std:lastevent() as b, ..., Stream(id
=10).std:lastevent() as j

pedroteixeira wrote:

>
>
> Hello,
>
> I've stumbled across a problem, but need some help to understand what it
> is the intended behavior of the NStreamQueryPlanBuilder class. Perhaps,
> someone has already had this problem, but I was not able to find in the
> archive.
>
>
> I have a query which joins 10 streams in the following manner:
>
> select a.value, b.value, c.value ... j.value from Stream(id = 1) as a,
> Stream(id = 2) as b, ..., Stream(id =10) as j
>
>
> The esper engine gets stuck in the "computeBestPath" method, consuming one
> of the CPUs allocated to the running thread. With yourkit, I extracted the
> following stack trace:
>
>
> com.espertech.esper.collection.PermutationEnumeration.nextElement()
> com.espertech.esper.collection.NumberSetPermutationEnumeration.nextElement()
> com.espertech.esper.epl.join.plan.NStreamQueryPlanBuilder.computeBestPath(int,
> QueryGraph, HistoricalDependencyGraph)
> com.espertech.esper.epl.join.plan.NStreamQueryPlanBuilder.build(QueryGraph,
> EventType[], boolean, boolean[], HistoricalDependencyGraph,
> HistoricalStreamIndexList[])
> com.espertech.esper.epl.join.plan.QueryPlanBuilder.getPlan(EventType[],
> List, QueryGraph, String[], boolean, boolean[],
> HistoricalDependencyGraph, HistoricalStreamIndexList[],
> ExprEvaluatorContext)
> com.espertech.esper.epl.join.JoinSetComposerFactoryImpl.makeComposer(List,
> ExprNode, EventType[], String[], Viewable[],
> SelectClauseStreamSelectorEnum, StreamJoinAnalysisResult,
> ExprEvaluatorContext)
> com.espertech.esper.core.EPStatementStartMethod.handleJoin(String[],
> EventType[], Viewable[], ResultSetProcessor,
> SelectClauseStreamSelectorEnum, StatementContext, List,
> StreamJoinAnalysisResult)
> com.espertech.esper.core.EPStatementStartMethod.startSelect(boolean)
> com.espertech.esper.core.EPStatementStartMethod.start(boolean, boolean,
> boolean)
> com.espertech.esper.core.StatementLifecycleSvcImpl.startInternal(String,
> StatementLifecycleSvcImpl$EPStatementDesc, boolean, boolean, boolean)
> com.espertech.esper.core.StatementLifecycleSvcImpl.start(String,
> StatementLifecycleSvcImpl$EPStatementDesc, boolean, boolean, boolean)
> com.espertech.esper.core.StatementLifecycleSvcImpl.createAndStart(StatementSpecRaw,
> String, boolean, String, String, Map, Object, EPIsolationUnitServices)
> com.espertech.esper.core.StatementLifecycleSvcImpl.createAndStart(StatementSpecRaw,
> String, boolean, String, Object, EPIsolationUnitServices)
> com.espertech.esper.core.EPAdministratorImpl.create(EPStatementObjectModel,
> String)
> com.espertech.esper.core.EPAdministratorImpl.create(EPStatementObjectModel)
>
>
>
> There seems to be some sort of caveat in the "computeBestPath" method,
> since it might have to try 15! combinations (in my case). Is that really
> intented?
>
> Thanks a bunch,
> Pedro
>
>
>

--
View this message in context: http://old.nabble.com/Stuck-at-NStreamQueryPlanBuilder.computeBestPath-tp26709887p26709902.html
Sent from the Esper User list mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

Re: Stuck at NStreamQueryPlanBuilder.computeBestPath

2009-12-09T05:10:18Z

TYPO. The actual query is more like this:

select a.value, b.value, c.value ... j.value from Stream(id = 1).std:lastevent() as a, Stream(id = 2).std:lastevent() as b, ..., Stream(id =10).std:lastevent() as j

pedroteixeira wrote:

Hello,

I've stumbled across a problem, but need some help to understand what it is the intended behavior of the NStreamQueryPlanBuilder class. Perhaps, someone has already had this problem, but I was not able to find in the archive.

I have a query which joins 10 streams in the following manner:

select a.value, b.value, c.value ... j.value from Stream(id = 1) as a, Stream(id = 2) as b, ..., Stream(id =10) as j

The esper engine gets stuck in the "computeBestPath" method, consuming one of the CPUs allocated to the running thread. With yourkit, I extracted the following stack trace:

com.espertech.esper.collection.PermutationEnumeration.nextElement()
com.espertech.esper.collection.NumberSetPermutationEnumeration.nextElement()
com.espertech.esper.epl.join.plan.NStreamQueryPlanBuilder.computeBestPath(int,
QueryGraph, HistoricalDependencyGraph)
com.espertech.esper.epl.join.plan.NStreamQueryPlanBuilder.build(QueryGraph,
EventType[], boolean, boolean[], HistoricalDependencyGraph,
HistoricalStreamIndexList[])
com.espertech.esper.epl.join.plan.QueryPlanBuilder.getPlan(EventType[],
List, QueryGraph, String[], boolean, boolean[],
HistoricalDependencyGraph, HistoricalStreamIndexList[],
ExprEvaluatorContext)
com.espertech.esper.epl.join.JoinSetComposerFactoryImpl.makeComposer(List,
ExprNode, EventType[], String[], Viewable[],
SelectClauseStreamSelectorEnum, StreamJoinAnalysisResult,
ExprEvaluatorContext)
com.espertech.esper.core.EPStatementStartMethod.handleJoin(String[],
EventType[], Viewable[], ResultSetProcessor,
SelectClauseStreamSelectorEnum, StatementContext, List,
StreamJoinAnalysisResult)
com.espertech.esper.core.EPStatementStartMethod.startSelect(boolean)
com.espertech.esper.core.EPStatementStartMethod.start(boolean, boolean, boolean)
com.espertech.esper.core.StatementLifecycleSvcImpl.startInternal(String,
StatementLifecycleSvcImpl$EPStatementDesc, boolean, boolean, boolean)
com.espertech.esper.core.StatementLifecycleSvcImpl.start(String,
StatementLifecycleSvcImpl$EPStatementDesc, boolean, boolean, boolean)
com.espertech.esper.core.StatementLifecycleSvcImpl.createAndStart(StatementSpecRaw,
String, boolean, String, String, Map, Object, EPIsolationUnitServices)
com.espertech.esper.core.StatementLifecycleSvcImpl.createAndStart(StatementSpecRaw,
String, boolean, String, Object, EPIsolationUnitServices)
com.espertech.esper.core.EPAdministratorImpl.create(EPStatementObjectModel,
String)
com.espertech.esper.core.EPAdministratorImpl.create(EPStatementObjectModel)

There seems to be some sort of caveat in the "computeBestPath" method, since it might have to try 15! combinations (in my case). Is that really intented?

Thanks a bunch,
Pedro

Stuck at NStreamQueryPlanBuilder.computeBestPath

2009-12-09T05:08:58Z

Hello,

I've stumbled across a problem, but need some help to understand what it is the intended behavior of the NStreamQueryPlanBuilder class. Perhaps, someone has already had this problem, but I was not able to find in the archive.

I have a query which joins 10 streams in the following manner:

select a.value, b.value, c.value ... j.value from Stream(id = 1) as a, Stream(id = 2) as b, ..., Stream(id =10) as j

The esper engine gets stuck in the "computeBestPath" method, consuming one of the CPUs allocated to the running thread. With yourkit, I extracted the following stack trace:

com.espertech.esper.collection.PermutationEnumeration.nextElement()
com.espertech.esper.collection.NumberSetPermutationEnumeration.nextElement()
com.espertech.esper.epl.join.plan.NStreamQueryPlanBuilder.computeBestPath(int,
QueryGraph, HistoricalDependencyGraph)
com.espertech.esper.epl.join.plan.NStreamQueryPlanBuilder.build(QueryGraph,
EventType[], boolean, boolean[], HistoricalDependencyGraph,
HistoricalStreamIndexList[])
com.espertech.esper.epl.join.plan.QueryPlanBuilder.getPlan(EventType[],
List, QueryGraph, String[], boolean, boolean[],
HistoricalDependencyGraph, HistoricalStreamIndexList[],
ExprEvaluatorContext)
com.espertech.esper.epl.join.JoinSetComposerFactoryImpl.makeComposer(List,
ExprNode, EventType[], String[], Viewable[],
SelectClauseStreamSelectorEnum, StreamJoinAnalysisResult,
ExprEvaluatorContext)
com.espertech.esper.core.EPStatementStartMethod.handleJoin(String[],
EventType[], Viewable[], ResultSetProcessor,
SelectClauseStreamSelectorEnum, StatementContext, List,
StreamJoinAnalysisResult)
com.espertech.esper.core.EPStatementStartMethod.startSelect(boolean)
com.espertech.esper.core.EPStatementStartMethod.start(boolean, boolean, boolean)
com.espertech.esper.core.StatementLifecycleSvcImpl.startInternal(String,
StatementLifecycleSvcImpl$EPStatementDesc, boolean, boolean, boolean)
com.espertech.esper.core.StatementLifecycleSvcImpl.start(String,
StatementLifecycleSvcImpl$EPStatementDesc, boolean, boolean, boolean)
com.espertech.esper.core.StatementLifecycleSvcImpl.createAndStart(StatementSpecRaw,
String, boolean, String, String, Map, Object, EPIsolationUnitServices)
com.espertech.esper.core.StatementLifecycleSvcImpl.createAndStart(StatementSpecRaw,
String, boolean, String, Object, EPIsolationUnitServices)
com.espertech.esper.core.EPAdministratorImpl.create(EPStatementObjectModel,
String)
com.espertech.esper.core.EPAdministratorImpl.create(EPStatementObjectModel)

There seems to be some sort of caveat in the "computeBestPath" method, since it might have to try 15! combinations (in my case). Is that really intented?

Thanks a bunch,
Pedro

Re: Retrieving events prior an event

2009-12-09T03:12:41Z

I made a mistake in my previous reply. The statement that is working looks like this in fact:
select * from B as evtB unidirectional, pattern[every evtA=A -> A -> A -> A -> A]

Which is even more ugly...

I've been looking at the mathc_recognize but as I expected I am a bit stuck on detecting the type of the event in the define clause. What I have so far is:

select * from BaseEvent
match_recognize (
measures A as start, B as stop
pattern(A X X X X B)
define
B as (??? AND B.var1="Val1" AND B.var2="Val2" )
)

So, to summarize, both A and B inherit from BaseEvent. The "???" in the define should be replaced by something that specifies the type of the event B (event A can be of any type). var1 and var2 are both members of the more derived type which B is representing. Is there any way to do this ?

Thanks for any feedback.
Cédric

At 11:02 9/12/2009, you wrote:

Hello Thomas,

I've looked at the documentation of Esper 3.2 and this is a new feature (we were still using 3.1). It looks indeed very promising to use the match-recognize. I will have to experiment with it but I think what might be a problem (not sure for now) is that you can only work with one type of events. Luckily, all our events have the same common base class so there is probably a work around here if we are able to specify a more defined type in the define clause. Is that possible ?
By the way, I realized that I made a small mistake in my previous example: what should be retrieved is "A3 B1" and "A12 B2" (and not "A12 B1"), sorry for the confusion.

Out of curiosity, is there a way to achieve this without using the match-recognize ? Any way using the prev or prior function ?

There is a way but it is really ugly and I really would like to find an easier solution. Here is what we have right now:
select * from B as evtB unidirectional, pattern[every A[5] as evtA]

This seems to work fine but it is really complex. Any other better looking way :-) ?

Thanks
Cédric

At 19:24 8/12/2009, you wrote:

Hi Cedrik,

here is a possible match-recognize (the regular expression only):
- A3 B1: (A X X X X B), select A and B
- A12 B1: (B X X X X A), select A and B

or both at the same time "(A X X X X B) | (B X X X X A)"

Best regards
Tom

From: Cédric Moonen <cedric.moonen@...>
To: user@...
Sent: Tue, December 8, 2009 9:07:18 AM
Subject: Re: [esper-user] Retrieving events prior an event

Thanks for the reply,

But I don't think this will really solve my problem. Let me show you with an example, which will probably be clearer than my explanation. Suppose the following sequence of events:
A1 A2 A3 A4 A5 A6 A7 B1 A8 A9 A10 A11 A12 A13 A14 A15 A16 B2 A17 A18

What I want to retrieve is:
- A3 B1 (A3 being the fifth A event before B10)
- A12 B1

I don't think this is really feasible using a pattern, is it ?

At 14:51 8/12/2009, you wrote:

May be a pattern using repeat or repeat until would be a better way?

2009/12/8 Cédric Moonen <cedric.moonen@... >

Hello,
I'm a bit stuck with a problem when trying to retrieve events that happened before a specific event. My scenario is that I have events coming in and all those events have the same base event (let's call it BaseEvent). Then, there are some specific events that I am looking for (SpecificEvent). I need to retrieve all those specific events with the base event that occured 5th-prior that specific event.
The first thing I did was something like this:
select * from SpecificEvent as event unidirectional, BaseEvent.win:length(5).std:firstevent() as prevEvent
But this didn't work as I expected and looking at the documentation of the std:firstevent, it is explained there that I should use the prev function instead because the window is restarted each time. So, I tried then to use the prev function but there's no explanation in the doc on how I have to specify that I want the full event and not one of the properties (although, it is stated that it should be possible). I tried to write something like this:
select event, prev(5, *) as prevEvent from SpecificEvent as event unidirectional, BaseEvent.win:length(5)
But it complains about the syntax error for the '*' symbol.
So my questions are:
1) How to write suche a statement ?
2) If there are multiple streams, how will the prev function "know" for which stream it has to select the event ?
I'm a bit confused with this prev function.
Thanks for your feedback.
Cédric

---------------------------------------------------------------------
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email

Cédric Moonen
Projectingenieur

Flanders' MECHATRONICS Technology Centre (FMTC)
Celestijnenlaan 300D
B-3001 Leuven
Belgium

GSM: +32-498-91.94.14
Fax: +32-16-32.80.64
cedric.moonen@...

www.fmtc.be

Cédric Moonen
Projectingenieur

Flanders' MECHATRONICS Technology Centre (FMTC)
Celestijnenlaan 300D
B-3001 Leuven
Belgium

GSM: +32-498-91.94.14
Fax: +32-16-32.80.64
cedric.moonen@...

www.fmtc.be

Cédric Moonen
Projectingenieur

Flanders' MECHATRONICS Technology Centre (FMTC)
Celestijnenlaan 300D
B-3001 Leuven
Belgium

GSM: +32-498-91.94.14
Fax: +32-16-32.80.64
cedric.moonen@...

www.fmtc.be

Re: Retrieving events prior an event

2009-12-09T02:02:00Z

Hello Thomas,

I've looked at the documentation of Esper 3.2 and this is a new feature (we were still using 3.1). It looks indeed very promising to use the match-recognize. I will have to experiment with it but I think what might be a problem (not sure for now) is that you can only work with one type of events. Luckily, all our events have the same common base class so there is probably a work around here if we are able to specify a more defined type in the define clause. Is that possible ?
By the way, I realized that I made a small mistake in my previous example: what should be retrieved is "A3 B1" and "A12 B2" (and not "A12 B1"), sorry for the confusion.

Out of curiosity, is there a way to achieve this without using the match-recognize ? Any way using the prev or prior function ?

There is a way but it is really ugly and I really would like to find an easier solution. Here is what we have right now:
select * from B as evtB unidirectional, pattern[every A[5] as evtA]

This seems to work fine but it is really complex. Any other better looking way :-) ?

Thanks
Cédric

At 19:24 8/12/2009, you wrote:

Hi Cedrik,

here is a possible match-recognize (the regular expression only):
- A3 B1: (A X X X X B), select A and B
- A12 B1: (B X X X X A), select A and B

or both at the same time "(A X X X X B) | (B X X X X A)"

Best regards
Tom

From: Cédric Moonen <cedric.moonen@...>
To: user@...
Sent: Tue, December 8, 2009 9:07:18 AM
Subject: Re: [esper-user] Retrieving events prior an event

Thanks for the reply,

But I don't think this will really solve my problem. Let me show you with an example, which will probably be clearer than my explanation. Suppose the following sequence of events:
A1 A2 A3 A4 A5 A6 A7 B1 A8 A9 A10 A11 A12 A13 A14 A15 A16 B2 A17 A18

What I want to retrieve is:
- A3 B1 (A3 being the fifth A event before B10)
- A12 B1

I don't think this is really feasible using a pattern, is it ?

At 14:51 8/12/2009, you wrote:

May be a pattern using repeat or repeat until would be a better way?

2009/12/8 Cédric Moonen <cedric.moonen@... >

Hello,

I'm a bit stuck with a problem when trying to retrieve events that happened before a specific event. My scenario is that I have events coming in and all those events have the same base event (let's call it BaseEvent). Then, there are some specific events that I am looking for (SpecificEvent). I need to retrieve all those specific events with the base event that occured 5th-prior that specific event.

The first thing I did was something like this:
select * from SpecificEvent as event unidirectional, BaseEvent.win:length(5).std:firstevent() as prevEvent

But this didn't work as I expected and looking at the documentation of the std:firstevent, it is explained there that I should use the prev function instead because the window is restarted each time. So, I tried then to use the prev function but there's no explanation in the doc on how I have to specify that I want the full event and not one of the properties (although, it is stated that it should be possible). I tried to write something like this:

select event, prev(5, *) as prevEvent from SpecificEvent as event unidirectional, BaseEvent.win:length(5)

But it complains about the syntax error for the '*' symbol.

So my questions are:
1) How to write suche a statement ?
2) If there are multiple streams, how will the prev function "know" for which stream it has to select the event ?

I'm a bit confused with this prev function.

Thanks for your feedback.
Cédric

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

Cédric Moonen
Projectingenieur

Flanders' MECHATRONICS Technology Centre (FMTC)
Celestijnenlaan 300D
B-3001 Leuven
Belgium

GSM: +32-498-91.94.14
Fax: +32-16-32.80.64
cedric.moonen@...

www.fmtc.be

Cédric Moonen
Projectingenieur

Flanders' MECHATRONICS Technology Centre (FMTC)
Celestijnenlaan 300D
B-3001 Leuven
Belgium

GSM: +32-498-91.94.14
Fax: +32-16-32.80.64
cedric.moonen@...

www.fmtc.be

Re: match_recognize clause with aggregations

2009-12-08T18:34:09Z

Yes the define-clause does not allow aggregation functions, we'll update the doc and improve error handling.

Best regards
Tom

From: Sandeep Poreddy <sandeep.poreddy@...>
To: user@...
Sent: Tue, December 8, 2009 1:06:57 PM
Subject: [esper-user] match_recognize clause with aggregations

Hello,

Need some help with match_recognize clause.

Consider the following EPL:

stmt2 = SELECT id, outsideTemp, timestamp1,tempString FROM OutputEventsOne

match_recognize(

measures A.id as id, A.outsideTemp as outsideTemp,

A.timestamp1 as a_timestamp1, sum(B.timestamp1 - A.timestamp1) as timestamp1, A.tempString as tempString

all matches

after match skip to current row

pattern ( A B+)

define

A as (A.tempString = 'LESS' ),

B as ( B.tempString = 'LESS') AND sum(B.timestamp1 - A.timestamp1) >3000);

The statement gets registered with the esper engine without any problem. But an exception is thrown when I run the application. I see that no exception is thrown when I remove the aggregate ‘sum’ from the statement. Is it not legal to use aggregations in the define part of match recognize clause?.

SEVERE: ##

com.espertech.esper.client.EPException

Caused by: java.lang.NullPointerException

at com.espertech.esper.epl.expression.ExprAggregateNode.evaluate(ExprAggregateNode.java:136)

at com.espertech.esper.epl.expression.ExprRelationalOpNode.evaluate(ExprRelationalOpNode.java:92)

at com.espertech.esper.epl.expression.ExprAndNode.evaluate(ExprAndNode.java:58)

at com.espertech.esper.rowregex.RegexNFAStateOneToMany.matches(RegexNFAStateOneToMany.java:36)

at com.espertech.esper.rowregex.EventRowRegexNFAView.step(EventRowRegexNFAView.java:792)

at com.espertech.esper.rowregex.EventRowRegexNFAView.update(EventRowRegexNFAView.java:305)

at com.espertech.esper.view.ZeroDepthStream.insert(ZeroDepthStream.java:70)

at com.espertech.esper.view.stream.StreamFactorySvcImpl$2.matchFound(StreamFactorySvcImpl.java:166)

at com.espertech.esper.core.EPRuntimeImpl.processStatementFilterSingle(EPRuntimeImpl.java:1031)

at com.espertech.esper.core.EPRuntimeImpl.processMatches(EPRuntimeImpl.java:824)

at com.espertech.esper.core.EPRuntimeImpl.processWrappedEvent(EPRuntimeImpl.java:398)

... 11 more

Thanks,

Sandeep

Re: Real-time hierarchical aggregated statistics

2009-12-08T18:27:12Z

Do you want to provide a little test class to use, with the SQL removed, we can look into.

Best regards
Tom

From: António Alegria <antoniomcalegria@...>
To: user@...
Sent: Tue, December 8, 2009 4:20:57 PM
Subject: Re: [esper-user] Real-time hierarchical aggregated statistics

The SQL fires only once (the pattern has no every) and the named window is a keepall, yes. That query does not return too many rows though.

On Sun, Dec 6, 2009 at 9:32 PM, Thomas Bernhardt <bernhardttom@...> wrote:

I'd recommend reading up on the performance tips in the section.
The named window is a keep-all policy and retains all events? You may want to retain only the last event per id.
The SQL query fires every 1 seconds and fills the named window?
Best regards
Tom
From: António Alegria <antoniomcalegria@...>
To: user@...
Sent: Fri, December 4, 2009 6:43:56 AM
Subject: Re: [esper-user] Real-time hierarchical aggregated statistics

Well, with 10 "StockTick" events per second esper starts to fall behind in updating all the tree in real time. My interest here is in to know if there is a more intelligent way to approaching this problem or if this is as good as it gets.

Thanks,

António Alegria

On Thu, Dec 3, 2009 at 9:27 PM, Alexandre Vasseur <avasseur@...> wrote:

What do you mean exactly with "isn't scalable" and "ineficient" ?
Alex

2009/12/3 António Alegria <antoniomcalegria@...>

Hi, we have the following problem we are trying to solve with Esper. We have, in fact, reached a solution but it isn't scalable. The problem has nothing to do with the stockmarket but I will present it as such.

Suppose Esper receives several StockTick events, with the following fields:

StockTick:
* symbol: string (e.g. "MSFT", "APPL", "GOOG", ...)

* price: float

Furthermore, each StockTick symbol can be associated with grouping hierarchy, such as business field, city, region, country, continent. This grouping information is collected from an SQL query and put in a named window with the following fields:

GroupTree:
* group: string
* member: string

Example events in this named windows are: ["USA", "Washington"], ["Washington", "Redmond"], ["IT", "MSFT"].

We want to develop EPL rules that, in real-time and for each arriving StockTick, calculate the average price for each category in this grouping hierarchy (e.g. average for each country, each region, etc...).

You can see our current (ineficient solution) in https://gist.github.com/f1bec8803ee7de907570

I would greatly appreciate it if someone could tell me the "right way" to do this.

Thank you,

António Alegria

Re: Real-time hierarchical aggregated statistics

2009-12-08T13:20:57Z

The SQL fires only once (the pattern has no every) and the named window is a keepall, yes. That query does not return too many rows though.

On Sun, Dec 6, 2009 at 9:32 PM, Thomas Bernhardt <bernhardttom@...> wrote:

I'd recommend reading up on the performance tips in the section.
The named window is a keep-all policy and retains all events? You may want to retain only the last event per id.
The SQL query fires every 1 seconds and fills the named window?
Best regards
Tom
From: António Alegria <antoniomcalegria@...>
To: user@...
Sent: Fri, December 4, 2009 6:43:56 AM
Subject: Re: [esper-user] Real-time hierarchical aggregated statistics

Well, with 10 "StockTick" events per second esper starts to fall behind in updating all the tree in real time. My interest here is in to know if there is a more intelligent way to approaching this problem or if this is as good as it gets.

Thanks,

António Alegria

On Thu, Dec 3, 2009 at 9:27 PM, Alexandre Vasseur <avasseur@...> wrote:

What do you mean exactly with "isn't scalable" and "ineficient" ?
Alex

2009/12/3 António Alegria <antoniomcalegria@...>

Hi, we have the following problem we are trying to solve with Esper. We have, in fact, reached a solution but it isn't scalable. The problem has nothing to do with the stockmarket but I will present it as such.

Suppose Esper receives several StockTick events, with the following fields:

StockTick:
* symbol: string (e.g. "MSFT", "APPL", "GOOG", ...)

* price: float

Furthermore, each StockTick symbol can be associated with grouping hierarchy, such as business field, city, region, country, continent. This grouping information is collected from an SQL query and put in a named window with the following fields:

GroupTree:
* group: string
* member: string

Example events in this named windows are: ["USA", "Washington"], ["Washington", "Redmond"], ["IT", "MSFT"].

We want to develop EPL rules that, in real-time and for each arriving StockTick, calculate the average price for each category in this grouping hierarchy (e.g. average for each country, each region, etc...).

You can see our current (ineficient solution) in https://gist.github.com/f1bec8803ee7de907570

I would greatly appreciate it if someone could tell me the "right way" to do this.

Thank you,

António Alegria

Re: Retrieving events prior an event

2009-12-08T10:24:58Z

Hi Cedrik,

here is a possible match-recognize (the regular expression only):
- A3 B1: (A X X X X B), select A and B
- A12 B1: (B X X X X A), select A and B

or both at the same time "(A X X X X B) | (B X X X X A)"

Best regards
Tom

From: Cédric Moonen <cedric.moonen@...>
To: user@...
Sent: Tue, December 8, 2009 9:07:18 AM
Subject: Re: [esper-user] Retrieving events prior an event

Thanks for the reply,

But I don't think this will really solve my problem. Let me show you with an example, which will probably be clearer than my explanation. Suppose the following sequence of events:
A1 A2 A3 A4 A5 A6 A7 B1 A8 A9 A10 A11 A12 A13 A14 A15 A16 B2 A17 A18

What I want to retrieve is:
- A3 B1 (A3 being the fifth A event before B10)
- A12 B1

I don't think this is really feasible using a pattern, is it ?

At 14:51 8/12/2009, you wrote:

May be a pattern using repeat or repeat until would be a better way?

2009/12/8 Cédric Moonen <cedric.moonen@... >

Hello,

I'm a bit stuck with a problem when trying to retrieve events that happened before a specific event. My scenario is that I have events coming in and all those events have the same base event (let's call it BaseEvent). Then, there are some specific events that I am looking for (SpecificEvent). I need to retrieve all those specific events with the base event that occured 5th-prior that specific event.

The first thing I did was something like this:

select * from SpecificEvent as event unidirectional, BaseEvent.win:length(5).std:firstevent() as prevEvent

But this didn't work as I expected and looking at the documentation of the std:firstevent, it is explained there that I should use the prev function instead because the window is restarted each time. So, I tried then to use the prev function but there's no explanation in the doc on how I have to specify that I want the full event and not one of the properties (although, it is stated that it should be possible). I tried to write something like this:

select event, prev(5, *) as prevEvent from SpecificEvent as event unidirectional, BaseEvent.win:length(5)

But it complains about the syntax error for the '*' symbol.

So my questions are:

1) How to write suche a statement ?

2) If there are multiple streams, how will the prev function "know" for which stream it has to select the event ?

I'm a bit confused with this prev function.

Thanks for your feedback.

Cédric

---------------------------------------------------------------------

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

Cédric Moonen
Projectingenieur

Flanders' MECHATRONICS Technology Centre (FMTC)
Celestijnenlaan 300D
B-3001 Leuven
Belgium

GSM: +32-498-91.94.14
Fax: +32-16-32.80.64
cedric.moonen@...

www.fmtc.be

match_recognize clause with aggregations

2009-12-08T10:06:57Z