Clean way to push messages into the stream from a memory cache
|
View:
New views
9 Messages
—
Rating Filter:
Alert me
|
 |
Clean way to push messages into the stream from a memory cache
by Dece
::
Rate this Message:
|
|
|
Re: Clean way to push messages into the stream from a memory cache
by Alexandre Vasseur
::
Rate this Message:
|
|
|
Re: Clean way to push messages into the stream from a memory cache
by Dece
::
Rate this Message:
|
|
|
Re: Clean way to push messages into the stream from a memory cache
by Alexandre Vasseur
::
Rate this Message:
|
|
|
Re: Clean way to push messages into the stream from a memory cache
by Dece
::
Rate this Message:
|
|
|
Re: Clean way to push messages into the stream from a memory cache
by Alexandre Vasseur
::
Rate this Message:
|
|
|
Re: Clean way to push messages into the stream from a memory cache
by Dece
::
Rate this Message:
|
|
|
Re: Clean way to push messages into the stream from a memory cache
by Thomas Bernhardt
::
Rate this Message:
Some parts of this message have been removed.
Learn more about Nabble's security policy.
Instead of manually parsing you could go with the statement object model (see compile method). That provides access to the where clause as well (string and object form). You could use that string or object and pass as a parameter to the method invocation join or the class handling that join, or create a new statement from just the where clause. Best regards, Tom From: Dece <decebel@...> To: user@... Sent: Tue, October 13, 2009 8:39:05 PM Subject: Re: [esper-user] Clean way to push messages into the stream from a memory cache Excuse me if I am repeating myself but I still have not got this clear And I see what you are hinting. The problem that I am trying to solve here is real time dynamic queries. I do not know what queries (select clause) I will get into the application in advance. Dynamic queries are submitted and I am manually parsing the select clause to identify the event names and then polling the data cloud for only those specific events. I am then setting up the statement and then using sendEvent() to send all the messages retrieved from the cloud. I like the idea you suggested about using a method in the join but I still need to parse the select to identify the event names. Also I am not taking advantage of the where clause. Esentially all the messages are passed thru the sql. many thanks Dece Alexandre Vasseur wrote: > > You can pass any argument(s) to your method - constatns as well as > argument > coming out from the stream(s) > > something like the following in pseudo syntax is valid: > event Foo { String name, int id } > method M(String x, int y, String t) > EPL: select * from Foo.win:time(...) f, method:M(f.x, f.y, "some > constant") > where ... > > There are some examples in the documentation I referenced. > As per saying that passing the event name as a string constant is > hardcoding, well... you already have the event name in the EPL statement. > > Alex > > On Tue, Oct 13, 2009 at 5:48 PM, Dece <decebel@...> wrote: > >> >> But won't that hard-code the arguments in the sql statement. >> >> I can get requests for different types of events. It need not necessarily >> be >> GameStart, GameEnd. >> Is there a way to get the string representation of the EventTypes >> mentioned >> in the select clause? >> >> Secondly, I would also like to pass simple where clauses in my method. >> The >> reason being, when the method queries the data cloud for an event type, >> it >> could return more than a million records and a lot of them would be >> irrelevant. However, if my method can also look at the where clause, then >> when I use the data cloud api, I can specifically ask for events with >> some >> condition. >> >> For example, >> select GameStart.TimeStamp, GameStart.UserId, GameStart.Avatar, >> GameEnd.TimeStamp, GameEnd.Score from GameStart.win:time(1 min), >> GameEnd.win:time(1 min) where GameStart.SessionId == GameEnd.SessionId >> and >> GameStart.Name == "Poker" >> >> Here, when I request GameStart events from the cloud using a method, it >> would be nice if I can also pass GameStart.Name == "Poker". The api that >> makes call to the data cloud will specify this extra criteria in its >> request >> and now we have a cleaner approach. >> >> What do you advice for such a situation? >> Once again appreciate your help, >> Thanks >> Dece >> >> >> Alexandre Vasseur wrote: >> > >> > it is likely your provided method will accept String type arguments, >> and >> > then your EPL will look like: >> > ... method:MyLookupLib.GetEventFromDataCloud("GameStart", "GameEnd") >> > Your method can also accep the full event as argument or some event >> > properties (SessionId etc) >> > Alex >> > >> > >> > >> > >> > >> > On Tue, Oct 13, 2009 at 3:54 PM, Dece <decebel@...> wrote: >> > >> >> >> >> Hi Alexandre, >> >> >> >> Your pointer to "Accessing Non-Relational Data via Method Invocation" >> >> looks >> >> promising. >> >> However there are a few questions that popped-up when I was trying to >> >> code: >> >> >> >> So, here is my original sql for example: >> >> >> >> select GameStart.TimeStamp, GameStart.UserId, GameStart.Avatar, >> >> GameEnd.TimeStamp, GameEnd.Score from GameStart.win:time(1 min), >> >> GameEnd.win:time(1 min) where GameStart.SessionId == GameEnd.SessionId >> >> >> >> >> >> Now when I receive this query from a user, I want to request the data >> >> cloud >> >> for all events of type GameStart and GameEnd. I am assuming once my >> >> method >> >> returns the GameStart Events and GameEndEnd Events, Esper can do the >> >> necessary join (GameStart.SessionId == GameEnd.SessionId) >> >> >> >> >> >> What would be the syntax for passing the event names into the methods. >> >> Below >> >> syntax looks bit weird ... >> >> >> >> >> >> select GameStart.TimeStamp, GameStart.UserId, GameStart.Avatar, >> >> GameEnd.TimeStamp, GameEnd.Score from >> >> method:MyLookupLib.GetEventFromDataCloud(GameStart, GameEnd) >> >> >> >> >> >> >> >> Thanks again, >> >> Dece >> >> >> >> >> >> Alexandre Vasseur wrote: >> >> > >> >> > Hi Dece >> >> > Your usage is a bit counter intuitive to what Esper/NEsper is >> designed >> >> for >> >> > ie continuous querying. >> >> > Could you detail a bit why you need to grab data from your external >> >> store >> >> > (your "data cloud") and perform one time query on it? This approach >> is >> >> > often >> >> > used for system warm up or recovery / event replay use cases. >> >> > If you want to look at related features: >> >> > - For adhoc querying, Esper/NEsper provides the concept of "named >> >> window". >> >> > >> >> >> >> > >> >> >> http://esper.codehaus.org/esper-3.1.0/doc/reference/en/html/api.html#api-runtime-ondemand >> >> > - Look at joins with reference data, considering your data cloud as >> a >> >> > reference data provider >> >> > >> >> >> http://esper.codehaus.org/esper-3.1.0/doc/reference/en/html/epl_clauses.html#joining_method >> >> > >> >> > Alex >> >> > >> >> > On Tue, Oct 13, 2009 at 6:26 AM, Dece <decebel@...> wrote: >> >> > >> >> >> >> >> >> Hi All, >> >> >> This is my first project using NEsper so pardon me if my mail is >> >> >> spammingly >> >> >> long or too lame. >> >> >> >> >> >> All my events are stored in a data cloud outside of my NEsper >> >> >> application. >> >> >> When I get a query request for some data, I use the >> >> >> EPStatementObjectModel >> >> >> to identify the stream/Event names from the query and I manually >> pull >> >> the >> >> >> messages from the data cloud corresponding to the stream name. >> >> >> After setting up the listener, I send the messages/events using >> >> >> sendEvent. >> >> >> >> >> >> >> >> >> For example, if I receive a query: >> >> >> select MyEvent.partNum, MyEvent.Price from MyEvent where >> >> >> MyEvent.partNum>100, I parse the from statement to identify the >> event >> >> >> name. >> >> >> In this case, it is, MyEvent. Then I make a request to the cloud >> for >> >> all >> >> >> MyEvent messages. Once I receive them, I do a sendEvent() to pump >> the >> >> >> messages. >> >> >> >> >> >> However, as you can see, I am not taking advantage of the where >> clause >> >> >> while >> >> >> making a request to the data cloud. >> >> >> >> >> >> Now functionally all this is fine except that this is very >> >> in-efficient >> >> >> as >> >> >> I >> >> >> do not take advantage of the where clause. The data cloud does not >> >> >> understand sql. >> >> >> >> >> >> So my question is what is a good design to pull events from a cloud >> >> and >> >> >> start streaming them into NEsper statements? >> >> >> >> >> >> Thanks a lot >> >> >> Gracias >> >> >> Dece >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> View this message in context: >> >> >> >> >> >> >> >> Sent from the Esper User list mailing list archive at Nabble.com. >> >> >> >> >> >> >> >> >> >> --------------------------------------------------------------------- >> >> >> To unsubscribe from this list, please visit: >> >> >> >> >> >> http://xircles.codehaus.org/manage_email >> >> >> >> >> >> >> >> >> >> >> > >> >> > >> >> >> >> -- >> >> View this message in context: >> >> >> http://www.nabble.com/Clean-way-to-push-messages-into-the-stream-from-a-memory-cache-tp25867020p25873329.html >> >> Sent from the Esper User list mailing list archive at Nabble.com. >> >> >> >> >> >> --------------------------------------------------------------------- >> >> To unsubscribe from this list, please visit: >> >> >> >> http://xircles.codehaus.org/manage_email >> >> >> >> >> >> >> > >> > >> >> -- >> View this message in context: >> Sent from the Esper User list mailing list archive at Nabble.com. >> >> >> --------------------------------------------------------------------- >> To unsubscribe from this list, please visit: >> >> http://xircles.codehaus.org/manage_email >> >> >> > > -- View this message in context: http://www.nabble.com/Clean-way-to-push-messages-into-the-stream-from-a-memory-cache-tp25867020p25883395.html Sent from the Esper User list mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: Clean way to push messages into the stream from a memory cache
by Dece
::
Rate this Message:
Ah .. thanks. That makes sense. I will explore more in that direction.
 |
| Free embeddable forum powered by Nabble | Forum Help |
