|
View:
New views
6 Messages
—
Rating Filter:
Alert me
|
|
|
Pipelines, Threaded services - is there any more documentation?Hello!
Right now I get used to Hivemind and I am starting
to like it very much :-)
BUT .... I really think there is a massive lack of
documentation.
I know (as I work for a software company myself)
that writing documentation is not really the favorite thing to do and I'm very
happy that my company has a own department doing this job, but on the
other hand: the coolest software is of no use if no one can use it
:-)
And even if I found out a lot by myself right now
it is frustrating to trial and error for 3 hours (or more) just to find out how
easy it is done if you know how to do it (in a sense it is even more frustrating
than it would be if it would be hard to do ;-)
The API help does not help me that much, because
they don't really describe how to do things in the context of the
framework.
If there is more documentation and I'm just
to stupid to find it, please point me to it!! Again, I like Hivemind very much already, I even
would volunteer to write some documentation.
What about the cookbook? I read in the newsgroup
about it, but I could not find it.
I don't know much about Hivemind right now,
but as I hope to find out more about it in the near future I
could write about the things I learn (and perhaps it is not to bad to
describe it from a beginners point of view), and if there would be a start,
perhaps others would join in? But after that outburst of frustration (after trial and error the whole
afternoon) some questions:
1.:
At the moment I have three services, and I need
them executed in a row.
A -> give results to -> B -> give results
to -> C
Each of them is used (or at least could be used)
alone from another point of the application, so I want to keep them
separated.
I found something about pipelines in the net, but I
could not find more documentation on how to use them, I even did not found out
if they are really intended to do what I need here.
2.:
I tried to build my own "pipeline" using threaded
services bundled together in a "parent" service.
I got it working, but: after processing about 2500
objects that way I get a java out of memory error.
After searching the internet a little more I found
out that I should clean up.
Can someone post me a little example of a threaded
service including the "clean up" part?
Thanks a lot for your time again!
Jochen
|
|
|
Re: Pipelines, Threaded services - is there any more documentation?Hi Jochen,
Right from the start, the Hivemind project has aimed to provide clear and concise instructions and information but as you also understand, this is of course not always possible. However, if you have some changes you'd like to contribute, I am quite confident such help is always welcome. I would suggest you create a JIRA for it, attach the changes you think see fit and someone will take a look and update the documentation to reflect this. As for you questions relating to the "pipeline" - I am not sure this is a good fit to your problem - the pipeline is very similar to the interceptors in that it allows arbitrary implementations to slot in between the service interface and the actual implementation and the pipeline, as opposed to the interceptors, work with the service interface definition directly and have the exact same parameters defined by the service interface (plus the additional parameter being the next service in the pipeline to invoke). So, in short, the pipeline is a "more typed" version of the interceptor. I have used the pipeline quite effectively for validation of parameters and simple caching. The scenario you are describing sounds more like the Filters in the Servlet API - each filter gets invoked and they may pass parameters "down" the chain by adding data to the servlet request - does that match more what you have in mind? Cheers, Johan Jochen Zimmermann wrote: > Hello! > > Right now I get used to Hivemind and I am starting to like it very much :-) > > BUT .... I really think there is a massive lack of documentation. > I know (as I work for a software company myself) that writing > documentation is not really the favorite thing to do and I'm very happy > that my company has a own department doing this job, but on the other > hand: the coolest software is of no use if no one can use it :-) > And even if I found out a lot by myself right now it is frustrating to > trial and error for 3 hours (or more) just to find out how easy it is > done if you know how to do it (in a sense it is even more frustrating > than it would be if it would be hard to do ;-) > > The API help does not help me that much, because they don't really > describe how to do things in the context of the framework. > > If there is more documentation and I'm just to stupid to find it, please > point me to it!! > > Again, I like Hivemind very much already, I even would volunteer to > write some documentation. > What about the cookbook? I read in the newsgroup about it, but I could > not find it. > I don't know much about Hivemind right now, but as I hope to find out > more about it in the near future I could write about the things I learn > (and perhaps it is not to bad to describe it from a beginners point of > view), and if there would be a start, perhaps others would join in? > > But after that outburst of frustration (after trial and error the whole > afternoon) some questions: > > 1.: > At the moment I have three services, and I need them executed in a row. > > A -> give results to -> B -> give results to -> C > > Each of them is used (or at least could be used) alone from another > point of the application, so I want to keep them separated. > > I found something about pipelines in the net, but I could not find more > documentation on how to use them, I even did not found out if they are > really intended to do what I need here. > > 2.: > I tried to build my own "pipeline" using threaded services bundled > together in a "parent" service. > I got it working, but: after processing about 2500 objects that way I > get a java out of memory error. > After searching the internet a little more I found out that I should > clean up. > Can someone post me a little example of a threaded service including the > "clean up" part? > > Thanks a lot for your time again! > > Jochen > > > > > -- you too? |
|
|
Re: Pipelines, Threaded services - is there any more documentation?Hi Johan!
Thanks for your answer! According to your answer I think the servlet filters describe more of what I need than the pipelines ... right now I solved it with a service that includes my three other services (injected by the framework of course) and handles the forwarding of the results. That works for me, I was just curios if the pipelines are meant to do something like that. To come back to the documentation: It was not my point to offend anyone involved in that project or the project itself (just to say it again :-) There is good documentation to get started, the most useful part is the calculator example. Beyond the example there is enough documentation to get an idea what you can do with Hivemind, but the explanation is to general .... Concrete example: Multithreading service model: It is explained how it works generally and then there is the following sentence in the documentation. "The service instance is discarded when notified to cleanup; this is controlled by the hivemind.ThreadEventNotifier service. If your application has any threaded services, you are responsible for invoking the fireThreadCleanup() method of the service." I really, really tried to work out how to do this with research in the internet, but I still can't do it .... and there are some more things like that in the documentation: it is mentioned, with some hint how to do it, but to vague to do it immediately. Perhaps it is really easy to do, but with just about 5 days Hivemind - experience it is not ... :-( Just some more concrete coding examples would be really great! If I ever find out how to do the task mentioned above I will work out an easy example and post it as you suggested, I promise this :-) Regards Jochen Original Message ----- From: "Johan Lindquist" <johan@...> To: <user@...> Sent: Saturday, April 07, 2007 12:48 PM Subject: Re: Pipelines, Threaded services - is there any more documentation? > Hi Jochen, > > Right from the start, the Hivemind project has aimed to provide clear > and concise instructions and information but as you also understand, > this is of course not always possible. However, if you have some > changes you'd like to contribute, I am quite confident such help is > always welcome. I would suggest you create a JIRA for it, attach the > changes you think see fit and someone will take a look and update the > documentation to reflect this. > > As for you questions relating to the "pipeline" - I am not sure this is > a good fit to your problem - the pipeline is very similar to the > interceptors in that it allows arbitrary implementations to slot in > between the service interface and the actual implementation and the > pipeline, as opposed to the interceptors, work with the service > interface definition directly and have the exact same parameters defined > by the service interface (plus the additional parameter being the next > service in the pipeline to invoke). So, in short, the pipeline is a > "more typed" version of the interceptor. I have used the pipeline quite > effectively for validation of parameters and simple caching. > > The scenario you are describing sounds more like the Filters in the > Servlet API - each filter gets invoked and they may pass parameters > "down" the chain by adding data to the servlet request - does that match > more what you have in mind? > > Cheers, > > Johan > > Jochen Zimmermann wrote: >> Hello! >> >> Right now I get used to Hivemind and I am starting to like it very much >> :-) >> >> BUT .... I really think there is a massive lack of documentation. >> I know (as I work for a software company myself) that writing >> documentation is not really the favorite thing to do and I'm very happy >> that my company has a own department doing this job, but on the other >> hand: the coolest software is of no use if no one can use it :-) >> And even if I found out a lot by myself right now it is frustrating to >> trial and error for 3 hours (or more) just to find out how easy it is >> done if you know how to do it (in a sense it is even more frustrating >> than it would be if it would be hard to do ;-) >> >> The API help does not help me that much, because they don't really >> describe how to do things in the context of the framework. >> >> If there is more documentation and I'm just to stupid to find it, please >> point me to it!! >> >> Again, I like Hivemind very much already, I even would volunteer to >> write some documentation. >> What about the cookbook? I read in the newsgroup about it, but I could >> not find it. >> I don't know much about Hivemind right now, but as I hope to find out >> more about it in the near future I could write about the things I learn >> (and perhaps it is not to bad to describe it from a beginners point of >> view), and if there would be a start, perhaps others would join in? >> >> But after that outburst of frustration (after trial and error the whole >> afternoon) some questions: >> >> 1.: >> At the moment I have three services, and I need them executed in a row. >> >> A -> give results to -> B -> give results to -> C >> >> Each of them is used (or at least could be used) alone from another >> point of the application, so I want to keep them separated. >> >> I found something about pipelines in the net, but I could not find more >> documentation on how to use them, I even did not found out if they are >> really intended to do what I need here. >> >> 2.: >> I tried to build my own "pipeline" using threaded services bundled >> together in a "parent" service. >> I got it working, but: after processing about 2500 objects that way I >> get a java out of memory error. >> After searching the internet a little more I found out that I should >> clean up. >> Can someone post me a little example of a threaded service including the >> "clean up" part? >> >> Thanks a lot for your time again! >> >> Jochen >> >> >> >> >> > > -- > you too? > |
|
|
Re: Pipelines, Threaded services - is there any more documentation?Hi Jochen,
I don't think anyone got offended by your comment, it holds as true for Hivemind as it does for many other projects out there. I agree that more easy to follow examples are needed (and it has been discussed on the dev list recently). The Hivemind 2.0 documentation currently lives on a WIKI, which will make it easier for people to contribute documentation but this is not the case for the 1.0 versions. In any case, doc updates will always be valuable to the project and most welcome. Btw, I hope I speak for the committers here too and not stepping on anyones toes ;) As for calling thread cleanup when a thread is finished, please have a look at the org.apache.hivemind.Registry API, it is defined there. [1] Cheers, Johan [1] http://hivemind.apache.org/hivemind1/hivemind/apidocs/index.html Jochen Zimmermann wrote: > Hi Johan! > > Thanks for your answer! > According to your answer I think the servlet filters describe more of > what I need than the pipelines ... right now I solved it with a service > that includes my three other services (injected by the framework of > course) and handles the forwarding of the results. That works for me, I > was just curios if the pipelines are meant to do something like that. > > To come back to the documentation: > It was not my point to offend anyone involved in that project or the > project itself (just to say it again :-) > There is good documentation to get started, the most useful part is the > calculator example. > Beyond the example there is enough documentation to get an idea what you > can do with Hivemind, but the explanation is to general .... > > Concrete example: Multithreading service model: > It is explained how it works generally and then there is the following > sentence in the documentation. > > "The service instance is discarded when notified to cleanup; this is > controlled by the hivemind.ThreadEventNotifier service. If your > application has any threaded services, you are responsible for invoking > the fireThreadCleanup() method of the service." > > I really, really tried to work out how to do this with research in the > internet, but I still can't do it .... and there are some more things > like that in the documentation: it is mentioned, with some hint how to > do it, but to vague to do it immediately. > > Perhaps it is really easy to do, but with just about 5 days Hivemind - > experience it is not ... :-( > Just some more concrete coding examples would be really great! > If I ever find out how to do the task mentioned above I will work out an > easy example and post it as you suggested, I promise this :-) > > Regards > Jochen > > > Original Message ----- From: "Johan Lindquist" <johan@...> > To: <user@...> > Sent: Saturday, April 07, 2007 12:48 PM > Subject: Re: Pipelines, Threaded services - is there any more > documentation? > > >> Hi Jochen, >> >> Right from the start, the Hivemind project has aimed to provide clear >> and concise instructions and information but as you also understand, >> this is of course not always possible. However, if you have some >> changes you'd like to contribute, I am quite confident such help is >> always welcome. I would suggest you create a JIRA for it, attach the >> changes you think see fit and someone will take a look and update the >> documentation to reflect this. >> >> As for you questions relating to the "pipeline" - I am not sure this is >> a good fit to your problem - the pipeline is very similar to the >> interceptors in that it allows arbitrary implementations to slot in >> between the service interface and the actual implementation and the >> pipeline, as opposed to the interceptors, work with the service >> interface definition directly and have the exact same parameters defined >> by the service interface (plus the additional parameter being the next >> service in the pipeline to invoke). So, in short, the pipeline is a >> "more typed" version of the interceptor. I have used the pipeline quite >> effectively for validation of parameters and simple caching. >> >> The scenario you are describing sounds more like the Filters in the >> Servlet API - each filter gets invoked and they may pass parameters >> "down" the chain by adding data to the servlet request - does that match >> more what you have in mind? >> >> Cheers, >> >> Johan >> >> Jochen Zimmermann wrote: >>> Hello! >>> >>> Right now I get used to Hivemind and I am starting to like it very >>> much :-) >>> >>> BUT .... I really think there is a massive lack of documentation. >>> I know (as I work for a software company myself) that writing >>> documentation is not really the favorite thing to do and I'm very happy >>> that my company has a own department doing this job, but on the other >>> hand: the coolest software is of no use if no one can use it :-) >>> And even if I found out a lot by myself right now it is frustrating to >>> trial and error for 3 hours (or more) just to find out how easy it is >>> done if you know how to do it (in a sense it is even more frustrating >>> than it would be if it would be hard to do ;-) >>> >>> The API help does not help me that much, because they don't really >>> describe how to do things in the context of the framework. >>> >>> If there is more documentation and I'm just to stupid to find it, please >>> point me to it!! >>> >>> Again, I like Hivemind very much already, I even would volunteer to >>> write some documentation. >>> What about the cookbook? I read in the newsgroup about it, but I could >>> not find it. >>> I don't know much about Hivemind right now, but as I hope to find out >>> more about it in the near future I could write about the things I learn >>> (and perhaps it is not to bad to describe it from a beginners point of >>> view), and if there would be a start, perhaps others would join in? >>> >>> But after that outburst of frustration (after trial and error the whole >>> afternoon) some questions: >>> >>> 1.: >>> At the moment I have three services, and I need them executed in a row. >>> >>> A -> give results to -> B -> give results to -> C >>> >>> Each of them is used (or at least could be used) alone from another >>> point of the application, so I want to keep them separated. >>> >>> I found something about pipelines in the net, but I could not find more >>> documentation on how to use them, I even did not found out if they are >>> really intended to do what I need here. >>> >>> 2.: >>> I tried to build my own "pipeline" using threaded services bundled >>> together in a "parent" service. >>> I got it working, but: after processing about 2500 objects that way I >>> get a java out of memory error. >>> After searching the internet a little more I found out that I should >>> clean up. >>> Can someone post me a little example of a threaded service including the >>> "clean up" part? >>> >>> Thanks a lot for your time again! >>> >>> Jochen >>> >>> >>> >>> >>> >> >> -- >> you too? >> > > -- you too? |
|
|
Re: Pipelines, Threaded services - is there any more documentation?Have you found the part of the hivemind 1 documentation which
is currently not available for hivemind 2? Like this one for example: http://hivemind.apache.org/hivemind1/hivemind-lib/PipelineFactory.html You can find an example for the "clean up" part in the HiveMind Servlet Filter (doFilter method): http://svn.apache.org/repos/asf/hivemind/hivemind2/trunk/framework/src/java/org/apache/hivemind/servlet/AutoloadingHiveMindFilter.java Achim Am Fri, 06 Apr 2007 23:58:08 +0200 schrieb Jochen Zimmermann <zet4080@...>: > Hello! > > Right now I get used to Hivemind and I am starting to like it very much > :-) > > BUT .... I really think there is a massive lack of documentation. > I know (as I work for a software company myself) that writing > documentation is not really the favorite thing to do and I'm very happy > that my company has a own department doing this job, but on the other > hand: the coolest software is of no use if no one can use it :-) > And even if I found out a lot by myself right now it is frustrating to > trial and error for 3 hours (or more) just to find out how easy it is > done if you know how to do it (in a sense it is even more frustrating > than it would be if it would be hard to do ;-) > > The API help does not help me that much, because they don't really > describe how to do things in the context of the framework. > > If there is more documentation and I'm just to stupid to find it, please > point me to it!! > > Again, I like Hivemind very much already, I even would volunteer to > write some documentation. > What about the cookbook? I read in the newsgroup about it, but I could > not find it. > I don't know much about Hivemind right now, but as I hope to find out > more about it in the near future I could write about the things I learn > (and perhaps it is not to bad to describe it from a beginners point of > view), and if there would be a start, perhaps others would join in? > > But after that outburst of frustration (after trial and error the whole > afternoon) some questions: > > 1.: > At the moment I have three services, and I need them executed in a row. > > A -> give results to -> B -> give results to -> C > > Each of them is used (or at least could be used) alone from another > point of the application, so I want to keep them separated. > > I found something about pipelines in the net, but I could not find more > documentation on how to use them, I even did not found out if they are > really intended to do what I need here. > > 2.: > I tried to build my own "pipeline" using threaded services bundled > together in a "parent" service. > I got it working, but: after processing about 2500 objects that way I > get a java out of memory error. > After searching the internet a little more I found out that I should > clean up. > Can someone post me a little example of a threaded service including the > "clean up" part? > > Thanks a lot for your time again! > > Jochen > > > > |
|
|
Re: Pipelines, Threaded services - is there any more documentation?Everybody is welcome to contribute documentation, examples, best practices
and lessons learned to the wiki: http://cwiki.apache.org/HIVEMIND2/user-guide.html Just signup here (link is on the top right): http://cwiki.apache.org/confluence/display/HIVEMIND2/Index and mail me so I can give you edit access. There is one catch only, if one day the wiki content should be included in a release, the edit access must be restricted to committers that filed a CLA: http://cwiki.apache.org/CWIKI/ Achim Am Sat, 07 Apr 2007 14:18:55 +0200 schrieb Johan Lindquist <johan@...>: > Hi Jochen, > > I don't think anyone got offended by your comment, it holds as true for > Hivemind as it does for many other projects out there. > > I agree that more easy to follow examples are needed (and it has been > discussed on the dev list recently). The Hivemind 2.0 documentation > currently lives on a WIKI, which will make it easier for people to > contribute documentation but this is not the case for the 1.0 versions. > > In any case, doc updates will always be valuable to the project and most > welcome. Btw, I hope I speak for the committers here too and not > stepping on anyones toes ;) > > As for calling thread cleanup when a thread is finished, please have a > look at the org.apache.hivemind.Registry API, it is defined there. [1] > > Cheers, > > Johan > > [1] http://hivemind.apache.org/hivemind1/hivemind/apidocs/index.html > > Jochen Zimmermann wrote: >> Hi Johan! >> >> Thanks for your answer! >> According to your answer I think the servlet filters describe more of >> what I need than the pipelines ... right now I solved it with a service >> that includes my three other services (injected by the framework of >> course) and handles the forwarding of the results. That works for me, I >> was just curios if the pipelines are meant to do something like that. >> >> To come back to the documentation: >> It was not my point to offend anyone involved in that project or the >> project itself (just to say it again :-) >> There is good documentation to get started, the most useful part is the >> calculator example. >> Beyond the example there is enough documentation to get an idea what you >> can do with Hivemind, but the explanation is to general .... >> >> Concrete example: Multithreading service model: >> It is explained how it works generally and then there is the following >> sentence in the documentation. >> >> "The service instance is discarded when notified to cleanup; this is >> controlled by the hivemind.ThreadEventNotifier service. If your >> application has any threaded services, you are responsible for invoking >> the fireThreadCleanup() method of the service." >> >> I really, really tried to work out how to do this with research in the >> internet, but I still can't do it .... and there are some more things >> like that in the documentation: it is mentioned, with some hint how to >> do it, but to vague to do it immediately. >> >> Perhaps it is really easy to do, but with just about 5 days Hivemind - >> experience it is not ... :-( >> Just some more concrete coding examples would be really great! >> If I ever find out how to do the task mentioned above I will work out an >> easy example and post it as you suggested, I promise this :-) >> >> Regards >> Jochen >> >> >> Original Message ----- From: "Johan Lindquist" <johan@...> >> To: <user@...> >> Sent: Saturday, April 07, 2007 12:48 PM >> Subject: Re: Pipelines, Threaded services - is there any more >> documentation? >> >> >>> Hi Jochen, >>> >>> Right from the start, the Hivemind project has aimed to provide clear >>> and concise instructions and information but as you also understand, >>> this is of course not always possible. However, if you have some >>> changes you'd like to contribute, I am quite confident such help is >>> always welcome. I would suggest you create a JIRA for it, attach the >>> changes you think see fit and someone will take a look and update the >>> documentation to reflect this. >>> >>> As for you questions relating to the "pipeline" - I am not sure this is >>> a good fit to your problem - the pipeline is very similar to the >>> interceptors in that it allows arbitrary implementations to slot in >>> between the service interface and the actual implementation and the >>> pipeline, as opposed to the interceptors, work with the service >>> interface definition directly and have the exact same parameters >>> defined >>> by the service interface (plus the additional parameter being the next >>> service in the pipeline to invoke). So, in short, the pipeline is a >>> "more typed" version of the interceptor. I have used the pipeline >>> quite >>> effectively for validation of parameters and simple caching. >>> >>> The scenario you are describing sounds more like the Filters in the >>> Servlet API - each filter gets invoked and they may pass parameters >>> "down" the chain by adding data to the servlet request - does that >>> match >>> more what you have in mind? >>> >>> Cheers, >>> >>> Johan >>> >>> Jochen Zimmermann wrote: >>>> Hello! >>>> >>>> Right now I get used to Hivemind and I am starting to like it very >>>> much :-) >>>> >>>> BUT .... I really think there is a massive lack of documentation. >>>> I know (as I work for a software company myself) that writing >>>> documentation is not really the favorite thing to do and I'm very >>>> happy >>>> that my company has a own department doing this job, but on the other >>>> hand: the coolest software is of no use if no one can use it :-) >>>> And even if I found out a lot by myself right now it is frustrating to >>>> trial and error for 3 hours (or more) just to find out how easy it is >>>> done if you know how to do it (in a sense it is even more frustrating >>>> than it would be if it would be hard to do ;-) >>>> >>>> The API help does not help me that much, because they don't really >>>> describe how to do things in the context of the framework. >>>> >>>> If there is more documentation and I'm just to stupid to find it, >>>> please >>>> point me to it!! >>>> >>>> Again, I like Hivemind very much already, I even would volunteer to >>>> write some documentation. >>>> What about the cookbook? I read in the newsgroup about it, but I could >>>> not find it. >>>> I don't know much about Hivemind right now, but as I hope to find out >>>> more about it in the near future I could write about the things I >>>> learn >>>> (and perhaps it is not to bad to describe it from a beginners point of >>>> view), and if there would be a start, perhaps others would join in? >>>> >>>> But after that outburst of frustration (after trial and error the >>>> whole >>>> afternoon) some questions: >>>> >>>> 1.: >>>> At the moment I have three services, and I need them executed in a >>>> row. >>>> >>>> A -> give results to -> B -> give results to -> C >>>> >>>> Each of them is used (or at least could be used) alone from another >>>> point of the application, so I want to keep them separated. >>>> >>>> I found something about pipelines in the net, but I could not find >>>> more >>>> documentation on how to use them, I even did not found out if they are >>>> really intended to do what I need here. >>>> >>>> 2.: >>>> I tried to build my own "pipeline" using threaded services bundled >>>> together in a "parent" service. >>>> I got it working, but: after processing about 2500 objects that way I >>>> get a java out of memory error. >>>> After searching the internet a little more I found out that I should >>>> clean up. >>>> Can someone post me a little example of a threaded service including >>>> the >>>> "clean up" part? >>>> >>>> Thanks a lot for your time again! >>>> >>>> Jochen >>>> >>>> >>>> >>>> >>>> >>> >>> -- >>> you too? >>> >> >> > |
| Free embeddable forum powered by Nabble | Forum Help |