|
New features for next Tim-Messaging releases
Hi guys,
some time has passed since the latest Tim-Messaging 2.0 release, and
I'm now starting to think about new features for next releases.
Here are a few new features I was thinking at:
1) Given that data-affinity is more and more import, it may be good to
improve affinity-routing for tim-masterworker.
2) It may be cool to add a json-over-http interface to tim-pipes,
providing restful access to topologies and pipes.
3) It may be even cooler to add a restful interface to
tim-masterworker, implementing a scalable, fault-tolerant, rest-based
master/worker server.
What do you think?
Do you have any other particular idea/feature/suggestion?
Thanks,
Cheers!
Sergio B.
--
Sergio Bossa
Software Passionate and Open Source Enthusiast.
URL: http://www.linkedin.com/in/sergiob_______________________________________________
tc-users mailing list
tc-users@...
http://lists.terracotta.org/mailman/listinfo/tc-users
|
|
Re: [tc-forge-dev] New features for next Tim-Messaging releases
Those all sound like cool features to me although I have little feel
for which of them are actually most in need.
I'm cc'ing some folks that might have a better feel on the needs of
actual tim-messaging and tim-pipes users. Sergio, maybe it would be
good to put these ideas up where people could discuss? Like your blog
or our forums. I'm sure we could point a little traffic your way
somehow.
On Aug 14, 2009, at 12:04 PM, Sergio Bossa wrote:
> Hi guys,
>
> some time has passed since the latest Tim-Messaging 2.0 release, and
> I'm now starting to think about new features for next releases.
> Here are a few new features I was thinking at:
>
> 1) Given that data-affinity is more and more import, it may be good to
> improve affinity-routing for tim-masterworker.
> 2) It may be cool to add a json-over-http interface to tim-pipes,
> providing restful access to topologies and pipes.
> 3) It may be even cooler to add a restful interface to
> tim-masterworker, implementing a scalable, fault-tolerant, rest-based
> master/worker server.
>
> What do you think?
> Do you have any other particular idea/feature/suggestion?
>
> Thanks,
> Cheers!
>
> Sergio B.
>
> --
> Sergio Bossa
> Software Passionate and Open Source Enthusiast.
> URL: http://www.linkedin.com/in/sergiob> _______________________________________________
> tc-forge-dev mailing list
> tc-forge-dev@...
> http://lists.terracotta.org/mailman/listinfo/tc-forge-dev_______________________________________________
tc-users mailing list
tc-users@...
http://lists.terracotta.org/mailman/listinfo/tc-users
|
|
Re: [tc-forge-dev] New features for next Tim-Messaging releases
I would add serialization of messages to the list. Serializing should
provide big improvements to throughput and messages behave more like
messages and not distributed memory. I think the latter is useful of
course but I think there's also a use for the former.
On Aug 14, 2009, at 10:13 AM, Alex Miller
< amiller@...> wrote:
> Those all sound like cool features to me although I have little feel
> for which of them are actually most in need.
>
> I'm cc'ing some folks that might have a better feel on the needs of
> actual tim-messaging and tim-pipes users. Sergio, maybe it would be
> good to put these ideas up where people could discuss? Like your blog
> or our forums. I'm sure we could point a little traffic your way
> somehow.
>
>
> On Aug 14, 2009, at 12:04 PM, Sergio Bossa wrote:
>
>> Hi guys,
>>
>> some time has passed since the latest Tim-Messaging 2.0 release, and
>> I'm now starting to think about new features for next releases.
>> Here are a few new features I was thinking at:
>>
>> 1) Given that data-affinity is more and more import, it may be good
>> to
>> improve affinity-routing for tim-masterworker.
>> 2) It may be cool to add a json-over-http interface to tim-pipes,
>> providing restful access to topologies and pipes.
>> 3) It may be even cooler to add a restful interface to
>> tim-masterworker, implementing a scalable, fault-tolerant, rest-based
>> master/worker server.
>>
>> What do you think?
>> Do you have any other particular idea/feature/suggestion?
>>
>> Thanks,
>> Cheers!
>>
>> Sergio B.
>>
>> --
>> Sergio Bossa
>> Software Passionate and Open Source Enthusiast.
>> URL: http://www.linkedin.com/in/sergiob>> _______________________________________________
>> tc-forge-dev mailing list
>> tc-forge-dev@...
>> http://lists.terracotta.org/mailman/listinfo/tc-forge-dev>
> _______________________________________________
> tc-forge-dev mailing list
> tc-forge-dev@...
> http://lists.terracotta.org/mailman/listinfo/tc-forge-dev_______________________________________________
tc-users mailing list
tc-users@...
http://lists.terracotta.org/mailman/listinfo/tc-users
|
|
Re: [tc-forge-dev] New features for next Tim-Messaging releases
yeah
2 modes. serialization-based vs. identity-based.
--Ari
On Aug 14, 2009, at 4:58 PM, Taylor Gautier wrote:
> I would add serialization of messages to the list. Serializing
> should provide big improvements to throughput and messages behave
> more like messages and not distributed memory. I think the latter is
> useful of course but I think there's also a use for the former.
>
> On Aug 14, 2009, at 10:13 AM, Alex Miller
> < amiller@...> wrote:
>
>> Those all sound like cool features to me although I have little feel
>> for which of them are actually most in need.
>>
>> I'm cc'ing some folks that might have a better feel on the needs of
>> actual tim-messaging and tim-pipes users. Sergio, maybe it would be
>> good to put these ideas up where people could discuss? Like your
>> blog
>> or our forums. I'm sure we could point a little traffic your way
>> somehow.
>>
>>
>> On Aug 14, 2009, at 12:04 PM, Sergio Bossa wrote:
>>
>>> Hi guys,
>>>
>>> some time has passed since the latest Tim-Messaging 2.0 release, and
>>> I'm now starting to think about new features for next releases.
>>> Here are a few new features I was thinking at:
>>>
>>> 1) Given that data-affinity is more and more import, it may be
>>> good to
>>> improve affinity-routing for tim-masterworker.
>>> 2) It may be cool to add a json-over-http interface to tim-pipes,
>>> providing restful access to topologies and pipes.
>>> 3) It may be even cooler to add a restful interface to
>>> tim-masterworker, implementing a scalable, fault-tolerant, rest-
>>> based
>>> master/worker server.
>>>
>>> What do you think?
>>> Do you have any other particular idea/feature/suggestion?
>>>
>>> Thanks,
>>> Cheers!
>>>
>>> Sergio B.
>>>
>>> --
>>> Sergio Bossa
>>> Software Passionate and Open Source Enthusiast.
>>> URL: http://www.linkedin.com/in/sergiob>>> _______________________________________________
>>> tc-forge-dev mailing list
>>> tc-forge-dev@...
>>> http://lists.terracotta.org/mailman/listinfo/tc-forge-dev>>
>> _______________________________________________
>> tc-forge-dev mailing list
>> tc-forge-dev@...
>> http://lists.terracotta.org/mailman/listinfo/tc-forge-dev_______________________________________________
tc-users mailing list
tc-users@...
http://lists.terracotta.org/mailman/listinfo/tc-users
|
|
Re: [tc-forge-dev] New features for next Tim-Messaging releases
On Sat, Aug 15, 2009 at 1:58 AM, Taylor
Gautier< tgautier@...> wrote:
> I would add serialization of messages to the list. Serializing should
> provide big improvements to throughput and messages behave more like
> messages and not distributed memory.
That's definitely a great idea: I'll put it on top of my list.
Talking about the serialized object, would it be better (for
increasing throughput) to serialize and share it in terracotta as a
string, or as a byte array?
--
Sergio Bossa
Software Passionate and Open Source Enthusiast.
URL: http://www.linkedin.com/in/sergiob_______________________________________________
tc-users mailing list
tc-users@...
http://lists.terracotta.org/mailman/listinfo/tc-users
|
|
Re: [tc-forge-dev] New features for next Tim-Messaging releases
On Fri, Aug 14, 2009 at 7:13 PM, Alex Miller< amiller@...> wrote:
> Sergio, maybe it would be good to put
> these ideas up where people could discuss? Like your blog or our forums.
> I'm sure we could point a little traffic your way somehow.
Thanks Alex.
I'll write a short blog post about it as soon as possible, and let you know.
Cheers,
Sergio B.
--
Sergio Bossa
Software Passionate and Open Source Enthusiast.
URL: http://www.linkedin.com/in/sergiob_______________________________________________
tc-users mailing list
tc-users@...
http://lists.terracotta.org/mailman/listinfo/tc-users
|
|
Re: New features for next Tim-Messaging releases
RE: [tc-users] New features for next Tim-Messaging releases
Yeah the rest-based interfaces sound real useful. We did something similar with the Authinator for a distributed cache at https://svn.terracotta.org/repo/forge/projects/authinator/trunk
The serialization based framework also sounds real useful.
Cheers
-----Original Message-----
From: tc-users-bounces@... [tc-users-bounces@...] On Behalf Of Sergio Bossa
Sent: Friday, August 14, 2009 10:04 AM
To: tc-forge-dev; tc-users
Subject: [tc-users] New features for next Tim-Messaging releases
Hi guys,
some time has passed since the latest Tim-Messaging 2.0 release, and
I'm now starting to think about new features for next releases.
Here are a few new features I was thinking at:
1) Given that data-affinity is more and more import, it may be good to
improve affinity-routing for tim-masterworker.
2) It may be cool to add a json-over-http interface to tim-pipes,
providing restful access to topologies and pipes.
3) It may be even cooler to add a restful interface to
tim-masterworker, implementing a scalable, fault-tolerant, rest-based
master/worker server.
What do you think?
Do you have any other particular idea/feature/suggestion?
Thanks,
Cheers!
Sergio B.
--
Sergio Bossa
Software Passionate and Open Source Enthusiast.
URL: http://www.linkedin.com/in/sergiob
_______________________________________________
tc-users mailing list
tc-users@...
http://lists.terracotta.org/mailman/listinfo/tc-users
_______________________________________________
tc-users mailing list
tc-users@...
http://lists.terracotta.org/mailman/listinfo/tc-users
|
|
Re: [tc-forge-dev] New features for next Tim-Messaging releases
Strings should be best.
On Aug 15, 2009, at 5:58 AM, Sergio Bossa < sergio.bossa@...>
wrote:
> On Sat, Aug 15, 2009 at 1:58 AM, Taylor
> Gautier< tgautier@...> wrote:
>
>> I would add serialization of messages to the list. Serializing should
>> provide big improvements to throughput and messages behave more like
>> messages and not distributed memory.
>
> That's definitely a great idea: I'll put it on top of my list.
>
> Talking about the serialized object, would it be better (for
> increasing throughput) to serialize and share it in terracotta as a
> string, or as a byte array?
>
> --
> Sergio Bossa
> Software Passionate and Open Source Enthusiast.
> URL: http://www.linkedin.com/in/sergiob_______________________________________________
tc-users mailing list
tc-users@...
http://lists.terracotta.org/mailman/listinfo/tc-users
|
|
Re: [tc-forge-dev] New features for next Tim-Messaging releases
Is there a way to let a rouge worker timeout if it gets stuck?
I've scanned the javadocs but couldn't find anything like that.
Sergio Bossa wrote:
> Hi guys,
>
> some time has passed since the latest Tim-Messaging 2.0 release, and
> I'm now starting to think about new features for next releases.
> Here are a few new features I was thinking at:
>
> 1) Given that data-affinity is more and more import, it may be good to
> improve affinity-routing for tim-masterworker.
> 2) It may be cool to add a json-over-http interface to tim-pipes,
> providing restful access to topologies and pipes.
> 3) It may be even cooler to add a restful interface to
> tim-masterworker, implementing a scalable, fault-tolerant, rest-based
> master/worker server.
>
> What do you think?
> Do you have any other particular idea/feature/suggestion?
>
> Thanks,
> Cheers!
>
> Sergio B.
>
>
_______________________________________________
tc-users mailing list
tc-users@...
http://lists.terracotta.org/mailman/listinfo/tc-users
|
|
Re: [tc-forge-dev] New features for next Tim-Messaging releases
|
|
Re: [tc-forge-dev] New features for next Tim-Messaging releases
On Friday 14 August 2009 01:04:24 pm Sergio Bossa wrote:
> some time has passed since the latest Tim-Messaging 2.0 release, and
> I'm now starting to think about new features for next releases.
> Here are a few new features I was thinking at:
>
> 1) Given that data-affinity is more and more import, it may be good to
> improve affinity-routing for tim-masterworker.
I definitely think this could be useful. Would there be some way of mapping a
work item to the data that it uses in some way?
> 2) It may be cool to add a json-over-http interface to tim-pipes,
> providing restful access to topologies and pipes.
> 3) It may be even cooler to add a restful interface to
> tim-masterworker, implementing a scalable, fault-tolerant, rest-based
> master/worker server.
We have actually built something like this on top of tim-messaging, which we
use to build and publish artifacts and web sites for the Terracotta Forge
projects. If you are interested in taking a look you can check out the
project from here:
https://svn.terracotta.org/repo/forge/projects/DistributedWorkQueue/trunkHaving a RESTful interface built-in to tim-messaging itself would of course be
very helpful to build systems such as that.
--
Jason Voegele
It is always the best policy to tell the truth, unless, of course,
you are an exceptionally good liar.
-- Jerome K. Jerome
_______________________________________________
tc-users mailing list
tc-users@...
http://lists.terracotta.org/mailman/listinfo/tc-users
|
|
Re: [tc-forge-dev] New features for next Tim-Messaging releases
On Mon, Aug 24, 2009 at 4:50 PM, Jason
Voegele< jvoegele@...> wrote:
> I definitely think this could be useful. Would there be some way of mapping a
> work item to the data that it uses in some way?
Isn't that somewhat accomplished by using Terracotta Cluster Events?
Or did I miss your point?
> We have actually built something like this on top of tim-messaging, which we
> use to build and publish artifacts and web sites for the Terracotta Forge
> projects. If you are interested in taking a look you can check out the
> project from here:
>
> https://svn.terracotta.org/repo/forge/projects/DistributedWorkQueue/trunkCool, I didn't know that: glad to see terracotta eats its own food :)
> Having a RESTful interface built-in to tim-messaging itself would of course be
> very helpful to build systems such as that.
Agree: it will definitely be in the todo-list.
Thanks for your feedback,
Cheers,
Sergio B.
--
Sergio Bossa
Software Passionate and Open Source Enthusiast.
URL: http://www.linkedin.com/in/sergiob_______________________________________________
tc-users mailing list
tc-users@...
http://lists.terracotta.org/mailman/listinfo/tc-users
|
|
Re: New features for next Tim-Messaging releases
|
|
Re: New features for next Tim-Messaging releases
Some parts of this message have been removed.
Learn more about Nabble's security policy.
Sounds great. Just make sure on the serialization part - what you are trying to optimize is the object count in Terracotta. So a simple test would be to run say 10,000 objects through TC and observe the object created count in the console and make sure it is constant.
Anything else indicates clustered objects (wrappers etc) that you must optimize out because the entire point is to ensure that message puts do not add to the object count. ----- Original Message ----- From: "Sergio Bossa" < sergio.bossa@...> To: "tc-forge-dev" < tc-forge-dev@...>, "tc-users" < tc-users@...> Sent: Tuesday, September 1, 2009 3:11:59 AM GMT -08:00 US/Canada Pacific Subject: Re: [tc-users] New features for next Tim-Messaging releases Hi guys, first of all: thanks everyone for your feedback! FYI, I've planned the following new features for the next release, which I'd like to cut in a few weeks: https://jira.terracotta.org/jira/browse/FORGE-549 (message serialization support) https://jira.terracotta.org/jira/browse/FORGE-548 (selective consumer support) https://jira.terracotta.org/jira/browse/FORGE-518 (message expiration support) I'm currently working on message serialization, which is almost finished and supports Java Serialization, Json, XML and GoogleProtoBuf encoding. Feedback is as always welcome. Cheers, Sergio B. -- Sergio Bossa Software Passionate and Open Source Enthusiast. URL: http://www.linkedin.com/in/sergiob _______________________________________________ tc-users mailing list tc-users@...http://lists.terracotta.org/mailman/listinfo/tc-users _______________________________________________
tc-users mailing list
tc-users@...
http://lists.terracotta.org/mailman/listinfo/tc-users
|
|
Re: [tc-forge-dev] New features for next Tim-Messaging releases
On Thu, Sep 3, 2009 at 3:57 AM, Taylor
Gautier< tgautier@...> wrote:
> Sounds great. Just make sure on the serialization part - what you are
> trying to optimize is the object count in Terracotta. So a simple test
> would be to run say 10,000 objects through TC and observe the object created
> count in the console and make sure it is constant.
It works just like that, now: live objects count is kept constant.
--
Sergio Bossa
Software Passionate and Open Source Enthusiast.
URL: http://www.linkedin.com/in/sergiob_______________________________________________
tc-users mailing list
tc-users@...
http://lists.terracotta.org/mailman/listinfo/tc-users
|
