Announce: New feature-complete version of the BeIT Memcached Client released.

[Henrik Schröder-3]

Hey everyone,

For those of you using memcached in a .Net environment, I am now happy to announce that the BeIT Memcached Client now supports check and set, append and prepend. Since those were the last missing features, this means that our client is now feature complete. All the commands and all of their optional parameters are now supported!

If you want to use cas, there's some new methods in the client called Gets which operate exactly like Get, except they also return the cas unique value as an out parameter. The new CheckAndSet methods of course require a cas unique value, and they return an enum that corresponds to the four different results you can get from a cas operation.

So, if anyone out there was waiting for check and set or append and prepend, you can download the latest version at: http://code.google.com/p/beitmemcached/

Enjoy!


/Henrik Schröder

[Dustin Sallings]

On May 30, 2008, at 16:30, Henrik Schröder wrote:

> For those of you using memcached in a .Net environment, I am now  
> happy to announce that the BeIT Memcached Client now supports check  
> and set, append and prepend. Since those were the last missing  
> features, this means that our client is now feature complete. All  
> the commands and all of their optional parameters are now supported!


        Have you been following the binary protocol development at all?  CAS  
is more of a fundamental property of operations there as opposed to  
being an operation on its own.  e.g. you can CAS a delete.

--
Dustin Sallings

[Henrik Schröder-3]

On Sat, May 31, 2008 at 1:37 AM, Dustin Sallings <dustin@...> wrote:

       Have you been following the binary protocol development at all?  CAS is more of a fundamental property of operations there as opposed to being an operation on its own.  e.g. you can CAS a delete.

 
No I haven't, for a bunch of reasons. First is that the current feature-set is good enough for our own needs, and second is that we're using the windows port of the server, and I think we'll see the binary protocol in that version very late, if at all. The only feature that's interesting for us that's on the horizon is tags, that would be really useful.

So, I doubt I'll add binary protocol support soon to our client unless some really strong need for it would emerge, and it doesn't look likely in the near future.


/Henrik Schröder

[Dustin Sallings]

On May 31, 2008, at 7:41, Henrik Schröder wrote:

> No I haven't, for a bunch of reasons. First is that the current  
> feature-set is good enough for our own needs,

        That's sort of why I responded.  I read ``feature-complete'' as  
``supporting all features.''

        In particular, CAS isn't fully complete in 1.2.5 and will probably  
not be backported to the text protocol due to the protocol disruption  
it'd likely introduce (we do CAS on append, prepend, set, add, update,  
and delete at least as well as returning CAS values from all commands).

> and second is that we're using the windows port of the server, and I  
> think we'll see the binary protocol in that version very late, if at  
> all.

        There seem to be very few people with Windows server interest.  Do  
you have any time to put into helping memcached continue working on  
the Windows platform?

--
Dustin Sallings

[Henrik Schröder-3]

On Sun, Jun 1, 2008 at 12:49 AM, Dustin Sallings <dustin@...> wrote:

       That's sort of why I responded.  I read ``feature-complete'' as ``supporting all features.''

Well, technically it supports all features in the 1.2.4 server. :-)

(Oh okay, it doesn't support the UDP protocol, but... who does?)
 

       In particular, CAS isn't fully complete in 1.2.5 and will probably not be backported to the text protocol due to the protocol disruption it'd likely introduce (we do CAS on append, prepend, set, add, update, and delete at least as well as returning CAS values from all commands).

Sounds reasonable, it's pretty evident that CAS is an afterthought with the separate gets and cas commands. I actually wondered about the reason for the binary protocol, I would guess that the difference in parsing variable-length text strings and fixed-length byte arrays is totally negligible compared to network latency, but I guess it makes it easier to add things like CAS to everything. :-)
 

       There seem to be very few people with Windows server interest.  Do you have any time to put into helping memcached continue working on the Windows platform?

Yes, it's pretty sad actually that it hasn't caught on very well in the Windows world. Sadly, I'm not a (good) C programmer, so I can't help. At my company we're pretty concerned about the bad performance of the Windows port of the memcached server, it affects our website Nonoba, so we might end up investing some resource sin getting it to work better, but I really can't promise anything. It's definitely in our interest to have a good working version of the server for Windows though, so we'll see. I suspect that the problems are related to the Windows port of libevent, and that might be pretty tricky to track down, and not something I would want to do with my meager C skills. :-)


/Henrik Schröder

[jrodenburg]

we're pretty concerned about the bad performance of the Windows port of the memcached server,

Henrik - can you elaborate on what you've found with this?  I'm not looking to resolve the issues, just trying to get a better picture of where the bodies are buried, and to convince an all-windows shop that it's OK to run a few linux instances to support certain application services.

[Stephen Johnston]

The problem we've seen is that memcached eats a complete core out of our four cores. The transactional (get, set, etc.) performance seems fine, but losing one out of 4 cores is a bit rough.

 

-Stephen

On Thu, Jun 5, 2008 at 2:01 PM, Jeff Rodenburg <jeff.rodenburg@...> wrote:

we're pretty concerned about the bad performance of the Windows port of the memcached server,

Henrik - can you elaborate on what you've found with this?  I'm not looking to resolve the issues, just trying to get a better picture of where the bodies are buried, and to convince an all-windows shop that it's OK to run a few linux instances to support certain application services.

--
Stephen Johnston
President/CEO
Guild Launch, LLC
http://www.guildlaunch.com/

Communities for online gamers, guilds and clans.

[Dustin Sallings]

On Jun 1, 2008, at 9:12, Henrik Schröder wrote:

> (Oh okay, it doesn't support the UDP protocol, but... who does?)

        Well, Brian supports it.  I haven't had a need for it myself yet,  
though.

> Sounds reasonable, it's pretty evident that CAS is an afterthought  
> with the separate gets and cas commands.

        New commands were mainly to keep existing clients happy.

> I actually wondered about the reason for the binary protocol, I  
> would guess that the difference in parsing variable-length text  
> strings and fixed-length byte arrays is totally negligible compared  
> to network latency, but I guess it makes it easier to add things  
> like CAS to everything. :-)

        The binary protocol is a lot easier to work with both in servers and  
clients, and certainly less work to ``parse.''  The cas-on-everything  
wasn't added until the fourth revision, though.

> Yes, it's pretty sad actually that it hasn't caught on very well in  
> the Windows world. Sadly, I'm not a (good) C programmer, so I can't  
> help. At my company we're pretty concerned about the bad performance  
> of the Windows port of the memcached server, it affects our website  
> Nonoba, so we might end up investing some resource sin getting it to  
> work better, but I really can't promise anything. It's definitely in  
> our interest to have a good working version of the server for  
> Windows though, so we'll see. I suspect that the problems are  
> related to the Windows port of libevent, and that might be pretty  
> tricky to track down, and not something I would want to do with my  
> meager C skills. :-)

        There are lots of ways to help.  Simply ensuring it builds is a good  
start.  I wouldn't even know how to go about compiling a C program in  
Windows if I had a place to do it.  :)

--
Dustin Sallings

[Henrik Schröder-3]

On Thu, Jun 5, 2008 at 8:01 PM, Jeff Rodenburg <jeff.rodenburg@...> wrote:

Henrik - can you elaborate on what you've found with this?  I'm not looking to resolve the issues, just trying to get a better picture of where the bodies are buried, and to convince an all-windows shop that it's OK to run a few linux instances to support certain application services.

About half a year ago we implemented memcached for a customer of ours, and we decided to try running the memcached servers on the physical web servers. Those were pretty old machines, a mixed bunch, and maybe 4 years old on average. After a little while, the memcached process on each just ate all the available CPU and promptly starved the IIS webserver processes. causing the entire web application to go down. However, even with no traffic to the website and no pressure on the memcached servers, they still consumed all CPU. We switched it around and made a separate memcached server cluster, but those machines were still brought down by memcached consuming all CPU. Finally, we installed some Linux on the same machines, and they just stay at 0% CPU while serving the entire web application perfectly. So on the same hardware and the same load, the Linux version totally outperforms the Windows version on such a scale that it cannot be platform differences, there's gotta be a bug or five in the Windows version which causes it to consume a lot more resources than it should. I would guess that the culprit is the libevent port or a combination of memcached + libevent that just doesn't play along well on Windows.

On our current project, we run memcached on two servers that are also web servers, and on both machines the memcached process consumes exactly 25% CPU. The weird thing is that those two servers have different hardware. One is a two-processor dual core Xeon at 2,5GHz, and the other is a two-processor dual core Xeon at 1,6GHz. The first one runs Windows Server 2008, the other Windows Server 2003. But the memcached process on each takes up exactly 25% CPU all the time. I can also see on the stats that the second server gets more memcached traffic than the first one, so the second server is slower than the first and gets more traffic, but the CPU use is 25% on both servers.

I first assumed that the memcached process just ate an entire core on both machines to produce such a perfect load number, but when I look at the CPU graphs, it doesn't. It's evenly spread, yet somehow capped at 25%. So the CPU consumption clearly isn't proportional to the load on the server or the speed of the CPU, which means it probably is a timing/polling(?) error of some sort, that somehow isn't confined to a single thread.

But the worst thing is that I don't know how to reproduce this state. If I restart the memcached processes, they go down to 0% CPU, stay there for a while, but later pop up to 25% when the web application has been running for a while. We did extensive testing when we developed our own memcached client, but we never ever encountered it then even though that put a lot more pressure on it than what the live web application does.

And that's about what I know about the problems.


/Henrik Schröder

[Henrik Schröder-3]

On Thu, Jun 5, 2008 at 10:26 PM, Dustin Sallings <dustin@...> wrote:

       There are lots of ways to help.  Simply ensuring it builds is a good start.  I wouldn't even know how to go about compiling a C program in Windows if I had a place to do it.  :)

Unfortunately, neither do I. :-)

I have plenty of friends who do though, so I can make a try. I have no idea which version of libevent the 1.2.4 Windows version of memcached is compiled with, maybe just upgrading that will help? We'll see.


/Henrik Schröder

[Henrik Schröder-3]

On Fri, Jun 6, 2008 at 12:27 AM, Henrik Schröder <skrolle@...> wrote:

On Thu, Jun 5, 2008 at 8:01 PM, Jeff Rodenburg <jeff.rodenburg@...> wrote:

Henrik - can you elaborate on what you've found with this?  I'm not looking to resolve the issues, just trying to get a better picture of where the bodies are buried, and to convince an all-windows shop that it's OK to run a few linux instances to support certain application services.

On our current project, we run memcached on two servers that are also web servers, and on both machines the memcached process consumes exactly 25% CPU. The weird thing is that those two servers have different hardware. One is a two-processor dual core Xeon at 2,5GHz, and the other is a two-processor dual core Xeon at 1,6GHz. The first one runs Windows Server 2008, the other Windows Server 2003. But the memcached process on each takes up exactly 25% CPU all the time. I can also see on the stats that the second server gets more memcached traffic than the first one, so the second server is slower than the first and gets more traffic, but the CPU use is 25% on both servers.

Ok, thanks to Brodie Thiesfield who managed to produce working Visual Studio projects of Libevent 1.4.4 and Memcached 1.2.5, I've compiled my own version. I took his project, added the old memcached icon (These things are important! :) ), fixed a file version number, and compiled everything in my Visual Studio 2005 with whatever optimizations it can do, and finally got to deploy this version live.

It's been running for a day now, and so far it looks good, still at 0% CPU utilization so hopefully whatever problems the older windows versions of memcached had are gone. I'll let it run for a week, and if it's still behaving after that time, I'll try to make available our binary for those that are interested.


/Henrik Schröder

[Stephen Johnston]

I would be very much interested in a Windows Binary that didn't eat up one of our cores.

 

-Stephen

On Wed, Jul 2, 2008 at 5:34 AM, Henrik Schröder <skrolle@...> wrote:

On Fri, Jun 6, 2008 at 12:27 AM, Henrik Schröder <skrolle@...> wrote:

On Thu, Jun 5, 2008 at 8:01 PM, Jeff Rodenburg <jeff.rodenburg@...> wrote:

Henrik - can you elaborate on what you've found with this?  I'm not looking to resolve the issues, just trying to get a better picture of where the bodies are buried, and to convince an all-windows shop that it's OK to run a few linux instances to support certain application services.

On our current project, we run memcached on two servers that are also web servers, and on both machines the memcached process consumes exactly 25% CPU. The weird thing is that those two servers have different hardware. One is a two-processor dual core Xeon at 2,5GHz, and the other is a two-processor dual core Xeon at 1,6GHz. The first one runs Windows Server 2008, the other Windows Server 2003. But the memcached process on each takes up exactly 25% CPU all the time. I can also see on the stats that the second server gets more memcached traffic than the first one, so the second server is slower than the first and gets more traffic, but the CPU use is 25% on both servers.

Ok, thanks to Brodie Thiesfield who managed to produce working Visual Studio projects of Libevent 1.4.4 and Memcached 1.2.5, I've compiled my own version. I took his project, added the old memcached icon (These things are important! :) ), fixed a file version number, and compiled everything in my Visual Studio 2005 with whatever optimizations it can do, and finally got to deploy this version live.

It's been running for a day now, and so far it looks good, still at 0% CPU utilization so hopefully whatever problems the older windows versions of memcached had are gone. I'll let it run for a week, and if it's still behaving after that time, I'll try to make available our binary for those that are interested.


/Henrik Schröder

[Dean Harding]

Stephen Johnston wrote:
> I would be very much interested in a Windows Binary that didn't eat up
> one of our cores.

We use a version of 1.2.4 that was ported to win32 that we subsequently
modified to compile in 64-bit mode that I could share if people are
interested.

Shouldn't take much effort for me to update it to 1.2.5 or whatever the
latest is, either (once we got it working, we didn't really bother with
updating it :p)

Dean.

[Henrik Schröder-3]

On Wed, Jul 2, 2008 at 1:46 PM, Dean Harding <dean.harding@...> wrote:

We use a version of 1.2.4 that was ported to win32 that we subsequently
modified to compile in 64-bit mode that I could share if people are
interested.

Which version of libevent did you compile that one against?


/Henrik Schröder

[Henrik Schröder-3]

I put up an archive at http://code.google.com/p/beitmemcached/downloads/list, grab it from there and test it out. Would love to get more people testing it so we know if the weird CPU usage is gone for good.


/Henrik Schröder

On Wed, Jul 2, 2008 at 1:33 PM, Stephen Johnston <stephen.johnston@...> wrote:

I would be very much interested in a Windows Binary that didn't eat up one of our cores.

 

-Stephen

On Wed, Jul 2, 2008 at 5:34 AM, Henrik Schröder <skrolle@...> wrote:

On Fri, Jun 6, 2008 at 12:27 AM, Henrik Schröder <skrolle@...> wrote:

On Thu, Jun 5, 2008 at 8:01 PM, Jeff Rodenburg <jeff.rodenburg@...> wrote:

Henrik - can you elaborate on what you've found with this?  I'm not looking to resolve the issues, just trying to get a better picture of where the bodies are buried, and to convince an all-windows shop that it's OK to run a few linux instances to support certain application services.

On our current project, we run memcached on two servers that are also web servers, and on both machines the memcached process consumes exactly 25% CPU. The weird thing is that those two servers have different hardware. One is a two-processor dual core Xeon at 2,5GHz, and the other is a two-processor dual core Xeon at 1,6GHz. The first one runs Windows Server 2008, the other Windows Server 2003. But the memcached process on each takes up exactly 25% CPU all the time. I can also see on the stats that the second server gets more memcached traffic than the first one, so the second server is slower than the first and gets more traffic, but the CPU use is 25% on both servers.

Ok, thanks to Brodie Thiesfield who managed to produce working Visual Studio projects of Libevent 1.4.4 and Memcached 1.2.5, I've compiled my own version. I took his project, added the old memcached icon (These things are important! :) ), fixed a file version number, and compiled everything in my Visual Studio 2005 with whatever optimizations it can do, and finally got to deploy this version live.

It's been running for a day now, and so far it looks good, still at 0% CPU utilization so hopefully whatever problems the older windows versions of memcached had are gone. I'll let it run for a week, and if it's still behaving after that time, I'll try to make available our binary for those that are interested.


/Henrik Schröder

[Josef Finsel]

Is that the 64 bit or 32 bit version?

Thanks


"If you see a whole thing - it seems that it's always beautiful. Planets, lives... But up close a world's all dirt and rocks. And day to day, life's a hard job, you get tired, you lose the pattern."
Ursula K. Le Guin

On Wed, Jul 2, 2008 at 7:55 AM, Henrik Schröder <skrolle@...> wrote:

I put up an archive at http://code.google.com/p/beitmemcached/downloads/list, grab it from there and test it out. Would love to get more people testing it so we know if the weird CPU usage is gone for good.


/Henrik Schröder

On Wed, Jul 2, 2008 at 1:33 PM, Stephen Johnston <stephen.johnston@...> wrote:

I would be very much interested in a Windows Binary that didn't eat up one of our cores.

 

-Stephen

On Wed, Jul 2, 2008 at 5:34 AM, Henrik Schröder <skrolle@...> wrote:

On Fri, Jun 6, 2008 at 12:27 AM, Henrik Schröder <skrolle@...> wrote:

On Thu, Jun 5, 2008 at 8:01 PM, Jeff Rodenburg <jeff.rodenburg@...> wrote:

Henrik - can you elaborate on what you've found with this?  I'm not looking to resolve the issues, just trying to get a better picture of where the bodies are buried, and to convince an all-windows shop that it's OK to run a few linux instances to support certain application services.

On our current project, we run memcached on two servers that are also web servers, and on both machines the memcached process consumes exactly 25% CPU. The weird thing is that those two servers have different hardware. One is a two-processor dual core Xeon at 2,5GHz, and the other is a two-processor dual core Xeon at 1,6GHz. The first one runs Windows Server 2008, the other Windows Server 2003. But the memcached process on each takes up exactly 25% CPU all the time. I can also see on the stats that the second server gets more memcached traffic than the first one, so the second server is slower than the first and gets more traffic, but the CPU use is 25% on both servers.

Ok, thanks to Brodie Thiesfield who managed to produce working Visual Studio projects of Libevent 1.4.4 and Memcached 1.2.5, I've compiled my own version. I took his project, added the old memcached icon (These things are important! :) ), fixed a file version number, and compiled everything in my Visual Studio 2005 with whatever optimizations it can do, and finally got to deploy this version live.

It's been running for a day now, and so far it looks good, still at 0% CPU utilization so hopefully whatever problems the older windows versions of memcached had are gone. I'll let it run for a week, and if it's still behaving after that time, I'll try to make available our binary for those that are interested.


/Henrik Schröder

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "memcached" group.
To post to this group, send email to memcached@...
To unsubscribe from this group, send email to memcached-unsubscribe@...
For more options, visit this group at http://groups.google.com/group/memcached?hl=en
-~----------~----~----~----~------~----~------~--~---

[Henrik Schröder-3]

32-bit, I never managed to compile a 64-bit version, and since I don't need one myself I didn't spend a lot of effort on getting one. Feel free to use Brodie's working src archives here: http://code.jellycan.com/memcached/ to compile one yourself.


/Henrik Schröder

On Wed, Jul 2, 2008 at 2:00 PM, Josef Finsel <carpdeus@...> wrote:

Is that the 64 bit or 32 bit version?

Thanks


"If you see a whole thing - it seems that it's always beautiful. Planets, lives... But up close a world's all dirt and rocks. And day to day, life's a hard job, you get tired, you lose the pattern."
Ursula K. Le Guin

On Wed, Jul 2, 2008 at 7:55 AM, Henrik Schröder <skrolle@...> wrote:

I put up an archive at http://code.google.com/p/beitmemcached/downloads/list, grab it from there and test it out. Would love to get more people testing it so we know if the weird CPU usage is gone for good.


/Henrik Schröder

On Wed, Jul 2, 2008 at 1:33 PM, Stephen Johnston <stephen.johnston@...> wrote:

I would be very much interested in a Windows Binary that didn't eat up one of our cores.

 

-Stephen

On Wed, Jul 2, 2008 at 5:34 AM, Henrik Schröder <skrolle@...> wrote:

On Fri, Jun 6, 2008 at 12:27 AM, Henrik Schröder <skrolle@...> wrote:

On Thu, Jun 5, 2008 at 8:01 PM, Jeff Rodenburg <jeff.rodenburg@...> wrote:

Henrik - can you elaborate on what you've found with this?  I'm not looking to resolve the issues, just trying to get a better picture of where the bodies are buried, and to convince an all-windows shop that it's OK to run a few linux instances to support certain application services.

On our current project, we run memcached on two servers that are also web servers, and on both machines the memcached process consumes exactly 25% CPU. The weird thing is that those two servers have different hardware. One is a two-processor dual core Xeon at 2,5GHz, and the other is a two-processor dual core Xeon at 1,6GHz. The first one runs Windows Server 2008, the other Windows Server 2003. But the memcached process on each takes up exactly 25% CPU all the time. I can also see on the stats that the second server gets more memcached traffic than the first one, so the second server is slower than the first and gets more traffic, but the CPU use is 25% on both servers.

Ok, thanks to Brodie Thiesfield who managed to produce working Visual Studio projects of Libevent 1.4.4 and Memcached 1.2.5, I've compiled my own version. I took his project, added the old memcached icon (These things are important! :) ), fixed a file version number, and compiled everything in my Visual Studio 2005 with whatever optimizations it can do, and finally got to deploy this version live.

It's been running for a day now, and so far it looks good, still at 0% CPU utilization so hopefully whatever problems the older windows versions of memcached had are gone. I'll let it run for a week, and if it's still behaving after that time, I'll try to make available our binary for those that are interested.


/Henrik Schröder

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "memcached" group.
To post to this group, send email to memcached@...
To unsubscribe from this group, send email to memcached-unsubscribe@...
For more options, visit this group at http://groups.google.com/group/memcached?hl=en
-~----------~----~----~----~------~----~------~--~---