« Return to Thread: [RFC PATCH 0/4] (Take 2): transcendent memory ("tmem") for Linux
Re: [Xen-devel] Re: [RFC PATCH 0/4] (Take 2): transcendent memory ("tmem") for Linux
by Anthony Liguori-2
::
Rate this Message:
Dan Magenheimer wrote:
> Hi Anthony --
>
> Thanks for the comments.
>
>
>> I have trouble mapping this to a VMM capable of overcommit
>> without just coming back to CMM2.
>>
>> In CMM2 parlance, ephemeral tmem pools is just normal kernel memory
>> marked in the volatile state, no?
>>
>
> They are similar in concept, but a volatile-marked kernel page
> is still a kernel page, can be changed by a kernel (or user)
> store instruction, and counts as part of the memory used
> by the VM. An ephemeral tmem page cannot be directly written
> by a kernel (or user) store,
Why does tmem require a special store?
A VMM can trap write operations pages can be stored on disk
transparently by the VMM if necessary. I guess that's the bit I'm missing.
>> It seems to me that an architecture built around hinting
>> would be more
>> robust than having to use separate memory pools for this type
>> of memory
>> (especially since you are requiring a copy to/from the pool).
>>
>
> Depends on what you mean by robust, I suppose. Once you
> understand the basics of tmem, it is very simple and this
> is borne out in the low invasiveness of the Linux patch.
> Simplicity is another form of robustness.
>
The main disadvantage I see is that you need to explicitly convert
portions of the kernel to use a data copying API. That seems like an
invasive change to me. Hinting on the other hand can be done in a
less-invasive way.
I'm not really arguing against tmem, just the need to have explicit
get/put mechanisms for the transcendent memory areas.
> The copy may be expensive on an older machine, but on newer
> machines copying a page is relatively inexpensive.
I don't think that's a true statement at all :-) If you had a workload
where data never came into the CPU cache (zero-copy) and now you
introduce a copy, even with new system, you're going to see a
significant performance hit.
> On a reasonable
> multi-VM-kernbench-like benchmark I'll be presenting at Linux
> Symposium next week, the overhead is on the order of 0.01%
> for a fairly significant savings in IOs.
>
But how would something like specweb do where you should be doing
zero-copy IO from the disk to the network? This is the area where I
would be concerned. For something like kernbench, you're already
bringing the disk data into the CPU cache anyway so I can appreciate
that the copy could get lost in the noise.
Regards,
Anthony Liguori
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
> Hi Anthony --
>
> Thanks for the comments.
>
>
>> I have trouble mapping this to a VMM capable of overcommit
>> without just coming back to CMM2.
>>
>> In CMM2 parlance, ephemeral tmem pools is just normal kernel memory
>> marked in the volatile state, no?
>>
>
> They are similar in concept, but a volatile-marked kernel page
> is still a kernel page, can be changed by a kernel (or user)
> store instruction, and counts as part of the memory used
> by the VM. An ephemeral tmem page cannot be directly written
> by a kernel (or user) store,
Why does tmem require a special store?
A VMM can trap write operations pages can be stored on disk
transparently by the VMM if necessary. I guess that's the bit I'm missing.
>> It seems to me that an architecture built around hinting
>> would be more
>> robust than having to use separate memory pools for this type
>> of memory
>> (especially since you are requiring a copy to/from the pool).
>>
>
> Depends on what you mean by robust, I suppose. Once you
> understand the basics of tmem, it is very simple and this
> is borne out in the low invasiveness of the Linux patch.
> Simplicity is another form of robustness.
>
The main disadvantage I see is that you need to explicitly convert
portions of the kernel to use a data copying API. That seems like an
invasive change to me. Hinting on the other hand can be done in a
less-invasive way.
I'm not really arguing against tmem, just the need to have explicit
get/put mechanisms for the transcendent memory areas.
> The copy may be expensive on an older machine, but on newer
> machines copying a page is relatively inexpensive.
I don't think that's a true statement at all :-) If you had a workload
where data never came into the CPU cache (zero-copy) and now you
introduce a copy, even with new system, you're going to see a
significant performance hit.
> On a reasonable
> multi-VM-kernbench-like benchmark I'll be presenting at Linux
> Symposium next week, the overhead is on the order of 0.01%
> for a fairly significant savings in IOs.
>
But how would something like specweb do where you should be doing
zero-copy IO from the disk to the network? This is the area where I
would be concerned. For something like kernbench, you're already
bringing the disk data into the CPU cache anyway so I can appreciate
that the copy could get lost in the noise.
Regards,
Anthony Liguori
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
« Return to Thread: [RFC PATCH 0/4] (Take 2): transcendent memory ("tmem") for Linux
| Free embeddable forum powered by Nabble | Forum Help |
