Generic Polyhedron Primitives

[Timothy M. Shead]

Folks:

A heads-up to get ready for the switch to generic polyhedron primitives
in the next day-or-so.  You can expect some compile problems and a lot
of broken tests while we sort-out the details.  Bart, I'll need you to
look at the subdivision surface library, you should prepare yourself for
a world without mesh->polyhedra.

Cheers,
Tim

[tshead.vcf]

---

begin:vcard
fn:Timothy Shead
n:Shead;Timothy
org:www.k-3d.org
email;internet:tshead@...
title:Founder
x-mozilla-html:FALSE
url:www.k-3d.org
version:2.1
end:vcard

---

------------------------------------------------------------------------------
Stay on top of everything new and different, both inside and
around Java (TM) technology - register by April 22, and save
$200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
300 plus technical and hands-on sessions. Register today.
Use priority code J9JMT32. http://p.sf.net/sfu/p
_______________________________________________
K3d-development mailing list
K3d-development@...
https://lists.sourceforge.net/lists/listinfo/k3d-development

[Timothy M. Shead]

Timothy M. Shead wrote:

> A heads-up to get ready for the switch to generic polyhedron primitives
> in the next day-or-so.  You can expect some compile problems and a lot
> of broken tests while we sort-out the details.  Bart, I'll need you to
> look at the subdivision surface library, you should prepare yourself for
> a world without mesh->polyhedra.

As promised, the changeover to generic polyhedron primitives is in svn.
   Because I can't build every module there will be compile errors in
modules I haven't touched - Collada being one prominent example.  Bart,
as I mentioned, I'm leaving subdivision surfaces to you :)  The
opengl_painters module won't build until Bart is done with the SDS
library.  Anders, it would be great if you could look at the UV editor.
 I will be working on legacy mesh conversion and selection.

Cheers,
Tim

[tshead.vcf]

---

begin:vcard
fn:Timothy Shead
n:Shead;Timothy
org:www.k-3d.org
email;internet:tshead@...
title:Founder
x-mozilla-html:FALSE
url:www.k-3d.org
version:2.1
end:vcard

---

------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensign option that enables unlimited
royalty-free distribution of the report engine for externally facing
server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
K3d-development mailing list
K3d-development@...
https://lists.sourceforge.net/lists/listinfo/k3d-development

[bART Janssens-2]

On Sat, Apr 25, 2009 at 7:37 AM, Timothy M. Shead <tshead@...> wrote:
> Timothy M. Shead wrote:
>
>> A heads-up to get ready for the switch to generic polyhedron primitives
>> in the next day-or-so.  You can expect some compile problems and a lot
>> of broken tests while we sort-out the details.  Bart, I'll need you to
>> look at the subdivision surface library, you should prepare yourself for
>> a world without mesh->polyhedra.

OK, I propose to change the signature of
k3d::sds::catmull_clark_subdivider::create_mesh as follows:

void create_mesh(const k3d::mesh::points_t& InputPoints, const
k3d::polyhedron::const_primitive& InputPolyhedron, const
k3d::mesh::selection_t& InputFaceSelection, k3d::mesh::points_t&
OutputPoints, k3d::polyhedron::primitive& OutputPolyhedron,
k3d::inode* Node = 0);

Same with update_mesh. If you want, can you take a look at
k3dsdk/subdivision_surface/catmull_clark.h and see if you have any
other remarks on the interface?

Cheers,

--
Bart

------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensign option that enables unlimited
royalty-free distribution of the report engine for externally facing
server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
K3d-development mailing list
K3d-development@...
https://lists.sourceforge.net/lists/listinfo/k3d-development

[Timothy M. Shead]

Bart Janssens wrote:

> On Sat, Apr 25, 2009 at 7:37 AM, Timothy M. Shead <tshead@...> wrote:
>> Timothy M. Shead wrote:
>>
>>> A heads-up to get ready for the switch to generic polyhedron primitives
>>> in the next day-or-so.  You can expect some compile problems and a lot
>>> of broken tests while we sort-out the details.  Bart, I'll need you to
>>> look at the subdivision surface library, you should prepare yourself for
>>> a world without mesh->polyhedra.
>
> OK, I propose to change the signature of
> k3d::sds::catmull_clark_subdivider::create_mesh as follows:
>
> void create_mesh(const k3d::mesh::points_t& InputPoints, const
> k3d::polyhedron::const_primitive& InputPolyhedron, const
> k3d::mesh::selection_t& InputFaceSelection, k3d::mesh::points_t&
> OutputPoints, k3d::polyhedron::primitive& OutputPolyhedron,
> k3d::inode* Node = 0);
>
> Same with update_mesh. If you want, can you take a look at
> k3dsdk/subdivision_surface/catmull_clark.h and see if you have any
> other remarks on the interface?

Seems reasonable to me.

Cheers,
Tim

[tshead.vcf]

---

begin:vcard
fn:Timothy Shead
n:Shead;Timothy
org:www.k-3d.org
email;internet:tshead@...
title:Founder
x-mozilla-html:FALSE
url:www.k-3d.org
version:2.1
end:vcard

---

------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensign option that enables unlimited
royalty-free distribution of the report engine for externally facing
server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
K3d-development mailing list
K3d-development@...
https://lists.sourceforge.net/lists/listinfo/k3d-development

[Timothy M. Shead]

Bart:

Just a "heads up" that I moved the CatmullClark plugin to its own module
- its dependency on the SDS library was masking other plugins that need
to be updated in the mesh module.

Cheers,
Tim

[tshead.vcf]

---

begin:vcard
fn:Timothy Shead
n:Shead;Timothy
org:www.k-3d.org
email;internet:tshead@...
title:Founder
x-mozilla-html:FALSE
url:www.k-3d.org
version:2.1
end:vcard

---

------------------------------------------------------------------------------
Register Now & Save for Velocity, the Web Performance & Operations
Conference from O'Reilly Media. Velocity features a full day of
expert-led, hands-on workshops and two days of sessions from industry
leaders in dedicated Performance & Operations tracks. Use code vel09scf
and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf
_______________________________________________
K3d-development mailing list
K3d-development@...
https://lists.sourceforge.net/lists/listinfo/k3d-development

[Anders Stenberg-3]

I'll try to get around to fixing up the UV editor. For the record,
it's still pretty dysfunctional though, so if it isn't disabled by
default maybe it should be. (I was somewhat waiting for the GPs though
so maybe I could start working some on it again. We'll see. :)

/Anders

On Sat, Apr 25, 2009 at 7:37 AM, Timothy M. Shead <tshead@...> wrote:

> Timothy M. Shead wrote:
>
>> A heads-up to get ready for the switch to generic polyhedron primitives
>> in the next day-or-so.  You can expect some compile problems and a lot
>> of broken tests while we sort-out the details.  Bart, I'll need you to
>> look at the subdivision surface library, you should prepare yourself for
>> a world without mesh->polyhedra.
>
> As promised, the changeover to generic polyhedron primitives is in svn.
>   Because I can't build every module there will be compile errors in
> modules I haven't touched - Collada being one prominent example.  Bart,
> as I mentioned, I'm leaving subdivision surfaces to you :)  The
> opengl_painters module won't build until Bart is done with the SDS
> library.  Anders, it would be great if you could look at the UV editor.
>  I will be working on legacy mesh conversion and selection.
>
> Cheers,
> Tim
>
> ------------------------------------------------------------------------------
> Crystal Reports - New Free Runtime and 30 Day Trial
> Check out the new simplified licensign option that enables unlimited
> royalty-free distribution of the report engine for externally facing
> server and web deployment.
> http://p.sf.net/sfu/businessobjects
> _______________________________________________
> K3d-development mailing list
> K3d-development@...
> https://lists.sourceforge.net/lists/listinfo/k3d-development
>
>

------------------------------------------------------------------------------
Register Now & Save for Velocity, the Web Performance & Operations
Conference from O'Reilly Media. Velocity features a full day of
expert-led, hands-on workshops and two days of sessions from industry
leaders in dedicated Performance & Operations tracks. Use code vel09scf
and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf
_______________________________________________
K3d-development mailing list
K3d-development@...
https://lists.sourceforge.net/lists/listinfo/k3d-development

[bART Janssens-2]

On Tue, Apr 28, 2009 at 2:56 AM, Timothy M. Shead <tshead@...> wrote:
> Just a "heads up" that I moved the CatmullClark plugin to its own module
> - its dependency on the SDS library was masking other plugins that need
> to be updated in the mesh module.

OK. The SDS library compiles again now, but I still have to test it
before I commit. I also noticed that some painters (i.e. the
face_painter) will have to be updated as well.

Cheers,

--
Bart

------------------------------------------------------------------------------
Register Now & Save for Velocity, the Web Performance & Operations
Conference from O'Reilly Media. Velocity features a full day of
expert-led, hands-on workshops and two days of sessions from industry
leaders in dedicated Performance & Operations tracks. Use code vel09scf
and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf
_______________________________________________
K3d-development mailing list
K3d-development@...
https://lists.sourceforge.net/lists/listinfo/k3d-development

[Timothy M. Shead]

Anders Stenberg wrote:
> I'll try to get around to fixing up the UV editor. For the record,
> it's still pretty dysfunctional though, so if it isn't disabled by
> default maybe it should be. (I was somewhat waiting for the GPs though
> so maybe I could start working some on it again. We'll see. :)

Fair enough, I've disabled this by default, though I hope you won't let
it bit-rot ...

Cheers,
Tim

[tshead.vcf]

---

begin:vcard
fn:Timothy Shead
n:Shead;Timothy
org:www.k-3d.org
email;internet:tshead@...
title:Founder
x-mozilla-html:FALSE
url:www.k-3d.org
version:2.1
end:vcard

---

------------------------------------------------------------------------------
Register Now & Save for Velocity, the Web Performance & Operations
Conference from O'Reilly Media. Velocity features a full day of
expert-led, hands-on workshops and two days of sessions from industry
leaders in dedicated Performance & Operations tracks. Use code vel09scf
and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf
_______________________________________________
K3d-development mailing list
K3d-development@...
https://lists.sourceforge.net/lists/listinfo/k3d-development

[Timothy M. Shead]

Bart Janssens wrote:
> On Tue, Apr 28, 2009 at 2:56 AM, Timothy M. Shead <tshead@...> wrote:
>> Just a "heads up" that I moved the CatmullClark plugin to its own module
>> - its dependency on the SDS library was masking other plugins that need
>> to be updated in the mesh module.
>
> OK. The SDS library compiles again now, but I still have to test it
> before I commit. I also noticed that some painters (i.e. the
> face_painter) will have to be updated as well.

Yup ... as you're looking into these, have in-mind what you'd like to
get out of hint-refactoring to simplify / cleanup the caching.

Cheers,
Tim


[tshead.vcf]

---

begin:vcard
fn:Timothy Shead
n:Shead;Timothy
org:www.k-3d.org
email;internet:tshead@...
title:Founder
x-mozilla-html:FALSE
url:www.k-3d.org
version:2.1
end:vcard

---

------------------------------------------------------------------------------
Register Now & Save for Velocity, the Web Performance & Operations
Conference from O'Reilly Media. Velocity features a full day of
expert-led, hands-on workshops and two days of sessions from industry
leaders in dedicated Performance & Operations tracks. Use code vel09scf
and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf
_______________________________________________
K3d-development mailing list
K3d-development@...
https://lists.sourceforge.net/lists/listinfo/k3d-development

[bART Janssens-2]

On Wed, Apr 29, 2009 at 2:26 AM, Timothy M. Shead <tshead@...> wrote:
> Yup ... as you're looking into these, have in-mind what you'd like to
> get out of hint-refactoring to simplify / cleanup the caching.

OK, I guess you've seen I've gotten the painters and the SDS module to
compile again. However, it's not working yet, hopefully only because
the k3d::polyhedron::validate methods are not implemented yet. I'd be
willing to take stab at those, but it'll most likely be for next week,
so if it's urgent, don't let me hold you back ;)

As for the hint refactoring, I'd like to implement the current caches
using pointer_demand_storage. I wonder if it would be possible to have
mesh_instance manage the hint mappings to the cache, instead of
calling mesh_changed explicitly.

Cheers,

--
Bart

------------------------------------------------------------------------------
The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
production scanning environment may not be a perfect world - but thanks to
Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700
Series Scanner you'll get full speed at 300 dpi even with all image
processing features enabled. http://p.sf.net/sfu/kodak-com
_______________________________________________
K3d-development mailing list
K3d-development@...
https://lists.sourceforge.net/lists/listinfo/k3d-development

[Timothy M. Shead]

Bart Janssens wrote:
> On Wed, Apr 29, 2009 at 2:26 AM, Timothy M. Shead <tshead@...> wrote:
>> Yup ... as you're looking into these, have in-mind what you'd like to
>> get out of hint-refactoring to simplify / cleanup the caching.
>
> OK, I guess you've seen I've gotten the painters and the SDS module to
> compile again. However, it's not working yet, hopefully only because
> the k3d::polyhedron::validate methods are not implemented yet. I'd be
> willing to take stab at those, but it'll most likely be for next week,
> so if it's urgent, don't let me hold you back ;)

Will-do!

> As for the hint refactoring, I'd like to implement the current caches
> using pointer_demand_storage.

Sounds good.

> I wonder if it would be possible to have
> mesh_instance manage the hint mappings to the cache, instead of
> calling mesh_changed explicitly.

You're going to have to explain this statement further - and as a
general observation, I think we need to have a larger conversation about
caching in general.  We should start talking about this now, before
there's pressure to write an implementation ;)  To get things started,
I'm going to reorganize & update the hint-related wiki pages to capture
exactly where we're at today.

Cheers,
Tim


[tshead.vcf]

---

begin:vcard
fn:Timothy Shead
n:Shead;Timothy
org:www.k-3d.org
email;internet:tshead@...
title:Founder
x-mozilla-html:FALSE
url:www.k-3d.org
version:2.1
end:vcard

---

------------------------------------------------------------------------------
The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
production scanning environment may not be a perfect world - but thanks to
Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700
Series Scanner you'll get full speed at 300 dpi even with all image
processing features enabled. http://p.sf.net/sfu/kodak-com
_______________________________________________
K3d-development mailing list
K3d-development@...
https://lists.sourceforge.net/lists/listinfo/k3d-development

[Timothy M. Shead]

Timothy M. Shead wrote:

> I'm going to reorganize & update the hint-related wiki pages to capture
> exactly where we're at today.

Following-up, I've done a major rewrite of

http://www.k-3d.org/wiki/Pipeline_Hints

and

http://www.k-3d.org/wiki/Hint_Mapping

so that they document all of the new functionality around hints.
Further, I moved all painter-cache-related material that I could find to

http://www.k-3d.org/wiki/Painter_Cache

... let's start capturing ideas on that page.

Cheers,
Tim

[tshead.vcf]

---

begin:vcard
fn:Timothy Shead
n:Shead;Timothy
org:www.k-3d.org
email;internet:tshead@...
title:Founder
x-mozilla-html:FALSE
url:www.k-3d.org
version:2.1
end:vcard

---

------------------------------------------------------------------------------
The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
production scanning environment may not be a perfect world - but thanks to
Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700
Series Scanner you'll get full speed at 300 dpi even with all image
processing features enabled. http://p.sf.net/sfu/kodak-com
_______________________________________________
K3d-development mailing list
K3d-development@...
https://lists.sourceforge.net/lists/listinfo/k3d-development

[bART Janssens-2]

On Thu, May 7, 2009 at 7:44 AM, Timothy M. Shead <tshead@...> wrote:
> http://www.k-3d.org/wiki/Painter_Cache
>
> ... let's start capturing ideas on that page.

OK, I've replaced that page with a new proposal, refined from two more
related pages I dug up at
http://www.k-3d.org/wiki/Node_Visibility
and
http://k-3d.org/wiki/Per_Render_Engine_Painters

Cheers,

--
Bart

------------------------------------------------------------------------------
The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
production scanning environment may not be a perfect world - but thanks to
Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700
Series Scanner you'll get full speed at 300 dpi even with all image
processing features enabled. http://p.sf.net/sfu/kodak-com
_______________________________________________
K3d-development mailing list
K3d-development@...
https://lists.sourceforge.net/lists/listinfo/k3d-development

[Timothy M. Shead]

Bart Janssens wrote:

> On Thu, May 7, 2009 at 7:44 AM, Timothy M. Shead <tshead@...> wrote:
>> http://www.k-3d.org/wiki/Painter_Cache
>>
>> ... let's start capturing ideas on that page.
>
> OK, I've replaced that page with a new proposal, refined from two more
> related pages I dug up at
> http://www.k-3d.org/wiki/Node_Visibility
> and
> http://k-3d.org/wiki/Per_Render_Engine_Painters

This is a great start; I've updated all three pages to reflect what's
actually implemented now, and to consolidate the three topics (I moved
some more cache-related stuff to the painter cache page under "archive",
you should throw-out anything you think is obsolete).  I'll make a
couple of points:

* Although it's great to put Node Visibility, Per-Render-Engine
Painters, and the Painter Cache in context, they should all represent
orthogonal functionality.  If any one of these depends on the others,
we're doing something fundamentally wrong.

* I want to approach the cache with more open eyes - in particular, I
think the current cache is far too complex, and I want to look at
simplified approaches such as LRU or Multi Queue:

http://en.wikipedia.org/wiki/Cache_algorithms

... I believe that this would eliminate existing / avoid introducing new
complexity and provide useful new functionality.

Cheers,
Tim


[tshead.vcf]

---

begin:vcard
fn:Timothy Shead
n:Shead;Timothy
org:www.k-3d.org
email;internet:tshead@...
title:Founder
x-mozilla-html:FALSE
url:www.k-3d.org
version:2.1
end:vcard

---

------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables
unlimited royalty-free distribution of the report engine
for externally facing server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
K3d-development mailing list
K3d-development@...
https://lists.sourceforge.net/lists/listinfo/k3d-development

[bART Janssens-2]

On Fri, May 15, 2009 at 5:47 AM, Timothy M. Shead <tshead@...> wrote:
> * Although it's great to put Node Visibility, Per-Render-Engine
> Painters, and the Painter Cache in context, they should all represent
> orthogonal functionality.  If any one of these depends on the others,
> we're doing something fundamentally wrong.

Agreed! However, I think there are no dependencies in the current
proposal. In fact, using k3d::imesh_source as type used in the painter
methods eliminates a dependency that exists today, namely that
mesh_instance must keep its output mesh pointer constant for the sake
of painter caches. I merely put them all on the page to illustrate
their role in the visualization pipeline. I have removed the archive
section, since most important points are repeated in the new version,
and the rest was obsolete.

> * I want to approach the cache with more open eyes - in particular, I
> think the current cache is far too complex, and I want to look at
> simplified approaches such as LRU or Multi Queue:

OK, I think the on-demand creation of the cached data is OK as it is.
Timely removal , on the other hand, is flawed and causes complexity.
Currently, data is only removed when either the mesh is deleted or
when the last painter using the cached data is deleted. There is no
check to see if a connection between a mesh and a painter changed and
caused data to become useless (tracking this would be a nightmare).
All the removal systems would be greatly simplified if we simply
removed all data that wasn't used in the last scene redraw. This would
be some sort of LRU I guess. Not sure if we can currently be notified
of redraw begin and end, but I assume there is no objection to
creating signals for that?

Cheers,

--
Bart

------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables
unlimited royalty-free distribution of the report engine
for externally facing server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
K3d-development mailing list
K3d-development@...
https://lists.sourceforge.net/lists/listinfo/k3d-development

[Timothy M. Shead]

Bart Janssens wrote:
> On Fri, May 15, 2009 at 5:47 AM, Timothy M. Shead <tshead@...> wrote:
>> * Although it's great to put Node Visibility, Per-Render-Engine
>> Painters, and the Painter Cache in context, they should all represent
>> orthogonal functionality.  If any one of these depends on the others,
>> we're doing something fundamentally wrong.
>
> Agreed! However, I think there are no dependencies in the current
> proposal.  I merely put them all on the page to illustrate
> their role in the visualization pipeline.

Fair enough, just wanted to be sure we were on the same page.

> In fact, using k3d::imesh_source as type used in the painter
> methods eliminates a dependency that exists today, namely that
> mesh_instance must keep its output mesh pointer constant for the sake
> of painter caches.

Glad you brought this up, my next point was going to be that using
k3d::mesh as a key into the cache seems like a mistake, and using
k3d::imesh_source seems like an even larger mistake.  Caching should be
happening at a much finer granularity.  Take a VBO point painter as the
simplest case: it caches a VBO based on an array of points.  If the
cache key is a pointer to the array, it doesn't matter whether the
owning mesh pointer changes or not.  Because pipeline_data uses
shared-object semantics, it's even possible for the same array of points
to be shared across multiple mesh instances, and the correct VBO cache
will still be used.

>> * I want to approach the cache with more open eyes - in particular, I
>> think the current cache is far too complex, and I want to look at
>> simplified approaches such as LRU or Multi Queue:
>
> OK, I think the on-demand creation of the cached data is OK as it is.
> Timely removal , on the other hand, is flawed and causes complexity.
> Currently, data is only removed when either the mesh is deleted or
> when the last painter using the cached data is deleted. There is no
> check to see if a connection between a mesh and a painter changed and
> caused data to become useless (tracking this would be a nightmare).
> All the removal systems would be greatly simplified if we simply
> removed all data that wasn't used in the last scene redraw. This would
> be some sort of LRU I guess. Not sure if we can currently be notified
> of redraw begin and end, but I assume there is no objection to
> creating signals for that?

Arguably we shouldn't even need to do that.  If there is a threshold on
cache size, you need only remove old items from the cache when adding a
new item causes the cache to cross that threshold.  There are two
strategies here: how the threshold is defined, and how to choose what
gets removed.  There's plenty of room for interested research there, so
they should be generic / separate from the cache itself, which should
merely handle the details of key-value lookup and storage.

Cheers,
Tim


[tshead.vcf]

---

begin:vcard
fn:Timothy Shead
n:Shead;Timothy
org:www.k-3d.org
email;internet:tshead@...
title:Founder
x-mozilla-html:FALSE
url:www.k-3d.org
version:2.1
end:vcard

---

------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables
unlimited royalty-free distribution of the report engine
for externally facing server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
K3d-development mailing list
K3d-development@...
https://lists.sourceforge.net/lists/listinfo/k3d-development

[bART Janssens-2]

On Sun, May 17, 2009 at 5:03 AM, Timothy M. Shead <tshead@...> wrote:

> Glad you brought this up, my next point was going to be that using
> k3d::mesh as a key into the cache seems like a mistake, and using
> k3d::imesh_source seems like an even larger mistake.  Caching should be
> happening at a much finer granularity.  Take a VBO point painter as the
> simplest case: it caches a VBO based on an array of points.  If the
> cache key is a pointer to the array, it doesn't matter whether the
> owning mesh pointer changes or not.  Because pipeline_data uses
> shared-object semantics, it's even possible for the same array of points
> to be shared across multiple mesh instances, and the correct VBO cache
> will still be used.

OK, agreed. In the past we had problems with this, but I think that
was because I used the address of the points array for pretty much
anything, and this address changed sometimes even when the topology
didn't. Using the correct array address per cache, i.e. points for
point data, edge_points for edge VBOs, ... should avoid any problems,
and with a new algorithm for cleanup, we don't need to track
explicitly what gets used for what. The imesh_source proposal has been
removed from the wiki.

>> All the removal systems would be greatly simplified if we simply
>> removed all data that wasn't used in the last scene redraw. This would
>> be some sort of LRU I guess. Not sure if we can currently be notified
>> of redraw begin and end, but I assume there is no objection to
>> creating signals for that?
>
> Arguably we shouldn't even need to do that.  If there is a threshold on
> cache size, you need only remove old items from the cache when adding a
> new item causes the cache to cross that threshold.  There are two
> strategies here: how the threshold is defined, and how to choose what
> gets removed.  There's plenty of room for interested research there, so
> they should be generic / separate from the cache itself, which should
> merely handle the details of key-value lookup and storage.

I've updated the wiki page with two alternatives. I prefer the "check
every redraw" method, since it will be closer to the optimal case than
fixing a cache size. The disadvantage is that we would need a new
"on_redraw" signal.

Cheers,

--
Bart

------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables
unlimited royalty-free distribution of the report engine
for externally facing server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
K3d-development mailing list
K3d-development@...
https://lists.sourceforge.net/lists/listinfo/k3d-development

[Timothy M. Shead]

Bart Janssens wrote:

> On Sun, May 17, 2009 at 5:03 AM, Timothy M. Shead <tshead@...> wrote:
>> Glad you brought this up, my next point was going to be that using
>> k3d::mesh as a key into the cache seems like a mistake, and using
>> k3d::imesh_source seems like an even larger mistake.  Caching should be
>> happening at a much finer granularity.  Take a VBO point painter as the
>> simplest case: it caches a VBO based on an array of points.  If the
>> cache key is a pointer to the array, it doesn't matter whether the
>> owning mesh pointer changes or not.  Because pipeline_data uses
>> shared-object semantics, it's even possible for the same array of points
>> to be shared across multiple mesh instances, and the correct VBO cache
>> will still be used.
>
> OK, agreed. In the past we had problems with this, but I think that
> was because I used the address of the points array for pretty much
> anything, and this address changed sometimes even when the topology
> didn't. Using the correct array address per cache, i.e. points for
> point data, edge_points for edge VBOs, ... should avoid any problems,
> and with a new algorithm for cleanup, we don't need to track
> explicitly what gets used for what.

Yep, that's the spirit!

We're still glossing-over some details here ... such as cached data that
is derived from multiple sources.  A cached triangulation for example,
needs to change whenever one of several arrays in a mesh / polyhedron
change.  Ideally, the key should be based on each of those arrays, so
this is an area for experimentation.

>
>>> All the removal systems would be greatly simplified if we simply
>>> removed all data that wasn't used in the last scene redraw. This would
>>> be some sort of LRU I guess. Not sure if we can currently be notified
>>> of redraw begin and end, but I assume there is no objection to
>>> creating signals for that?
>> Arguably we shouldn't even need to do that.  If there is a threshold on
>> cache size, you need only remove old items from the cache when adding a
>> new item causes the cache to cross that threshold.  There are two
>> strategies here: how the threshold is defined, and how to choose what
>> gets removed.  There's plenty of room for interested research there, so
>> they should be generic / separate from the cache itself, which should
>> merely handle the details of key-value lookup and storage.
>
> I've updated the wiki page with two alternatives. I prefer the "check
> every redraw" method, since it will be closer to the optimal case than
> fixing a cache size. The disadvantage is that we would need a new
> "on_redraw" signal.

Sounds good, I would nuance this in a couple of ways:

* "Used since the last redraw" is a slippery concept, since cached data
in a world of MeshPainterLinks may only be used in some viewports.  If
those viewports aren't getting updated, that data won't appear to be in
use.  I think that ordering cached data based on a "hit count" is as
complex as we need to get.  One reason I found the Multi Queue stuff
interesting is that we can easily include cache metadata that reflects
the size and the time-to-generate cached data.  You might allow a large,
slow-to-generate chunk of data to live longer even if its hit-count is
low, for example.

* I'm still not convinced of the need to know about when redraws happen.
 For example, it would be possible for a "garbage collector" strategy
object to determine when the cache is cleaned-up.  Such a strategy
object might be written to only work when the UI is idle, saving CPU
cycles during interactive rendering.

Cheers,
Tim


[tshead.vcf]

---

begin:vcard
fn:Timothy Shead
n:Shead;Timothy
org:www.k-3d.org
email;internet:tshead@...
title:Founder
x-mozilla-html:FALSE
url:www.k-3d.org
version:2.1
end:vcard

---

------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables
unlimited royalty-free distribution of the report engine
for externally facing server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
K3d-development mailing list
K3d-development@...
https://lists.sourceforge.net/lists/listinfo/k3d-development

[bART Janssens-2]

On Mon, May 18, 2009 at 8:37 AM, Timothy M. Shead <tshead@...> wrote:
> We're still glossing-over some details here ... such as cached data that
> is derived from multiple sources.  A cached triangulation for example,
> needs to change whenever one of several arrays in a mesh / polyhedron
> change.  Ideally, the key should be based on each of those arrays, so
> this is an area for experimentation.

Well, as far as cache update is concerned, this is not really an
issue: the hint will be passed along, and if the hint indicates that
there is reason to recalculate, it will be done, regardless of the key
type. For triangle data we could use the face_first_loops array, for
example.

On the other hand, getting the correct data is more critical. If the
face_first_loops array happens to be shared across meshes, problems
will occur. Maybe in the case of multi-array painters using the
pointer to the primitive itself would be the best choice. That means
the key type would have to remain templated, although I have a feeling
there should be a simpler way.

>>>> All the removal systems would be greatly simplified if we simply
>>>> removed all data that wasn't used in the last scene redraw. This would
>>>> be some sort of LRU I guess. Not sure if we can currently be notified
>>>> of redraw begin and end, but I assume there is no objection to
>>>> creating signals for that?
>>> Arguably we shouldn't even need to do that.  If there is a threshold on
>>> cache size, you need only remove old items from the cache when adding a
>>> new item causes the cache to cross that threshold.  There are two
>>> strategies here: how the threshold is defined, and how to choose what
>>> gets removed.  There's plenty of room for interested research there, so
>>> they should be generic / separate from the cache itself, which should
>>> merely handle the details of key-value lookup and storage.
>>
>> I've updated the wiki page with two alternatives. I prefer the "check
>> every redraw" method, since it will be closer to the optimal case than
>> fixing a cache size. The disadvantage is that we would need a new
>> "on_redraw" signal.
>
> Sounds good, I would nuance this in a couple of ways:

OK, I think the best way forward is to start with the stack-based LRU
algorithm. This seems the simplest to implement, and we can always
optimize when we feel the need.

Cheers,

--
Bart

------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables
unlimited royalty-free distribution of the report engine
for externally facing server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
K3d-development mailing list
K3d-development@...
https://lists.sourceforge.net/lists/listinfo/k3d-development

[Timothy M. Shead]

Bart Janssens wrote:

> On Mon, May 18, 2009 at 8:37 AM, Timothy M. Shead <tshead@...> wrote:
>> We're still glossing-over some details here ... such as cached data that
>> is derived from multiple sources.  A cached triangulation for example,
>> needs to change whenever one of several arrays in a mesh / polyhedron
>> change.  Ideally, the key should be based on each of those arrays, so
>> this is an area for experimentation.
>
> Well, as far as cache update is concerned, this is not really an
> issue: the hint will be passed along, and if the hint indicates that
> there is reason to recalculate, it will be done, regardless of the key
> type. For triangle data we could use the face_first_loops array, for
> example.
>
> On the other hand, getting the correct data is more critical. If the
> face_first_loops array happens to be shared across meshes, problems
> will occur. Maybe in the case of multi-array painters using the
> pointer to the primitive itself would be the best choice. That means
> the key type would have to remain templated, although I have a feeling
> there should be a simpler way.

Fair enough ... to figure-out what the key / API should be, it would be
useful to document all of the data types that we currently cache ...
could you add that to the wiki?  Point VBOs and cached triangulations
come to mind, what else is there?  While the list of cached data isn't
set in stone, I wouldn't expect it to grow all that often.  It will also
be useful to enumerate what hints need to be created to support
efficient updates.

Cheers,
Tim

[tshead.vcf]

---

begin:vcard
fn:Timothy Shead
n:Shead;Timothy
org:www.k-3d.org
email;internet:tshead@...
title:Founder
x-mozilla-html:FALSE
url:www.k-3d.org
version:2.1
end:vcard

---

------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables
unlimited royalty-free distribution of the report engine
for externally facing server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
K3d-development mailing list
K3d-development@...
https://lists.sourceforge.net/lists/listinfo/k3d-development