Discussion about #2474255 - DoF artifacts at image edges

[Ricardo Mayor]

Hi,

This bug is similar to "Bad render for image's first row and column". The clipping volume test is culling geometry needed during the sampling process. This geometry is culled because the view frustum clipping volume doesn't not take into account geometry that produces a circle of confusion that intersects with the screen window. I've been thinking in this problem and these are my three possible approaches.

a.-Using a max CoC

In this approach, the size of the view frustum clipping volume is enlarged by the max CoC size. The Max CoC is calculated as the max CoC in far and near clipping plane. This calculation is done while the camera is initialized and used as and extra frame to the screen window.

b.- Using an enlarged bounding box

Each bounding box is enlarged proportionally to the CoC that the original bounding box produces. This enlarged bounding box projected in the screen window must contain the CoC of the original bounding box. This new BB is used in the cull surface test. This extra bound can be calculated only for the cull surface test.

c.- Brand new cull surface test:

The view frustum clipping volume is actually defined with planes. This planes are not capable of define a volume in which all geometry inside of this plane affects the screen window. The last approach is to use a curved surface for each lateral of the bounding box. This curved surface is calculated as the original plane function plus and additional function. This additional function includes the space outside the original bounding plane which CoC affects the screen window.

In and outs

A.- It's easy to implement but very inefficient. if the near clipping plane is more or less 0, the max CoC is very high. In the attached rib, this is CoC of near a far clipping planes:

Znear = 1.1920929E-7 -> CoC = 131071991

Zfar = 3.4028235E38 -> CoC = 7,8125

B.- It needs a little bit of work, but It's accurate. The culling test is less efficient because it needs extra calculation. It needs at least one transformation from screen space to camera space in each test.

C.- This approach needs more coding than the others. Before that, I need to take a deeper look in all the maths involved. In the other side is accurate and, IMO It can produce the faster solution.

I think B is suitable for a quick develoment but C is the best solution in the long term.

Now It's late, but I'll post a graphical example for the approach C tomorrow. I attach an example rib and a graph view of CoC for further discussion.

------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
Aqsis-development mailing list
Aqsis-development@...
https://lists.sourceforge.net/lists/listinfo/aqsis-development

[Chris Foster-5]

Hi Ricardo,

I think you're right about the causes for this problem, nicely done once again.

On Tue, Sep 15, 2009 at 7:52 AM, Ricardo Mayor <ricardo.mayor@...> wrote:
> In and outs
> A.- It's easy to implement but very inefficient. if the near clipping plane
> is more or less 0, the max CoC is very high. In the attached rib, this is
> CoC of near a far clipping planes:
> Znear = 1.1920929E-7 -> CoC = 131071991
> Zfar = 3.4028235E38 -> CoC = 7,8125

I think solution A will not give acceptable results; as you say, it's
highly inefficient.

> B.- It needs a little bit of work, but It's accurate. The culling test is
> less efficient because it needs extra calculation. It needs at least one
> transformation from screen space to camera space in each test.

This seems like a viable long term solution to me.  I would expect that
the extra transformation has a fairly negligable impact on performance.
Consider that this happens at a per-primitive level, *before* anything
has been turned into micropolygons.  The cost of shading and sampling
micropolygons should dwarf the per-primitive culling costs in nearly
every scene.

> C.- This approach needs more coding than the others. Before that, I need to
> take a deeper look in all the maths involved. In the other side is accurate
> and, IMO It can produce the faster solution.

I'm not convinced by solution C for two reasons:
  - Culling against a general volume rather than a plane is not going to
    have an efficient implementation in general.
  - Even if it does, the implementation is likely to be *much* more
    complicated, and simplicity is preferred unless it conflicts
    severely with performance.
however, feel free to tell me why I am wrong ;-)


Instead, I'd like to suggest a fourth solution:

D.- Remove the direct frustrum-based clipping volume test entirely, and
    perform the culling entirely in screen/raster space.

Now, I might be missing something important, but it looks to me like the
the combination of the two tests

    // If the primitive is completely outside of the hither-yon z
range, cull it.
    if ( Bound.vecMin().z() >= m_optCache.clipFar ||
         Bound.vecMax().z() <= m_optCache.clipNear )
        return true;

        [...]

    // If the bounds are completely outside the viewing frustum, cull
the primitive.
    if( Bound.vecMin().x() > QGetRenderContext()->cropWindowXMax() ||
        Bound.vecMin().y() > QGetRenderContext()->cropWindowYMax() ||
        Bound.vecMax().x() < QGetRenderContext()->cropWindowXMin() ||
        Bound.vecMax().y() < QGetRenderContext()->cropWindowYMin() )
        return ( true );

inside CqImageBuffer::CullSurface() should do the trick for us even with the
frustrum culling turned off.


Maybe I'm missing something vitally important about how the frustrum-based
culling works, or is it just an optimization to avoid the potentially more
expensive screen/raster space culling?  If it's actually a useful optimization
then one possibility would be to turn it off only when depth of field
rendering is enabled.

Cheers,
~Chris

------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
Aqsis-development mailing list
Aqsis-development@...
https://lists.sourceforge.net/lists/listinfo/aqsis-development

[Ricardo Mayor]

Hi Chris,

Just a quick update. I've been busy today but I have some spare time to deal think on this.

I think solution A will not give acceptable results; as you say, it's
highly inefficient.

OK, open the trash bin.............. 

> B.- It needs a little bit of work, but It's accurate. The culling test is
> less efficient because it needs extra calculation. It needs at least one
> transformation from screen space to camera space in each test.

This seems like a viable long term solution to me.  I would expect that
the extra transformation has a fairly negligable impact on performance.
Consider that this happens at a per-primitive level, *before* anything
has been turned into micropolygons.  The cost of shading and sampling
micropolygons should dwarf the per-primitive culling costs in nearly
every scene.

 

I don't like this approach because you have to deal with transformation from camera between spaces. IMO, one of the best things that view frustum clipping volume has is that you cull the surface in the camera space. You dont' need to deal with transformation between spaces during the process. Another important thing is that the culling test is easy and quick ( take a look at plane.h, cqplane::whicside()).

 

> C.- This approach needs more coding than the others. Before that, I need to
> take a deeper look in all the maths involved. In the other side is accurate
> and, IMO It can produce the faster solution. 

I'm not convinced by solution C for two reasons:
 - Culling against a general volume rather than a plane is not going to
   have an efficient implementation in general.
 - Even if it does, the implementation is likely to be *much* more
   complicated, and simplicity is preferred unless it conflicts
   severely with performance.
however, feel free to tell me why I am wrong ;-)

I've been dealing with maths for this case and I think that this approach is not too complicated indeed. The final function is easy to implement. The culling test is not going to be difficult to understand and it's similar to the actual approach.  See the attached example, is a quick look to the final function.

Instead, I'd like to suggest a fourth solution:

D.- Remove the direct frustrum-based clipping volume test entirely, and
   perform the culling entirely in screen/raster space.

Now, I might be missing something important, but it looks to me like the
the combination of the two tests

   // If the primitive is completely outside of the hither-yon z
range, cull it.
   if ( Bound.vecMin().z() >= m_optCache.clipFar ||
        Bound.vecMax().z() <= m_optCache.clipNear )
       return true;

       [...]

   // If the bounds are completely outside the viewing frustum, cull
the primitive.
   if( Bound.vecMin().x() > QGetRenderContext()->cropWindowXMax() ||
       Bound.vecMin().y() > QGetRenderContext()->cropWindowYMax() ||
       Bound.vecMax().x() < QGetRenderContext()->cropWindowXMin() ||
       Bound.vecMax().y() < QGetRenderContext()->cropWindowYMin() )
       return ( true );

inside CqImageBuffer::CullSurface() should do the trick for us even with the
frustrum culling turned off.


Maybe I'm missing something vitally important about how the frustrum-based
culling works, or is it just an optimization to avoid the potentially more
expensive screen/raster space culling?  If it's actually a useful optimization
then one possibility would be to turn it off only when depth of field
rendering is enabled.

The problem with this option is that you need to transform from camera space to screen space. This is an extra work.

The attached example is the approach C. In this example I take the  geometry and depth of field definition of dof_edge_bug but the projection is perspective. This graph is a 2d graph where:

x represents  depth

y represents one of the projected dimensions.

black line is the  actual frustum plane => f(x) = 0.5 x  

Red is the Circle of confusion  CoC(x) = 50 * 0.3125 * abs(1/x - 0.5)

blue is the new frustum curve (approach C) => f(x) = x * ( 50 * 0.3125 * abs(1/x - 0.5)  + 0.5 )

All the geometry under the new frustum curve may have influence in the final render. All the space between the black and the blue line is geometry that is actually culled.

Please take into account that  this is a quick test !

As you can see, the new frustum covers a great surface.This is due to the high depth of field. 

------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
Aqsis-development mailing list
Aqsis-development@...
https://lists.sourceforge.net/lists/listinfo/aqsis-development

[Paul Gregory-4]

2009/9/15 Chris Foster <chris42f@...>

I think solution A will not give acceptable results; as you say, it's
highly inefficient.

I agree  here, especially as the default clipping planes are RI_EPSILON and RI_INFINITY, this just isn't workable.
 

This seems like a viable long term solution to me.  I would expect that
the extra transformation has a fairly negligable impact on performance.
Consider that this happens at a per-primitive level, *before* anything
has been turned into micropolygons.  The cost of shading and sampling
micropolygons should dwarf the per-primitive culling costs in nearly
every scene.

Again, can't disagree, of the 3 choices, this one seems the most sensible to me. No point overcomplicating things for such a small part of the pipeline. The only caveat I'd say is, feel free if you so desire to investigate C and if it shows that more efficient culling does have an impact further down the pipeline, we can review the situation.
 

Instead, I'd like to suggest a fourth solution:

D.- Remove the direct frustrum-based clipping volume test entirely, and
   perform the culling entirely in screen/raster space.

Now, I might be missing something important, but it looks to me like the
the combination of the two tests

   // If the primitive is completely outside of the hither-yon z
range, cull it.
   if ( Bound.vecMin().z() >= m_optCache.clipFar ||
        Bound.vecMax().z() <= m_optCache.clipNear )
       return true;

       [...]

   // If the bounds are completely outside the viewing frustum, cull
the primitive.
   if( Bound.vecMin().x() > QGetRenderContext()->cropWindowXMax() ||
       Bound.vecMin().y() > QGetRenderContext()->cropWindowYMax() ||
       Bound.vecMax().x() < QGetRenderContext()->cropWindowXMin() ||
       Bound.vecMax().y() < QGetRenderContext()->cropWindowYMin() )
       return ( true );

inside CqImageBuffer::CullSurface() should do the trick for us even with the
frustrum culling turned off.


Maybe I'm missing something vitally important about how the frustrum-based
culling works, or is it just an optimization to avoid the potentially more
expensive screen/raster space culling?  If it's actually a useful optimization
then one possibility would be to turn it off only when depth of field
rendering is enabled.

The clipping volume test is just an 'early out' that isn't normally too expensive, and has the advantage that it's a generic plane based clipping volume, and therefore can easily be extended to support the RiClippingPlane functionality when I get round to wiring it up. The second test against the crop window listed above is slightly different in that it considers the crop window, which the view frustum clipping doesn't.

I'd prefer to keep the volume clipping if only to support RiClippingPlane at some point, but whether it needs to include the frustum or not is debatable.  Might be worth doing some comparative tests, with and without frustum clipping, to see the impact on performance, I suspect under normal (no crop window) conditions, it'll be negligible.

Paul

------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
Aqsis-development mailing list
Aqsis-development@...
https://lists.sourceforge.net/lists/listinfo/aqsis-development

[Chris Foster-5]

Hi guys,

On Wed, Sep 16, 2009 at 4:51 PM, Paul Gregory <pgregory@...> wrote:

>> This seems like a viable long term solution to me.  I would expect that
>> the extra transformation has a fairly negligable impact on performance.
>> Consider that this happens at a per-primitive level, *before* anything
>> has been turned into micropolygons.  The cost of shading and sampling
>> micropolygons should dwarf the per-primitive culling costs in nearly
>> every scene.
>>
>
> Again, can't disagree, of the 3 choices, this one seems the most sensible to
> me. No point overcomplicating things for such a small part of the pipeline.
> The only caveat I'd say is, feel free if you so desire to investigate C and
> if it shows that more efficient culling does have an impact further down the
> pipeline, we can review the situation.

Agreed.  I'm not against option C if it turns out to be simple enough.  From
the graphs that Ricardo has just sent, it appears that option C implies that
the *outside* of each CoC-modified clipping "plane" is a convex region.  This
is vitally important: it makes simple culling viable because it allows us to
test only the vertices of the bounding box (which itself is convex) in order
to determine whether the object may be culled.  (If the outside volume of the
clipping surface is not convex, this isn't the case.)

[...]

>> Maybe I'm missing something vitally important about how the frustrum-based
>> culling works, or is it just an optimization to avoid the potentially more
>> expensive screen/raster space culling?  If it's actually a useful
>> optimization
>> then one possibility would be to turn it off only when depth of field
>> rendering is enabled.
>>
>
> The clipping volume test is just an 'early out' that isn't normally too
> expensive, and has the advantage that it's a generic plane based clipping
> volume, and therefore can easily be extended to support the RiClippingPlane
> functionality when I get round to wiring it up. The second test against the
> crop window listed above is slightly different in that it considers the crop
> window, which the view frustum clipping doesn't.
>
> I'd prefer to keep the volume clipping if only to support RiClippingPlane at
> some point, but whether it needs to include the frustum or not is
> debatable.  Might be worth doing some comparative tests, with and without
> frustum clipping, to see the impact on performance, I suspect under normal
> (no crop window) conditions, it'll be negligible.

Thanks Paul, this really clears things up.  Regarding RiClippingPlane, I
wonder if it might have some extra requirements in addition to those needed
for a frustrum clipping volume?  The RISpec says that

" All geometry on the positive side of the plane (that is, in the direction that
" the normal points) will be clipped from the scene.

I would take this to mean that micropolygons must be clipped against the
plane, along with individual samples as well.  This would be necessary if
RiClippingPlane were used to support simulating mirrored surfaces with a
reflection pass.  (All the geometry on the other side of the mirror plane must
be clipped for the reflection map to render correctly.)

~Chris.

------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
Aqsis-development mailing list
Aqsis-development@...
https://lists.sourceforge.net/lists/listinfo/aqsis-development

[Paul Gregory-4]

2009/9/16 Chris Foster <chris42f@...>

Thanks Paul, this really clears things up.  Regarding RiClippingPlane, I
wonder if it might have some extra requirements in addition to those needed
for a frustrum clipping volume?  The RISpec says that

" All geometry on the positive side of the plane (that is, in the direction that
" the normal points) will be clipped from the scene.

I would take this to mean that micropolygons must be clipped against the
plane, along with individual samples as well.  This would be necessary if
RiClippingPlane were used to support simulating mirrored surfaces with a
reflection pass.  (All the geometry on the other side of the mirror plane must
be clipped for the reflection map to render correctly.)

Yes, the clipping would need to be extended to MPs too, that's why I didn't just wire up the RiClippingPlane code when the clipping volume was introduced, it's not a massive job, but not a trivial one either.

Paul

------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
Aqsis-development mailing list
Aqsis-development@...
https://lists.sourceforge.net/lists/listinfo/aqsis-development

[Ricardo Mayor]

Hi,
I'll start working on option B, which seems to be the most valued option. Anyway, I understand that this part of the pipeline will have a future revision to include RiClippingPlane or micropolygons, so I'll try to encapsulate all the functionality. This will make easier future changes or other options like C.

Best regards

>> This seems like a viable long term solution to me.  I would expect that

>> the extra transformation has a fairly negligable impact on performance.
>> Consider that this happens at a per-primitive level, *before* anything
>> has been turned into micropolygons.  The cost of shading and sampling
>> micropolygons should dwarf the per-primitive culling costs in nearly
>> every scene.
>>
>
> Again, can't disagree, of the 3 choices, this one seems the most sensible to
> me. No point overcomplicating things for such a small part of the pipeline.
> The only caveat I'd say is, feel free if you so desire to investigate C and
> if it shows that more efficient culling does have an impact further down the
> pipeline, we can review the situation.

Agreed.  I'm not against option C if it turns out to be simple enough.  From
the graphs that Ricardo has just sent, it appears that option C implies that
the *outside* of each CoC-modified clipping "plane" is a convex region.  This
is vitally important: it makes simple culling viable because it allows us to
test only the vertices of the bounding box (which itself is convex) in order
to determine whether the object may be culled.  (If the outside volume of the
clipping surface is not convex, this isn't the case.)

------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
Aqsis-development mailing list
Aqsis-development@...
https://lists.sourceforge.net/lists/listinfo/aqsis-development

[Chris Foster-5]

On Wed, Sep 16, 2009 at 5:54 PM, Ricardo Mayor <ricardo.mayor@...> wrote:
> I'll start working on option B, which seems to be the most valued option.

Maybe option B and C are essentially equivalent?  On the one hand, we expand
the bounding box to take into account the CoC.  On the other hand, we expand
the clipping surface to take into account the CoC.  Either option requires
computing the CoC radius at the object depth doesn't it?  They seem basically
the same to me.

> Anyway, I understand that this part of the pipeline will have a future
> revision to include RiClippingPlane or micropolygons,

The micropolygon clipping for RiClippingPlane would happen elsewhere in the
code, and would *not* depend on the DoF settings:  RiClippingPlane is a purely
*geometric* operation, whereas the viewing frustrum clipping is an
optimization based on culling non-visible objects in the final image.

They're really somewhat different things, but share some common elements.

> so I'll try to encapsulate all the functionality. This will make easier
> future changes or other options like C.

Sounds good.

~Chris.

------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
Aqsis-development mailing list
Aqsis-development@...
https://lists.sourceforge.net/lists/listinfo/aqsis-development

[Ricardo Mayor]

Yes, both options are equivalent. The main difference is that option B needs a Coc calculation and an explicit transformation between screen and camera spaces to test bounding box with planes. In option C this transformation and the COC calculation are included in the function that defines the clipping region. That is the difference

On Wed, Sep 16, 2009 at 9:07 AM, Chris Foster <chris42f@...> wrote:

On Wed, Sep 16, 2009 at 5:54 PM, Ricardo Mayor <ricardo.mayor@...> wrote:
> I'll start working on option B, which seems to be the most valued option.

Maybe option B and C are essentially equivalent?  On the one hand, we expand
the bounding box to take into account the CoC.  On the other hand, we expand
the clipping surface to take into account the CoC.  Either option requires
computing the CoC radius at the object depth doesn't it?  They seem basically
the same to me.

------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
Aqsis-development mailing list
Aqsis-development@...
https://lists.sourceforge.net/lists/listinfo/aqsis-development

[Chris Foster-5]

On Wed, Sep 16, 2009 at 6:15 PM, Ricardo Mayor <ricardo.mayor@...> wrote:
> Yes, both options are equivalent. The main difference is that option B needs
> a Coc calculation and an explicit transformation between screen and camera
> spaces to test bounding box with planes. In option C this transformation and
> the COC calculation are included in the function that defines the clipping
> region. That is the difference

I would have thought that for option B, the only additional thing you
need to do is to compute the max CoC for the bound, transform that CoC
from raster into the camera space (essentially dividing by the image
dimensions), and add it onto the existing bound which is already in
camera space.  Then test the result against the frustrum clipping
planes.

The max CoC calculation is really cheap at this level (it's only a
floating point divide + a few multiplications and additions).
Likewise, expanding the camera space bound by that CoC should be
really computationally cheap too... to me they look about the same
computational cost.

~Chris.

------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
Aqsis-development mailing list
Aqsis-development@...
https://lists.sourceforge.net/lists/listinfo/aqsis-development