Clouds in hi resolution pano (Celeste 2.0?)

[Yuval Levy-6]

hi all,

I'm currently dealing with a lot of clouds (when traveling quickly
through many locations I have no other choice than take the meteo
conditions as they are). Since the introduction of Celeste (that works
great, thanks Tim!) this is no longer an issue for the proper
alignment and stitching of everything but the clouds. However in multi-
rows high resolution shots the clouds end up being an issue at the
blending stage. The rows become visible.

The lazy way to deal with this is to mask out the sky and replace it
with whatever photo editor is at hand, but the result inevitably looks
artificial (shade zones on the mountains and so).

When Tim was coding Celeste, we sparred about what kind of mask should
Celeste generate around the clouds, if any.

Now, having such a mask could be useful for the following idea,
assuming the clouds movement is constant throughout the photoshooting:
1. generate control points for the static parts in the image. Use them
to position the images in relationship to one another and create the
master panoramas, using Celeste to prune CPs from the clouds and mask
the coulded areas.
2. generate control points in the clouds (area masked by Celeste) and
calculate the translation related to the positioning in 1 (which is
the translation vector multiplied by the time differential between the
reference image and the current image)
3. use the translated cloud images to generate an additional panorama
of the sky
4. mask the sky out from the static panorama and add the sky panorama
as a layer

there will be some areas of the sky that will be "empty" (e.g. when a
cloud moves behind an object or out from it), but those will be much
smaller areas to deal with in an image editor than generating the sky
artificially or dealing with the shift across all images.

to do this we need:
- an additional category of CPs to compose the sky panorama (sky-CPs)
- a measurement of the displacement of those CPs related to the
position of the image in the static pano
- some math to average / optimize the displacement measures
- some glue/script to generate the second panorama
- a tool to mask the sky from the rest of the panorama (I don't think
that Celeste's mask are fine enough for that).
- some glue/script to add the mask and the sky to the resulting
panorama.

does this sound logic? or have I missed something? should I record
this as a feature request? maybe a future GSoC project?

Yuv
-
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group.
A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ
To post to this group, send email to hugin-ptx@...
To unsubscribe from this group, send email to hugin-ptx-unsubscribe@...
For more options, visit this group at http://groups.google.com/group/hugin-ptx
-~----------~----~----~----~------~----~------~--~---

[Luís Henrique Camargo Quiroz]

Yuv escreveu:
> hi all,
>   
    Hi! See point 2 below, please.
> I'm currently dealing with a lot of clouds (when traveling quickly
> through many locations I have no other choice than take the meteo
> conditions as they are). Since the introduction of Celeste (that works
> great, thanks Tim!) this is no longer an issue for the proper
> alignment and stitching of everything but the clouds. However in multi-
> rows high resolution shots the clouds end up being an issue at the
> blending stage. The rows become visible.
>
> The lazy way to deal with this is to mask out the sky and replace it
> with whatever photo editor is at hand, but the result inevitably looks
> artificial (shade zones on the mountains and so).
>
> When Tim was coding Celeste, we sparred about what kind of mask should
> Celeste generate around the clouds, if any.
>
> Now, having such a mask could be useful for the following idea,
> assuming the clouds movement is constant throughout the photoshooting:
> 1. generate control points for the static parts in the image. Use them
> to position the images in relationship to one another and create the
> master panoramas, using Celeste to prune CPs from the clouds and mask
> the coulded areas.
> 2. generate control points in the clouds (area masked by Celeste) and
> calculate the translation related to the positioning in 1 (which is
> the translation vector multiplied by the time differential between the
> reference image and the current image)
>   

    Ok, but, if I don´t misunderstood: in the perhaps usual case of wind
in mainly the same direction and speed, if someone take shots for a 360
deg panorama, the algorithm has to deal with many cases: clouds
approaching the camera, then (say, rotating camera clockwise) coming
from left, then going farther, then coming from right, and at 360=0
again towards the photographer. I think that any solution to this
problem - cloud movement at 360 degrees apnos -  would "distort" the
sky  somewhere in the result.
    But, in other cases, when clouds are always going in the same
direction/speed, this solution is great!
> 3. use the translated cloud images to generate an additional panorama
> of the sky
> 4. mask the sky out from the static panorama and add the sky panorama
> as a layer
>
> there will be some areas of the sky that will be "empty" (e.g. when a
> cloud moves behind an object or out from it), but those will be much
> smaller areas to deal with in an image editor than generating the sky
> artificially or dealing with the shift across all images.
>
> to do this we need:
> - an additional category of CPs to compose the sky panorama (sky-CPs)
> - a measurement of the displacement of those CPs related to the
> position of the image in the static pano
> - some math to average / optimize the displacement measures
> - some glue/script to generate the second panorama
> - a tool to mask the sky from the rest of the panorama (I don't think
> that Celeste's mask are fine enough for that).
> - some glue/script to add the mask and the sky to the resulting
> panorama.
>
> does this sound logic? or have I missed something? should I record
> this as a feature request? maybe a future GSoC project?
>
> Yuv
> -
>   

    Thanks for your attention,

    Luís Henrique

--
Luis Henrique Camargo Quiroz, M.Sc. - Internal Combustion Engines Group
IPT - Sao Paulo State Technological Researches Institute
phone: [phone]  fax: [phone]  www.ipt.br  BRAZIL
http://luishcq.tripod.com - http://www.christusrex.org/www2/cantgreg



--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group.
A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ
To post to this group, send email to hugin-ptx@...
To unsubscribe from this group, send email to hugin-ptx-unsubscribe@...
For more options, visit this group at http://groups.google.com/group/hugin-ptx
-~----------~----~----~----~------~----~------~--~---

[Jim Watters]

Wow!  sounds complicated.  I must admit my normal panorama shooting
process does not require using Celeste.

Wouldn't the simple solution be for Celeste to remove points if there is
a mixture of cloud and non-cloud points joining two images.  But if the
only points[1] joining two images are cloud points then they should not
be removed.

[1]  Requires at least 2 non-cloud point to be considered as having
non-cloud points.

Jim Watters

Yuv wrote:

> hi all,
>
> I'm currently dealing with a lot of clouds (when traveling quickly
> through many locations I have no other choice than take the meteo
> conditions as they are). Since the introduction of Celeste (that works
> great, thanks Tim!) this is no longer an issue for the proper
> alignment and stitching of everything but the clouds. However in multi-
> rows high resolution shots the clouds end up being an issue at the
> blending stage. The rows become visible.
>
> The lazy way to deal with this is to mask out the sky and replace it
> with whatever photo editor is at hand, but the result inevitably looks
> artificial (shade zones on the mountains and so).
>
> When Tim was coding Celeste, we sparred about what kind of mask should
> Celeste generate around the clouds, if any.
>
> Now, having such a mask could be useful for the following idea,
> assuming the clouds movement is constant throughout the photoshooting:
> 1. generate control points for the static parts in the image. Use them
> to position the images in relationship to one another and create the
> master panoramas, using Celeste to prune CPs from the clouds and mask
> the coulded areas.
> 2. generate control points in the clouds (area masked by Celeste) and
> calculate the translation related to the positioning in 1 (which is
> the translation vector multiplied by the time differential between the
> reference image and the current image)
> 3. use the translated cloud images to generate an additional panorama
> of the sky
> 4. mask the sky out from the static panorama and add the sky panorama
> as a layer
>
> there will be some areas of the sky that will be "empty" (e.g. when a
> cloud moves behind an object or out from it), but those will be much
> smaller areas to deal with in an image editor than generating the sky
> artificially or dealing with the shift across all images.
>
> to do this we need:
> - an additional category of CPs to compose the sky panorama (sky-CPs)
> - a measurement of the displacement of those CPs related to the
> position of the image in the static pano
> - some math to average / optimize the displacement measures
> - some glue/script to generate the second panorama
> - a tool to mask the sky from the rest of the panorama (I don't think
> that Celeste's mask are fine enough for that).
> - some glue/script to add the mask and the sky to the resulting
> panorama.
>
> does this sound logic? or have I missed something? should I record
> this as a feature request? maybe a future GSoC project?
>
> Yuv
>  

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group.
A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ
To post to this group, send email to hugin-ptx@...
To unsubscribe from this group, send email to hugin-ptx-unsubscribe@...
For more options, visit this group at http://groups.google.com/group/hugin-ptx
-~----------~----~----~----~------~----~------~--~---

[luca vascon-2]

Uhu!!!
I want it for lagoon waves!!!
:-)

> hi all,
>
> I'm currently dealing with a lot of clouds (when traveling quickly
> through many locations I have no other choice than take the meteo
> conditions as they are). Since the introduction of Celeste (that works
> great, thanks Tim!) this is no longer an issue for the proper
> alignment and stitching of everything but the clouds. However in multi-
> rows high resolution shots the clouds end up being an issue at the
> blending stage. The rows become visible.
>
> The lazy way to deal with this is to mask out the sky and replace it
> with whatever photo editor is at hand, but the result inevitably looks
> artificial (shade zones on the mountains and so).
>
> When Tim was coding Celeste, we sparred about what kind of mask should
> Celeste generate around the clouds, if any.
>
> Now, having such a mask could be useful for the following idea,
> assuming the clouds movement is constant throughout the photoshooting:
> 1. generate control points for the static parts in the image. Use them
> to position the images in relationship to one another and create the
> master panoramas, using Celeste to prune CPs from the clouds and mask
> the coulded areas.
> 2. generate control points in the clouds (area masked by Celeste) and
> calculate the translation related to the positioning in 1 (which is
> the translation vector multiplied by the time differential between the
> reference image and the current image)
> 3. use the translated cloud images to generate an additional panorama
> of the sky
> 4. mask the sky out from the static panorama and add the sky panorama
> as a layer
>
> there will be some areas of the sky that will be "empty" (e.g. when a
> cloud moves behind an object or out from it), but those will be much
> smaller areas to deal with in an image editor than generating the sky
> artificially or dealing with the shift across all images.
>
> to do this we need:
> - an additional category of CPs to compose the sky panorama (sky-CPs)
> - a measurement of the displacement of those CPs related to the
> position of the image in the static pano
> - some math to average / optimize the displacement measures
> - some glue/script to generate the second panorama
> - a tool to mask the sky from the rest of the panorama (I don't think
> that Celeste's mask are fine enough for that).
> - some glue/script to add the mask and the sky to the resulting
> panorama.
>
> does this sound logic? or have I missed something? should I record
> this as a feature request? maybe a future GSoC project?
>
> Yuv
> -
> >
>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group.
A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ
To post to this group, send email to hugin-ptx@...
To unsubscribe from this group, send email to hugin-ptx-unsubscribe@...
For more options, visit this group at http://groups.google.com/group/hugin-ptx
-~----------~----~----~----~------~----~------~--~---

[Yuval Levy-6]

Jim Watters wrote:
> Wow!  sounds complicated.  I must admit my normal panorama shooting
> process does not require using Celeste.
>
> Wouldn't the simple solution be for Celeste to remove points if there is
> a mixture of cloud and non-cloud points joining two images.


I think I did not explain myself properly. The problem is not bad CP
on moving clouds. It is the moving sky in relationship with the static
subject. It could be also a sky of star.

Look at <http://groups.google.com/group/hugin-ptx/web/Screenshot-Fast
%20Panorama%20preview.png>

The static scene stitches perfectly, but the background sky has moved
between the row containing the tip of the clock-tower and the row
below it.

The easy way out is to fiddle with the clone tool or some other image
editing tool around the seam between the two rows.

The smart way out IMO would be to mask out the static features (the
top of the clock-tower) from the top row and then shift to correct for
the cloud movement over time. Probably shifting the whole row is good
enough, though shifting each composing image proportionally to the
timestamp difference may be more accurate. This will leave much less
fiddling in the image editor: mostly where the sky has moved behind a
static feature, and a little also where the clouds have changed shape
over time (easy, because no accuracy is required).

Yuv
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group.
A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ
To post to this group, send email to hugin-ptx@...
To unsubscribe from this group, send email to hugin-ptx-unsubscribe@...
For more options, visit this group at http://groups.google.com/group/hugin-ptx
-~----------~----~----~----~------~----~------~--~---

[rlhelinski]

This is a good idea, but IMHO it would require the user creating a
mask for each image so that the tools can treat the different regions
separately, perhaps by using alpha channels. Perhaps this could be
manifested in something like the fuzzy select tool in the GIMP.

Also, think of the boundary conditions that could be a show stopper
for something like this. What if you move the sky in the second row to
line up with the first row, but you end up with an area for which you
have no information. In the screenshot, I think this area would be to
the left. What should the tool do then?

Ryan

On Jun 22, 4:16 am, Yuv <goo...@...> wrote:

> Jim Watters wrote:
> > Wow!  sounds complicated.  I must admit my normal panorama shooting
> > process does not require using Celeste.
>
> > Wouldn't the simple solution be for Celeste to remove points if there is
> > a mixture of cloud and non-cloud points joining two images.
>
> I think I did not explain myself properly. The problem is not bad CP
> on moving clouds. It is the moving sky in relationship with the static
> subject. It could be also a sky of star.
>
> Look at <http://groups.google.com/group/hugin-ptx/web/Screenshot-Fast
> %20Panorama%20preview.png>
>
> The static scene stitches perfectly, but the background sky has moved
> between the row containing the tip of the clock-tower and the row
> below it.
>
> The easy way out is to fiddle with the clone tool or some other image
> editing tool around the seam between the two rows.
>
> The smart way out IMO would be to mask out the static features (the
> top of the clock-tower) from the top row and then shift to correct for
> the cloud movement over time. Probably shifting the whole row is good
> enough, though shifting each composing image proportionally to the
> timestamp difference may be more accurate. This will leave much less
> fiddling in the image editor: mostly where the sky has moved behind a
> static feature, and a little also where the clouds have changed shape
> over time (easy, because no accuracy is required).
>
> Yuv

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group.
A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ
To post to this group, send email to hugin-ptx@...
To unsubscribe from this group, send email to hugin-ptx-unsubscribe@...
For more options, visit this group at http://groups.google.com/group/hugin-ptx
-~----------~----~----~----~------~----~------~--~---

[Yuval Levy-6]

Hi Ryan,

rlhelinski wrote:
> IMHO it would require the user creating a
> mask for each image so that the tools can treat the different regions
> separately, perhaps by using alpha channels. Perhaps this could be
> manifested in something like the fuzzy select tool in the GIMP.

may be a good idea too, although if Celeste can provide the masks (it
does already, and although not as fine as needed, they may be a good
starting point for some magic-wand-like tool), why do it manually?


> Also, think of the boundary conditions that could be a show stopper
> for something like this. What if you move the sky in the second row to
> line up with the first row, but you end up with an area for which you
> have no information. In the screenshot, I think this area would be to
> the left. What should the tool do then?

there will be inevitably such areas, e.g. when the sky has moved behind
a static object (such as the clock-tower in the example). Leaving the
empty area for human to fill with an image editor is what comes to my
mind as one option.

some manual touch up will be inevitable, the goal is not to eliminate
manual intervention, but to reduce it - from what it is currently (a lot
of shifting and editing all across the seam and around the border of
static objects) to something less (e.g. only around the border of static
objects, and maybe some refining of masks).

Yuv

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group.
A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ
To post to this group, send email to hugin-ptx@...
To unsubscribe from this group, send email to hugin-ptx-unsubscribe@...
For more options, visit this group at http://groups.google.com/group/hugin-ptx
-~----------~----~----~----~------~----~------~--~---