Presets and profiles

[Christian Fredrik Kalager Schaller-3]

Hi everyone,
Daniel G. Taylor and myself have been working on defining a
specification for how we want to handle things like profiles and presets
for transcoding to various devices. The initial usecase is for devices
you do not have attached to your computer, but we love to get feedback
on how we might want to adjust the current specifications to work better
for that usecase too.

There are two documents so far, the first one discussing element level
presets using the GStreamer preset interface:
http://gstreamer.freedesktop.org/wiki/PresetDesign

The second is a device level description of the Device in XML:
http://gstreamer.freedesktop.org/wiki/DeviceProfile

Both documents will probably see quite some changes still, so please
feel free to give feedback and suggestions.

We do hope that this stuff ends up being used beyond Arista and
Transmageddon once it matures a bit, for instance Sound Juicer could use
the Audio information in here for example.

Christian





_______________________________________________
gnome-multimedia mailing list
gnome-multimedia@...
http://mail.gnome.org/mailman/listinfo/gnome-multimedia

[Ross Burton]

On Mon, 2009-05-04 at 11:30 +0100, Christian Fredrik Kalager Schaller wrote:
> We do hope that this stuff ends up being used beyond Arista and
> Transmageddon once it matures a bit, for instance Sound Juicer could use
> the Audio information in here for example.

I'd love to see a replacement in SJ for the gnome-media profiles which
are quite limited.  I'll have a good read of these documents later.

Ross
--
Ross Burton                                 mail: ross@...
                                          jabber: ross@...
                                           www: http://burtonini.com


_______________________________________________
gnome-multimedia mailing list
gnome-multimedia@...
http://mail.gnome.org/mailman/listinfo/gnome-multimedia

[Adam Williamson-4]

On Mon, 2009-05-04 at 11:30 +0100, Christian Fredrik Kalager Schaller
wrote:

> Hi everyone,
> Daniel G. Taylor and myself have been working on defining a
> specification for how we want to handle things like profiles and presets
> for transcoding to various devices. The initial usecase is for devices
> you do not have attached to your computer, but we love to get feedback
> on how we might want to adjust the current specifications to work better
> for that usecase too.
>
> There are two documents so far, the first one discussing element level
> presets using the GStreamer preset interface:
> http://gstreamer.freedesktop.org/wiki/PresetDesign
>
> The second is a device level description of the Device in XML:
> http://gstreamer.freedesktop.org/wiki/DeviceProfile
>
> Both documents will probably see quite some changes still, so please
> feel free to give feedback and suggestions.
>
> We do hope that this stuff ends up being used beyond Arista and
> Transmageddon once it matures a bit, for instance Sound Juicer could use
> the Audio information in here for example.

There's an issue on this topic where I know what the problem is, but I
don't know what a good solution would be: what do you do about devices
where the support depends on the installed software or firmware?

Obvious example is a Windows Mobile smartphone, like mine. (But there
are other cases, I think the PSP would be one, for instance...and Nokia
phones, where you can get third party player apps that extend the
functionality, too.) As it comes from my provider, it could only handle
a limited range of audio formats (no Vorbis support, for e.g.) But now
I've installed TCPMP and the Vorbis plugin on it, it *does* support
Vorbis. And then it may come with different default software installed
if you buy it from another carrier, but as far as any kind of hardware
detection would be concerned, it'd look like the same device...

Should we just require some kind of interface layer provide this
information - as in the iPod example, where libgpod could do it based on
the device firmware? For Windows Mobile, when the device is connected in
sync mode these capabilities could theoretically be detected (as you can
tell what software's installed on the device), but not when connected in
mass storage mode. Do we set up some kind of interface for the user to
define what capabilities the device has, to cover situations like this?
Or is this just an insoluble issue? Hopefully someone smarter than me
has ideas. :)
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net

_______________________________________________
gnome-multimedia mailing list
gnome-multimedia@...
http://mail.gnome.org/mailman/listinfo/gnome-multimedia

[Daniel Taylor-6]

On Mon, 2009-05-04 at 14:10 +0200, Christophe Fergeau wrote:

> Hi,
>
> 2009/5/4 Christian Fredrik Kalager Schaller <uraeus@...>
>        
>         The second is a device level description of the Device in XML:
>         http://gstreamer.freedesktop.org/wiki/DeviceProfile
>
> As this document indicates, the video settings will depend on the iPod
> model. All iPods supporting video (except for the iphone/ipod touch)
> export XML data describing the video formats they support, see
> http://www.ipodlinux.org/wiki/Device_Information and grep for
> "VideoCodec" for an example of what kind of information is available
> on the iPod. Dunno if that's enough to build a device profile or not..
> If that's enough and if you think libgpod should export this
> information, let me know exactly how you'd like it to be done ;)

This is indeed enough to know how to encode for the device - H.264
Baseline @ Level 1.3 with maximum bitrate, width, height and Low
Complexity AAC up to 160kbps @ 48000Hz. No container info but since it's
an iPod we know to use MP4 and name it e.g. m4v.

I'm a bit disappointed that the newer iDevices (TM) don't have the plist
on them... any idea why? Seems like a step in the wrong direction
removing them. I was hoping to add support for transcoding directly to
iPods into Arista with libgpod's Python bindings and this info would
really be useful while waiting for the real device profiles to get
finalized and working.

Either way if this info is somehow obtainable from newer generation
iDevices I think it makes sense to definitely expose it, either as just
an xml file on the device or as properties in the libgpod Database (or
other) object.
--
Daniel G. Taylor
http://programmer-art.org


_______________________________________________
gnome-multimedia mailing list
gnome-multimedia@...
http://mail.gnome.org/mailman/listinfo/gnome-multimedia

[Daniel Taylor-6]

Hey,

On Mon, 2009-05-04 at 10:20 -0700, Adam Williamson wrote:

> * snip *
>
> There's an issue on this topic where I know what the problem is, but I
> don't know what a good solution would be: what do you do about devices
> where the support depends on the installed software or firmware?
>
> Obvious example is a Windows Mobile smartphone, like mine. (But there
> are other cases, I think the PSP would be one, for instance...and Nokia
> phones, where you can get third party player apps that extend the
> functionality, too.) As it comes from my provider, it could only handle
> a limited range of audio formats (no Vorbis support, for e.g.) But now
> I've installed TCPMP and the Vorbis plugin on it, it *does* support
> Vorbis. And then it may come with different default software installed
> if you buy it from another carrier, but as far as any kind of hardware
> detection would be concerned, it'd look like the same device...

This sounds like a total and utter mess to me. I'm in favor of only
supporting the latest released firmwares and a format we think is good
enough, but that's probably not acceptable for many people. The idea of
this whole project is to simplify things, not make them even more
complicated ;-)

> Should we just require some kind of interface layer provide this
> information - as in the iPod example, where libgpod could do it based on
> the device firmware? For Windows Mobile, when the device is connected in
> sync mode these capabilities could theoretically be detected (as you can
> tell what software's installed on the device), but not when connected in
> mass storage mode. Do we set up some kind of interface for the user to
> define what capabilities the device has, to cover situations like this?
> Or is this just an insoluble issue? Hopefully someone smarter than me
> has ideas. :)

Getting info from the device is a great idea, but what about when the
user wants to convert something for her iPod that isn't currently
connected for example? From a purely UI perspective this presents a mess
- who knows what iPod with which software version they have? Same with
smartphones? And how would we present this to the user, with a thousand
devices each with a dozen hard/software revision options? People
generally just want stuff to work.

I've also been thinking about this a lot and can't seem to come up with
an elegant solution that provides everything most people want.

Take care,
--
Daniel G. Taylor
http://programmer-art.org


_______________________________________________
gnome-multimedia mailing list
gnome-multimedia@...
http://mail.gnome.org/mailman/listinfo/gnome-multimedia

[Daniel Taylor-6]

On Mon, 2009-05-04 at 11:30 +0100, Christian Fredrik Kalager Schaller
wrote:
> Hi everyone,

Hey,

> Daniel G. Taylor and myself have been working on defining a
> specification for how we want to handle things like profiles and presets
> for transcoding to various devices. The initial usecase is for devices
> you do not have attached to your computer, but we love to get feedback
> on how we might want to adjust the current specifications to work better
> for that usecase too.
>
> There are two documents so far, the first one discussing element level
> presets using the GStreamer preset interface:
> http://gstreamer.freedesktop.org/wiki/PresetDesign
>
> The second is a device level description of the Device in XML:
> http://gstreamer.freedesktop.org/wiki/DeviceProfile
>
> Both documents will probably see quite some changes still, so please
> feel free to give feedback and suggestions.
>
> We do hope that this stuff ends up being used beyond Arista and
> Transmageddon once it matures a bit, for instance Sound Juicer could use
> the Audio information in here for example.

Just thought I'd mention that there are some great comments on the wiki
left by Mike Smith.

I agree that the use of two different formats for the files seems
unnecessary - I believe it's because of how the presets for GStreamer
are currently setup, right? I'd prefer one format and don't really care
which one, though obviously currently Arista is using XML (and the
format is not stable yet).

In case anyone else misunderstands the passes section in both Arista and
the DeviceProfile wiki page: it's merely a way to define what should be
done for each pass of an encode. In the example we have only one pass
that is based on constant quality, but you could easily see where you'd
want two pass bitrate-constrained encodes (see Nokia N810 for example),
and the passes section allows you to define multiple passes to
accomplish this. As a working example, in Arista trunk it currently runs
through each pass one after another to produce the final file. In each
pass element you put the options for that pass (or in the DeviceProfiles
case the profiles that all elements must provide given the caps like
video/x-h264, which are defined on the PresetDesign page).

I also agree that the syntax is not well defined in the wiki for certain
elements. Here is how it works in Arista:

Each element that gives a constraint is allowed to be either a single
value (e.g. <param>10</param>) or an inclusive range (e.g. <param>10,
30</param>). Internally the single parameter is turned into a range of
e.g. [10, 10] and all calculations in while setting up the transcoder
are done using those ranges (scaling, resampling, etc). I agree that one
shortcoming is not allowing a list of discrete values or allowing
something like e.g. <param>320, 720 % 16</param> to say that the value
can be between 320 and 720 but must be a multiple of 16. I'm working on
this for Arista and will probably be updating the GStreamer wiki with
more info soon as well.

Perhaps it is more useful to direct people to post here rather than the
wiki? Having conversations on a wiki has never really been very
successful for me.

Take care,
--
Daniel G. Taylor
http://programmer-art.org


_______________________________________________
gnome-multimedia mailing list
gnome-multimedia@...
http://mail.gnome.org/mailman/listinfo/gnome-multimedia

[Daniel Taylor-6]

Hey,

On Tue, 2009-05-05 at 10:35 +0200, Christophe Fergeau wrote:

> * snip*
> Not sure exactly which devices you refer to by "newer iDevices". There
> are roughly 2 ipods families for this matter, the iPod Classic/iPod
> Nano (called iPod from now on) on one side, and the iPod Touch/iPhone
> (called iPhone from now on) on the other side.

By newer iDevices I mean what you call the iPhone. ;-)

> For the iPod, the plist xml file doesn't exist on the visible device
> filesystem, but it can be extracted through a SCSI Inquiry command,
> and libgpod installs a hal callout which issues this command and
> writes the plist file to disk when the iPod is plugged in.
>
> For the iPhone, I suspect this information is available somehow from
> the iPhone but at this point in time, I have no idea where to get it
> from. Maybe it's available from the custom USB protocol, maybe it's
> available somewhere in the filesystem, I don't know :) But I'm pretty
> sure it can be found somehow.

I have an iPod Touch, if there's anything I can do to help out here
short of sending it to you (I need it for work) let me know!

Take care,
--
Daniel G. Taylor
http://programmer-art.org


_______________________________________________
gnome-multimedia mailing list
gnome-multimedia@...
http://mail.gnome.org/mailman/listinfo/gnome-multimedia

[Stefan Kost]

Daniel G. Taylor schrieb:

> On Mon, 2009-05-04 at 11:30 +0100, Christian Fredrik Kalager Schaller
> wrote:
>  
>> Hi everyone,
>>    
>
> Hey,
>
>  
>> Daniel G. Taylor and myself have been working on defining a
>> specification for how we want to handle things like profiles and presets
>> for transcoding to various devices. The initial usecase is for devices
>> you do not have attached to your computer, but we love to get feedback
>> on how we might want to adjust the current specifications to work better
>> for that usecase too.
>>
>> There are two documents so far, the first one discussing element level
>> presets using the GStreamer preset interface:
>> http://gstreamer.freedesktop.org/wiki/PresetDesign
>>
>> The second is a device level description of the Device in XML:
>> http://gstreamer.freedesktop.org/wiki/DeviceProfile
>>
>> Both documents will probably see quite some changes still, so please
>> feel free to give feedback and suggestions.
>>
>> We do hope that this stuff ends up being used beyond Arista and
>> Transmageddon once it matures a bit, for instance Sound Juicer could use
>> the Audio information in here for example.
>>    
>
> Just thought I'd mention that there are some great comments on the wiki
> left by Mike Smith.
>
> I agree that the use of two different formats for the files seems
> unnecessary - I believe it's because of how the presets for GStreamer
> are currently setup, right? I'd prefer one format and don't really care
> which one, though obviously currently Arista is using XML (and the
> format is not stable yet).
>
> In case anyone else misunderstands the passes section in both Arista and
> the DeviceProfile wiki page: it's merely a way to define what should be
> done for each pass of an encode. In the example we have only one pass
> that is based on constant quality, but you could easily see where you'd
> want two pass bitrate-constrained encodes (see Nokia N810 for example),
> and the passes section allows you to define multiple passes to
> accomplish this. As a working example, in Arista trunk it currently runs
> through each pass one after another to produce the final file. In each
> pass element you put the options for that pass (or in the DeviceProfiles
> case the profiles that all elements must provide given the caps like
> video/x-h264, which are defined on the PresetDesign page).
>
> I also agree that the syntax is not well defined in the wiki for certain
> elements. Here is how it works in Arista:
>
> Each element that gives a constraint is allowed to be either a single
> value (e.g. <param>10</param>) or an inclusive range (e.g. <param>10,
> 30</param>). Internally the single parameter is turned into a range of
> e.g. [10, 10] and all calculations in while setting up the transcoder
> are done using those ranges (scaling, resampling, etc). I agree that one
> shortcoming is not allowing a list of discrete values or allowing
> something like e.g. <param>320, 720 % 16</param> to say that the value
> can be between 320 and 720 but must be a multiple of 16. I'm working on
> this for Arista and will probably be updating the GStreamer wiki with
> more info soon as well.
>
> Perhaps it is more useful to direct people to post here rather than the
> wiki? Having conversations on a wiki has never really been very
> successful for me.
>
> Take care,
>  

Okay here are my comments (now removed from the wiki):
Totally agree with Mike regarding the devce capabilities. What about even:
video/x-h264, width=[320, 640], height=[240, 480], framerate=(fraction)1/30
you could e.g. reuse the capssrting and put it on a capsfilter as it is
(except the media-type).

Also agree that xml feature are slight overkill here. The purpose of the
device profile is to describe the suggested format and limitation for a
certain device.

On IRC I was suggesting to also take os-versions into account, so that
we end up with profiles like this:
PC.WindowsXP      // use avi with mpeg4 and mp3
PC.WindowsVista   // use asf with wmv and wma
PC.Linux          // use ogg with theora and vorbis
N8x0.Maemo4       // use avi with mpeg4 and mp3
N8x0.Maemo5       // use mp4 with h264 and aac

It could be consided to have a unversioned fallback (N8x0 that even
could be a symlink incase someone does not know the version).

Stefan

> ------------------------------------------------------------------------
>
> _______________________________________________
> gnome-multimedia mailing list
> gnome-multimedia@...
> http://mail.gnome.org/mailman/listinfo/gnome-multimedia
>  

_______________________________________________
gnome-multimedia mailing list
gnome-multimedia@...
http://mail.gnome.org/mailman/listinfo/gnome-multimedia

[Christian Fredrik Kalager Schaller-2]

Let me reply to Stefans and also to MikeS comments from the Wiki (soon
to be removed).

I don't have any strong opinion on the formating of various values, but
I guess making them as GStreamer like as possible is nice in terms of
ease of use in the code and for people to make mental maps of how things
in the profiles map to GStreamer code.

So switching to MikeS proposed:
<channels>[1,6]</channels> or <channels>{1,2,6}</channels>

and Stefans
width=[320, 640], height=[240, 480], framerate=(fraction)1/30

would be fine with me.

As for Stefan and MikeS comment about the two different formats for the
presets themselves and the device profiles. The reason is that in both
cases I simply went with what is already there. I do see however that
the preset system support XML presets so we can use that if having the
same file format for both is preferred. Personally I think it is a minor
issue.

Mike claim that presets apply to only a specific element, which is
wrong. Or at least it is supposed to be wrong. The whole idea about
using presets is to be able to define equivalent presets to two
different encoders for the same format. If we didn't care about that
goal we don't need presets, we can just do what Arista do now and define
the exact attributes we want to use in the device profile XML file.

As for the comment on adjusting the bitrate, I don't think that is part
of scope. I mean the point of these files is to suggest sensible
defaults. If people want to override them that is fine, but I don't see
the value of having a key for overide ability in the device profile
file. I mean most values in there can be overridden, including the codec
choices etc. Only reason I could see for having a bitrate value in the
XML would be if some devices only allows specific bitrate ranges.

Daniel explained the passes stuff in his email so I will not comment on
that.

One important aspect to mention here is that the XML file is not
complete yet. We do want to add some kind of listing of ALL supported
media format combinations on the device. So that applications can find
out which formats it should allow setting if they have an option for
choosing containers and codecs (like Transmageddon does).

Christian



On Tue, 2009-05-05 at 16:35 +0300, Stefan Kost wrote:

> Daniel G. Taylor schrieb:
> > On Mon, 2009-05-04 at 11:30 +0100, Christian Fredrik Kalager Schaller
> > wrote:
> >  
> >> Hi everyone,
> >>    
> >
> > Hey,
> >
> >  
> >> Daniel G. Taylor and myself have been working on defining a
> >> specification for how we want to handle things like profiles and presets
> >> for transcoding to various devices. The initial usecase is for devices
> >> you do not have attached to your computer, but we love to get feedback
> >> on how we might want to adjust the current specifications to work better
> >> for that usecase too.
> >>
> >> There are two documents so far, the first one discussing element level
> >> presets using the GStreamer preset interface:
> >> http://gstreamer.freedesktop.org/wiki/PresetDesign
> >>
> >> The second is a device level description of the Device in XML:
> >> http://gstreamer.freedesktop.org/wiki/DeviceProfile
> >>
> >> Both documents will probably see quite some changes still, so please
> >> feel free to give feedback and suggestions.
> >>
> >> We do hope that this stuff ends up being used beyond Arista and
> >> Transmageddon once it matures a bit, for instance Sound Juicer could use
> >> the Audio information in here for example.
> >>    
> >
> > Just thought I'd mention that there are some great comments on the wiki
> > left by Mike Smith.
> >
> > I agree that the use of two different formats for the files seems
> > unnecessary - I believe it's because of how the presets for GStreamer
> > are currently setup, right? I'd prefer one format and don't really care
> > which one, though obviously currently Arista is using XML (and the
> > format is not stable yet).
> >
> > In case anyone else misunderstands the passes section in both Arista and
> > the DeviceProfile wiki page: it's merely a way to define what should be
> > done for each pass of an encode. In the example we have only one pass
> > that is based on constant quality, but you could easily see where you'd
> > want two pass bitrate-constrained encodes (see Nokia N810 for example),
> > and the passes section allows you to define multiple passes to
> > accomplish this. As a working example, in Arista trunk it currently runs
> > through each pass one after another to produce the final file. In each
> > pass element you put the options for that pass (or in the DeviceProfiles
> > case the profiles that all elements must provide given the caps like
> > video/x-h264, which are defined on the PresetDesign page).
> >
> > I also agree that the syntax is not well defined in the wiki for certain
> > elements. Here is how it works in Arista:
> >
> > Each element that gives a constraint is allowed to be either a single
> > value (e.g. <param>10</param>) or an inclusive range (e.g. <param>10,
> > 30</param>). Internally the single parameter is turned into a range of
> > e.g. [10, 10] and all calculations in while setting up the transcoder
> > are done using those ranges (scaling, resampling, etc). I agree that one
> > shortcoming is not allowing a list of discrete values or allowing
> > something like e.g. <param>320, 720 % 16</param> to say that the value
> > can be between 320 and 720 but must be a multiple of 16. I'm working on
> > this for Arista and will probably be updating the GStreamer wiki with
> > more info soon as well.
> >
> > Perhaps it is more useful to direct people to post here rather than the
> > wiki? Having conversations on a wiki has never really been very
> > successful for me.
> >
> > Take care,
> >  
>
> Okay here are my comments (now removed from the wiki):
> Totally agree with Mike regarding the devce capabilities. What about even:
> video/x-h264, width=[320, 640], height=[240, 480], framerate=(fraction)1/30
> you could e.g. reuse the capssrting and put it on a capsfilter as it is
> (except the media-type).
>
> Also agree that xml feature are slight overkill here. The purpose of the
> device profile is to describe the suggested format and limitation for a
> certain device.
>
> On IRC I was suggesting to also take os-versions into account, so that
> we end up with profiles like this:
> PC.WindowsXP      // use avi with mpeg4 and mp3
> PC.WindowsVista   // use asf with wmv and wma
> PC.Linux          // use ogg with theora and vorbis
> N8x0.Maemo4       // use avi with mpeg4 and mp3
> N8x0.Maemo5       // use mp4 with h264 and aac
>
> It could be consided to have a unversioned fallback (N8x0 that even
> could be a symlink incase someone does not know the version).
>
> Stefan
>
> > ------------------------------------------------------------------------
> >
> > _______________________________________________
> > gnome-multimedia mailing list
> > gnome-multimedia@...
> > http://mail.gnome.org/mailman/listinfo/gnome-multimedia
> >  
>
> _______________________________________________
> gnome-multimedia mailing list
> gnome-multimedia@...
> http://mail.gnome.org/mailman/listinfo/gnome-multimedia

_______________________________________________
gnome-multimedia mailing list
gnome-multimedia@...
http://mail.gnome.org/mailman/listinfo/gnome-multimedia

[Stefan Kost]

hi,

Christian Fredrik Kalager Schaller schrieb:

> Let me reply to Stefans and also to MikeS comments from the Wiki (soon
> to be removed).
>
> I don't have any strong opinion on the formating of various values, but
> I guess making them as GStreamer like as possible is nice in terms of
> ease of use in the code and for people to make mental maps of how things
> in the profiles map to GStreamer code.
>
> So switching to MikeS proposed:
> <channels>[1,6]</channels> or <channels>{1,2,6}</channels>
>
> and Stefans
> width=[320, 640], height=[240, 480], framerate=(fraction)1/30
>
> would be fine with me.
>
> As for Stefan and MikeS comment about the two different formats for the
> presets themselves and the device profiles. The reason is that in both
> cases I simply went with what is already there. I do see however that
> the preset system support XML presets so we can use that if having the
> same file format for both is preferred. Personally I think it is a minor
> issue.
>  

Ack. I was merly pointing ou that gst has useful function to serialize
and deserialize things, but the xml format as it is could not benefit
from it.

> Mike claim that presets apply to only a specific element, which is
> wrong. Or at least it is supposed to be wrong. The whole idea about
> using presets is to be able to define equivalent presets to two
> different encoders for the same format. If we didn't care about that
> goal we don't need presets, we can just do what Arista do now and define
> the exact attributes we want to use in the device profile XML file.
>  
Imho Mike said one preset configure one element. There is
interoperability via preset metadata like tags or names.
> As for the comment on adjusting the bitrate, I don't think that is part
> of scope. I mean the point of these files is to suggest sensible
> defaults. If people want to override them that is fine, but I don't see
> the value of having a key for overide ability in the device profile
> file. I mean most values in there can be overridden, including the codec
> choices etc. Only reason I could see for having a bitrate value in the
> XML would be if some devices only allows specific bitrate ranges.
>  
Once we have all those things in place, we could think of a preset
editor (aka expert mode). Something like
http://buzztard.svn.sourceforge.net/viewvc/buzztard/trunk/buzztard/docs/help/bt-edit/C/figures/machine-properties.png
One sees all the useful properties on the left and the presets on the
right. Loading a presets shows the settings. One can modify them and
save it (as a new preset). If we go that way it would be nice to get
more hints from the elements to construct a sane ui.
For filtering properties:
http://bugzilla.gnome.org/show_bug.cgi?id=522205
Describing dependencies is tough though. We would need a mechanism to
tell which other properties to make sensitive/insensitive when one
property changes. Like we have a notify: signal for each property there
could be a sensitive: signal on gstobject.

Stefan

> Daniel explained the passes stuff in his email so I will not comment on
> that.
>
> One important aspect to mention here is that the XML file is not
> complete yet. We do want to add some kind of listing of ALL supported
> media format combinations on the device. So that applications can find
> out which formats it should allow setting if they have an option for
> choosing containers and codecs (like Transmageddon does).
>
> Christian
>
>
>
> On Tue, 2009-05-05 at 16:35 +0300, Stefan Kost wrote:
>  
>> Daniel G. Taylor schrieb:
>>    
>>> On Mon, 2009-05-04 at 11:30 +0100, Christian Fredrik Kalager Schaller
>>> wrote:
>>>  
>>>      
>>>> Hi everyone,
>>>>    
>>>>        
>>> Hey,
>>>
>>>  
>>>      
>>>> Daniel G. Taylor and myself have been working on defining a
>>>> specification for how we want to handle things like profiles and presets
>>>> for transcoding to various devices. The initial usecase is for devices
>>>> you do not have attached to your computer, but we love to get feedback
>>>> on how we might want to adjust the current specifications to work better
>>>> for that usecase too.
>>>>
>>>> There are two documents so far, the first one discussing element level
>>>> presets using the GStreamer preset interface:
>>>> http://gstreamer.freedesktop.org/wiki/PresetDesign
>>>>
>>>> The second is a device level description of the Device in XML:
>>>> http://gstreamer.freedesktop.org/wiki/DeviceProfile
>>>>
>>>> Both documents will probably see quite some changes still, so please
>>>> feel free to give feedback and suggestions.
>>>>
>>>> We do hope that this stuff ends up being used beyond Arista and
>>>> Transmageddon once it matures a bit, for instance Sound Juicer could use
>>>> the Audio information in here for example.
>>>>    
>>>>        
>>> Just thought I'd mention that there are some great comments on the wiki
>>> left by Mike Smith.
>>>
>>> I agree that the use of two different formats for the files seems
>>> unnecessary - I believe it's because of how the presets for GStreamer
>>> are currently setup, right? I'd prefer one format and don't really care
>>> which one, though obviously currently Arista is using XML (and the
>>> format is not stable yet).
>>>
>>> In case anyone else misunderstands the passes section in both Arista and
>>> the DeviceProfile wiki page: it's merely a way to define what should be
>>> done for each pass of an encode. In the example we have only one pass
>>> that is based on constant quality, but you could easily see where you'd
>>> want two pass bitrate-constrained encodes (see Nokia N810 for example),
>>> and the passes section allows you to define multiple passes to
>>> accomplish this. As a working example, in Arista trunk it currently runs
>>> through each pass one after another to produce the final file. In each
>>> pass element you put the options for that pass (or in the DeviceProfiles
>>> case the profiles that all elements must provide given the caps like
>>> video/x-h264, which are defined on the PresetDesign page).
>>>
>>> I also agree that the syntax is not well defined in the wiki for certain
>>> elements. Here is how it works in Arista:
>>>
>>> Each element that gives a constraint is allowed to be either a single
>>> value (e.g. <param>10</param>) or an inclusive range (e.g. <param>10,
>>> 30</param>). Internally the single parameter is turned into a range of
>>> e.g. [10, 10] and all calculations in while setting up the transcoder
>>> are done using those ranges (scaling, resampling, etc). I agree that one
>>> shortcoming is not allowing a list of discrete values or allowing
>>> something like e.g. <param>320, 720 % 16</param> to say that the value
>>> can be between 320 and 720 but must be a multiple of 16. I'm working on
>>> this for Arista and will probably be updating the GStreamer wiki with
>>> more info soon as well.
>>>
>>> Perhaps it is more useful to direct people to post here rather than the
>>> wiki? Having conversations on a wiki has never really been very
>>> successful for me.
>>>
>>> Take care,
>>>  
>>>      
>> Okay here are my comments (now removed from the wiki):
>> Totally agree with Mike regarding the devce capabilities. What about even:
>> video/x-h264, width=[320, 640], height=[240, 480], framerate=(fraction)1/30
>> you could e.g. reuse the capssrting and put it on a capsfilter as it is
>> (except the media-type).
>>
>> Also agree that xml feature are slight overkill here. The purpose of the
>> device profile is to describe the suggested format and limitation for a
>> certain device.
>>
>> On IRC I was suggesting to also take os-versions into account, so that
>> we end up with profiles like this:
>> PC.WindowsXP      // use avi with mpeg4 and mp3
>> PC.WindowsVista   // use asf with wmv and wma
>> PC.Linux          // use ogg with theora and vorbis
>> N8x0.Maemo4       // use avi with mpeg4 and mp3
>> N8x0.Maemo5       // use mp4 with h264 and aac
>>
>> It could be consided to have a unversioned fallback (N8x0 that even
>> could be a symlink incase someone does not know the version).
>>
>> Stefan
>>
>>    
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> gnome-multimedia mailing list
>>> gnome-multimedia@...
>>> http://mail.gnome.org/mailman/listinfo/gnome-multimedia
>>>  
>>>      
>> _______________________________________________
>> gnome-multimedia mailing list
>> gnome-multimedia@...
>> http://mail.gnome.org/mailman/listinfo/gnome-multimedia
>>    
>
> _______________________________________________
> gnome-multimedia mailing list
> gnome-multimedia@...
> http://mail.gnome.org/mailman/listinfo/gnome-multimedia
>  

_______________________________________________
gnome-multimedia mailing list
gnome-multimedia@...
http://mail.gnome.org/mailman/listinfo/gnome-multimedia

[David Joaquim-2]

Hi,

The idea to develop highlevel gstreamer libraries is really nice!

Some thoughts about your proposal.

First I see one problem, or rather a limitation of your DeviceProfile.

Why a profile should necessarily be associated with a device ?

I'm thinking about a profile library that replaces the current horrible,
gnome-media profile "framework".
The profile system may be used by any gnome application (Sound-Juicer,
...), but may also define profiles that did not correspond to any
specific device.
Who uses flac for a portable player, or even a TV ?
The fact is that people may want to be able to create a any gstreamer
supported format profile.

Now, the DeviceProfile is not a bad idea at all. It is definitely a
great idea!
We can imagine to build it on top of the general profile library.
Strictly speaking an DeviceProfile is a profile with some limitations
(the one from the device).

We can also put device information into profile, merging GeneralProfile
with DeviceProfile.
Like: This profile can be used for: iThing, Nokia, ...

What do you think ?

Second, the presets.
Preset is cool, but seems limited to just one element. (Did I
misunderstood something ?)
The problem is that in many cases your pipeline contains many elements
and also caps.

A simple example is this mp3 encoding :

... ! rate=44100, channels=2 ! audioconvert ! lame mode=4 vbr-quality=7
! xingmux ! id3v2mux ! ...

Where xingmux is used to add xingheader to the vbr file, and the
id3v2mux used to pass tags to the output file.

Therefore, how to configure all of them at the same time ?


Then :)

Let me explain how I imagine things from now.

We need three things :

* A Profile Library (definitely with a device specific support!)
* A transcoding library
* A set of UI components and application

The profile library is used to create, store, edit, profile (low level).

The transcoding library is used to transcode files between differents
formats.
It uses directly profiles to configure the transcoding operation.
The main problem is to generate the pipeline from the profile desciption.

The set of graphical component is designed for integration, simlilarly
to the current gnome-media-profile.
There are predefined components, each gnome application can directly
include it.

A profile manager can also be part of the graphical components. As
standalone/integrated application.


What I can offer:
I'm currently a developper of gnac, an audio converter:
http://gnac.sourceforge.net/download/

What we have :
* a profile manager: really simpler than the gnome-media one.

The main problem now is that the UI of the manager is not automatically
generated.
You cannot, for example, add a new format like the celt codec without
writing some code.
The goal is that you can pass a "format description" and then the UI is
automatically generated.
It also needs many code cleaning :)

* a C transcoding library
It currently supports, audio transcoding, video transcoding, external
subtitle integration (incrustation, or inclusion if the output format
supports it).

I think we can adapt this library to fit the new needs.
The main point is to define what will be the interface of the library.

I hope this contribution will help to build a new shiny set of highlevel
components to gstreamer!

 
David

Christian Fredrik Kalager Schaller a écrit :

> Hi everyone,
> Daniel G. Taylor and myself have been working on defining a
> specification for how we want to handle things like profiles and presets
> for transcoding to various devices. The initial usecase is for devices
> you do not have attached to your computer, but we love to get feedback
> on how we might want to adjust the current specifications to work better
> for that usecase too.
>
> There are two documents so far, the first one discussing element level
> presets using the GStreamer preset interface:
> http://gstreamer.freedesktop.org/wiki/PresetDesign
>
> The second is a device level description of the Device in XML:
> http://gstreamer.freedesktop.org/wiki/DeviceProfile
>
> Both documents will probably see quite some changes still, so please
> feel free to give feedback and suggestions.
>
> We do hope that this stuff ends up being used beyond Arista and
> Transmageddon once it matures a bit, for instance Sound Juicer could use
> the Audio information in here for example.
>
> Christian
>
>
>
>
>
> _______________________________________________
> gnome-multimedia mailing list
> gnome-multimedia@...
> http://mail.gnome.org/mailman/listinfo/gnome-multimedia
>  

_______________________________________________
gnome-multimedia mailing list
gnome-multimedia@...
http://mail.gnome.org/mailman/listinfo/gnome-multimedia

[Stefan Kost]

David Joaquim schrieb:

<snip>

> Then :)
>
> Let me explain how I imagine things from now.
>
> We need three things :
>
> * A Profile Library (definitely with a device specific support!)
> * A transcoding library
> * A set of UI components and application
>
> The profile library is used to create, store, edit, profile (low level).
>
> The transcoding library is used to transcode files between differents
> formats.
> It uses directly profiles to configure the transcoding operation.
> The main problem is to generate the pipeline from the profile desciption.

Please consider encoding and transcoding as a special case for decoding and
encoding (or just remuxing). Applications like pitivi,jokosher,buzztard & co
would like to benefit from the whole profile&preset work too.

Stefan
_______________________________________________
gnome-multimedia mailing list
gnome-multimedia@...
http://mail.gnome.org/mailman/listinfo/gnome-multimedia

[David Joaquim-2]

Stefan Kost a écrit :
>
> Please consider encoding and transcoding as a special case for decoding and
> encoding (or just remuxing). Applications like pitivi,jokosher,buzztard & co
> would like to benefit from the whole profile&preset work too.
>
> Stefan
>  

Interresting!
I didn't thought about that.

The difficult part is to provide a convenient way to encode from
raw-video/audio to the specified profile.
Am I right ?

We can develop something like an encodebin :)
The transcodebin will use a decodebin and an encodebin.

I mean, for transcoding, the decoding part is "just" a decodebin, with
no additionnal work.

Let's say an encoding library then.
With an optional direct from file encoding.

It can be a good idea to make a list of all application that may want to
use theses libraries and define a set of use cases!

* Converters (transcoders)
* Extractors (sound-juicer, music players)
* Multimedia editors / authoring tool

Do I forget someone ?
_______________________________________________
gnome-multimedia mailing list
gnome-multimedia@...
http://mail.gnome.org/mailman/listinfo/gnome-multimedia

[David Joaquim-2]

By the way, a "device" can also be an website like youtube.
Providing specific profile to publish on youtube can be great too!
_______________________________________________
gnome-multimedia mailing list
gnome-multimedia@...
http://mail.gnome.org/mailman/listinfo/gnome-multimedia

[Stefan Kost]

David Joaquim schrieb:

> Stefan Kost a écrit :
>> Please consider encoding and transcoding as a special case for decoding and
>> encoding (or just remuxing). Applications like pitivi,jokosher,buzztard & co
>> would like to benefit from the whole profile&preset work too.
>>
>> Stefan
>>  
>
> Interresting!
> I didn't thought about that.
>
> The difficult part is to provide a convenient way to encode from
> raw-video/audio to the specified profile.
> Am I right ?

Jep, you have n-audio and m-video tracks and you want to offer the user a
convinient way to encode into for archival or for a specific device or service.

>
> We can develop something like an encodebin :)
> The transcodebin will use a decodebin and an encodebin.
>
> I mean, for transcoding, the decoding part is "just" a decodebin, with
> no additionnal work.

Almost. The transcodebin would need to be smart to stop decodebin from decoding
streams if remuxing is enough. And an encodebin does not make too much sense as
you need to specify encoders and muxers anyway. We coudl have a function in the
library altough, that given the specs, creates and ordinary bin and adds the
requested elements there. We would need to define the needed info for doing that:
- Transcoding: you get media-specs from deodebin
- Encoding: the application provides it
These media-specs could e.g. be a list of pads.
Now we let the user choose a format that can handle the spec. (like if there is
video, we don't offer wav).
When you have the target format, create a bin for it and link pads. In case of
transcoding if would eventualy restart the decodebin to avoid the decoders.


>
> Let's say an encoding library then.
> With an optional direct from file encoding.
>
> It can be a good idea to make a list of all application that may want to
> use theses libraries and define a set of use cases!
>
> * Converters (transcoders)
> * Extractors (sound-juicer, music players)
    Thoggen for DVDs :)
> * Multimedia editors / authoring tool
I think its really two things Encoding (kind of transcoding from raw) and
Transcoding.

>
> Do I forget someone ?

Stefan
_______________________________________________
gnome-multimedia mailing list
gnome-multimedia@...
http://mail.gnome.org/mailman/listinfo/gnome-multimedia

[David Joaquim-2]

> Almost. The transcodebin would need to be smart to stop decodebin from decoding
> streams if remuxing is enough.
Sure. It may be a interesting problem to solve by the way.

> And an encodebin does not make too much sense as
> you need to specify encoders and muxers anyway.
> We coudl have a function in the
> library altough, that given the specs, creates and ordinary bin and adds the
> requested elements there.
Yes. We need a piece of code that builds the tail of the pipeline from a
profile and a given stream.
But, It can be presented as a single bin, black box, using ghost pads
inside).
>  We would need to define the needed info for doing that:
> - Transcoding: you get media-specs from deodebin
> - Encoding: the application provides it
> These media-specs could e.g. be a list of pads.
>  
The pads seems to be a good choice.

> Now we let the user choose a format that can handle the spec. (like if there is
> video, we don't offer wav).
> When you have the target format, create a bin for it and link pads. In case of
> transcoding if would eventualy restart the decodebin to avoid the decoders.
>
>
>  
>> Let's say an encoding library then.
>> With an optional direct from file encoding.
>>
>> It can be a good idea to make a list of all application that may want to
>> use theses libraries and define a set of use cases!
>>
>> * Converters (transcoders)
>> * Extractors (sound-juicer, music players)
>>    
>     Thoggen for DVDs :)
>  

This one seems to be the most difficult case :)
Multiple video streams, subtitles and audio...
+ removable media source
>> * Multimedia editors / authoring tool
>>    
> I think its really two things Encoding (kind of transcoding from raw) and
> Transcoding.
>  
I totally agree, but it seems possible to handle both at the same time I
think.

David
_______________________________________________
gnome-multimedia mailing list
gnome-multimedia@...
http://mail.gnome.org/mailman/listinfo/gnome-multimedia

[Christian Fredrik Kalager Schaller-3]

On Tue, 2009-05-05 at 20:13 +0200, David Joaquim wrote:
> Hi,
>
> The idea to develop highlevel gstreamer libraries is really nice!
>
> Some thoughts about your proposal.
>
> First I see one problem, or rather a limitation of your DeviceProfile.
> Why a profile should necessarily be associated with a device ?
The device profiles is not meant to solve all the problems in the world,
they are meant to solve a limited set of problems :)

> I'm thinking about a profile library that replaces the current horrible,
> gnome-media profile "framework".
> The profile system may be used by any gnome application (Sound-Juicer,
> ...), but may also define profiles that did not correspond to any
> specific device.
> Who uses flac for a portable player, or even a TV ?
> The fact is that people may want to be able to create a any gstreamer
> supported format profile.
Well there comes a point where defining profiles stop having a meaning :)
A profile that allows you to do anything isn't really a profile anymore.

> Now, the DeviceProfile is not a bad idea at all. It is definitely a
> great idea!
> We can imagine to build it on top of the general profile library.
> Strictly speaking an DeviceProfile is a profile with some limitations
> (the one from the device).
>
> We can also put device information into profile, merging GeneralProfile
> with DeviceProfile.
> Like: This profile can be used for: iThing, Nokia, ...
Not sure what you want to accomplish

> What do you think ?
>
> Second, the presets.
> Preset is cool, but seems limited to just one element. (Did I
> misunderstood something ?)
No they are limited to one element, but of course any application and/or
library interacting with them would group them in some way.

> The problem is that in many cases your pipeline contains many elements
> and also caps.
>
> A simple example is this mp3 encoding :
>
> ... ! rate=44100, channels=2 ! audioconvert ! lame mode=4 vbr-quality=7
> ! xingmux ! id3v2mux ! ...
>
> Where xingmux is used to add xingheader to the vbr file, and the
> id3v2mux used to pass tags to the output file.
>
> Therefore, how to configure all of them at the same time ?

Well that would be the job of a helper library or the application right?
If you have two encoders and all you want to do is have them both use a
'High Quality' preset you have an api applying that preset to both your
encoders.

> Then :)
>
> Let me explain how I imagine things from now.
>
> We need three things :
>
> * A Profile Library (definitely with a device specific support!)
> * A transcoding library
> * A set of UI components and application
>
> The profile library is used to create, store, edit, profile (low level).
>
> The transcoding library is used to transcode files between differents
> formats.
> It uses directly profiles to configure the transcoding operation.

Not necessarily. Profiles makes sense for certain kind of problems,
for instance devices which has a relatively static set of properties.
So a profile system is good for getting values you can feed into a
transcoding/encoding library, but it would never be the only source for
such information. One could for instance have an application where you
set every value manually.

> The main problem is to generate the pipeline from the profile desciption.
Well in some sense  implementing it is the easy problem, the hard thing
is deciding what kind of API and what kind of features such a library
would offer.

> The set of graphical component is designed for integration, simlilarly
> to the current gnome-media-profile.
> There are predefined components, each gnome application can directly
> include it.
>
> A profile manager can also be part of the graphical components. As
> standalone/integrated application.
>
>
> What I can offer:
> I'm currently a developper of gnac, an audio converter:
> http://gnac.sourceforge.net/download/
>
> What we have :
> * a profile manager: really simpler than the gnome-media one.
>
> The main problem now is that the UI of the manager is not automatically
> generated.
> You cannot, for example, add a new format like the celt codec without
> writing some code.
> The goal is that you can pass a "format description" and then the UI is
> automatically generated.
> It also needs many code cleaning :)
>
> * a C transcoding library
> It currently supports, audio transcoding, video transcoding, external
> subtitle integration (incrustation, or inclusion if the output format
> supports it).
>
> I think we can adapt this library to fit the new needs.
> The main point is to define what will be the interface of the library.
>
> I hope this contribution will help to build a new shiny set of highlevel
> components to gstreamer!

What kind of API do you provide at this point?

Also one thing you probably want to do right away is relicense all the
code to the LGPLv2.1. If it is to be integrated into GStreamer at some
point it would need to be under that license.

Christian

_______________________________________________
gnome-multimedia mailing list
gnome-multimedia@...
http://mail.gnome.org/mailman/listinfo/gnome-multimedia

[David Joaquim-2]

Christian Fredrik Kalager Schaller a écrit :
> The device profiles is not meant to solve all the problems in the world,
> they are meant to solve a limited set of problems :)
>  
Definitly, but for me profile is more general that just device.
>
> Well there comes a point where defining profiles stop having a meaning :)
> A profile that allows you to do anything isn't really a profile anymore.
>  
I think about it as a preconfigured set of settings.
Avoiding to reconfigure each parameter every time.

We may first agree on the definition of profile :p

> Not sure what you want to accomplish
>
>  
A profile can be a triplet :
 - A video pipeline
 - An audio pipeline
 - A muxer.

When you define a profile, you set the values for the triplet.
theora for video, vorbis for audio, oggmux for the muxer, with a set of
per element properties.

A device profile is the same, but, with various limitations.
For example a profile for iPod is limited to a subset of all video/audio
codecs and muxer.
x264 for video with a maximal size, complexity and bitrate, aac for
audio and mp4 muxer.

You can configure the iPod profile using these limitations.

Of course the final profile is a static thing. You just select it, and
use it.
I was talking about how users may want to configure a profile.

We can provide a finite set of predefined profile.
However,  I think it is more interesting to also provide a interface
that permits to create customized profiles.

The lambda user will use the predefined profile without asking questions.
An advanced user may create his own profile and save it into the profile
list.
Later he can reuse it.


> Well that would be the job of a helper library or the application right?
> If you have two encoders and all you want to do is have them both use a
> 'High Quality' preset you have an api applying that preset to both your
> encoders.
>  
Sure.
My point was that end user doesn't care about which element he is
currently using.
He just want to know which parameters it can interact with.
The idea is to pack multiple element's properties (they which are
interesting), and provide an high level configuration.

If you want to encode a video in a different format and size, you don't
care about configuring format element (the encoder), and the resizer.
You just want to specify theora 300x200, and the api will do the job to
configure the underlying things.

> Not necessarily. Profiles makes sense for certain kind of problems,
> for instance devices which has a relatively static set of properties.
> So a profile system is good for getting values you can feed into a
> transcoding/encoding library, but it would never be the only source for
> such information. One could for instance have an application where you
> set every value manually.
>  
In that case you may also want to create the pipeline manually :)
I think profile is the convenient way to configure an encoding.
As I mean profile is like a set of parameters.
>> The main problem is to generate the pipeline from the profile desciption.
>>    
> Well in some sense  implementing it is the easy problem, the hard thing
> is deciding what kind of API and what kind of features such a library
> would offer.
>  
Of course, but being generic make the generation harder.
>  
> What kind of API do you provide at this point?
>  
For the moment we have something like that (not exactly, but lets say it
is the case):

EncodingItem {
    GFile *source;
    GFile *destination;
    Profile *profile;
}

void
transcode(EncodingItem *item);

The transcode function takes the source get the parameters from the
profile (we use something like gst-launch now) and encode and write it
to the destination.
Internally we use a decodebin to get raw audio and video and then
connect the pads to the encoders.

As pointed out in an other post, we can do better job in case of remuxing.

About the profile manager.

Similarly to the Banshee player, we have a set of profile-descriptors.
It defines the format (ogg, mp3, aac, flac, ...).
It essentially defines what properties you can interact with and their
default values.

We have an UI that permits the interaction with the parameters and save
the result into a profile.
A part of the UI is still manually done, but we want to generate it
automatically from the profile descriptor.
The code is not so clear, we may benefit to code it in python or another
high-level language.


> Also one thing you probably want to do right away is relicense all the
> code to the LGPLv2.1. If it is to be integrated into GStreamer at some
> point it would need to be under that license.
>  
Yes, thanks to point that out.
I have to agree with the other developers, but I think it will be ok.


I hope you have a more precise idea of what I mean by profile.
I'm pretty sure we didn't think about it the same way :)

David
_______________________________________________
gnome-multimedia mailing list
gnome-multimedia@...
http://mail.gnome.org/mailman/listinfo/gnome-multimedia

[Zeeshan Ali (Khattak)-2]

> * a C transcoding library
>It currently supports, audio transcoding, video transcoding, external
>subtitle integration (incrustation, or inclusion if the output format
>supports it).
>
> I think we can adapt this library to fit the new needs.
>The main point is to define what will be the interface of the library.

  FWIW, i can use such a library in Rygel[1] as well. Currently I have
my own transcoding classes for transcoding from any format to mp3,
lpcm and mpeg ts stream. If you are interested you can look at the
code in *transcoder*.vala files here:
http://git.gnome.org/cgit/rygel/tree/src/rygel/

--
Regards,

Zeeshan Ali (Khattak)
FSF member#5124

[1] http://live.gnome.org/Rygel
_______________________________________________
gnome-multimedia mailing list
gnome-multimedia@...
http://mail.gnome.org/mailman/listinfo/gnome-multimedia

[Bastien Nocera]

On Tue, 2009-05-05 at 23:47 +0300, Stefan Kost wrote:
<snip>

> > It can be a good idea to make a list of all application that may want to
> > use theses libraries and define a set of use cases!
> >
> > * Converters (transcoders)
> > * Extractors (sound-juicer, music players)
>     Thoggen for DVDs :)
> > * Multimedia editors / authoring tool
> I think its really two things Encoding (kind of transcoding from raw) and
> Transcoding.
>
> >
> > Do I forget someone ?

It could be useful in Totem as well. Movie -> Save As...

_______________________________________________
gnome-multimedia mailing list
gnome-multimedia@...
http://mail.gnome.org/mailman/listinfo/gnome-multimedia