DoxyGen Style Kernel API reference.

[brian gianforcaro]

I'm interested in setting up a Doxygen or Doxygen-like documentation reference for DragonFly-BSD. I have been very interested in working on DragonFly for a considerable amount of time. I've realized that for the most part I just have no idea how most of the code works or what underlying API's the kernel and userland use. I do realize a large amount of effort goes into writing and keeping man pages up to date, documenting these very things. However, I feel like having an on line resource to see for example, function call graph's and data structures explained might lower the barrier to entry for new developers. And possibly even help out the experienced dev's when they've forgotten how pieces of code works or interacts.

I imagine this documentation would be built nightly off of master. So developers always have the freshest doc's available.

I have noticed that Net/Open/FreeBSD don't have anything similar either. Would other developers be interested in such a resource? I certainly don't want to put my time and effort into something that would likely never be used.

Some examples of other projects doxygen documentation.
    * D-Bus documentation http://freedesktop.org/software/dbus/doc/api/html/index.html
    * Visualization Toolkit http://www.vtk.org/doc/nightly/html/index.html
    * IBM's International Components for Unicode http://www.icu-project.org/apiref/icu4c/

If you could just let me know any of the pro's/con's you see and what your vote would be. Also if you have any specific requests for what you would like the automatically generated documentation to contain, I can put them on my todo list.

Thanks,
- Brian Gianforcaro

[Simon 'corecode' Schubert]

Brian Gianforcaro wrote:
> I'm interested in setting up a Doxygen or Doxygen-like documentation reference for DragonFly-BSD. I have been very interested in working on DragonFly for a considerable amount of time. I've realized that for the most part I just have no idea how most of the code works or what underlying API's the kernel and userland use. I do realize a large amount of effort goes into writing and keeping man pages up to date, documenting these very things. However, I feel like having an on line resource to see for example, function call graph's and data structures explained might lower the barrier to entry for new developers. And possibly even help out the experienced dev's when they've forgotten how pieces of code works or interacts.

I think that's a good idea.  However you will realize that a lot of the functions are not documented enough to make such a reference useful.  Considerable work would have to go into actually documenting the functions.

cheers
  simon

[Samuel J. Greear-4]

> I think that's a good idea.  However you will realize that a lot of the
> functions are not documented enough to make such a reference useful.
>  Considerable work would have to go into actually documenting the functions.

Knowing (as in, having a generated list of) what functions need to be
documented would be useful in its own right.

[Emil Mikulic-2]

On Thu, Sep 03, 2009 at 01:28:49PM -0400, Brian Gianforcaro wrote:
> I have noticed that Net/Open/FreeBSD don't have anything similar either.

Alexander Leidinger has been working on doxygen-generated docs
for FreeBSD.

See for example: http://www.leidinger.net/FreeBSD/dox/vm/html/

--Emil

[Justin C. Sherrill]

On Thu, September 3, 2009 1:28 pm, Brian Gianforcaro wrote:
>
> I'm interested in setting up a Doxygen or Doxygen-like documentation
> reference for DragonFly-BSD.

I can't think of a reason to not do it.  My only question is who would
maintain it?  I don't know how much effort it would take over time, and
I'd hate to have something bit-rot.  If you're willing to keep it fresh,
so to speak, that's great!

[Sascha Wildner]

Brian Gianforcaro schrieb:
> If you could just let me know any of the pro's/con's you see and what your vote would be.

In the doxygen scenario, who would actually _write_ the documentation?
The same people who don't write or adjust the man9 manpages right now? :)

Note, that this is not meant as offense or complain to any developer.
People don't like writing documentation, have no time, forget about it,
coding is more important and fun etc. etc. No real problem with me, it's
the usual way of things.

I just that I don't see how a different format, way of presentation etc.
would help with this fundamental issue, so all in all, I don't feel too
enthusiastic about the idea.

Sascha

--
http://yoyodyne.ath.cx

[Sdävtaker]

If you doing it, Im up to help.
Damian

On Fri, Sep 4, 2009 at 06:15, Sascha Wildner <saw@...> wrote:

Brian Gianforcaro schrieb:

If you could just let me know any of the pro's/con's you see and what your vote would be.

In the doxygen scenario, who would actually _write_ the documentation? The same people who don't write or adjust the man9 manpages right now? :)

Note, that this is not meant as offense or complain to any developer. People don't like writing documentation, have no time, forget about it, coding is more important and fun etc. etc. No real problem with me, it's the usual way of things.

I just that I don't see how a different format, way of presentation etc. would help with this fundamental issue, so all in all, I don't feel too enthusiastic about the idea.

Sascha

--
http://yoyodyne.ath.cx

--
Sdävtaker prays to Rikku goddess for a good treasure.

[Erik Wikström-2]

On 2009-09-04 11:15, Sascha Wildner wrote:

> Brian Gianforcaro schrieb:
>> If you could just let me know any of the pro's/con's you see and what your vote would be.
>
> In the doxygen scenario, who would actually _write_ the documentation?
> The same people who don't write or adjust the man9 manpages right now? :)
>
> Note, that this is not meant as offense or complain to any developer.
> People don't like writing documentation, have no time, forget about it,
> coding is more important and fun etc. etc. No real problem with me, it's
> the usual way of things.
>
> I just that I don't see how a different format, way of presentation etc.
> would help with this fundamental issue, so all in all, I don't feel too
> enthusiastic about the idea.

An advantage with doxygen like documentation is that the documentation
is in the code together with the code, the man-pages are separate files
that have to be opened and edited separately. This makes it much easier
to just update the comments while updating the code, with man-pages you
have to open a totally different file and use (to me at least) obscure
formatting.

But I do agree with you that it would probably not be well updated, I
know that I myself forget to update comments from time to time,
especially doxygen-comments.

--
Erik Wikström

[Stathis Kamperis-2]

2009/9/3 Brian Gianforcaro <b.gianfo@...>:
>
> I'm interested in setting up a Doxygen or Doxygen-like documentation reference for DragonFly-BSD.

Would that require modifying existing comments to properly expose the
various information to doxygen parser ?

Cheers,
Stathis

[brian gianforcaro]

Depending on how nice you want the documentation to be, and how good
the existing documentation is.

I'd be more than willing to put all my effort into modifying the
existing comments to be doxygen compliant, if this what is finally
decided.

- Brian Gianforcaro

On Fri, Sep 4, 2009 at 1:53 PM, Stathis Kamperis<ekamperi@...> wrote:

> 2009/9/3 Brian Gianforcaro <b.gianfo@...>:
>>
>> I'm interested in setting up a Doxygen or Doxygen-like documentation reference for DragonFly-BSD.
>
> Would that require modifying existing comments to properly expose the
> various information to doxygen parser ?
>
> Cheers,
> Stathis
>

[Jeffrey Hsu]

On Fri, Sep 4, 2009 at 2:48 PM, Brian Gianforcaro <b.gianfo@...> wrote:

Depending on how nice you want the documentation to be, and how good
the existing documentation is.

I'd be more than willing to put all my effort into modifying the
existing comments to be doxygen compliant, if this what is finally
decided.

This is a lot of effort.  If the real goal is to better understand the code, I think your time would be better spent directly reading the code (not the comments) with a good source code browser like cscope or lxr.

[Sascha Wildner]

Brian Gianforcaro schrieb:
> Depending on how nice you want the documentation to be, and how good
> the existing documentation is.
>
> I'd be more than willing to put all my effort into modifying the
> existing comments to be doxygen compliant, if this what is finally
> decided.

I wonder, how would the existing man9 manpages fit into the picture?

Maintaining man9 and doxygen in parallel? I think there is even less
chance that developers will care about both man9 & doxygen than taking
care of man9 alone.

Fold the info from the manpages into the code as comments and discard
man9 entirely? That sounds like either a lot of comments would have to
be added or (in case you should decide to take only selected portions of
man9 into the comments) a lot of documentation would have to be discarded.

Note that I'm just trying to realistically picture what would happen if
we decide to use doxygen for kernel documentation.

Sascha

--
http://yoyodyne.ath.cx

[Simon 'corecode' Schubert]

Sascha Wildner wrote:

> Brian Gianforcaro schrieb:
>> Depending on how nice you want the documentation to be, and how good
>> the existing documentation is.
>>
>> I'd be more than willing to put all my effort into modifying the
>> existing comments to be doxygen compliant, if this what is finally
>> decided.
> Maintaining man9 and doxygen in parallel? I think there is even less
> chance that developers will care about both man9 & doxygen than taking
> care of man9 alone.

I actually don't think so.  Docs can be copied from the doxygen to the
man page format.

I worked on the linux source in the last days, and I have to say that
proper doxygen comments are *really* helpful for navigating in unknown code.

I strongly support Brian's enthusiasm and suggest just going forward and
doing it.  Let's see how it is working out.  The more documentation the
better.  I don't think we should turn this into a lengthy discussion.

cheers
   simon

[Sascha Wildner]

Simon 'corecode' Schubert schrieb:

>> Maintaining man9 and doxygen in parallel? I think there is even less
>> chance that developers will care about both man9 & doxygen than taking
>> care of man9 alone.
>
> I actually don't think so.  Docs can be copied from the doxygen to the
> man page format.
>
> I worked on the linux source in the last days, and I have to say that
> proper doxygen comments are *really* helpful for navigating in unknown code.
>
> I strongly support Brian's enthusiasm and suggest just going forward and
> doing it.  Let's see how it is working out.  The more documentation the
> better.  I don't think we should turn this into a lengthy discussion.
>

Who said doxygen is not helpful? I'm not against doxygen, if that is the
impression. But do you think Linux's kernel documentation is being cared
about by 2-3 people like ours is?

The questions I asked were valid questions about the future of man9 in a
doxygen scenario. Good plans that will work should have answers to
questions raised and be able to withstand a little discussion, don't you
think?

So please let's not imply that discussion can kill good plans. On the
contrary, it helps to separate the plans based on enthusiasm only from
those that are very clear about the real work and implications involved
and yet have a good chance to work out good (which is all I care for at
this point). And please let's not place ourselves on either ends of the
opinion scale only (either totally pro or anti doxygen).

And regarding "docs can be copied from the doxygen to the man page
format": You totally missed my point. _Who_ is going to do that work or
care about it? For example, take stuff that comes in new after the
switch to doxygen. Do you think developers want to write doxygen
comments _and_ a new manual page? :)

To me, it is obvious that any attempt to maintain both man9 and doxygen
in parallel is not going to work because I can't imagine developers will
happily want to double the documentation work. Which I'm fine with btw,
one thing less to care about. Just thought I'd point it out in case
there was a different impression.

Sascha

--
http://yoyodyne.ath.cx

[Dennis Melentyev-2]

Hi Sascha and all,

2009/9/4 Sascha Wildner <saw@...>:

> Brian Gianforcaro schrieb:
>>
>> Depending on how nice you want the documentation to be, and how good
>> the existing documentation is.
>>
>> I'd be more than willing to put all my effort into modifying the
>> existing comments to be doxygen compliant, if this what is finally
>> decided.
>
> I wonder, how would the existing man9 manpages fit into the picture?
>
> Maintaining man9 and doxygen in parallel? I think there is even less chance
> that developers will care about both man9 & doxygen than taking care of man9
> alone.
>
> Fold the info from the manpages into the code as comments and discard man9
> entirely? That sounds like either a lot of comments would have to be added
> or (in case you should decide to take only selected portions of man9 into
> the comments) a lot of documentation would have to be discarded.
>
> Note that I'm just trying to realistically picture what would happen if we
> decide to use doxygen for kernel documentation.

Sascha, I do understand your concern. Have 25 developers next door :)

Usually, this kind of problem is only about the "process" (name it
development culture, habits, etc).

In many [not only commercial] projects some submit/commit rules are present:
0. Description of the problem it solves
1. We appreciate your patch in "diff -u blah-blah-blah" format
2. It should include man page changes (final or preliminary)
3. Codestyle should conform http://..../
4. Codestyle includes comment rules (usually handled while in review cycle).

Here, from bugzilla, for example:
http://www.bugzilla.org/docs/developer.html#intro
http://www.bugzilla.org/docs/developer.html#perl-style
http://www.bugzilla.org/docs/developer.html#apidocs

Each patch/submission always has a code review part. And almost NEVER
gets commited immediately (unless trivial changes).

Writing doxygen/javadoc-like comments is not a big effort: just
purpose of the module, function, arguments and some key notes. It is
more like helping future developers (and author himself, if coming
back to sources after another year of not touching it) with future
maintenance tasks.

Writing a man page is much more serious task. It IS a documentation,
while Doxygen is more like API description/outline.

PS. just my 0.02 UAH for what it worth.
--
Dennis Melentyev

[Matthew Dillon]

    I've looked at the web site and I do like the concept.  I think we
    could incorporate Doxygen elements in the kernel code as an
    augmentation of our section 9 manual pages.
         
    To be clear, I'm talking about this style of documentation:

    /**
     * Short descript.  Longer description ....
     * ...
     *
     * @param arg1 description
     * ...
     * @return description
     *
     */

    I don't think we need to impose any major formalities, it can be
    as simple as a one-liner.  In otherwords, however little or much
    the programmer wants to throw in there would be fine, at least as
    long as it doesn't get past 15 lines or so :-).

    There are some practical issues, however.  When I built doxygen
    from pkgsrc it brought in a ridiculous number of packages, including
    ghostscript.

    I wonder... could we just write our own source code parser that is
    compatible with a subset of Doxygen formatting conventions and generate
    the HTML and MAN templates ourselves?  Without using the actual doxygen
    package?

                                        -Matt

[Simon 'corecode' Schubert]

Matthew Dillon wrote:

>     I've looked at the web site and I do like the concept.  I think we
>     could incorporate Doxygen elements in the kernel code as an
>     augmentation of our section 9 manual pages.
>  
>     To be clear, I'm talking about this style of documentation:
>
>     /**
>      * Short descript.  Longer description ....
>      * ...
>      *
>      * @param arg1 description
>      * ...
>      * @return description
>      *
>      */
>
>     I don't think we need to impose any major formalities, it can be
>     as simple as a one-liner.  In otherwords, however little or much
>     the programmer wants to throw in there would be fine, at least as
>     long as it doesn't get past 15 lines or so :-).
>
>     There are some practical issues, however.  When I built doxygen
>     from pkgsrc it brought in a ridiculous number of packages, including
>     ghostscript.
>
>     I wonder... could we just write our own source code parser that is
>     compatible with a subset of Doxygen formatting conventions and generate
>     the HTML and MAN templates ourselves?  Without using the actual doxygen
>     package?

I think it is okay to use the pkgsrc package on those systems where
developers want to generate the documentation.  Not everybody will need
to generate it, not even all developers.

cheers
   simon

--
   <3 the future  +++  RENT this banner advert  +++   ASCII Ribbon   /"\
   rock the past  +++  space for low CHF NOW!1  +++     Campaign     \ /
Party Enjoy Relax   |   http://dragonflybsd.org      Against  HTML   \
Dude 2c 2 the max   !   http://golden-apple.biz       Mail + News   / \

[Simon 'corecode' Schubert]

Matthew Dillon wrote:

>     I've looked at the web site and I do like the concept.  I think we
>     could incorporate Doxygen elements in the kernel code as an
>     augmentation of our section 9 manual pages.
>  
>     To be clear, I'm talking about this style of documentation:
>
>     /**
>      * Short descript.  Longer description ....
>      * ...
>      *
>      * @param arg1 description
>      * ...
>      * @return description
>      *
>      */
>
>     I don't think we need to impose any major formalities, it can be
>     as simple as a one-liner.  In otherwords, however little or much
>     the programmer wants to throw in there would be fine, at least as
>     long as it doesn't get past 15 lines or so :-).
>
>     There are some practical issues, however.  When I built doxygen
>     from pkgsrc it brought in a ridiculous number of packages, including
>     ghostscript.
>
>     I wonder... could we just write our own source code parser that is
>     compatible with a subset of Doxygen formatting conventions and generate
>     the HTML and MAN templates ourselves?  Without using the actual doxygen
>     package?

I think it is okay to use the pkgsrc package on those systems where
developers want to generate the documentation.  Not everybody will need
to generate it, not even all developers.

cheers
   simon

--
   <3 the future  +++  RENT this banner advert  +++   ASCII Ribbon   /"\
   rock the past  +++  space for low CHF NOW!1  +++     Campaign     \ /
Party Enjoy Relax   |   http://dragonflybsd.org      Against  HTML   \
Dude 2c 2 the max   !   http://golden-apple.biz       Mail + News   / \