templates, macros and spaces

[Jeff Garland]

Joel de Guzman wrote:

>> Or is this already possible with the template mechanism in some other
>> way?
>
> That's a perfect job for Python! :) I guess we should provide a
> boost python wrapper support for accessing the parser state info.
> This is Dave's wish list and I intend to pursue it along these
> lines. Python will have access to the "collectors", the (soon
> to be) "code stacks", filenames, symbol tables, etc. Then, in
> Quickbook, we can pass info to and from Python.

Sorry to be a stick in the mud, but please don't increase the Quickbook
tool chain by adding Python to perform basic functions that don't
require it now.

Jeff


_______________________________________________
Boost-docs mailing list
Boost-docs@...
Unsubscribe and other administrative requests: https://lists.sourceforge.net/lists/listinfo/boost-docs

[Joel de Guzman-2]

Jeff Garland wrote:

> Joel de Guzman wrote:
>
>>> Or is this already possible with the template mechanism in some other
>>> way?
>> That's a perfect job for Python! :) I guess we should provide a
>> boost python wrapper support for accessing the parser state info.
>> This is Dave's wish list and I intend to pursue it along these
>> lines. Python will have access to the "collectors", the (soon
>> to be) "code stacks", filenames, symbol tables, etc. Then, in
>> Quickbook, we can pass info to and from Python.
>
> Sorry to be a stick in the mud, but please don't increase the Quickbook
> tool chain by adding Python to perform basic functions that don't
> require it now.

Understood. As I see it, the basic std.qbk library should not require
any scripting. I think there should be a way to enable scripting only
when needed. IOTW, the user should not pay for things s/he does not
need. With that restriction, templates should still simplify quickbook
code to about 10-20% its current size.

Pardon me if I wasn't clear with my reply to Dan. The [heading] markup
will not be a template (again, basic std.qbk library should not require
any scripting). What I meant is that if someone wants to add some user-
markup that does something similar to [heading], it would (as I imagine)
have to be through a scripting interface.

Regards,
--
Joel de Guzman
http://www.boost-consulting.com
http://spirit.sf.net



_______________________________________________
Boost-docs mailing list
Boost-docs@...
Unsubscribe and other administrative requests: https://lists.sourceforge.net/lists/listinfo/boost-docs

[Jeff Garland]

Joel de Guzman wrote:

> Understood. As I see it, the basic std.qbk library should not require
> any scripting. I think there should be a way to enable scripting only
> when needed. IOTW, the user should not pay for things s/he does not
> need. With that restriction, templates should still simplify quickbook
> code to about 10-20% its current size.
>
> Pardon me if I wasn't clear with my reply to Dan. The [heading] markup
> will not be a template (again, basic std.qbk library should not require
> any scripting). What I meant is that if someone wants to add some user-
> markup that does something similar to [heading], it would (as I imagine)
> have to be through a scripting interface.

No problem -- thx for the clarification and the continuing effort on
this 'weekend project' -- who knows, in 6 months I might be needing that
Python extension ;-)

Jeff



_______________________________________________
Boost-docs mailing list
Boost-docs@...
Unsubscribe and other administrative requests: https://lists.sourceforge.net/lists/listinfo/boost-docs

[David Abrahams]

Joel de Guzman <joel@...> writes:

> Jeff Garland wrote:
>> Joel de Guzman wrote:
>>
>>>> Or is this already possible with the template mechanism in some other
>>>> way?
>>> That's a perfect job for Python! :) I guess we should provide a
>>> boost python wrapper support for accessing the parser state info.
>>> This is Dave's wish list and I intend to pursue it along these
>>> lines. Python will have access to the "collectors", the (soon
>>> to be) "code stacks", filenames, symbol tables, etc. Then, in
>>> Quickbook, we can pass info to and from Python.
>>
>> Sorry to be a stick in the mud, but please don't increase the Quickbook
>> tool chain by adding Python to perform basic functions that don't
>> require it now.
>
> Understood. As I see it, the basic std.qbk library should not require
> any scripting. I think there should be a way to enable scripting only
> when needed. IOTW, the user should not pay for things s/he does not
> need. With that restriction, templates should still simplify quickbook
> code to about 10-20% its current size.
>
> Pardon me if I wasn't clear with my reply to Dan. The [heading] markup
> will not be a template (again, basic std.qbk library should not require
> any scripting). What I meant is that if someone wants to add some user-
> markup that does something similar to [heading], it would (as I imagine)
> have to be through a scripting interface.

Sorry, but I really disagree with this approach.  In a sense, the
biggest reason for the mess that is the current Qbk toolchain is the
lack of Python.  There's no reason for us to go through docbook and
XSLT in order to generate HTML pages (or PDF, for that matter -- LaTeX
is a much more reliable path to good PDFs than FOP is, and it will be
so for some time, AFAICT).

Here's how it works in docutils: the front-end parser builds an
AST/DOM (in Python of course -- a much better language for ASTs than
C++ because of their very dynamic structure).  The various back-end
translators (also written in Python) are simply SAX-like visitors that
generate "events" for the entry and exit of the various AST nodes.  So
when entering the "table" node, the translator's visit_table method
(if any) is called, and when leaving, its depart_table method is
called.

The code for translating the DOM into HTML, LaTeX, or any number of
other formats they already have support for is surprisingly
straightforward and maintainable.  With a system like this, if we
wanted to write a Qbk->Docbook translator in addition to the Qbk->HTML
translator it would be simple.

--
Dave Abrahams
Boost Consulting
www.boost-consulting.com



_______________________________________________
Boost-docs mailing list
Boost-docs@...
Unsubscribe and other administrative requests: https://lists.sourceforge.net/lists/listinfo/boost-docs

[Rene Rivera]

David Abrahams wrote:
> (in Python of course -- a much better language for ASTs than
> C++ because of their very dynamic structure).

Not that I disagree with that. But I think preferring more than one
language, I'm counting the C++ implementation, is going to be
detrimental to the eventual flexibility of quickbooc. After all I might
want to use Lua or PHP, also languages well suited for dynamic nature of
ASTs.

> The various back-end
> translators (also written in Python) are simply SAX-like visitors that
> generate "events" for the entry and exit of the various AST nodes.  So
> when entering the "table" node, the translator's visit_table method
> (if any) is called, and when leaving, its depart_table method is
> called.
>
> The code for translating the DOM into HTML, LaTeX, or any number of
> other formats they already have support for is surprisingly
> straightforward and maintainable.  With a system like this, if we
> wanted to write a Qbk->Docbook translator in addition to the Qbk->HTML
> translator it would be simple.

One thought I had yesterday about the backend quandary is to generate a
quickbook specific format and have the back ends transform that into the
real target format. Basically it's what Dave is suggesting but having an
external version of the AST. If the AST format is XML based for
BoostBook/DocBook the translation could still be done with XSLT. And
hence would be a minimal impact on the current toolchain. For doing a
backend on Python, PHP, etc. it would be easy to translate that into an
internal AST as most such languages already have libraries for XML->AST
parsing.


--
-- Grafik - Don't Assume Anything
-- Redshift Software, Inc. - http://redshift-software.com
-- rrivera/acm.org - grafik/redshift-software.com
-- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo


_______________________________________________
Boost-docs mailing list
Boost-docs@...
Unsubscribe and other administrative requests: https://lists.sourceforge.net/lists/listinfo/boost-docs

[David Abrahams]

Rene Rivera <grafik.list@...> writes:

> David Abrahams wrote:
>> (in Python of course -- a much better language for ASTs than
>> C++ because of their very dynamic structure).
>
> Not that I disagree with that. But I think preferring more than one
> language, I'm counting the C++ implementation, is going to be
> detrimental to the eventual flexibility of quickbooc.  After all I
> might want to use Lua or PHP, also languages well suited for dynamic
> nature of ASTs.

Sorry, I don't get it.  You can always use a Python backend that
writes XML and read that in with Lua or PHP as you're suggesting
below.  It sounds like you're saying that, because someone might be
interested in any number of different languages that are superior to
C++ for doing this backend work, we should stick exclusively to C++, a
language that's inferior for this sort of work.

In fairness, my suggestion does impose the requirement to have Python
installed, even on developers who aren't interested in Python, but
doesn't everybody have Python already? ;-)

>> The various back-end translators (also written in Python) are
>> simply SAX-like visitors that generate "events" for the entry and
>> exit of the various AST nodes.  So when entering the "table" node,
>> the translator's visit_table method (if any) is called, and when
>> leaving, its depart_table method is called.
>>
>> The code for translating the DOM into HTML, LaTeX, or any number of
>> other formats they already have support for is surprisingly
>> straightforward and maintainable.  With a system like this, if we
>> wanted to write a Qbk->Docbook translator in addition to the Qbk->HTML
>> translator it would be simple.
>
> One thought I had yesterday about the backend quandary is to generate a
> quickbook specific format and have the back ends transform that into the
> real target format. Basically it's what Dave is suggesting but having an
> external version of the AST.

Isn't that just BoostBook?

> If the AST format is XML based for BoostBook/DocBook the translation
> could still be done with XSLT.

You say that like it's a good thing. :)

> And hence would be a minimal impact on the current toolchain. For
> doing a backend on Python, PHP, etc. it would be easy to translate
> that into an internal AST as most such languages already have
> libraries for XML->AST parsing.

Isn't that where we are today?

--
Dave Abrahams
Boost Consulting
www.boost-consulting.com



_______________________________________________
Boost-docs mailing list
Boost-docs@...
Unsubscribe and other administrative requests: https://lists.sourceforge.net/lists/listinfo/boost-docs

[Reece Dunn]

Dave Abrahams wrote:

> Rene Rivera <grafik.list@...> writes:
>
> > David Abrahams wrote:
> >> (in Python of course -- a much better language for ASTs than
> >> C++ because of their very dynamic structure).
> >
> > Not that I disagree with that. But I think preferring more than one
> > language, I'm counting the C++ implementation, is going to be
> > detrimental to the eventual flexibility of quickbooc.  After all I
> > might want to use Lua or PHP, also languages well suited for dynamic
> > nature of ASTs.
>
> Sorry, I don't get it.  You can always use a Python backend that
> writes XML and read that in with Lua or PHP as you're suggesting
> below.  It sounds like you're saying that, because someone might be
> interested in any number of different languages that are superior to
> C++ for doing this backend work, we should stick exclusively to C++, a
> language that's inferior for this sort of work.

This essentially comes down to where in the QuickBook processing you want
to run the extended functionality. The current implementation is a good
QuickBook -> DocBook translator. It currently provides basic transformation
and manipulation facilities via macros and the new template feature.

What you are wanting is the ability to hook into the QuickBook processor
as it is transforming the document and provide powerful scripting where
you need to extend the QuickBook facilities -- sort of like uber-templates :).

> In fairness, my suggestion does impose the requirement to have Python
> installed, even on developers who aren't interested in Python, but
> doesn't everybody have Python already? ;-)

Most people who use Boost, but not everyone.

> > One thought I had yesterday about the backend quandary is to generate a
> > quickbook specific format and have the back ends transform that into the
> > real target format. Basically it's what Dave is suggesting but having an
> > external version of the AST.
>
> Isn't that just BoostBook?

You can use BoostBook as the XML format to represent the AST. In that case,
you don't need to transform QuickBook to yet another intermediate XML format.

> > If the AST format is XML based for BoostBook/DocBook the translation
> > could still be done with XSLT.
>
> You say that like it's a good thing. :)

XSLT is very powerful and useful in certain circumstaces, but adds another step
in the transformation tool chain. Allowing scripting hooks to output a custom
format (defaulting to BoostBook) would allow you to speed up the transformation
and produce cleaner native output (do I really want all of the DocBook RTF or
HTML output?)

> > And hence would be a minimal impact on the current toolchain. For
> > doing a backend on Python, PHP, etc. it would be easy to translate
> > that into an internal AST as most such languages already have
> > libraries for XML->AST parsing.
>
> Isn't that where we are today?

Yes. With BoostBook, you can post-process the XML after QuickBook. Being
able to use scripting hooks/bindings to Python (optionally via a --scripting
flag?) would allow more powerful transformations on the source document.

As an example, you could embed compressed sections in DocBook and then
decompress them in Python. Try doing that in XSLT!

- Reece
_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE!
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

_______________________________________________
Boost-docs mailing list
Boost-docs@...
Unsubscribe and other administrative requests: https://lists.sourceforge.net/lists/listinfo/boost-docs

[Rene Rivera]

Reece Dunn wrote:

> Dave Abrahams wrote:
>> Rene Rivera <grafik.list@...> writes:
>>
>>> David Abrahams wrote:
>>>> (in Python of course -- a much better language for ASTs than
>>>> C++ because of their very dynamic structure).
>>> Not that I disagree with that. But I think preferring more than one
>>> language, I'm counting the C++ implementation, is going to be
>>> detrimental to the eventual flexibility of quickbooc.  After all I
>>> might want to use Lua or PHP, also languages well suited for dynamic
>>> nature of ASTs.
>> Sorry, I don't get it.  You can always use a Python backend that
>> writes XML and read that in with Lua or PHP as you're suggesting
>> below.  It sounds like you're saying that, because someone might be
>> interested in any number of different languages that are superior to
>> C++ for doing this backend work, we should stick exclusively to C++, a
>> language that's inferior for this sort of work.

I don't think I was saying that :-( I did somewhat imply that
integrating C++ with other systems is easier than integrating, for
example, Python and PHP. I.e. it would be easier and considerably safer
to make a quickbook PHP module than it would be to have PHP run Python.

> This essentially comes down to where in the QuickBook processing you want
> to run the extended functionality. The current implementation is a good
> QuickBook -> DocBook translator. It currently provides basic transformation
> and manipulation facilities via macros and the new template feature.

Yes. And it was my understanding that quickbook currently makes use of
some assumptions given that the output will be processed by a docbook
processor. Not sure what those are as Joel wasn't specific when he
mentioned it to me.

> What you are wanting is the ability to hook into the QuickBook processor
> as it is transforming the document and provide powerful scripting where
> you need to extend the QuickBook facilities -- sort of like uber-templates :).

Not sure if that's what we all want ;-) I should come clean and mention
what I currently want... I have a use I want to put quickbook to. I
need/want to use the quickbook syntax within a wiki context. This comes
from evaluating the variety of wiki formats and realizing that they just
plain suck. The structuring features of quickbook alone blow away any
current wiki format. So ultimately I want a quickbook library I can plug
into a web server, either directly or through PHP, to translate from
user edited quickbook content to XHTML fragments, as fast as possible.

But to put it closer to "home". It would really cool if the officially
supported Boost wiki used quicbook content. Also it would be nice to be
able to dynamically present Boost documentation on the website without
having to go through the slow docbook translation and check in into CVS.

>> In fairness, my suggestion does impose the requirement to have Python
>> installed, even on developers who aren't interested in Python, but
>> doesn't everybody have Python already? ;-)
>
> Most people who use Boost, but not everyone.

And more likely the decision of what tools to use are not up to the
developer.


--
-- Grafik - Don't Assume Anything
-- Redshift Software, Inc. - http://redshift-software.com
-- rrivera/acm.org - grafik/redshift-software.com
-- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo


_______________________________________________
Boost-docs mailing list
Boost-docs@...
Unsubscribe and other administrative requests: https://lists.sourceforge.net/lists/listinfo/boost-docs

[David Abrahams]

Rene Rivera <grafik.list@...> writes:

> Not sure if that's what we all want ;-) I should come clean and mention
> what I currently want... I have a use I want to put quickbook to. I
> need/want to use the quickbook syntax within a wiki context. This comes
> from evaluating the variety of wiki formats and realizing that they just
> plain suck. The structuring features of quickbook alone blow away any
> current wiki format. So ultimately I want a quickbook library I can plug
> into a web server, either directly or through PHP, to translate from
> user edited quickbook content to XHTML fragments, as fast as possible.

Like the Trac wiki currently does with ReStructuredText (using Python,
of course ;-)

I've been hoping we'll make a Trac plugin for QuickBook, too.

> But to put it closer to "home". It would really cool if the
> officially supported Boost wiki used quicbook content. Also it would
> be nice to be able to dynamically present Boost documentation on the
> website without having to go through the slow docbook translation
> and check in into CVS.

Yep.

>>> In fairness, my suggestion does impose the requirement to have Python
>>> installed, even on developers who aren't interested in Python, but
>>> doesn't everybody have Python already? ;-)
>>
>> Most people who use Boost, but not everyone.
>
> And more likely the decision of what tools to use are not up to the
> developer.

What developer?  I mean, who are we trying to reach with QuickBook?
Is it so crucial to be able to serve that small minority of developers
who are allowed to use C++ but not Python?

--
Dave Abrahams
Boost Consulting
www.boost-consulting.com



_______________________________________________
Boost-docs mailing list
Boost-docs@...
Unsubscribe and other administrative requests: https://lists.sourceforge.net/lists/listinfo/boost-docs

[Jeff Garland]

David Abrahams wrote:

> What developer?  I mean, who are we trying to reach with QuickBook?
> Is it so crucial to be able to serve that small minority of developers
> who are allowed to use C++ but not Python?

Ok, since I started the fight I'll jump back in.  I have no problem if
Python replaces xslt, etc in the toolchain.  My only objection making
the tool-chain even longer than it already is -- it's a pain now and
Python is one more thing I don't normally use.  My current quickbook use
actually circumvents bjam and boost build because I have a 15 minute
perl hack that converts boostbook into docbook.  Yes, I can setup Python
in all the environments I use.  It's really just that I work on lots of
different computers and it gets to be a pain to constantly setup complex
environments. One day you need to switch from Linux to windows and
another hour gets burned figuring out how to get all the tools installed...

Jeff



_______________________________________________
Boost-docs mailing list
Boost-docs@...
Unsubscribe and other administrative requests: https://lists.sourceforge.net/lists/listinfo/boost-docs

[David Abrahams]

Jeff Garland <jeff@...> writes:

> It's really just that I work on lots of different computers and it
> gets to be a pain to constantly setup complex environments. One day
> you need to switch from Linux to windows and another hour gets
> burned figuring out how to get all the tools installed...

I hear ya!  Fortunately, there's nothing special involved in setting
up Python.

--
Dave Abrahams
Boost Consulting
www.boost-consulting.com



_______________________________________________
Boost-docs mailing list
Boost-docs@...
Unsubscribe and other administrative requests: https://lists.sourceforge.net/lists/listinfo/boost-docs

[James Mansion]

>installed, even on developers who aren't interested in Python, but
>doesn't everybody have Python already? ;-)

No. ;-)  Boost is a C++ library system, I think the toolchain should
be C++ based with a simple script to bootstrap the build tool
if you must.

(OK, I have Python on *one* of my Win32 machines, but only recently)





_______________________________________________
Boost-docs mailing list
Boost-docs@...
Unsubscribe and other administrative requests: https://lists.sourceforge.net/lists/listinfo/boost-docs

[Joel de Guzman-2]

Rene Rivera wrote:

> Yes. And it was my understanding that quickbook currently makes use of
> some assumptions given that the output will be processed by a docbook
> processor. Not sure what those are as Joel wasn't specific when he
> mentioned it to me.

Not that much, I think. QuickBook is still pretty much output-agnostic.
There's stuff like the Document Info section, but looking at it again,
I see no reason why that would be a problem to decoupling the back-end.
Recall that QuickBook was initially HTML generator. I think decoupling
the backend is indeed a possibility.

Regards,
--
Joel de Guzman
http://www.boost-consulting.com
http://spirit.sf.net



_______________________________________________
Boost-docs mailing list
Boost-docs@...
Unsubscribe and other administrative requests: https://lists.sourceforge.net/lists/listinfo/boost-docs

[Joel de Guzman-2]

David Abrahams wrote:
> Rene Rivera <grafik.list@...> writes:

>>>> In fairness, my suggestion does impose the requirement to have Python
>>>> installed, even on developers who aren't interested in Python, but
>>>> doesn't everybody have Python already? ;-)
>>> Most people who use Boost, but not everyone.
>> And more likely the decision of what tools to use are not up to the
>> developer.
>
> What developer?  I mean, who are we trying to reach with QuickBook?
> Is it so crucial to be able to serve that small minority of developers
> who are allowed to use C++ but not Python?

I'd like QuickBook to move outside the realms of Boost too,
as a generic document generation mechanism. So, yes, the
lesser dependencies, the better.

Regards,
--
Joel de Guzman
http://www.boost-consulting.com
http://spirit.sf.net



_______________________________________________
Boost-docs mailing list
Boost-docs@...
Unsubscribe and other administrative requests: https://lists.sourceforge.net/lists/listinfo/boost-docs

[David Abrahams]

"James Mansion" <james@...> writes:

>>installed, even on developers who aren't interested in Python, but
>>doesn't everybody have Python already? ;-)
>
> No. ;-)  Boost is a C++ library system, I think the toolchain should
> be C++ based with a simple script to bootstrap the build tool
> if you must.

That's too simplistic a viewpoint.  We're trying to deliver C++ code
and documentation.  It matters a whole lot more _that_ we do it, and
do it well, than it matters how we do it.  There are some jobs that
C++ handles very poorly by comparison to other tools, and if there's a
better tool out there we shouldn't be afraid to exploit it.

Besides, Python is written in an old-fashioned variant of C++ ;-)

--
Dave Abrahams
Boost Consulting
www.boost-consulting.com



_______________________________________________
Boost-docs mailing list
Boost-docs@...
Unsubscribe and other administrative requests: https://lists.sourceforge.net/lists/listinfo/boost-docs