libbozohttpd and lua

[Alistair Crooks]

Just to add some fuel to the fires...

I have need of a webserver that can embedded in other things.  I've
taken bozohttpd, and split it into libbozohttpd and a main.c (a short
driver program).  The embedded version still has one iffy bit - the
current way of interrupting output which is taking too long is to have
an alarm(3) raised.  This doesn't scale, and won't embed well.  At the
same time, this raises questions about the whole way of architecting
an embedded server, whilst keeping the small, simple nature of
bozohttpd intact.

The libbozohttpd + main.c version passes all the testsuite tests.

I've also rewritten the driver program as a small lua script (and some
supporting C glue). The lua script is < 160 lines and that includes
license and comments, and all of the options httpd(8) supports.
Feedback on that is solicited and will be gratefully received.

Everything can be found in:

        http://www.netbsd.org/~agc/libhttpd-lua-20091105-2030.tgz

Regards,
Alistair

[Antti Kantee]

On Fri Nov 06 2009 at 07:12:27 +0000, Alistair Crooks wrote:

> Just to add some fuel to the fires...
>
> I have need of a webserver that can embedded in other things.  I've
> taken bozohttpd, and split it into libbozohttpd and a main.c (a short
> driver program).  The embedded version still has one iffy bit - the
> current way of interrupting output which is taking too long is to have
> an alarm(3) raised.  This doesn't scale, and won't embed well.  At the
> same time, this raises questions about the whole way of architecting
> an embedded server, whilst keeping the small, simple nature of
> bozohttpd intact.

What is a driver program?  Like inetd (which bozo used to require)?

Your problem sounds like a case for threads or non-blocking handles
which needs to be massaged from the driver until data has been sent.
But I guess you don't want to do it that way?

> I've also rewritten the driver program as a small lua script (and some
> supporting C glue). The lua script is < 160 lines and that includes
> license and comments, and all of the options httpd(8) supports.
> Feedback on that is solicited and will be gratefully received.

Sounds cool.  I'm soliciting comparison data which has been a little
hard to find on this subject.  The obvious ones are:
* how big is main.c
* how big is the C glue
* how does it perform (apachebench should do the trick?)

And the "86 developers out of 100 prefer X to the leading brand"
subjective metric:
* do you consider the lua version significantly easier/better (keeping
  in mind you have to write the C glue and maintain it)

(don't have a terminal for easy targz viewing currently)

[Martin Husemann]

On Fri, Nov 06, 2009 at 10:41:39AM +0200, Antti Kantee wrote:
> And the "86 developers out of 100 prefer X to the leading brand"
> subjective metric:
> * do you consider the lua version significantly easier/better (keeping
>   in mind you have to write the C glue and maintain it)

Two more nits:
 * how easy is it to add more command line options to the Lua code now
   without touching any C code?
 * would you consider the Lua driver a kind of configuration file, thus
   breaking the bozo paradigm?

Martin

[Alistair Crooks]

On Fri, Nov 06, 2009 at 10:41:39AM +0200, Antti Kantee wrote:

> On Fri Nov 06 2009 at 07:12:27 +0000, Alistair Crooks wrote:
> > Just to add some fuel to the fires...
> >
> > I have need of a webserver that can embedded in other things.  I've
> > taken bozohttpd, and split it into libbozohttpd and a main.c (a short
> > driver program).  The embedded version still has one iffy bit - the
> > current way of interrupting output which is taking too long is to have
> > an alarm(3) raised.  This doesn't scale, and won't embed well.  At the
> > same time, this raises questions about the whole way of architecting
> > an embedded server, whilst keeping the small, simple nature of
> > bozohttpd intact.
>
> What is a driver program?  Like inetd (which bozo used to require)?

A driver program is something that calls the routines in the library
from the command line.  Its usage was common in Computing Science in
the 1970s.  You kids.

I wasn't aware that bozo required inetd - indeed, I used to run it from
the command line on a number of systems.
 
> Your problem sounds like a case for threads or non-blocking handles
> which needs to be massaged from the driver until data has been sent.
> But I guess you don't want to do it that way?

I was thinking more of an event driven system, like libevent or libev.
But that kind of usage would mean that the requirements for bozohttpd
just got much, much larger than before.
 
> > I've also rewritten the driver program as a small lua script (and some
> > supporting C glue). The lua script is < 160 lines and that includes
> > license and comments, and all of the options httpd(8) supports.
> > Feedback on that is solicited and will be gratefully received.
>
> Sounds cool.  I'm soliciting comparison data which has been a little
> hard to find on this subject.  The obvious ones are:
> * how big is main.c

% grep -e ';$' main.c | wc -l
     104
% wc -l main.c
     413 main.c
%

> * how big is the C glue

% grep -e ';$' bindings/lua/glue.c | wc -l
      96
% wc -l bindings/lua/glue.c
     276 bindings/lua/glue.c
%
 
(For the rationale behind counting the number of semicolons at the end
of line being a non-commented line of code, I refer the honourable
gentleman to some speaker at a Unix conference from the 80s or 90s - I
think they were from Bell Labs.  I realise that this doesn't narrow it
down much).

> * how does it perform (apachebench should do the trick?)

Not sure how it performs.

Regards,
Alistair

[Alistair Crooks]

On Fri, Nov 06, 2009 at 09:55:28AM +0100, Martin Husemann wrote:
> On Fri, Nov 06, 2009 at 10:41:39AM +0200, Antti Kantee wrote:
> > And the "86 developers out of 100 prefer X to the leading brand"
> > subjective metric:
> > * do you consider the lua version significantly easier/better (keeping
> >   in mind you have to write the C glue and maintain it)
>
> Two more nits:
>  * how easy is it to add more command line options to the Lua code now
>    without touching any C code?

The lua code uses optparse.lua (kind of equivalent to getopt_long). Each option
is specified using one line of lua, e.g.

        opt.add_option{"-u", "--enable-users", action="store_true", dest="enableusers", help="--enable-users"}

(Options are parsed with the "options,args = opt.parse_args()" call)

Deciding what to do if an option is specified is 3 lines of lua:

        if options.enableusers then
                bozohttpd.set_pref(prefs, "enable users", "true")
        end

In the C code, there is the corresponding "if name set to value, then
do this":

        if ((cp = bozo_get_pref(prefs, "enable users")) != NULL &&
            strcmp(cp, "true") == 0) {
                httpd->enable_users = 1;
        }

Now I'm not saying everyone should do it like this, just that I find it
a very useful and clear way of organising option setting.

If the option code was already there in the C library, just waiting to
be called, then you'd just have to enable it with the 3 + 1 lines of
lua code above. But you could say very much the same about C.

>  * would you consider the Lua driver a kind of configuration file, thus
>    breaking the bozo paradigm?

No - the lua program decides what to do if the options are given, and then
calls code to do things. I'd consider the config file to be setting the
options. Two different things.

Regards,
Alistair