Best Practices for managing repetitive Apache config file entries?

I'm working on a mod_perl project where we would like to implement to
Apache related goals:

1. Automate repetitive configurations. For developers, we repeat the
same configs with the only difference being the "~user" location. In
other cases, the differences are not far from s/alpha/beta.

2. Put the Apache configurations under source control. Here the goal is
to tie changes to the Apache configs to other code changes in the
project.

Here's my outline for now we might do this. Are there other options
that work well for you?


1. We would be a template of Apache configs unders source control.
2. When the template changes, we'll run a script to generate all the
sections. Apache would pull this in as an include file.

In the event a launch is rolled back, the template could reverted to an
older version, and the include-file could be regenerated from the older
version.

Sound reasonable? What else would you recommend?

    Mark



--
 . . . . . . . . . . . . . . . . . . . . . . . . . . .
   Mark Stosberg            Principal Developer  
   mark@...     Summersault, LLC    
   765-939-9301 ext 202     database driven websites
 . . . . . http://www.summersault.com/ . . . . . . . .

[Bill Moseley]

On Fri, Nov 6, 2009 at 10:30 AM, Mark Stosberg <mark@...> wrote:

I'm working on a mod_perl project where we would like to implement to
Apache related goals:

1. Automate repetitive configurations. For developers, we repeat the
same configs with the only difference being the "~user" location. In
other cases, the differences are not far from s/alpha/beta.

2. Put the Apache configurations under source control. Here the goal is
to tie changes to the Apache configs to other code changes in the
project.

I generate httpd.conf files on the fly (i.e. when I start Apache) from Template toolkit and YAML files.

The script merges the configurations from multilple YAML files -- i.e. form a main config.yaml file and then when, say, running on "staginge", from staging.yml.  Then it merges with a local file which allows individual developers to override settings in their home directory.  Then the apache-specific config is passed to a script that builds the httpd.conf files and starts (or stops) apache.

--
Bill Moseley
moseley@...

[Dirk-Willem van Gulik]

Mark Stosberg wrote:
...
> 1. We would be a template of Apache configs unders source control.

> 2. When the template changes, we'll run a script to generate all the
> sections. Apache would pull this in as an include file.

> Sound reasonable? What else would you recommend?

I often use the svn hooks for this. Another thing to look at is mod perl
its ability to generate things on the fly - ideal for generating a lot
of ~user variations.

Thanks,

Dw

[Jeff Nokes]

For the mod_perl apps I have worked on, we do something very similar.  We have a few apache config templates, the main one and separate ones for all of the vhosts (i.e. tenants); these files are set up with macro replacements in place:

   ex.  <VirtualHost  macro_APP_HOST:macro_APACHE_HTTP_PORT>

And we have what we call "rules" files, that contain the macro substitutions for these:

   ex.    s|macro_APP_HOST|dev-03.yourcompany.com|g
          s|macro_APACHE_HTTP_PORT|8010|g

The rules are kept is separate files, one file per developer, one file for each QA environment, and one file for production use.  They are just `sed` substitutions.  We just run the rules file once per deployment pretty much, and everything is set up for that environment.  For staging and production environments, we have n hosts in our server pools so we run a 2nd substitution command and the end of the rules, to make sure we have the right physical host name where necessary.

We check in all conf and rules files, plus the scripts/tools used to do the macro substitutions into source control.  We have aliases set up to run these rule replacements, and then start up apache, etc.  We use file naming conventions like:

   _httpd.conf
   _somevhost.httpd.conf  (Included via the main httpd.conf)
   ...
   username.somedevhost.rules

When the substitution script runs, it will spit out the files:

   http.conf
   somevhost.httpd.conf
   ...

We found this to be the most straight-forward and simple approach for our needs.

Hope this helps,
- Jeff




----- Original Message ----
From: Mark Stosberg <mark@...>
To: modperl@...
Sent: Fri, November 6, 2009 10:30:23 AM
Subject: Best Practices for managing repetitive Apache config file entries?


I'm working on a mod_perl project where we would like to implement to
Apache related goals:

1. Automate repetitive configurations. For developers, we repeat the
same configs with the only difference being the "~user" location. In
other cases, the differences are not far from s/alpha/beta.

2. Put the Apache configurations under source control. Here the goal is
to tie changes to the Apache configs to other code changes in the
project.

Here's my outline for now we might do this. Are there other options
that work well for you?


1. We would be a template of Apache configs unders source control.
2. When the template changes, we'll run a script to generate all the
sections. Apache would pull this in as an include file.

In the event a launch is rolled back, the template could reverted to an
older version, and the include-file could be regenerated from the older
version.

Sound reasonable? What else would you recommend?

    Mark



--
. . . . . . . . . . . . . . . . . . . . . . . . . . .
   Mark Stosberg            Principal Developer  
  mark@...     Summersault, LLC    
   765-939-9301 ext 202     database driven websites
. . . . . http://www.summersault.com/ . . . . . . . .

[David Nicol]

On Fri, Nov 6, 2009 at 12:30 PM, Mark Stosberg <mark@...> wrote:


> What else

another possibility would be to embed some kind of macro language in
apache config file parsing. Perhaps Apache could offer m4
preprocessing?

http://en.wikipedia.org/wiki/M4_%28computer_language%29

[Randal L. Schwartz]

>>>>> "David" == David Nicol <davidnicol@...> writes:

David> another possibility would be to embed some kind of macro language in
David> apache config file parsing. Perhaps Apache could offer m4
David> preprocessing?

Eww.  No.  M4 needs to die a timely death.

Template Toolkit is far closer to "sane" for this, and can be embedded
to be used at launch time, as others have already said.

--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - [phone]
<merlyn@...> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion

[Michael Peters]

On 11/06/2009 02:02 PM, Bill Moseley wrote:

> I generate httpd.conf files on the fly (i.e. when I start Apache) from
> Template toolkit and YAML files.

This is very similar to what we do. We have a script to start/stop
apache which will first generate the real config from the templated
config. We use HTML::Template and Config::ApacheFormat but the idea is
the same. And our templated files also live in svn.

--
Michael Peters
Plus Three, LP

On Fri, 6 Nov 2009 11:02:55 -0800
Bill Moseley <moseley@...> wrote:

> On Fri, Nov 6, 2009 at 10:30 AM, Mark Stosberg <mark@...> wrote:
>
> >
> > I'm working on a mod_perl project where we would like to implement to
> > Apache related goals:
> >
> > 1. Automate repetitive configurations. For developers, we repeat the
> > same configs with the only difference being the "~user" location. In
> > other cases, the differences are not far from s/alpha/beta.
> >
> > 2. Put the Apache configurations under source control. Here the goal is
> > to tie changes to the Apache configs to other code changes in the
> > project.
> >
>
> I generate httpd.conf files on the fly (i.e. when I start Apache) from
> Template toolkit and YAML files.
>
> The script merges the configurations from multilple YAML files -- i.e. form
> a main config.yaml file and then when, say, running on "staginge", from
> staging.yml.  Then it merges with a local file which allows individual
> developers to override settings in their home directory.  Then the
> apache-specific config is passed to a script that builds the httpd.conf
> files and starts (or stops) apache.

That's another interesting approach to consider.

I should also mention we use several of the Apache configs under CGI as
part of our development model, so our solution won't be
modperl-specific. (And I appreciate this approach is not
modperl-specific).

   Mark

--
 . . . . . . . . . . . . . . . . . . . . . . . . . . .
   Mark Stosberg            Principal Developer  
   mark@...     Summersault, LLC    
   765-939-9301 ext 202     database driven websites
 . . . . . http://www.summersault.com/ . . . . . . . .