Storing config values in-memory between sessions

[cottoncrucible]

Hi,

We run a script, server.cgi, running on a Apache2 instance using
mod_perl support for 'older' CGI scripts.

server.cgi is called by multiple individual clients at a time,
returning an XML.  It contains some config values that change from
time to time (once a week) e.g, campaign end date

As it seldom changes, we dont want to use a database for these 'config
values', yet need a way to retain in memory and dirty its' value from
time to time.

Confused slightly by the Apache phases and how it plays with older CGI
scripts, what would be the best way to implement this in-memory cache
that can be dirtied from time to time?

Regards,
Mahesh

[Alan Young]

2009/10/28 Mahesh Khambadkone <mahesh@...>:
> Confused slightly by the Apache phases and how it plays with older CGI
> scripts, what would be the best way to implement this in-memory cache
> that can be dirtied from time to time?

I would suggest using memcached.  It's a daemon that will hold your
data in memory and can be accessed in the way you are talking about.
--
Alan

[Torsten Foertsch]

On Thu 29 Oct 2009, Mahesh Khambadkone wrote:
> As it seldom changes, we dont want to use a database for these
> 'config values', yet need a way to retain in memory and dirty its'
> value from time to time.

Have a look at MMapDB which I have just uploaded to CPAN.

I wrote this module some time ago exactly for that case: configuration
data that seldom changes. Before I have used BerkeleyDB. But that is a
bit touchy when it comes to sudden process death. I wanted something
simple, stable and fast. So an MMapDB database cannot be modified by a
connected process because it is mapped read-only. Changing data means
rewriting the database file and reconnect the new one. This also means
the whole database is shared memory between all connected processes.

There is a tie() based interface. So, even dumping human readable data
is simple:

perl -MMMapDB -MData::Dumper -e '
  print Dumper(MMapDB->new(filename=>shift)->start->main_index);
' /etc/opt/TRANSCONFIG/transconfig.mmdb

And a more complicated but faster interface:

perl -MMMapDB -le '
  # open the DB
  my $db=MMapDB->new(filename=>shift)->start;
  # get data record positions
  my @pos=$db->index_lookup($db->mainidx, qw!actn opi /svn!);
  # fetch a data item
  print $db->data_record($pos[0])->[2];
' /etc/opt/TRANSCONFIG/transconfig.mmdb

One even can store large values there and pass references to them around
and thus avoid copying and mallocing that happens normally when you do
$x=$y:

perl -MMMapDB -MDevel::Peek -e '
  my $db=MMapDB->new(filename=>shift)->start;
  my @pos=$db->index_lookup($db->mainidx, qw!actn opi /svn!);
  # get a reference to the data item to avoid copying
  my $v=\$db->data_record($pos[0])->[2];
  print $$v, "\n";
  Dump $$v
' /etc/opt/TRANSCONFIG/transconfig.mmdb
File: '/opt/svnbook'.$MATCHED_PATH_INFO
SV = PV(0x9d4b50) at 0x7c9248
  REFCNT = 1
  FLAGS = (POK,READONLY,pPOK)
  PV = 0x7f956388d240 "File: '/opt/svnbook'.$MATCHED_PATH_INFO"
  CUR = 39
  LEN = 0

You see $v points to a read-only variable. The PV pointer of this
variable references the string inside the read-only mapped database
file. Now you can do $x=$v and only the reference is copied.

Torsten

--
Need professional mod_perl support?
Just hire me: torsten.foertsch@...

[Perrin Harkins-3]

On Thu, Oct 29, 2009 at 12:13 AM, Mahesh Khambadkone
<mahesh@...> wrote:
> Confused slightly by the Apache phases and how it plays with older CGI
> scripts, what would be the best way to implement this in-memory cache
> that can be dirtied from time to time?

You can't keep things in memory with CGI scripts because they are
spawned and exit on each request.  You'll need to cache things in a
database or file.  The dirt-simple solution is a file that you write
periodically from a cron job, and read from your CGI scripts.  If you
read it frequently, it will be kept in the filesystem cache so the
read will be extremely fast.

- Perrin