Re: [Bacula-devel] Encryption Status

[Dan Langille]

resent, this time to bacula-users, the list upon which this thread
started.

On 5 Feb 2006 at 18:33, Landon Fuller wrote:

> In the spirit of status reports -- Bacula's File Daemon now has complete
> support for signing and encryption data prior to sending it to the
> Storage Daemon, and decrypting said data upon receipt from the Storage
> Daemon.

Now that's pretty cool!

How does it work?  Just simple public key encryption type thing?

> There is a small memory leak I need to track down, and some remaining
> bits and pieces to implement, but I wanted to fill everyone in on the
> current status before another weekend slipped by.

That's always the case... pretty hard to track down too I bet.

> Additionally, I'd like to thank Robert Moreda and Allenta Consulting of
> Spain for their donation of $180 to the EFF. This donation brings the
> total donations for the Bacula Encryption Project to my initial goal $3,000.

Good!

I've been using the TLS feature for a few weeks now.  I'm pretty
happy with it.  It's been working every time withojut fail.   No more
stunnel for me.

--
Dan Langille : Software Developer looking for work
my resume: http://www.freebsddiary.org/dan_langille.php




-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Bacula-users mailing list
Bacula-users@...
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Landon Fuller]

Dan Langille wrote:

> On 5 Feb 2006 at 18:33, Landon Fuller wrote:
>
>
>>In the spirit of status reports -- Bacula's File Daemon now has complete
>>support for signing and encryption data prior to sending it to the
>>Storage Daemon, and decrypting said data upon receipt from the Storage
>>Daemon.
>
>
> Now that's pretty cool!

Thanks!

> How does it work?  Just simple public key encryption type thing?

Right - a session key is randomly generated for each backup job, and
that session key is then encrypted using "recipients'" public keys. By
specifying multiple public keys, one can specify multiple recipients
that may decrypt the backup.

The downside to the current implementation is that a copy of the
encrypted session keys is saved with *every file*. The size of this data
structure isn't huge, but it does add up. If I recall correctly, with a
single 2048 bit key the on disk structure was ~280 bytes. For 100,000
files, that adds up to 26 megabytes.

I'm not sure how much of an issue this is for potential users of FD-side
data encryption. The upside is that the backups are much more resilient
to tape/disk corruption.

As far as generating the keys, I'm just using self-signed PEM encoded
x509 certificates and private keys. They be generated with a couple
openssl(1) one-liners.

One other issue worth raising -- The director can currently overwrite
any file on the FD, including the encryption keys or the FD
configuration file, thus exposing private data to the director.

I thought that we could solve this by either:
    1) Provide a "Allow Restore" flag that allows one to disable
restoration until it's actually needed, or
    2) Provide a "Restore Root" directive that allows the specification
of a restoration root under which all restored files must live. I prefer
this option, but 1) is certainly easier.

Thoughts?

>>There is a small memory leak I need to track down, and some remaining
>>bits and pieces to implement, but I wanted to fill everyone in on the
>>current status before another weekend slipped by.
>
>
> That's always the case... pretty hard to track down too I bet.

I really do love valgrind ... =)

> I've been using the TLS feature for a few weeks now.  I'm pretty
> happy with it.  It's been working every time withojut fail.   No more
> stunnel for me.

Glad to hear it. We've been using the TLS implementation ever since the
  tail-end of the 1.37 release cycle, zero problems. One feature I'd
really like to see is the implementation of SSH-style public key
validation -- caching the public key on first connect and then
validating against the cache. It'd be a nice alternative to maintaining
x509 infrastructure. I don't know when I'll have time to tackle the
problem, but if anyone is interested in the project I'd be happy to
provide some pointers. It's not too hard to do ...

-landonf

[Landon Fuller]

Landon Fuller wrote:
> One other issue worth raising -- The director can currently overwrite
> any file on the FD, including the encryption keys or the FD
> configuration file, thus exposing private data to the director.

Something else I forgot to mention; the file daemon also ensures data
integrity by signing each file. Currently, only file data is signed --
permissions, ownership, et al are not.

AFAIK, during a restore, the storage daemon will provide the stream data
in the same order it was written by the file daemon. If that's true, the
easiest way to add extra file attributes/streams to the signature is to
checksum them as we send them to (and receive them from) the storage daemon.

Kern, is it reasonable to assume that the Storage Daemon will always
provide per-file stream data in the order it was written by the File
Daemon? If not, I'd guess the alternative is to cache the file
attributes on restore and checksum them in the standard order.

Thanks,
-landonf

[Dan Langille]

On 12 Feb 2006 at 13:49, Landon Fuller wrote:

> Dan Langille wrote:
> > On 5 Feb 2006 at 18:33, Landon Fuller wrote:
> >
> >
> >>In the spirit of status reports -- Bacula's File Daemon now has complete
> >>support for signing and encryption data prior to sending it to the
> >>Storage Daemon, and decrypting said data upon receipt from the Storage
> >>Daemon.
> >
> >
> > Now that's pretty cool!
>
> Thanks!
>
> > How does it work?  Just simple public key encryption type thing?
>
> Right - a session key is randomly generated for each backup job, and
> that session key is then encrypted using "recipients'" public keys. By
> specifying multiple public keys, one can specify multiple recipients
> that may decrypt the backup.

Multiple decryption keys is a nice feature.

> The downside to the current implementation is that a copy of the
> encrypted session keys is saved with *every file*. The size of this data
> structure isn't huge, but it does add up. If I recall correctly, with a
> single 2048 bit key the on disk structure was ~280 bytes. For 100,000
> files, that adds up to 26 megabytes.

100,000 files is a big backup anyway.  So another 26MB isn't much.  
I'm backing up to DLT 7000...  26MB is less than 1% of a tape.

> I'm not sure how much of an issue this is for potential users of FD-side
> data encryption. The upside is that the backups are much more resilient
> to tape/disk corruption.

I think it's pretty minimal.  The first time they get a bad spot on a
tape, they'll be glad to have it.

> As far as generating the keys, I'm just using self-signed PEM encoded
> x509 certificates and private keys. They be generated with a couple
> openssl(1) one-liners.
>
> One other issue worth raising -- The director can currently overwrite
> any file on the FD, including the encryption keys or the FD
> configuration file, thus exposing private data to the director.

Overwriting is not something I'd thought of.  Perhaps it's time to
come up with some FD-side restrictions:

Protected {
        NoAccess = /path/to/my/keys
}

This is a directory level directive.  The FD will neither read nor
write anything under that directory.  This cannot be overridden by
the Director.

> I thought that we could solve this by either:
>     1) Provide a "Allow Restore" flag that allows one to disable
> restoration until it's actually needed, or
>     2) Provide a "Restore Root" directive that allows the specification
> of a restoration root under which all restored files must live. I prefer
> this option, but 1) is certainly easier.

I like #1.  It allows a user to have confidence that files cannot be
updated without their knowledge.  #2 means that the FD specifies the
"Where" parameter that we see in the restore command.  The DIR would
not be able to override this parameter.

> > I've been using the TLS feature for a few weeks now.  I'm pretty
> > happy with it.  It's been working every time withojut fail.   No more
> > stunnel for me.
>
> Glad to hear it. We've been using the TLS implementation ever since the
>   tail-end of the 1.37 release cycle, zero problems. One feature I'd
> really like to see is the implementation of SSH-style public key
> validation -- caching the public key on first connect and then
> validating against the cache. It'd be a nice alternative to maintaining
> x509 infrastructure. I don't know when I'll have time to tackle the
> problem, but if anyone is interested in the project I'd be happy to
> provide some pointers. It's not too hard to do ...

SSH keys would be a nice option, I agree.  Many people are familiar
with SSH keys and it's one less thing for them to learn.


--
Dan Langille : Software Developer looking for work
my resume: http://www.freebsddiary.org/dan_langille.php




-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Bacula-users mailing list
Bacula-users@...
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Dan Langille]

On 12 Feb 2006 at 14:07, Landon Fuller wrote:

> Landon Fuller wrote:
> > One other issue worth raising -- The director can currently overwrite
> > any file on the FD, including the encryption keys or the FD
> > configuration file, thus exposing private data to the director.
>
> Something else I forgot to mention; the file daemon also ensures data
> integrity by signing each file. Currently, only file data is signed --
> permissions, ownership, et al are not.

Signing it is nice.  Does any signature verification occur?  e.g. the
SD as it receives the data during backup?  Before SD sends during
restore?  When the FD receives during restore?

> AFAIK, during a restore, the storage daemon will provide the stream
> data in the same order it was written by the file daemon. If that's
> true, the easiest way to add extra file attributes/streams to the
> signature is to checksum them as we send them to (and receive them
> from) the storage daemon.

After checksum'ing them, what would you do with them?

> Kern, is it reasonable to assume that the Storage Daemon will always
> provide per-file stream data in the order it was written by the File
> Daemon? If not, I'd guess the alternative is to cache the file
> attributes on restore and checksum them in the standard order.

What would happen if we received out-of-order packets?

--
Dan Langille : Software Developer looking for work
my resume: http://www.freebsddiary.org/dan_langille.php




-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Bacula-users mailing list
Bacula-users@...
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Phil Stracchino]

Dan Langille wrote:
> On 12 Feb 2006 at 14:07, Landon Fuller wrote:
>>Kern, is it reasonable to assume that the Storage Daemon will always
>>provide per-file stream data in the order it was written by the File
>>Daemon? If not, I'd guess the alternative is to cache the file
>>attributes on restore and checksum them in the standard order.
>
>
> What would happen if we received out-of-order packets?

One assumes this would be taken care of in the TCp stack, unless I'm
misunderstanding the question.

--
 Phil Stracchino       phil.stracchino@...
    Renaissance Man, Unix generalist, Perl hacker
 Mobile: [phone]         Landline: 603-886-3518


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Bacula-users mailing list
Bacula-users@...
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Dan Langille]

On 13 Feb 2006 at 9:02, Phil Stracchino wrote:

> Dan Langille wrote:
> > On 12 Feb 2006 at 14:07, Landon Fuller wrote:
> >>Kern, is it reasonable to assume that the Storage Daemon will always
> >>provide per-file stream data in the order it was written by the File
> >>Daemon? If not, I'd guess the alternative is to cache the file
> >>attributes on restore and checksum them in the standard order.
> >
> >
> > What would happen if we received out-of-order packets?
>
> One assumes this would be taken care of in the TCp stack, unless I'm
> misunderstanding the question.

That's the question I'm posing.  Does the TCP stack handle that, or
does Bacula?

--
Dan Langille : Software Developer looking for work
my resume: http://www.freebsddiary.org/dan_langille.php




-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Bacula-users mailing list
Bacula-users@...
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Dan Langille]

On 5 Feb 2006 at 18:33, Landon Fuller wrote:

> In the spirit of status reports -- Bacula's File Daemon now has complete
> support for signing and encryption data prior to sending it to the
> Storage Daemon, and decrypting said data upon receipt from the Storage
> Daemon.

That's only the Unix bacula-fd, right?

Similarly, the TLS code you wrote was also just for Unix?

I'm pretty sure the answer to both is yes.



--
Dan Langille : Software Developer looking for work
my resume: http://www.freebsddiary.org/dan_langille.php




-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Bacula-users mailing list
Bacula-users@...
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Landon Fuller]

On Feb 14, 2006, at 13:50, Dan Langille wrote:

> On 5 Feb 2006 at 18:33, Landon Fuller wrote:
>
>> In the spirit of status reports -- Bacula's File Daemon now has  
>> complete
>> support for signing and encryption data prior to sending it to the
>> Storage Daemon, and decrypting said data upon receipt from the  
>> Storage
>> Daemon.
>
> That's only the Unix bacula-fd, right?
>
> Similarly, the TLS code you wrote was also just for Unix?
>
> I'm pretty sure the answer to both is yes.

The only thing missing for win32 TLS/Encryption support is code to  
seed OpenSSL PRNG using Microsoft's CryptoAPI.

-landonf

[Kern Sibbald]

On Sunday 12 February 2006 23:07, Landon Fuller wrote:

> Landon Fuller wrote:
> > One other issue worth raising -- The director can currently overwrite
> > any file on the FD, including the encryption keys or the FD
> > configuration file, thus exposing private data to the director.
>
> Something else I forgot to mention; the file daemon also ensures data
> integrity by signing each file. Currently, only file data is signed --
> permissions, ownership, et al are not.
>
> AFAIK, during a restore, the storage daemon will provide the stream data
> in the same order it was written by the file daemon. If that's true, the
> easiest way to add extra file attributes/streams to the signature is to
> checksum them as we send them to (and receive them from) the storage
> daemon.
>
> Kern, is it reasonable to assume that the Storage Daemon will always
> provide per-file stream data in the order it was written by the File
> Daemon?

Yes, it is a strict requirement.

> If not, I'd guess the alternative is to cache the file
> attributes on restore and checksum them in the standard order.
>
> Thanks,
> -landonf


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
Bacula-users mailing list
Bacula-users@...
https://lists.sourceforge.net/lists/listinfo/bacula-users