towards gnunet 0.9.x

View: New views
10 Messages — Rating Filter:   Alert me  

towards gnunet 0.9.x

by Christian Grothoff :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Dear all,

I've just created a new "branch" in the SVN repository
(https://gnunet.org/svn/gnunet/) which contains about 50 kLOC towards an
implementation of GNUnet using a radically new architectural redesign.  The
code there only has TCP support and no real applications, but basic
(encrypted) messaging between peers is working.  In other words, this is not
useful for anyone but developers and those interested in contributing to the
discussion as to how the code should evolve.  Note that development on the
existing 0.8.x-branch is expected to continue (albeit at a slower pace) for a
while.  Once we've sorted out the quirks in the new tree, moving existing
applications from 0.8.x to 0.9.x should not be too much work.

I want to briefly state the main advantages that the new architecture will
provide:

1) Fault isolation.  We're using more processes, literally replacing all
  threads, which will ensure that if one component crashes it doesn't take the
  whole system down -- and we can tell *which* component is responsible, as
  opposed to component A corrupting memory and the code crashing in component
  B. Also, 1000 lines of locking code can disappear.

2) Language independence.  Since plugins into the framework run as independent
  processes, it will be easier to contribute to GNUnet using languages other
  than C/C++.

3) We're also addressing various long-standing API and protocol issues,
   including more consistent naming conventions, better support for multi-
   homing and IPv6.

A more detailed rationale describing not only what changed but also why can be
found in the repository at https://gnunet.org/svn/gnunet/RATIONALE.  


Happy hacking!

Christian


_______________________________________________
GNUnet-developers mailing list
GNUnet-developers@...
http://lists.gnu.org/mailman/listinfo/gnunet-developers

Re: towards gnunet 0.9.x

by Milan Bouchet-Valat :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Wow! Never tired of rewriting GNUnet to get it master the world! ;-)
Reading the rationale, I see fantastic plans that really seem to allow
for a very robust and full-featured networking system.

Reading the analysis, a question has occurred to me : why didn't you
choose C++ instead of plain C? At least with namespaces and classes, it
would help making the different parts of the code cleaner. You know I
hate C, while that's the language I code the most in... ;-)
Also, using a general purpose library such as the GLib or Qt's fundation
classes could be useful. Just wondering, not asking you to change your
code now that it's done...


On the details part, I particularly liked that one:
> * With discovery being part of the transport service,
>   it is now also possible to "learn" our external
>   IP address from other peers (we just add plausible
>   addresses to the list; other peers will discard
>   those addresses that don't work for them!)
That was one of the utopic designs I had imagined to solve that issue.
Glad you chose it!

Now, I've noticed that GNUnet's UPnP support doesn't work for the two
router I've used, while some Internet applications do. One is Transmission
Bittorent, another is Pidgin Instant Messaging. They automatically open
the ports you need to make them work, and get your IP address. Looking at
Transmission's code [1], there are three files about UPnP and NAT-PMP, and
it's including a little lib called libnatpmp. It seems that these two
protocols are concurrents, and my modems are using the latter.

It shouldn't be too hard to get working in GNUnet, and that would provide
people with easy ports redirection, and of a "certified" external IP address,
else the general framework would get it from other peers. I could give a try
a this one of these days (weeks). Are there special thoughts about it, or
where it should go, etc.?


Cheers, and good luck!




_______________________________________________
GNUnet-developers mailing list
GNUnet-developers@...
http://lists.gnu.org/mailman/listinfo/gnunet-developers

Re: towards gnunet 0.9.x

by Christian Grothoff :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Friday 29 May 2009 03:28:30 am Milan Bouchet-Valat wrote:

> Wow! Never tired of rewriting GNUnet to get it master the world! ;-)
> Reading the rationale, I see fantastic plans that really seem to allow
> for a very robust and full-featured networking system.
>
> Reading the analysis, a question has occurred to me : why didn't you
> choose C++ instead of plain C? At least with namespaces and classes, it
> would help making the different parts of the code cleaner. You know I
> hate C, while that's the language I code the most in... ;-)
> Also, using a general purpose library such as the GLib or Qt's fundation
> classes could be useful. Just wondering, not asking you to change your
> code now that it's done...

I didn't think C++ would really help.  And using glib/qt as a foundation would
hugely increase the footprint of the background processes, which is not good
either.  Not to mention I did not feel like these libraries would really
contain much useful code: most of the code is networking-related at this
point.

> On the details part, I particularly liked that one:
> > * With discovery being part of the transport service,
> >   it is now also possible to "learn" our external
> >   IP address from other peers (we just add plausible
> >   addresses to the list; other peers will discard
> >   those addresses that don't work for them!)
>
> That was one of the utopic designs I had imagined to solve that issue.
> Glad you chose it!

:-).

> Now, I've noticed that GNUnet's UPnP support doesn't work for the two
> router I've used, while some Internet applications do. One is Transmission
> Bittorent, another is Pidgin Instant Messaging. They automatically open
> the ports you need to make them work, and get your IP address. Looking at
> Transmission's code [1], there are three files about UPnP and NAT-PMP, and
> it's including a little lib called libnatpmp. It seems that these two
> protocols are concurrents, and my modems are using the latter.

libnatpmp is free software, so the question is mostly if it is good enough to
be used directly.  I'd certainly like to see PMP and UPnP support in 0.9.x.  
IMO the first step would be to write a little library/API that could be used
by any transport to obtain an external IP/port and/or create an entry in the
port mapping done by the NAT box. Modifying the transports to add the
resulting address to their list should then be quite easy.  

The repository contains some highly preliminary incomplete code (largely
copied from the 0.8.x tree) for UPnP. If you want to work on making this nice,
that'd be great.

> It shouldn't be too hard to get working in GNUnet, and that would provide
> people with easy ports redirection, and of a "certified" external IP
> address, else the general framework would get it from other peers. I could
> give a try a this one of these days (weeks). Are there special thoughts
> about it, or where it should go, etc.?

It should probably go into src/upnp, or something like src/upnppmp or some
other name capturing both (src/nat?).  If the libraries that you write or use
can operate without ever blocking (which good network libs should allow), then
I think a simple shared wrapper library (ideally encapsulating both UPnP and
PMP support) would do.  

gnunet/src/include/gnunet_upnp_service.h contains my first draft at a simple
API that I'd like to see.  Maybe too simple, suggestions would be welcome. Oh,
and if the API is implemented as a simple library, the header should be called
"something_lib" in the end, not "something_service" (since we use the latter
for things implemented by an external process where the shared library is just
a convenience wrapper).

Best,

Christian


_______________________________________________
GNUnet-developers mailing list
GNUnet-developers@...
http://lists.gnu.org/mailman/listinfo/gnunet-developers

Re: towards gnunet 0.9.x

by Milan Bouchet-Valat :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Le vendredi 29 mai 2009 à 23:43 -0600, Christian Grothoff a écrit :

> > Now, I've noticed that GNUnet's UPnP support doesn't work for the two
> > router I've used, while some Internet applications do. One is Transmission
> > Bittorent, another is Pidgin Instant Messaging. They automatically open
> > the ports you need to make them work, and get your IP address. Looking at
> > Transmission's code [1], there are three files about UPnP and NAT-PMP, and
> > it's including a little lib called libnatpmp. It seems that these two
> > protocols are concurrents, and my modems are using the latter.
>
> libnatpmp is free software, so the question is mostly if it is good enough to
> be used directly.  I'd certainly like to see PMP and UPnP support in 0.9.x.  
> IMO the first step would be to write a little library/API that could be used
> by any transport to obtain an external IP/port and/or create an entry in the
> port mapping done by the NAT box. Modifying the transports to add the
> resulting address to their list should then be quite easy.
libnatpmp seems to be good to do what it is intended for, it would only need to
be wrapped I guess. Apparently the lib is not packaged, so project copy the few
files it provides. Not a big deal.

I don't think the transports should be modified to call the new API
directly, since the IP may be determined using config files
(static/DNS), or the new mechanism you described. I'd rather make the
core call the library trying to get the IP, and for every transport it
knows about, to open ports in the router. Now, I'm not familiar with the
networking code in GNUnet...

> The repository contains some highly preliminary incomplete code (largely
> copied from the 0.8.x tree) for UPnP. If you want to work on making this nice,
> that'd be great.
About that previous implementation: I guess it was working with 0.8.x,
so the UPnP-specific code can be kept more or less as-is internally?
Actually, I've checked again, and my boxes are UPnP, except that they
don't work with GNUnet's implementation ('make check' fails). So I think
I'll start the wrapper lib with UPnP, trying to get it working on my box
(that's what it's for, isn't it?) and then with NAT-PMP. Or maybe
getting the UPnP implementation used by Transmission would be better,
what do you think? This way, I could reuse their whole framework that
unifies both protocols.

> It should probably go into src/upnp, or something like src/upnppmp or some
> other name capturing both (src/nat?).  If the libraries that you write or use
> can operate without ever blocking (which good network libs should allow), then
> I think a simple shared wrapper library (ideally encapsulating both UPnP and
> PMP support) would do.
src/nat sounds good to me, if that's not too close to the NAT transport.
Regarding non-blocking calls, I'm not used to network and threads
coding, but I'll see what I can do. I guess that means running each call
in a separate thread?

> gnunet/src/include/gnunet_upnp_service.h contains my first draft at a simple
> API that I'd like to see.  Maybe too simple, suggestions would be welcome. Oh,
> and if the API is implemented as a simple library, the header should be called
> "something_lib" in the end, not "something_service" (since we use the latter
> for things implemented by an external process where the shared library is just
> a convenience wrapper).
I'd rather split the lib in different calls: one should get the IP,
another open a port. There the question of signals triggered when the
router advertises a IP address change, but that's for a second step.

I guess I'll have to ask you advice often... ;-)




_______________________________________________
GNUnet-developers mailing list
GNUnet-developers@...
http://lists.gnu.org/mailman/listinfo/gnunet-developers

Re: towards gnunet 0.9.x

by Christian Grothoff :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Saturday 30 May 2009 04:15:27 am Milan Bouchet-Valat wrote:

> Le vendredi 29 mai 2009 à 23:43 -0600, Christian Grothoff a écrit :
> > > Now, I've noticed that GNUnet's UPnP support doesn't work for the two
> > > router I've used, while some Internet applications do. One is
> > > Transmission Bittorent, another is Pidgin Instant Messaging. They
> > > automatically open the ports you need to make them work, and get your
> > > IP address. Looking at Transmission's code [1], there are three files
> > > about UPnP and NAT-PMP, and it's including a little lib called
> > > libnatpmp. It seems that these two protocols are concurrents, and my
> > > modems are using the latter.
> >
> > libnatpmp is free software, so the question is mostly if it is good
> > enough to be used directly.  I'd certainly like to see PMP and UPnP
> > support in 0.9.x. IMO the first step would be to write a little
> > library/API that could be used by any transport to obtain an external
> > IP/port and/or create an entry in the port mapping done by the NAT box.
> > Modifying the transports to add the resulting address to their list
> > should then be quite easy.
>
> libnatpmp seems to be good to do what it is intended for, it would only
> need to be wrapped I guess. Apparently the lib is not packaged, so project
> copy the few files it provides. Not a big deal.

Ok.

> I don't think the transports should be modified to call the new API
> directly, since the IP may be determined using config files
> (static/DNS), or the new mechanism you described. I'd rather make the
> core call the library trying to get the IP, and for every transport it
> knows about, to open ports in the router. Now, I'm not familiar with the
> networking code in GNUnet...

The core really knows nothing about IP addresses (or transports).  Each
transport plugin is responsible for providing and validating addresses --
nobody else even knows what the format of an address is, or what "port" is
being used (if at all).  So really the transport plugins are the ones that
will absolutely have to do this.  But it should be a tiny modification given
the right API.

> > The repository contains some highly preliminary incomplete code (largely
> > copied from the 0.8.x tree) for UPnP. If you want to work on making this
> > nice, that'd be great.
>
> About that previous implementation: I guess it was working with 0.8.x,
> so the UPnP-specific code can be kept more or less as-is internally?

Yes, except that the existing UPnP code will have to be rewritten to make use
of our new util libraries.  Should be a relatively minor change.

> Actually, I've checked again, and my boxes are UPnP, except that they
> don't work with GNUnet's implementation ('make check' fails). So I think
> I'll start the wrapper lib with UPnP, trying to get it working on my box
> (that's what it's for, isn't it?) and then with NAT-PMP.

Sounds good.  The code does work with my UPnP NAT box, but obviously making it
work with additional systems would be great.

> Or maybe
> getting the UPnP implementation used by Transmission would be better,
> what do you think? This way, I could reuse their whole framework that
> unifies both protocols.

I would not mind, the existing UPnP implementation was adapted from
pidgin/gaim and is not terribly nice to begin with.  So if Transmission has a
nice way to do it, I'd be happy to switch.

> > It should probably go into src/upnp, or something like src/upnppmp or
> > some other name capturing both (src/nat?).  If the libraries that you
> > write or use can operate without ever blocking (which good network libs
> > should allow), then I think a simple shared wrapper library (ideally
> > encapsulating both UPnP and PMP support) would do.
>
> src/nat sounds good to me, if that's not too close to the NAT transport.

The NAT transport is history (no longer required, thanks to the new way how
we're dealing with HELLOs).

> Regarding non-blocking calls, I'm not used to network and threads
> coding, but I'll see what I can do. I guess that means running each call
> in a separate thread?

No, it means not using any threads.  It means using a "select" loop.  Read the
src/include/gnunet_scheduler_lib.h API.  Essentially, instead of doing a
blocking read/write call, you ask the scheduler to call you once the socket is
ready to do the operation, and then you do it.  Rather easy.

> > gnunet/src/include/gnunet_upnp_service.h contains my first draft at a
> > simple API that I'd like to see.  Maybe too simple, suggestions would be
> > welcome. Oh, and if the API is implemented as a simple library, the
> > header should be called "something_lib" in the end, not
> > "something_service" (since we use the latter for things implemented by an
> > external process where the shared library is just a convenience wrapper).
>
> I'd rather split the lib in different calls: one should get the IP,
> another open a port. There the question of signals triggered when the
> router advertises a IP address change, but that's for a second step.

Splitting it like that is fine with me as well.

> I guess I'll have to ask you advice often... ;-)

That's fine, I'll try to hang out on IRC ;-).

Christian


_______________________________________________
GNUnet-developers mailing list
GNUnet-developers@...
http://lists.gnu.org/mailman/listinfo/gnunet-developers

Re: towards gnunet 0.9.x

by Milan Bouchet-Valat :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Le samedi 30 mai 2009 à 12:47 -0600, Christian Grothoff a écrit :
> The core really knows nothing about IP addresses (or transports).  Each
> transport plugin is responsible for providing and validating addresses --
> nobody else even knows what the format of an address is, or what "port" is
> being used (if at all).  So really the transport plugins are the ones that
> will absolutely have to do this.  But it should be a tiny modification given
> the right API.
No issue, except about the IP adress. Since it can be detected via
different means, we need some kind of centralization, i.e. a function
that will check for config files, then NAT, and finally use the new
framework for that. Should it be done in the nat library, or should it
just provide a function to be called by another lib?

> > Actually, I've checked again, and my boxes are UPnP, except that they
> > don't work with GNUnet's implementation ('make check' fails). So I think
> > I'll start the wrapper lib with UPnP, trying to get it working on my box
> > (that's what it's for, isn't it?) and then with NAT-PMP.
>
> Sounds good.  The code does work with my UPnP NAT box, but obviously making it
> work with additional systems would be great.
Latest update: 'make check' now reports my IP address, so something must
have improved in GNUnet or in my box in this regard. So I'll consider
what's the more logical/robust/clean solution, assuming the current
implementation works.

> > Regarding non-blocking calls, I'm not used to network and threads
> > coding, but I'll see what I can do. I guess that means running each call
> > in a separate thread?
>
> No, it means not using any threads.  It means using a "select" loop.  Read the
> src/include/gnunet_scheduler_lib.h API.  Essentially, instead of doing a
> blocking read/write call, you ask the scheduler to call you once the socket is
> ready to do the operation, and then you do it.  Rather easy.
Good, I'll look at that.

> > I guess I'll have to ask you advice often... ;-)
>
> That's fine, I'll try to hang out on IRC ;-).
Don't spend your nights on IRC for me, I think I'm going to work on this
sparsely (between exams and holidays!). I'm simply planning to follow
your schedule for 0.9.0... ;-)


Additionally, here are a few things I think could be good to keep in
mind when designing 0.9:

- We need a way for the daemon to send warning messages to the clients,
e.g. show a conspicuous message in gnunet-gtk about TCP transport
failing, library not found, etc. Logging is not usable for all cases and
is very misleading for new users that don't understand what fails. Maybe
GNUNET_ERROR already allows for this, though.

- It would be nice to allow downloads to go on on background, while
closing all clients. I think this should be performed in the daemon
itself - starting a new per-user process for that would not allow you to
close your session and come back, for example. While this feature is not
required to be present soon, keeping in the design the ability to add it
at some point would be nice

- And posting a news on the website to sum up what you said in your mail
would help people to keep in touch with the development. We've heard
many times that the sites looked like it was dead - giving details and
showing there's work going on will encourage users to persevere...

Cheers




_______________________________________________
GNUnet-developers mailing list
GNUnet-developers@...
http://lists.gnu.org/mailman/listinfo/gnunet-developers

Re: towards gnunet 0.9.x

by Christian Grothoff :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Sunday 31 May 2009 04:41:06 am Milan Bouchet-Valat wrote:

> Le samedi 30 mai 2009 à 12:47 -0600, Christian Grothoff a écrit :
> > The core really knows nothing about IP addresses (or transports).  Each
> > transport plugin is responsible for providing and validating addresses --
> > nobody else even knows what the format of an address is, or what "port"
> > is being used (if at all).  So really the transport plugins are the ones
> > that will absolutely have to do this.  But it should be a tiny
> > modification given the right API.
>
> No issue, except about the IP adress. Since it can be detected via
> different means, we need some kind of centralization, i.e. a function
> that will check for config files, then NAT, and finally use the new
> framework for that. Should it be done in the nat library, or should it
> just provide a function to be called by another lib?

No, we don't need that type of centralization: we now support having
*multiple* IP addresses.  This is required for proper IPv6 support anyway.  So
all that will really happen is that the UPnP/PMP library should allow us to
register a callback which will be called whenever there is an external IP
address to be added to (or removed from) our IP list.  I've thought about it a
bit more and here is what I'm thinking of for the API right now:

// add_remove: YES: add, NO: remove
// addr: see man 2 bind
void (*GNUNET_NAT_AddressCallback)(void *cls, int add_remove,
                                    const struct sockaddr *addr,
                                    socklen_t addrlen)

struct GNUNET_NAT_Handle;

struct GNUNET_NAT_Handle *
GNUNET_NAT_register (struct GNUNET_SCHEDULER_Handle *sched,
                                struct GNUNET_CONFIGURATION_Handle *cfg,
                               const struct sockaddr *addr, socklen_t addrlen,
                             GNUNET_NAT_AddressCallback callback,
                             void *callback_cls);
void
GNUNET_NAT_unregister (struct GNUNET_NAT_Handle *h);


Calling "register" is asking for the given IP address (which can be v4 or v6)
to be made available "more globally" (mapped by the NAT box).  If such a "more
global" address becomes available, the callback should be invoked with the
address other peers should bind to.  If the mapping changes, the callback
would be called with "NO" (on the old address) and then "YES" on the updated
address.  

Note that "register" could be called on a range of IP addresses (127.0.0.1,
 ::1, 192.168.1.42) and ports (2086, 1080, ...).  The NAT could would have to
decide if that address was somehow "mappable" (and do nothing if not) and for
those that are do the right thing.

> > > Actually, I've checked again, and my boxes are UPnP, except that they
> > > don't work with GNUnet's implementation ('make check' fails). So I
> > > think I'll start the wrapper lib with UPnP, trying to get it working on
> > > my box (that's what it's for, isn't it?) and then with NAT-PMP.
> >
> > Sounds good.  The code does work with my UPnP NAT box, but obviously
> > making it work with additional systems would be great.
>
> Latest update: 'make check' now reports my IP address, so something must
> have improved in GNUnet or in my box in this regard. So I'll consider
> what's the more logical/robust/clean solution, assuming the current
> implementation works.

Ok! Good to know.  I don't think anyone touched the UPnP code recently.

> > > Regarding non-blocking calls, I'm not used to network and threads
> > > coding, but I'll see what I can do. I guess that means running each
> > > call in a separate thread?
> >
> > No, it means not using any threads.  It means using a "select" loop.
> > Read the src/include/gnunet_scheduler_lib.h API.  Essentially, instead of
> > doing a blocking read/write call, you ask the scheduler to call you once
> > the socket is ready to do the operation, and then you do it.  Rather
> > easy.
>
> Good, I'll look at that.
>
> > > I guess I'll have to ask you advice often... ;-)
> >
> > That's fine, I'll try to hang out on IRC ;-).
>
> Don't spend your nights on IRC for me, I think I'm going to work on this
> sparsely (between exams and holidays!). I'm simply planning to follow
> your schedule for 0.9.0... ;-)
>
>
> Additionally, here are a few things I think could be good to keep in
> mind when designing 0.9:
>
> - We need a way for the daemon to send warning messages to the clients,
> e.g. show a conspicuous message in gnunet-gtk about TCP transport
> failing, library not found, etc. Logging is not usable for all cases and
> is very misleading for new users that don't understand what fails. Maybe
> GNUNET_ERROR already allows for this, though.

We have the "component" in the log call which tells us what part of the system
caused the problem. My current thinking here is that for GUIs we should have
an easy way for users to inspect the log file.  And of course, if error
messages are expected to happen, we should have some way in the API between
the two processes to pass the specific message.  But I'm not thinking of doing
that for any messages logged using GNUNET_log.

> - It would be nice to allow downloads to go on on background, while
> closing all clients. I think this should be performed in the daemon
> itself - starting a new per-user process for that would not allow you to
> close your session and come back, for example. While this feature is not
> required to be present soon, keeping in the design the ability to add it
> at some point would be nice

First of all, there won't be "the daemon itself".  And yes, I was thinking of
having a daemon for managing FS activities in the background.  However, I'm
not anywhere near touching that code yet.

> - And posting a news on the website to sum up what you said in your mail
> would help people to keep in touch with the development. We've heard
> many times that the sites looked like it was dead - giving details and
> showing there's work going on will encourage users to persevere...

Hmpf.  More work :-).  I'll see what I can find time to do...

Christian



_______________________________________________
GNUnet-developers mailing list
GNUnet-developers@...
http://lists.gnu.org/mailman/listinfo/gnunet-developers

Parent Message unknown Re: towards gnunet 0.9.x

by Christian Grothoff :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

I was having a similar problem the other day.  Downgrading (!) to SVN 1.5.1
helped.

Christian

On Sunday 07 June 2009 01:50:15 am Sony Mohanty wrote:

> Hi Christian
>
> I am not able to open in open the link https://gnunet.org/svn/gnunet/.
>
> Regards
> Sony
>
> --- On Fri, 5/29/09, Christian Grothoff <christian@...> wrote:
> > From: Christian Grothoff <christian@...>
> > Subject: [GNUnet-developers] towards gnunet 0.9.x
> > To: gnunet-developers@...
> > Date: Friday, May 29, 2009, 1:02 AM
> > Dear all,
> >
> > I've just created a new "branch" in the SVN repository
> > (https://gnunet.org/svn/gnunet/) which contains about 50
> > kLOC towards an
> > implementation of GNUnet using a radically new
> > architectural redesign.  The
> > code there only has TCP support and no real applications,
> > but basic
> > (encrypted) messaging between peers is working.  In
> > other words, this is not
> > useful for anyone but developers and those interested in
> > contributing to the
> > discussion as to how the code should evolve.  Note
> > that development on the
> > existing 0..8.x-branch is expected to continue (albeit at a
> > slower pace) for a
> > while.  Once we've sorted out the quirks in the new
> > tree, moving existing
> > applications from 0.8.x to 0.9.x should not be too much
> > work.
> >
> > I want to briefly state the main advantages that the new
> > architecture will
> > provide:
> >
> > 1) Fault isolation.  We're using more processes,
> > literally replacing all
> >   threads, which will ensure that if one component
> > crashes it doesn't take the
> >   whole system down -- and we can tell *which*
> > component is responsible, as
> >   opposed to component A corrupting memory and the
> > code crashing in component
> >   B. Also, 1000 lines of locking code can disappear.
> >
> > 2) Language independence.  Since plugins into the
> > framework run as independent
> >   processes, it will be easier to contribute to GNUnet
> > using languages other
> >   than C/C++.
> >
> > 3) We're also addressing various long-standing API and
> > protocol issues,
> >    including more consistent naming
> > conventions, better support for multi-
> >    homing and IPv6.
> >
> > A more detailed rationale describing not only what changed
> > but also why can be
> > found in the repository at https://gnunet.org/svn/gnunet/RATIONALE.
> >
> >
> > Happy hacking!
> >
> > Christian
> >
> >
> > _______________________________________________
> > GNUnet-developers mailing list
> > GNUnet-developers@...
> > http://lists.gnu.org/mailman/listinfo/gnunet-developers



_______________________________________________
GNUnet-developers mailing list
GNUnet-developers@...
http://lists.gnu.org/mailman/listinfo/gnunet-developers

NAT library (Was: towards gnunet 0.9.x)

by Milan Bouchet-Valat :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi!

I've worked a little on the NAT front, and I've come to something that
basically builds. Now, I need to rework most of the generic code around
UPnP and NAT-PMP actual implementations, since Transmission was working
on a very different basis than ours. But there are a few things that I
don't really understand (see below).


Le dimanche 31 mai 2009 à 12:31 -0600, Christian Grothoff a écrit :

> No, we don't need that type of centralization: we now support having
> *multiple* IP addresses.  This is required for proper IPv6 support anyway.  So
> all that will really happen is that the UPnP/PMP library should allow us to
> register a callback which will be called whenever there is an external IP
> address to be added to (or removed from) our IP list.  I've thought about it a
> bit more and here is what I'm thinking of for the API right now:
>
> // add_remove: YES: add, NO: remove
> // addr: see man 2 bind
> void (*GNUNET_NAT_AddressCallback)(void *cls, int add_remove,
>                                     const struct sockaddr *addr,
>                                     socklen_t addrlen)
>
> struct GNUNET_NAT_Handle;
>
> struct GNUNET_NAT_Handle *
> GNUNET_NAT_register (struct GNUNET_SCHEDULER_Handle *sched,
>                                 struct GNUNET_CONFIGURATION_Handle *cfg,
>                                const struct sockaddr *addr, socklen_t addrlen,
>                              GNUNET_NAT_AddressCallback callback,
>                              void *callback_cls);
> void
> GNUNET_NAT_unregister (struct GNUNET_NAT_Handle *h);
>
>
> Calling "register" is asking for the given IP address (which can be v4 or v6)
> to be made available "more globally" (mapped by the NAT box).  If such a "more
> global" address becomes available, the callback should be invoked with the
> address other peers should bind to.  If the mapping changes, the callback
> would be called with "NO" (on the old address) and then "YES" on the updated
> address.
>From what I know of UPnP and NAT-PMP, I understand that you cannot
choose the addresses (nor internal nor external) for which the
redirection is performed: the internal address we used to send the
request will be used for internal redirection, and the external address
is chosen by the NAT box. So we can't take a sockaddr, but merely a port
number. Moreover, isn't NAT kind of obsolete on IPv6? My knowledge in
this realm is very limited, though.


> Note that "register" could be called on a range of IP addresses (127.0.0.1,
>  ::1, 192.168.1.42) and ports (2086, 1080, ...).  The NAT could would have to
> decide if that address was somehow "mappable" (and do nothing if not) and for
> those that are do the right thing.
You mean called several times, once for each address/port? That means we
need a GNUNET_NAT_Handler_Init() function, and then pass the struct to
register(). But I can't see how we could pass addresses, as I said
above.

Other than that, I could not find the cron jobs API that was in GNUnet
0.8. I'd need some kind of timers to regularly check that the NAT is
still working. I'd also need a basic list support if we want to use
several ports, though I can use a custom singly-linked list if nothing
exists.


Cheers




_______________________________________________
GNUnet-developers mailing list
GNUnet-developers@...
http://lists.gnu.org/mailman/listinfo/gnunet-developers

Re: NAT library (Was: towards gnunet 0.9.x)

by Christian Grothoff :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Thursday 10 September 2009 19:22:26 Milan Bouchet-Valat wrote:
> Hi!
>
> I've worked a little on the NAT front, and I've come to something that
> basically builds. Now, I need to rework most of the generic code around
> UPnP and NAT-PMP actual implementations, since Transmission was working
> on a very different basis than ours. But there are a few things that I
> don't really understand (see below).

I'll try to answer below.
 

> Le dimanche 31 mai 2009 à 12:31 -0600, Christian Grothoff a écrit :
> > No, we don't need that type of centralization: we now support having
> > *multiple* IP addresses.  This is required for proper IPv6 support
> > anyway.  So all that will really happen is that the UPnP/PMP library
> > should allow us to register a callback which will be called whenever
> > there is an external IP address to be added to (or removed from) our IP
> > list.  I've thought about it a bit more and here is what I'm thinking of
> > for the API right now:
> >
> > // add_remove: YES: add, NO: remove
> > // addr: see man 2 bind
> > void (*GNUNET_NAT_AddressCallback)(void *cls, int add_remove,
> >                                     const struct sockaddr *addr,
> >                                     socklen_t addrlen)
> >
> > struct GNUNET_NAT_Handle;
> >
> > struct GNUNET_NAT_Handle *
> > GNUNET_NAT_register (struct GNUNET_SCHEDULER_Handle *sched,
> >                                 struct GNUNET_CONFIGURATION_Handle *cfg,
> >                                const struct sockaddr *addr, socklen_t
> > addrlen, GNUNET_NAT_AddressCallback callback, void *callback_cls);
> > void
> > GNUNET_NAT_unregister (struct GNUNET_NAT_Handle *h);
> >
> >
> > Calling "register" is asking for the given IP address (which can be v4 or
> > v6) to be made available "more globally" (mapped by the NAT box).  If
> > such a "more global" address becomes available, the callback should be
> > invoked with the address other peers should bind to.  If the mapping
> > changes, the callback would be called with "NO" (on the old address) and
> > then "YES" on the updated address.
>
> From what I know of UPnP and NAT-PMP, I understand that you cannot
> choose the addresses (nor internal nor external) for which the
> redirection is performed: the internal address we used to send the
> request will be used for internal redirection, and the external address
> is chosen by the NAT box. So we can't take a sockaddr, but merely a port
> number. Moreover, isn't NAT kind of obsolete on IPv6? My knowledge in
> this realm is very limited, though.

Yes, you are right in that the NAT box has little choice as far as the
internal address goes.  However, imagine you have a PC with two Ethernet
cards, both connected to a NAT box:

                                     PC
            eth0 (10.1.0.1)          eth1( 10.2.0.1)
                     |                                   |
          NAT (global: 1.2.3.4)       NAT (global: 5.6.7.8)

With the above API, you can specifically ask either NAT to create a port
mapping.  You may ask for "10.1.0.1:42" and are told "1.2.3.4:401", and then
you could ask for "10.2.0.1:55" and are told "5.6.7.8:421".  Naturally, in
most cases the system will only have one IP address (or the caller could leave
it at all-zeros for "unknown") and then the given struct sockaddr is really
effectively only a port number.  And as long as your implementation doesn't
support multiple network interfaces, you may just choose to only take the port
anyway.

As far as IPv6 goes, yes, sane people think that it should eliminate NAT but
there are people that try to combine NAT with IPv6.  So in case that insanity
prevails, our API would at least be prepared.  Not to mention that you may
have IPv6 between you and the NAT box and then have an IPv4 address from NAT
to the outside world (imagine your internal network moved entirely to IPv6,
not dual-stack, and your NAT box did 6over4 magic and stuff like that).  So
multiple scenarios are conceivable where IPv6 support can make sense.  

And of course an implementation (of my proposed API) is always allowed to just
return NULL to say "couldn't do that" / "not yet supported".

Final remark: we may need to add an additinal argument to specify a protocol
type (UDP, TCP, etc.) in the "GNUNET_NAT_register" call, I forgot about that
last time.
 

> > Note that "register" could be called on a range of IP addresses
> > (127.0.0.1,
> >
> >  ::1, 192.168.1.42) and ports (2086, 1080, ...).  The NAT could would
> >  :: have to
> >
> > decide if that address was somehow "mappable" (and do nothing if not) and
> > for those that are do the right thing.
>
> You mean called several times, once for each address/port? That means we
> need a GNUNET_NAT_Handler_Init() function, and then pass the struct to
> register(). But I can't see how we could pass addresses, as I said
> above.

Yes, the intend was that we would call it as often as needed (once for each
mapping).  I don't see why you would need some kind of "init" function; I'd
just do the full handshake with the NAT box (or PMP-box) for each mapping that
is being installed.  I don't think efficiency is a concern here to the point
that doing this once for each port (and keeping state for each mapping) would
be an issue.

> Other than that, I could not find the cron jobs API that was in GNUnet
> 0.8. I'd need some kind of timers to regularly check that the NAT is
> still working. I'd also need a basic list support if we want to use
> several ports, though I can use a custom singly-linked list if nothing
> exists.

Cron-ish stuff *is* there.  GNUNET_SCHEDULER_add_delayed replaces "cron". The
only major difference is that at the end of your "cron" function (or its
continuation) you have to (explicitly / manually) schedule your cron-task
*again* (each time).  

As for list support, you can obviously write your own custom singly-linked
lists in C easily.  However, I still think the answer here is to have one NAT
handle per port and to have each NAT handle have its own "cron" task with the
scheduler (also helps keep each individual task short).  Tasks with the
scheduler should be considered "cheap" -- they are not threads! -- but they
also are expected to run "quickly".

I hope this clarifies things.

Happy hacking!

Christian
--
http://grothoff.org/christian/


_______________________________________________
GNUnet-developers mailing list
GNUnet-developers@...
http://lists.gnu.org/mailman/listinfo/gnunet-developers