Resolving many ( > 200 ) items seems to lead to apparent congestion in the dbus

[Daniel Wynne]

Hi All!

Since we sometimes deal with more than 250 hosts, we often run into
timeouts from the Resolvers. For now, we immediately try again until all
Hosts are resolved. This can take up to 10 mins...

After restarting the application we end up resolving only about 50% of
the hosts. A restart of the dbus allows the application to find all
hosts again.

Is there a knwon issue about the dbus system that "free"-ed items remain
for an amount of time? Is there an easy workaround for this?
Do you think it would be good practice to establish an queue system to
avoid the resolver-timeouts?

Thanx again for the great support so far!

Daniel


_______________________________________________
avahi mailing list
avahi@...
http://lists.freedesktop.org/mailman/listinfo/avahi

[Lennart Poettering-8]

On Fri, 15.05.09 10:17, Daniel Wynne (daniel.wynne@...) wrote:

> Hi All!
>
> Since we sometimes deal with more than 250 hosts, we often run into
> timeouts from the Resolvers. For now, we immediately try again until all
> Hosts are resolved. This can take up to 10 mins...
>
> After restarting the application we end up resolving only about 50% of
> the hosts. A restart of the dbus allows the application to find all
> hosts again.
>
> Is there a knwon issue about the dbus system that "free"-ed items remain
> for an amount of time? Is there an easy workaround for this?
> Do you think it would be good practice to establish an queue system to
> avoid the resolver-timeouts?

How are you using this? Are you resolving 250 hosts one after the
other?

Avahi enforces limits at all places, i.e. on the cache, timeouts,
queries sent out and so on, to make sure we don't flood the network,
and that Avahi cannot be used for a DoS attack.

mDNS is intended to be used in smaller networks, up to 200 hosts or
something like that. I.e. small offices, homes. If you have larger
networks than that mDNS is not the protocol you should be using.

Lennart

--
Lennart Poettering                        Red Hat, Inc.
lennart [at] poettering [dot] net
http://0pointer.net/lennart/           GnuPG 0x1A015CC4
_______________________________________________
avahi mailing list
avahi@...
http://lists.freedesktop.org/mailman/listinfo/avahi

[Daniel Wynne]

Lennart Poettering schrieb:

> On Fri, 15.05.09 10:17, Daniel Wynne (daniel.wynne@...) wrote:
>
>  
>> Hi All!
>>
>> Since we sometimes deal with more than 250 hosts, we often run into
>> timeouts from the Resolvers. For now, we immediately try again until all
>> Hosts are resolved. This can take up to 10 mins...
>>
>> After restarting the application we end up resolving only about 50% of
>> the hosts. A restart of the dbus allows the application to find all
>> hosts again.
>>
>> Is there a knwon issue about the dbus system that "free"-ed items remain
>> for an amount of time? Is there an easy workaround for this?
>> Do you think it would be good practice to establish an queue system to
>> avoid the resolver-timeouts?
>>    
>
> How are you using this? Are you resolving 250 hosts one after the
> other?
>
> Avahi enforces limits at all places, i.e. on the cache, timeouts,
> queries sent out and so on, to make sure we don't flood the network,
> and that Avahi cannot be used for a DoS attack.
>
> mDNS is intended to be used in smaller networks, up to 200 hosts or
> something like that. I.e. small offices, homes. If you have larger
> networks than that mDNS is not the protocol you should be using.
>
> Lennart
>
>  

Hi Lennart!

Though its not the standard case, we deal with more than 250 hosts for
testing purpose. We want to resolve all of them, either without spamming
the DBus-System or crash into any Avahi-limitation.
First, we tried resolving everything right on the way, but this lead to
apparent DBus-congestion, which could only be solved by restarting the
whole service. After that we tried a queued system which allows only a
few resolvers to be coexistent, but this did not lead to any perceptible
improvement. The DBus still seems to be congested after a short while.
Only a restart of the service solves the problem. Without deep insight I
would assume, that the DBus-objects created are not freed-up cleanly or
lets say immediately. They seem to remain and so the bus gets filled-up.
To give you a more detailed overview I explain to you what exactly happens:
- We browse the .local domain for HTTP and HTTPS services with two
coexistent browsers
- For every resolver we create, we create another one to receive
additional address records
- The first resolver is freed immediately after resolving, the second
after the last address has been resolved
- This works fins for about 100 Hosts, then it starts to stumble. The
other resolvers achieve timeouts. If a timeout occurs we trie to resolve
again.
- After about 1min we receive another circa 50 hosts before the next
stumbling
- Then about every 1-2 minutes we receive a variable amount of hosts
We also tried a queued algorithm with only a certain amount of
coexistent resolvers without a noticeable improvement.

As you have a now a litlle more detailed view on what we do, could you
please give us some hints on how to improve the use of Avahi?
Maybe its not an DBus but an Avahi issue?
What are the exact limits of Avahi concerning cache, timeouts queries,
etc...?
How does the free-mechanism work exactly, especially on the DBus-side?

Thanx again!

Daniel

_______________________________________________
avahi mailing list
avahi@...
http://lists.freedesktop.org/mailman/listinfo/avahi

[Daniel Wynne]

I would like to attach some Information to my previous posting.
Another issue is, that on startup we have to restart the avahi service.
Otherwise we startup with only about 5-6 found services.



On Tue, 2009-05-19 at 08:10 +0200, Daniel Wynne wrote:

> Lennart Poettering schrieb:
> > On Fri, 15.05.09 10:17, Daniel Wynne (daniel.wynne@...) wrote:
> >
> >  
> >> Hi All!
> >>
> >> Since we sometimes deal with more than 250 hosts, we often run into
> >> timeouts from the Resolvers. For now, we immediately try again until all
> >> Hosts are resolved. This can take up to 10 mins...
> >>
> >> After restarting the application we end up resolving only about 50% of
> >> the hosts. A restart of the dbus allows the application to find all
> >> hosts again.
> >>
> >> Is there a knwon issue about the dbus system that "free"-ed items remain
> >> for an amount of time? Is there an easy workaround for this?
> >> Do you think it would be good practice to establish an queue system to
> >> avoid the resolver-timeouts?
> >>    
> >
> > How are you using this? Are you resolving 250 hosts one after the
> > other?
> >
> > Avahi enforces limits at all places, i.e. on the cache, timeouts,
> > queries sent out and so on, to make sure we don't flood the network,
> > and that Avahi cannot be used for a DoS attack.
> >
> > mDNS is intended to be used in smaller networks, up to 200 hosts or
> > something like that. I.e. small offices, homes. If you have larger
> > networks than that mDNS is not the protocol you should be using.
> >
> > Lennart
> >
> >  
> Hi Lennart!
>
> Though its not the standard case, we deal with more than 250 hosts for
> testing purpose. We want to resolve all of them, either without spamming
> the DBus-System or crash into any Avahi-limitation.
> First, we tried resolving everything right on the way, but this lead to
> apparent DBus-congestion, which could only be solved by restarting the
> whole service. After that we tried a queued system which allows only a
> few resolvers to be coexistent, but this did not lead to any perceptible
> improvement. The DBus still seems to be congested after a short while.
> Only a restart of the service solves the problem. Without deep insight I
> would assume, that the DBus-objects created are not freed-up cleanly or
> lets say immediately. They seem to remain and so the bus gets filled-up.
> To give you a more detailed overview I explain to you what exactly happens:
> - We browse the .local domain for HTTP and HTTPS services with two
> coexistent browsers
> - For every resolver we create, we create another one to receive
> additional address records
> - The first resolver is freed immediately after resolving, the second
> after the last address has been resolved
> - This works fins for about 100 Hosts, then it starts to stumble. The
> other resolvers achieve timeouts. If a timeout occurs we trie to resolve
> again.
> - After about 1min we receive another circa 50 hosts before the next
> stumbling
> - Then about every 1-2 minutes we receive a variable amount of hosts
> We also tried a queued algorithm with only a certain amount of
> coexistent resolvers without a noticeable improvement.
>
> As you have a now a litlle more detailed view on what we do, could you
> please give us some hints on how to improve the use of Avahi?
> Maybe its not an DBus but an Avahi issue?
> What are the exact limits of Avahi concerning cache, timeouts queries,
> etc...?
> How does the free-mechanism work exactly, especially on the DBus-side?
>
> Thanx again!
>
> Daniel
>
> _______________________________________________
> avahi mailing list
> avahi@...
> http://lists.freedesktop.org/mailman/listinfo/avahi

_______________________________________________
avahi mailing list
avahi@...
http://lists.freedesktop.org/mailman/listinfo/avahi

[Lennart Poettering-8]

On Tue, 19.05.09 08:10, Daniel Wynne (daniel.wynne@...) wrote:

> Hi Lennart!
>
> Though its not the standard case, we deal with more than 250 hosts for
> testing purpose. We want to resolve all of them, either without spamming
> the DBus-System or crash into any Avahi-limitation.
> First, we tried resolving everything right on the way, but this lead to
> apparent DBus-congestion, which could only be solved by restarting the
> whole service. After that we tried a queued system which allows only a
> few resolvers to be coexistent, but this did not lead to any perceptible
> improvement. The DBus still seems to be congested after a short
> while.

D-Bus "congestion"? I don't think that exists. What exactly makes you
think D-Bus could be 'congested'?

> Only a restart of the service solves the problem. Without deep insight I
> would assume, that the DBus-objects created are not freed-up cleanly or
> lets say immediately. They seem to remain and so the bus gets filled-up.
> To give you a more detailed overview I explain to you what exactly happens:
> - We browse the .local domain for HTTP and HTTPS services with two
> coexistent browsers
> - For every resolver we create, we create another one to receive
> additional address records
> - The first resolver is freed immediately after resolving, the second
> after the last address has been resolved
> - This works fins for about 100 Hosts, then it starts to stumble. The
> other resolvers achieve timeouts. If a timeout occurs we trie to resolve
> again.
> - After about 1min we receive another circa 50 hosts before the next
> stumbling
> - Then about every 1-2 minutes we receive a variable amount of hosts
> We also tried a queued algorithm with only a certain amount of
> coexistent resolvers without a noticeable improvement.

This might have to do something with the internal limits Avahi applies
on almost everything: in this case possibly the size of the chache?

Also note that if you issue a lot of requests the local IP stack
packet queueing might already drop packets. Lost packets will most
likely result in timeouts.

> As you have a now a litlle more detailed view on what we do, could you
> please give us some hints on how to improve the use of Avahi?

Dunno. This really depends on the problem. I don't have such a setup
here, so I cannot debug this myself.

If you get timeouts then either the query or the response packets got
lost. Try wireshark to find that out.

> Maybe its not an DBus but an Avahi issue?

mDNS/DNS-SD is not a reliable protocol. It wasn't designed to be
one. Don't make assumptions on its reliabilty that you shouldn't make.

> What are the exact limits of Avahi concerning cache, timeouts queries,
> etc...?

The max cache size is controlled via AVAHI_CACHE_ENTRIES_MAX in
avahi-core/cache. It is set to 500. Given the number of hosts this
might actually be way to low for your use case. Try to increase it to
5000.

> How does the free-mechanism work exactly, especially on the
> DBus-side?

Resolvers/browsers are ref counted by the D-Bus clients. I.e. if two
clients issue the same queries they end up with the same internal
browser/resolver. Also, a browser/resolver stub will be kept around
after the browser resolver died until the next time we'd have to
reissue a request for it in the hope we might be able to reuse it
later on for another client. That allows us to resuse the same
browser/resolver for repeated queries of the same data.

Lennart

--
Lennart Poettering                        Red Hat, Inc.
lennart [at] poettering [dot] net
http://0pointer.net/lennart/           GnuPG 0x1A015CC4
_______________________________________________
avahi mailing list
avahi@...
http://lists.freedesktop.org/mailman/listinfo/avahi

[Daniel Wynne]

On Wed, 2009-05-27 at 04:12 +0200, Lennart Poettering wrote:

> On Tue, 19.05.09 08:10, Daniel Wynne (daniel.wynne@...) wrote:
>
> > Hi Lennart!
> >
> > Though its not the standard case, we deal with more than 250 hosts for
> > testing purpose. We want to resolve all of them, either without spamming
> > the DBus-System or crash into any Avahi-limitation.
> > First, we tried resolving everything right on the way, but this lead to
> > apparent DBus-congestion, which could only be solved by restarting the
> > whole service. After that we tried a queued system which allows only a
> > few resolvers to be coexistent, but this did not lead to any perceptible
> > improvement. The DBus still seems to be congested after a short
> > while.
>
> D-Bus "congestion"? I don't think that exists. What exactly makes you
> think D-Bus could be 'congested'?

So this was a misunderstanding, sorry. In a previous thread somebody
mentioned that every D-Bus client can have at maximum 250 objects. I
assumed that every resolver browser is a client's object. If this is
correct, the limitation is forced by Avahi and not the D-Bus system.

> > Only a restart of the service solves the problem. Without deep insight I
> > would assume, that the DBus-objects created are not freed-up cleanly or
> > lets say immediately. They seem to remain and so the bus gets filled-up.
> > To give you a more detailed overview I explain to you what exactly happens:
> > - We browse the .local domain for HTTP and HTTPS services with two
> > coexistent browsers
> > - For every resolver we create, we create another one to receive
> > additional address records
> > - The first resolver is freed immediately after resolving, the second
> > after the last address has been resolved
> > - This works fins for about 100 Hosts, then it starts to stumble. The
> > other resolvers achieve timeouts. If a timeout occurs we trie to resolve
> > again.
> > - After about 1min we receive another circa 50 hosts before the next
> > stumbling
> > - Then about every 1-2 minutes we receive a variable amount of hosts
> > We also tried a queued algorithm with only a certain amount of
> > coexistent resolvers without a noticeable improvement.
>
> This might have to do something with the internal limits Avahi applies
> on almost everything: in this case possibly the size of the chache?
>
> Also note that if you issue a lot of requests the local IP stack
> packet queueing might already drop packets. Lost packets will most
> likely result in timeouts.

Is there an easy way to verify this? Could not find any proper logfile.

> > As you have a now a litlle more detailed view on what we do, could you
> > please give us some hints on how to improve the use of Avahi?
>
> Dunno. This really depends on the problem. I don't have such a setup
> here, so I cannot debug this myself.
>
> If you get timeouts then either the query or the response packets got
> lost. Try wireshark to find that out.
>
> > Maybe its not an DBus but an Avahi issue?
>
> mDNS/DNS-SD is not a reliable protocol. It wasn't designed to be
> one. Don't make assumptions on its reliabilty that you shouldn't make.
>
> > What are the exact limits of Avahi concerning cache, timeouts queries,
> > etc...?
>
> The max cache size is controlled via AVAHI_CACHE_ENTRIES_MAX in
> avahi-core/cache. It is set to 500. Given the number of hosts this
> might actually be way to low for your use case. Try to increase it to
> 5000.

This solution is not really applicable since it requires recompiling the sources.

> > How does the free-mechanism work exactly, especially on the
> > DBus-side?
>
> Resolvers/browsers are ref counted by the D-Bus clients. I.e. if two
> clients issue the same queries they end up with the same internal
> browser/resolver. Also, a browser/resolver stub will be kept around
> after the browser resolver died until the next time we'd have to
> reissue a request for it in the hope we might be able to reuse it
> later on for another client. That allows us to resuse the same
> browser/resolver for repeated queries of the same data.
>
> Lennart

So this means that every created browser resolver remains as a
D-Bus-Object and therefore might be responsible for the congestion
mentioned before.
Is there a way to cleanly free the browser resolvers immediately?

Thanx again and again ! :-)



_______________________________________________
avahi mailing list
avahi@...
http://lists.freedesktop.org/mailman/listinfo/avahi

[Lennart Poettering-8]

On Wed, 27.05.09 09:49, Daniel Wynne (daniel.wynne@...) wrote:

> > > Though its not the standard case, we deal with more than 250 hosts for
> > > testing purpose. We want to resolve all of them, either without spamming
> > > the DBus-System or crash into any Avahi-limitation.
> > > First, we tried resolving everything right on the way, but this lead to
> > > apparent DBus-congestion, which could only be solved by restarting the
> > > whole service. After that we tried a queued system which allows only a
> > > few resolvers to be coexistent, but this did not lead to any perceptible
> > > improvement. The DBus still seems to be congested after a short
> > > while.
> >
> > D-Bus "congestion"? I don't think that exists. What exactly makes you
> > think D-Bus could be 'congested'?
>
> So this was a misunderstanding, sorry. In a previous thread somebody
> mentioned that every D-Bus client can have at maximum 250 objects. I
> assumed that every resolver browser is a client's object. If this is
> correct, the limitation is forced by Avahi and not the D-Bus system.

Yes. That is true. For security reasons Avahi enforces limits on all
resources a local or remote client can allocate and control.

> > This might have to do something with the internal limits Avahi applies
> > on almost everything: in this case possibly the size of the chache?
> >
> > Also note that if you issue a lot of requests the local IP stack
> > packet queueing might already drop packets. Lost packets will most
> > likely result in timeouts.
>
> Is there an easy way to verify this? Could not find any proper
> logfile.

Edit avahi-core.c and the function avahi_cache_update(). Look for these
line:

if (c->n_entries >= AVAHI_CACHE_ENTRIES_MAX)
   return;

Then add a log line there that prints something when an entry is not
added to the cache because it reached its max size. i.e.

if (c->n_entries >= AVAHI_CACHE_ENTRIES_MAX) {
    avahi_log_debug("cache: overrun");
    return;
}

or something like that. Then run avahi-daemon --debug to get a peek on
the debug output.

> > The max cache size is controlled via AVAHI_CACHE_ENTRIES_MAX in
> > avahi-core/cache. It is set to 500. Given the number of hosts this
> > might actually be way to low for your use case. Try to increase it to
> > 5000.
>
> This solution is not really applicable since it requires recompiling
> the sources.

It's free software. Sources available.

> Is there a way to cleanly free the browser resolvers immediately?

No.

But this 'recycling' of resolves/browsers shouldn't hurt in your case.

Lennart

--
Lennart Poettering                        Red Hat, Inc.
lennart [at] poettering [dot] net
http://0pointer.net/lennart/           GnuPG 0x1A015CC4
_______________________________________________
avahi mailing list
avahi@...
http://lists.freedesktop.org/mailman/listinfo/avahi

[Daniel Wynne]

Lennart Poettering schrieb:

> On Wed, 27.05.09 09:49, Daniel Wynne (daniel.wynne@...) wrote:
>
>  
>>>> Though its not the standard case, we deal with more than 250 hosts for
>>>> testing purpose. We want to resolve all of them, either without spamming
>>>> the DBus-System or crash into any Avahi-limitation.
>>>> First, we tried resolving everything right on the way, but this lead to
>>>> apparent DBus-congestion, which could only be solved by restarting the
>>>> whole service. After that we tried a queued system which allows only a
>>>> few resolvers to be coexistent, but this did not lead to any perceptible
>>>> improvement. The DBus still seems to be congested after a short
>>>> while.
>>>>        
>>> D-Bus "congestion"? I don't think that exists. What exactly makes you
>>> think D-Bus could be 'congested'?
>>>      
>> So this was a misunderstanding, sorry. In a previous thread somebody
>> mentioned that every D-Bus client can have at maximum 250 objects. I
>> assumed that every resolver browser is a client's object. If this is
>> correct, the limitation is forced by Avahi and not the D-Bus system.
>>    
>
> Yes. That is true. For security reasons Avahi enforces limits on all
> resources a local or remote client can allocate and control.
>
>  
>>> This might have to do something with the internal limits Avahi applies
>>> on almost everything: in this case possibly the size of the chache?
>>>
>>> Also note that if you issue a lot of requests the local IP stack
>>> packet queueing might already drop packets. Lost packets will most
>>> likely result in timeouts.
>>>      
>> Is there an easy way to verify this? Could not find any proper
>> logfile.
>>    
>
> Edit avahi-core.c and the function avahi_cache_update(). Look for these
> line:
>
> if (c->n_entries >= AVAHI_CACHE_ENTRIES_MAX)
>    return;
>
> Then add a log line there that prints something when an entry is not
> added to the cache because it reached its max size. i.e.
>
> if (c->n_entries >= AVAHI_CACHE_ENTRIES_MAX) {
>     avahi_log_debug("cache: overrun");
>     return;
> }
>
> or something like that. Then run avahi-daemon --debug to get a peek on
> the debug output.
>
>  
>>> The max cache size is controlled via AVAHI_CACHE_ENTRIES_MAX in
>>> avahi-core/cache. It is set to 500. Given the number of hosts this
>>> might actually be way to low for your use case. Try to increase it to
>>> 5000.
>>>      
>> This solution is not really applicable since it requires recompiling
>> the sources.
>>    
>
> It's free software. Sources available.
>
>  

Since we do not want to recompile Avahi for every host using our
software, this solution is not applicable for us.

>> Is there a way to cleanly free the browser resolvers immediately?
>>    
>
> No.
>
> But this 'recycling' of resolves/browsers shouldn't hurt in your case.
>
>  
But I think thats exactly the problem in our case. In our Testnetwork
reside about 250 cameras we want to find and resolve via Avahi. So the
cache is way big enough. But if every resolver created is hold for
possible recycling and not deleted, the number of devices resolved is
limited to about 100 since we have to create two resolvers for each one
( see previous thread about multiple IP address resolving ). This fits
exactly to the behaviour we monitor. Seems like we are locked out with
the current Avahi version ? Is there a chance that some restrictions to
Avahi, implemented to avoid misuse through DoS attacks, will be loosen
due to usability issues? Many programmers and users have probs with
those as many previous threads show.
> Lennart
>
>  

_______________________________________________
avahi mailing list
avahi@...
http://lists.freedesktop.org/mailman/listinfo/avahi

[Lennart Poettering-8]

On Thu, 28.05.09 13:43, Daniel Wynne (daniel.wynne@...) wrote:

> But I think thats exactly the problem in our case. In our Testnetwork
> reside about 250 cameras we want to find and resolve via Avahi. So the
> cache is way big enough.

No. The limit of 500 entries in the cache that Avahi enforces is for
RRs, not services or hosts.

For each service you need (at least) one A, one PTR one SRV and one
TXT RR. Usually even more. I.e. if you register 250 services and want
to browser for them all at the same time you need as absolute minimum
space for 1000 entries, and practically even more.

Lennart

--
Lennart Poettering                        Red Hat, Inc.
lennart [at] poettering [dot] net
http://0pointer.net/lennart/           GnuPG 0x1A015CC4
_______________________________________________
avahi mailing list
avahi@...
http://lists.freedesktop.org/mailman/listinfo/avahi

[Daniel Wynne]

Hi Lennart!

After a short interrupt I am back in avahi-business ;-)

PLaying around a little bit with parameters I boiled it down to
AVAHI_CACHE_ENTRIES_MAX
OBJECTS_PER_CLIENT_MAX

These Parameters are slightly tight I think when it comes to bigger
networks i.e. more than 100 Services.

You said these limitations are implemented to avoid misuse, i.e.
DOS-Attacks.

Do you think it is possible in future releases of Avahi to loosen these
limitations a little bit, to enable more comprehensive usage of Avahi,
even for professional applications like ours ;-)
DOS Attacks are way above the limitations of 500, even way above 5000. I
think when talking about DOS we are in millions and above.

What do you say?

Kind Regards

Daniel
 


On Sun, 2009-06-07 at 01:47 +0200, Lennart Poettering wrote:

> On Thu, 28.05.09 13:43, Daniel Wynne (daniel.wynne@...) wrote:
>
> > But I think thats exactly the problem in our case. In our Testnetwork
> > reside about 250 cameras we want to find and resolve via Avahi. So the
> > cache is way big enough.
>
> No. The limit of 500 entries in the cache that Avahi enforces is for
> RRs, not services or hosts.
>
> For each service you need (at least) one A, one PTR one SRV and one
> TXT RR. Usually even more. I.e. if you register 250 services and want
> to browser for them all at the same time you need as absolute minimum
> space for 1000 entries, and practically even more.
>
> Lennart
>

_______________________________________________
avahi mailing list
avahi@...
http://lists.freedesktop.org/mailman/listinfo/avahi

[Trent Lloyd]

FWIW, I've always felt this was a little restrictive as well but never  
really ran into it in production.

I am personally keen on upping them a little - Daniel, can you tell us  
what values seem to work well for you?

Thanks,
Trent

On 27/06/2009, at 12:04 AM, Daniel Wynne wrote:

> Hi Lennart!
>
> After a short interrupt I am back in avahi-business ;-)
>
> PLaying around a little bit with parameters I boiled it down to
> AVAHI_CACHE_ENTRIES_MAX
> OBJECTS_PER_CLIENT_MAX
>
> These Parameters are slightly tight I think when it comes to bigger
> networks i.e. more than 100 Services.
>
> You said these limitations are implemented to avoid misuse, i.e.
> DOS-Attacks.
>
> Do you think it is possible in future releases of Avahi to loosen  
> these
> limitations a little bit, to enable more comprehensive usage of Avahi,
> even for professional applications like ours ;-)
> DOS Attacks are way above the limitations of 500, even way above  
> 5000. I
> think when talking about DOS we are in millions and above.
>
> What do you say?
>
> Kind Regards
>
> Daniel
>
>
>
> On Sun, 2009-06-07 at 01:47 +0200, Lennart Poettering wrote:
>> On Thu, 28.05.09 13:43, Daniel Wynne (daniel.wynne@...)  
>> wrote:
>>
>>> But I think thats exactly the problem in our case. In our  
>>> Testnetwork
>>> reside about 250 cameras we want to find and resolve via Avahi. So  
>>> the
>>> cache is way big enough.
>>
>> No. The limit of 500 entries in the cache that Avahi enforces is for
>> RRs, not services or hosts.
>>
>> For each service you need (at least) one A, one PTR one SRV and one
>> TXT RR. Usually even more. I.e. if you register 250 services and want
>> to browser for them all at the same time you need as absolute minimum
>> space for 1000 entries, and practically even more.
>>
>> Lennart
>>
>
> _______________________________________________
> avahi mailing list
> avahi@...
> http://lists.freedesktop.org/mailman/listinfo/avahi
>

_______________________________________________
avahi mailing list
avahi@...
http://lists.freedesktop.org/mailman/listinfo/avahi

[Daniel Wynne]

Hi Trent!

These are great news for us! :-)
We were seriously concerned about the issue, that the Avahi mDNS
implementation prevented us from developing Linux camera management
applications for huge setups.

In the following I provide a short calculation to give you a hint of
what we need:

    - Right now we have setups with deployments of up to 800 devices We
do not think that deployments with even more devices in one network make
sense, as this already can be considered as a individual feasibility
study. But lets calculate with this worst case scenario and for the sake
of convenience we even round it up to N = 1000 devices per network.

    - Since every device typically comes up with 2-3 ip addresses, we
have to reserve cache entries for N*( 3 A's, 1 PTR, 1 SRV, 1 TXT. ) =>
AVAHI_CACHE_ENTRIES_MAX >= 6*N = 6000 entries.

    - As the dbus limitations are concerned the maximum number of dbus
objects per client will have to be increased, too. Since we have to
create N*3 AddressRecordBrowsers, N*1 ServiceResolver, 1 Client, 2
ServiceBrowsers => OBJECTS_PER_CLIENT_MAX >= 4*N + 3 = 4003 ~ 4000 entries

As I do not have that much insight into the Avahi-code there might be
even more limits that might have to be adapted to achieve a coherent
configuration.

If you need any additional information, let me know.

Thank you Guys for the great support so far!

Kind Regards

Daniel


Trent Lloyd wrote:

> FWIW, I've always felt this was a little restrictive as well but never  
> really ran into it in production.
>
> I am personally keen on upping them a little - Daniel, can you tell us  
> what values seem to work well for you?
>
> Thanks,
> Trent
>
> On 27/06/2009, at 12:04 AM, Daniel Wynne wrote:
>
>  
>> Hi Lennart!
>>
>> After a short interrupt I am back in avahi-business ;-)
>>
>> PLaying around a little bit with parameters I boiled it down to
>> AVAHI_CACHE_ENTRIES_MAX
>> OBJECTS_PER_CLIENT_MAX
>>
>> These Parameters are slightly tight I think when it comes to bigger
>> networks i.e. more than 100 Services.
>>
>> You said these limitations are implemented to avoid misuse, i.e.
>> DOS-Attacks.
>>
>> Do you think it is possible in future releases of Avahi to loosen  
>> these
>> limitations a little bit, to enable more comprehensive usage of Avahi,
>> even for professional applications like ours ;-)
>> DOS Attacks are way above the limitations of 500, even way above  
>> 5000. I
>> think when talking about DOS we are in millions and above.
>>
>> What do you say?
>>
>> Kind Regards
>>
>> Daniel
>>
>>
>>
>> On Sun, 2009-06-07 at 01:47 +0200, Lennart Poettering wrote:
>>    
>>> On Thu, 28.05.09 13:43, Daniel Wynne (daniel.wynne@...)  
>>> wrote:
>>>
>>>      
>>>> But I think thats exactly the problem in our case. In our  
>>>> Testnetwork
>>>> reside about 250 cameras we want to find and resolve via Avahi. So  
>>>> the
>>>> cache is way big enough.
>>>>        
>>> No. The limit of 500 entries in the cache that Avahi enforces is for
>>> RRs, not services or hosts.
>>>
>>> For each service you need (at least) one A, one PTR one SRV and one
>>> TXT RR. Usually even more. I.e. if you register 250 services and want
>>> to browser for them all at the same time you need as absolute minimum
>>> space for 1000 entries, and practically even more.
>>>
>>> Lennart
>>>
>>>      
>> _______________________________________________
>> avahi mailing list
>> avahi@...
>> http://lists.freedesktop.org/mailman/listinfo/avahi
>>
>>    
>
> _______________________________________________
> avahi mailing list
> avahi@...
> http://lists.freedesktop.org/mailman/listinfo/avahi
>  

--
---------------------------------------------------------------------
Daniel Wynne, Softwareentwicklung
MOBOTIX AG - Security Vision Systems
Luxemburger Straße 6, 67657 Kaiserslautern
---------------------------------------------------------------------
daniel.wynne@..., http://www.mobotix.com
---------------------------------------------------------------------
Vorstandvorsitzender: Dr. Ralf Hinkel
Vorstandsmitglieder: Lutz Coelen, Horst Pfiffi
Aufsichtsratsvorsitzender: Dr. Thomas Hoch
Registergericht: HRB Kaiserslautern 3724  

_______________________________________________
avahi mailing list
avahi@...
http://lists.freedesktop.org/mailman/listinfo/avahi

[Lennart Poettering-8]

On Mon, 29.06.09 11:05, Daniel Wynne (daniel.wynne@...) wrote:

>
> Hi Trent!
>
> These are great news for us! :-)
> We were seriously concerned about the issue, that the Avahi mDNS
> implementation prevented us from developing Linux camera management
> applications for huge setups.
>
> In the following I provide a short calculation to give you a hint of
> what we need:
>
>     - Right now we have setups with deployments of up to 800 devices We
> do not think that deployments with even more devices in one network make
> sense, as this already can be considered as a individual feasibility
> study. But lets calculate with this worst case scenario and for the sake
> of convenience we even round it up to N = 1000 devices per network.
>
>     - Since every device typically comes up with 2-3 ip addresses, we
> have to reserve cache entries for N*( 3 A's, 1 PTR, 1 SRV, 1 TXT. ) =>
> AVAHI_CACHE_ENTRIES_MAX >= 6*N = 6000 entries.
>
>     - As the dbus limitations are concerned the maximum number of dbus
> objects per client will have to be increased, too. Since we have to
> create N*3 AddressRecordBrowsers, N*1 ServiceResolver, 1 Client, 2
> ServiceBrowsers => OBJECTS_PER_CLIENT_MAX >= 4*N + 3 = 4003 ~ 4000 entries
>
> As I do not have that much insight into the Avahi-code there might be
> even more limits that might have to be adapted to achieve a coherent
> configuration.
>
> If you need any additional information, let me know.
>
> Thank you Guys for the great support so far!

Hmm, with allowing more simulatenous clients the memory usage of the
cache will increase too. I wonder how bad that actually wuld
be. I.e. Daniel, how does the VSZ of the avahi daemon change when
connected to a en empty network vs. one of your fully equipped ones?

Lennart

--
Lennart Poettering                        Red Hat, Inc.
lennart [at] poettering [dot] net
http://0pointer.net/lennart/           GnuPG 0x1A015CC4
_______________________________________________
avahi mailing list
avahi@...
http://lists.freedesktop.org/mailman/listinfo/avahi

[Daniel Wynne]

Lennart Poettering wrote:

> On Mon, 29.06.09 11:05, Daniel Wynne (daniel.wynne@...) wrote:
>
>  
>> Hi Trent!
>>
>> These are great news for us! :-)
>> We were seriously concerned about the issue, that the Avahi mDNS
>> implementation prevented us from developing Linux camera management
>> applications for huge setups.
>>
>> In the following I provide a short calculation to give you a hint of
>> what we need:
>>
>>     - Right now we have setups with deployments of up to 800 devices We
>> do not think that deployments with even more devices in one network make
>> sense, as this already can be considered as a individual feasibility
>> study. But lets calculate with this worst case scenario and for the sake
>> of convenience we even round it up to N = 1000 devices per network.
>>
>>     - Since every device typically comes up with 2-3 ip addresses, we
>> have to reserve cache entries for N*( 3 A's, 1 PTR, 1 SRV, 1 TXT. ) =>
>> AVAHI_CACHE_ENTRIES_MAX >= 6*N = 6000 entries.
>>
>>     - As the dbus limitations are concerned the maximum number of dbus
>> objects per client will have to be increased, too. Since we have to
>> create N*3 AddressRecordBrowsers, N*1 ServiceResolver, 1 Client, 2
>> ServiceBrowsers => OBJECTS_PER_CLIENT_MAX >= 4*N + 3 = 4003 ~ 4000 entries
>>
>> As I do not have that much insight into the Avahi-code there might be
>> even more limits that might have to be adapted to achieve a coherent
>> configuration.
>>
>> If you need any additional information, let me know.
>>
>> Thank you Guys for the great support so far!
>>    
>
> Hmm, with allowing more simulatenous clients the memory usage of the
> cache will increase too. I wonder how bad that actually wuld
> be. I.e. Daniel, how does the VSZ of the avahi daemon change when
> connected to a en empty network vs. one of your fully equipped ones?
>
> Lennart
>
>  

Hi Lennart,

I could not test it with the 800 devices but at least with a network of
about 250 devices.
These are the results:

Original Parameters:

AVAHI_CACHE_ENTRIES_MAX = 500
OBJECTS_PER_CLIENT_MAX = 500

Changed Parameters:

AVAHI_CACHE_ENTRIES_MAX = 6000
OBJECTS_PER_CLIENT_MAX = 4000


Original Configuration, 2 Devices in the network
avahi-daemon uses 352 K

Original Configuration, 250 Devices in the network
avahi-daemon uses 1,820 K ( about 90 services were found at first, after
time number increases but never reaches the actual number of services in
the network, even after hours )

Changed Configuration, 250 Devices in the network
avahi-daemon uses 352 K

Changed Configuration, 250 Devices in the network
avahi-daemon uses 2,764 K ( as all services were found immediately )


Hope it helps! If you need more info let me know!


Daniel


_______________________________________________
avahi mailing list
avahi@...
http://lists.freedesktop.org/mailman/listinfo/avahi

[Daniel Wynne]

Daniel Wynne wrote:

> Lennart Poettering wrote:
>  
>> On Mon, 29.06.09 11:05, Daniel Wynne (daniel.wynne@...) wrote:
>>
>>  
>>    
>>> Hi Trent!
>>>
>>> These are great news for us! :-)
>>> We were seriously concerned about the issue, that the Avahi mDNS
>>> implementation prevented us from developing Linux camera management
>>> applications for huge setups.
>>>
>>> In the following I provide a short calculation to give you a hint of
>>> what we need:
>>>
>>>     - Right now we have setups with deployments of up to 800 devices We
>>> do not think that deployments with even more devices in one network make
>>> sense, as this already can be considered as a individual feasibility
>>> study. But lets calculate with this worst case scenario and for the sake
>>> of convenience we even round it up to N = 1000 devices per network.
>>>
>>>     - Since every device typically comes up with 2-3 ip addresses, we
>>> have to reserve cache entries for N*( 3 A's, 1 PTR, 1 SRV, 1 TXT. ) =>
>>> AVAHI_CACHE_ENTRIES_MAX >= 6*N = 6000 entries.
>>>
>>>     - As the dbus limitations are concerned the maximum number of dbus
>>> objects per client will have to be increased, too. Since we have to
>>> create N*3 AddressRecordBrowsers, N*1 ServiceResolver, 1 Client, 2
>>> ServiceBrowsers => OBJECTS_PER_CLIENT_MAX >= 4*N + 3 = 4003 ~ 4000 entries
>>>
>>> As I do not have that much insight into the Avahi-code there might be
>>> even more limits that might have to be adapted to achieve a coherent
>>> configuration.
>>>
>>> If you need any additional information, let me know.
>>>
>>> Thank you Guys for the great support so far!
>>>    
>>>      
>> Hmm, with allowing more simulatenous clients the memory usage of the
>> cache will increase too. I wonder how bad that actually wuld
>> be. I.e. Daniel, how does the VSZ of the avahi daemon change when
>> connected to a en empty network vs. one of your fully equipped ones?
>>
>> Lennart
>>
>>  
>>    
>
> Hi Lennart,
>
> I could not test it with the 800 devices but at least with a network of
> about 250 devices.
> These are the results:
>
> Original Parameters:
>
> AVAHI_CACHE_ENTRIES_MAX = 500
> OBJECTS_PER_CLIENT_MAX = 500
>
> Changed Parameters:
>
> AVAHI_CACHE_ENTRIES_MAX = 6000
> OBJECTS_PER_CLIENT_MAX = 4000
>
>
> Original Configuration, 2 Devices in the network
> avahi-daemon uses 352 K
>
> Original Configuration, 250 Devices in the network
> avahi-daemon uses 1,820 K ( about 90 services were found at first, after
> time number increases but never reaches the actual number of services in
> the network, even after hours )
>
> Changed Configuration, 250 Devices in the network
> avahi-daemon uses 352 K
>
> Changed Configuration, 250 Devices in the network
> avahi-daemon uses 2,764 K ( as all services were found immediately )
>
>
> Hope it helps! If you need more info let me know!
>
>
> Daniel
>
>
> _______________________________________________
> avahi mailing list
> avahi@...
> http://lists.freedesktop.org/mailman/listinfo/avahi
>  

BTW

You could also make the cache size configurable with a maximum allowed
size. This would help the one side without affecting the other...


--
---------------------------------------------------------------------
Daniel Wynne, Softwareentwicklung
MOBOTIX AG - Security Vision Systems
Luxemburger Straße 6, 67657 Kaiserslautern
---------------------------------------------------------------------
daniel.wynne@..., http://www.mobotix.com
---------------------------------------------------------------------
Vorstandvorsitzender: Dr. Ralf Hinkel
Vorstandsmitglieder: Lutz Coelen, Horst Pfiffi
Aufsichtsratsvorsitzender: Dr. Thomas Hoch
Registergericht: HRB Kaiserslautern 3724  

_______________________________________________
avahi mailing list
avahi@...
http://lists.freedesktop.org/mailman/listinfo/avahi

[Daniel Wynne]

Hi Folks,

just returning from my summer vacation I reflected about past projects.

Thereby I also wondered about our little conversation concerning Avahi
cache sizes.

Trent, you made those promising hints that you will loosen the strict
limitations to meet requirements of larger network implementations.

Could you, or any other responsible person please make a definite
authorative statement about this issue so we can proceed with development?
Me and my boss would be very appreciated about this ;-)

Kind Regards

Daniel


Daniel wrote:

> Hi Trent!
>
> These are great news for us! :-)
> We were seriously concerned about the issue, that the Avahi mDNS
> implementation prevented us from developing Linux camera management
> applications for huge setups.
>
> In the following I provide a short calculation to give you a hint of
> what we need:
>
>     - Right now we have setups with deployments of up to 800 devices We
> do not think that deployments with even more devices in one network make
> sense, as this already can be considered as a individual feasibility
> study. But lets calculate with this worst case scenario and for the sake
> of convenience we even round it up to N = 1000 devices per network.
>
>     - Since every device typically comes up with 2-3 ip addresses, we
> have to reserve cache entries for N*( 3 A's, 1 PTR, 1 SRV, 1 TXT. ) =>
> AVAHI_CACHE_ENTRIES_MAX >= 6*N = 6000 entries.
>
>     - As the dbus limitations are concerned the maximum number of dbus
> objects per client will have to be increased, too. Since we have to
> create N*3 AddressRecordBrowsers, N*1 ServiceResolver, 1 Client, 2
> ServiceBrowsers => OBJECTS_PER_CLIENT_MAX >= 4*N + 3 = 4003 ~ 4000 entries
>
> As I do not have that much insight into the Avahi-code there might be
> even more limits that might have to be adapted to achieve a coherent
> configuration.
>
> If you need any additional information, let me know.
>
> Thank you Guys for the great support so far!
>
> Kind Regards
>
> Daniel
>
>
> Trent Lloyd wrote:
>  
>> FWIW, I've always felt this was a little restrictive as well but never  
>> really ran into it in production.
>>
>> I am personally keen on upping them a little - Daniel, can you tell us  
>> what values seem to work well for you?
>>
>> Thanks,
>> Trent
>>
>> On 27/06/2009, at 12:04 AM, Daniel Wynne wrote:
>>
>>  
>>    
>>> Hi Lennart!
>>>
>>> After a short interrupt I am back in avahi-business ;-)
>>>
>>> PLaying around a little bit with parameters I boiled it down to
>>> AVAHI_CACHE_ENTRIES_MAX
>>> OBJECTS_PER_CLIENT_MAX
>>>
>>> These Parameters are slightly tight I think when it comes to bigger
>>> networks i.e. more than 100 Services.
>>>
>>> You said these limitations are implemented to avoid misuse, i.e.
>>> DOS-Attacks.
>>>
>>> Do you think it is possible in future releases of Avahi to loosen  
>>> these
>>> limitations a little bit, to enable more comprehensive usage of Avahi,
>>> even for professional applications like ours ;-)
>>> DOS Attacks are way above the limitations of 500, even way above  
>>> 5000. I
>>> think when talking about DOS we are in millions and above.
>>>
>>> What do you say?
>>>
>>> Kind Regards
>>>
>>> Daniel
>>>
>>>
>>>
>>> On Sun, 2009-06-07 at 01:47 +0200, Lennart Poettering wrote:
>>>    
>>>      
>>>> On Thu, 28.05.09 13:43, Daniel Wynne (daniel.wynne@...)  
>>>> wrote:
>>>>
>>>>      
>>>>        
>>>>> But I think thats exactly the problem in our case. In our  
>>>>> Testnetwork
>>>>> reside about 250 cameras we want to find and resolve via Avahi. So  
>>>>> the
>>>>> cache is way big enough.
>>>>>        
>>>>>          
>>>> No. The limit of 500 entries in the cache that Avahi enforces is for
>>>> RRs, not services or hosts.
>>>>
>>>> For each service you need (at least) one A, one PTR one SRV and one
>>>> TXT RR. Usually even more. I.e. if you register 250 services and want
>>>> to browser for them all at the same time you need as absolute minimum
>>>> space for 1000 entries, and practically even more.
>>>>
>>>> Lennart
>>>>
>>>>      
>>>>        
>>> _______________________________________________
>>> avahi mailing list
>>> avahi@...
>>> http://lists.freedesktop.org/mailman/listinfo/avahi
>>>
>>>    
>>>      
>> _______________________________________________
>> avahi mailing list
>> avahi@...
>> http://lists.freedesktop.org/mailman/listinfo/avahi
>>  
>>    
>
>
>  

_______________________________________________
avahi mailing list
avahi@...
http://lists.freedesktop.org/mailman/listinfo/avahi

[Daniel Wynne]

Hi again!

Is there anyone responsible available who is capable of answering my
previous question? Everybody on vacation? ;-)

Kind Regards

Daniel



Daniel wrote:

> Hi Folks,
>
> just returning from my summer vacation I reflected about past projects.
>
> Thereby I also wondered about our little conversation concerning Avahi
> cache sizes.
>
> Trent, you made those promising hints that you will loosen the strict
> limitations to meet requirements of larger network implementations.
>
> Could you, or any other responsible person please make a definite
> authorative statement about this issue so we can proceed with development?
> Me and my boss would be very appreciated about this ;-)
>
> Kind Regards
>
> Daniel
>
>
> Daniel wrote:
>  
>> Hi Trent!
>>
>> These are great news for us! :-)
>> We were seriously concerned about the issue, that the Avahi mDNS
>> implementation prevented us from developing Linux camera management
>> applications for huge setups.
>>
>> In the following I provide a short calculation to give you a hint of
>> what we need:
>>
>>     - Right now we have setups with deployments of up to 800 devices We
>> do not think that deployments with even more devices in one network make
>> sense, as this already can be considered as a individual feasibility
>> study. But lets calculate with this worst case scenario and for the sake
>> of convenience we even round it up to N = 1000 devices per network.
>>
>>     - Since every device typically comes up with 2-3 ip addresses, we
>> have to reserve cache entries for N*( 3 A's, 1 PTR, 1 SRV, 1 TXT. ) =>
>> AVAHI_CACHE_ENTRIES_MAX >= 6*N = 6000 entries.
>>
>>     - As the dbus limitations are concerned the maximum number of dbus
>> objects per client will have to be increased, too. Since we have to
>> create N*3 AddressRecordBrowsers, N*1 ServiceResolver, 1 Client, 2
>> ServiceBrowsers => OBJECTS_PER_CLIENT_MAX >= 4*N + 3 = 4003 ~ 4000 entries
>>
>> As I do not have that much insight into the Avahi-code there might be
>> even more limits that might have to be adapted to achieve a coherent
>> configuration.
>>
>> If you need any additional information, let me know.
>>
>> Thank you Guys for the great support so far!
>>
>> Kind Regards
>>
>> Daniel
>>
>>
>> Trent Lloyd wrote:
>>  
>>    
>>> FWIW, I've always felt this was a little restrictive as well but never  
>>> really ran into it in production.
>>>
>>> I am personally keen on upping them a little - Daniel, can you tell us  
>>> what values seem to work well for you?
>>>
>>> Thanks,
>>> Trent
>>>
>>> On 27/06/2009, at 12:04 AM, Daniel Wynne wrote:
>>>
>>>  
>>>    
>>>      
>>>> Hi Lennart!
>>>>
>>>> After a short interrupt I am back in avahi-business ;-)
>>>>
>>>> PLaying around a little bit with parameters I boiled it down to
>>>> AVAHI_CACHE_ENTRIES_MAX
>>>> OBJECTS_PER_CLIENT_MAX
>>>>
>>>> These Parameters are slightly tight I think when it comes to bigger
>>>> networks i.e. more than 100 Services.
>>>>
>>>> You said these limitations are implemented to avoid misuse, i.e.
>>>> DOS-Attacks.
>>>>
>>>> Do you think it is possible in future releases of Avahi to loosen  
>>>> these
>>>> limitations a little bit, to enable more comprehensive usage of Avahi,
>>>> even for professional applications like ours ;-)
>>>> DOS Attacks are way above the limitations of 500, even way above  
>>>> 5000. I
>>>> think when talking about DOS we are in millions and above.
>>>>
>>>> What do you say?
>>>>
>>>> Kind Regards
>>>>
>>>> Daniel
>>>>
>>>>
>>>>
>>>> On Sun, 2009-06-07 at 01:47 +0200, Lennart Poettering wrote:
>>>>    
>>>>      
>>>>        
>>>>> On Thu, 28.05.09 13:43, Daniel Wynne (daniel.wynne@...)  
>>>>> wrote:
>>>>>
>>>>>      
>>>>>        
>>>>>          
>>>>>> But I think thats exactly the problem in our case. In our  
>>>>>> Testnetwork
>>>>>> reside about 250 cameras we want to find and resolve via Avahi. So  
>>>>>> the
>>>>>> cache is way big enough.
>>>>>>        
>>>>>>          
>>>>>>            
>>>>> No. The limit of 500 entries in the cache that Avahi enforces is for
>>>>> RRs, not services or hosts.
>>>>>
>>>>> For each service you need (at least) one A, one PTR one SRV and one
>>>>> TXT RR. Usually even more. I.e. if you register 250 services and want
>>>>> to browser for them all at the same time you need as absolute minimum
>>>>> space for 1000 entries, and practically even more.
>>>>>
>>>>> Lennart
>>>>>
>>>>>      
>>>>>        
>>>>>          
>>>> _______________________________________________
>>>> avahi mailing list
>>>> avahi@...
>>>> http://lists.freedesktop.org/mailman/listinfo/avahi
>>>>
>>>>    
>>>>      
>>>>        
>>> _______________________________________________
>>> avahi mailing list
>>> avahi@...
>>> http://lists.freedesktop.org/mailman/listinfo/avahi
>>>  
>>>    
>>>      
>>  
>>    
>
> _______________________________________________
> avahi mailing list
> avahi@...
> http://lists.freedesktop.org/mailman/listinfo/avahi

_______________________________________________
avahi mailing list
avahi@...
http://lists.freedesktop.org/mailman/listinfo/avahi

[Daniel Wynne]

Hey Guys!

Really dont want to be annoying, but it would be really nice if we could
get at least one reliable statement from somebody responsible regarding
the issue below.

Thanx

Daniel

> Hi again!
>
> Is there anyone responsible available who is capable of answering my
> previous question? Everybody on vacation? ;-)
>
> Kind Regards
>
> Daniel
>
>
>
> Daniel wrote:
>  
>> Hi Folks,
>>
>> just returning from my summer vacation I reflected about past projects.
>>
>> Thereby I also wondered about our little conversation concerning Avahi
>> cache sizes.
>>
>> Trent, you made those promising hints that you will loosen the strict
>> limitations to meet requirements of larger network implementations.
>>
>> Could you, or any other responsible person please make a definite
>> authorative statement about this issue so we can proceed with development?
>> Me and my boss would be very appreciated about this ;-)
>>
>> Kind Regards
>>
>> Daniel
>>
>>
>> Daniel wrote:
>>  
>>    
>>> Hi Trent!
>>>
>>> These are great news for us! :-)
>>> We were seriously concerned about the issue, that the Avahi mDNS
>>> implementation prevented us from developing Linux camera management
>>> applications for huge setups.
>>>
>>> In the following I provide a short calculation to give you a hint of
>>> what we need:
>>>
>>>     - Right now we have setups with deployments of up to 800 devices We
>>> do not think that deployments with even more devices in one network make
>>> sense, as this already can be considered as a individual feasibility
>>> study. But lets calculate with this worst case scenario and for the sake
>>> of convenience we even round it up to N = 1000 devices per network.
>>>
>>>     - Since every device typically comes up with 2-3 ip addresses, we
>>> have to reserve cache entries for N*( 3 A's, 1 PTR, 1 SRV, 1 TXT. ) =>
>>> AVAHI_CACHE_ENTRIES_MAX >= 6*N = 6000 entries.
>>>
>>>     - As the dbus limitations are concerned the maximum number of dbus
>>> objects per client will have to be increased, too. Since we have to
>>> create N*3 AddressRecordBrowsers, N*1 ServiceResolver, 1 Client, 2
>>> ServiceBrowsers => OBJECTS_PER_CLIENT_MAX >= 4*N + 3 = 4003 ~ 4000 entries
>>>
>>> As I do not have that much insight into the Avahi-code there might be
>>> even more limits that might have to be adapted to achieve a coherent
>>> configuration.
>>>
>>> If you need any additional information, let me know.
>>>
>>> Thank you Guys for the great support so far!
>>>
>>> Kind Regards
>>>
>>> Daniel
>>>
>>>
>>> Trent Lloyd wrote:
>>>  
>>>    
>>>      
>>>> FWIW, I've always felt this was a little restrictive as well but never  
>>>> really ran into it in production.
>>>>
>>>> I am personally keen on upping them a little - Daniel, can you tell us  
>>>> what values seem to work well for you?
>>>>
>>>> Thanks,
>>>> Trent
>>>>
>>>> On 27/06/2009, at 12:04 AM, Daniel Wynne wrote:
>>>>
>>>>  
>>>>    
>>>>      
>>>>        
>>>>> Hi Lennart!
>>>>>
>>>>> After a short interrupt I am back in avahi-business ;-)
>>>>>
>>>>> PLaying around a little bit with parameters I boiled it down to
>>>>> AVAHI_CACHE_ENTRIES_MAX
>>>>> OBJECTS_PER_CLIENT_MAX
>>>>>
>>>>> These Parameters are slightly tight I think when it comes to bigger
>>>>> networks i.e. more than 100 Services.
>>>>>
>>>>> You said these limitations are implemented to avoid misuse, i.e.
>>>>> DOS-Attacks.
>>>>>
>>>>> Do you think it is possible in future releases of Avahi to loosen  
>>>>> these
>>>>> limitations a little bit, to enable more comprehensive usage of Avahi,
>>>>> even for professional applications like ours ;-)
>>>>> DOS Attacks are way above the limitations of 500, even way above  
>>>>> 5000. I
>>>>> think when talking about DOS we are in millions and above.
>>>>>
>>>>> What do you say?
>>>>>
>>>>> Kind Regards
>>>>>
>>>>> Daniel
>>>>>
>>>>>
>>>>>
>>>>> On Sun, 2009-06-07 at 01:47 +0200, Lennart Poettering wrote:
>>>>>    
>>>>>      
>>>>>        
>>>>>          
>>>>>> On Thu, 28.05.09 13:43, Daniel Wynne (daniel.wynne@...)  
>>>>>> wrote:
>>>>>>
>>>>>>      
>>>>>>        
>>>>>>          
>>>>>>            
>>>>>>> But I think thats exactly the problem in our case. In our  
>>>>>>> Testnetwork
>>>>>>> reside about 250 cameras we want to find and resolve via Avahi. So  
>>>>>>> the
>>>>>>> cache is way big enough.
>>>>>>>        
>>>>>>>          
>>>>>>>            
>>>>>>>              
>>>>>> No. The limit of 500 entries in the cache that Avahi enforces is for
>>>>>> RRs, not services or hosts.
>>>>>>
>>>>>> For each service you need (at least) one A, one PTR one SRV and one
>>>>>> TXT RR. Usually even more. I.e. if you register 250 services and want
>>>>>> to browser for them all at the same time you need as absolute minimum
>>>>>> space for 1000 entries, and practically even more.
>>>>>>
>>>>>> Lennart
>>>>>>
>>>>>>      
>>>>>>        
>>>>>>          
>>>>>>            
>>>>> _______________________________________________
>>>>> avahi mailing list
>>>>> avahi@...
>>>>> http://lists.freedesktop.org/mailman/listinfo/avahi
>>>>>
>>>>>    
>>>>>      
>>>>>        
>>>>>          
>>>> _______________________________________________
>>>> avahi mailing list
>>>> avahi@...
>>>> http://lists.freedesktop.org/mailman/listinfo/avahi
>>>>  
>>>>    
>>>>      
>>>>        
>>>  
>>>    
>>>      
>> _______________________________________________
>> avahi mailing list
>> avahi@...
>> http://lists.freedesktop.org/mailman/listinfo/avahi
>>    
>
> _______________________________________________
> avahi mailing list
> avahi@...
> http://lists.freedesktop.org/mailman/listinfo/avahi
>  

--
---------------------------------------------------------------------
Daniel Wynne, Softwareentwicklung
MOBOTIX AG - Security Vision Systems
Kaiserstrasse, 67722 Langmeil
---------------------------------------------------------------------
daniel.wynne@..., http://www.mobotix.com
---------------------------------------------------------------------
Vorstandvorsitzender: Dr. Ralf Hinkel
Vorstandsmitglieder: Lutz Coelen
Aufsichtsratsvorsitzender: Dr. Thomas Hoch
Registergericht: HRB Kaiserslautern 3724  

_______________________________________________
avahi mailing list
avahi@...
http://lists.freedesktop.org/mailman/listinfo/avahi

[Trent Lloyd]

lennart is really the only person that can respond to this since he  
moved to git I no longer have direct commit access so he is the gate  
keeper!

Trent

On 16/10/2009, at 3:03 PM, Daniel Wynne wrote:

> Hey Guys!
>
> Really dont want to be annoying, but it would be really nice if we  
> could
> get at least one reliable statement from somebody responsible  
> regarding
> the issue below.
>
> Thanx
>
> Daniel
>
>> Hi again!
>>
>> Is there anyone responsible available who is capable of answering my
>> previous question? Everybody on vacation? ;-)
>>
>> Kind Regards
>>
>> Daniel
>>
>>
>>
>> Daniel wrote:
>>
>>> Hi Folks,
>>>
>>> just returning from my summer vacation I reflected about past  
>>> projects.
>>>
>>> Thereby I also wondered about our little conversation concerning  
>>> Avahi
>>> cache sizes.
>>>
>>> Trent, you made those promising hints that you will loosen the  
>>> strict
>>> limitations to meet requirements of larger network implementations.
>>>
>>> Could you, or any other responsible person please make a definite
>>> authorative statement about this issue so we can proceed with  
>>> development?
>>> Me and my boss would be very appreciated about this ;-)
>>>
>>> Kind Regards
>>>
>>> Daniel
>>>
>>>
>>> Daniel wrote:
>>>
>>>
>>>> Hi Trent!
>>>>
>>>> These are great news for us! :-)
>>>> We were seriously concerned about the issue, that the Avahi mDNS
>>>> implementation prevented us from developing Linux camera management
>>>> applications for huge setups.
>>>>
>>>> In the following I provide a short calculation to give you a hint  
>>>> of
>>>> what we need:
>>>>
>>>>    - Right now we have setups with deployments of up to 800  
>>>> devices We
>>>> do not think that deployments with even more devices in one  
>>>> network make
>>>> sense, as this already can be considered as a individual  
>>>> feasibility
>>>> study. But lets calculate with this worst case scenario and for  
>>>> the sake
>>>> of convenience we even round it up to N = 1000 devices per network.
>>>>
>>>>    - Since every device typically comes up with 2-3 ip addresses,  
>>>> we
>>>> have to reserve cache entries for N*( 3 A's, 1 PTR, 1 SRV, 1  
>>>> TXT. ) =>
>>>> AVAHI_CACHE_ENTRIES_MAX >= 6*N = 6000 entries.
>>>>
>>>>    - As the dbus limitations are concerned the maximum number of  
>>>> dbus
>>>> objects per client will have to be increased, too. Since we have to
>>>> create N*3 AddressRecordBrowsers, N*1 ServiceResolver, 1 Client, 2
>>>> ServiceBrowsers => OBJECTS_PER_CLIENT_MAX >= 4*N + 3 = 4003 ~  
>>>> 4000 entries
>>>>
>>>> As I do not have that much insight into the Avahi-code there  
>>>> might be
>>>> even more limits that might have to be adapted to achieve a  
>>>> coherent
>>>> configuration.
>>>>
>>>> If you need any additional information, let me know.
>>>>
>>>> Thank you Guys for the great support so far!
>>>>
>>>> Kind Regards
>>>>
>>>> Daniel
>>>>
>>>>
>>>> Trent Lloyd wrote:
>>>>
>>>>
>>>>
>>>>> FWIW, I've always felt this was a little restrictive as well but  
>>>>> never
>>>>> really ran into it in production.
>>>>>
>>>>> I am personally keen on upping them a little - Daniel, can you  
>>>>> tell us
>>>>> what values seem to work well for you?
>>>>>
>>>>> Thanks,
>>>>> Trent
>>>>>
>>>>> On 27/06/2009, at 12:04 AM, Daniel Wynne wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Hi Lennart!
>>>>>>
>>>>>> After a short interrupt I am back in avahi-business ;-)
>>>>>>
>>>>>> PLaying around a little bit with parameters I boiled it down to
>>>>>> AVAHI_CACHE_ENTRIES_MAX
>>>>>> OBJECTS_PER_CLIENT_MAX
>>>>>>
>>>>>> These Parameters are slightly tight I think when it comes to  
>>>>>> bigger
>>>>>> networks i.e. more than 100 Services.
>>>>>>
>>>>>> You said these limitations are implemented to avoid misuse, i.e.
>>>>>> DOS-Attacks.
>>>>>>
>>>>>> Do you think it is possible in future releases of Avahi to loosen
>>>>>> these
>>>>>> limitations a little bit, to enable more comprehensive usage of  
>>>>>> Avahi,
>>>>>> even for professional applications like ours ;-)
>>>>>> DOS Attacks are way above the limitations of 500, even way above
>>>>>> 5000. I
>>>>>> think when talking about DOS we are in millions and above.
>>>>>>
>>>>>> What do you say?
>>>>>>
>>>>>> Kind Regards
>>>>>>
>>>>>> Daniel
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Sun, 2009-06-07 at 01:47 +0200, Lennart Poettering wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On Thu, 28.05.09 13:43, Daniel Wynne (daniel.wynne@...)
>>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> But I think thats exactly the problem in our case. In our
>>>>>>>> Testnetwork
>>>>>>>> reside about 250 cameras we want to find and resolve via  
>>>>>>>> Avahi. So
>>>>>>>> the
>>>>>>>> cache is way big enough.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> No. The limit of 500 entries in the cache that Avahi enforces  
>>>>>>> is for
>>>>>>> RRs, not services or hosts.
>>>>>>>
>>>>>>> For each service you need (at least) one A, one PTR one SRV  
>>>>>>> and one
>>>>>>> TXT RR. Usually even more. I.e. if you register 250 services  
>>>>>>> and want
>>>>>>> to browser for them all at the same time you need as absolute  
>>>>>>> minimum
>>>>>>> space for 1000 entries, and practically even more.
>>>>>>>
>>>>>>> Lennart
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> _______________________________________________
>>>>>> avahi mailing list
>>>>>> avahi@...
>>>>>> http://lists.freedesktop.org/mailman/listinfo/avahi
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> avahi mailing list
>>>>> avahi@...
>>>>> http://lists.freedesktop.org/mailman/listinfo/avahi
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>> _______________________________________________
>>> avahi mailing list
>>> avahi@...
>>> http://lists.freedesktop.org/mailman/listinfo/avahi
>>>
>>
>> _______________________________________________
>> avahi mailing list
>> avahi@...
>> http://lists.freedesktop.org/mailman/listinfo/avahi
>>
>
>
> --
> ---------------------------------------------------------------------
> Daniel Wynne, Softwareentwicklung
> MOBOTIX AG - Security Vision Systems
> Kaiserstrasse, 67722 Langmeil
> ---------------------------------------------------------------------
> daniel.wynne@..., http://www.mobotix.com
> ---------------------------------------------------------------------
> Vorstandvorsitzender: Dr. Ralf Hinkel
> Vorstandsmitglieder: Lutz Coelen
> Aufsichtsratsvorsitzender: Dr. Thomas Hoch
> Registergericht: HRB Kaiserslautern 3724
>
> _______________________________________________
> avahi mailing list
> avahi@...
> http://lists.freedesktop.org/mailman/listinfo/avahi
>

_______________________________________________
avahi mailing list
avahi@...
http://lists.freedesktop.org/mailman/listinfo/avahi