MingWin compiling bug 705

[Faker Moatamri]

Hi Gustavo & George,

Yes again Mingwin, it is still an issue and buildbot doesn't build with
Mingwin. The issue is reported in bug 705. Thanks Gustavo for the patch
but it needs more work right? As far as I understand, we need to disable
code using socket in net-anim which requires to add some #ifdef to the
code right?

Gustavo, George, can you please make the patch in bug 705 work so that
we can keep on supporting Mingwin, hoping that other problems won't arise?

Thanks
Faker Moatamri

[George Riley]

Yes, we can put needed ifdef's in to get Mingwin to compile. Of course  
the animation
will not be supported there.

Am cc'ing Josh for information.

George

--------------------------------------------------
George Riley
Associate Professor
Georgia Tech
Electrical and Computer Engineering
riley@...
404-894-4767
Klaus Advanced Computing Building
Room 3360
266 Ferst Drive
Atlanta, Georgia 30332-0765

ECE6110 Web page:
http://users.ece.gatech.edu/~riley/ece6110/

ECE3090 Web page:
http://users.ece.gatech.edu/~riley/ece3090/





On Nov 3, 2009, at 11:37 AM, Faker Moatamri wrote:

> Hi Gustavo & George,
>
> Yes again Mingwin, it is still an issue and buildbot doesn't build  
> with Mingwin. The issue is reported in bug 705. Thanks Gustavo for  
> the patch but it needs more work right? As far as I understand, we  
> need to disable code using socket in net-anim which requires to add  
> some #ifdef to the code right?
>
> Gustavo, George, can you please make the patch in bug 705 work so  
> that we can keep on supporting Mingwin, hoping that other problems  
> won't arise?
>
> Thanks
> Faker Moatamri

[craigdo]

> Gustavo, George, can you please make the patch in bug 705 work so that
> we can keep on supporting Mingwin, hoping that other problems won't
> arise?

Keep on supporting Mingwin?

I'll say it again.  MinGW is not a supported platform for ns-3.  I have seen
no indication that this has been discussed anywhere and I've seen no
decision made by the project.  As far as I can make out, our "official"
windows story consists entirely of Cygwin.

If we are going to make MinGW a supported platform, we (as a group) should
make a positive decision and actually consistently add it to the list of
supported platforms and have someone actually test it before each release.
We need to let people know that they have got to find a MinGW installation
and test their code under MinGW before committing it.

I don't know what the issue is, but MinGW support keeps trying to sneak in
the back door instead of politely knocking on the front.  In different
documents, MinGW will sometimes just appear out of nowhere.  I noticed that
it has been added as a supported platform in the RELEASE_NOTES on ns-3-dev.
This seems to have happened for ns-3.5.1-RC2.  (Huh?  We silently added a
new supported platform in a second release candidate?)  It is not, however,
added to the list of supported platforms on the main web site, nor the wiki.
There was no discussion nor notification to developers to start testing
things under MinGW, no indication that someone had actually verified that
everything was working -- it just silently appeared in the RELEASE_NOTES.

That is completely bogus.  Either it *is* a supported platform or it *is
not*.  Either we (as a project) decide it is worthwhile to support it or
not.  It is ridiculous to find no mention of MinGW on the project web site,
but find it listed as supported in the RELEASE_NOTES.  It is ridiculous to
find no mention of MinGW in one paragraph of the wiki as a supported
platform only to find it in a product matrix in the very next section.  See:

  http://www.nsnam.org/getting_started.html#os

and

  http://www.nsnam.org/wiki/index.php/Installation#Supported_platforms

Can we, as a project, get our act together on this and make a positive (or
negative) decision and consistently document this decision?  Can we let
developers actually know that their code needs to deal with the possibility
that it may be run in some MSVCRT-like environment or write itself out of
the build?  I doubt most of our developers are actually seriously
considering that sockets may actually turn out to be Winsock, for example.

If we decide positively, will someone pick up the ball and make sure all of
the documentation agrees with our decision and run all of the
samples/examples that are built to make sure they work?

So, now for the real question:  Does ns-3 view MinGW as a supported
platform?

-0

I am neutral, tending slightly negative.  If there is a significant demand
for running ns-3 on MinGW, sure -- let's officially support it; but let's
document and advertize that fact.  If nobody is going to use it, and all it
does is get in the way while pretending to be some kind of alternate
platform to enforce arbitrary portability guidelines, then no -- let's not
bother.  It's just a pain.

What I am not neutral about is the half-in, half-out, partially-tested,
snuck-in nature of current MinGW "support."

Regards,

-- Craig

[Faker Moatamri]

> So, now for the real question:  Does ns-3 view MinGW as a supported
> platform?
>
> -0
>
You're right, we should have only one word about mingwin. Either it is or
it is not supported. If it is not supported, no bug should be filed for it
and buildbot should not compile Mingwin.
I would also give -0 or even -1 for Mingwin support as it lacks a lot of
libraries and it will be only a pain for maintainers. Moreover, Cygwin is
doing the work under windows, so there is no point of going in the hassle
of supporting Mingwin while Cygwin is there.
Tom, Mathieu, others, what do you think?

[craigdo]

> > So, now for the real question:  Does ns-3 view MinGW as a supported
> > platform?
> >
> > -0
> >
> You're right, we should have only one word about mingwin. Either it is
> or
> it is not supported. If it is not supported, no bug should be filed for
> it
> and buildbot should not compile Mingwin.
> I would also give -0 or even -1 for Mingwin support as it lacks a lot
> of
> libraries and it will be only a pain for maintainers. Moreover, Cygwin
> is
> doing the work under windows, so there is no point of going in the
> hassle
> of supporting Mingwin while Cygwin is there.
> Tom, Mathieu, others, what do you think?

In defense of the MinGW platform, Python bindings work there but do not work under Cygwin.

[Tom Henderson-2]

On Tue, 3 Nov 2009 21:32:58 +0100 (CET), Faker.Moatamri@...
wrote:
>> So, now for the real question:  Does ns-3 view MinGW as a supported
>> platform?
>>
>> -0
>>
> You're right, we should have only one word about mingwin. Either it is or
> it is not supported. If it is not supported, no bug should be filed for
it
> and buildbot should not compile Mingwin.
> I would also give -0 or even -1 for Mingwin support as it lacks a lot of
> libraries and it will be only a pain for maintainers. Moreover, Cygwin is
> doing the work under windows, so there is no point of going in the hassle
> of supporting Mingwin while Cygwin is there.
> Tom, Mathieu, others, what do you think?

Two weeks ago, I asked on both the users and developers list whether there
are any users who would like MinGW to be supported, and there were no
responses.  Without user demand, and given that we have two existing
solutions for Windows (Cygwin and virtualization), I have a hard time
prioritizing the addition of MinGW support as officially supported
platform.

Another option we have used for ns-2 in the past is to have volunteer
maintainers for non-supported platforms of interest.  Cygwin, Solaris, and
icc were maintained like this, with the maintainers of those platforms
sending in patches around release-candidate time to get the release back
into shape.  

We could also try to take some data on this by preparing an
ns-allinone-3.6-mingw release and posting it/announcing it as a MinGW
variant and see whether it gets used.

Tom

[Faker Moatamri]

>
> Two weeks ago, I asked on both the users and developers list whether there
> are any users who would like MinGW to be supported, and there were no
> responses.  Without user demand, and given that we have two existing
> solutions for Windows (Cygwin and virtualization), I have a hard time
> prioritizing the addition of MinGW support as officially supported
> platform.
>
I noticed that also, it is part of why I voted -1.
> Another option we have used for ns-2 in the past is to have volunteer
> maintainers for non-supported platforms of interest.  Cygwin, Solaris, and
> icc were maintained like this, with the maintainers of those platforms
> sending in patches around release-candidate time to get the release back
> into shape.
>
> We could also try to take some data on this by preparing an
> ns-allinone-3.6-mingw release and posting it/announcing it as a MinGW
> variant and see whether it gets used.
This is kind of last chance for Mingwin surviving, afterwards, we can
decide afterwards if we keep it or not. I agree with this idea.
>
> Tom
>

[Gustavo Carneiro]

I vote +0 for supporting MinGW: I can promise to try to make the code I
maintain work on MinGW (already does, I think), but I do not care enough to
spend a lot of time fixing other code I did not write.

I personally do not use MinGW, but I do not want to be accused of writing
non-portable code, and MinGW is a good way to test the limits of portability
of the code.  The longer we keep writing non-portable code the harder it
will be to come back if in the future we decide to support native win32.


2009/11/3 <craigdo@...>

> > Gustavo, George, can you please make the patch in bug 705 work so that
> > we can keep on supporting Mingwin, hoping that other problems won't
> > arise?
>
> Keep on supporting Mingwin?
>
> I'll say it again.  MinGW is not a supported platform for ns-3.  I have
> seen
> no indication that this has been discussed anywhere and I've seen no
> decision made by the project.  As far as I can make out, our "official"
> windows story consists entirely of Cygwin.
>
> If we are going to make MinGW a supported platform, we (as a group) should
> make a positive decision and actually consistently add it to the list of
> supported platforms and have someone actually test it before each release.
> We need to let people know that they have got to find a MinGW installation
> and test their code under MinGW before committing it.
>
> I don't know what the issue is, but MinGW support keeps trying to sneak in
> the back door instead of politely knocking on the front.  In different
> documents, MinGW will sometimes just appear out of nowhere.  I noticed that
> it has been added as a supported platform in the RELEASE_NOTES on ns-3-dev.
> This seems to have happened for ns-3.5.1-RC2.  (Huh?  We silently added a
> new supported platform in a second release candidate?)  It is not, however,
> added to the list of supported platforms on the main web site, nor the
> wiki.
> There was no discussion nor notification to developers to start testing
> things under MinGW, no indication that someone had actually verified that
> everything was working -- it just silently appeared in the RELEASE_NOTES.
>
> That is completely bogus.  Either it *is* a supported platform or it *is
> not*.  Either we (as a project) decide it is worthwhile to support it or
> not.  It is ridiculous to find no mention of MinGW on the project web site,
> but find it listed as supported in the RELEASE_NOTES.  It is ridiculous to
> find no mention of MinGW in one paragraph of the wiki as a supported
> platform only to find it in a product matrix in the very next section.
>  See:
>
>  http://www.nsnam.org/getting_started.html#os
>
> and
>
>  http://www.nsnam.org/wiki/index.php/Installation#Supported_platforms
>
> Can we, as a project, get our act together on this and make a positive (or
> negative) decision and consistently document this decision?  Can we let
> developers actually know that their code needs to deal with the possibility
> that it may be run in some MSVCRT-like environment or write itself out of
> the build?  I doubt most of our developers are actually seriously
> considering that sockets may actually turn out to be Winsock, for example.
>
> If we decide positively, will someone pick up the ball and make sure all of
> the documentation agrees with our decision and run all of the
> samples/examples that are built to make sure they work?
>
> So, now for the real question:  Does ns-3 view MinGW as a supported
> platform?
>
> -0
>
> I am neutral, tending slightly negative.  If there is a significant demand
> for running ns-3 on MinGW, sure -- let's officially support it; but let's
> document and advertize that fact.  If nobody is going to use it, and all it
> does is get in the way while pretending to be some kind of alternate
> platform to enforce arbitrary portability guidelines, then no -- let's not
> bother.  It's just a pain.
>
> What I am not neutral about is the half-in, half-out, partially-tested,
> snuck-in nature of current MinGW "support."
>
> Regards,
>
> -- Craig
>
>
>

--
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert

[Mathieu Lacage]

On Tue, 2009-11-03 at 14:21 -0700, Tom Henderson wrote:

> > You're right, we should have only one word about mingwin. Either it is or
> > it is not supported. If it is not supported, no bug should be filed for
> it
> > and buildbot should not compile Mingwin.

This is clearly wrong: not being officially supported merely means that
no paid developer will invest explicit resources in mingw. However, it
is the duty of paid developers to invest cycles to help mingw
contributors to fix mingw. i.e., bugs filed against mingw should not be
summarily closed as WONTFIX. They should be at least left open with a
way for others to keep track of them.

> > I would also give -0 or even -1 for Mingwin support as it lacks a lot of
> > libraries and it will be only a pain for maintainers. Moreover, Cygwin is
> > doing the work under windows, so there is no point of going in the hassle
> > of supporting Mingwin while Cygwin is there.
> > Tom, Mathieu, others, what do you think?
>
> Two weeks ago, I asked on both the users and developers list whether there
> are any users who would like MinGW to be supported, and there were no
> responses.  Without user demand, and given that we have two existing

I did not notice that email.

> solutions for Windows (Cygwin and virtualization), I have a hard time
> prioritizing the addition of MinGW support as officially supported
> platform.

I don't think that anyone is seriously suggesting to make MingW an
officially supported platform but I personally strongly believe that not
refusing patches to fix mingw and actually working with the mingw
contributors (gustavo) to produce correct and useful patches is
important, not because mingw is important per se, but because supporting
more platforms generally increases test coverage and improves the
quality of our codebase.

I think that shipping a release which is known to not even compile on
mingw is generally not a good idea. Again, not because I care about
mingw but because not even compiling or passing basic sanity checks on
mingw is a sign of low code quality.

Mathieu

[Faker Moatamri]

Hi all,
Please find attached  a patch to make Mingw compile/Build. It contains
already the changes done by Gustavo. Please review it and if no one
disagrees, I will upload it to the server by friday.
Best regards
Faker Moatamri

diff -r 3827a2a06b38 src/contrib/net-anim/animation-interface.cc
--- a/src/contrib/net-anim/animation-interface.cc Tue Nov 03 12:50:18 2009 +0300
+++ b/src/contrib/net-anim/animation-interface.cc Wed Nov 04 15:41:34 2009 +0100
@@ -21,9 +21,17 @@
 #include <stdio.h>
 #include <sstream>
 
+#include "ns3/net-anim-config.h"
+
+#ifdef MINGW
+#include <fcntl.h>
+#endif
+
 // Socket related includes
-#include <sys/socket.h>
-#include <netinet/in.h>
+#if defined(HAVE_SYS_SOCKET_H) && defined(HAVE_NETINET_IN_H)
+# include <sys/socket.h>
+# include <netinet/in.h>
+#endif
 
 // ns3 includes
 #include "ns3/animation-interface.h"
@@ -58,6 +66,7 @@
 
 bool AnimationInterface::SetServerPort (uint16_t port)
 {
+#if (!(defined(HAVE_SYS_SOCKET_H)) || !(defined(HAVE_NETINET_IN_H)))
   int s = socket (AF_INET, SOCK_STREAM, 0);
   struct sockaddr_in addr;
   addr.sin_family = AF_INET;
@@ -77,6 +86,8 @@
   int t = 1;
   setsockopt (s, SOL_SOCKET, SO_LINGER, &t, sizeof(t));
   return true;
+#endif
+  return false;//never reached unless the above is disabled
 }
 
 bool AnimationInterface::SetInternalAnimation ()
diff -r 3827a2a06b38 src/contrib/net-anim/point-to-point-dumbbell-helper.cc
--- a/src/contrib/net-anim/point-to-point-dumbbell-helper.cc Tue Nov 03 12:50:18 2009 +0300
+++ b/src/contrib/net-anim/point-to-point-dumbbell-helper.cc Wed Nov 04 15:41:34 2009 +0100
@@ -21,10 +21,6 @@
 #include <iostream>
 #include <sstream>
 
-// Socket related includes
-#include <sys/socket.h>
-#include <netinet/in.h>
-
 // ns3 includes
 #include "ns3/animation-interface.h"
 #include "ns3/point-to-point-dumbbell-helper.h"
diff -r 3827a2a06b38 src/contrib/net-anim/wscript
--- a/src/contrib/net-anim/wscript Tue Nov 03 12:50:18 2009 +0300
+++ b/src/contrib/net-anim/wscript Wed Nov 04 15:41:34 2009 +0100
@@ -1,5 +1,11 @@
 ## -*- Mode: python; py-indent-offset: 4; indent-tabs-mode: nil; coding: utf-8; -*-
 
+def configure(conf):
+    conf.check(header_name='sys/socket.h', define_name='HAVE_SYS_SOCKET_H')
+    conf.check(header_name='netinet/in.h', define_name='HAVE_NETINET_IN_H')
+    conf.write_config_header('ns3/net-anim-config.h', top=True)
+
+
 def build(bld):
     obj = bld.create_ns3_module('net-anim')
     obj.source = [
diff -r 3827a2a06b38 src/contrib/wscript
--- a/src/contrib/wscript Tue Nov 03 12:50:18 2009 +0300
+++ b/src/contrib/wscript Wed Nov 04 15:41:34 2009 +0100
@@ -14,9 +14,11 @@
     conf.report_optional_feature("XmlIo", "XmlIo",
                                  conf.env['ENABLE_LIBXML2'],
                                  "library 'libxml-2.0 >= 2.7' not found")
+    conf.write_config_header('ns3/contrib-config.h', top=True)
+
     conf.sub_config('stats')
+    conf.sub_config('net-anim')
 
-    conf.write_config_header('ns3/contrib-config.h', top=True)
 
 def build(bld):
     module = bld.create_ns3_module('contrib', ['simulator', 'common'])

[craigdo]

[ ... ]

> supporting more platforms generally increases test coverage and improves
the
> quality of our codebase.

I disagree.  Supporting more Unix-like platforms *may* increase the quality
of our codebase.  It also may introduce hacks and special cases to
compensate for bugs or different capabilities in the various platforms and
compilers.  Supporting something like MinGW will simply introduce complexity
for no good reason as far as I can tell (if there is no great user demand
for it).

We'll have to make porting classes or special case code which increases the
amount of code in the product and provides more paths that must be tested,
not fewer.  We'll need to introduce more special cases in wscripts (on
"win32", for example to compensate for features not available in win32)
which introduces more complexity that also must be tested.  We'll introduce
differences in the product that must be documented and tested for graceful
degradation (see the product matrix I mentioned earlier).

I think that having to factor the codebase more, and introducing
special-case code that is rarely if ever run in the real world actually
degrades the quality of our codebase.

Why in the world would we want to do all of this to not officially support a
platform virtually nobody uses?

> I think that shipping a release which is known to not even compile on
> mingw is generally not a good idea. Again, not because I care about
> mingw but because not even compiling or passing basic sanity checks on
> mingw is a sign of low code quality.

I disagree.  That is like saying that an application that is designed to run
on, say KDE has low code quality because it does not run on Windows 7.  That
does not follow.  It says that the application writers do not consider
portability between Unix and Windows 7 important enough to write a
portability layer or jump through the required hoops.

There are things that are missing on MinGW and will never be there.  If we
want to write something that is really portable to MinGW/win32, we need to
do a lot more than just accommodate '/' versus '\'.  If we want a product
that is portable to win32, then emulation code should be able to use
Winsock, and the real-time scheduler should have a component that uses
Windows CreateThread and WaitForSingleObject.  I think it is an illusion to
think that having some of our product compile on MinGW makes our code
somehow better.  What we end up with is a crippled win32 product and a bunch
of special case code in the build.  I could say the same thing about Cygwin,
too; but it is at least pretending to be Unix-like and provides a modicum of
Unix-ness so we don't have to deal with as many OS portability issues.

It seems to me that we are really building a Linux product.  If someone
wants to do serious work, then that is really the only option since that is
where you get real-time, emu, tap, NSC, simu, mysql, etc., etc.  We are
constantly asking, "What Would Linux Do" when thinking about how features
should be implemented.

There was talk some time ago about producing a general purpose simulator
core product.  Now, I think if we want to do that, or make a portable
simulator subset, then it would be a good idea to define it and then build
and test on wildly different OSes and run-times to ensure that we have
something that really is OS *independent*.  That seems to me to be a very
different goal than simply making part of ns-3 work on MinGW, though.

Anyway, my point is that I think we need to tighten up this story and
clearly understand and state what the MinGW and portability story actually
is.  If MinGW is not supported, we need to stop saying so and acting as if
it was.  We need to state a real distinction between "supported" and "tested
on" if we are going to imply a difference.  If we are going to build and
test it every night, we need to clearly state what we are implying by doing
so.  If someone checks in code that breaks on MinGW, we need to clearly
state that either this is okay or not; and how bugs will be dealt with and
with what priority.  Having a buildbot report failure every night to
ns-commits seems like a bad plan since it openly communicates poor quality
of the entire project.

-- Craig

[Mathieu Lacage]

On Wed, 2009-11-04 at 11:46 -0800, craigdo@... wrote:

> I disagree.  Supporting more Unix-like platforms *may* increase the quality
> of our codebase.  It also may introduce hacks and special cases to
> compensate for bugs or different capabilities in the various platforms and
> compilers.  Supporting something like MinGW will simply introduce complexity
> for no good reason as far as I can tell (if there is no great user demand
> for it).

Yes, mingw is not a unix platform but it's a gcc platform and that makes
it orders of magnitude easier to support than a plain win32 system.
i.e., it's not so far removed from a unix system.

> Anyway, my point is that I think we need to tighten up this story and
> clearly understand and state what the MinGW and portability story actually
> is.  If MinGW is not supported, we need to stop saying so and acting as if

I do not think anyone said mingw is supported.

> it was.  We need to state a real distinction between "supported" and "tested
> on" if we are going to imply a difference.  If we are going to build and
> test it every night, we need to clearly state what we are implying by doing
> so.  If someone checks in code that breaks on MinGW, we need to clearly
> state that either this is okay or not; and how bugs will be dealt with and

It is clearly ok to check in code which breaks mingw because it's not an
officially supported platform _but_, as I said previously, it's not ok
to summarily dismiss patches to fix mingw support on the grounds that
it's not a supported platform and, just like every patch submitted by
anyone, it is the duty of every maintainer to help contributors to put
together a correct patch. I don't care about mingw personally, but I
don't care about wifi block ack support either. However, in both cases,
I will work with the relevant contributors to help them merge their
code.

i.e., I believe that when gustavo submits a patch to fix mingw support,
the relevant maintainers should work with him just like they should work
with every contributor to merge their patches in a timely manner.

With that being said, I will stop sending emails about this: I think
that I made my point pretty clearly. Whoever decides on the fate of
mingw support will do so on their own.

Mathieu

[craigdo]

> I do not think anyone said mingw is supported.

This is from our current RELEASE_NOTES in ns-3-dev:

Supported platforms
-------------------
ns-3.4 has been tested on the following platforms:
  - linux x86 gcc 4.2, 4.1, and, 3.4.6.
  - linux x86_64 gcc 4.4.0, 4.3.2, 4.2.3, 4.2.1, 4.1.3, 3.4.6
  - MacOS X ppc and x86 (gcc 4.0.x and 4.2.x)
  - cygwin gcc 3.4.4 (debug only), gcc 4.3.2 (debug and optimized)
  - mingw gcc 3.4.5 (debug only)

And what triggered my email the other day was:

> Gustavo, George, can you please make the patch in bug 705 work so that
> we can keep on supporting Mingwin, hoping that other problems won't
> arise?

So, if it's not supported, we shouldn't say so; and we shouldn't build and
test it every night, sending email to ns-commits saying it's broken.

-- Craig

[Mathieu Lacage]

On Wed, 2009-11-04 at 12:35 -0800, craigdo@... wrote:

> So, if it's not supported, we shouldn't say so; and we shouldn't build and
> test it every night, sending email to ns-commits saying it's broken.

I am perfectly fine with making RELEASE_NOTES reflect more accurately
the current status of testing of ns-3, but I don't really see the point
of disabling actually useful automated testing. Who other than a handful
of developers actually read the buildbot emails ?

Mathieu

[craigdo]

[ ... ]

> I am perfectly fine with making RELEASE_NOTES reflect more accurately
> the current status of testing of ns-3, but I don't really see
> the point
> of disabling actually useful automated testing. Who other
> than a handful
> of developers actually read the buildbot emails ?

It's a question of "false positive" reports on ns-commits and access to the false positive reports on the web through the waterfall
report.  MinGW is a bright red light on the waterfall report saying that the build is broken.  It stands out starkly.  The link is
to the waterfall is at the top of:

  http://www.nsnam.org/wiki/index.php/Main_Page

You can also get to the test.py-generated HTML from there.

I talked to Faker about moving to positive acknowledgements from the buildbots -- that is, they send out success of failure messages
whenever they run.  This avoids situations where a buildbot has simply not been working for days or weeks and nobody notices since
its failed state doesn't change.

If I see a bunch of emails from the various buildbots saying "success" I get warm fuzzy feelings.  Whenever I see "failed" I
basically stop what I'm doing and investigate to see if there's something I need to do.  If I'm looking in from the outside, I might
wonder why this particular build is failing every time I look at the status.

I think it is fine to run the tests, but I'm not sure about sending out email all the time saying it's broken, or adding a red light
to the waterfall display saying it's broken.

-- Craig