Upgrading our minimum required flex version for 8.5

[Andrew Dunstan]

Andrew Dunstan wrote:

>
> OK, the fly in this ointment turns out to be MSVC. The latest flex
> from GnuWin32 is 2.5.4a, and building 2.5.35 for Windows is turning
> out to be quite a pain. Luckily, MinGW has a pre-built modified 2.5.33
> available, and I have installed this (also needed msys-regex), and
> then I jury rigged my MSVC build to use it, so we still have one MSVC
> working OK in the buildfarm. But we can hardly ask people to install
> MinGW/MSys so they can build with MSVC, that's horribly ugly. I'll
> work on getting version 2.5.35 build for Windows in a way that works
> standalone, and push it somewhere (maybe the dev wiki).
>
>

Well, it looks like there's a reason GnuWin32 hasn't advanced beyond
2.5.4a - after that the flex developers proceeded to make flex use a
filter chain methodology that requires the use of fork(). Making it run  
on Windows without the  support of Msys or Cygwin would involve some
significant surgery, I suspect.

Maybe for the time being we need to think about keeping scan.c in CVS.
It's not like scan.l gets updated all that often.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Tom Lane-2]

Andrew Dunstan <andrew@...> writes:
> Well, it looks like there's a reason GnuWin32 hasn't advanced beyond
> 2.5.4a - after that the flex developers proceeded to make flex use a
> filter chain methodology that requires the use of fork(). Making it run  
> on Windows without the  support of Msys or Cygwin would involve some
> significant surgery, I suspect.

Egad, this is a mess :-(.  I noticed in the flex changelog that they'd
switched to using m4 instead of implementing all the text processing
themselves.  I suppose this is a consequence of that.

But I'm not prepared to agree that M$ lameness should restrict us to
using only a 1990s version of flex.  Didn't somebody mention upthread
that there is a Windows port of 2.5.33 available?

> Maybe for the time being we need to think about keeping scan.c in CVS.
> It's not like scan.l gets updated all that often.

We could if we had to, though it amounts to saying that Windows-based
developers don't get to touch the scanner.

                        regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Andrew Dunstan]

Tom Lane wrote:

> Andrew Dunstan <andrew@...> writes:
>  
>> Well, it looks like there's a reason GnuWin32 hasn't advanced beyond
>> 2.5.4a - after that the flex developers proceeded to make flex use a
>> filter chain methodology that requires the use of fork(). Making it run  
>> on Windows without the  support of Msys or Cygwin would involve some
>> significant surgery, I suspect.
>>    
>
> Egad, this is a mess :-(.  I noticed in the flex changelog that they'd
> switched to using m4 instead of implementing all the text processing
> themselves.  I suppose this is a consequence of that.
>
> But I'm not prepared to agree that M$ lameness should restrict us to
> using only a 1990s version of flex.  Didn't somebody mention upthread
> that there is a Windows port of 2.5.33 available?
>  

Not one that is standalone. It needs some runtime support for Unix-like
process control, either Cygwin or Msys. So my Cygwin and Mingw/MSys
buildfarm members are quite happily using flex built for their
respective environments. I am leveraging the fact that I run all three
of my Windows based buildfarm members on the same Windows instance to
let the MSVC member use the MSys flex.

>  
>> Maybe for the time being we need to think about keeping scan.c in CVS.
>> It's not like scan.l gets updated all that often.
>>    
>
> We could if we had to, though it amounts to saying that Windows-based
> developers don't get to touch the scanner.
>
>
>  

True, but I'm not going to invest a large number of cycles on porting
this. I'm not very happy about it either. I guess anyone wanting to
develop on Windows and affect the scanner could install Cygwin or MSys.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Chuck McDevitt]

> -----Original Message-----
> From: pgsql-hackers-owner@... [mailto:pgsql-hackers-
> owner@...] On Behalf Of Tom Lane
> Sent: Monday, July 13, 2009 7:43 PM
> To: Andrew Dunstan
> Cc: pgsql-hackers@...
> Subject: Re: [HACKERS] Upgrading our minimum required flex version for
> 8.5
>
> Andrew Dunstan <andrew@...> writes:
> > Well, it looks like there's a reason GnuWin32 hasn't advanced beyond
> > 2.5.4a - after that the flex developers proceeded to make flex use a
> > filter chain methodology that requires the use of fork(). Making it
> run
> > on Windows without the  support of Msys or Cygwin would involve some
> > significant surgery, I suspect.
>
> Egad, this is a mess :-(.  I noticed in the flex changelog that they'd
> switched to using m4 instead of implementing all the text processing
> themselves.  I suppose this is a consequence of that.
>
> But I'm not prepared to agree that M$ lameness should restrict us to
> using only a 1990s version of flex.  Didn't somebody mention upthread
> that there is a Windows port of 2.5.33 available?
>
> > Maybe for the time being we need to think about keeping scan.c in
> CVS.
> > It's not like scan.l gets updated all that often.
>
> We could if we had to, though it amounts to saying that Windows-based
> developers don't get to touch the scanner.
>
> regards, tom lane

Flex 2.5.33 and bison 2.3 are available from mingw for windows.

http://sourceforge.net/projects/mingw/files/

Since mingw programs don't need Cygwin installed, these should probably be OK for most Windows people.

But if really needed, flex 2.5.33 could be ported (m4 is already ported).

I'm also wonderings why flex is a problem, since there is a GNUwin32 (native) port of Bison 2.4.1 and m4,
And Bison uses m4 these days, doesn't it?  Perhaps it wouldn't be so hard to update flex to use the same m4 calling that bison uses?



Chuck McDevitt

--
Sent via pgsql-hackers mailing list (pgsql-hackers@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Andrew Dunstan]

Chuck McDevitt wrote:

>
> Flex 2.5.33 and bison 2.3 are available from mingw for windows.
>
> http://sourceforge.net/projects/mingw/files/
>
> Since mingw programs don't need Cygwin installed, these should probably be OK for most Windows people.
>
> But if really needed, flex 2.5.33 could be ported (m4 is already ported).
>
> I'm also wonderings why flex is a problem, since there is a GNUwin32 (native) port of Bison 2.4.1 and m4,
> And Bison uses m4 these days, doesn't it?  Perhaps it wouldn't be so hard to update flex to use the same m4 calling that bison uses?
>
>
>  

It is part of the MSys utilities, and requires that you have MSys
installed. MSys is in effect a Cygwin equivalent. The flex from MinGW
will NOT run standalone. I have tried.

If you think it can be ported to run standalone (i.e. without either
Cygwin or MSys), have at it. Or take my advice and don't waste time trying.

cheers

andrew



--
Sent via pgsql-hackers mailing list (pgsql-hackers@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Andrew Dunstan]

Andrew Dunstan wrote:

>
>
> Chuck McDevitt wrote:
>>
>> Flex 2.5.33 and bison 2.3 are available from mingw for windows.
>>
>> http://sourceforge.net/projects/mingw/files/
>>
>> Since mingw programs don't need Cygwin installed, these should
>> probably be OK for most Windows people.
>>
>> But if really needed, flex 2.5.33 could be ported (m4 is already
>> ported).
>>
>> I'm also wonderings why flex is a problem, since there is a GNUwin32
>> (native) port of Bison 2.4.1 and m4,
>> And Bison uses m4 these days, doesn't it?  Perhaps it wouldn't be so
>> hard to update flex to use the same m4 calling that bison uses?
>>
>>
>>  
>
> It is part of the MSys utilities, and requires that you have MSys
> installed. MSys is in effect a Cygwin equivalent. The flex from MinGW
> will NOT run standalone. I have tried.
>
> If you think it can be ported to run standalone (i.e. without either
> Cygwin or MSys), have at it. Or take my advice and don't waste time
> trying.
>
>

What I have done is built a version of flex 2.5.35 with Cygwin, and
bundled the .exe with the Cygwin DLL (the only other thing it should
need, for our purposes) in a zip file. It can currently be fetched from
<http://developer.postgresql.org/~adunstan/windows-flex-2.5.35.zip>.
That way people won't have to do a full Cygwin or MSys install to use it.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Tom Lane-2]

Andrew Dunstan <andrew@...> writes:
> What I have done is built a version of flex 2.5.35 with Cygwin, and
> bundled the .exe with the Cygwin DLL (the only other thing it should
> need, for our purposes) in a zip file. It can currently be fetched from
> <http://developer.postgresql.org/~adunstan/windows-flex-2.5.35.zip>.
> That way people won't have to do a full Cygwin or MSys install to use it.

Sounds good.  The other idea that was suggested (putting scan.c into
CVS) seems problematic on reflection.  I don't believe that CVS will
guarantee that scan.c has a same-or-newer timestamp as scan.l if both
are pulled from CVS; so you might find make trying to rebuild scan.c
anyway.

                        regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Magnus Hagander-2]

On Tue, Jul 14, 2009 at 05:10, Andrew Dunstan<andrew@...> wrote:
> Tom Lane wrote:

>>> Maybe for the time being we need to think about keeping scan.c in CVS.
>>> It's not like scan.l gets updated all that often.
>>>
>>
>> We could if we had to, though it amounts to saying that Windows-based
>> developers don't get to touch the scanner.
>>
>>
>>
>
> True, but I'm not going to invest a large number of cycles on porting this.
> I'm not very happy about it either. I guess anyone wanting to develop on
> Windows and affect the scanner could install Cygwin or MSys.

I think requiring that for messing with the scanner is acceptable. As
it is now, requiring that to do *any* development or compiling on
HEAD, is a serious regression.

FWIW, it seems the version that Andrew put up doesn't work in one of
my test environments, and also not in at last one of Dave's. I will
test it in my second test environment later today to be sure.

--
 Magnus Hagander
 Self: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

--
Sent via pgsql-hackers mailing list (pgsql-hackers@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Dave Page-7]

On Wed, Jul 22, 2009 at 12:42 PM, Magnus Hagander<magnus@...> wrote:

> FWIW, it seems the version that Andrew put up doesn't work in one of
> my test environments, and also not in at last one of Dave's. I will
> test it in my second test environment later today to be sure.

It doesn't work in any of my bf animals, or build machines :-(

--
Dave Page
EnterpriseDB UK:   http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Andrew Dunstan]

Dave Page wrote:

> On Wed, Jul 22, 2009 at 12:42 PM, Magnus Hagander<magnus@...> wrote:
>
>  
>> FWIW, it seems the version that Andrew put up doesn't work in one of
>> my test environments, and also not in at last one of Dave's. I will
>> test it in my second test environment later today to be sure.
>>    
>
> It doesn't work in any of my bf animals, or build machines :-(
>  

Darn. I'm working on it.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Tom Lane-2]

Dave Page <dpage@...> writes:
> It doesn't work in any of my bf animals, or build machines :-(

?? narwhal seems to have gone green.

                        regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Dave Page-7]

On Wed, Jul 22, 2009 at 2:16 PM, Tom Lane<tgl@...> wrote:
> Dave Page <dpage@...> writes:
>> It doesn't work in any of my bf animals, or build machines :-(
>
> ?? narwhal seems to have gone green.

Narwhal is mingw/msys. The misbehaving flex is the one provided by
Andrew for use with VC++ (where the aim is to avoid having mingw/msys
of course).


--
Dave Page
EnterpriseDB UK:   http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Andrew Dunstan]

Tom Lane wrote:
> Dave Page <dpage@...> writes:
>  
>> It doesn't work in any of my bf animals, or build machines :-(
>>    
>
> ?? narwhal seems to have gone green.
>  


Yeah, the problem is with MSVC, Narwal is a Mingw box.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Greg Smith-12]

I got bit by this tonight as part of testing a patch on CentOS 5, which
like RHEL 5 still ships flex 2.5.4.  I just wrote a little guide on how to
grab a source RPM from a Fedora version and install it to work around that
problem:
http://notemagnet.blogspot.com/2009/07/upgrading-flex-from-source-rpm-to.html

Kind of annoying, but as special software you have to install on a server
just to build something from CVS goes it's only a minor inconvenience.

--
* Greg Smith gsmith@... http://www.gregsmith.com Baltimore, MD

--
Sent via pgsql-hackers mailing list (pgsql-hackers@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Dave Page-7]

On Thu, Jul 23, 2009 at 2:57 AM, Greg Smith<gsmith@...> wrote:
> I got bit by this tonight as part of testing a patch on CentOS 5, which like
> RHEL 5 still ships flex 2.5.4.  I just wrote a little guide on how to grab a
> source RPM from a Fedora version and install it to work around that problem:
> http://notemagnet.blogspot.com/2009/07/upgrading-flex-from-source-rpm-to.html
>
> Kind of annoying, but as special software you have to install on a server
> just to build something from CVS goes it's only a minor inconvenience.

*sigh*. So does OS X Tiger. And (unsurprisingly), CentOS 4, which is
what we use for all our Linux build servers.

This is getting quite tiresome :-(

--
Dave Page
EnterpriseDB UK:   http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Magnus Hagander-2]

On Thu, Jul 23, 2009 at 10:52, Dave Page<dpage@...> wrote:

> On Thu, Jul 23, 2009 at 2:57 AM, Greg Smith<gsmith@...> wrote:
>> I got bit by this tonight as part of testing a patch on CentOS 5, which like
>> RHEL 5 still ships flex 2.5.4.  I just wrote a little guide on how to grab a
>> source RPM from a Fedora version and install it to work around that problem:
>> http://notemagnet.blogspot.com/2009/07/upgrading-flex-from-source-rpm-to.html
>>
>> Kind of annoying, but as special software you have to install on a server
>> just to build something from CVS goes it's only a minor inconvenience.
>
> *sigh*. So does OS X Tiger. And (unsurprisingly), CentOS 4, which is
> what we use for all our Linux build servers.
>
> This is getting quite tiresome :-(

Much as I dislike it, we may need to revisit the idea about putting
the flex output files in CVS...


--
 Magnus Hagander
 Self: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

--
Sent via pgsql-hackers mailing list (pgsql-hackers@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Andrew Dunstan]

Magnus Hagander wrote:

> On Thu, Jul 23, 2009 at 10:52, Dave Page<dpage@...> wrote:
>  
>> On Thu, Jul 23, 2009 at 2:57 AM, Greg Smith<gsmith@...> wrote:
>>    
>>> I got bit by this tonight as part of testing a patch on CentOS 5, which like
>>> RHEL 5 still ships flex 2.5.4.  I just wrote a little guide on how to grab a
>>> source RPM from a Fedora version and install it to work around that problem:
>>> http://notemagnet.blogspot.com/2009/07/upgrading-flex-from-source-rpm-to.html
>>>
>>> Kind of annoying, but as special software you have to install on a server
>>> just to build something from CVS goes it's only a minor inconvenience.
>>>      
>> *sigh*. So does OS X Tiger. And (unsurprisingly), CentOS 4, which is
>> what we use for all our Linux build servers.
>>
>> This is getting quite tiresome :-(
>>    
>
> Much as I dislike it, we may need to revisit the idea about putting
> the flex output files in CVS...
>
>  

Why? This only affects developers building from a CVS pull. You don't
need any flex at all to build from a tarball. If developers can't
install flex on a *nix box they need to get out of the business.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Tom Lane-2]

Andrew Dunstan <andrew@...> writes:
> Magnus Hagander wrote:
>> Much as I dislike it, we may need to revisit the idea about putting
>> the flex output files in CVS...

> Why? This only affects developers building from a CVS pull. You don't
> need any flex at all to build from a tarball. If developers can't
> install flex on a *nix box they need to get out of the business.

I wonder if it would be helpful to have a buildfarm option whereby
it would fetch the latest nightly-snapshot tarball and use that instead
of a CVS pull.  This would have the dual advantage of actually testing
builds from tarballs and requiring less stuff on the buildfarm machine.
It wouldn't be useful for more-than-once-a-day builds, but a lot of
the machines only build that often anyhow.

                        regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Andrew Dunstan]

Tom Lane wrote:

> Andrew Dunstan <andrew@...> writes:
>  
>> Magnus Hagander wrote:
>>    
>>> Much as I dislike it, we may need to revisit the idea about putting
>>> the flex output files in CVS...
>>>      
>
>  
>> Why? This only affects developers building from a CVS pull. You don't
>> need any flex at all to build from a tarball. If developers can't
>> install flex on a *nix box they need to get out of the business.
>>    
>
> I wonder if it would be helpful to have a buildfarm option whereby
> it would fetch the latest nightly-snapshot tarball and use that instead
> of a CVS pull.  This would have the dual advantage of actually testing
> builds from tarballs and requiring less stuff on the buildfarm machine.
> It wouldn't be useful for more-than-once-a-day builds, but a lot of
> the machines only build that often anyhow.
>
>
>  

That is certainly doable. It would be in effect a forced run, because we
would have no notion of what had changed. Presumably, this would only be
for HEAD - we don't do daily snapshots of the back branches, do we?

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Tom Lane-2]

Andrew Dunstan <andrew@...> writes:
> Tom Lane wrote:
>> I wonder if it would be helpful to have a buildfarm option whereby
>> it would fetch the latest nightly-snapshot tarball and use that instead
>> of a CVS pull.

> That is certainly doable. It would be in effect a forced run, because we
> would have no notion of what had changed. Presumably, this would only be
> for HEAD - we don't do daily snapshots of the back branches, do we?

There was just recently some discussion of providing daily snapshots
for the most recent back branch or two.  I'm not sure if that's up
and running, but the consensus seemed to be that it was a good idea.

                        regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers