Nabble - Pike

2 segfaults

2009-12-07T17:42:20Z

While playing around with Debug.locate_references i noticed that it
fails on ({}). The problem seems to be that in
print_short_svalue_compact the svalue is not completely initialized and
at some later point s->subtype is added to a pointer. 1-line patch attached.

There is another problem when the master is printed during a run of
locate_references. As the one before it can be triggered by
Debug.locate_references(({})) but only after fixing the first bug.
The current_path variable inherited from the 7.4 compat master is type
PIKE_T_FREE for some reason and describe_svalue doesnt like that. If
pike is compiled with dmalloc, pike segfaults because the free location
of current_path seems to be uninitialized, too. Not sure where to fix that.

arne

diff --git a/src/svalue.c b/src/svalue.c
index a6b07d1..273fc61 100644
--- a/src/svalue.c
+++ b/src/svalue.c
@@ -1871,6 +1871,7 @@ PMOD_EXPORT void print_short_svalue_compact (FILE *out, const union anything *a,
else {
struct svalue sval;
sval.type = type;
+ sval.subtype = 0;
sval.u = *a;
print_svalue_compact (out, &sval);
}

From forum: Pike - Dev

Re: UNIX socket support broken

2009-12-07T05:46:28Z

On Mon, 7 Dec 2009, Pontus Rodling wrote:

> Hi!
> It seems that some updates related to path length limits in
> modules/files/file.c and modules/files/socket.c broke the UNIX socket
> support.
> More specifically Stdio.Port()->bind_unix() and Stdio.File()->connect_unix().
>
> v1.416 in file.c and v1.104 in socket.c.
> Commit message: "Fixed length-limit in bind_unix(). Added automatic retry on
> EINTR to bind_unix()."
>
> Calling Stdio.Port()->bind_unix(...) returned 0 and strerror(errno()) gave me
> "Invalid argument", I didn't check the errno from Stdio.File()->connect_unix
> but I'm guessing it returned the same thing.

Thanks, there was a trivial typo in bind_unix(), which caused it to
always fail.

I couldn't reproduce any problems with connect_unix().

> I tried reverting the code in those commits and it solved the problem.
>
> // Pontus Rodling

--
Henrik Grubbström grubba@...
Roxen Internet Software AB

From forum: Pike - User

UNIX socket support broken

2009-12-06T17:54:32Z

Hi!
It seems that some updates related to path length limits in
modules/files/file.c and modules/files/socket.c broke the UNIX socket
support.
More specifically Stdio.Port()->bind_unix() and
Stdio.File()->connect_unix().

v1.416 in file.c and v1.104 in socket.c.
Commit message: "Fixed length-limit in bind_unix(). Added automatic
retry on EINTR to bind_unix()."

Calling Stdio.Port()->bind_unix(...) returned 0 and strerror(errno())
gave me "Invalid argument", I didn't check the errno from
Stdio.File()->connect_unix but I'm guessing it returned the same thing.

I tried reverting the code in those commits and it solved the problem.

// Pontus Rodling

From forum: Pike - User

Re: Pike crash

2009-12-04T14:02:52Z

In the last episode (Dec 04), Daniel Bengtsson said:

> On Thu, 03 Dec 2009 21:10:54 +0100, Dan Nelson <dnelson@...> wrote:
> > In the last episode (Dec 03), Daniel Bengtsson said:
> >> I am having problem with a Pike script crashing Pike. The problem
> >> happens on a windows machine (vista), the same script has never caused
> >> any issues on linux. Using pike 7.6 I saw no output at all when it
> >> happened, but using 7.8.352 I get the following message in the console:
> >>
> >> "This application has requested the Runtime to terminate it in an
> >> unusual way. Please contact the application's support team for more
> >> information"
> >>
> >> According to log messages it suggests that the last operation might be
> >> a call to mv on a directory.
> >>
> >> Any suggestions how to debug this or find out what the problem is ?
> >
> > Event Viewer might have more information regarding the location of the
> > crash. You could also install the Debugging Tools for Windows and use
> > windbg to attach to the process and see what the call stack looks like.
> >
> > http://www.microsoft.com/whdc/devtools/debugging/
>
> In the event viewer I can see this at the same timestamp as the crash:
>
> "Faulting application pike.exe, version 0.0.0.0, time stamp 0x4aba8624,
> faulting module pike.exe, version 0.0.0.0, time stamp 0x4aba8624,
> exception code 0x40000015, fault offset 0x000faf4a, process id 0xf30,
> application start time 0x01ca739c94dc9180."

Google says that 0x40000015 is STATUS_FATAL_APP_EXIT "Fatal Application
Exit", which doesn't say all that much. Running pike from within windbg (or
attaching windbg to a running pike before it crashes) will let you get the
stacks for the faulting thread. I think the "!analyze" command is all you
have to run once it faults. If "!analyze" doesn't print anything
interesting, "~* kp" should at least print a stack trace for all threads.
The windows pike distribution includes a symbol file, so the stack trace
should have function names and arguments in it.

--
Dan Nelson
dnelson@...

From forum: Pike - User

eureka reinstall

2009-12-04T06:30:10Z

That got postponed dur to me getting sick during the planned service
window, but there will be another try in an hour or so. This time I'll
split the mirror try and prepare the new OS on the detached
disk in another machine, so there should be very little downtime
today.

From forum: Pike - Dev

Counting the indices of a mapping

2009-12-04T04:00:04Z

Apply sizeof() to each value of the top-level mapping and add the
result together:

`+(0, @map(values(foo), sizeof))

(I've added the extra 0 to handle the case of foo being empty.)

From forum: Pike - User

Re: Pike crash

2009-12-04T03:57:26Z

On Thu, 03 Dec 2009 21:10:54 +0100, Dan Nelson <dnelson@...>
wrote:

> In the last episode (Dec 03), Daniel Bengtsson said:
>> I am having problem with a Pike script crashing Pike. The problem
>> happens
>> on a windows machine (vista), the same script has never caused any
>> issues
>> on linux. Using pike 7.6 I saw no output at all when it happened, but
>> using 7.8.352 I get the following message in the console:
>>
>> "This application has requested the Runtime to terminate it in an
>> unusual
>> way. Please contact the application's support team for more information"
>>
>> According to log messages it suggests that the last operation might be a
>> call to mv on a directory.
>>
>> Any suggestions how to debug this or find out what the problem is ?
>
> Event Viewer might have more information regarding the location of the
> crash. You could also install the Debugging Tools for Windows and use
> windbg to attach to the process and see what the call stack looks like.
>
> http://www.microsoft.com/whdc/devtools/debugging/

In the event viewer I can see this at the same timestamp as the crash:

"Faulting application pike.exe, version 0.0.0.0, time stamp 0x4aba8624,
faulting module pike.exe, version 0.0.0.0, time stamp 0x4aba8624,
exception code 0x40000015, fault offset 0x000faf4a, process id 0xf30,
application start time 0x01ca739c94dc9180."

/Daniel

From forum: Pike - User

Counting the indices of a mapping

2009-12-04T03:40:45Z

I know you can easily get the indices of a mapping with indices( mapping
foo ).

Is there an east/fast way to get the second level of indices, or to just
count the number of indices of a nested mapping?

For example, assume the following:

mapping foo =
([
"element a" :
([
"foo" : 1,
"bar" : 2
])

"element b" :
([
"stuff" : 1,
"more stuff" : 2
])

"element c" :
([
"even more stuff" : 1,
"mega stuff" : 2
])
])

Is there a fast/easy way to count up the second level of indices, where
the result would be 6.

--
Michael A. Hartman, J.D.
President and CEO, Frogdice, Inc.
http://www.frogdice.com

From forum: Pike - User

sscanf parttern matching (was bug?)

2009-12-03T18:27:45Z

On Thu, Dec 03, 2009 at 10:55:02PM +0000, Martin Stjernholm, Roxen IS @ Pike developers forum wrote:
> If there was backtracking, the second case would return ({"abc-"})
> since %d didn't parse "-" successfully. Now it's just lost.

on a related topic: is there a way to find out if the format has been
parsed successfully? ie. in the above example, i'd like to get an error
instead of an incomplete result. yes, i could check manually, but that
implies that i understand or know the format, which is not aleays the
case as it could come from somewhere else, even from user input.

i'd like to use sscanf as an alternative to regexp pattern matching

greetings, martin.

From forum: Pike - Dev

sscanf bug?

2009-12-03T14:55:02Z

Yes, I guess it should. Wonder how much of a compatibility problem it
would be to change that.

Personally I don't really like the dwim in %s in situations like that;
I prefer to use %[...] instead. %s essentially becomes e.g.
%[^0-9+-] if followed by %d, but there's no full backtracking if the
following input specifier fails. Compare:

> array_sscanf("abc-123", "%s%d");
(2) Result: ({ /* 2 elements */
"abc",
-123
})
> array_sscanf("abc-", "%s%d");
(3) Result: ({ /* 1 element */
"abc"
})

If there was backtracking, the second case would return ({"abc-"})
since %d didn't parse "-" successfully. Now it's just lost.

From forum: Pike - Dev

Re: Pike crash

2009-12-03T12:10:54Z

In the last episode (Dec 03), Daniel Bengtsson said:

> I am having problem with a Pike script crashing Pike. The problem happens
> on a windows machine (vista), the same script has never caused any issues
> on linux. Using pike 7.6 I saw no output at all when it happened, but
> using 7.8.352 I get the following message in the console:
>
> "This application has requested the Runtime to terminate it in an unusual
> way. Please contact the application's support team for more information"
>
> According to log messages it suggests that the last operation might be a
> call to mv on a directory.
>
> Any suggestions how to debug this or find out what the problem is ?

Event Viewer might have more information regarding the location of the
crash. You could also install the Debugging Tools for Windows and use
windbg to attach to the process and see what the call stack looks like.

http://www.microsoft.com/whdc/devtools/debugging/

--
Dan Nelson
dnelson@...

From forum: Pike - User

Pike crash

2009-12-03T03:21:45Z

Hi,

I am having problem with a Pike script crashing Pike. The problem happens
on a windows machine (vista), the same script has never caused any issues
on linux. Using pike 7.6 I saw no output at all when it happened, but
using 7.8.352 I get the following message in the console:

"This application has requested the Runtime to terminate it in an unusual
way. Please contact the application's support team for more information"

According to log messages it suggests that the last operation might be a
call to mv on a directory.

Any suggestions how to debug this or find out what the problem is ?

/Daniel

From forum: Pike - User

sscanf bug?

2009-12-02T23:30:30Z

hi,

shouldn't those two examples be the same?

> array_sscanf("124dsf134", "%d%s%3d");
(5) Result: ({ /* 2 elements */
124,
"dsf134"
})
> array_sscanf("124dsf134", "%d%s%d");
(6) Result: ({ /* 3 elements */
124,
"dsf",
134
})

greetings, martin.

From forum: Pike - Dev

GTK2.FileChooserDialog missing in latest snapshot?

2009-12-02T12:32:38Z

Hey all.

I can't figure out why, but it seems that GTK2.FileChooserDialog is missing in the latest snapshot. I get this error when using it:

Too many arguments to `() (function call) (expected 0 arguments)

and indeed, the commandline pike version does not suggest the FileChooserDialog either, when trying to autocomplete.

I compiled the latest stable build again on the same computer, and it works again. (Perhaps I should learn to use stable builds as long as I'm a beginner...)

Regards,

yvan

--
Copyright only exists in the imagination of those who do not have any.

From forum: Pike - User

Re: More rtldebug to get better info when gc_mark problems happen.

2009-11-30T05:35:03Z

Pikefarm spotted it form me; it broke 11 different builds. ;-)

From forum: Pike - Dev

Re: More rtldebug to get better info when gc_mark problems happen.

2009-11-30T05:14:35Z

Quite right. Thanks for spotting it.

From forum: Pike - Dev

More rtldebug to get better info when gc_mark problems happen.

2009-11-30T04:00:03Z

Um, the latest commit to Makefile (by mast) was a mistake, no?

From forum: Pike - Dev

Re: segfault when trying out embedding demo program

2009-11-30T02:19:00Z

On Fri, 27 Nov 2009, H. William Welliver III wrote:

> Oh, I should also probably mention 2 things: first, I'm actively using
> embedded pike with good success using the Objective-C OCPikeInterpreter class
> to build Native Mac OSX GUI applications and second, that I have difficulty
> getting pike to exit happily at times... usually things end with pike
> thinking a fatal error has occurred and that there is no error handling
> context, even though there isn't actually an error that I'm aware of. If you

Note that predef::exit() is implemented via a throw of an error with
severity THROW_EXIT, which won't be caught by normal catches. Take a look
at the code in pike_embed.c:init_pike_runtime() to see how to catch those.

> do see this problem, do post about it and maybe we can figure out what we're
> doing wrong. :)
>
> Good luck!
>
> Bill

--
Henrik Grubbström grubba@...
Roxen Internet Software AB

From forum: Pike - User

patch for string_builder_vsprintf

2009-11-30T02:10:03Z

Applied similar patch (with a bugfix regarding handling of 0.0).
Thanks.

From forum: Pike - Dev

The OOB working test

2009-11-28T07:30:02Z

The difference is not between 7.6 and 7.8. I don't know what it is
though; it doesn't seem just random either.

From forum: Pike - Dev

patch for string_builder_vsprintf

2009-11-28T07:20:03Z

Looks ok to me, but I've only checked it superficially. Maybe Grubba
who wrote that function should have a say.

From forum: Pike - Dev

Protocols.DNS.server

2009-11-28T06:44:08Z

I've added multi-port-functionality and the ability to reply to DNS
queries asynchronously to Protocols.DNS.server.
A diff file is attached.

Tobias

diff --git a/modules/Protocols.pmod/DNS.pmod b/modules/Protocols.pmod/DNS.pmod
index c63dec7..0f74b9f 100644
--- a/modules/Protocols.pmod/DNS.pmod
+++ b/modules/Protocols.pmod/DNS.pmod
@@ -623,9 +623,9 @@ class server
//!
inherit protocol;

- inherit Stdio.UDP : udp;
+ //inherit Stdio.UDP : udp;

- protected void send_reply(mapping r, mapping q, mapping m)
+ protected void send_reply(mapping r, mapping q, mapping m, Stdio.UDP udp)
{
// FIXME: Needs to handle truncation somehow.
if(!r)
@@ -636,7 +636,7 @@ class server
r->rd = q->rd;
r->qd = r->qd || q->qd;
string s = mkquery(r);
- udp::send(m->ip, m->port, s);
+ udp->send(m->ip, m->port, s);
}

//! Reply to a query (stub).
@@ -647,6 +647,10 @@ class server
//! @param udp_data
//! Raw UDP data.
//!
+ //! @param cb
+ //! Callback you can call with the result instead of returning it.
+ //! In that case, return @expr{0@} (zero).
+ //!
//! Overload this function to implement the proper lookup.
//!
//! @returns
@@ -666,7 +670,8 @@ class server
//! @member array|void "ns"
//! @member array|void "ar"
//! @endmapping
- protected mapping reply_query(mapping query, mapping udp_data)
+ protected mapping reply_query(mapping query, mapping udp_data,
+ function(mapping:void) cb)
{
// Override this function.
//
@@ -676,18 +681,32 @@ class server
return 0;
}

- protected void handle_query(mapping q, mapping m)
+ protected void handle_query(mapping q, mapping m, Stdio.UDP udp)
{
- mapping r = reply_query(q, m);
- send_reply(r, q, m);
+ int(0..1) result_available = 0;
+ void _cb(mapping r) {
+ if(result_available)
+ error("DNS: you can either use the callback OR return the reply, not "
+ "both.\n");
+ result_available = 1;
+ send_reply(r, q, m, udp);
+ };
+ mapping r = reply_query(q, m, _cb);
+ if(r && result_available)
+ error("DNS: you can either use the callback OR return the reply, not "
+ "both.\n");
+ if(r) {
+ result_available = 1;
+ send_reply(r, q, m, udp);
+ }
}

- protected void handle_response(mapping r, mapping m)
+ protected void handle_response(mapping r, mapping m, Stdio.UDP udp)
{
// This is a stub intended to simplify servers which allow recursion
}

- protected private void rec_data(mapping m)
+ protected private void rec_data(mapping m, Stdio.UDP udp)
{
mixed err;
mapping q;
@@ -698,27 +717,40 @@ class server
describe_backtrace(err));
if(m && m->data && sizeof(m->data)>=2)
send_reply((["rcode":1]),
- mkmapping(({"id"}), array_sscanf(m->data, "%2c")), m);
+ mkmapping(({"id"}), array_sscanf(m->data, "%2c")), m, udp);
}
else if(q->qr)
- handle_response(q, m);
+ handle_response(q, m, udp);
else
- handle_query(q, m);
+ handle_query(q, m, udp);
}

- void create(int|void port)
+ void create(int|string|void arg1, string|int ... args)
{
- if(!port)
- port = 53;
- if(!udp::bind(port))
- error("DNS: failed to bind port "+port+".\n");
-#if 0
- werror("Protocols.DNS.server(%O)\n"
- "UDP Address: %s\n"
- "%s\n", port, udp::query_address(),
- describe_backtrace(backtrace()));
-#endif /* 0 */
- udp::set_read_callback(rec_data);
+ if(!arg1 && !sizeof(args))
+ arg1 = 53;
+ if(!sizeof(args))
+ if(stringp(arg1))
+ args = ({ arg1, 53 });
+ else
+ args = ({ 0, arg1 });
+ else
+ args = ({ arg1 }) + args;
+ if(sizeof(args)&1)
+ error("DNS: if you specify more than one argument, the number of "
+ "arguments needs to be even (server(ip1, port1, ip2, port2, "
+ "...)).\n");
+ for(int i;i<sizeof(args);i+=2) {
+ Stdio.UDP udp = Stdio.UDP();
+ if(args[i]) {
+ if (!udp->bind(args[i+1],args[i]))
+ error("DNS: failed to bind host:port %s:%d.\n", args[i],args[i+1]);
+ } else {
+ if(!udp->bind(args[i+1]))
+ error("DNS: failed to bind port %d.\n", args[i+1]);
+ }
+ udp->set_read_callback(rec_data, udp);
+ }
}

}

From forum: Pike - Dev

The OOB working test

2009-11-28T05:05:03Z

My guess is that the set of detected operating system features differs
between the two. Try comparing the output from running just the
preprocessor.

From forum: Pike - Dev

The OOB working test

2009-11-28T04:30:02Z

I noticed that the OOB check fails with TIMEOUT in 7.8 on my linux
box. It produces SEMIWORKING in 7.6, so something is different in the
configure script, although the test itself is identical.

The log doesn't say much. Here's from 7.8:

configure:70034: checking how well OOB handling works
configure:70366: /home/mast/Pike/no-changes-7.8/build/linux-x86_64/smartlink gcc -o conftest -g -O3 -pipe -ggdb3 -m64 -I/home/mast/Pike/no-changes-7.8/build/linux-x86_64/bundles/include -I/usr/local/include -I/home/mast/Pike/no-changes-7.8/src -I/home/mast/Pike/no-changes-7.8/build/linux-x86_64 -L/home/mast/Pike/no-changes-7.8/build/linux-x86_64/bundles/lib64 -L/home/mast/Pike/no-changes-7.8/build/linux-x86_64/bundles/lib/64 -L/home/mast/Pike/no-changes-7.8/build/linux-x86_64/bundles/lib/. -L/usr/local/lib -L/usr/lib/gcc/x86_64-linux-gnu/4.4 -R/usr/lib/gcc/x86_64-linux-gnu/4.4 conftest.c -ldl -lrt -lnsl -lm -lpthread -lcrypt >&5
conftest.c: In function 'my_socketpair':
conftest.c:362: warning: incompatible implicit declaration of built-in function 'memset'
conftest.c: In function 'ping':
conftest.c:527: warning: incompatible implicit declaration of built-in function 'exit'
conftest.c:535: warning: incompatible implicit declaration of built-in function 'exit'
conftest.c: In function 'pong':
conftest.c:552: warning: incompatible implicit declaration of built-in function 'exit'
conftest.c: In function 'pang':
conftest.c:570: warning: incompatible implicit declaration of built-in function 'exit'
conftest.c: In function 'main':
conftest.c:591: warning: incompatible implicit declaration of built-in function 'exit'
configure:70366: $? = 0
configure:70366: ./conftest
my_socketpair: succeeded
configure:70366: $? = 0
configure:70381: result: TIMEOUT

Compared to 7.6:

configure:50000: checking how well OOB handling works
configure:50342: /home/mast/Pike/frozen/build/linux-x86_64/smartlink gcc -o conftest -g -O2 -pipe -Wa,--execstack -I/home/mast/Pike/frozen/build/linux-x86_64/bundles/include -I/usr/local/include -I/home/mast/Pike/frozen/src -I/home/mast/Pike/frozen/build/linux-x86_64 -L/home/mast/Pike/frozen/build/linux-x86_64/bundles/lib -z execstack -R/usr/local/lib -L/usr/local/lib -R/usr/X11R6/lib -L/usr/X11R6/lib -R/usr/lib/gcc/x86_64-linux-gnu/4.4 -L/usr/lib/gcc/x86_64-linux-gnu/4.4 conftest.c -ldl -lrt -lnsl -lm -lpthread -lcrypt >&5
conftest.c: In function 'my_socketpair':
conftest.c:333: warning: incompatible implicit declaration of built-in function 'memset'
conftest.c: In function 'ping':
conftest.c:498: warning: incompatible implicit declaration of built-in function 'exit'
conftest.c:506: warning: incompatible implicit declaration of built-in function 'exit'
conftest.c: In function 'pong':
conftest.c:523: warning: incompatible implicit declaration of built-in function 'exit'
conftest.c: In function 'pang':
conftest.c:541: warning: incompatible implicit declaration of built-in function 'exit'
conftest.c: In function 'main':
conftest.c:562: warning: incompatible implicit declaration of built-in function 'exit'
configure:50345: $? = 0
configure:50351: ./conftest
my_socketpair: succeeded
configure:50354: $? = 0
configure:50375: result: SEMIWORKING

I've tried changing the compiler flags to match 7.6 (-O2 instead of
-O3 etc), but it doesn't change anything. What could the problem be?

From forum: Pike - Dev

Re: segfault when trying out embedding demo program

2009-11-26T21:05:34Z

Oh, I should also probably mention 2 things: first, I'm actively using
embedded pike with good success using the Objective-C
OCPikeInterpreter class to build Native Mac OSX GUI applications and
second, that I have difficulty getting pike to exit happily at
times... usually things end with pike thinking a fatal error has
occurred and that there is no error handling context, even though
there isn't actually an error that I'm aware of. If you do see this
problem, do post about it and maybe we can figure out what we're doing
wrong. :)

Good luck!

Bill

From forum: Pike - User

Re: segfault when trying out embedding demo program

2009-11-26T21:00:18Z

I just verified that example 2 does work (on OSX 10.6). Note that in the examples, you have to set the PIKE_MASTER environment variable, otherwise you will get a segfault. I imagine that this may be the problem. Note that push_pike_env() is no longer present, so you should remove that line from the examples.

I don't know if there's any very good docs on embedding; basically you'll be using the same API you'd use for building c-modules. You'll probably also need to become familiar with apply() and friends. I'd imagine that you would want to build a c-module to provide an interface in pike for anything in the embedding application that you'd want to access from pike. From there, it's up to you.

You might want to look at OCPikeInterpreter.m in the src directory; it's Objective-C, but should give you some idea of how to run pike code from C (in particular the call -evalString:.

Bill

On Nov 26, 2009, at 1:13 PM, Yvan Vander Sanden wrote:

I got my pike library ready now, and had some time to try out the actual embedding.

I took the example program at http://hww3.riverweb.com/dist/pike_embedding/

Compiling and linking went ok, but the test program just segfaults. this is how i compile:

gcc -I src/ -I build/linux -c main.c -o main.o
gcc main.o -o testpike -Lbuild/linux -lpike

libpike is build/linux as well as in /lib

I am using a recent code snapshot and pike itself does work.

- Is the example program still current?
- Is there any good documentation for embedding pike, except from the link i mentioned above?

Regards,

yvan

--
Copyright only exists in the imagination of those who do not have any.

From forum: Pike - User

segfault when trying out embedding demo program

2009-11-26T10:13:05Z

I got my pike library ready now, and had some time to try out the actual embedding.

I took the example program at http://hww3.riverweb.com/dist/pike_embedding/

Compiling and linking went ok, but the test program just segfaults. this is how i compile:

gcc -I src/ -I build/linux -c main.c -o main.o
gcc main.o -o testpike -Lbuild/linux -lpike

libpike is build/linux as well as in /lib

I am using a recent code snapshot and pike itself does work.

- Is the example program still current?
- Is there any good documentation for embedding pike, except from the link i mentioned above?

Regards,

yvan

--
Copyright only exists in the imagination of those who do not have any.

From forum: Pike - User

patch for string_builder_vsprintf

2009-11-23T09:24:36Z

Hello,

I noticed that only %f is supported in string_builder_vsprintf(). I
would like to use something else in JSON2, so it would be nice if
someone could review this and maybe commit.

arne

diff --git a/src/stralloc.c b/src/stralloc.c
index eb60b8b..e75d45b 100644
--- a/src/stralloc.c
+++ b/src/stralloc.c
@@ -2945,13 +2945,17 @@ PMOD_EXPORT void string_builder_vsprintf(struct string_builder *s,
flags, min_width, precision);
break;

- /* FIMXE: TODO: Doubles (ie 'a', 'e', 'E', 'g', 'G'). */
-
/* %f used in modules/Image/colors.c. */
+ case 'a':
+ case 'g':
+ case 'G':
+ case 'e':
+ case 'E':
case 'f':
{
double val = va_arg(args, double);
size_t bytes;
+ char nfmt[] = { '%', *(fmt-1), 0 };

if (PIKE_ISNAN(val)) {
/* NaN */
@@ -2974,9 +2978,9 @@ PMOD_EXPORT void string_builder_vsprintf(struct string_builder *s,
break;
}
/* FIXME: Field lengths and precision. */
- if ((bytes = SNPRINTF(NULL, 0, "%f", val))) {
+ if ((bytes = SNPRINTF(NULL, 0, nfmt, val))) {
p_wchar0 *p = string_builder_allocate(s, bytes, 0);
- size_t check = SNPRINTF((char*)p, bytes+1, "%f", val);
+ size_t check = SNPRINTF((char*)p, bytes+1, nfmt, val);
if (check != bytes) {
Pike_fatal("string_builder_vsprintf(): snprintf(%f) is not "
"trustworthy: %"PRINTSIZET"u != %"PRINTSIZET"u\n",

From forum: Pike - Dev

Re: mysql charset issue

2009-11-14T17:10:12Z

> Great, seems to work.

Ok, good. Fix is committed to 7.4, 7.6 and 7.8.

> For me the charset is going into the connect options (i.e.
> RECONNECT_CHARSET_IS_SET), i guess

That's right, I changed that too.

From forum: Pike - Dev

Re: do_async_method_url ignoring content-type

2009-11-14T14:10:02Z

> So i guess it would be better to do a set_write_callback(0) after
> write if !sizeof(out_buffer) and to reinstall the callback when more
> data is available?

..and remove the initial callback install in create(). Yes, that is
what I'd do too.

From forum: Pike - Dev

Re: do_async_method_url ignoring content-type

2009-11-14T13:54:14Z

Martin Stjernholm, Roxen IS @ Pike developers forum wrote:
>> The send_output method which writes to the buffer triggers a write
>> attempt. So that should not be a problem.
>
> I see. Then I guess it works, although personally I think it's a bit
> obscure to rely on the internal write cb blocker. Anyway, I've applied
> your patch. Thanks.

Its what I usually do. So i guess it would be better to do a
set_write_callback(0) after write if !sizeof(out_buffer) and to
reinstall the callback when more data is available?

From forum: Pike - Dev

Re: mysql charset issue

2009-11-14T13:52:03Z

Martin Stjernholm wrote:
> Aha, I somehow thought the setting would survive the reconnect if
> mysql_set_character_set is used. Of course it doesn't. Then the
> approach is a bit different. Please try this patch instead.
>
>

Great, seems to work. For me the charset is going into the connect
options (i.e. RECONNECT_CHARSET_IS_SET), i guess, so i am not seeing any
SET NAMES magic after reconnect.

From forum: Pike - Dev

Re: do_async_method_url ignoring content-type

2009-11-14T13:35:02Z

> The send_output method which writes to the buffer triggers a write
> attempt. So that should not be a problem.

I see. Then I guess it works, although personally I think it's a bit
obscure to rely on the internal write cb blocker. Anyway, I've applied
your patch. Thanks.

From forum: Pike - Dev

Re: mysql charset issue

2009-11-14T13:22:38Z

Aha, I somehow thought the setting would survive the reconnect if
mysql_set_character_set is used. Of course it doesn't. Then the
approach is a bit different. Please try this patch instead.

Index: src/modules/Mysql/mysql.c =================================================================== RCS file: /pike/data/cvsroot/Pike/7.8/src/modules/Mysql/mysql.c,v retrieving revision 1.119 diff -u -r1.119 mysql.c --- src/modules/Mysql/mysql.c 5 Nov 2009 14:15:15 -0000 1.119 +++ src/modules/Mysql/mysql.c 14 Nov 2009 21:15:35 -0000 @@ -157,6 +157,9 @@ #define MYSQL_DISALLOW() #endif /* _REENTRANT */ +#define PIKE_MYSQL_FLAG_STORE_RESULT 1 +#define PIKE_MYSQL_FLAG_TYPED_RESULT 2 + #define CHECK_8BIT_NONBINARY_STRING(FUNC, ARG) do { \ if (sp[ARG-1-args].type != T_STRING || \ sp[ARG-1-args].u.string->size_shift || \ @@ -214,12 +217,10 @@ free_mapping (PIKE_MYSQL->options); PIKE_MYSQL->options = NULL; } -#ifndef HAVE_MYSQL_SET_CHARACTER_SET if (PIKE_MYSQL->conn_charset) { free_string (PIKE_MYSQL->conn_charset); PIKE_MYSQL->conn_charset = NULL; } -#endif MYSQL_ALLOW(); @@ -347,14 +348,48 @@ val->u.string->str); } #endif /* MYSQL_SET_CHARSET_DIR */ -#ifdef HAVE_MYSQL_SET_CHARSET_NAME +#endif /* HAVE_MYSQL_OPTIONS */ + if ((val = simple_mapping_string_lookup(options, "mysql_charset_name")) && - (val->type == T_STRING) && (!val->u.string->size_shift)) { - mysql_options(PIKE_MYSQL->mysql, MYSQL_SET_CHARSET_NAME, - val->u.string->str); + (val->type == T_STRING) && (!val->u.string->size_shift) && + !string_has_null (val->u.string)) { + if (PIKE_MYSQL->conn_charset) + free (PIKE_MYSQL->conn_charset); + copy_shared_string (PIKE_MYSQL->conn_charset, val->u.string); } -#endif /* MYSQL_SET_CHARSET_NAME */ -#endif /* HAVE_MYSQL_OPTIONS */ +} + +static void low_query(INT32 args, char *name, int flags); + +static void connection_set_charset() +{ +#ifdef HAVE_MYSQL_SET_CHARACTER_SET + int res; + MYSQL *mysql = PIKE_MYSQL->mysql; + struct pike_string *charset = PIKE_MYSQL->conn_charset; + MYSQL_ALLOW(); + res = mysql_set_character_set (mysql, charset->str); + MYSQL_DISALLOW(); + if (res) { + const char *err; + MYSQL_ALLOW(); + err = mysql_error(mysql); + MYSQL_DISALLOW(); + Pike_error("Setting the charset failed: %s\n", err); + } + +#else + /* Old libs (< 4.1.13) doesn't support changing the connection + * charset. We emulate it by setting the charset ourselves. Note + * that this doesn't work with mysql_real_escape_string, but that + * function isn't used. */ + push_constant_text ("SET NAMES '"); + ref_push_string (PIKE_MYSQL->conn_charset); + push_constant_text ("'"); + f_add (3); + low_query (1, "set_charset", PIKE_MYSQL_FLAG_STORE_RESULT); + pop_stack(); +#endif } static void pike_mysql_reconnect (int reconnect) @@ -404,10 +439,11 @@ options = (unsigned int)val->u.integer; } -#if !defined (HAVE_MYSQL_SET_CHARACTER_SET) && defined (HAVE_MYSQL_OPTIONS) && defined (HAVE_MYSQL_SET_CHARSET_NAME) +#if defined (HAVE_MYSQL_OPTIONS) && defined (HAVE_MYSQL_SET_CHARSET_NAME) if (PIKE_MYSQL->conn_charset) mysql_options (mysql, MYSQL_SET_CHARSET_NAME, PIKE_MYSQL->conn_charset->str); +#define RECONNECT_CHARSET_IS_SET #endif MYSQL_ALLOW(); @@ -488,6 +524,11 @@ } } } + +#ifndef RECONNECT_CHARSET_IS_SET + if (PIKE_MYSQL->conn_charset) + connection_set_charset(); +#endif } /* @@ -642,24 +683,13 @@ pike_mysql_reconnect (0); -#ifndef HAVE_MYSQL_SET_CHARACTER_SET #ifdef HAVE_MYSQL_CHARACTER_SET_NAME - { + if (!PIKE_MYSQL->conn_charset) { const char *charset = mysql_character_set_name (PIKE_MYSQL->mysql); - if (PIKE_MYSQL->conn_charset) - free_string (PIKE_MYSQL->conn_charset); - if (charset) + if (charset) /* Just paranoia; mysql_character_set_name should + * always return a string. */ PIKE_MYSQL->conn_charset = make_shared_string (charset); - else - /* Just paranoia; mysql_character_set_name should always return - * a string. */ - PIKE_MYSQL->conn_charset = NULL; } -#else - if (PIKE_MYSQL->conn_charset) - free_string (PIKE_MYSQL->conn_charset); - PIKE_MYSQL->conn_charset = NULL; -#endif #endif pop_n_elems(args); @@ -861,9 +891,6 @@ pop_n_elems(args); } -#define PIKE_MYSQL_FLAG_STORE_RESULT 1 -#define PIKE_MYSQL_FLAG_TYPED_RESULT 2 - static void low_query(INT32 args, char *name, int flags) { MYSQL *mysql = PIKE_MYSQL->mysql; @@ -1771,58 +1798,20 @@ SIMPLE_ARG_ERROR ("set_charset", 0, "The charset name cannot contain a NUL character."); -#ifdef HAVE_MYSQL_SET_CHARACTER_SET - { - int res; - MYSQL *mysql = PIKE_MYSQL->mysql; - MYSQL_ALLOW(); - res = mysql_set_character_set (mysql, charset->str); - MYSQL_DISALLOW(); - if (res) { - const char *err; - MYSQL_ALLOW(); - err = mysql_error(mysql); - MYSQL_DISALLOW(); - Pike_error("Setting the charset failed: %s\n", err); - } - } - -#else /* !HAVE_MYSQL_SET_CHARACTER_SET */ - push_constant_text ("SET NAMES '"); - ref_push_string (charset); - push_constant_text ("'"); - f_add (3); - low_query (1, "set_charset", PIKE_MYSQL_FLAG_STORE_RESULT); - args++; if (PIKE_MYSQL->conn_charset) free_string (PIKE_MYSQL->conn_charset); copy_shared_string (PIKE_MYSQL->conn_charset, charset); -#endif - + connection_set_charset(); pop_n_elems (args); } static void f_get_charset (INT32 args) { pop_n_elems (args); -#ifdef HAVE_MYSQL_SET_CHARACTER_SET - { - /* mysql_character_set_name should always exist if - * mysql_set_character_set exists. */ - const char *charset = mysql_character_set_name (PIKE_MYSQL->mysql); - if (charset) - push_text (charset); - else - /* Just paranoia; mysql_character_set_name should always return - * a string. */ - push_constant_text ("latin1"); - } -#else if (PIKE_MYSQL->conn_charset) ref_push_string (PIKE_MYSQL->conn_charset); else push_constant_text ("latin1"); -#endif } static void f__can_send_as_latin1 (INT32 args) Index: src/modules/Mysql/precompiled_mysql.h =================================================================== RCS file: /pike/data/cvsroot/Pike/7.8/src/modules/Mysql/precompiled_mysql.h,v retrieving revision 1.20 diff -u -r1.20 precompiled_mysql.h --- src/modules/Mysql/precompiled_mysql.h 5 Nov 2009 13:49:31 -0000 1.20 +++ src/modules/Mysql/precompiled_mysql.h 14 Nov 2009 21:15:35 -0000 @@ -59,13 +59,7 @@ MYSQL *mysql; struct pike_string *host, *database, *user, *password; /* Reconnect */ struct mapping *options; -#ifndef HAVE_MYSQL_SET_CHARACTER_SET - /* Old libs (< 4.1.13) doesn't support changing the connection - * charset. We emulate it by storing the charset ourselves. Note - * that this doesn't work with mysql_real_escape_string, but that - * function isn't used. */ struct pike_string *conn_charset; -#endif }; struct precompiled_mysql_result {

From forum: Pike - Dev

Re: mysql charset issue

2009-11-14T11:52:53Z

Hm... confusion. the PIKE_MYSQL->conn_charset is used only in case
mysql_set_charset is not available. So maybe instead add it to the mysql
options as if it was given to create? Just speculating here, dont really
know the code. Tell me if there is anything I can do to help you debug.

arne

Arne Goedeke wrote:

> The patch doesnt change anything for me right now. the "old" mysql i was
> running was 5.0.70, not 5.0.7 ;-). So I do have mysql_set_character_set
> and pike is using it. The library sends an SET NAMES utf8 when using
> set_charset("unicode"). After the reconnect it sends a
> SET character_set_client=latin1.
>
> I think the reason for this is, that set_charset() does not change
> PIKE_MYSQL->conn_charset accordingly. I guess its different when I
> create the sql object with the appropriate options without changing the
> charset later on.
>
> arne
>
> Martin Stjernholm, Roxen IS @ Pike developers forum wrote:
>> Hmm, ok. Could you please try this patch without upgrading instead?
>> That way I can verify that it fixes the problem.
>>
>> Index: src/modules/Mysql/mysql.c
>> ===================================================================
>> RCS file: /pike/data/cvsroot/Pike/7.8/src/modules/Mysql/mysql.c,v
>> retrieving revision 1.119
>> diff -u -r1.119 mysql.c
>> --- src/modules/Mysql/mysql.c 5 Nov 2009 14:15:15 -0000 1.119
>> +++ src/modules/Mysql/mysql.c 14 Nov 2009 18:26:17 -0000
>> @@ -157,6 +157,9 @@
>> #define MYSQL_DISALLOW()
>> #endif /* _REENTRANT */
>>
>> +#define PIKE_MYSQL_FLAG_STORE_RESULT 1
>> +#define PIKE_MYSQL_FLAG_TYPED_RESULT 2
>> +
>> #define CHECK_8BIT_NONBINARY_STRING(FUNC, ARG) do { \
>> if (sp[ARG-1-args].type != T_STRING || \
>> sp[ARG-1-args].u.string->size_shift || \
>> @@ -357,6 +360,8 @@
>> #endif /* HAVE_MYSQL_OPTIONS */
>> }
>>
>> +static void low_query(INT32 args, char *name, int flags);
>> +
>> static void pike_mysql_reconnect (int reconnect)
>> {
>> MYSQL *mysql = PIKE_MYSQL->mysql;
>> @@ -488,6 +493,17 @@
>> }
>> }
>> }
>> +
>> +#ifndef HAVE_MYSQL_SET_CHARACTER_SET
>> + if (PIKE_MYSQL->conn_charset) {
>> + push_constant_text ("SET NAMES '");
>> + ref_push_string (PIKE_MYSQL->conn_charset);
>> + push_constant_text ("'");
>> + f_add (3);
>> + low_query (1, "reconnect", PIKE_MYSQL_FLAG_STORE_RESULT);
>> + pop_stack();
>> + }
>> +#endif
>> }
>>
>> /*
>> @@ -861,9 +877,6 @@
>> pop_n_elems(args);
>> }
>>
>> -#define PIKE_MYSQL_FLAG_STORE_RESULT 1
>> -#define PIKE_MYSQL_FLAG_TYPED_RESULT 2
>> -
>> static void low_query(INT32 args, char *name, int flags)
>> {
>> MYSQL *mysql = PIKE_MYSQL->mysql;
>

From forum: Pike - Dev