Re: pqxx and Kexi

View: New views
5 Messages — Rating Filter:   Alert me  

Parent Message unknown Re: pqxx and Kexi

by Adam Pigg :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Thursday 15 October 2009 10:36:22 you wrote:

> Adam Pigg wrote:
> > Hi
>
> Hi Adam!
>
> > Im porting kexi to pqxx 3 from 2.6
> >
> > So far all ive ran into is the lack of pqxx::sqlesc.  I know this has
> > moved to the transaction and connection classes (why?) but in the kexi
> > class hierarchy, i need to escape a string in a class which doesnt have
> > access to a connection object?
>
> This move was made necessary by a security fix in libpq itself.  As it
> turned out, its escaping function needs to know the encoding the string
> is in, because some multibyte encodings have characters that contain the
> byte that in ASCII or UTF-8 would have been e.g. a single quote.
>
> So for example you might have a two-byte character consisting of some
> byte X and one that matches the ASCII character "'": X'
>
> In that case, the naïve previous escaping function would just double
> that byte to "escape" it, producing X'' and voilà: a closing quote has
> been injected into a string--by the exact function that's supposed to
> prevent it.
>
> I think your options are:
>
> a) Write your own escaping function based on your own knowledge of the
> encoding that the code is going to run in.  I suppose Qt has some
> facility for breaking a string in the current locale's encoding down
> into unicode characters.
Jeroen,

Thanks for getting back, is it just the ' character which needs escaped by
doubling it?

In which case, a QString::replace() could suffice, Jaroslaw, the main kexi dev
can advise on that.
>
> b) Give the class that needs this (temporary) access to a connection or
> transaction.
>
The way i made it compile was to have a connection and transaction object, not
actually connected to anything...i guess this is wrong as it isnt  possible to
check the encoding without being connected.

Btw, i needed the transaction object as the connection versions of esc() arnt
const, but transaction has both const and non const, you will know if this is
intentional :)

Cheers

Adam
> c) Postpone the escaping to some point where the connection is available.
>
> Not the best news, I know!  Wish we could have avoided this, but there
> you go.
>
>
> Jeroen
>
_______________________________________________
Kexi mailing list
Kexi@...
https://mail.kde.org/mailman/listinfo/kexi

Re: pqxx and Kexi

by Jaroslaw Staniek :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

2009/10/15 Adam Pigg <piggz1@...>:

>> This move was made necessary by a security fix in libpq itself.  As it
>> turned out, its escaping function needs to know the encoding the string
>> is in, because some multibyte encodings have characters that contain the
>> byte that in ASCII or UTF-8 would have been e.g. a single quote.
>>
>> So for example you might have a two-byte character consisting of some
>> byte X and one that matches the ASCII character "'": X'
>>
>> In that case, the naïve previous escaping function would just double
>> that byte to "escape" it, producing X'' and voilà: a closing quote has
>> been injected into a string--by the exact function that's supposed to
>> prevent it.
>>
>> I think your options are:
>>
>> a) Write your own escaping function based on your own knowledge of the
>> encoding that the code is going to run in.  I suppose Qt has some
>> facility for breaking a string in the current locale's encoding down
>> into unicode characters.
> Jeroen,
>
> Thanks for getting back, is it just the ' character which needs escaped by
> doubling it?
>
> In which case, a QString::replace() could suffice, Jaroslaw, the main kexi dev
> can advise on that.

Guys,
Thanks for continuing the discussion here.
QString::replace() works in test cases but in pessimistic case it's O(n^2).
I do recommend using const QChar * QString::constData () const-
iterating over the \0-terminated list.

If we implement our escaping (doesn't libpq itself have it if not
libpqxx?), it's good idea to implement at least
KexiDB::Driver::valueToSQL(), escapeString(), and check if
escapeBLOB() is OK.

BTW, there's another job regarding pgsql support: the driver based on
just libpq. Unfortunaltely that was never delivered.
I think we would get benefits from going this way eventually
(efficiency since you can see we have to implement some low-level
features anyway).
Either I find someone interested or myself I'll be back to this task
after 2.2...

--
regards / pozdrawiam, Jaroslaw Staniek
 Kexi & KOffice (http://www.kexi-project.org, http://www.koffice.org)
 KDE Libraries for MS Windows (http://windows.kde.org)
 http://www.linkedin.com/in/jstaniek
_______________________________________________
Kexi mailing list
Kexi@...
https://mail.kde.org/mailman/listinfo/kexi

Parent Message unknown Re: pqxx and Kexi

by Jaroslaw Staniek :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

2009/10/15 Jeroen Vermeulen <jtv@...>:

> Jaroslaw S wrote:
>
>> QString::replace() works in test cases but in pessimistic case it's
>> O(n^2).
>> I do recommend using const QChar * QString::constData () const-
>> iterating over the \0-terminated list.
>
> That reminds me that there's another way, actually!  Depends on your design
> whether it's feasible, but you could use a prepared statement (unnamed if
> you want) for whatever you do with the string.  That way you don't need to
> escape it at all.
>
> (I also just implemented parameterized statements today, but that's not in a
> release yet).

Prepared statements are present in kexidb API already as you know. I
think you mean about using them (directly using libpq/libpqxx API, not
kexidb API) for internal purposes of the driver. I'd say go for it,
just consider keeping the given statement structure after the first
use internally. This will add speed and simplify things.
I planned to do the same for other drivers, including sqlite, long ago.

--
regards / pozdrawiam, Jaroslaw Staniek
 Kexi & KOffice (http://www.kexi-project.org, http://www.koffice.org)
 KDE Libraries for MS Windows (http://windows.kde.org)
 http://www.linkedin.com/in/jstaniek
_______________________________________________
Kexi mailing list
Kexi@...
https://mail.kde.org/mailman/listinfo/kexi

Re: pqxx and Kexi

by Adam Pigg :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Thursday 15 October 2009 20:22:15 Jaroslaw S wrote:

> 2009/10/15 Adam Pigg <piggz1@...>:
> >> This move was made necessary by a security fix in libpq itself.  As it
> >> turned out, its escaping function needs to know the encoding the string
> >> is in, because some multibyte encodings have characters that contain the
> >> byte that in ASCII or UTF-8 would have been e.g. a single quote.
> >>
> >> So for example you might have a two-byte character consisting of some
> >> byte X and one that matches the ASCII character "'": X'
> >>
> >> In that case, the naïve previous escaping function would just double
> >> that byte to "escape" it, producing X'' and voilà: a closing quote has
> >> been injected into a string--by the exact function that's supposed to
> >> prevent it.
> >>
> >> I think your options are:
> >>
> >> a) Write your own escaping function based on your own knowledge of the
> >> encoding that the code is going to run in.  I suppose Qt has some
> >> facility for breaking a string in the current locale's encoding down
> >> into unicode characters.
> >
> > Jeroen,
> >
> > Thanks for getting back, is it just the ' character which needs escaped
> > by doubling it?
> >
> > In which case, a QString::replace() could suffice, Jaroslaw, the main
> > kexi dev can advise on that.
>
> Guys,
> Thanks for continuing the discussion here.
> QString::replace() works in test cases but in pessimistic case it's O(n^2).
> I do recommend using const QChar * QString::constData () const-
> iterating over the \0-terminated list.
>
> If we implement our escaping (doesn't libpq itself have it if not
> libpqxx?), it's good idea to implement at least
> KexiDB::Driver::valueToSQL(), escapeString(), and check if
> escapeBLOB() is OK.

I think in this case libpq wouldnt help because it is libpq's requirement to
have a connection to be able to escape properly based on the encoding of the
database. (?)

Does mysql/other db's we support not have the same issue?  Could the proper
solution not be to move Driver::escape* to Connection::escape*, that way
strings could be escaped based on the encoding of the connection in use.


>
> BTW, there's another job regarding pgsql support: the driver based on
> just libpq. Unfortunaltely that was never delivered.
> I think we would get benefits from going this way eventually
> (efficiency since you can see we have to implement some low-level
> features anyway).
> Either I find someone interested or myself I'll be back to this task
> after 2.2...
>
_______________________________________________
Kexi mailing list
Kexi@...
https://mail.kde.org/mailman/listinfo/kexi

Re: pqxx and Kexi

by Jaroslaw Staniek :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

2009/10/16 Adam Pigg <piggz1@...>:

> On Thursday 15 October 2009 20:22:15 Jaroslaw S wrote:
>> 2009/10/15 Adam Pigg <piggz1@...>:
>> >> This move was made necessary by a security fix in libpq itself.  As it
>> >> turned out, its escaping function needs to know the encoding the string
>> >> is in, because some multibyte encodings have characters that contain the
>> >> byte that in ASCII or UTF-8 would have been e.g. a single quote.
>> >>
>> >> So for example you might have a two-byte character consisting of some
>> >> byte X and one that matches the ASCII character "'": X'
>> >>
>> >> In that case, the naïve previous escaping function would just double
>> >> that byte to "escape" it, producing X'' and voilà: a closing quote has
>> >> been injected into a string--by the exact function that's supposed to
>> >> prevent it.
>> >>
>> >> I think your options are:
>> >>
>> >> a) Write your own escaping function based on your own knowledge of the
>> >> encoding that the code is going to run in.  I suppose Qt has some
>> >> facility for breaking a string in the current locale's encoding down
>> >> into unicode characters.
>> >
>> > Jeroen,
>> >
>> > Thanks for getting back, is it just the ' character which needs escaped
>> > by doubling it?
>> >
>> > In which case, a QString::replace() could suffice, Jaroslaw, the main
>> > kexi dev can advise on that.
>>
>> Guys,
>> Thanks for continuing the discussion here.
>> QString::replace() works in test cases but in pessimistic case it's O(n^2).
>> I do recommend using const QChar * QString::constData () const-
>> iterating over the \0-terminated list.
>>
>> If we implement our escaping (doesn't libpq itself have it if not
>> libpqxx?), it's good idea to implement at least
>> KexiDB::Driver::valueToSQL(), escapeString(), and check if
>> escapeBLOB() is OK.
>
> I think in this case libpq wouldnt help because it is libpq's requirement to
> have a connection to be able to escape properly based on the encoding of the
> database. (?)
>
> Does mysql/other db's we support not have the same issue?  Could the proper
> solution not be to move Driver::escape* to Connection::escape*, that way
> strings could be escaped based on the encoding of the connection in use.

I have nothing against it so far. The only thing is that we may need
to keep the Driver::escape* functions (maybe with extra "encoding"
parameter) too, for cases when we want to encode strings before we
have working connection. That's assuming there's such a case in the
wild.

--
regards / pozdrawiam, Jaroslaw Staniek
 Kexi & KOffice (http://www.kexi-project.org, http://www.koffice.org)
 KDE Libraries for MS Windows (http://windows.kde.org)
 http://www.linkedin.com/in/jstaniek
_______________________________________________
Kexi mailing list
Kexi@...
https://mail.kde.org/mailman/listinfo/kexi