Problem in a unicode project

[Kincorn]

I'm a new user, and Crypto++ is really wonderful and powerful,
haha ...

When using FileSink with path including non-ascii chars (e.g.
chinese), I've got the exception:
"FileSink: error opening file for writing: D:\Project\Crypto\测试
\privkey.txt"
But it's well run in ascii environment.

And then, I found out that something's wrong in function wbstowcs_s()
in STL class fstream as the default local setting is C-Locale. So I
can call function setlocale(LC_ALL, szCodePage) to resolve it, but
it's really troubled, isn't it? And the key-problem is, when I use a
special CodePage, it cann't work in another CodePage environment!

And my project is unicode-based, all the code like "std::string, char
[], ..." in Crypto++ may result unpredictable issues.

Has CryptoPP got the unicode version? If not, how can I fix the
problem?
Thanks for your advice...


Thanks,
Kincorn

Development environment:
PC x86
Windows XP SP2
Visual Studio 2005
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the "Crypto++ Users" Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscribe@....
More information about Crypto++ and this group is available at http://www.cryptopp.com.
-~----------~----~----~----~------~----~------~--~---

[Eugene Zolenko-2]

That's not CryptoPP problem, but rather broken standard C++ unicode
support.

On windows you can use non-standard extention to fstreams -- they can
be created from FILE handles, and FILEs can be created using _wfopen.
And FileSink/Source can be created using fstreams.

std::wstring path = L"D:\bla.txt";
FILE* handle = _wfopen(path, L"rb");
std::ifstream file(handle);
CryptoPP::FileSource(file, ...);

On Linux you will have to convert you std::wstring into UTF8 encoded
std::string first. (Using your favorite UTF8 conversion libraries :)).

Note that using UTF8 converted string on windows doesn't seem to work.

On Jun 24, 10:10 pm, Kincorn <fly...@...> wrote:

> I'm a new user, and Crypto++ is really wonderful and powerful,
> haha ...
>
> When using FileSink with path including non-ascii chars (e.g.
> chinese), I've got the exception:
> "FileSink: error opening file for writing: D:\Project\Crypto\测试
> \privkey.txt"
> But it's well run in ascii environment.
>
> And then, I found out that something's wrong in function wbstowcs_s()
> in STL class fstream as the default local setting is C-Locale. So I
> can call function setlocale(LC_ALL, szCodePage) to resolve it, but
> it's really troubled, isn't it? And the key-problem is, when I use a
> special CodePage, it cann't work in another CodePage environment!
>
> And my project is unicode-based, all the code like "std::string, char
> [], ..." in Crypto++ may result unpredictable issues.
>
> Has CryptoPP got the unicode version? If not, how can I fix the
> problem?
> Thanks for your advice...
>
> Thanks,
> Kincorn
>
> Development environment:
> PC x86
> Windows XP SP2
> Visual Studio 2005

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the "Crypto++ Users" Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscribe@....
More information about Crypto++ and this group is available at http://www.cryptopp.com.
-~----------~----~----~----~------~----~------~--~---

[Kincorn]

Thank you, Eugene, I've just tried the code, and it works exactly.
It's a nice solution.
Is there a better one?
Maybe someone has made a Unicode version, replacing std::string by
std::wstring in the lib-code CryptoPP.


On Jun 26, 2:15 am, Eugene Zolenko <zolen...@...> wrote:

> That's not CryptoPP problem, but rather broken standard C++ unicode
> support.
>
> On windows you can use non-standard extention to fstreams -- they can
> be created from FILE handles, and FILEs can be created using _wfopen.
> And FileSink/Source can be created using fstreams.
>
> std::wstring path = L"D:\bla.txt";
> FILE* handle = _wfopen(path, L"rb");
> std::ifstream file(handle);
> CryptoPP::FileSource(file, ...);
>
> On Linux you will have to convert you std::wstring into UTF8 encoded
> std::string first. (Using your favorite UTF8 conversion libraries :)).
>
> Note that using UTF8 converted string on windows doesn't seem to work.
>
> On Jun 24, 10:10 pm, Kincorn <fly...@...> wrote:
>
>
>
> > I'm a new user, and Crypto++ is really wonderful and powerful,
> > haha ...
>
> > When using FileSink with path including non-ascii chars (e.g.
> > chinese), I've got the exception:
> > "FileSink: error opening file for writing: D:\Project\Crypto\测试
> > \privkey.txt"
> > But it's well run in ascii environment.
>
> > And then, I found out that something's wrong in function wbstowcs_s()
> > in STL class fstream as the default local setting is C-Locale. So I
> > can call function setlocale(LC_ALL, szCodePage) to resolve it, but
> > it's really troubled, isn't it? And the key-problem is, when I use a
> > special CodePage, it cann't work in another CodePage environment!
>
> > And my project is unicode-based, all the code like "std::string, char
> > [], ..." in Crypto++ may result unpredictable issues.
>
> > Has CryptoPP got the unicode version? If not, how can I fix the
> > problem?
> > Thanks for your advice...
>
> > Thanks,
> > Kincorn
>
> > Development environment:
> > PC x86
> > Windows XP SP2
> > Visual Studio 2005- Hide quoted text -
>
> - Show quoted text -

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the "Crypto++ Users" Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscribe@....
More information about Crypto++ and this group is available at http://www.cryptopp.com.
-~----------~----~----~----~------~----~------~--~---

[Eugene Zolenko-2]

Actually somebody did create Source/Sink based on CFile (MFC). If you
only use windows and can use MFC, that would work for you.

Look here:
http://www.codeproject.com/KB/MFC/CryptoPPCFileMFC.aspx

> Thank you, Eugene, I've just tried the code, and it works exactly.
> It's a nice solution.
> Is there a better one?
> Maybe someone has made a Unicode version, replacing std::string by
> std::wstring in the lib-code CryptoPP.
>
>
> On Jun 26, 2:15 am, Eugene Zolenko <zolen...@...> wrote:
>  
>> That's not CryptoPP problem, but rather broken standard C++ unicode
>> support.
>>
>> On windows you can use non-standard extention to fstreams -- they can
>> be created from FILE handles, and FILEs can be created using _wfopen.
>> And FileSink/Source can be created using fstreams.
>>
>> std::wstring path = L"D:\bla.txt";
>> FILE* handle = _wfopen(path, L"rb");
>> std::ifstream file(handle);
>> CryptoPP::FileSource(file, ...);
>>
>> On Linux you will have to convert you std::wstring into UTF8 encoded
>> std::string first. (Using your favorite UTF8 conversion libraries :)).
>>
>> Note that using UTF8 converted string on windows doesn't seem to work.
>>
>> On Jun 24, 10:10 pm, Kincorn <fly...@...> wrote:
>>
>>
>>
>>    
>>> I'm a new user, and Crypto++ is really wonderful and powerful,
>>> haha ...
>>>      
>>> When using FileSink with path including non-ascii chars (e.g.
>>> chinese), I've got the exception:
>>> "FileSink: error opening file for writing: D:\Project\Crypto\测试
>>> \privkey.txt"
>>> But it's well run in ascii environment.
>>>      
>>> And then, I found out that something's wrong in function wbstowcs_s()
>>> in STL class fstream as the default local setting is C-Locale. So I
>>> can call function setlocale(LC_ALL, szCodePage) to resolve it, but
>>> it's really troubled, isn't it? And the key-problem is, when I use a
>>> special CodePage, it cann't work in another CodePage environment!
>>>      
>>> And my project is unicode-based, all the code like "std::string, char
>>> [], ..." in Crypto++ may result unpredictable issues.
>>>      
>>> Has CryptoPP got the unicode version? If not, how can I fix the
>>> problem?
>>> Thanks for your advice...
>>>      
>>> Thanks,
>>> Kincorn
>>>      
>>> Development environment:
>>> PC x86
>>> Windows XP SP2
>>> Visual Studio 2005- Hide quoted text -
>>>      

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the "Crypto++ Users" Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscribe@....
More information about Crypto++ and this group is available at http://www.cryptopp.com.
-~----------~----~----~----~------~----~------~--~---

[Kincorn]

Thanks, the class is useful for my project.

On Jun 26, 10:51 am, Eugene Zolenko <zolen...@...> wrote:

> Actually somebody did create Source/Sink based on CFile (MFC). If you
> only use windows and can use MFC, that would work for you.
>
> Look here:http://www.codeproject.com/KB/MFC/CryptoPPCFileMFC.aspx
>
>
>
> > Thank you, Eugene, I've just tried the code, and it works exactly.
> > It's a nice solution.
> > Is there a better one?
> > Maybe someone has made a Unicode version, replacing std::string by
> > std::wstring in the lib-code CryptoPP.
>
> > On Jun 26, 2:15 am, Eugene Zolenko <zolen...@...> wrote:
>
> >> That's not CryptoPP problem, but rather broken standard C++ unicode
> >> support.
>
> >> On windows you can use non-standard extention to fstreams -- they can
> >> be created from FILE handles, and FILEs can be created using _wfopen.
> >> And FileSink/Source can be created using fstreams.
>
> >> std::wstring path = L"D:\bla.txt";
> >> FILE* handle = _wfopen(path, L"rb");
> >> std::ifstream file(handle);
> >> CryptoPP::FileSource(file, ...);
>
> >> On Linux you will have to convert you std::wstring into UTF8 encoded
> >> std::string first. (Using your favorite UTF8 conversion libraries :)).
>
> >> Note that using UTF8 converted string on windows doesn't seem to work.
>
> >> On Jun 24, 10:10 pm, Kincorn <fly...@...> wrote:
>
> >>> I'm a new user, and Crypto++ is really wonderful and powerful,
> >>> haha ...
>
> >>> When using FileSink with path including non-ascii chars (e.g.
> >>> chinese), I've got the exception:
> >>> "FileSink: error opening file for writing: D:\Project\Crypto\测试
> >>> \privkey.txt"
> >>> But it's well run in ascii environment.
>
> >>> And then, I found out that something's wrong in function wbstowcs_s()
> >>> in STL class fstream as the default local setting is C-Locale. So I
> >>> can call function setlocale(LC_ALL, szCodePage) to resolve it, but
> >>> it's really troubled, isn't it? And the key-problem is, when I use a
> >>> special CodePage, it cann't work in another CodePage environment!
>
> >>> And my project is unicode-based, all the code like "std::string, char
> >>> [], ..." in Crypto++ may result unpredictable issues.
>
> >>> Has CryptoPP got the unicode version? If not, how can I fix the
> >>> problem?
> >>> Thanks for your advice...
>
> >>> Thanks,
> >>> Kincorn
>
> >>> Development environment:
> >>> PC x86
> >>> Windows XP SP2
> >>> Visual Studio 2005- Hide quoted text -- Hide quoted text -
>
> - Show quoted text -

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the "Crypto++ Users" Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscribe@....
More information about Crypto++ and this group is available at http://www.cryptopp.com.
-~----------~----~----~----~------~----~------~--~---

[Wei Dai]

Eugene Zolenko wrote:
> On windows you can use non-standard extention to fstreams -- they can
> be created from FILE handles, and FILEs can be created using _wfopen.
> And FileSink/Source can be created using fstreams.
>
> std::wstring path = L"D:\bla.txt";
> FILE* handle = _wfopen(path, L"rb");
> std::ifstream file(handle);
> CryptoPP::FileSource(file, ...);

It turns out that ifstream::open in MSVC 2005 and later can handle a Unicode
filename directly.

> On Linux you will have to convert you std::wstring into UTF8 encoded
> std::string first. (Using your favorite UTF8 conversion libraries :)).

wcstombs() can be used for this, and it seems to be a standard Unix
function.

I've checked in changes to enable FileSource, FileSink, and FileStore to
handle Unicode filenames on Unix and MSVC 2005 and later using the above
methods. Please take a look at
http://cryptopp.svn.sourceforge.net/viewvc/cryptopp?view=rev&revision=472 if
you'd like to review the code.


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the "Crypto++ Users" Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscribe@....
More information about Crypto++ and this group is available at http://www.cryptopp.com.
-~----------~----~----~----~------~----~------~--~---

[Eugene Zolenko-2]

I would streamline the logic of StoreInitialize like this, removing
goto and half of ifdefs. A goto for name selection is overkill :).

(I didn't compile this though)

void FileStore::StoreInitialize(const NameValuePairs ¶meters)
{
        m_waiting = false;
        m_stream = NULL;
        m_file.release();

        const char* narrowName = NULL;
        const wchar_t* wideName = NULL;
        bool useWideName = false;

        if(parameters.GetValue(Name::InputFileName(), narrowName))
                useWideName = false;
        #if defined(CRYPTOPP_UNIX_AVAILABLE) || _MSC_VER >= 1400
        else if (parameters.GetValue(Name::InputFileNameWide(), wideName))
                useWideName = true;
        #endif
        else
        {
                parameters.GetValue(Name::InputStreamPointer(), m_stream);
                return;
        }

        ios::openmode binary = parameters.GetValueWithDefault
(Name::InputBinaryMode(), true) ? ios::binary : ios::openmode(0);
        m_file.reset(new std::ifstream);

        if(useWideName)
        {
        #if _MSC_VER >= 1400
                m_file->open(wideName, ios::in | binary);
        #else
                m_file->open(StringNarrow(wideName).c_str(), ios::in | binary);
        #endif
        }
        else
                m_file->open(narrowName, ios::in | binary);

        if (!*m_file)
                throw OpenErr(fileName);
        m_stream = m_file.get();
}

On Jul 10, 7:51 pm, "Wei Dai" <wei...@...> wrote:

> Eugene Zolenko wrote:
> > On windows you can use non-standard extention to fstreams -- they can
> > be created from FILE handles, and FILEs can be created using _wfopen.
> > And FileSink/Source can be created using fstreams.
>
> > std::wstring path = L"D:\bla.txt";
> > FILE* handle = _wfopen(path, L"rb");
> > std::ifstream file(handle);
> > CryptoPP::FileSource(file, ...);
>
> It turns out that ifstream::open in MSVC 2005 and later can handle a Unicode
> filename directly.
>
> > On Linux you will have to convert you std::wstring into UTF8 encoded
> > std::string first. (Using your favorite UTF8 conversion libraries :)).
>
> wcstombs() can be used for this, and it seems to be a standard Unix
> function.
>
> I've checked in changes to enable FileSource, FileSink, and FileStore to
> handle Unicode filenames on Unix and MSVC 2005 and later using the above
> methods. Please take a look athttp://cryptopp.svn.sourceforge.net/viewvc/cryptopp?view=rev&revision...if
> you'd like to review the code.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the "Crypto++ Users" Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscribe@....
More information about Crypto++ and this group is available at http://www.cryptopp.com.
-~----------~----~----~----~------~----~------~--~---

[Wei Dai]

Eugene Zolenko wrote:
> I would streamline the logic of StoreInitialize like this, removing
> goto and half of ifdefs. A goto for name selection is overkill :).

I was trying to keep the preexisting logic as much as possible to minimize
the possibility of introducing new errors. But you're right, the result is a
bit too convoluted. I've adopted your logic and improved it a bit (your code
doesn't compile as written). The new revision is at
http://cryptopp.svn.sourceforge.net/viewvc/cryptopp?view=rev&revision=473.
 


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the "Crypto++ Users" Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscribe@....
More information about Crypto++ and this group is available at http://www.cryptopp.com.
-~----------~----~----~----~------~----~------~--~---