[PATCH] remove unnecessary encoding in the default filters, now that everything is a unicode object internally

View: New views
3 Messages — Rating Filter:   Alert me  

[PATCH] remove unnecessary encoding in the default filters, now that everything is a unicode object internally

by Mike Bonnet :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

There is no longer any value in having the filters return encoded str objects,
since they need to be decoded back to unicode before they can be combined with
the rest of the template text (which is managed as a unicode object).  This
patch maintains API compatibility but fundamentally changes the behavior of
some of the filters.  RawOrEncodedUnicode and EncodeUnicode should be
deprecated and removed from the API.

---
 cheetah/Filters.py |   35 +++++++----------------------------
 1 files changed, 7 insertions(+), 28 deletions(-)

diff --git a/cheetah/Filters.py b/cheetah/Filters.py
index 452afc5..d452439 100644
--- a/cheetah/Filters.py
+++ b/cheetah/Filters.py
@@ -29,40 +29,19 @@ class Filter(object):
         if val is None:
             return u''
         if isinstance(val, unicode):
-            if encoding:
-                return val.encode(encoding)
-            else:
-                return val
+            # ignore the encoding and return the unicode object
+            return val
         else:
             try:
-                return str(val)
-            except UnicodeEncodeError:
                 return unicode(val)
-        return u''
+            except UnicodeDecodeError:
+                # we could put more fallbacks here, but we'll just pass the str
+                # on and let DummyTransaction worry about it
+                return str(val)
 
 RawOrEncodedUnicode = Filter
 
-class EncodeUnicode(Filter):
-    def filter(self, val,
-               encoding='utf8',
-               str=str,
-               **kw):
-        """Encode Unicode strings, by default in UTF-8.
-
-        >>> import Cheetah.Template
-        >>> t = Cheetah.Template.Template('''
-        ... $myvar
-        ... ${myvar, encoding='utf16'}
-        ... ''', searchList=[{'myvar': u'Asni\xe8res'}],
-        ... filter='EncodeUnicode')
-        >>> print t
-        """
-        if isinstance(val, unicode):
-            return val
-        if val is None:
-            return ''
-        return str(val)
-
+EncodeUnicode = Filter
 
 class Markdown(EncodeUnicode):
     '''
--
1.6.2.5


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Cheetahtemplate-discuss mailing list
Cheetahtemplate-discuss@...
https://lists.sourceforge.net/lists/listinfo/cheetahtemplate-discuss

Re: [PATCH] remove unnecessary encoding in the default filters, now that everything is a unicode object internally

by tyler-53 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message


On Fri, 16 Oct 2009, Mike Bonnet wrote:

> There is no longer any value in having the filters return encoded str objects,
> since they need to be decoded back to unicode before they can be combined with
> the rest of the template text (which is managed as a unicode object).  This
> patch maintains API compatibility but fundamentally changes the behavior of
> some of the filters.  RawOrEncodedUnicode and EncodeUnicode should be
> deprecated and removed from the API.

Ack'd. Merged into rtyler/next

If internal testing goes well, I'll roll this into 2.4.0/2.3.0

>
> ---
>  cheetah/Filters.py |   35 +++++++----------------------------
>  1 files changed, 7 insertions(+), 28 deletions(-)
>
> diff --git a/cheetah/Filters.py b/cheetah/Filters.py
> index 452afc5..d452439 100644
> --- a/cheetah/Filters.py
> +++ b/cheetah/Filters.py
> @@ -29,40 +29,19 @@ class Filter(object):
>          if val is None:
>              return u''
>          if isinstance(val, unicode):
> -            if encoding:
> -                return val.encode(encoding)
> -            else:
> -                return val
> +            # ignore the encoding and return the unicode object
> +            return val
>          else:
>              try:
> -                return str(val)
> -            except UnicodeEncodeError:
>                  return unicode(val)
> -        return u''
> +            except UnicodeDecodeError:
> +                # we could put more fallbacks here, but we'll just pass the str
> +                # on and let DummyTransaction worry about it
> +                return str(val)
>  
>  RawOrEncodedUnicode = Filter
>  
> -class EncodeUnicode(Filter):
> -    def filter(self, val,
> -               encoding='utf8',
> -               str=str,
> -               **kw):
> -        """Encode Unicode strings, by default in UTF-8.
> -
> -        >>> import Cheetah.Template
> -        >>> t = Cheetah.Template.Template('''
> -        ... $myvar
> -        ... ${myvar, encoding='utf16'}
> -        ... ''', searchList=[{'myvar': u'Asni\xe8res'}],
> -        ... filter='EncodeUnicode')
> -        >>> print t
> -        """
> -        if isinstance(val, unicode):
> -            return val
> -        if val is None:
> -            return ''
> -        return str(val)
> -
> +EncodeUnicode = Filter
>  
>  class Markdown(EncodeUnicode):
>      '''
> --
> 1.6.2.5
>
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference
> _______________________________________________
> Cheetahtemplate-discuss mailing list
> Cheetahtemplate-discuss@...
> https://lists.sourceforge.net/lists/listinfo/cheetahtemplate-discuss
Cheers,
-R. Tyler Ballance
--------------------------------------
 GitHub: http://github.com/rtyler
Twitter: http://twitter.com/agentdero
   Blog: http://unethicalblogger.com


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Cheetahtemplate-discuss mailing list
Cheetahtemplate-discuss@...
https://lists.sourceforge.net/lists/listinfo/cheetahtemplate-discuss

attachment0 (204 bytes) Download Attachment

fyi, the cheetah website will be offline for a few hours tomorrow

by Tavis Rudd :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

The server hosting the site is sitting in my closet and our building's
power vault is scheduled for 'cleaning' tomorrow so site will be
offline for a few hours.
Tavis

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Cheetahtemplate-discuss mailing list
Cheetahtemplate-discuss@...
https://lists.sourceforge.net/lists/listinfo/cheetahtemplate-discuss