Concerns about Zend_Filter_Input

[Ed Finkler]

I wrote up a short piece of code for the Zend DevZone last night, and sent it to Cal Evans.  This code was a basic bootstrap file that utilized Zend_Filter_Input and the Zend::registry methods to provide much safer access to user input while getting around scoping issues.  This morning, after Cal looked it over, he told me something I'd not realized (as it's not mentioned in the 0.8.0 changelog, and the docs are still present on the main FW site): that Zend_Filter_Input has been removed entirely from the ZFW.

Clearly I'm late to the party on this, so I want to ask a few questions before formulating a full opinion on the issue.  The mailing list archives gave me some ideas about the issues involved, but not a clear breakdown of the what and why.

1) Why exactly was Zend_Filter_Input dropped completely?  My impression is that it was because some components that needed access to the superglobals data were not designed to support Zend_Filter_Input, and that Zend_Filter_Input didn't yet support things like iterative access.

2) Will a similar replacement component, one that forces developers to access user input via a filtering API, be included in the 1.0RC?

Thanks in advance,

-- 

Ed Finkler

Web and Security Archive Administrator

CERIAS - Purdue University

http://www.cerias.purdue.edu/

v: 765.496.6762  f: 765.496.3181

AIM: funkatronwork

ICQ: 301234029

[Kevin McArthur-2]

Some parts of this message have been removed. Learn more about Nabble's security policy.

It was dropped for Zend_Validate afaik. However, I did submit a patch that added iterative access for ZFI, that wasnt applied because the component was being deprecated.

 

Search the archives, the debate on point 2 has been hashed before.

 

Kevin

----- Original Message -----

From: coj@...

To: fw-core@...

Sent: Tuesday, March 13, 2007 10:31 AM

Subject: [fw-core] Concerns about Zend_Filter_Input

I wrote up a short piece of code for the Zend DevZone last night, and sent it to Cal Evans.  This code was a basic bootstrap file that utilized Zend_Filter_Input and the Zend::registry methods to provide much safer access to user input while getting around scoping issues.  This morning, after Cal looked it over, he told me something I'd not realized (as it's not mentioned in the 0.8.0 changelog, and the docs are still present on the main FW site): that Zend_Filter_Input has been removed entirely from the ZFW.

Clearly I'm late to the party on this, so I want to ask a few questions before formulating a full opinion on the issue.  The mailing list archives gave me some ideas about the issues involved, but not a clear breakdown of the what and why.

1) Why exactly was Zend_Filter_Input dropped completely?  My impression is that it was because some components that needed access to the superglobals data were not designed to support Zend_Filter_Input, and that Zend_Filter_Input didn't yet support things like iterative access.

2) Will a similar replacement component, one that forces developers to access user input via a filtering API, be included in the 1.0RC?

Thanks in advance,

-- 

Ed Finkler

Web and Security Archive Administrator

CERIAS - Purdue University

http://www.cerias.purdue.edu/

v: 765.496.6762  f: 765.496.3181

AIM: funkatronwork

ICQ: 301234029

[Bill Karwin from Zend]

Some parts of this message have been removed. Learn more about Nabble's security policy.

Hi Ed,

 

Actually, the implementations of Zend_Filter and Zend_Filter_Input in earlier preview releases of Zend Framework have been _replaced_ by a much better object-oriented class design that provides similar functionality.  It’s better because it also serves as an extensible framework for developers to implement custom filters and validators.  And also, an application needs to load only the filters and validators that it needs, instead of a monolithic class that contains all filters and validators.

 

You can see the proposal that described this change here:

http://framework.zend.com/wiki/x/Wi8

 

You can see the documentation for the new Zend_Filter and Zend_Validate classes here:

http://framework.zend.com/manual/en/zend.filter.html

http://framework.zend.com/manual/en/zend.validate.html

 

Neither the old classes nor the new classes force the developer to access input data through filters.  Usage of these classes is encouraged, but it is up to the developer to utilize them.  Both in the old design and in the new design, it is possible to write PHP applications that access inputs such as superglobals directly, without the enforcement of any filtering.

 

The goal has always been to promote safe programming practices and reduce risks of security issues in application code, by providing classes that help developers filter out malicious content using a simple object-oriented API.  But we don’t want to constrain developers to a single solution.

 

There is still more work to do, to make the process of filtering and validation even easier to use. We’ll enhance the solution in future versions of Zend Framework after 1.0.  For example, we intend to design an easy-to-use component to apply filtering/validation logic against aggregations of web input data.

 

Regards,

Bill Karwin

 

---

From: Ed Finkler [mailto:coj@...]
Sent: Tuesday, March 13, 2007 10:32 AM
To: Zend FW Core List
Subject: [fw-core] Concerns about Zend_Filter_Input

 

I wrote up a short piece of code for the Zend DevZone last night, and sent it to Cal Evans.  This code was a basic bootstrap file that utilized Zend_Filter_Input and the Zend::registry methods to provide much safer access to user input while getting around scoping issues.  This morning, after Cal looked it over, he told me something I'd not realized (as it's not mentioned in the 0.8.0 changelog, and the docs are still present on the main FW site): that Zend_Filter_Input has been removed entirely from the ZFW.

 

Clearly I'm late to the party on this, so I want to ask a few questions before formulating a full opinion on the issue.  The mailing list archives gave me some ideas about the issues involved, but not a clear breakdown of the what and why.

 

1) Why exactly was Zend_Filter_Input dropped completely?  My impression is that it was because some components that needed access to the superglobals data were not designed to support Zend_Filter_Input, and that Zend_Filter_Input didn't yet support things like iterative access.

 

2) Will a similar replacement component, one that forces developers to access user input via a filtering API, be included in the 1.0RC?

 

Thanks in advance,

 

-- 

Ed Finkler

Web and Security Archive Administrator

CERIAS - Purdue University

http://www.cerias.purdue.edu/

v: 765.496.6762  f: 765.496.3181

AIM: funkatronwork

ICQ: 301234029

 

[Ed Finkler]

Kevin McArthur wrote:
> It was dropped for Zend_Validate afaik. However, I did submit a patch
> that added iterative access for ZFI, that wasnt applied because the
> component was being deprecated.

*nod*

> Search the archives, the debate on point 2 has been hashed before.

I saw it being hashed here:
<http://www.nabble.com/Re%3A--fw-general--Zend_Filter-design-proposal-tf2954795s16154.html>
but not resolved, per se.  Evidently it was, though! 8)

--
Ed Finkler
Web and Security Archive Administrator
CERIAS - Purdue University
http://www.cerias.purdue.edu/
v: 765.496.6762  f: 765.496.3181

[Ed Finkler]

Bill Karwin wrote:

> Hi Ed,
>
>  
>
> Actually, the implementations of Zend_Filter and Zend_Filter_Input in
> earlier preview releases of Zend Framework have been _/replaced/_ by a
> much better object-oriented class design that provides similar
> functionality.  It’s better because it also serves as an extensible
> framework for developers to implement custom filters and validators.  
> And also, an application needs to load only the filters and validators
> that it needs, instead of a monolithic class that contains all filters
> and validators.

I can see that creating my own filters is significantly easier, and
that's a win, no doubt.

> There is still more work to do, to make the process of filtering and
> validation even easier to use. We’ll enhance the solution in future
> versions of Zend Framework after 1.0.  For example, we intend to design
> an easy-to-use component to apply filtering/validation logic against
> aggregations of web input data.

Let me reiterate my second question:

"2) Will a similar replacement component, one that forces developers to
access user input via a filtering API, be included in the 1.0RC?"

I've certainly been known to miss things staring me in the face before,
but I don't see anything in the new Zend_Filter that would allow me to
do bootstrap filtering like I could with Zend_Input_Filter:

<code>
$f_POST   = new Zend_Filter_Input($_POST);
$f_GET    = new Zend_Filter_Input($_GET);
$f_COOKIE = new Zend_Filter_Input($_COOKIE);
$f_SERVER = new Zend_Filter_Input($_SERVER);
</code>

Can any of the filter classes that are now part of the ZFW process
arrays?  All of the usage cases I see are handling scalars.

Again, I want to make sure I'm not missing something before forming an
opinion.

--
Ed Finkler
Web and Security Archive Administrator
CERIAS - Purdue University
http://www.cerias.purdue.edu/
v: 765.496.6762  f: 765.496.3181

[Bill Karwin from Zend]

> -----Original Message-----
> From: Ed Finkler [mailto:coj@...]
>
> Let me reiterate my second question:
>
> "2) Will a similar replacement component, one that forces developers
to
> access user input via a filtering API, be included in the 1.0RC?"

No, there will not be such a component in ZF 1.0.

We're very close to our scheduled feature-freeze for Zend Framework 1.0
(March 15).  A solution to satisfy the use case you describe is not
going to be implemented by our deadline.

> Can any of the filter classes that are now part of the ZFW process
> arrays?  All of the usage cases I see are handling scalars.

Right, currently the Zend_Filter classes operate on scalars.  Using
these classes to apply filtering and validation to arrays or other
aggregations of data is left for the developer to do, in a manner
appropriate for his application.

We encourage proposals for solutions that utilize the Zend_Filter and
Zend_Validate classes, to apply filtering and validation to aggregations
of input data.  So far there has been discussion, but no design has been
formally proposed.

We will evaluate the proposals and a solution will be added in a future
release of Zend Framework (e.g. 1.1 or later).

Regards,
Bill Karwin

[Kevin McArthur-2]

Why the push to 1.0 so fast, when things can't as readily be changed after?

With so many changes in 0.8 and now some serious 0.9 changes, might make
sense to slow down the run-up to 1.0 a bit. If not, then the same level of
change will occur post 1.0 and instead of point releases, there'll be a
bi-monthly majorversion jump.

I'd propose a 0.8.5, 0.9.5 and 0.9.9 before 1.0 might give time to get all
the API changes out of the way.

$0.02

Kevin
----- Original Message -----
From: "Bill Karwin" <bill.k@...>
To: "Ed Finkler" <coj@...>
Cc: "Zend FW Core List" <fw-core@...>
Sent: Tuesday, March 13, 2007 12:58 PM
Subject: RE: [fw-core] Concerns about Zend_Filter_Input


> -----Original Message-----
> From: Ed Finkler [mailto:coj@...]
>
> Let me reiterate my second question:
>
> "2) Will a similar replacement component, one that forces developers
to
> access user input via a filtering API, be included in the 1.0RC?"

No, there will not be such a component in ZF 1.0.

We're very close to our scheduled feature-freeze for Zend Framework 1.0
(March 15).  A solution to satisfy the use case you describe is not
going to be implemented by our deadline.

> Can any of the filter classes that are now part of the ZFW process
> arrays?  All of the usage cases I see are handling scalars.

Right, currently the Zend_Filter classes operate on scalars.  Using
these classes to apply filtering and validation to arrays or other
aggregations of data is left for the developer to do, in a manner
appropriate for his application.

We encourage proposals for solutions that utilize the Zend_Filter and
Zend_Validate classes, to apply filtering and validation to aggregations
of input data.  So far there has been discussion, but no design has been
formally proposed.

We will evaluate the proposals and a solution will be added in a future
release of Zend Framework (e.g. 1.1 or later).

Regards,
Bill Karwin

[Simon Mundy]

I've got to say I agree Kevin.

In the leadup to 0.9 there's been a few issues raised on several  
components that I feel should be adopted for 1.0 and I'm not quite  
sure that the impending release schedule should dictate their  
inclusion/exclusion.

Take Db_Table, for example. Bill's busting a boiler getting the new  
branch out the door for 0.9 but then that still leaves little time  
between 0.9 and 1.0 for a sufficiently large audience of developers  
to put it through its paces and perhaps make suggestions for minor  
functionality tweaks.

I'd agree with a 0.9.5 (RC 1) and perhaps 0.9.6 (RC 2) to allow one  
last revision to APIs on the major components (not saying they all  
need it, but perhaps it's time to revisit the framework with fresh  
eyes to ensure it is watertight) and to tie up loose ends. After all,  
those who are not on the bleeding edge are still waiting for 0.9 to  
be released with a _lot_ of changes and additions that need roadtesting.

> Why the push to 1.0 so fast, when things can't as readily be  
> changed after?
>
> With so many changes in 0.8 and now some serious 0.9 changes, might  
> make sense to slow down the run-up to 1.0 a bit. If not, then the  
> same level of change will occur post 1.0 and instead of point  
> releases, there'll be a bi-monthly majorversion jump.
>
> I'd propose a 0.8.5, 0.9.5 and 0.9.9 before 1.0 might give time to  
> get all the API changes out of the way.
>
> $0.02
>
> Kevin
> ----- Original Message ----- From: "Bill Karwin" <bill.k@...>
> To: "Ed Finkler" <coj@...>
> Cc: "Zend FW Core List" <fw-core@...>
> Sent: Tuesday, March 13, 2007 12:58 PM
> Subject: RE: [fw-core] Concerns about Zend_Filter_Input
>
>
>> -----Original Message-----
>> From: Ed Finkler [mailto:coj@...]
>>
>> Let me reiterate my second question:
>>
>> "2) Will a similar replacement component, one that forces developers
> to
>> access user input via a filtering API, be included in the 1.0RC?"
>
> No, there will not be such a component in ZF 1.0.
>
> We're very close to our scheduled feature-freeze for Zend Framework  
> 1.0
> (March 15).  A solution to satisfy the use case you describe is not
> going to be implemented by our deadline.
>
>> Can any of the filter classes that are now part of the ZFW process
>> arrays?  All of the usage cases I see are handling scalars.
>
> Right, currently the Zend_Filter classes operate on scalars.  Using
> these classes to apply filtering and validation to arrays or other
> aggregations of data is left for the developer to do, in a manner
> appropriate for his application.
>
> We encourage proposals for solutions that utilize the Zend_Filter and
> Zend_Validate classes, to apply filtering and validation to  
> aggregations
> of input data.  So far there has been discussion, but no design has  
> been
> formally proposed.
>
> We will evaluate the proposals and a solution will be added in a  
> future
> release of Zend Framework (e.g. 1.1 or later).
>
> Regards,
> Bill Karwin

--

Simon Mundy | Director | PEPTOLAB

""" " "" """""" "" "" """"""" " "" """"" " """"" "  """""" "" "
202/258 Flinders Lane | Melbourne | Victoria | Australia | 3000
Voice [phone] | Mobile [phone] | Fax [phone]  
4124
http://www.peptolab.com

[Ed Finkler]

Thank you for clarifying the situation, Bill.

Let me start off with some context.  If we do a search for 'php' in the 2006 records of the NIST NVD, over *40%* of the total number of vulnerabilities are returned.  Removing language level issues, which would account for a small number of these, we're left with an enormous number of PHP application vulnerabilities -- well over 1/3 of all the 2006 entries in the NVD, for sure.

I believe those numbers say a lot about the following things:

1. How popular PHP is

2. How much web applications are a target

3. How many PHP devs are incapable of writing secure applications

I don't necessarily fault PHP as a language for this, as I think any language of similar ease and popularity would suffer many of the same problems.  And as more and more of our business and social activity is conducted online, web applications are inevitably going to increasingly be targeted.

But we can do a *lot* about #3, in two major areas: _education_, and _empowerment_.

Education is fairly easy to understand, I think: books, training courses, articles, tutorials, etc.  In a sense, the ZFW project has a significant opportunity to educate PHP developers by serving as an example to the community.  Zend's profile in the community gives it an opportunity to raise the bar, and change expectations about what effective, secure web app development entails.

Empowering developers, in my mind, means making it easier to do the right thing.  Our toolsets -- editors, debuggers, frameworks, etc -- should make it _easier_ to write secure applications.  The default action should be the secure action.  It shouldn't be _impossible_ to do potentially dangerous stuff, but it should require clear intent on the part of the developer.

This is where I feel the Zend Framework has *regressed* as of version 0.8.  Out of the box, it is *more difficult* to write secure applications with ZFW as of 0.8.0 than it was in 0.7.0.

Stripping Tags.  In 0.7:

<pre>

require "Zend/Filter.php";

$searchtxt = Zend_Filter::noTags($searchtxt);

</pre>

In 0.8:

<pre>

require "Zend/Filter/StripTags.php";

$noTags    = new Zend_Filter_StripTags();

$searchtxt = $noTags->filter($searchtxt);

</pre>

Only one extra line, but when you're having to instantiate a new filter every time you want different functionality, it adds up.  And if you're not using class autoloading, you have to do a require statement for every filter class, whereas in 0.7 you only needed to do a single require.

In some ways, the new Zend_Filter is a win, especially in terms of expandability, but it lacks the proper hooks for making filtering and validation as speedy and easy as it was before.  Make no mistake: verbosity *matters*.  I am not alone in preferring an option that makes me type less.

More disturbing, though, is the total removal of the Zend_Filter_Input component from the ZFW.  ZFI was a very effective tool for limiting access to user input arrays like $_POST and $_COOKIE, wrapping them in an object and forcing the developer to use object methods to access the data.

<code>

require "Zend/Filter/Input.php";

$f_POST = new Zend_Filter_Input($_POST);

$searchtext = $f_POST->noTags('searchtxt');

$id = $f_POST->getInt('id');

// you can only get the raw input with the getRaw method

$unfiltered_input = $f_POST->('xssWaitingToHappen');

</code>

It wasn't perfect -- as of 0.7 it lacked iterative access and didn't work well with multidimensional arrays -- but it established a strong new paradigm for PHP development that was a long time in coming.  The vast majority of vulnerabilities in PHP apps come from mishandling user input, and Zend_Filter_Input made it significantly easier to do the right thing, and significantly harder to do the wrong thing.  This was a *major* jewel for the Zend Framework, something no other high-profile PHP framework was offering (AFAIK).  And it evidently had been considered a fundamental component, having been present in the core since the initial public release (0.1.2).  Now, it's gone, with no comparable functionality in place, and no concrete plans to restore it.

Zend has a significant opportunity to influence how PHP development is done with the ZFW 1.0 release.  The Zend Framework will be, for better or worse, considered an example as to how PHP development *should* be done.  It's disappointing, then, to see the framework move backwards in terms of easing secure development while it speeds towards 1.0.  This opportunity will not come again.  I hope these decisions are reconsidered, and the feature freeze is delayed so that previously-existing functionality can be restored.

Thanks,

-- 

Ed Finkler

Web and Security Archive Administrator

CERIAS - Purdue University

http://www.cerias.purdue.edu/

v: 765.496.6762  f: 765.496.3181

AIM: funkatronwork

ICQ: 301234029

On Mar 13, 2007, at 3:58 PM, Bill Karwin wrote:

-----Original Message-----

From: Ed Finkler [coj@...]

Let me reiterate my second question:

"2) Will a similar replacement component, one that forces developers

to

access user input via a filtering API, be included in the 1.0RC?"

No, there will not be such a component in ZF 1.0.

We're very close to our scheduled feature-freeze for Zend Framework 1.0

(March 15).  A solution to satisfy the use case you describe is not

going to be implemented by our deadline.

Can any of the filter classes that are now part of the ZFW process

arrays?  All of the usage cases I see are handling scalars.

Right, currently the Zend_Filter classes operate on scalars.  Using

these classes to apply filtering and validation to arrays or other

aggregations of data is left for the developer to do, in a manner

appropriate for his application.

We encourage proposals for solutions that utilize the Zend_Filter and

Zend_Validate classes, to apply filtering and validation to aggregations

of input data.  So far there has been discussion, but no design has been

formally proposed.

We will evaluate the proposals and a solution will be added in a future

release of Zend Framework (e.g. 1.1 or later).

Regards,

Bill Karwin

[Christopher Thompson-2]

I also don't think it makes sense given the changes made (and needed) to
do 1.0 so fast. On the other hand, it might be better to push a 1.0 out,
let the community use it for 6 months, and then make the necessary
changes for a 2.0 that will be the more stable long term release. I
think the current track means that 1.0 will really be a public beta and
2.0 will be the real 1.0.


Kevin McArthur wrote:

> Why the push to 1.0 so fast, when things can't as readily be changed after?
>
> With so many changes in 0.8 and now some serious 0.9 changes, might make
> sense to slow down the run-up to 1.0 a bit. If not, then the same level
> of change will occur post 1.0 and instead of point releases, there'll be
> a bi-monthly majorversion jump.
>
> I'd propose a 0.8.5, 0.9.5 and 0.9.9 before 1.0 might give time to get
> all the API changes out of the way.
>
> $0.02
>
> Kevin
> ----- Original Message ----- From: "Bill Karwin" <bill.k@...>
> To: "Ed Finkler" <coj@...>
> Cc: "Zend FW Core List" <fw-core@...>
> Sent: Tuesday, March 13, 2007 12:58 PM
> Subject: RE: [fw-core] Concerns about Zend_Filter_Input
>
>
>> -----Original Message-----
>> From: Ed Finkler [mailto:coj@...]
>>
>> Let me reiterate my second question:
>>
>> "2) Will a similar replacement component, one that forces developers
> to
>> access user input via a filtering API, be included in the 1.0RC?"
>
> No, there will not be such a component in ZF 1.0.
>
> We're very close to our scheduled feature-freeze for Zend Framework 1.0
> (March 15).  A solution to satisfy the use case you describe is not
> going to be implemented by our deadline.
>
>> Can any of the filter classes that are now part of the ZFW process
>> arrays?  All of the usage cases I see are handling scalars.
>
> Right, currently the Zend_Filter classes operate on scalars.  Using
> these classes to apply filtering and validation to arrays or other
> aggregations of data is left for the developer to do, in a manner
> appropriate for his application.
>
> We encourage proposals for solutions that utilize the Zend_Filter and
> Zend_Validate classes, to apply filtering and validation to aggregations
> of input data.  So far there has been discussion, but no design has been
> formally proposed.
>
> We will evaluate the proposals and a solution will be added in a future
> release of Zend Framework (e.g. 1.1 or later).
>
> Regards,
> Bill Karwin
>

[Kevin McArthur-2]

The problem with this is that all the community docs and books will be
written to a 1.0 standard.

Changing a lot for 2.0 will invalidate this prior work (and cause problems
if 2.0 comes quickly)

Kevin
----- Original Message -----
From: "Christopher Thompson" <cxthompson@...>
Cc: "Zend FW Core List" <fw-core@...>
Sent: Tuesday, March 13, 2007 5:15 PM
Subject: Re: [fw-core] Concerns about Zend_Filter_Input

>I also don't think it makes sense given the changes made (and needed) to do
>1.0 so fast. On the other hand, it might be better to push a 1.0 out, let
>the community use it for 6 months, and then make the necessary changes for
>a 2.0 that will be the more stable long term release. I think the current
>track means that 1.0 will really be a public beta and 2.0 will be the real
>1.0.
>
>
> Kevin McArthur wrote:
>> Why the push to 1.0 so fast, when things can't as readily be changed
>> after?
>>
>> With so many changes in 0.8 and now some serious 0.9 changes, might make
>> sense to slow down the run-up to 1.0 a bit. If not, then the same level
>> of change will occur post 1.0 and instead of point releases, there'll be
>> a bi-monthly majorversion jump.
>>
>> I'd propose a 0.8.5, 0.9.5 and 0.9.9 before 1.0 might give time to get
>> all the API changes out of the way.
>>
>> $0.02
>>
>> Kevin
>> ----- Original Message ----- From: "Bill Karwin" <bill.k@...>
>> To: "Ed Finkler" <coj@...>
>> Cc: "Zend FW Core List" <fw-core@...>
>> Sent: Tuesday, March 13, 2007 12:58 PM
>> Subject: RE: [fw-core] Concerns about Zend_Filter_Input
>>
>>
>>> -----Original Message-----
>>> From: Ed Finkler [mailto:coj@...]
>>>
>>> Let me reiterate my second question:
>>>
>>> "2) Will a similar replacement component, one that forces developers
>> to
>>> access user input via a filtering API, be included in the 1.0RC?"
>>
>> No, there will not be such a component in ZF 1.0.
>>
>> We're very close to our scheduled feature-freeze for Zend Framework 1.0
>> (March 15).  A solution to satisfy the use case you describe is not
>> going to be implemented by our deadline.
>>
>>> Can any of the filter classes that are now part of the ZFW process
>>> arrays?  All of the usage cases I see are handling scalars.
>>
>> Right, currently the Zend_Filter classes operate on scalars.  Using
>> these classes to apply filtering and validation to arrays or other
>> aggregations of data is left for the developer to do, in a manner
>> appropriate for his application.
>>
>> We encourage proposals for solutions that utilize the Zend_Filter and
>> Zend_Validate classes, to apply filtering and validation to aggregations
>> of input data.  So far there has been discussion, but no design has been
>> formally proposed.
>>
>> We will evaluate the proposals and a solution will be added in a future
>> release of Zend Framework (e.g. 1.1 or later).
>>
>> Regards,
>> Bill Karwin
>>

[Ersin Yilmaz]

Some parts of this message have been removed. Learn more about Nabble's security policy.

Could not agree more with Ed. The application I'm currently working on is still using ZF 0.7.0.  And I'm hesitant to upgrade to 0.8.0 until I can replace Zend_Filter_Input with another component.

We are doing the exact same thing in our bootstrapper file -  we are registering $_GET, $_POST, $_REQUEST globals with Zend_Filter_Input to disallow direct access, and this has worked great so far.

I also vote for not rushing into 1.0.

thanks,
Ersin

On 3/13/07, Ed Finkler <coj@...> wrote:

Thank you for clarifying the situation, Bill.

Let me start off with some context.  If we do a search for 'php' in the 2006 records of the NIST NVD, over *40%* of the total number of vulnerabilities are returned.  Removing language level issues, which would account for a small number of these, we're left with an enormous number of PHP application vulnerabilities -- well over 1/3 of all the 2006 entries in the NVD, for sure.

I believe those numbers say a lot about the following things:

1. How popular PHP is

2. How much web applications are a target

3. How many PHP devs are incapable of writing secure applications

I don't necessarily fault PHP as a language for this, as I think any language of similar ease and popularity would suffer many of the same problems.  And as more and more of our business and social activity is conducted online, web applications are inevitably going to increasingly be targeted.

But we can do a *lot* about #3, in two major areas: _education_, and _empowerment_.

Education is fairly easy to understand, I think: books, training courses, articles, tutorials, etc.  In a sense, the ZFW project has a significant opportunity to educate PHP developers by serving as an example to the community.  Zend's profile in the community gives it an opportunity to raise the bar, and change expectations about what effective, secure web app development entails.

Empowering developers, in my mind, means making it easier to do the right thing.  Our toolsets -- editors, debuggers, frameworks, etc -- should make it _easier_ to write secure applications.  The default action should be the secure action.  It shouldn't be _impossible_ to do potentially dangerous stuff, but it should require clear intent on the part of the developer.

This is where I feel the Zend Framework has *regressed* as of version 0.8.  Out of the box, it is *more difficult* to write secure applications with ZFW as of 0.8.0 than it was in 0.7.0.

Stripping Tags.  In 0.7:

<pre>

require "Zend/Filter.php";

$searchtxt = Zend_Filter::noTags($searchtxt);

</pre>

In 0.8:

<pre>

require "Zend/Filter/StripTags.php";

$noTags    = new Zend_Filter_StripTags();

$searchtxt = $noTags->filter($searchtxt);

</pre>

Only one extra line, but when you're having to instantiate a new filter every time you want different functionality, it adds up.  And if you're not using class autoloading, you have to do a require statement for every filter class, whereas in 0.7 you only needed to do a single require.

In some ways, the new Zend_Filter is a win, especially in terms of expandability, but it lacks the proper hooks for making filtering and validation as speedy and easy as it was before.  Make no mistake: verbosity *matters*.  I am not alone in preferring an option that makes me type less.

More disturbing, though, is the total removal of the Zend_Filter_Input component from the ZFW.  ZFI was a very effective tool for limiting access to user input arrays like $_POST and $_COOKIE, wrapping them in an object and forcing the developer to use object methods to access the data.

<code>

require "Zend/Filter/Input.php";

$f_POST = new Zend_Filter_Input($_POST);

$searchtext = $f_POST->noTags('searchtxt');

$id = $f_POST->getInt('id');

// you can only get the raw input with the getRaw method

$unfiltered_input = $f_POST->('xssWaitingToHappen');

</code>

It wasn't perfect -- as of 0.7 it lacked iterative access and didn't work well with multidimensional arrays -- but it established a strong new paradigm for PHP development that was a long time in coming.  The vast majority of vulnerabilities in PHP apps come from mishandling user input, and Zend_Filter_Input made it significantly easier to do the right thing, and significantly harder to do the wrong thing.  This was a *major* jewel for the Zend Framework, something no other high-profile PHP framework was offering (AFAIK).  And it evidently had been considered a fundamental component, having been present in the core since the initial public release ( 0.1.2).  Now, it's gone, with no comparable functionality in place, and no concrete plans to restore it.

Zend has a significant opportunity to influence how PHP development is done with the ZFW 1.0 release.  The Zend Framework will be, for better or worse, considered an example as to how PHP development *should* be done.  It's disappointing, then, to see the framework move backwards in terms of easing secure development while it speeds towards 1.0.  This opportunity will not come again.  I hope these decisions are reconsidered, and the feature freeze is delayed so that previously-existing functionality can be restored.

Thanks,

-- 

Ed Finkler

Web and Security Archive Administrator

CERIAS - Purdue University

http://www.cerias.purdue.edu/

v: 765.496.6762  f: 765.496.3181

AIM: funkatronwork

ICQ: 301234029

On Mar 13, 2007, at 3:58 PM, Bill Karwin wrote:

-----Original Message-----

From: Ed Finkler [coj@...]

Let me reiterate my second question:

"2) Will a similar replacement component, one that forces developers

to

access user input via a filtering API, be included in the 1.0RC?"

No, there will not be such a component in ZF 1.0.

We're very close to our scheduled feature-freeze for Zend Framework 1.0

(March 15).  A solution to satisfy the use case you describe is not

going to be implemented by our deadline.

Can any of the filter classes that are now part of the ZFW process

arrays?  All of the usage cases I see are handling scalars.

Right, currently the Zend_Filter classes operate on scalars.   Using

these classes to apply filtering and validation to arrays or other

aggregations of data is left for the developer to do, in a manner

appropriate for his application.

We encourage proposals for solutions that utilize the Zend_Filter and

Zend_Validate classes, to apply filtering and validation to aggregations

of input data.  So far there has been discussion, but no design has been

formally proposed.

We will evaluate the proposals and a solution will be added in a future

release of Zend Framework ( e.g. 1.1 or later).

Regards,

Bill Karwin

--
Ersin Yilmaz
ersiny@...

[Christopher Thompson-2]

Docs can be rewritten. I think reality may trump designing at this
point. So far we have a boat that we know floats and rumor has it that
Matthew can make it go straight. But you need to launch it to see how it
really sails.

I have a strong suspicion that the version we are all using in two years
is different than the 1.0 code. And the fastest way from here to there
is to release early and often.

That does make the 1.0 launch less than optimal from the code
maintenance POV, but such is life. You can't expect no changes moving to
2.0. And perhaps some tooling can ease identifying what needs to change
when upgrading (and maybe even suggest or do the change).


Kevin McArthur wrote:

> The problem with this is that all the community docs and books will be
> written to a 1.0 standard.
>
> Changing a lot for 2.0 will invalidate this prior work (and cause
> problems if 2.0 comes quickly)
>
> Kevin
> ----- Original Message ----- From: "Christopher Thompson"
> <cxthompson@...>
> Cc: "Zend FW Core List" <fw-core@...>
> Sent: Tuesday, March 13, 2007 5:15 PM
> Subject: Re: [fw-core] Concerns about Zend_Filter_Input
>
>
>> I also don't think it makes sense given the changes made (and needed)
>> to do 1.0 so fast. On the other hand, it might be better to push a 1.0
>> out, let the community use it for 6 months, and then make the
>> necessary changes for a 2.0 that will be the more stable long term
>> release. I think the current track means that 1.0 will really be a
>> public beta and 2.0 will be the real 1.0.
>>
>>
>> Kevin McArthur wrote:
>>> Why the push to 1.0 so fast, when things can't as readily be changed
>>> after?
>>>
>>> With so many changes in 0.8 and now some serious 0.9 changes, might
>>> make sense to slow down the run-up to 1.0 a bit. If not, then the
>>> same level of change will occur post 1.0 and instead of point
>>> releases, there'll be a bi-monthly majorversion jump.
>>>
>>> I'd propose a 0.8.5, 0.9.5 and 0.9.9 before 1.0 might give time to
>>> get all the API changes out of the way.
>>>
>>> $0.02
>>>
>>> Kevin
>>> ----- Original Message ----- From: "Bill Karwin" <bill.k@...>
>>> To: "Ed Finkler" <coj@...>
>>> Cc: "Zend FW Core List" <fw-core@...>
>>> Sent: Tuesday, March 13, 2007 12:58 PM
>>> Subject: RE: [fw-core] Concerns about Zend_Filter_Input
>>>
>>>
>>>> -----Original Message-----
>>>> From: Ed Finkler [mailto:coj@...]
>>>>
>>>> Let me reiterate my second question:
>>>>
>>>> "2) Will a similar replacement component, one that forces developers
>>> to
>>>> access user input via a filtering API, be included in the 1.0RC?"
>>>
>>> No, there will not be such a component in ZF 1.0.
>>>
>>> We're very close to our scheduled feature-freeze for Zend Framework 1.0
>>> (March 15).  A solution to satisfy the use case you describe is not
>>> going to be implemented by our deadline.
>>>
>>>> Can any of the filter classes that are now part of the ZFW process
>>>> arrays?  All of the usage cases I see are handling scalars.
>>>
>>> Right, currently the Zend_Filter classes operate on scalars.  Using
>>> these classes to apply filtering and validation to arrays or other
>>> aggregations of data is left for the developer to do, in a manner
>>> appropriate for his application.
>>>
>>> We encourage proposals for solutions that utilize the Zend_Filter and
>>> Zend_Validate classes, to apply filtering and validation to aggregations
>>> of input data.  So far there has been discussion, but no design has been
>>> formally proposed.
>>>
>>> We will evaluate the proposals and a solution will be added in a future
>>> release of Zend Framework (e.g. 1.1 or later).
>>>
>>> Regards,
>>> Bill Karwin
>>>
>
>

[Eric Coleman-3]

Same with Ed and Ersin... also kinda liked the static member access  
in certin circumstances when i needed to just double check some  
values...

On Mar 13, 2007, at 8:35 PM, Ersin Yilmaz wrote:

> Could not agree more with Ed. The application I'm currently working  
> on is
> still using ZF 0.7.0.  And I'm hesitant to upgrade to 0.8.0 until I  
> can
> replace Zend_Filter_Input with another component.
>
> We are doing the exact same thing in our bootstrapper file -  we are
> registering $_GET, $_POST, $_REQUEST globals with Zend_Filter_Input to
> disallow direct access, and this has worked great so far.
>
> I also vote for not rushing into 1.0.
>
> thanks,
> Ersin
>
> On 3/13/07, Ed Finkler <coj@...> wrote:
>>
>> Thank you for clarifying the situation, Bill.
>>
>> Let me start off with some context.  If we do a search for 'php'  
>> in the
>> 2006 records of the NIST NVD, over *40%* of the total number of
>> vulnerabilities are returned.  Removing language level issues,  
>> which would
>> account for a small number of these, we're left with an enormous  
>> number of
>> PHP application vulnerabilities -- well over 1/3 of all the 2006  
>> entries in
>> the NVD, for sure.
>>
>> I believe those numbers say a lot about the following things:
>> 1. How popular PHP is
>> 2. How much web applications are a target
>> 3. How many PHP devs are incapable of writing secure applications
>>
>> I don't necessarily fault PHP as a language for this, as I think any
>> language of similar ease and popularity would suffer many of the same
>> problems.  And as more and more of our business and social  
>> activity is
>> conducted online, web applications are inevitably going to  
>> increasingly be
>> targeted.
>>
>> But we can do a *lot* about #3, in two major areas: _education_, and
>> _empowerment_.
>>
>> Education is fairly easy to understand, I think: books, training  
>> courses,
>> articles, tutorials, etc.  In a sense, the ZFW project has a  
>> significant
>> opportunity to educate PHP developers by serving as an example to the
>> community.  Zend's profile in the community gives it an  
>> opportunity to raise
>> the bar, and change expectations about what effective, secure web app
>> development entails.
>>
>> Empowering developers, in my mind, means making it easier to do  
>> the right
>> thing.  Our toolsets -- editors, debuggers, frameworks, etc --  
>> should make
>> it _easier_ to write secure applications.  The default action  
>> should be the
>> secure action.  It shouldn't be _impossible_ to do potentially  
>> dangerous
>> stuff, but it should require clear intent on the part of the  
>> developer.
>>
>> This is where I feel the Zend Framework has *regressed* as of  
>> version 0.8.
>> Out of the box, it is *more difficult* to write secure  
>> applications with ZFW
>> as of 0.8.0 than it was in 0.7.0.
>>
>> Stripping Tags.  In 0.7:
>> <pre>
>> require "Zend/Filter.php";
>> $searchtxt = Zend_Filter::noTags($searchtxt);
>> </pre>
>>
>> In 0.8:
>> <pre>
>> require "Zend/Filter/StripTags.php";
>> $noTags    = new Zend_Filter_StripTags();
>> $searchtxt = $noTags->filter($searchtxt);
>> </pre>
>>
>> Only one extra line, but when you're having to instantiate a new  
>> filter
>> every time you want different functionality, it adds up.  And if  
>> you're not
>> using class autoloading, you have to do a require statement for  
>> every filter
>> class, whereas in 0.7 you only needed to do a single require.
>>
>> In some ways, the new Zend_Filter is a win, especially in terms of
>> expandability, but it lacks the proper hooks for making filtering and
>> validation as speedy and easy as it was before.  Make no mistake:  
>> verbosity
>> *matters*.  I am not alone in preferring an option that makes me  
>> type less.
>>
>>
>> More disturbing, though, is the total removal of the  
>> Zend_Filter_Input
>> component from the ZFW.  ZFI was a very effective tool for  
>> limiting access
>> to user input arrays like $_POST and $_COOKIE, wrapping them in an  
>> object
>> and forcing the developer to use object methods to access the data.
>>
>> <code>
>> require "Zend/Filter/Input.php";
>> $f_POST = new Zend_Filter_Input($_POST);
>> $searchtext = $f_POST->noTags('searchtxt');
>> $id = $f_POST->getInt('id');
>>
>> // you can only get the raw input with the getRaw method
>> $unfiltered_input = $f_POST->('xssWaitingToHappen');
>> </code>
>>
>> It wasn't perfect -- as of 0.7 it lacked iterative access and  
>> didn't work
>> well with multidimensional arrays -- but it established a strong new
>> paradigm for PHP development that was a long time in coming.  The  
>> vast
>> majority of vulnerabilities in PHP apps come from mishandling user  
>> input,
>> and Zend_Filter_Input made it significantly easier to do the right  
>> thing,
>> and significantly harder to do the wrong thing.  This was a  
>> *major* jewel
>> for the Zend Framework, something no other high-profile PHP  
>> framework was
>> offering (AFAIK).  And it evidently had been considered a fundamental
>> component, having been present in the core since the initial  
>> public release
>> (0.1.2).  Now, it's gone, with no comparable functionality in  
>> place, and
>> no concrete plans to restore it.
>>
>> Zend has a significant opportunity to influence how PHP  
>> development is
>> done with the ZFW 1.0 release.  The Zend Framework will be, for  
>> better or
>> worse, considered an example as to how PHP development *should* be  
>> done.
>> It's disappointing, then, to see the framework move backwards in  
>> terms of
>> easing secure development while it speeds towards 1.0.  This  
>> opportunity
>> will not come again.  I hope these decisions are reconsidered, and  
>> the
>> feature freeze is delayed so that previously-existing  
>> functionality can be
>> restored.
>>
>> Thanks,
>>
>> --
>> Ed Finkler
>> Web and Security Archive Administrator
>> CERIAS - Purdue University
>> http://www.cerias.purdue.edu/
>> v: 765.496.6762  f: 765.496.3181
>> AIM: funkatronwork
>> ICQ: 301234029
>>
>>
>>
>>
>> On Mar 13, 2007, at 3:58 PM, Bill Karwin wrote:
>>
>> -----Original Message-----
>> From: Ed Finkler [mailto:coj@...  
>> <coj@...>]
>>
>> Let me reiterate my second question:
>>
>> "2) Will a similar replacement component, one that forces developers
>>
>> to
>>
>> access user input via a filtering API, be included in the 1.0RC?"
>>
>>
>> No, there will not be such a component in ZF 1.0.
>>
>> We're very close to our scheduled feature-freeze for Zend  
>> Framework 1.0
>> (March 15).  A solution to satisfy the use case you describe is not
>> going to be implemented by our deadline.
>>
>> Can any of the filter classes that are now part of the ZFW process
>> arrays?  All of the usage cases I see are handling scalars.
>>
>>
>> Right, currently the Zend_Filter classes operate on scalars.  Using
>> these classes to apply filtering and validation to arrays or other
>> aggregations of data is left for the developer to do, in a manner
>> appropriate for his application.
>>
>> We encourage proposals for solutions that utilize the Zend_Filter and
>> Zend_Validate classes, to apply filtering and validation to  
>> aggregations
>> of input data.  So far there has been discussion, but no design  
>> has been
>> formally proposed.
>>
>> We will evaluate the proposals and a solution will be added in a  
>> future
>> release of Zend Framework (e.g. 1.1 or later).
>>
>> Regards,
>> Bill Karwin
>>
>>
>>
>
>
> --
> Ersin Yilmaz
> ersiny@...

[Ed Finkler]

Christopher Thompson wrote:
> Docs can be rewritten. I think reality may trump designing at this
> point. So far we have a boat that we know floats and rumor has it that
> Matthew can make it go straight. But you need to launch it to see how it
> really sails.

The push to a quick 1.0 release is motivated, as Bill said, by the
desire by users to have a stable API to work with.  That's great, but I
think it's contrary to that end to *drop* features and lose backwards
compatibility less than a month before feature freeze.

I'm a minor dev for ZFW, but I'm much more of a user of it, and while I
certainly want a stable API, this is not how I'd expect or like it to
happen.  Adding a new system that's more flexible is *great*, but losing
several of the things that were attractive features of ZFW in the
process is, I think, a serious mistake.

But, it's my two cents.  What happens will happen. 8)

--
Ed Finkler
Web and Security Archive Administrator
CERIAS - Purdue University
http://www.cerias.purdue.edu/
v: 765.496.6762  f: 765.496.3181

[Rob Allen-3]

Ed Finkler wrote:

>
> This is where I feel the Zend Framework has *regressed* as of version
> 0.8.  Out of the box, it is *more difficult* to write secure
> applications with ZFW as of 0.8.0 than it was in 0.7.0.
>
> Stripping Tags.  In 0.7:
> <pre>
> require "Zend/Filter.php";
> $searchtxt = Zend_Filter::noTags($searchtxt);
> </pre>
>
> In 0.8:
> <pre>
> require "Zend/Filter/StripTags.php";
> $noTags    = new Zend_Filter_StripTags();
> $searchtxt = $noTags->filter($searchtxt);
> </pre>
>
> Only one extra line, but when you're having to instantiate a new filter
> every time you want different functionality, it adds up.  And if you're
> not using class autoloading, you have to do a require statement for
> every filter class, whereas in 0.7 you only needed to do a single require.
>
> In some ways, the new Zend_Filter is a win, especially in terms of
> expandability, but it lacks the proper hooks for making filtering and
> validation as speedy and easy as it was before.  Make no mistake:
> verbosity *matters*.  I am not alone in preferring an option that makes
> me type less.
>

I found exactly the same issue
(http://www.nabble.com/Zend_Filter:-Simple-use-cases--t3225574s16154.html).
No one in the "core team" responded though, so I don't know why the
simple case has been made that much more hassle to write.

Personally, I've come to the conclusion that in a lot of cases, it's
generally easier (i.e. less lines of code in the action function) to
just roll your own solution than use Zend_Filter or Zend_Validate.


> More disturbing, though, is the total removal of the Zend_Filter_Input
> component from the ZFW.  ZFI was a very effective tool for limiting
> access to user input arrays like $_POST and $_COOKIE, wrapping them in
> an object and forcing the developer to use object methods to access the
> data.

I really liked Zend_Filter_Input too because it made the "safe" way the
easiest way. Usually in PHP, the easiest way is the most dangerous way
and ZFI was the first time I had seen a solution that reversed that.
(e.g. Zend_View provides an escape() function rather than an asRaw()
function)


Moving forward from where we are at the moment, I'd like to see a way of
integrating filtering into the request object so that I could do:

$searchTxt = $this->_request->getFilteredParam('searchTxt','StripTags');

or something similar so that my controller code is less verbose.


Regards,

Rob...

[weierophinney]

-- Rob Allen <rob@...> wrote
(on Thursday, 15 March 2007, 10:01 AM +0000):

> Ed Finkler wrote:
> > More disturbing, though, is the total removal of the Zend_Filter_Input
> > component from the ZFW.  ZFI was a very effective tool for limiting
> > access to user input arrays like $_POST and $_COOKIE, wrapping them in
> > an object and forcing the developer to use object methods to access the
> > data.
>
> I really liked Zend_Filter_Input too because it made the "safe" way the
> easiest way. Usually in PHP, the easiest way is the most dangerous way
> and ZFI was the first time I had seen a solution that reversed that.
> (e.g. Zend_View provides an escape() function rather than an asRaw()
> function)
>
>
> Moving forward from where we are at the moment, I'd like to see a way of
> integrating filtering into the request object so that I could do:
>
> $searchTxt = $this->_request->getFilteredParam('searchTxt','StripTags');
>
> or something similar so that my controller code is less verbose.

I've been waiting to do any filter integration until after Zend_Validate
and Zend_Filter stabilized. I expect you will start seeing this kind of
integration post-1.0; I've already got it on my roadmap.

--
Matthew Weier O'Phinney
PHP Developer            | matthew@...
Zend - The PHP Company   | http://www.zend.com/

[Jim Scherer]

As a newbie I was very disappointed to see Zend_Filter_Input go away. I've tried to understand the issues around input filtering since I started tinkering with PHP several years ago but I don't feel that I have ever really grasped what it is exactly that I have to do. So to take away a simple helper like Zend_Filter_Input from me means that I'm not filtering input, and that's unfortunate. Hopefully future code examples of Zend_Filter and Zend_Validate will help me to understand how to properly incorporate input filtering into my applications.

I would also like to voice my disapproval of any push to a quick 1.0 release. One of the reasons I decided to invest my time understanding the ZF had to do with the involvement and guidance provided by Zend. My trust has been placed in the people from Zend and so far has been rewarded.

Ed Finkler wrote:

Christopher Thompson wrote:
> Docs can be rewritten. I think reality may trump designing at this
> point. So far we have a boat that we know floats and rumor has it that
> Matthew can make it go straight. But you need to launch it to see how it
> really sails.

The push to a quick 1.0 release is motivated, as Bill said, by the
desire by users to have a stable API to work with.  That's great, but I
think it's contrary to that end to *drop* features and lose backwards
compatibility less than a month before feature freeze.

I'm a minor dev for ZFW, but I'm much more of a user of it, and while I
certainly want a stable API, this is not how I'd expect or like it to
happen.  Adding a new system that's more flexible is *great*, but losing
several of the things that were attractive features of ZFW in the
process is, I think, a serious mistake.

But, it's my two cents.  What happens will happen. 8)

--
Ed Finkler
Web and Security Archive Administrator
CERIAS - Purdue University
http://www.cerias.purdue.edu/
v: 765.496.6762  f: 765.496.3181

[Rob Allen-3]

Matthew Weier O'Phinney wrote:
> -- Rob Allen <rob@...> wrote
> (on Thursday, 15 March 2007, 10:01 AM +0000):

>> Moving forward from where we are at the moment, I'd like to see a way of
>> integrating filtering into the request object so that I could do:
>>
>> $searchTxt = $this->_request->getFilteredParam('searchTxt','StripTags');
>>
>> or something similar so that my controller code is less verbose.
>
> I've been waiting to do any filter integration until after Zend_Validate
> and Zend_Filter stabilized. I expect you will start seeing this kind of
> integration post-1.0; I've already got it on my roadmap.
>

That's good :)

I'm thinking that 1.1 or 1.2 will end up being the "release that
everyone uses" as it'll have all the stuff that didn't quite make 1.0
but is actually really useful.

Regards,

Rob...

[russx2]

I must agree with regards to it being a shame that the Zend_Filter_Input class has been removed.

I've read the discussions regarding the new atomic filters/validators and agree on an academic level that the new system is a purer way to go about it. But when you get down to it, it does seem like considerable overkill for the average developer using the Zend Framework. Having to instantiate a filter object for each type of filtering required? It just seems... too much.

However, it does seem to be agreed that a similar system to the orginal Zend_Filter_Input will be wrapped around the current implementation and be included "post 1.0". Why release 1.0 before such a key part of the framework is in place?

It just a seems a bit of a shame - I'm concerned that the people who have been waiting for the 1.0 will look at it and walk away.

Russ Hall