« Return to Thread: Discussion point: CONSPEC - Context-specific Issues
Re: Patr Traversal and some challenges for CWE
by Steven M. Christey-2
::
Rate this Message:
On Wed, 14 Nov 2007, Chris Telfer wrote:
> There are usually a huge number of different ways for a programmer to
> produce the same type of weakness. We already acknowledge this in some
> part through the notion of "complexities" that color a flaw (ala
> SAMATE). I believe that different ways to express the same weakness
> (e.g. '/' vs '\\' as a path separator) are simply "complexities" for a
> weakness that don't apply generally across all weakness types.
Lately, Bob Martin and I have begun exploring certain concepts of "chains"
of weaknesses, as well as "composites" of properties whose combination
forms a weakness.
An increasingly-reported "chain" would be an integer overflow that causes
insufficient allocation of memory, which then leads to a buffer overflow.
A second chain involging integer overflow would lead to an infinite loop,
and could be independent of memory allocation altogether (we see examples
of this in CVE).
The "weakness that is better known by the attack term 'Symbolic Link
Following'" can be thought of as a composite in which one needs
predictability (of filenames), a race condition, path equivalence (where
two paths ultimately point to the same file object), and a shared
directory to exist all at once, in order to be present. Removing any one
of these elements would prevent the issue or, at worst, significantly
reduce its scope.
Many of the CWE path traversal variants are probably reflections of
different chains and/or composites. This gets closer to understanding the
variety of ways for programmers to cause the same problem.
Note: Bob and I are only in the early stages of developing these concepts.
> As an example of the danger, imagine the scenario: You have tool A that
> can cover a particular weakness, for example path traversal with '/', in
> a myriad of contexts programming contexts. However, tool A doesn't find
> '\\' traversal problems. Another tool, B, has covers all path traversal
> issues but only covers them in the narrowest interpretation of each of
> the weakness complexities and thus doesn't find nearly the breadth of
> issues that will actually arise in code. Breaking the CWEs down to
> fine-grained weaknesses would give tool B an artificially high and
> dubious evaluation rating. Due to this, tool vendors will have more
> incentive to build tools like B rather than A.
I've been thinking lately that one way to handle this might be to define
one or more "tool evaluation" views that have a fairly fixed level of
abstraction and omit all nodes that are higher or lower levels. If chosen
well, these views could provide more consistent evaluation ratings, at the
expense of precision. This is just an early thought, however.
> Incidentally, I think that the ideal CWE compliance metric would grade
> CWE compliance according to how _well_ a tool finds a flaw and not just
> _whether_ it does or doesn't in some situations that we have test cases
> for.
I suspect the SAMATE project hopes to get to this point.
> Evaluating coverage of a CWE should be separated from the definition
> of "what defines this weakness" and will require much more thought.
This is an intriguing proposal that might address a few separate problems
at once - or, at worst, put them in separate buckets.
- Steve
> There are usually a huge number of different ways for a programmer to
> produce the same type of weakness. We already acknowledge this in some
> part through the notion of "complexities" that color a flaw (ala
> SAMATE). I believe that different ways to express the same weakness
> (e.g. '/' vs '\\' as a path separator) are simply "complexities" for a
> weakness that don't apply generally across all weakness types.
Lately, Bob Martin and I have begun exploring certain concepts of "chains"
of weaknesses, as well as "composites" of properties whose combination
forms a weakness.
An increasingly-reported "chain" would be an integer overflow that causes
insufficient allocation of memory, which then leads to a buffer overflow.
A second chain involging integer overflow would lead to an infinite loop,
and could be independent of memory allocation altogether (we see examples
of this in CVE).
The "weakness that is better known by the attack term 'Symbolic Link
Following'" can be thought of as a composite in which one needs
predictability (of filenames), a race condition, path equivalence (where
two paths ultimately point to the same file object), and a shared
directory to exist all at once, in order to be present. Removing any one
of these elements would prevent the issue or, at worst, significantly
reduce its scope.
Many of the CWE path traversal variants are probably reflections of
different chains and/or composites. This gets closer to understanding the
variety of ways for programmers to cause the same problem.
Note: Bob and I are only in the early stages of developing these concepts.
> As an example of the danger, imagine the scenario: You have tool A that
> can cover a particular weakness, for example path traversal with '/', in
> a myriad of contexts programming contexts. However, tool A doesn't find
> '\\' traversal problems. Another tool, B, has covers all path traversal
> issues but only covers them in the narrowest interpretation of each of
> the weakness complexities and thus doesn't find nearly the breadth of
> issues that will actually arise in code. Breaking the CWEs down to
> fine-grained weaknesses would give tool B an artificially high and
> dubious evaluation rating. Due to this, tool vendors will have more
> incentive to build tools like B rather than A.
I've been thinking lately that one way to handle this might be to define
one or more "tool evaluation" views that have a fairly fixed level of
abstraction and omit all nodes that are higher or lower levels. If chosen
well, these views could provide more consistent evaluation ratings, at the
expense of precision. This is just an early thought, however.
> Incidentally, I think that the ideal CWE compliance metric would grade
> CWE compliance according to how _well_ a tool finds a flaw and not just
> _whether_ it does or doesn't in some situations that we have test cases
> for.
I suspect the SAMATE project hopes to get to this point.
> Evaluating coverage of a CWE should be separated from the definition
> of "what defines this weakness" and will require much more thought.
This is an intriguing proposal that might address a few separate problems
at once - or, at worst, put them in separate buckets.
- Steve
« Return to Thread: Discussion point: CONSPEC - Context-specific Issues
| Free embeddable forum powered by Nabble | Forum Help |
