Nabble - CWE Research List

CWE Version 1.6 Now Available

2009-11-02T19:53:01Z

CWE Version 1.6 [1] has been posted on the CWE List page. A detailed
report [2] is available that lists specific changes between Version 1.5
and Version 1.6.

The main changes include: (1) creation of 4 new entries with no entries
deprecated; (2) cleanup of the general-purpose Other_Notes field in 84
entries, which typically moved content into other more relevant fields
within those entries, especially Common_Consequences; (3) modified
descriptions for 49 entries stemming from the Other_Notes modification and
continued efforts to establish a common vocabulary; (4) promotion of three
entries from "Draft" to "Usable" status; and (5) updated relationships for
50 entries, including a partial restructuring of CWE-119 to better handle
closely-related buffer-overflow concepts. There were no schema changes in
this version.

The new entries are:

CWE-786 Access of Memory Location Before Start of Buffer
CWE-787 Out-of-bounds Write
CWE-788 Access of Memory Location After End of Buffer
CWE-789 Uncontrolled Memory Allocation

The "Stakeholder Field Priorities" document [3] has been modified to
reflect additional stakeholders, new CWE fields, and changing priorities.
The CWE/SANS Top 25 document has been updated to reflect the latest
changes in names and attack patterns. PDF documents have been updated to
display graphs of views such as the Research View (CWE-1000) and the
Development View (CWE-699), and a "Printable CWE" document lists all of
the entries in CWE.

Please send any comments or concerns to cwe@..., or post them to
this list.

Regards,
Steve Christey
CWE Technical Lead

[1] http://cwe.mitre.org/data/index.html
[2] http://cwe.mitre.org/data/reports.html
[3] http://cwe.mitre.org/data/reports/stakeholder_field_priorities.html

OWASP Hartford: October 2009

2009-10-02T12:28:13Z

OWASP Hartford: October 2009

When: Tuesday, October 13, 2009 5:00 PM-7:00 PM (GMT-05:00) Eastern Time (US & Canada).
Where: The Hartford: Tower Building, 22nd Floor

*~*~*~*~*~*~*~*~*~*

The Open Web Application Security Project (OWASP) is a worldwide free and open community focused on improving the security of application software. Our mission is to make application security "visible," so that people and organizations can make informed decisions about application security risks. Everyone is free to participate in OWASP and all of our materials are available under a free and open software license.

The agenda for this meeting is posted at: http://www.owasp.org/index.php/Hartford

To receive future invites, please subscribe to our mailing list at: https://lists.owasp.org/mailman/listinfo/owasp-hartford

************************************************************
This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information.  If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited.  If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
************************************************************

Re: A Proposal to Precisely Define Weaknesses

2009-10-02T10:35:54Z

Michael and Paul,
Yours is an interesting proposal, and the goals are laudable. I have
organized my comments by corresponding section in your proposal.

Section 1.
I accept the premise that careful definitions are better, within the limits of
what is possible or practical.

Section 2.
I find your examples of difficult cases interesting and well-chosen. They
illustrate things that IMO are wrong with not just the practices of many
programmers, but also with the "C" language and its compilers. Inasmuch as the
examples have several issues, asking which single weakness that code represents
is itself an ill-posed question. I would think that language and
compiler/hardware bugs are out-of-scope for the CWE. Yet, I believe that
you are correct in that the CWE should be robust vs these difficulties.

The code in the SRD test case 201 is wrong at the semantics level, and at that
level it is obviously a buffer overflow. That is because the semantics of the
type declaration are violated. I believe that the point of the example is that
the compiler happens to allocate extra memory for efficiency reasons, so the
"actual" buffer is not overflowed. However, what is wrong here is that the
combination of C language, compiler and architecture introduced an ambiguity
about what constitutes the buffer. The conceptual, "semantic buffer" is
different from the actual one. It's a fallacy to think that a program that
doesn't trample other memory structures or doesn't crash, due to a happenstance
combination of compiler and architecture, is correct. IMO it is unfair to ask
that the CWE definitions resolve ambiguities within languages and compilers
themselves. I think that the CWE should operate and apply at the semantic level
of the code because this is where programmers do their work. Ambiguities and
bugs within specific languages and compilers should be the problem of the
relevant standards bodies or compiler vendors. I am tempted to say that the
fact that the program doesn't crash or produce an error is a weakness in the
specification of the "C" language ;).

The example with alloca() makes it clear that rare cases can be overlooked
when writing definitions for CWE. It's rare because its use is
discouraged and it is not in POSIX.1-2001, according to various man pages.
Consequently I find this oversight not upsetting, even though your point is
well made. I agree that it would be better if things like this didn't happen.

The SRD test case 188 is similar to 201 and rather obvious to me; it is wrong
at the semantics level, as there does not exist an element at index 17 for buf1,
and at that level it is obviously a buffer overflow. As an example possible
consequence, with most compilers and in most environments the contents of buf2
get overwritten, and if those contents had a security meaning then all kinds of
bad things could happen. That the access doesn't go outside the struct and
the program happens not to crash is irrelevant.

So, how do we make the CWE robust vs these issues? As a general comment on
those examples, it seems that several were discussed because the consequences
of the violations were possibly or likely benign. I'd argue that the possible
consequences don't matter, and it many cases they are hard to fathom anyway.
What matters (but see below) is that at a semantic level, a close similarity is
established between those violations and weaknesses known to produce
vulnerable, exploitable code, for the purpose of being classified as instances
of those weaknesses.

I love the example of a network programming structure with a payload, and how
you describe it. It is obviously a hack, albeit a popular hack that works
well and is convenient. It is also the perfect counterexample to what I've
said above. The literal semantics of the code are violated, yet few people
would consider this an example of CWE-121. This is because at a higher level,
we can figure out the intent of the programmer and that the code is safe.
However, if we need to guess at programmer intent on a regular basis in order
to figure out that something is or isn't a weakness, the task is much harder
and difficult to automate. I believe, though, that a scanner could be programmed
to treat an array with a declared size of zero as possibly being a declaration
that the size is probably variable and will be determined later. If it is so
popular, then perhaps the standard needs to be changed and allow a special
notation for an array of a size TBD (to be determined), so programmers can
write what they mean? I think that this is really a problem with standards and
compilers, out of scope for the CWE to resolve. Nevertheless, to achieve the
goal of robustness, the CWE folks are stuck with defining an augmented or
variant "C" instead of the one defined in the standard. If this example is
only one of a few exceptions, though, it is manageable.

Section 3.
a) I have reservations on the practicality of requiring the CWE definitions to
be reviewed by experts external to MITRE. The CVE had a review and vote model
that was overwhelmed by the creation rate of new vulnerabilities. I don't
expect that the number of CWE entries will be anywhere as high as the number of
CVE entries, but in theory the number of ways in which people can get things
wrong is very high and the known ones could keep increasing every year. The
number of replies to your proposal might be indicative of the number of
volunteers MITRE would get for reviewing CWE entries. Still, the fact that the
SC22/WG23 effort is making progress is encouraging. I guess we'll really know
only if the CWE folks actually make a request.

b) I'm slightly worried that focusing on a "prime definition" will set the stage
for excessive rules- or wording- lawyering, instead of using "reasoned common
sense". This phenomenon is common wherever a single prime definition exists.
For example, in some games lacking examples of what the rules mean, the exact
grammar and definitions of words get analysed to a ridiculous extent. Let's
mention also "legalese", the distortion of human languages attempting to
close loopholes and reach precise meanings. Its existence is almost proof
that if we want precise and correct prime definitions, we shouldn't use a
natural language.

c) Another aspect is that whereas designating a single, prime definition removes
ambiguity and inconsistencies from consideration, assuming that they are not
contained within the prime definition itself, it also removes redundancy, which
is very useful for catching errors. In a way, I see this proposal as having
disturbing similarities to removing checksums from network messages, or
considering them as wrong if a calculated checksum doesn't match, which defeats
the purpose. The human mind uses many points of references to form a concept;
I'm not sure that a single definition would be able to capture all the
viewpoints and nuances of a problem. I see this as a different issue from your
discussion point as to whether elements refer to completely orthogonal ideas.
In Steve's answer, I note that indeed differences between fields can result in
deprecating an entry. This confirms the usefulness of redundancy in spotting
"definitional confusion/vagueness". Granted, if things were done really well
to start with, like you suggest, then perhaps the redundancy wouldn't be
needed (and in a perfect world we wouldn't need backups ;-)). I like the way
it's been handled by the CWE team.

d) A minor concern is that focusing on a single definition also devalues the
other fields, so perhaps they will get less attention than they deserve. I
think all are useful for humans attempting to learn or understand what a
particular weakness is. Could this, as a side effect, make the CWE slightly
more difficult to use, e.g., because there would be fewer examples?

Regards,
Pascal Meunier

On Fri, 1 May 2009 14:51:02 -0400
"Michael Koo" <koo@...> wrote:

> A Proposal to Precisely Define Weaknesses
>
> by Paul E. Black
>
> 1 May 2009
>
>
>
> Although much work has been done, which has greatly improved the definitions
> of source code weaknesses, there are still inconsistencies, ambiguities, and
> errors. Better definitions are needed as one element to speed the work of
> detecting and preventing weaknesses.
>
>
>
> We propose, first, that one element of a CWE be designated the prime
> definition. If there is inconsistency between the prime definition and a
> description, example, or definition in another element, the prime definition
> is considered to be correct. If there is an error, the prime definition is
> the first to be fixed. Other descriptions and definitions can later be
> brought into alignment. Second we propose a specific set of reviews and
> other steps as necessary and sufficient to consider a prime definition to be
> thoroughly reviewed.
>
>
>
> This document has three sections: motivation for precisely and accurately
> defining weaknesses, comments on the current state of definitions, and the
> proposal itself.
>
>
>
> If you disagree (or agree!) with the proposal or have suggestions, please
> respond. Feel free to share this.
>
>
>
>
>
> SECTION 1. WHY CAREFULLY DEFINE WEAKNESSES?
>
>
>
> An important prelude to preventing, mitigating, or detecting weaknesses in
> software and systems is to have clear, unambiguous, widely-accepted
> definitions of such weaknesses. Consider the following questions that might
> occur to someone learning about software weaknesses. What precisely is a
> buffer overflow? Is it the same as a heap overflow or an unbounded
> transfer? Is one just a refinement of another? If an integer overflow
> leads to memory violation, which weakness is it? Is it both or is there
> some other relation between them? Precise definitions could answer these
> questions and others.
>
>
>
> Many people have done excellent work toward clearly defining types of
> weakness. (Hereafter we usually use the term "weakness" to mean a weakness
> class or type, not a particular instance of a weakness type.) Some papers
> and work are "Seven Pernicious Kingdoms: A Taxonomy of Software Security
> Errors" (Tsipenyuk, Chess, and McGraw), "The CLASP Application Security
> Process", (Viega), "The Preliminary List of Vulnerability Examples for
> Researchers (PLOVER)" (Christey), "The Ten Most Critical Web Application
> Security Vulnerabilities" (OWASP), "19 Deadly Sins of Software Security
> Programming Flaws and How to Fix Them" (Howard, LeBlanc, and Viega), "A
> Taxonomy of Computer Program Security Flaws, with Examples" (Landwehr, Bull,
> McDermott, and Choi), (see http://cwe.mitre.org/about/sources.html for more)
> "Structured CWE Descriptions" (Christey, Harris, Heinbockel) available at
> http://cwe.mitre.org/documents/structured_descriptions/, ISO/IEC Project
> 22.24772: Programming Language Vulnerabilities available at
> http://aitc.aitcnet.org/isai/ and many others. KDM Analytics has produced
> formal definitions of some 30 weaknesses.
>
>
>
> Since 2006 MITRE has led the Common Weakness Enumeration (CWE) effort to
> collect, reconcile, organize, and define weaknesses. Not only has this work
> improved and unified definitions, it has also led to discovery of
> fundamental clarifying concepts, such as chains and composites of weaknesses
> ("Chains and Composites", Steve Christey,
> http://cwe.mitre.org/data/reports/chains_and_composites.html).
>
> Precise definitions can lead to further better understanding of weaknesses,
> their causes, cures, and preventions.
>
>
>
> There are still inconsistencies and ambiguities within CWEs. Consider that
> a CWE may have many descriptive elements. Some in CWE Version 1.0 are
>
> Black_Box_Definition (possibly more than one)
>
> White_Box_Definition (possibly more than one)
>
> Description
>
> Description_Summary
>
> Extended_Description
>
> Compound_Element
>
> Name (Weakness)
>
> Demonstrative_Example
>
> Observed_Example
>
> Note
>
> Theoretical_Note
>
> Could inconsistencies be eliminated by deleting all but one such element in
> each CWE? That is not practical. One definition cannot serve all people in
> all instances. Training and reference needs succinct prose definitions and
> examples. Automated tools need a definition written in machine-readable
> languages. When discussion gets specific, people need the nuances and
> details of what constitutes that weakness. Proving that a weakness is
> absent or is prevented by some approach needs a rigorous, mathematically
> formal definition.
>
>
>
> Improving the state-of-practice of software development to reduce instances
> of weaknesses and the vulnerabilities they cause takes work from language
> designers, compiler writers, educators, assurance tool developers, auditors
> and developers of guidance, people who specify and contract software
> development, researchers, vulnerability trackers, software engineers, and
> many more. If people in these roles disagree about what constitutes a
> particular weakness, or even whether it is a weakness at all, communication
> is difficult at best. At worst they may work at cross purposes. Broadly
> accepted definitions should allow different groups to work together more
> effectively.
>
>
>
> Unambiguous, complete definitions allows those in the field to understand
> precisely what different software assurance tools, services, technologies,
> or methods can detect, mitigate, or prevent.
>
> Formal definitions may allow tools to automatically check for weaknesses,
> create wrappers to filter out attacks to exploit them, or even rewrite the
> code to eliminate them.
>
>
>
>
>
> SECTION 2. DOES THE CURRENT METHOD NEED IMPROVEMENT?
>
>
>
> Currently there isn't a documented process to validate weakness definitions.
> Descriptions, definitions, and examples in the CWE have been reviewed and
> improved by different entities and the quality and consistency have improved
> dramatically over the years. Yet, even casual examination shows that most,
> if not all, CWEs need further work.
>
>
>
> Given the number of CWEs and their range of complexity, frequency, and
> severity, it is probably not worthwhile to precisely define every single
> weakness. But for those which need precise definitions, it is not clear
> exactly what should be. As with code, an essentially unlimited amount of
> checking and review could be done for each weakness. The community should
> agree on the types and amounts of validation that are necessary and
> sufficient.
>
>
>
> To be specific, currently every CWE has a summary description. A CWE may
> also have alternate term descriptions, demonstrative examples, or a white
> box definition. Thousands of these descriptions have been reviewed and
> improved greatly. But which ones are merely informative and which ones are
> intended to be the carefully reviewed, precise and accurate? Consider
> CWE-121, Stack-based Buffer Overflow. (Text taken from CWE version 1.3).
> Here is the description summary:
>
>
>
> "A stack-based buffer overflow condition is a condition where the
>
> buffer being overwritten is allocated on the stack (i.e., is a
>
> local variable or, rarely, a parameter to a function)."
>
>
>
> Here is the White Box Definition:
>
>
>
> "A buffer overflow where the buffer from the Buffer Write
>
> Operation is statically allocated"
>
>
>
> Is the following an instance of CWE-121? Note that alloca() gets memory
> from the stack, not the heap.
>
>
>
> #define BUFSIZE 256
>
> char *buf;
>
> int main(int argc, char **argv) {
>
> buf = (char *)alloca(BUFSIZE);
>
> strcpy(buf, argv[1]);
>
> }
>
>
>
> The description summary uses "i.e." meaning "that is" or a restatement. A
> strict reading excludes this example because the buffer is only referenced
> by a global variable.
>
>
>
> The white box definition says the buffer is statically allocated, that is,
> allocated automatically by the compiler or language support routine, not by
> functions invoked at run-time, such as alloca().
>
> Again a strict reading excludes this example because the buffer is
> dynamically allocated.
>
>
>
> Yet most people would agree that the example code has an instance of
> Stack-based Buffer Overflow and that the descriptions have minor
> inaccuracies. Even ignoring this particular example, we see there is
> inconsistency between the descriptions.
>
>
>
> The following code snippets highlight not merely quibbles about wording, but
> raise the fundamental question of what CWE-121 means.
>
>
>
> typedef struct
>
> {
>
> char buf1[10];
>
> char buf2[10];
>
> } my_struct;
>
>
>
> my_struct s;
>
>
>
> s.buf1[17] = 'A';
>
>
>
> (from SRD test case 188)
>
>
>
> The C99 standard [1] Sect. 6.7.2.1 Structure and union specifiers, page 102,
> para 5 says fields are "allocated in order". So buf2 makes the structure at
> least big enough that the access doesn't go outside the structure. Any
> particular compiler probably allocates the structure consistently, so the
> above code likely have a specific behavior in each environment, although it
> might differ from environment to environment. Is this an instance as
> CWE-121?
>
>
>
> It is common in network programming to have structures and code like the
> following:
>
>
>
> struct {
>
> int header;
>
> char payload[ 0 ];
>
> } *p;
>
>
>
> p = malloc(sizeof *p + 2);
>
> p->payload[0] = 'A';
>
> p->payload[1] = '\0';
>
>
>
> (adapted from Aurelien Delaitre)
>
>
>
> Nothing in the C standard justifies this code, but most compilers will treat
> it reasonably. Since the code does not strictly follow the C standard, is
> its behavior undefined so that this should not be considered an instance of
> CWE-121?
>
>
>
> Compilers usually allocate structures in multiples of four bytes, so the
> following code should be fine. That is, there is memory for 12 characters
> in the structure. Should this example be considered to write outside of the
> buffer or not? On what grounds?
>
>
>
> typedef struct
>
> {
>
> int int_field;
>
> char buf[10];
>
> } my_struct;
>
>
>
> my_struct s;
>
>
>
> s.buf[10] = 'A';
>
>
>
> (from SRD test case 201)
>
>
>
> We see that precise definitions are important to make consistent judgments.
> We also see that it is very difficult to write a definition which is precise
> and accurate.
>
>
>
> Given all the work which the CWE embodies, how can the effect of
> inconsistencies be minimized? Where should work be concentrated to achieve
> good definitions? All CWEs have a status of draft or incomplete. How much
> and what kind of work should be done before a definition is considered to be
> "thoroughly reviewed", that is, good enough for the community to move on to
> another one? What language, either formal (mathematically precise) or
> prose, should be used to state definitions clearly and so the style is
> similar across CWEs?
>
> What should be done to minimize assumptions or artifacts arising from the
> use of a particular formal language or style?
>
>
>
>
>
> SECTION 3. THE PROPOSAL
>
>
>
> This proposal is part of a vision to improve software assurance by having
> widely-used clear definitions of software weakness classes.
>
> The goals of this proposal are to
>
> * increase precision (minimize ambiguity and inconsistency) in CWE
>
> entries and
>
> * increase accuracy (minimize mistakes indicated by broad agreement).
>
>
>
> The proposal has two points: (1) the need for an prime definition and
>
> (2) a review process. Each point has several parts or supporting clauses
> which are somewhat independent.
>
>
>
> POINT ONE: The community should have one prime definition of each weakness.
>
>
>
> PROPOSAL CLAUSE 1: the Common Weakness Enumeration (CWE) is the repository
> of the prime definition.
>
>
>
> PROPOSAL CLAUSE 2: each weakness has exactly one prime definition.
>
>
>
> All other notes, summaries, descriptions, categories, components, examples,
> definitions, etc. are subordinate to it. For instance, if a summary seems
> to contradict the prime definition, the prime definition is assumed to be
> correct and the summary should be reinterpreted or changed.
>
> There may be times when the prime definition is just wrong and needs to be
> corrected (see review process below), but short of that, the prime
> definition is authoritative.
>
>
>
> The prime definition is a complete description of the weakness. All details
> and nuances are in that element. One need not read any other element to
> understand what is or is not an instance, although other elements may be
> helpful as examples or restatements.
>
>
>
> It is important for the community to choose a prime definition so work can
> be focused on it. Having prime definitions lets us decide that a CWE is
> precisely and accurately described. Other summaries, descriptions, views,
> names, etc. can be brought into agreement as time, resources, and need
> arise.
>
>
>
> The prime description will likely be legalistic and dry, like most formal
> descriptions. One would read something else to initially get a sense of
> what it is or to be reminded of it. Summaries, relationships, and notes
> serve such purposes. Rather the prime definition answers questions when
> differences of opinion or interpretation arise.
>
>
>
> A clearly defined vocabulary is needed. Likely there will be stock phrases,
> too. This vocabulary and the definitions themselves should be based on the
> work already done by many contributors to the CWE.
>
> One example is "Structured CWE Descriptions" (Christey, Harris, Heinbockel),
> available at http://cwe.mitre.org/documents/structured_descriptions/ Another
> is the work by KDM Analytics. Starting from scratch would be wasteful.
>
>
>
> DISCUSSION POINT: if elements refer to completely orthogonal ideas, there
> could be more than one prime element and still has no chance of (internal)
> contradiction.
>
>
>
> We don't think this occurs in CWE.
>
>
>
> DISCUSSION POINT: "prime definition" is only one possible name. Other
> possibilities are master, attested, decisive, prevailing, key, normative,
> authoritative, official, sanctioned, or commonly accepted element or
> definition.
>
>
>
> PROPOSAL CLAUSE 3: different kinds of weaknesses are best expressed with
> different prime elements.
>
>
>
> Most weaknesses are manifest in code, like hard-coded password
>
> (CWE-259) or leftover debug code (CWE-489). (Note that these two are not
> 100% discernible from code alone.) However some weaknesses are "black box",
> behavioral, functional, or external weaknesses, like denial of service
> (CWE-730), configuration (CWE-16), or violation of secure design principles
> (CWE-657).
>
>
>
> PROPOSAL CLAUSE 4: each kind of weakness (see CLAUSE 3) has a prime element.
>
>
>
> For instance, the prime definition for all weaknesses manifest in code might
> be White_Box_Definition.
>
>
>
> DISCUSSION POINT: what is the right element for each kind of weakness? For
> weaknesses manifest in code White_Box_Definition seem most appropriate.
> Application behavior could be described in Black_Box_Definition.
>
>
>
>
>
> POINT TWO: How should the Community Decide a Definition is Right?
>
>
>
> An implementation may be checked for correctness against its specification.
> But how can a specification be checked? The short answer is that it can't
> be fully checked. However, we can take steps to validate a definition and
> minimize mistakes.
>
>
>
> PROPOSAL CLAUSE 5: a CWE prime definition is acceptable when the following
> are done:
>
> (A) the definition is published on the CWE discussion list, and
>
> there is no (or only limited) disagreement that it is precise
>
> (unambiguous) and accurate (correct).
>
> (B) the definition agrees with SC22/WG23 Programming Language
>
> Vulnerabilities if there is a correspondence.
>
> (C) the definition is carefully reviewed by two experts, who find it
>
> to be precise and accurate.
>
> (D) the definition is written in two different "executable" forms
>
> which are tested and deemed to find/generate/mitigate the
>
> weakness (and nothing else).
>
>
>
> We think these steps are the minimum needed to come up with good
> definitions. We need the general consensus of the community to make sure it
> is broadly agreeable. We want definitions which are consistent with other
> standards rather than creating yet another
>
> (inconsistent) "standard". We need experts to make sure the definition says
> the right thing. Finally, we need executable forms to make sure the
> definition is precise.
>
>
>
> Since CWEs are still being developed and the community is self-selected, We
> don't think unanimous agreement is needed.
>
> Nevertheless, any objections should be taken very seriously. The desired
> goal is unanimity.
>
>
>
> The community of concern is large and there are other efforts. As much as
> possible the definitions should be in harmony. One particular effort is
> PDTR 24772 a draft of which is document N0138 at
> http://aitc.aitcnet.org/isai/ Here are some correspondences between CWEs
> and entries in PDTR 24772:
>
> 6.15 Numeric Conversion Errors [FLC]
>
> CWE 192
>
> 6.17 Boundary Beginning Violation [XYX]
>
> CWE 123
>
> CWE 129
>
> 6.18 Unchecked Array Indexing [XYZ]
>
> CWE 129
>
> 6.19 Buffer Overflow in Stack [XYW]
>
> CWE 121
>
>
>
> We need reviews from at least two very different experts to minimize the
> chance that the definition only reflects the view of one person or one
> group. Why not three or more? The only objective reason we offer is that
> economy suggests the fewest possible, and two is the least number greater
> than one.
>
>
>
> Who are the experts? We don't have specific people in mind. It seems it
> should be people who have a lot of experience in the field. It also seems
> like it should be people who have thought about such definitions, so its
> likely to be authors, security researchers, and tool makers. Maybe we
> should designate a pool of experts and any two of them are acceptable. (And
> no objections from "experts"?) Expert availability will probably be a
> factor.
>
>
>
> The definition should be "formalized" in at least two completely different
> ways to minimize the chance of unstated assumptions or severe bias to one
> form of expression. Some ways may be a language for generating test cases,
> rules for a source code scanner to find the weakness, or a purely formal
> description in a highly mathematical notation for proving program
> properties.
>
>
>
> The executable forms and the expert reviews should be completely
> independent. That is, a expert writing an "executable" form only counts as
> an expert review (C above) or an executable (D above), but not both. Also
> the two experts should not be from the same organization.
>
>
>
> It was suggested that one more step be added: that the definition be
> validated to accurately describe some subset of reported instances of the
> weakness, like the CVE. The aim is to make sure the definition corresponds
> to real examples.
>
>
>
> This proposal does not address how the process would be executed, in
> particular where the resources would come from. Nevertheless, We think it
> is important to begin by getting community agreement on these two points.
>
>
>
> -paul-
>
>
>

CWE 1.4 released

2009-05-28T15:34:29Z

CWE 1.4 has been released. Changes include: (1) creation of 15 new
entries, most of which are newly-identified weaknesses; (2) deprecation of
one entry that inadvertently combined multiple weaknesses; (3) usage of a
more established vocabulary in the names and descriptions of 89 entries;
(4) updated relationships for 35 entries; (5) improvements and additions
to demonstrative examples for 75 entries; (6) updated CAPEC attack
patterns for 31 entries; and changes to 198 total entries.

A detailed report is available that lists specific changes between Version
1.3 and Version 1.4:

http://cwe.mitre.org/data/reports/diff_reports/v1.3_v1.4.html

We've also updated the glossary for terms used in CWE (but please note
these are still in development):

http://cwe.mitre.org/documents/glossary/index.html

We've also created new PDF files that contain the entire contents of CWE.
Note that these "Printable CWE" documents are hundreds of pages long, so
you may want to think twice before printing them:

http://cwe.mitre.org/data/index.html

The CWE Top 25 document has also been updated to reflect the latest
changes in names, mitigations, and attack patterns. Note that mitigations
were not affected much:

http://cwe.mitre.org/top25/index.html

There were no schema changes in this version.

The new entries are:

761 Free of Pointer not at Start of Buffer
762 Mismatched Memory Management Routines
763 Release of Invalid Pointer or Reference
764 Multiple Locks of a Critical Resource
765 Multiple Unlocks of a Critical Resource
766 Critical Variable Declared Public
767 Access to Critical Private Variable via Public Method
768 Incorrect Short Circuit Evaluation
769 File Descriptor Exhaustion
770 Allocation of Resources Without Limits or Throttling
771 Missing Reference to Active Allocated Resource
772 Missing Release of Resource after Effective Lifetime
773 Missing Reference to Active File Descriptor or Handle
774 Allocation of File Descriptors or Handles Without Limits or Throttling
775 Missing Release of File Descriptor or Handle after Effective Lifetime

The main additions are for throttling/limiting (as exploited by "resource
exhaustion" attacks) and improper free/delete operations (which previously
could only be classified under the high-level CWE-404). These new entries
reflect some of the changes that we are making to certain "regions" of
CWE. When we released 1.3, we performed a similar regional reorganization
for error handling, as reflected in CWE-754 and CWE-755. We plan to post
short summaries that further explain this kind of organization.

This is the largest number of new weakness-focused entries since the
release of CWE 1.0 last year. (Past releases have often included many new
categories in order to support new views.) In the foreseeable future, we
expect to add more weakness-focused entries as we simultaneously improve
the quality and completeness of existing entries. If you have any
suggestions for new weaknesses or major gaps in CWE, feel free to contact
us at cwe@.... We can use non-disclosure agreements (NDA) if
desired.

Finally, Bob Martin and I would like to thank CWE team members Janis
Kenderdine, Conor Harris, Scott Bennett, and Tom Stracener for all their
contributions to this version.

Thank you for your support of CWE!

Steve Christey
CWE Technical Lead

Re: A Proposal to Precisely Define Weaknesses

2009-05-04T16:58:26Z

Paul and Michael, thank you for raising these important issues along
with your extensive commentary :-)

All - feel free to send any thoughts to this list or privately to
cwe@....

I will respond to a small piece of Paul's proposal to help get the
ball rolling. This post will concentrate on where the MITRE team is
with respect to the maturity of existing CWE definitions. Later posts
will cover Paul's suggestions from a process standpoint.

> We propose, first, that one element of a CWE be designated the prime
> definition.

Our approach for the past year has been to treat the CWE
Description_Summary element as an emerging "prime" definition. We use
the Extended_Summary to provide some additional details, informal
examples, etc.

For Description_Summary, our goal has been to create a focused
description that is as clear as possible, avoids vague terms, and
omits as much extraneous information as possible. We do this by
trying to identify the core error, and to avoid conflating these with
closely-related errors (e.g. chains) and attacks. Recently, we have
been avoiding identifying the typical consequence of the weakness,
which is informally covered in the Extended_Summary element (and often
detailed in the Common_Consequences element).

We also try to write Description_Summary as a single sentence, which
forces us to concentrate on the core issue as much as possible. One
emerging style that has worked for us is to describe the software's
erroneous behavior with respect to any associated resources that are
affected by that behavior. The reason *why* the behavior is erroneous
is moved to the Extended_Summary.

We also believe that there needs to be very close alignment between
the Name attribute and the Description_Summary.

Many of our description summaries have been inherited from previous
efforts in which the names or definitions were not so precise; we did
not help our own cause in earlier years when we wrote our own
descriptions, either.

Starting with Draft 9, we have made significant improvements to many
descriptions. Since Draft 9 was released, we have modified the
descriptions for 311 different entries, and we've modified the names
for 98 entries. If you read the difference reports
(http://cwe.mitre.org/data/reports.html) you can see that Name and
Description are often one of the most frequently updated elements for
each new version.

There has been some tension in how to capture the Description_Summary
effectively and clearly, without being too verbose or too difficult to
understand. As Paul has mentioned, CWE has multiple audiences. In
some cases, we can write a terse Description_Summary using
vulnerability theory terms, but the casual reader might not understand
most of the words in that description.

There are also significant challenges when trying to be more precise,
in that we often realize that we're not sure what the actual CWE entry
is really about! Consider CWE-377 (currently named "Insecure
Temporary File"). Its description is: "Creating and using insecure
temporary files can leave application and system data vulnerable to
attack." The immediate question that comes to mind is - what is
"insecure" about the temporary file? It has bad permissions? It's
created in a directory with bad permissions? Its name contains
sensitive information? It's subject to symlink following? It's not
deleted after use? It's accidentally packaged up with other files
before it's sent somewhere else? This vague name and description are
actually hiding several lower-level weaknesses - yet for certain
audiences, these distinctions are not important, and to adopt a more
precise definition would not help certain audiences.

The labor to come up with an acceptable Description_Summary for all
CWE entries, estimating 30 minutes each, would be approximately 8
staff weeks for weakness-focused entries. Some of this labor has been
spread over time, but there is still room for improvement. And note
that in many cases, it will take more than 30 minutes to get clarity -
for example, the buffer overflow examples given by Paul.

To take Paul's example of the current CWE-121 description:

A stack-based buffer overflow condition is a condition where the
buffer being overwritten is allocated on the stack (i.e., is a local
variable or, rarely, a parameter to a function).

Under current MITRE practices, ideally we would change this
Description_Summary, perhaps in the following fashion (note: this is
an informal example only):

- the phrase regarding local variables and function parameters would
be relocated elsewhere in CWE - probably to the
Extended_Description

- the "overwrite" phrase would be avoided or otherwise clarified

- a next-generation description might be:

The software allocates a buffer using memory that is stored on
the stack, but it writes to an address that is outside of the
implied boundaries of that buffer.

While such a description may still have some subtle problems, it is
closer to our current guidelines for CWE descriptions: it focuses
largely on behavior ("allocate"/"write") and resources
("buffer"/"stack"), while avoiding specific examples or variants
("local variable"), while also attempting to more clearly identify
concepts such as "overflow" and "overwrite" whose terms have many
audience-specific interpretations.

Note that CWE-121 has another larger problem that we frequently
wrestle with. This CWE's abstraction has a particular perspective
focused on the type of resource (i.e. "buffer on stack") that leads to
conceptual overlap with other resource-independent CWEs (like CWE-125
"out-of-bounds read"). This part of the tree, like many other parts
of CWE, still needs additional research that more clearly identifies
the various perspectives and "layers" that come from various CWE
audiences. The CWE content team is getting better at identifying when
perspective/layering problems are preventing a clear identification of
a weakness or set of related weaknesses, but fixing these problems is
sometimes difficult.

As Paul has noted, there are many other fields that are related to
other descriptive aspects of a weakness, such as relationship notes,
background details, terminology notes, and white box definitions. All
of these have specific roles. They have not been the highest priority
to us, but we clean them up or fill them out when we run across them
(especially when they are in Other_Notes, which historically was a
general-purpose field before we extended the schema for Draft 9 and
version 1.0).

> If there is inconsistency between the prime definition and a
> description, example, or definition in another element, the prime
> definition is considered to be correct.

This is roughly the approach that we have taken. However, in some
cases, we may believe that the prime definition is excessively vague
or inconsistent with the rest of the CWE entry. We will sometimes
deprecate an entry outright if, as a whole, the entry can be
interpreted to cover multiple distinct weaknesses (assuming the entry
is not a category or higher-level asbtraction). As a result of the
deprecation, we may split it into newer entries that are more clearly
written.

Also, when there is a mismatch between the Name and the Description,
we will strongly consider deprecating the entry, since it is clear to
me that many people map to CWE identifiers based only on the name
without reading the description.

A current example of this problem is CWE-217, which has a very general
name of "Failure to Protect Stored Data from Modification," which was
inherited from CLASP. But the rest of the original CLASP item - and
much of CWE-217 itself - is focused on a very low-level Java problem
(actually, if you look at the demonstrative examples, it's about a
couple distinct problems). CWE-217 will be deprecated in the next
version, with a couple replacements that are much more clear.

Another example of definitional confusion/vagueness is CWE-391. As
its maintenance notes currently say: "This entry needs significant
modification. It currently combines information from three different
taxonomies, but each taxonomy is talking about a slightly different
issue." However, for us to fully deal with CWE-391, we have been
forced into defining a general model for error handling, which is
incomplete because of perspective/layering problems (see CWE-754 and
CWE-755 for a start).

>A clearly defined vocabulary is needed. Likely there will be stock
>phrases, too. This vocabulary and the definitions themselves should
>be based on the work already done by many contributors to the CWE.

We expect to be updating the vulnerability theory document
(http://cwe.mitre.org/documents/vulnerability_theory/intro.html) in
the coming months, which handles some of the gaps.

In addition, we have begun maintaining a glossary of terms here:

http://cwe.mitre.org/documents/glossary/index.html

While these are not perfect, and they are often still evolving, these
reflect some of the ongoing vocabulary that we are trying to develop.

As Paul has noted, there is other work being done by other
contributors to CWE that can be leveraged or consulted. Since CWE
operates within multiple audiences, this will sometimes be a difficult
task. For example, the current ISO/IEC Project 22.24772 document
appears to use the term "encapsulation" in a different way than
McGraw/Chess/Tsipenyuk did, which was also different than how others
may define it. (Look for my "what is encapsulation?" post coming
sometime in 2009). When terms like this become overloaded, the
solution for one audience will not necessarily work for another
audience.

The main point of this post was to provide some context: what our
current approach to CWE descriptions has been, and some of the
challenges that will be faced if the community decides that it is
important to adopt a "prime definition."

Any and all feedback is welcome. Thanks again to Paul and Mike for
raising this question. I look forward to everyone's thoughts.

- Steve

A Proposal to Precisely Define Weaknesses

2009-05-01T11:51:02Z

A Proposal to Precisely Define Weaknesses

by Paul E. Black

1 May 2009

Although much work has been done, which has greatly improved the definitions of source code weaknesses, there are still inconsistencies, ambiguities, and errors. Better definitions are needed as one element to speed the work of detecting and preventing weaknesses.

We propose, first, that one element of a CWE be designated the prime definition. If there is inconsistency between the prime definition and a description, example, or definition in another element, the prime definition is considered to be correct. If there is an error, the prime definition is the first to be fixed. Other descriptions and definitions can later be brought into alignment. Second we propose a specific set of reviews and other steps as necessary and sufficient to consider a prime definition to be thoroughly reviewed.

This document has three sections: motivation for precisely and accurately defining weaknesses, comments on the current state of definitions, and the proposal itself.

If you disagree (or agree!) with the proposal or have suggestions, please respond. Feel free to share this.

SECTION 1. WHY CAREFULLY DEFINE WEAKNESSES?

An important prelude to preventing, mitigating, or detecting weaknesses in software and systems is to have clear, unambiguous, widely-accepted definitions of such weaknesses. Consider the following questions that might occur to someone learning about software weaknesses. What precisely is a buffer overflow? Is it the same as a heap overflow or an unbounded transfer? Is one just a refinement of another? If an integer overflow leads to memory violation, which weakness is it? Is it both or is there some other relation between them? Precise definitions could answer these questions and others.

Many people have done excellent work toward clearly defining types of weakness. (Hereafter we usually use the term "weakness" to mean a weakness class or type, not a particular instance of a weakness type.) Some papers and work are "Seven Pernicious Kingdoms: A Taxonomy of Software Security Errors" (Tsipenyuk, Chess, and McGraw), "The CLASP Application Security Process", (Viega), "The Preliminary List of Vulnerability Examples for Researchers (PLOVER)" (Christey), "The Ten Most Critical Web Application Security Vulnerabilities" (OWASP), "19 Deadly Sins of Software Security Programming Flaws and How to Fix Them" (Howard, LeBlanc, and Viega), "A Taxonomy of Computer Program Security Flaws, with Examples" (Landwehr, Bull, McDermott, and Choi), (see http://cwe.mitre.org/about/sources.html for more) "Structured CWE Descriptions" (Christey, Harris, Heinbockel) available at http://cwe.mitre.org/documents/structured_descriptions/, ISO/IEC Project 22.24772: Programming Language Vulnerabilities available at http://aitc.aitcnet.org/isai/ and many others. KDM Analytics has produced formal definitions of some 30 weaknesses.

Since 2006 MITRE has led the Common Weakness Enumeration (CWE) effort to collect, reconcile, organize, and define weaknesses. Not only has this work improved and unified definitions, it has also led to discovery of fundamental clarifying concepts, such as chains and composites of weaknesses ("Chains and Composites", Steve Christey, http://cwe.mitre.org/data/reports/chains_and_composites.html).

Precise definitions can lead to further better understanding of weaknesses, their causes, cures, and preventions.

There are still inconsistencies and ambiguities within CWEs. Consider that a CWE may have many descriptive elements. Some in CWE Version 1.0 are

Black_Box_Definition (possibly more than one)

White_Box_Definition (possibly more than one)

Description

Description_Summary

Extended_Description

Compound_Element

Name (Weakness)

Demonstrative_Example

Observed_Example

Note

Theoretical_Note

Could inconsistencies be eliminated by deleting all but one such element in each CWE? That is not practical. One definition cannot serve all people in all instances. Training and reference needs succinct prose definitions and examples. Automated tools need a definition written in machine-readable languages. When discussion gets specific, people need the nuances and details of what constitutes that weakness. Proving that a weakness is absent or is prevented by some approach needs a rigorous, mathematically formal definition.

Improving the state-of-practice of software development to reduce instances of weaknesses and the vulnerabilities they cause takes work from language designers, compiler writers, educators, assurance tool developers, auditors and developers of guidance, people who specify and contract software development, researchers, vulnerability trackers, software engineers, and many more. If people in these roles disagree about what constitutes a particular weakness, or even whether it is a weakness at all, communication is difficult at best. At worst they may work at cross purposes. Broadly accepted definitions should allow different groups to work together more effectively.

Unambiguous, complete definitions allows those in the field to understand precisely what different software assurance tools, services, technologies, or methods can detect, mitigate, or prevent.

Formal definitions may allow tools to automatically check for weaknesses, create wrappers to filter out attacks to exploit them, or even rewrite the code to eliminate them.

SECTION 2. DOES THE CURRENT METHOD NEED IMPROVEMENT?

Currently there isn't a documented process to validate weakness definitions. Descriptions, definitions, and examples in the CWE have been reviewed and improved by different entities and the quality and consistency have improved dramatically over the years. Yet, even casual examination shows that most, if not all, CWEs need further work.

Given the number of CWEs and their range of complexity, frequency, and severity, it is probably not worthwhile to precisely define every single weakness. But for those which need precise definitions, it is not clear exactly what should be. As with code, an essentially unlimited amount of checking and review could be done for each weakness. The community should agree on the types and amounts of validation that are necessary and sufficient.

To be specific, currently every CWE has a summary description. A CWE may also have alternate term descriptions, demonstrative examples, or a white box definition. Thousands of these descriptions have been reviewed and improved greatly. But which ones are merely informative and which ones are intended to be the carefully reviewed, precise and accurate? Consider CWE-121, Stack-based Buffer Overflow. (Text taken from CWE version 1.3). Here is the description summary:

"A stack-based buffer overflow condition is a condition where the

buffer being overwritten is allocated on the stack (i.e., is a

local variable or, rarely, a parameter to a function)."

Here is the White Box Definition:

"A buffer overflow where the buffer from the Buffer Write

Operation is statically allocated"

Is the following an instance of CWE-121? Note that alloca() gets memory from the stack, not the heap.

#define BUFSIZE 256

char *buf;

int main(int argc, char **argv) {

buf = (char *)alloca(BUFSIZE);

strcpy(buf, argv[1]);

}

The description summary uses "i.e." meaning "that is" or a restatement. A strict reading excludes this example because the buffer is only referenced by a global variable.

The white box definition says the buffer is statically allocated, that is, allocated automatically by the compiler or language support routine, not by functions invoked at run-time, such as alloca().

Again a strict reading excludes this example because the buffer is dynamically allocated.

Yet most people would agree that the example code has an instance of Stack-based Buffer Overflow and that the descriptions have minor inaccuracies. Even ignoring this particular example, we see there is inconsistency between the descriptions.

The following code snippets highlight not merely quibbles about wording, but raise the fundamental question of what CWE-121 means.

typedef struct

{

char buf1[10];

char buf2[10];

} my_struct;

my_struct s;

s.buf1[17] = 'A';

(from SRD test case 188)

The C99 standard [1] Sect. 6.7.2.1 Structure and union specifiers, page 102, para 5 says fields are "allocated in order". So buf2 makes the structure at least big enough that the access doesn't go outside the structure. Any particular compiler probably allocates the structure consistently, so the above code likely have a specific behavior in each environment, although it might differ from environment to environment. Is this an instance as CWE-121?

It is common in network programming to have structures and code like the following:

struct {

int header;

char payload[ 0 ];

} *p;

p = malloc(sizeof *p + 2);

p->payload[0] = 'A';

p->payload[1] = '\0';

(adapted from Aurelien Delaitre)

Nothing in the C standard justifies this code, but most compilers will treat it reasonably. Since the code does not strictly follow the C standard, is its behavior undefined so that this should not be considered an instance of CWE-121?

Compilers usually allocate structures in multiples of four bytes, so the following code should be fine. That is, there is memory for 12 characters in the structure. Should this example be considered to write outside of the buffer or not? On what grounds?

typedef struct

{

int int_field;

char buf[10];

} my_struct;

my_struct s;

s.buf[10] = 'A';

(from SRD test case 201)

We see that precise definitions are important to make consistent judgments. We also see that it is very difficult to write a definition which is precise and accurate.

Given all the work which the CWE embodies, how can the effect of inconsistencies be minimized? Where should work be concentrated to achieve good definitions? All CWEs have a status of draft or incomplete. How much and what kind of work should be done before a definition is considered to be "thoroughly reviewed", that is, good enough for the community to move on to another one? What language, either formal (mathematically precise) or prose, should be used to state definitions clearly and so the style is similar across CWEs?

What should be done to minimize assumptions or artifacts arising from the use of a particular formal language or style?

SECTION 3. THE PROPOSAL

This proposal is part of a vision to improve software assurance by having widely-used clear definitions of software weakness classes.

The goals of this proposal are to

* increase precision (minimize ambiguity and inconsistency) in CWE

entries and

* increase accuracy (minimize mistakes indicated by broad agreement).

The proposal has two points: (1) the need for an prime definition and

(2) a review process. Each point has several parts or supporting clauses which are somewhat independent.

POINT ONE: The community should have one prime definition of each weakness.

PROPOSAL CLAUSE 1: the Common Weakness Enumeration (CWE) is the repository of the prime definition.

PROPOSAL CLAUSE 2: each weakness has exactly one prime definition.

All other notes, summaries, descriptions, categories, components, examples, definitions, etc. are subordinate to it. For instance, if a summary seems to contradict the prime definition, the prime definition is assumed to be correct and the summary should be reinterpreted or changed.

There may be times when the prime definition is just wrong and needs to be corrected (see review process below), but short of that, the prime definition is authoritative.

The prime definition is a complete description of the weakness. All details and nuances are in that element. One need not read any other element to understand what is or is not an instance, although other elements may be helpful as examples or restatements.

It is important for the community to choose a prime definition so work can be focused on it. Having prime definitions lets us decide that a CWE is precisely and accurately described. Other summaries, descriptions, views, names, etc. can be brought into agreement as time, resources, and need arise.

The prime description will likely be legalistic and dry, like most formal descriptions. One would read something else to initially get a sense of what it is or to be reminded of it. Summaries, relationships, and notes serve such purposes. Rather the prime definition answers questions when differences of opinion or interpretation arise.

A clearly defined vocabulary is needed. Likely there will be stock phrases, too. This vocabulary and the definitions themselves should be based on the work already done by many contributors to the CWE.

One example is "Structured CWE Descriptions" (Christey, Harris, Heinbockel), available at http://cwe.mitre.org/documents/structured_descriptions/ Another is the work by KDM Analytics. Starting from scratch would be wasteful.

DISCUSSION POINT: if elements refer to completely orthogonal ideas, there could be more than one prime element and still has no chance of (internal) contradiction.

We don't think this occurs in CWE.

DISCUSSION POINT: "prime definition" is only one possible name. Other possibilities are master, attested, decisive, prevailing, key, normative, authoritative, official, sanctioned, or commonly accepted element or definition.

PROPOSAL CLAUSE 3: different kinds of weaknesses are best expressed with different prime elements.

Most weaknesses are manifest in code, like hard-coded password

(CWE-259) or leftover debug code (CWE-489). (Note that these two are not 100% discernible from code alone.) However some weaknesses are "black box", behavioral, functional, or external weaknesses, like denial of service (CWE-730), configuration (CWE-16), or violation of secure design principles (CWE-657).

PROPOSAL CLAUSE 4: each kind of weakness (see CLAUSE 3) has a prime element.

For instance, the prime definition for all weaknesses manifest in code might be White_Box_Definition.

DISCUSSION POINT: what is the right element for each kind of weakness? For weaknesses manifest in code White_Box_Definition seem most appropriate. Application behavior could be described in Black_Box_Definition.

POINT TWO: How should the Community Decide a Definition is Right?

An implementation may be checked for correctness against its specification. But how can a specification be checked? The short answer is that it can't be fully checked. However, we can take steps to validate a definition and minimize mistakes.

PROPOSAL CLAUSE 5: a CWE prime definition is acceptable when the following are done:

(A) the definition is published on the CWE discussion list, and

there is no (or only limited) disagreement that it is precise

(unambiguous) and accurate (correct).

(B) the definition agrees with SC22/WG23 Programming Language

Vulnerabilities if there is a correspondence.

to be precise and accurate.

(D) the definition is written in two different "executable" forms

which are tested and deemed to find/generate/mitigate the

weakness (and nothing else).

We think these steps are the minimum needed to come up with good definitions. We need the general consensus of the community to make sure it is broadly agreeable. We want definitions which are consistent with other standards rather than creating yet another

(inconsistent) "standard". We need experts to make sure the definition says the right thing. Finally, we need executable forms to make sure the definition is precise.

Since CWEs are still being developed and the community is self-selected, We don't think unanimous agreement is needed.

Nevertheless, any objections should be taken very seriously. The desired goal is unanimity.

The community of concern is large and there are other efforts. As much as possible the definitions should be in harmony. One particular effort is PDTR 24772 a draft of which is document N0138 at http://aitc.aitcnet.org/isai/ Here are some correspondences between CWEs and entries in PDTR 24772:

6.15 Numeric Conversion Errors [FLC]

CWE 192

6.17 Boundary Beginning Violation [XYX]

CWE 123

CWE 129

6.18 Unchecked Array Indexing [XYZ]

CWE 129

6.19 Buffer Overflow in Stack [XYW]

CWE 121

We need reviews from at least two very different experts to minimize the chance that the definition only reflects the view of one person or one group. Why not three or more? The only objective reason we offer is that economy suggests the fewest possible, and two is the least number greater than one.

Who are the experts? We don't have specific people in mind. It seems it should be people who have a lot of experience in the field. It also seems like it should be people who have thought about such definitions, so its likely to be authors, security researchers, and tool makers. Maybe we should designate a pool of experts and any two of them are acceptable. (And no objections from "experts"?) Expert availability will probably be a factor.

The definition should be "formalized" in at least two completely different ways to minimize the chance of unstated assumptions or severe bias to one form of expression. Some ways may be a language for generating test cases, rules for a source code scanner to find the weakness, or a purely formal description in a highly mathematical notation for proving program properties.

The executable forms and the expert reviews should be completely independent. That is, a expert writing an "executable" form only counts as an expert review (C above) or an executable (D above), but not both. Also the two experts should not be from the same organization.

It was suggested that one more step be added: that the definition be validated to accurately describe some subset of reported instances of the weakness, like the CVE. The aim is to make sure the definition corresponds to real examples.

This proposal does not address how the process would be executed, in particular where the resources would come from. Nevertheless, We think it is important to begin by getting community agreement on these two points.

-paul-

Re: More frequent CWE releases and CWE versioning

2008-10-14T09:10:01Z

On Tue, 14 Oct 2008, Pascal Meunier wrote:

> I'm sorry, that was sloppy thinking on my part. I was grouping schemas
> and views together, whereas when you say "schema" you clearly have only
> something like the XML schema definition in mind. I thought it would be
> interesting to separate changes in relationships defined in views from
> changes in the CWE entries themselves, and to keep track of each
> category separately.

An interesting point - if you only care mostly about one view, then you'd
want to know what's changed within that view. I've been considering
view-specific diff reports that highlight which relationships were added
or removed; these could be part of the larger diff. Within the XML
itself, the individual view node could potentially record its own
"sub-version" that changes whenever the view changes (alternately, the
modification record in the content history could be updated).

Would people find it useful to be able to track when a view's
relationships have changed?

- Steve

Re: More frequent CWE releases and CWE versioning

2008-10-14T09:03:00Z

Steven M. Christey wrote, On 10/13/2008 09:28 PM:

(...)

> When you talk about views changing, do you mean the XML downloads of
> specific views? Or when we change relationships within views? I view the
> latter as a content change. The former is basically just part of our web
> downloads that are automatically generated from the core XML.
>

I'm sorry, that was sloppy thinking on my part. I was grouping schemas
and views together, whereas when you say "schema" you clearly have only
something like the XML schema definition in mind. I thought it would be
interesting to separate changes in relationships defined in views from
changes in the CWE entries themselves, and to keep track of each
category separately. The noNamespaceSchemaLocation attribute doesn't
allow for that.

Regards,
Pascal

Re: More frequent CWE releases and CWE versioning

2008-10-13T18:28:48Z

On Sun, 12 Oct 2008, Pascal Meunier wrote:

> IMO ideally you need two version numbers. You need a schema/view
> version and a content version.

The raw XML already references the schema in the noNamespaceSchemaLocation
attribute, and we hope that the schema won't change so often. So, I've
been viewing this as sufficient information. However, we expect to force
a major content version change if the schema also changes significantly.

That said, we expect to be making small modifications to the schema in the
next month or so - basically, we want to add a couple new values to the
Introductory_Phase element. We expect we'll have similar small schema
changes as we add new programming languages to the Language element under
Applicable_Platforms.

> In option 1, a change in a view can be
> confused with a major change in individual CWE entries.

We will need to be more precise about what should constitute a "major"
change versus a "minor" change. View maintenance is largely a matter of
modifying relationships between CWE entries (and/or editing the view node
itself).

> This is how it could work in practice:
>
> -The XML download could be named cwec_s1.0_c1.0.xml

The schema is currently at 4.0, so using your scheme, the download would
be named:

cwec_s4.0_c1.0.xml

since the schema is currently at 4.0:

http://cwedev1.mitre.org/data/xsd/cwe_schema_v4.0.xsd

If we go this route, I'd advocate listing the content version first, then
the schema version, since I'd expect that the content version would be the
most important to people.

So if we changed the version to 1.0.1, we'd have:

cwec_c1.0.1_s4.0.xml

Either way, that seems like a lot of dots and numbers, so its not
aesthetically pleasing to me.

> -I suspect it would be better (less ambiguous) if the content version
> would never reset when you modify the schema or views, so if you had
> schema/views 1.0 and content 1.6, if you adopt schema/views 2.0 then the
> content version would be at least 1.6, possibly 1.7 or higher depending
> on the extent of changes.

I would expect this to be the case. The schema and content versions are
closely related, but they are clearly separable in my mind.

When you talk about views changing, do you mean the XML downloads of
specific views? Or when we change relationships within views? I view the
latter as a content change. The former is basically just part of our web
downloads that are automatically generated from the core XML.

> Files with only the differences or a list of updated CWEs between
> content versions would be a bonus.

The diff's will list the updated CWEs. I don't think we can produce files
with only the differences at this stage, but we'll definitely consider it
for the future.

- Steve

More frequent CWE releases and CWE versioning

2008-10-10T11:04:09Z

All,

We are planning to release a new version of CWE on Tuesday, October 14.
But before we do so, we'd like to get your opinion on how we should label
things when CWE data changes.

This new version does not have the large number of changes that we saw
occurring between drafts. The schema remains the same. We've updated
a few dozen CWE entries and added a new entry, but otherwise the
content is the same.

For the foreseeable future, we are planning a more frequent update
schedule - currently estimated at once a month, although the frequency may
vary depending on "quality-based" criteria or external deadlines.

So, we now face a question of the best way to capture these ongoing
versions.

The CWE schema supports two separate elements that could be used for
versioning:

Catalog_Version - a string

Catalog_Date - a date

In the past, we've only used the Catalog_Version - so currently, it's
at "1.0."

We believe there are two main choices for how we can handle versions
in the future. We wanted to get feedback from the researcher list
because you are some of our most active users of CWE.

The options are:

1) Versions for each change

Use Catalog_Version to contain sub-versions for every new release
of CWE, no matter how small the change; and use larger versions to
reflect significant content changes, such as major changes to an
important view.

The Catalog_Date would be filled out, but only as a convenience for
users, so that they don't have to go to the CWE web site to figure
out what date CWE was released on.

So, the new release would be labeled "1.0.1"

The XML download would be cwec_v1.0.1.xml

2) Versions for significant changes

Catalog_Version would not be modified if only small changes
occurred, as will be the case for October 14. We would only modify
the version number when significant content changes occur, as
mentioned previously.

The Catalog_Date would be changed whenever content changed within
the same version, so that people who care about the small
differences can check to see if their copy of CWE needs updating.

So, the new release would remain labeled "1.0", and the
Catalog_Date would be "2008-10-14". The XML download would remain
at cwec_v1.0.xml.

By creating new versions for each change, including minor changes, this
would ensure that there is no confusion between copies of CWE; any two
parties who say they use "CWE 1.0.x" would be using the exact same data.
However, casual CWE users would not understand the rationale for why the
versions are updated, so they might update their data more frequently than
necessary.

By limiting new versions to significant changes only, then any version
change will send a strong signal that it's important for people to update
their copy of CWE. People who care about minor versions can still use the
date field if they want.

Note that in either case, we will be providing version-to-version diffs on
the web site.

What would work best for you? Do you expect to follow every single update
of CWE, or only the most significant changes?

Thanks,
Steve

1.0 confusion

2008-09-12T08:55:56Z

Posting this to the list for further discussion.

-----Original Message-----
From: Steven M. Christey [mailto:coley@...]
Sent: Friday, September 12, 2008 11:52 AM
To: Chris Eng
Cc: Steven M. Christey; Robert A. Martin
Subject: Re: 1.0 confusion

Chris,

Would you mind posting this question to the research list? Others will
probably run into similar questions.

> 1) Why does each Relationship node have a View_ID and a
> Relationship_Target?

A Relationship only exists within the context of a particular view.
This
technically begin in Draft 9 but wasn't necessarily that noticeable
until
1.0.

> If I am trying to figure out who is the most applicable parent for a
> given node, how do I prioritize if there are multiple ChildOf
> Relationships?

1) You pick a view, X, through which you plan to navigate, then you pick
ChildOf in that view.

2) If there are multiple ChildOf in view X (i.e. where
Relationship_View_ID=X), then you pick the relationship with
ordinal=primary for that view. There's only one primary allowed per
(relationship:view) tuple in each node. If there's multiple
ordinals=primary, then each one is in a separate view.

The primary navigational views in CWE 1.0 are View 1000 (Research
Concepts) and view 699 (Development Concepts; the main CWE organization
before 1.0). In general, deferring to view 1000 instead of 699 is more
likely to get you closely-related entries - however, that's based on
view-1000's perspective of the world.

Note that we only have relationships in a single direction in the XML.
We plan on publishing XML with bidirectional relationships, which should
minimize programming effort in some cases.

[Note to Bob and self: we should probably have a way of more cleanly
capturing only primary relationships for people who don't want to do
multi-parent navigation].

> 2) I am trying to figure out how to take the OWASP views and generate
a
> list of all the CWE nodes that belong to it, which we will then use
for
> our reporting. I thought I could start with all the nodes that are
> listed explicitly as part of CWE 723, for example, and then add all
the
> descendants of those nodes. But when I do that, I end up missing some
> nodes, such as CWE 73 which should be part of OWASP A2-2004.

A very interesting problem that I don't have a fully-automated answer
for.

In general, you could use something we informally call "view-hopping,"
although there can be limitations.

Let's say you're following view 711, the 2004 OWASP Top Ten. Once you
run
out of "711" child relationships to follow, you could then "hop" over to
view-1000 (natural hierarchy) relationships.

So navigating under view-711 from cwe-723, you'll reach a child CWE-330
(randomness). This has no cildren under view-711, but if you "hop" to
relationships under view 1000, CWE-329 Not Using a Random IV with CBC
Mode
and many other variants starts to show up.

View-hopping probably makes the most sense under view-1000, since it is
focused on closely related weaknesses instead of arbitrary categories.

But there are limitations here, and the example you used, CWE-73, is one
of them. You won't find it under view A2 (as you saw), and it's not a
descendant under view-1000 either, because the Research Concept view
doesn't treat CWE-73 as an instance of input validation. However, you
do
reach CWE-20 through the 699 Development Concepts view.

711 --> 722 (A1 - Unvalidated Input) [VIEW 711]

722 has child 20 (Insufficient Input Validation) [VIEW 711]

20 has many children under VIEW 1000... but not 73

20 also has children under VIEW 699... including 73

But since view 1000 and view 699 are organized so differently, I can see
how you'd wind up in trouble if you just navigated both of them
automatically.

If you're going the opposite direction, maybe you could try something
like:

73

view 1000: ChildOf CWE-610 (which is ChildOf 664, no OWASP cat's
in the path)

view 699: ChildOf CWE-20 [under 699]
CWE-20 ChildOf CWE-722 [under 711]
CWE-722 is A1-2004

So 73 is part of A1-2004.

I don't think there's a fully automated solution here, but if you
navigate
up view-1000 relationships *and* view-699 relationships, you're likely
to
find what you want.

> 3) You have CWE 79 (XSS) listed under OWASP A1-2004. But XSS is
really
> an Output Validation problem, not Input Validation, plus it has its
own
> OWASP category dedicated to it, OWASP A4-2004.

This is because when I created 711, I made a literal mapping from what
the
OWASP document said to what CWEs were available. I agree that XSS is
more
related to output than input (we say "sanitization" instead of
"validation" because things like encoding/escaping don't feel like
"validation"). However, the OWASP 2004 document specifically lists XSS
under its A1 category.

The ordinal=primary for XSS, though, is for the A4 XSS relationship.

Hope this all makes sense.

- Steve

Release of CWE 1.0

2008-09-09T21:49:35Z

All,

CWE 1.0 has been released!

We've added a lot of pages to the site. Many new pages have undergone
a redesign; you might need to refresh your browser cache to see the
new presentation.

The most efficient way for you to access what you want is probably
through the links provided in this email, much of which is also
reflected in this page:

http://cwe.mitre.org/data/reports/major_chgs_1.0.html

Major changes have been made to the CWE schema, which will be much
more stable than previous versions. It is expected that the schema
will not change in any substantial fashion for the near future. The
new schema addresses the main outstanding limitations of past
versions, provides internal consistency, fixes outstanding
limitations, and supports ease of content editing by the CWE team. We
thank Sean Barnum of Cigital for his active contributions in this
area.

Schema change summary:

http://cwe.mitre.org/data/reports/diff_xsd_3.0_4.0.html

Schema documentation:

http://cwe.mitre.org/documents/schema/index.html

Engagement with key stakeholders in the community has led to
additional content enhancements in CWE 1.0. Many entries contain
modifications that were contributed by external parties.

* Cigital provided additional demonstrative examples, mitigations,
and times of introduction.

* KDM Analytics provided additional white box definitions.

* Veracode suggested the creation of an OWASP Top Ten 2004 view
(CWE-711) because of its use in PCI, and they provided supporting
CWE mappings.

Links: everywhere. Look at the Modification credits in the Content
History sections of individual entries.

Engagement with members of the community has also resulted in
significant enhancements to the Development Concepts (CWE-699) and
Research Concepts (CWE-1000) views, which are the most heavily
featured on the CWE web site. We have also created a Seven Pernicious
Kingdoms view (CWE-700). A comparison of these views is available, as
well as a description of how they evolved. We are especially grateful
for feedback from representatives from Cigital, Fortify, and Veracode.

Development Concepts view (check out the graph tab):

http://cwe.mitre.org/data/definitions/699.html

Research Concepts view (check out the graph tab):

http://cwe.mitre.org/data/definitions/1000.html

Evolution of the views:

http://cwe.mitre.org/documents/views/view-evolution.html

Comparison:

http://cwe.mitre.org/documents/views/view-comparison.html

List of all views:

http://cwe.mitre.org/data/index.html

In addition to 39 new entries, all 695 entries from CWE Draft 9 have
been modified in some fashion, mostly from external contributions and
from relationship changes in support of various views.

Detailed change report:

content: http://cwe.mitre.org/data/reports/diff_draft_9_v1.0.html

schema: http://cwe.mitre.org/data/reports/diff_xsd_3.0_4.0.html

There are additional documents that have been published, including:

(1) an analysis of CWE's ability to support tool mappings, of
interest to tool vendors, academic researchers, and tool
analysts:

http://cwe.mitre.org/documents/mapping_analysis/index.html

(2) PDF graphical depictions of various CWE views, including
"coverage graphs" that show how members of one view are located
within another view:

http://cwe.mitre.org/data/pdfs.html

(3) an evolving glossary of terms:

http://cwe.mitre.org/documents/glossary/index.html

Of course, the work doesn't end here, but we believe that CWE 1.0 is a
significant improvement to the past drafts of CWE. It would not be
possible without hard work from the community and the CWE team. Bob
Martin and Steve Christey would like to thank CWE team members Janis
Kenderdine, Conor Harris, and Mark Loveless for all their efforts in
bringing CWE to a new level of maturity.

As always, feedback is welcome here on the list or to cwe@....

Enjoy!

Steve Christey, CWE Technical Lead
Bob Martin, CWE Project Lead

RE: CWE 1.0 to be released Tuesday, September 9, 2008

2008-09-09T21:48:57Z

On Tue, 9 Sep 2008, paulslewis66 wrote:

> May I ask if CWE references will be included within the XML CVE feeds in
> a similar way the searchable database is?

We do not track CWE references within CVE. You're probably talking about
NIST's NVD (nvd.nist.gov), which is an extension of CVE. NVD has been
mapping to CWE on individual pages, but it is not yet included in their
downloads. However, NVD has stated that they will start including CWE
names in the downloads within a matter of weeks.

For those who were not aware of NVD's use of CWE, see:

http://nvd.nist.gov/cwe.cfm

The current selection of CWE identifiers used in NVD faces many of the
same issues that have been brought up by Information-technology Promotion
Agency, Japan (IPA), specifically that sometimes you can't assign CWE
identifiers when you are dealing with incomplete third-party vulnerability
information. We are aware of this limitation and hope to address it in
the coming months.

- Steve

RE: CWE 1.0 to be released Tuesday, September 9, 2008

2008-09-09T00:38:18Z

Hello,

May I ask if CWE references will be included within the XML CVE feeds in a similar way the searchable database is?

many thanks

Paul S Lewis

Re: Some asking about CWE.

2008-09-07T20:35:58Z

Dear Mr. Steven M. Christey,

I am very appreciate all your help.
I am looking forward to the CWE Version 1.0 release.

We(IPA) think that we will adopt CWE.
We would like to participate in the following CWE community initiative.
http://cwe.mitre.org/community/index.html

Could you add our organization to the CWE community initiative ?
Information-technology Promotion Agency, Japan (IPA)
http://www.ipa.go.jp/index-e.html

Thank you again for your explanation about questions.

Sincerely yours,
Tadashi Yamagishi
IT Security Center (ISEC)
Information-technology Promotion Agency, Japan (IPA)
E-mail: t-yamagi@...

Steven M. Christey wrote:

> On Mon, 25 Aug 2008, Tadashi Yamagishi wrote:
>
>> Question1:About Hierarchy diagram.
>> I made CWE-635(Weaknesses Used by NVD) a hierarchy diagram
>> referring to cwe_classification_tree.pdf.
>> The hierarchy diagram is appended.
>> cwe_classification_tree.pdf shows the following.
>> CWE-20 is a child of CWE-19.
>> CWE-22 is a child of CWE-21.
>> CWE-134 is a child of CWE-133.
>> However, CWE-1000(Natural Hierarchy) shows another parents.
>> I am confused. Are two or more parents permitted in CWE ?
>
> Yes. This is for two reasons:
>
> (1) there are multiple ways of looking at the same weakness, so we want
> to support these different ways. So, different views can express
> different relationships.
>
> (2) Some weaknesses can be fairly complex, so they can be classified in
> different ways, even within the same view. Ideally, it would be
> good to have a view in which every weakness can have only one
> parent. This is very difficult to achieve in practice; we think
> that the concept of chains and composites helps to explain why
> this classification is so difficult. We are making significant
> progress within the "natural hierarchy" (view 1000), but we
> will not have this finished by the release of CWE 1.0.
>
>> Question2:About the classification of Dos( Denial of Service ).
>> DoS is not classified in CWE.
>
> DoS is not classified because it's a "consequence" of some weakness - just
> like "loss of integrity" is a consequence. In CWE 1.0, we will have
> multiple ways of trying to determine which weaknesses can lead to a DoS:
>
> - a new OWASP Top Ten 2004 view has category A9, Denial of Service.
>
> - as a result of schema changes in 1.0, our Consequence element has
> been improved so that you will be able to search CWE for
> Consequence_Scope = Availability.
>
>
>> How do you classify it when the cause of the DoS is not understood
>> in the vulnerability report?
>
> This is a very difficult question, and I'm not sure how to handle it. In
> the case of databases of public vulnerabilities (like NVD), the databases
> often don't have solid information about the underlying weaknesses that
> led to the vulnerability. Sometimes, the only vulnerability information
> is something like "Product X can crash from a malformed packet" - you
> might see this in a software vendor advisory, for example.
>
> Since the "DoS" phrase alone doesn't talk about any specific weakness, CWE
> is not currently capable of modeling it.
>
> This is an example of a challenge: how should you map to CWE when you're
> dealing with issues that aren't exactly weaknesses? We hope to be able to
> develop methods of handling these kinds of issues. In fact, one of the
> upcoming white papers will cover some of the common difficulties that
> people will face when mapping to CWE. In addition, sometime after the
> release of CWE 1.0, we will try to improve the current NVD classifications
> so that they are more suitable for handling incomplete vulnerability
> information. Hopefully we will be able to address this issue somehow.
>
>
> - Steve
>

Re: Some asking about CWE.

2008-09-05T13:46:23Z

On Mon, 25 Aug 2008, Tadashi Yamagishi wrote:

> Question1:About Hierarchy diagram.
> I made CWE-635(Weaknesses Used by NVD) a hierarchy diagram
> referring to cwe_classification_tree.pdf.
> The hierarchy diagram is appended.
> cwe_classification_tree.pdf shows the following.
> CWE-20 is a child of CWE-19.
> CWE-22 is a child of CWE-21.
> CWE-134 is a child of CWE-133.
> However, CWE-1000(Natural Hierarchy) shows another parents.
> I am confused. Are two or more parents permitted in CWE ?

Yes. This is for two reasons:

(1) there are multiple ways of looking at the same weakness, so we want
to support these different ways. So, different views can express
different relationships.

(2) Some weaknesses can be fairly complex, so they can be classified in
different ways, even within the same view. Ideally, it would be
good to have a view in which every weakness can have only one
parent. This is very difficult to achieve in practice; we think
that the concept of chains and composites helps to explain why
this classification is so difficult. We are making significant
progress within the "natural hierarchy" (view 1000), but we
will not have this finished by the release of CWE 1.0.

> Question2:About the classification of Dos( Denial of Service ).
> DoS is not classified in CWE.

DoS is not classified because it's a "consequence" of some weakness - just
like "loss of integrity" is a consequence. In CWE 1.0, we will have
multiple ways of trying to determine which weaknesses can lead to a DoS:

- a new OWASP Top Ten 2004 view has category A9, Denial of Service.

- as a result of schema changes in 1.0, our Consequence element has
been improved so that you will be able to search CWE for
Consequence_Scope = Availability.

> How do you classify it when the cause of the DoS is not understood
> in the vulnerability report?

This is a very difficult question, and I'm not sure how to handle it. In
the case of databases of public vulnerabilities (like NVD), the databases
often don't have solid information about the underlying weaknesses that
led to the vulnerability. Sometimes, the only vulnerability information
is something like "Product X can crash from a malformed packet" - you
might see this in a software vendor advisory, for example.

Since the "DoS" phrase alone doesn't talk about any specific weakness, CWE
is not currently capable of modeling it.

This is an example of a challenge: how should you map to CWE when you're
dealing with issues that aren't exactly weaknesses? We hope to be able to
develop methods of handling these kinds of issues. In fact, one of the
upcoming white papers will cover some of the common difficulties that
people will face when mapping to CWE. In addition, sometime after the
release of CWE 1.0, we will try to improve the current NVD classifications
so that they are more suitable for handling incomplete vulnerability
information. Hopefully we will be able to address this issue somehow.

- Steve

CWE 1.0 to be released Tuesday, September 9, 2008

2008-09-05T13:29:09Z

All,

We will be releasing CWE 1.0 on Tuesday, September 9.

While we had a goal for everything to be finished by the end of August, we
want CWE 1.0 to be as stable as possible, especially with respect to the
schema. Ironing out the issues with the schema has taken more time than
we wanted, plus we have added a number of new elements.

During our interactions with various community members over the summer,
we've realized that it would be best for us to write a number of white
papers, as well as creating some new views. These were somewhat
unexpected additions that came in July and August, so this introduced more
work than we had originally expected.

We didn't want to release CWE 1.0 too early, only to make some more
changes soon after we released it because it was incomplete. So we've
slipped in our schedule, but the quality will be higher.

This is our biggest release to date, and we believe that it will be worth
the wait. Thank you for your patience.

- Steve

Some asking about CWE.

2008-08-25T02:52:26Z

Dear CWE group,

I am Tadashi Yamagishi
in Information-technology Promotion Agency, Japan (IPA).
We(IPA) have Vulnerability Countermeasure
Information Database (JVN iPedia) for Japanese IT user.
http://jvndb.jvn.jp/index_en.html

JVN iPedia adapted CVSS(Common Vulnerability Scoring System) last year.
The next step, I think that JVN iPedia need CWE.
I am studying CWE draft 9 now.
I have three questions about CWE.

Question1:About Hierarchy diagram.
I made CWE-635(Weaknesses Used by NVD) a hierarchy diagram
referring to cwe_classification_tree.pdf.
The hierarchy diagram is appended.
cwe_classification_tree.pdf shows the following.
CWE-20 is a child of CWE-19.
CWE-22 is a child of CWE-21.
CWE-134 is a child of CWE-133.
However, CWE-1000(Natural Hierarchy) shows another parents.
I am confused. Are two or more parents permitted in CWE ?
I think that cwe_classification_tree.pdf and CWE-1000
are comprehensible when it is the same.

Question2:About the classification of Dos( Denial of Service ).
DoS is not classified in CWE.
How do you classify it when the cause of the DoS is not understood
in the vulnerability report?

Question3:About XSS vulnerabilities.
There are lots of XSS vulnerabilities
by the UTF-7 encoded string problems in Japan.
for example:
CVE - CVE-2008-1468
http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-1468
JVNDB-2008-000018 - JVN iPedia
http://jvndb.jvn.jp/contents/en/2008/JVNDB-2008-000018.html

CVE - CVE-2008-2168
http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-2168

CVE - CVE-2008-0005
http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-0005

I want to classify the detail of XSS.
Can I choose a CWE-ID more detail than CWE-79
about XSS(UTF-7 encoded string problems)?

I look forward to your reply.

Sincerely yours,
Tadashi Yamagishi
IT Security Center (ISEC)
Information-technology Promotion Agency, Japan (IPA)
E-mail: t-yamagi@...

CWE_635.PNG (17K) Download Attachment

FW: Natural Hierarchy email

2008-07-30T07:32:42Z

Hi All,

In earlier CWE drafts, the CWE hierarchy was mostly based on weakness
abstractions - that is parent / child relationships were focused on the
abstraction of the core weakness being addressed by each. But
weaknesses in some areas of the hierarchy were grouped together based
on a common attribute such as language, resource or consequence, the
relevance of which can change based on the context of the weakness
occurrence or the perspective of the viewer. As a result, weaknesses
were sometimes children of categories that had little to do with the
underlying problem, but provided a convenient way in which to view
issues from certain perspectives. While these perspective and context
based views into CWE are very useful for some, a view based solely on
weaknesses and their abstractions is also needed.

In an effort to provide this additional perspective into CWE, the CWE
team began adding additional relationships to some entries in the
release of draft 9. These relationships are based on the fundamental
weakness behind each entry and how it relates to other weaknesses in
terms of abstraction and small variations on similar issues. We are
attempting to identify relationships that are as independent of
specific language, technology, programming idiom, and resource as
possible.

The slight shift in focus between drafts 7 and 8 led to several new
schema constructs being added for draft 9, one of which being the Class
/ Base / Variant attribute added to the weaknesses structure - a
derivation of the draft 8 Grouping attribute - in order to facilitate
the abstraction-based relationships. The CWE team has expanded on this
for the release of version 1.0, and can be demonstrated in part by the
top level of the natural hierarchy, view-1000, and a small subset of
their children shown below.

* Range Errors
o Unbounded Transfer ('Classic Buffer Overflow')
o Incorrect Offset into a File
* Equivalence [Draft Concept for 1.0]
o Insufficient Comparison
o Failure to Resolve Case Sensitivity
o Encoding Error
* Incorrect Calculation
o Off-by-one Error
o Divide by Zero
* Failure to Fulfill API Contract (aka 'API Abuse')
o Failure to Follow Specification
o Failure to Provide Specified Functionality
* Coding Rule Violation
o Violation of Secure Design Principles
o Use of Inherently Dangerous Function
o Use of Potentially Dangerous Function
* Security Features (Protection Mechanism Failure) [Merging is new for
1.0]
o Incomplete Blacklist
o Weak Encryption
o Insufficient Authentication
* Use of Insufficiently Random Values
o Insufficient Entropy
o Small Space of Random Values
* Interaction Error
o Compiler Removal of Code to Clear Buffers
o Interpretation Conflict
o Reliance on Data Memory Layout
o Behavioral Change in New Version or Environment
* Insufficient Control of Resource Through its Lifetime
o Improper Resource Shutdown or Release
o Incorrect or Incomplete Initialization
o External Influence of Sphere Definition
o Incorrect Resource Transfer Between Spheres
o Operation on Resource in Wrong Phase of Lifetime
* Insufficient Control Flow Management
o Insufficient Synchronization
o Always-Incorrect Control Flow Implementation
o Incorrect Behavior Order
o Deployment of Wrong Handler
* Failure to Handle Exceptional Conditions
o Missing Error Handling Mechanism
o Unexpected Status Code or Return Value
* Failure to Maintain Cohesion in Structured Messages or Data [New
Concept for 1.0]
o Failure to Sanitize Special Elements
o Deletion of Data Structure Sentinel
o Improper Null Termination

It is important to emphasize that CWE will still be navigable by common
attributes, such as language, consequence and platform after the
release of CWE version 1.0 through various "views". Prior relationships
were preserved in the development of the natural hierarchy and can
still be viewed from the individual CWE Definition pages or through an
alternate view where the relationships carry more meaning. For example,
CWE will be supporting a Programming Concepts view as well as a 7
Pernicious Kingdoms view starting with version 1.0.

The 7 Pernicious Kingdom view will be based on the original 7PK paper
and is focused primarily on usefulness to developers and grouping
weaknesses by categories. The Programming Concepts view will be an
expansion of sorts from the 7PK view which will incorporate the
additional content that has been added to CWE. These views will
accommodate navigation similar to previous drafts of CWE using
categories such as pointer issues, mobile code issues, error handling,
data handling, cleansing, comparison and canonicalization, time and
state, temporary file issues, and channel and path errors. Different
views allow for the organization of the content in the manner that
makes the most sense for the intended audience of each view.

The natural hierarchy as described above is the result of a
reorganization of the draft 8 hierarchy. We focused more on
abstractions of the core issues behind each weakness to try to unify
the perspective throughout the natural hierarchy. This entails moving
away from relationships based on weakness context, environment, or
technologies. In draft 8 of CWE, for example, CWE-5 Misconfiguration:
Data Transmission without Encryption is a child of CWE-4 J2EE
Environment Issues. While this is a useful relationship, especially for
a J2EE developer, CWE-4 is a technology specific grouping of CWE-5 and
there may or may not be any relationship between CWE-5 and its siblings
other than a common technology. In version 1.0 of CWE, an additional
"ChildOf" relationship was added to CWE-5 relating it to CWE-319
Plaintext Transmission of Sensitive Data because CWE-5 is a technology
specific variant of the more abstract weakness, CWE-319. It is
important to note that CWE-5 is still a "ChildOf" CWE-4. This
relationship is maintained in a different "View", another schema
construct added for draft 9.

To further illustrate these changes, CWE-444 HTTP Request Smuggling was
a child of CWE-442 Web Problems in draft 8. Once again, this
relationship might be a useful way for a web developer to come across
relevant problems, but it doesn't tell us very much about the more
abstract issue behind CWE-444 nor does it give us any indication as to
how CWE-444 is related to its siblings aside from web technologies. For
draft 9, a relationship was added to CWE-444 making it a "ChildOf"
CWE-436 Interpretation Conflict, which is more aligned with the
fundamental problem behind CWE-444.

A hierarchy focused on abstraction can allow for more intuitive and
consistent navigation of the CWE tree from top to bottom. This approach
may be more useful to researchers, educators and coverage mapping for
code analysis tool vendors. Organizing the weaknesses hierarchically by
abstraction helps the CWE team identify gaps and inconsistencies in CWE
and it also helps tool vendors identify gaps in their mappings. For
example, duplicates CWE-132 Miscalculated Null Termination and CWE-170
Improper Null Termination used to be in disjoint segments of the
hierarchy. When reorganized by core weakness, they fell in the same
location; thus identifying the content of each entry as duplicates was
much more obvious. Furthermore, applying "Chains" and "Composites" to
an abstraction-based hierarchy can be useful for identifying higher
level trends in weakness occurrences, potential mitigation strategies
and their impact, and is another useful mechanism for finding gaps in
CWE coverage. In addition, these concepts have helped us to understand
why classification has been such a difficult challenge in the past.

Another benefit of collocating similar core weaknesses is the ease of
applying a consistent vocabulary across CWE. Weaknesses that had no
relationships before are brought together and allow the CWE team to see
inconspicuous relationships and trends. For example, CWE-696 Incorrect
Behavior Order was created as a weakness class for several entries
where the fundamental problem was performing operations in the
incorrect sequence. The first level of children of CWE-696 under the
natural hierarchy now read:

* CWE-179 Incorrect Behavior Order: Early Validation
* CWE-408 Incorrect Behavior Order: Early Amplification
* CWE-551 Incorrect Behavior Order: Authorization Before Parsing
and Canonicalization

As a result, when we peer into other views of CWE and come across these
weaknesses, we can immediately identify what the core issue is.

The natural hierarchy view is simply one view meant to unify the
weaknesses within CWE in a way that can be understood by a broad
audience based on underlying factors of each weakness rather than the
context in which the weakness occurs or how the weakness can be
mitigated. By ignoring such variables as environment, consequence,
mitigation, attacks, and relationship impact, we have been able to find
the factors that make each weakness unique and canonical. Likewise, we
have been able to identify gaps and create a more mature CWE model,
have identified areas for further research, and laid the groundwork for
a more solid understanding of the complexity of weaknesses and their
impact. With such a foundation we can then start adding in the
previously discounted variables to create a more complex and thorough
understanding of issues caused by the existence of these weaknesses.

Any thoughts, suggestions or concerns are welcome, either to this
thread or cwe@....

-Conor Harris

RE: Industry Analyst Coverage of CWE

2008-07-09T09:40:26Z

You typically start this process by filling out the briefing requests
on each analyst site:

http://www.forrester.com/Briefing/Process
http://www.burtongroup.com/Contact/VendorBriefing.aspx
http://www.gartner.com/it/about/vbriefings_faq.jsp

-----Original Message-----
From: Robert A. Martin [mailto:ramartin@...]
Sent: Wednesday, July 09, 2008 12:24 PM
To: McGovern, James F (HTSC, IT); cwe-research-list@...
Subject: Re: Industry Analyst Coverage of CWE

Hi James,

That would be something we would be interested in doing but probably
through direct discussion with specific interested individuals at each
organization.

Anyone with suggestions on who to work with at these or other such
groups or on ideas about how to educate them about CWE please feel free
to contact me directly or through this list.

We know some of the appropriate people but would welcome suggestions and
ideas.

Bob Martin
CWE Project Leader

At 11:57 AM -0400 7/9/08, McGovern, James F (HTSC, IT) wrote:
> As soon as the next version of CWE is released, is there an
>equivalent plan to communicate this to Gartner, Forrester and The
Burton Group?

>
>
>***********************************************************************
>** This communication, including attachments, is for the exclusive use
>of addressee and may contain proprietary, confidential and/or
>privileged information. If you are not the intended recipient, any
>use, copying, disclosure, dissemination or distribution is strictly
>prohibited. If you are not the intended recipient, please notify the
>sender immediately by return e-mail, delete this communication and
>destroy all copies.
>***********************************************************************
>**

*************************************************************************
This communication, including attachments, is
for the exclusive use of addressee and may contain proprietary,
confidential and/or privileged information. If you are not the intended
recipient, any use, copying, disclosure, dissemination or distribution is
strictly prohibited. If you are not the intended recipient, please notify
the sender immediately by return e-mail, delete this communication and
destroy all copies.
*************************************************************************

Re: Industry Analyst Coverage of CWE

2008-07-09T09:24:24Z

Hi James,

That would be something we would be interested in doing but probably
through direct discussion with specific interested individuals at
each organization.

Anyone with suggestions on who to work with at these or other such
groups or on ideas about how to educate them about CWE please feel
free to contact me directly or through this list.

We know some of the appropriate people but would welcome suggestions and ideas.

Bob Martin
CWE Project Leader

At 11:57 AM -0400 7/9/08, McGovern, James F (HTSC, IT) wrote:

> As soon as the next version of CWE is released, is there an equivalent
>plan to communicate this to Gartner, Forrester and The Burton Group?
>
>
>*************************************************************************
>This communication, including attachments, is
>for the exclusive use of addressee and may contain proprietary,
>confidential and/or privileged information. If you are not the intended
>recipient, any use, copying, disclosure, dissemination or distribution is
>strictly prohibited. If you are not the intended recipient, please notify
>the sender immediately by return e-mail, delete this communication and
>destroy all copies.
>*************************************************************************

Industry Analyst Coverage of CWE

2008-07-09T08:57:59Z

As soon as the next version of CWE is released, is there an equivalent
plan to communicate this to Gartner, Forrester and The Burton Group?

*************************************************************************
This communication, including attachments, is
for the exclusive use of addressee and may contain proprietary,
confidential and/or privileged information. If you are not the intended
recipient, any use, copying, disclosure, dissemination or distribution is
strictly prohibited. If you are not the intended recipient, please notify
the sender immediately by return e-mail, delete this communication and
destroy all copies.
*************************************************************************

Re: Suggestion of Repositioning CWE #244

2008-07-07T17:11:42Z

On Thu, 3 Jul 2008, ljknews wrote:

> At 3:08 PM -0400 7/3/08, koo wrote:
>
> > We suggest that CWE #244, Failure to Clear Heap Memory Before Release,
>
> It seems to me that it would be sufficient for the operating
> system to clear the memory before reallocation to a process.

One thing that I try to do when thinking about classification within CWE
is to separate the solution from the weakness itself. Most weaknesses
could have multiple solutions.

> Is there a separate item for clearing stack memory ? That
> would seem vulnerable in the same ways.

We don't have a CWE that's about stack memory.

But, this raises an abstraction question that we've been dealing with, and
which I touched on in the fall of 2007.

The general question is: do we create an individual CWE for stack memory?
How about the other resource types that were mentioned by Robert Seacord?

Both "failure to clear stack memory" and "failure to clear heap memory"
are related to resources. Their common parent is "memory," which we
currently think of as a fairly basic resource that's reasonable to cover
in CWE. So maybe there's a conceptual parent, "failure to clear memory."
Then you could go up another level, to the general concept of "resource",
i.e. "failure to clear resource" (which is basically a rephrasing of 404
and/or 459).

This "resource-specific abstraction" happens in other places in CWE, for
example CWE-122 (Heap-based buffer overflow) and CWE-121 (Stack-based
buffer overflow), as well as the various descendants of CWE-552 Files or
Directories Accessible to External Parties; each descendant covers a
different type of file, such as a backup or log file.

The general issue is, how specific must we get in order to create CWEs?
This was discussed in the fall. A combinatorial explosion could result if
we go too deep; we lose expressiveness if we're not specific enough.
This problem is now less severe since we have abstraction levels (Class,
Base, Variant) - we'll usually label resource-specific abstractions as
variants, so these could be removed from various views that don't want to
go that deep. It might also be useful to label the "dimension" along
which variants can occur, such as "resource-specific."

If we have this type of data available, then we don't need to reach the
same depth across all of CWE. We could add new nodes on an "as-needed"
basis if there is sufficient demand for it, and those nodes would exist in
some views, but not others.

In this particular case, the question is - is there a need to create
separate CWE entries for the failure to "clear" different types of memory,
and/or the different types of resources in general?

- Steve

Re: Suggestion of Repositioning CWE #244

2008-07-07T16:51:58Z

On Thu, 3 Jul 2008, koo wrote:

> We suggest that CWE #244, Failure to Clear Heap Memory Before Release, also
> be a child of CWE #226, Sensitive Information Uncleared Before Release.
> #244 is just the specific case of (mis)using realloc() for sensitive
> information.

This makes sense. Pending further commentary from the researcher list, we
will do this.

> They are both children of CWE #633, Weaknesses that Affect Memory, but
> don't have any other connection in the CWE that we can see.

This is a good example of the kind of "cleanup" and mapping task that we
are hoping to partially address with improved mapping guidance and
organizational views such as the natural hierarchy. We hope that the
natural hierarchy would provide a mechanism for more easily finding these
relationships.

> For that matter we suggest #244 NOT be a child of #404, Improper Resource
> Shutdown or Release. #404 is about freeing what you allocate, unlocking
> what you lock, etc. - it can lead to memory leak or resource leak.

We intend for 404 to be a pretty high-level entry that covers any
situation in which the developer does not properly "get rid of" resources
such as memory, locks, files, processes, and connections. Note how this
is separable from the *consequences* of those actions, such as information
leaks or resource consumption. It might be good for us to modify 404 to
better emphasize how we view it.

> #244 talks about sensitive information being exposed because it is not
> erased from a resource before being released.

Here, #244 suffers a little bit from a perspective problem.
Specifically, it is currently written to emphasize the information leak,
instead of the root cause of using memory allocation routines that might
release memory back to the OS in an uncontrolled fashion.

So, the descriptive text for 244 would need to be modified a bit to "fix"
the perspective to focus more on the weakness.

Now, about the relationships between 226, 244, and 404.

As you observed, 226 and 244 don't share any parents besides CWE-633,
Weaknesses that Affect Memory. 226 is currently classified under CWE-200,
Information Leak. But "information leak" is usually a consequence (i.e.,
resultant) - so it's typically the final link of a chain. While we
currently say that 226 is a ChildOf information leak, it would be better
represented as a CanPrecede relationship.

We then face the question of where 226 fits under the natural hierarchy,
since neither of its two parents are "natural parents:" 630 is actually a
view, and 200 is a chaining relationship, not a parent/child relationship.

We still think that 244 belongs somewhere under 404, probably as a
grandchild through its ChildOf relationship with 226 (i.e. 244 as a
ChildOf 226, and 226 as a ChildOf 404). 404 itself is a child of 664,
Insufficient Control of a Resource Through its Lifetime, which we view as
a likely "pillar" (top-level node) of the natural hierarchy.

There is another perspective issue to consider. Some might argue that
244, which is specifically about using realloc() to resize buffers, should
fall under CWE-227, API Abuse. This is how it was categorized in Seven
Pernicious Kingdoms, for example. CWE 1.0 will support a Seven Pernicious
Kingroms view (CWE-700, specifically) that will preserve this
relationship. However, we are also trying to figure out how to clearly
define CWE-227 in a way that doesn't effectively include everything that's
a weakness - after all, writing code that allows writing outside of a
memory buffer could distincly be counted as "API abuse" as well. We think
that CWE-227 has a place in the natural hierarchy, but that means that 244
would have multiple natural parents. Currently, we are allowing this to
happen - so "hierarchy" is currently a misnomer.

I hope this addresses your questions and helps to illuminate some of the
issues we face leading up to CWE 1.0.

- Steve

Re: Suggestion of Repositioning CWE #244

2008-07-07T07:15:10Z

Pascal & everyone,

Here is the recommendation we wrote for this rule for the C programming language:

MEM03-A. Clear sensitive information stored in reusable resources returned for reuse

Besides the neat use if alliteration, we list the following examples of reusable resources:

dynamically allocated memory
statically allocated memory
automatically allocated (stack) memory
memory caches
disk
disk caches

thanks,
rCs

ljknews wrote:

At 3:08 PM -0400 7/3/08, koo wrote:

We suggest that CWE #244, Failure to Clear Heap Memory Before Release,

It seems to me that it would be sufficient for the operating
system to clear the memory before reallocation to a process.
Why be concerned about the state when no process can access
it ?

Can you, or should you, as the paranoid secure programmer of an
application, trust the OS to do wipe heap memory before it passes the
memory on to another process or even uses it itself?

Is there a separate item for clearing stack memory ?  That
would seem vulnerable in the same way


There probably should be one, c.f. GCC Mudflap Pointer Debugging, the
-wipe-stack option at http://gcc.gnu.org/wiki/Mudflap_Pointer_Debugging

Koo's suggestion makes sense to me (moving 244).

Cheers,
Pascal

-- 
Robert C. Seacord
Senior Vulnerability Analyst
CERT/CC 

Work: 412-268-7608
FAX: 412-268-6989

Re: Suggestion of Repositioning CWE #244

2008-07-07T05:59:09Z

At 8:31 AM -0400 7/7/08, Pascal Meunier wrote:

> ljknews wrote:
>> At 3:08 PM -0400 7/3/08, koo wrote:
>>
>>> We suggest that CWE #244, Failure to Clear Heap Memory Before Release,
>>
>> It seems to me that it would be sufficient for the operating
>> system to clear the memory before reallocation to a process.
>> Why be concerned about the state when no process can access
>> it ?
>>
> Can you, or should you, as the paranoid secure programmer of an
> application, trust the OS to do wipe heap memory before it passes the
> memory on to another process or even uses it itself?

On the operating system I use, absolutely.
--
Larry Kilgallen

Re: Suggestion of Repositioning CWE #244

2008-07-07T05:28:59Z

ljknews wrote:
> At 3:08 PM -0400 7/3/08, koo wrote:
>
>> We suggest that CWE #244, Failure to Clear Heap Memory Before Release,
>
> It seems to me that it would be sufficient for the operating
> system to clear the memory before reallocation to a process.
> Why be concerned about the state when no process can access
> it ?
>
Can you, or should you, as the paranoid secure programmer of an
application, trust the OS to do wipe heap memory before it passes the
memory on to another process or even uses it itself?

> Is there a separate item for clearing stack memory ? That
> would seem vulnerable in the same way

There probably should be one, c.f. GCC Mudflap Pointer Debugging, the
-wipe-stack option at http://gcc.gnu.org/wiki/Mudflap_Pointer_Debugging

Koo's suggestion makes sense to me (moving 244).

Cheers,
Pascal

Re: Suggestion of Repositioning CWE #244

2008-07-03T13:37:46Z

At 3:08 PM -0400 7/3/08, koo wrote:

> We suggest that CWE #244, Failure to Clear Heap Memory Before Release,

It seems to me that it would be sufficient for the operating
system to clear the memory before reallocation to a process.
Why be concerned about the state when no process can access
it ?

Is there a separate item for clearing stack memory ? That
would seem vulnerable in the same ways.
--
Larry Kilgallen

Suggestion of Repositioning CWE #244

2008-07-03T12:08:50Z

We suggest that CWE #244, Failure to Clear Heap Memory Before Release, also be a child of CWE #226, Sensitive Information Uncleared Before Release. #244 is just the specific case of (mis)using realloc() for sensitive information.

They are both children of CWE #633, Weaknesses that Affect Memory, but don't have any other connection in the CWE that we can see.

For that matter we suggest #244 NOT be a child of #404, Improper Resource Shutdown or Release. #404 is about freeing what you allocate, unlocking what you lock, etc. - it can lead to memory leak or resource leak. #244 talks about sensitive information being exposed because it is not erased from a resource before being released.

For your convenience, here are the URLs and descriptions

http://cwe.mitre.org/data/definitions/244.html

Using realloc() to resize buffers that store sensitive

information can leave the sensitive information exposed to

attack, because it is not removed from memory.

http://cwe.mitre.org/data/definitions/226.html

The software does not fully clear previously used information in

a data structure, file, or other resource, before making that

resource available to a party in another control sphere.

http://cwe.mitre.org/data/definitions/404.html

The program fails to release - or incorrectly releases - a

system resource before it is made available for re-use.

Michael Koo

on Behalf of SAMATE Team

Identifying Perspective Issues in CWE

2008-07-02T13:22:08Z

All,

As we've been developing the natural hierarchy, we've started paying
more attention to the problem in which CWE supports multiple different
perspectives. This is posing challenges for developing the natural
hierarchy, but it is also reflected in how tools map to CWE.

The purpose of this message is to raise awareness about these
challenges, and to solicit feedback from CWE researchers on ways of
handling these.

Here's a live example to work with. A fairly well-known research team
called Security Reason recently released an advisory that maps to a
CWE number:

CVE-2008-2665
http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-2665

PHP 5.2.6 posix_access() (posix ext) safe_mode bypass
URL:http://securityreason.com/achievement_securityalert/54

In short, PHP's safe mode feature is a protection mechanism. It is
can limit which OS resources can be accessed by a PHP application,
such as files or potentially dangerous functions. The PHP interpreter
uses an incorrect order of behaviors related to canonicalization,
causing its safe mode check to be insufficient. This could allow an
attacker to conduct path traversal attacks against applications.

Here, Security Reason mapped to CWE-264, which is a Category for
"Permissions, Privileges, and Access Controls." They presumably did
this mapping because of the emphasis on bypassing safe mode - or,
because it's in the NVD Slice, which Bob Martin and I have been
advocating as a reasonable mapping system. At any rate, the "safe
mode" functionality is the LOCATION of the issue - a protection
mechanism intended to provide access control. It is not really
talking about WHAT the issue actually is.

Many people probably would have mapped this same issue to CWE-22 for
path traversal, which is the ultimate consequence. (For those who are
interested in chains, notice how path traversal is the end link in
this case.)

Alternately, you could map to the more generic "protection mechanism
failure" (CWE-693), a new entry, which might be regarded as a
"property of the consequence," or alternately, an intermediate link in
a chain.

Then there's another weakness that people often call the "root cause."
It appears that some inputs are checked against safe mode, but then
those inputs are later canonicalized in a way that generates a result
that bypasses safe mode. This is CWE-180, "Incorrect Behavior Order:
Validate Before Canonicalize." This entry's perspective is largely
based on "code properties" - i.e., it basically involves properties of
code that are effectively independent of the types of security
problems they introduce (Consider CWE-227, API Abuse, as another
example of an entry focused on code properties. The implications of
the API abuse will vary widely depending on the functionality and
assumptions of the API).

So, we have at least 4 potential CWEs that this issue could be mapped
to, because each CWE is written from different perspectives, or
applies to different parts of the chain. This should have obvious
implications for understanding tool capabilities. If one tool maps to
CWE-x, and another tool maps to CWE-y, then it will appear like two
unrelated problems are being covered, when they might just be
different pieces of the same problem.

The question becomes, which mapping should tool vendors or researchers
use, and in which context? Ideally, we would like mapping to be
repeatable, i.e. independent of who's doing the mapping.

Should CWE get rid of CWE-264 entirely, since it's a category?
Absolutely not! It's a very useful way to group things in a way that
reflects how humans think, and essential for some views. And in the
CVE/NVD world where you don't have much information, and you only want
a couple dozen CWE's to map to, it serves its purpose well. But,
neither does CWE-264 belong in the natural hierarchy, because it's not
about a specific type of weakness. So, we're moving it out of the
natural hierarchy (along with other categories), into a new view that
we're informally referring to as a "programming concepts" view.

What about defining guidelines that would map to CWE-22 (path
traversal)? This is probably how most people would map these days.
One could argue that CWE-22 should only be applied in cases where
there's a failure to even TRY to prevent '..' and related sequences.
In this specific case though, in my personal opinion, the most
appropriate mapping is CWE-180, the root cause.

So - even if we establish mapping guidelines that say "stay away from
categories," a mapper would still be looking at a couple different
weaknesses. If we suggest: "stay away from categories AND
consequences," then the mapper would (hopefully) arrive at CWE-180 as
well. This assumes that the person doing the mapping is following
these guidelines, however - which in practice doesn't always happen.

Suggesting that a mapper should "find the root cause" might be more
useful, but that would require a certain mindset that not all mappers
are familiar with, especially if they conduct application
vulnerability research. In addition, tools aren't necessarily focused
on root causes. If we go in this direction, then we might want to
actively discourage mapping to weaknesses that are solely (or
primarily) about consequences, such as NULL pointer dereferences.

Then there's another possible guideline - "map to everything that
matches." This has certain strengths and limitations. It would
effectively support audiences with multiple perspectives, so it would
be useful for people to understand the general contents of a single
tool or repository. But it probably wouldn't work well for a deep
analysis of multiple tools.

The issues I've discussed are illustrative; I don't expect us to come
up with any quick and easy answers.

But, now that we're more specifically aware that we have these perspective
challenges, we are working on identifying and labeling the different
perspectives that are currently being used in CWE. We are currently
examining various tool repositories to provide real-world examples for us
to work with. This effort won't be complete for the release of CWE 1.0,
but I'd like to be able to describe the problem in an understandable way.

Any suggestions or concerns are more than welcome, whether to this list or
to cwe@....

- Steve

Upcoming plans for CWE 1.0

2008-07-01T09:45:42Z

All,

MITRE plans to release CWE 1.0 sometime in August. Here is a summary
of our main goals for that release.

1) Finish existing systemic changes. We want CWE 1.0 to represent a
stable point in CWE's development, which means finalizing the
systemic changes that we've been making over the past year or two.
For this, we are de-prioritizing general "content maintenance" -
i.e., localized modification of individual entries - except as
those modifications might relate to the systemic changes. After
CWE 1.0 is released, we plan to move more into a content
development and refinement mode, in which there will be greater
emphasis on accuracy and completeness of individual entries.

2) Stable schema. We have been making significant schema changes over
the past year, primarily to support our development of views, as
well as the needs of new stakeholders. Our primary goal for CWE
1.0 is to have the schema be stable. We are conducting an internal
review and have outlined the major limitations that still need to
be addressed.

3) Viable views. We have been developing the view concept and
implementation for almost a year now, and we think we finally have
a handle on how we want to support them. So CWE 1.0 will have
multiple views that support different use-cases and stakeholders,
and the schema infrastructure will be in place to support adding
more views without requiring schema modifications.

4) Refinement of the Natural Hierarchy. We have come to realize that
we need to do a better job of communicating what we're trying to
accomplish with the Natural Hierarchy (CWE-1000). In short, we are
attempting to build a classification scheme based on inherent
features of weaknesses of large portions of CWE weaknesses, and
their inter-relationships. My personal hope is that it will take
Seven Pernicious Kingdoms and CLASP one step further. All past
versions of CWE have had multiple ways of grouping weaknesses
together that would lead to difficulty or inconsistency in
performing mappings. It would also be difficult to infer where
knowledge gaps existed. The MITRE team has found that the ongoing
development of the natural hierarchy has helped us significantly in
understanding much of what we have in CWE, and why. Academic
researchers might be especially interested in its development.

Ironically, the natural hierarchy might not seem so "natural" to
regular developers; so, we need to actively support the developer
view. This is one major challenge that we face.

In the coming weeks, we will be releasing a more detailed white
paper to the community on MITRE's goals for the natural hierarchy.
Traces of it exist in CWE Draft 9, but it is far from complete (and
we've since made significant inroads in our 1.0 development). To
get an idea of where we are headed, see: CWE-664 ("Insufficient
Control of a Resource Through its Lifetime"), CWE-682 ("Incorrect
Calculation"), and CWE-691 ("Insufficient Control Flow
Management"). If you are particularly interested in this effort,
then contact us offline and we will give you our current status.

5) More active community engagement. Leading up to CWE 1.0, we will
be actively engaging community members on important issues for CWE.
This discussion list will be one of the main places in which we
solicit feedback. So, this summer is the time to voice any
concerns you have.

6) Resolution of outstanding issues. In the fall of 2007, we brought
up various issues related to CWE content maintenance, including
which types of issues we should include, and what level of
abstraction we should use. We will be actively resolving many of
those issues. See the Working Documents section at
http://cwe.mitre.org/community/workingdocs.html for a refresher.

7) Identifying CWE gaps with respect to current tools, including
guidance for mapping. Several tool vendors have sent us updated
lists of their checks, some of which had CWE mappings. We are
evaluating these mappings to ensure that CWE 1.0 will be able to
support the existing perspectives under which these tools operate.
This might include creating high-level entries that act as
placeholders for future content creation of lower-level entries.
We will not have the time to create new entries for every gap that
we encounter, at least by the release of 1.0, but we will have a
solid understanding of what remains to be done.

Re: Examples of Name and Relationship Changes in CWE Draft 9

2008-04-23T07:26:56Z

Steven M. Christey wrote:
> we changed the names of over 200 entries in Draft 9 alone
...
> 401 Memory Leak
>
> RENAMED: Failure to Release Memory Before Removing Last Reference
> (aka 'Memory Leak')

I like the _idea_ of more-specific-names, but this one isn't quite right
on two counts:

1. Many memory leaks are due to circular structures, e.g., A references
B, and B references A, yet NOTHING refers to either. This _IS_ a memory
leak, but not by this name.

2. Memory leaks only happen if the run-time doesn't support the
necessary kind of garbage collection. Some systems build in
reference-counting collectors, which means you don't need to worry about
releasing memory UNLESS there's a circularity.

I think what you mean is something like
"Failure to Release Memory After It Becomes Unreferenceable"

--- David A. Wheeler

Examples of Name and Relationship Changes in CWE Draft 9

2008-04-21T14:15:21Z

If you examine the difference report at
http://cwe.mitre.org/data/reports/diff_draft_8_9.html , you will see that
we changed the names of over 200 entries in Draft 9 alone, and we added
275 relationships while removing 75. We also changed nearly 200
descriptions.

The main goals were:

- make the CWE name and description more clear about the weakness
being covered, and try to keep the perspective on the weakness
itself, instead of the attack or consequence - but preserve such
terminology if it's commonplace.

- when the CWE is identifying a weakness, try to classify it under
the Natural Hierarchy view (CWE-1000), i.e. it should have a parent
that is a Weakness (Variant, Base, or Class). If a new node is
necessary, create it (or flag the issue for review after Draft 9's
release).

We tried to change the names so that a CWE consumer would not have to
depend so much on looking up the item's description and context notes,
just to figure out what the item was talking about. We tried to
remove perspective problems where feasible, such as when a name was
too focused on the associated attack.

The litmus test for a name change was simple: if a CWE analyst didn't
know what the issue was about upon reading the name, then most CWE
users probably wouldn't know either. As a result, we removed a lot of
non-specific terms such as "insecure," "improper," and "erroneous," or
tried to develop some consistency when we needed to use more general
terms, such as "sanitization" as an over-arching term that could cover
failure to filter, decode, quote, validate, etc.

We didn't identify all the names that needed fixing, but 37% of CWE
entries were modified, so this was a solid start.

We definitely didn't identify the natural parents for every entry,
although this effort did produce many of the new entries that were
added to Draft 9. We expect this to be an ongoing process. See the
CWE-1000 definition for additional explanation of the natural
hierarchy.

Below are a few examples of the name changes, along with relationships
that we added, to give people a sense of what we did and why.

--------------------------------------------------------

582: Mobile Code: Unsafe Array Declaration

- what's unsafe about it - is this permissions? buffer overflow?
something else?

New name: Array Declared Public, Final, and Static

--------------------------------------------------------

568: Erroneous Finalize Method

- what's the error? does the software define the finalize method
incorrectly? is this permissions? Does the method do too much?
too little? operates on the wrong object? has a memory leak?
sends private data?

New name: finalize() Method Without super.finalize()

Old parent: 399: Resource Management Errors

New parents:

573 - Failure to Follow Specification
404 - Improper Resource Shutdown or Release

--------------------------------------------------------

73: Path Manipulation

- name is attack-focused

- how is it being manipulated - symbolic link? long pathname? path
traversal? appending "%20" to retrive source code?

- first code example is path traversal (CWE-22)

- second code example may or may not be path traversal

- RENAMED: "External Control of File Name or Path"

- ADDED ChildOf 99 Insufficient Control of Resource Identifiers (aka
'Resource Injection')

--------------------------------------------------------

4: J2EE Environment Issues

This is a general category node whose name is self-explanatory. In
draft 8, however, its children rarely had any natural parents.

child: 5 J2EE Misconfiguration: Insecure Transport

RENAMED: J2EE Misconfiguration: Data Transmission Without Encryption

ADDED: ChildOf 311 Failure to Encrypt Sensitive Data

child: 555 J2EE Misconfiguration: Password in Configuration File

RENAMED: J2EE Misconfiguration: Plaintext Password in
Configuration File

ADDED: ChildOf 522 Insufficiently Protected Credentials

DESCRIPTION: modified

child: 6 J2EE Misconfiguration: Insufficient Session-ID Length

ADDED: ChildOf 334 Small Space of Random Values

child: 7 J2EE Misconfiguration: Missing Error Handling

Unchanged

child: 8 J2EE Misconfiguration: Entity Bean Declared Remote

ADDED: ChildOf 668 Exposure of Resource to Wrong Sphere

child: 9 J2EE Misconfiguration: Weak Access Permissions

RENAMED: J2EE Misconfiguration: Weak Access Permissions for EJB Methods

ADDED: ChildOf 275 Permission Issues

--------------------------------------------------------

597 Erroneous String Compare

- what's the error - only a portion of the string is compared? It
compares a string in a case-insensitive manner? It doesn't handle
when one string is shorter than the other?

RENAMED: Use of Wrong Operator in String Comparison

--------------------------------------------------------

591 Memory Locking

- is this about not locking memory? Locking it incorrectly? is
this a category of all different types of weaknesses that can
occur during memory locking?

RENAMED: Sensitive Data Storage in Improperly Locked Memory

--------------------------------------------------------

590 Improperly Freeing Heap Memory

- does this mean double free? running free() on an object that was
allocated using new() ?

RENAMED: Free of Invalid Pointer Not on the Heap

--------------------------------------------------------

560 Often Misused: umask()

- is this about setting an insecure umask? Not specifying a umask
and using one that you've inherited from the caller of your
program?

RENAMED: Use of umask() with chmod-style Argument

FORMER PARENT: 559 "Often Misused: Arguments and Parameters"

New parent: 687 Function Call With Incorrectly Specified Argument Value

--------------------------------------------------------

474 Inconsistent Implementations

- is this about things like how web browsers can behave differently?

RENAMED: Use of Function with Inconsistent Implementations

--------------------------------------------------------

401 Memory Leak

RENAMED: Failure to Release Memory Before Removing Last Reference
(aka 'Memory Leak')

Natural parents: none in draft 8

ADDED: ChildOf 404 Improper Resource Shutdown or Release

CWE Draft 9 Major Schema Changes, and Outstanding Issues

2008-04-16T14:16:41Z

All,

We've significantly modified the schema for Draft 9. The primary
driver was to improve support for multiple views, and to better
distinguish between the different types of elements that we are
covering in CWE. Thanks to Sean Barnum for figuring out the bulk of
this. The MITRE team took his inputs and made some small tweaks here
and there.

As CWE is at a crossroads with respect to the schema, we welcome any
feedback or alternatives to our current approaches. Specifically,
while we have chosen XML so far, we are open to leveraging other
techniques to storing and working with the data, if those techniques
are more effective. For example, if it makes sense to store CWE in a
database and use an application server to help present and link
everything together, we are open to pursuing that. We also plan to
investigate RDF, XGGML, and other languages that might be more
directly supportive of graph-based relationships.

Please note that even if we stay with XML and related technologies, we
expect that the schema will still need to change a little bit.
However, we believe that one requirement for "CWE 1.0" is to have
stable schema. In Draft 9, we are definitely a lot closer than we
were. Bob and I will post our requirements for "CWE 1.0" once they've
been finalized.

For Draft 9, some of the highest level schema changes are covered
here:

http://cwe.mitre.org/data/reports/diff_xsd_10_3.0.html

The rest of this document assumes that you read the preceding
document.

Any and all feedback would be appreciated, especially if there are
still outstanding issues in the schema that prevent you from using CWE
as extensively as you would want to.

Schema Evaluation Criteria
--------------------------

Here are some of the criteria that I think we should be applying while
finalizing the schema:

- Expressiveness: we should be able to express everything that we
want to. In Draft 9, some examples of this are the creation of
explicit views, and the requirement for relationships to specify
the views they are part of. But, we still don't have a way of
saying things like "this issue theoretically affects any language
that performs direct memory management, but it's especially common
in C." That's important, because if C is not explicitly mentioned
in an element, then that element won't be part of the C language
view.

- Extraction: it should be as easy as possible for CWE users to
extract the data that they want, using commonly available XML
parsers and related tools. In Draft 9, the relevant data for
named chains are not necessarily easy to extract.

- Maintenance

- minimize maintenance costs: the MITRE team, and outside
contributors, should be able to quickly represent the necessary
information.

- minimize preventable errors in data entry: we want to minimize
errors in the CWE representation that cannot be caught by an XML
validator, but nonetheless require consistency.

- minimize XML "bloat": this is hopefully self-explanatory. The
relationships in Draft 9 might exhibit some bloat, although at
the same time, there's a major benefit to their increased
expressiveness.

- Flexibility: ideally, the schema would remain stable, while
allowing us to build in additional capabilities. For Draft 9, we
believe that we've added flexibility for defining new kinds of
relationships and views. The introduction of compound elements
will hopefully allow us to support other kinds of concepts besides
chains and composites that might arise in the future; for example,
some CWE nodes are really talking about multiple distinct issues
and could be called "loose composites."

In light of these criteria, I wanted to explain some of the rationale
for the schema changes, and what we have left ahead of us for CWE 1.0.

Views
-----

We added a number of views to CWE Draft 9. For the most part, this
involved converting weakness/"groupings" from Draft 8, into the new
Views type for draft 9.

See http://cwe.mitre.org/data/index.html for a list of views.

Slices are basically lists of elements, without any relationships
between them. Membership in a slice can be explicit or implicit. In
explicit slices, all the relevant entries have some ChildOf
relationship where the View node is the parent; see CWE-630
(Weaknesses Examined by SAMATE) and CWE-635 (Weaknesses Used by NVD)
for examples.

In implicit slices, the slice has some filtering criteria that define
membership, and there aren't any relationships within the XML that are
explicitly defined. For example, CWE-658 is a slice that covers
weaknesses found in the C language. This implicit slice has a Filter
that specifies that member entries have "C" under the
Applicable_Platforms field.

The Comprehensive CWE Dictionary view, CWE-2000, is actually an
implicit slice that selects everything from CWE by using a filter that
always returns true.

Views can also be graphs, such as CWE-1000 (Natural Hierarchy).
Currently, graphs are expected to have explicit ChildOf relationships
within the member elements. Before Draft 9, everything was
effectively under the Natural Hierarchy. In Draft 9, however, some of
those elements have been removed from the Natural Hierarchy
altogether, like deprecated nodes and the resource-based view.

We suspect that some individual views might be best described as a
combination of slices *and* graphs, with a combination of implicit or
explicit membership. A view might be best expressed via some set of
explicit relationships (maybe between some implicit slices), then
defaulting to the relationships of a different view at some point.

The most concrete example of this is CWE-631 (Resource-specific
Weaknesses), at:

http://cwe.mitre.org/data/graphs/631.html

The higher-level nodes have explicit relationships defined within View
631. Its children - such as the Category node CWE-632 (Weaknesses
that Affect Files or Directories) - have explicitly specified children
such as CWE-22 (Path Traversal). That is, Path Traversal has an
explicit "ChildOf CWE-632" relationship. However, instead of the
explicit relationships, CWE-632 could potentially be defined as an
implicit slice of "all elements that have an Affected_Resource field
of File/Directory." That would reduce maintenance costs and improve
accuracy, but it is not possible in Draft 9, because CWE-632 is a
Category type - it's *in* a view, but not a view itself.

In addition, the resource-based view, CWE-631, could be more
comprehensive by "view hopping." In Draft 9, CWE-631 stops at CWE-22
(Path Traversal), but there are several children under CWE-22 that
would also match - except those children are only listed under the
natural hierarchy (view CWE-1000). It would probably be quite tedious
and error-prone just to copy all the natural hierarchy relationships
over to this new view. This might be best handled by allowing views
to link to each other, but this is not possible in Draft 9. In
addition, the "hops" might wind up including elements that were not
intended.

Finally, we have encountered some difficulties in generating a
"Comprehensive Graph" that merges all views together - the natural
hierarchy, the resource-based graph, the language-specific slices,
etc. So, there isn't a single graph on the CWE web site that covers
the entire CWE. We do have a PDF file that contains most nodes; it
focuses on the natural hierarchy (CWE-1000), and all other nodes are
effectively "orphans." We don't necessarily have to solve this
problem for a comprehensive view - after all, it's not clear who would
have a need for such a thing - but I thought it was worthwhile to
mention.

Relationships
-------------

The expression of relationships has changed significantly for Draft 9.
Much of this is covered by the schema diff report listed at the top of
this document, but there are some fields that I wanted to highlight.

Relationship_Type:

The Draft 8 version of "Relationship_Type" has been renamed to
"Relationship_Nature". The Draft 9 version of this field is
intended to identify the type of the entry that is being linked to.
Since we now have multiple types of entries in CWE, this field might
be useful in simplifying some extraction and presentation logic for
XSLT's. We have not needed this field in generating the web site
for Draft 9, although it might be convenient for others. However,
this field is currently being manually maintained, and this value
was often incorrect, because we changed the types of a number of
elements in Draft 9, which immediately invalidated this field in
dozens of relationships. We are able to perform a consistency check
to ensure that these values are correct before release, but it's
still a little bit of labor.

As a result, we will be looking at this field more closely, trying
to balance utility to the community with maintenance costs to the
CWE team.

Relationship_View_IDs:

We anticipate that, in the future, we will have multiple views that
share a lot of the same structure. As one example - CWE's Natural
Hierarchy (CWE-1000) is beginning to diverge more from the Seven
Pernicious Kingdoms (SPK) way of organizing the world, so it might
be reasonable to create a view into CWE that's useful for people who
are knowledgeable about SPK. The Natural Hierarchy and an SPK view
would probably have a lot of different elements near the top of the
tree, but they would share a lot at a lower level.

With closely overlapping views, this would produce a large number of
duplicate relationships that might contribute significantly to XML
bloat. The MITRE team decided that allowing multiple
Relationship_View_IDs would be a useful shorthand that might be
easier to maintain.

Current Challenges
------------------

Here are some of the current challenges that we still face, and plan
to resolve by CWE 1.0.

1) The Draft 9 schema does not have the expressiveness to define the
more complex views, and there are some associated maintenance
costs, as outlined in the previous sections.

2) Chains and composites, views, and categories all have some
overlapping uses that we'd like to clarify and, to the degree
possible, unify.

For example, both chains and composites involve a small selection
of entries from CWE, and dictate relationships between them. In
this sense, they can be regarded as views - perhaps micro-views.
Yet, we expect that they will have a distinct and important role
throughout CWE.

As another example, the resource-based view (CWE-632) has children
that are categories. These categories might be best described by
defining what their membership should be, but in Draft 9, this type
of automatic population is only possible through filters in View
elements. So, we had to manually create ChildOf relationships.

3) Relationship Directionality

Some views, like CWE-635 (Weaknesses Used by NVD), are defined more
by external criteria than anything that is implicit within
individual nodes, so these are explicit slices. In terms of
maintenance costs and ease of extraction, it might be best for
CWE-635 to explicitly state what its "members" are. Instead, each
member has a ChildOf relationship, with View_ID=635, that is a
ChildOf 635. Thus, maintenance of the NVD slice is done not by
operating on the slice itself, but by operating on its individual
members. This proved to be moderately expensive for us to do when
we changed the membership of the SAMATE view in Draft 8 - it took
an hour or so to edit some nodes to remove the SAMATE relationship,
and then edit other nodes to add the SAMATE relationship; if we
could just edit the SAMATE list directly, it would have been a
5-minute task. However, as I understand it, one of the mantras of
knowledge management is that data is kept as close to individual
nodes as possible; but relationships "belong" to multiple nodes,
even though in Draft 9 they are only explicit in one node.

It would be possible for us to automate some of those maintenance
tasks, but that would involve additional development.

Also, we have multiple relationships that are mutual, but only
expressed in one direction. For example, "X ChildOf Y" might be
specified in the XML, which implies "Y ParentOf X" - but we have no
ParentOf relationships that are explicitly stated. The same thing
applies for relationships that support chains and composites. As a
result, extraction logic can be complicated, because an entry
doesn't explicitly know what its children are. As a result of this
complexity, the extraction logic can be hard to maintain, and
sometimes computationally expensive. We have encountered this
problem in various ways while generating web site pages.

One possibility would be to create separate XML files and
representations for the relationships (and maybe for views),
possibly with separate schema. This might preserve expressiveness
and simplify maintenance, but it might make it more difficult for
some people to extract.

4) Named Chains

There are a couple issues with named chains. See
http://cwe.mitre.org/data/reports/chains_and_composites.html for
background.

All the data that's required to determine the links of a named
chain are within the XML, so there is sufficient expressiveness.
However, extraction is a little more difficult. For a named chain
X, the code has to search throughout all of CWE for entries with
all the CanPrecede relationships with a Chain_ID of X, then order
them appropriately. If you want to know what elements are in a
named chain, you HAVE to do this navigation throughout CWE - a
named chain does not explicitly state what its links are. Just
like composites explicitly state which items they require, it might
be reasonable to have named chains explicitly know what their
starting links are.

In addition, named chains can be difficult to classify, especially
under the natural hierarchy. Because named chains are new, we
decided not to create a separate view to handle them. We do have a
view that lists chain elements (CWE-679), but that view is actually
an implicit slice for extracting components of all the CanPrecede
relationships, whether they're related to a Named Chain or not.

The extraction and presentation logic for presenting chains in
general was too complicated for us to handle cleanly by the release
of Draft 9, so they are generated by external programs, instead of
through XSLT.

Release of Draft 9

2008-04-14T14:16:02Z

All,

Just in case some of the researchers are not subscribed to the CWE
announce list, following is the announcement of Draft 9. We have
accomplished a lot in this draft, and I hope you will agree.

In the next day or so, I'll post some more extensive discussion of some of
the schema changes and some outstanding issues, and I'll discuss some of
the content changes as well.

If you have any questions or concerns, please notify the CWE team, or post
to this list. We do want to have some active discussion on this list, and
the transition between Draft 9 and "CWE 1.0" will provide ample
opportunity.

- Steve

=============

The ninth draft of CWE has been posted on the CWE List page. It
includes several important changes. A report is available that
lists specific differences between Draft 8 and Draft 9.

Specific changes for Draft 9 include: significant schema changes
to better distinguish and link between Weaknesses, Categories,
Views, Chains, and Composites; 39 new entries, many of which
improve the organization of CWE; changes to the names or
descriptions of over 200 entries, in order to more accurately
reflect each entry; modification of relationships related to
classification under a "natural hierarchy" view; addition of a
Status field to reflect the maturity of each entry; an updated
report on prioritization of fields; the introduction of named
chains; and many other changes affecting over 450 entries.