list to table

[jay-132]

Hi All,

Am new to using newgroup for help, but have exhausted my searching
online for a solution

I have a long list of data of associations between values with a value
to that association as follows..

(var) to (var) = (var) hits
A B 1
B A 1
A B 3
B A 3
A C 7
C A 2

And need to build a table as follows that accumulates the above:

row is (from)
column is (to)

   A B C
A 0 4 7
B 4 0 0
C 2 0 0

Just can't seam to figure out how to manage this programatically.

Any help or guidance much appreciated !!!


Cheers,


Jay
--
http://mail.python.org/mailman/listinfo/python-list

[Scott David Daniels]

jay wrote:

> ...
> I have a long list of data of associations between values with a value
> to that association as follows...
> A B 1
> B A 1
> ... And need to build a table as follows that accumulates the above:
>
> row is (from)
> column is (to)
>
>    A B C
> A 0 4 7
> B 4 0 0
> C 2 0 0
>
> Just can't seam to figure out how to manage this programatically.

How to solve this:  Define (quite carefully) what you mean by
"build a table" for yourself.  Then, determine a single step
that either gets you closer to the answer (going forward) or
(working backward) determine something (typically some data
structure) that you could use to produce the result you need.
Keep pushing at both ends until you can get them to meet in
the middle.

--Scott David Daniels
Scott.Daniels@...
--
http://mail.python.org/mailman/listinfo/python-list

[Alf P. Steinbach]

* jay:

>
> I have a long list of data of associations between values with a value
> to that association as follows..
>
> (var) to (var) = (var) hits
> A B 1
> B A 1
> A B 3
> B A 3
> A C 7
> C A 2
>
> And need to build a table as follows that accumulates the above:
>
> row is (from)
> column is (to)
>
>    A B C
> A 0 4 7
> B 4 0 0
> C 2 0 0
>
> Just can't seam to figure out how to manage this programatically.

You're not very clear on what A, B and C are. Assuming that they're constants
that denote unique values you can do something like

<code>
table = dict()
for k1, k2, n in list:
     position = (k1, k2)
     if position not in table:
         table[position] = n
     else:
         table[position] += n
</code>

Disclaimer: I'm a Python newbie so there may be some much easier way...


Cheers & hth.,

- Alf
--
http://mail.python.org/mailman/listinfo/python-list

[Jon Clements-2]

On Nov 5, 4:09 pm, "Alf P. Steinbach" <al...@...> wrote:

> * jay:
>
>
>
>
>
> > I have a long list of data of associations between values with a value
> > to that association as follows..
>
> > (var) to (var) = (var) hits
> > A B 1
> > B A 1
> > A B 3
> > B A 3
> > A C 7
> > C A 2
>
> > And need to build a table as follows that accumulates the above:
>
> > row is (from)
> > column is (to)
>
> >    A B C
> > A 0 4 7
> > B 4 0 0
> > C 2 0 0
>
> > Just can't seam to figure out how to manage this programatically.
>
> You're not very clear on what A, B and C are. Assuming that they're constants
> that denote unique values you can do something like
>
> <code>
> table = dict()
> for k1, k2, n in list:
>      position = (k1, k2)
>      if position not in table:
>          table[position] = n
>      else:
>          table[position] += n
> </code>
>
> Disclaimer: I'm a Python newbie so there may be some much easier way...
>
> Cheers & hth.,
>
> - Alf

I read the OP as homework (I'm thinking Scott did as well), however,
your code would be much nicer re-written using collections.defaultdict
(int)... which I don't think is giving anything away...

However, the challenge of making it 'tabular' or whatever the
requirement of 'table' is, is still there.

Jon.

--
http://mail.python.org/mailman/listinfo/python-list

[Alf P. Steinbach]

* Jon Clements:
>
> I read the OP as homework (I'm thinking Scott did as well),

Sorry. Need to recalibrate that neural network. Back-propagation initiated...
Done! :-)


> however,
> your code would be much nicer re-written using collections.defaultdict
> (int)... which I don't think is giving anything away...

Thanks!

This sent me searching everywhere, because the documentation of '+=' and other
"augmented assignment statements" says

   "The target is only evaluated once.",

like in C++, which implies a kind of reference to mutable object.

I couldn't immediately see how subscription could apparently return a reference
to mutable int object in Python in a way so that it worked transparently (the
idiom in C++).

However, it worked to replace collections.defaultdict with a class overriding
__getitem__ and __setitem__, so I guess that's how it works, that in this case
'+=' is simply translated like 'x += n' -> 'temp = x; x = temp + n'.

Is this a correct understanding, and if so, what exactly does the documentation
mean for the general case?

E.g.

def foo():
     print( "foo" )
     d = dict(); d[43] = 666
     return d

def bar():
     print( "bar" )
     return 43;

foo()[bar()] += 1

produces

   foo
   bar

so here it's not translated like 'foo()[bar()] = foo()[bar()] + 1' but evidently
more like 'a = foo(); i = bar(); a.__setitem__(i, a.__getitem__(i) + 1)'?

If so, is this behavior defined anywhere?

I did find discussion (end of §6.2 of the language reference) of the case where
the target is an attibute reference, with this example:

class A:
     x = 3    # class variable
a = A()
a.x += 1     # writes a.x as 4 leaving A.x as 3

:-)


Cheers, & thanks,

- Alf
--
http://mail.python.org/mailman/listinfo/python-list

[Gabriel Genellina-7]

En Thu, 05 Nov 2009 21:23:27 -0300, Alf P. Steinbach <alfps@...>  
escribió:
> * Jon Clements:

> This sent me searching everywhere, because the documentation of '+=' and  
> other "augmented assignment statements" says
>
>    "The target is only evaluated once.",
>
> like in C++, which implies a kind of reference to mutable object.

Not exactly. For an augmented assignment, the target (left side) may be an  
identifier, an atttribute reference, or a subscription/slicing. In the  
first case, the comment simply does not apply (there is a single one  
opportunity to evaluate the target).

a += some(expression) means: evaluate "a", evaluate some(expression),  
perform the operation +=, bind the resulting object to the name "a".

a.attr += some(expression) means: evaluate "a", ask it for its attribute  
"attr", evaluate some(expression), perform the operation +=, ask the  
(already known) object "a" to store the resulting object as its attribute  
"attr".

a[index] += some(expression) performs a similar sequence of steps; "a" is  
evaluated only once, then it is asked to retrieve its element at [index],  
the computation is performed, and finally the (already known) object "a"  
is asked to store the result at [index].

"The target is only evaluated once." means that it is NOT reevaluated  
before the final result is stored. Same applies to "index" above, although  
this is not explicited. Perhaps it is more clear with a longer expression:  
a[b+c].c[d(e[f])].h += 1 computes the a[b+c].c[d(e[f])] part only once.

> I couldn't immediately see how subscription could apparently return a  
> reference to mutable int object in Python in a way so that it worked  
> transparently (the idiom in C++).

It does not. It perform first a "getitem" operation followed by a  
"setitem" to store the result. A normal dictionary would fail when asked  
for an inexistent item; a defaultdict creates and returns a default value  
in such case.

py> import collections
py> d = collections.defaultdict(int)
py> d['a'] += 1
py> d['a'] += 1
py> d['a']
2
py> d['b']
0

note: int() returns 0; defaultdict(lambda: 0) could have been used.

> However, it worked to replace collections.defaultdict with a class  
> overriding __getitem__ and __setitem__, so I guess that's how it works,  
> that in this case '+=' is simply translated like 'x += n' -> 'temp = x;  
> x = temp + n'.
>
> Is this a correct understanding, and if so, what exactly does the  
> documentation mean for the general case?

If x is a simple name, the only difference between x += n and x = x+n is  
the method invoked to perform the operation (__iadd__ vs __add__ in  
arbitrary objects). Both x and n are evaluated only once - and this is not  
an optimization, nor a shortcut, simply there is no need to do otherwise  
(please make sure you understand this).

 From the docs for the operator module: "Many operations have an “in-place”  
version. The following functions provide a more primitive access to  
in-place operators than the usual syntax does; for example, the statement  
x += y is equivalent to x = operator.iadd(x, y). Another way to put it is  
to say that z = operator.iadd(x, y) is equivalent to the compound  
statement z = x; z += y."
http://docs.python.org/library/operator.html

> foo()[bar()] += 1
>
> so here it's not translated like 'foo()[bar()] = foo()[bar()] + 1' but  
> evidently more like 'a = foo(); i = bar(); a.__setitem__(i,  
> a.__getitem__(i) + 1)'?

Yes, something like that.

> If so, is this behavior defined anywhere?

Isn't the description at  
http://docs.python.org/reference/simple_stmts.html#assignment-statements 
enough? It goes to some detail describing, e.g., what a.x = 1 means. The  
next section explains what a.x += 1 means in terms of the former case.

> I did find discussion (end of §6.2 of the language reference) of the  
> case where the target is an attibute reference, with this example:
>
> class A:
>      x = 3    # class variable
> a = A()
> a.x += 1     # writes a.x as 4 leaving A.x as 3

Do you want to discuss this example?

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

[Alf P. Steinbach]

* Gabriel Genellina:
> En Thu, 05 Nov 2009 21:23:27 -0300, Alf P. Steinbach <alfps@...>
> escribió:
>
[snip]
>  From the docs for the operator module: "Many operations have an
> “in-place” version. The following functions provide a more primitive
> access to in-place operators than the usual syntax does; for example,
> the statement x += y is equivalent to x = operator.iadd(x, y). Another
> way to put it is to say that z = operator.iadd(x, y) is equivalent to
> the compound statement z = x; z += y."
> http://docs.python.org/library/operator.html

Thanks!

>> foo()[bar()] += 1
>>
>> so here it's not translated like 'foo()[bar()] = foo()[bar()] + 1' but
>> evidently more like 'a = foo(); i = bar(); a.__setitem__(i,
>> a.__getitem__(i) + 1)'?
>
> Yes, something like that.
>
>> If so, is this behavior defined anywhere?
>
> Isn't the description at
> http://docs.python.org/reference/simple_stmts.html#assignment-statements 
> enough? It goes to some detail describing, e.g., what a.x = 1 means. The
> next section explains what a.x += 1 means in terms of the former case.

No, it wasn't exactly enough for me, coming most recently from a C++ background.

One reason was as mentioned that the C++ standard has essentially the /same
wording/ about "only evaluated once" but with a more strict meaning; in C++,
with the built-in += operator

    a()[foo()] += 1;

not only avoids calling a() and foo() twice, it also avoids doing the internal
indexing twice, while in python the internal indexing, locating that item, is
performed first in __getitem__ and then in __setitem__ (unless that is optimized
away at lower level by caching last access, but that in itself has overhead).

Another reason was that §6.2 does explicitly discuss attribute references as
targets, but not subscription as target. It would have been more clear to me if
all (four?) possible target forms were discussed. Happily you did now discuss
that in the part that I snipped above, but would've been nice, and easier for
for an other-language-thinking person :-), if it was in documentation.


>> I did find discussion (end of §6.2 of the language reference) of the
>> case where the target is an attibute reference, with this example:
>>
>> class A:
>>      x = 3    # class variable
>> a = A()
>> a.x += 1     # writes a.x as 4 leaving A.x as 3
>
> Do you want to discuss this example?

Thanks but no, it's OK, I understand it.


Cheers,

- Alf
--
http://mail.python.org/mailman/listinfo/python-list

[Gabriel Genellina-7]

En Fri, 06 Nov 2009 04:29:05 -0300, Alf P. Steinbach <alfps@...>  
escribió:

> * Gabriel Genellina:
>> En Thu, 05 Nov 2009 21:23:27 -0300, Alf P. Steinbach <alfps@...>  
>> escribió:
>>
>>> foo()[bar()] += 1
>>>
> One reason was as mentioned that the C++ standard has essentially the  
> /same wording/ about "only evaluated once" but with a more strict  
> meaning; in C++, with the built-in += operator
>
>     a()[foo()] += 1;
>
> not only avoids calling a() and foo() twice, it also avoids doing the  
> internal indexing twice, while in python the internal indexing, locating  
> that item, is performed first in __getitem__ and then in __setitem__  
> (unless that is optimized away at lower level by caching last access,  
> but that in itself has overhead).

Yes, that's a common misunderstanding in people coming from other  
languages with a different semantics for "assignment" and "variable".  
You're not alone :)

Python does not have "lvalues" as in C++. In the statement x=1, the left  
hand side does not denote an object, but a name. x.attr=1 and x[index]=1  
act more like a function call (they *are* function calls actually) than  
assignments.

> Another reason was that §6.2 does explicitly discuss attribute  
> references as targets, but not subscription as target. It would have  
> been more clear to me if all (four?) possible target forms were  
> discussed. Happily you did now discuss that in the part that I snipped  
> above, but would've been nice, and easier for for an  
> other-language-thinking person :-), if it was in documentation.

Yes, probably that section should be improved (except the final example  
added, the text hasn't changed since it was first written, more than 9  
years ago).

Reading reference material may be terribly boring, I admit. Most people  
read only the specific sections required to solve a specific problem; and  
some concepts that are introduced earlier in the book are missed or  
skipped.
If you haven't already done so, try at least to read these two sections  
 from the Language Reference: 3.1. Objects, values and types, and 4.1.  
Naming and binding. They define the most important concepts in Python; the  
rest are just details.

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

[John Posner]

Gabriel Genellina said:

>
>> Another reason was that §6.2 does explicitly discuss attribute
>> references as targets, but not subscription as target. It would have
>> been more clear to me if all (four?) possible target forms were
>> discussed. Happily you did now discuss that in the part that I
>> snipped above, but would've been nice, and easier for for an
>> other-language-thinking person :-), if it was in documentation.
>
> Yes, probably that section should be improved (except the final
> example added, the text hasn't changed since it was first written,
> more than 9 years ago).
>

FWIW ... following on a discussion in this forum in March of this year
[1], I submitted issue #5621 at bugs.python.org, along with a proposed
rewording of the documentation of this case:

   self.x = self.x + 1

I did *not* address the case in which the target is a subscription -- it
wasn't the focus of the forum discussion. IMHO, the subscription case is
much less subject to misunderstanding than the
instance-attribute-vs-class-attribute case. My change was accepted (with
some rearrangements made by Georg Brandl), but has not yet been included
in any Py2 or Py3 release, AFAIK.

-John

[1] http://mail.python.org/pipermail/python-list/2009-March/175054.html

--
http://mail.python.org/mailman/listinfo/python-list

[Gabriel Genellina-7]

En Fri, 06 Nov 2009 12:44:38 -0300, John Posner <jjposner@...>  
escribió:
> Gabriel Genellina said:

>> Yes, probably that section should be improved (except the final example  
>> added, the text hasn't changed since it was first written, more than 9  
>> years ago).

> FWIW ... following on a discussion in this forum in March of this year I  
> submitted issue #5621 at bugs.python.org [...]
> My change was accepted (with some rearrangements made by Georg Brandl),  
> but has not yet been included in any Py2 or Py3 release, AFAIK.

That's strange. Your modified text appears in the online version of the  
documentation for 2.6.4:
http://www.python.org/doc/2.6.4/reference/simple_stmts.html#assignment-statements
but not in the source package nor the .chm file included in the Windows  
binaries for the same release.
Same with 3.1.1

Looks like an error in the build process.

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list