Information as lingua franca

View: New views
4 Messages — Rating Filter:   Alert me  

Information as lingua franca

by Russ Abbott :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Mikhail,

You suggested:

One could attempt to measure mutual information I(X;Y) between *gene sequences* X and *functional behaviours* Y - without specifying or even assuming a precise relationship between f and Y ["f" and "between" switched. RA]. It could be said that genes (or the proteins they produce) encode *information* RATHER THAN specific regularities about the environment!

Would you be willing to elaborate on this? For example, if you apply this to software, would this imply that you can measure the mutual information between the code and the function it computes? Can you work out a simple example for me?  One of my favorites is the idiom that exchanges the values in two variables x and y.

temp = x;
x = y;
y = temp;

How would you express the mutual information in the code (above) vs. the function (the values in x and y were exchanged)?

-- Russ

On Nov 24, 2007 11:00 PM, <Mikhail.Prokopenko@...> wrote:
Hi Russ and all,
 
I agree, of course, that genotype-phenotype relationship is not a simple X = f(Y) relationship :)  And emergence plays a role in the process: I completely agree that "there is probably significantly more to be said than that genes (or the proteins they produce) encode specific regularities about the environment." 
 
Russ argued that
---------
Nature typically works that way; it builds new stuff on top of existing stuff.  So if one has a genetic mechanism that results in some functionality, that functionality may be incorporated into some other functionality if the new functionality turns out to be useful.

This view would propose to look at biological mechanisms as increasingly complex levels of abstraction. Some of those levels of abstraction produce functionality in the world. Others produce internal functionality. The overall effect result is that beings with certain functionalities survive better in certain environments. 
---------
 
This may be a good representation. If we are to learn from Nature (and I don't mean bio-mimetics) we have to start somewhere. Information Theory may be one good tool that transcends simple X = f(Y) relationships (read: simple correlations), and captures the [probably nested] relationships leading to functionality. For example, one could attempt to measure mutual information I(X;Y) between *gene sequences* X and *functional behaviours* Y - without specifying or even assuming a precise relationship f between and Y. It could be said that genes (or the proteins they produce) encode *information* RATHER THAN specific regularities about the environment! 
 
If we assume that a) environment has a "statistical structure", b) agents' genome has a "statistical structure", c) agent's dynamics/functionality in the environment have "statistical structure", then What's the information dynamics among these (and other conceivable) abstraction levels, is a valid question, I think.

In short, the view you advocated (increasingly complex levels of abstraction), as far as I understand, is not at odds with IDSO.
 
I'm curious if "neutral emergence" view (Andrew Weeks, Susan Stepney, Fiona Polack)
is related to this thread?  Susan, would you like to comment on this?
 
Cheers,
Mikhail
P.S. I'm still using IDSO but happy to switch to "information-mediated" rather than "information-driven" at any time when we reach a consensus :)
 


From: Abbott, Russ
Sent: Saturday, 24 November 2007 6:25 AM
To: Prokopenko, Mikhail (ICT Centre, North Ryde)
Cc: IDSO-CSIRO
Subject: Re: [IDSO] strong IDSO

Thanks, I'll have a look at the paper. 

Regarding your question:

If, say, y is a gene, and h(y) is the protein that corresponds to some regularity in nature ... then wouldn't one be able to say that gene y itself also corresponds to this regularity?

It seems to me that neither the gene nor the protein typically corresponds to some regularity in nature. In my original message I should have said "is advantageous given some regularity in nature" instead of "corresponds to some regularity in nature." 

Pardon the following fairly long discussion.

The example I used was skin color.  I freely admit to not being an expert in the field. The Wikipedia article (and I don't claim that Wikipedia is always a reliable source) paints a fairly complex picture of the relationship between genes, skin color, latitude, etc. One of the basic mechanisms (it claims) is that like chimpanzees, humans had light skin under our fur. When we lost our fur, dark skin became essential in those parts of the world where the sun was brightest as a way to protect against skin cancer. The genetic configuration for dark skin (lots of melanin) is apparently fairly specific. Any mutation results in significantly lighter skin, which tends to be disadvantageous for survival in the hot sun of Africa. According to this theory, the environment selected for dark skin, not light skin. When that environmental pressure wasn't present, mutation wasn't so disadvantageous, and lighter skin survived.  In addition, lighter skin is advantageous for the production of vitamin D.

So the cartoon version of this is that there is a gene for skin color.  One specific DNA sequence produces dark skin; all other versions produce lighter skin. Dark skin is advantageous where the sun is hot and strong (because of the skin cancer connection). Light skin is advantageous where the sun is weak (because of the vitamin D connection). Both of these connections are correlated with the strength of the sun. But neither of them is directly connected. That is, dark skin with weak sun is OK if you have another source of vitamin D (as the article says the Inuit do). Light skin with strong sun is OK if you have lots of fur (as our primate ancestors did).

So I think it's a stretch to say that either a gene or the protein it produces records information about the environment. It's much more complex than that.

And even if it were more directly connected (without the fur or the other sources of vitamin D), the gene controls the production of melanin. Assume that's all it does.  (It's probably even more complex than that.) The amount of melanin has the effect of making the skin "lighter" or "darker." But why does that matter? It matters because that affects how the skin will respond to sunlight. But isn't that backwards? Light colors reflect sunlight, and dark colors accept it. In this case, though melanin acts as a sun (UV) block rather than as a reflective color. So the more melanin, the less sun.  But:
  • Do we care how the specific configuration of nucleic acids produces lots of melanin and other configurations produce less? Or is that a level of abstraction that we can ignore?
  • Do we care about the mechanism whereby melanin acts as a sun block? Or is that another level of abstraction that we can ignore? 
  • And finally, do we care about what happens when UV rays are blocked or not blocked , i.e., the biological mechanisms that produce skin cancer and vitamin D? Or is that still  another level of abstraction we can ignore? 
As you know, I'm all in favor of understanding nature in terms of emergent phenomena.  So some of these mechanisms may be subsumed under the mantle of emergence, and we may be able to talk about them as levels of abstraction.  But even so, it seems to me that there is probably significantly more to be said than that genes (or the proteins they produce) encode specific regularities about the environment.

Having gone this far, I would say that it probably makes more sense to understand genes and the proteins they generate as part of a larger collection of mechanisms that working together produce useful (or not-so-useful) results.  Nature typically works that way; it builds new stuff on top of existing stuff.  So if one has a genetic mechanism that results in some functionality, that functionality may be incorporated into some other functionality if the new functionality turns out to be useful.

This view would propose to look at biological mechanisms as increasingly complex levels of abstraction. Some of those levels of abstraction produce functionality in the world. Others produce internal functionality. The overall effect result is that beings with certain functionalities survive better in certain environments.

For example, being able to extract oxygen from water is good if you live in the water, and being able to use oxygen in the atmosphere is good if you live in an atmosphere with oxygen. It's not so much that genes encode information about the environment; it's that genes provide certain functionalities, which are useful in certain environments.

Again, apologies for the length of this post. I got carried away.

-- Russ

On Nov 23, 2007 12:21 AM, <Mikhail.Prokopenko@...> wrote:
Hi Russ and all,
 
The paper I mentioned earlier
    M. Piraveenan, D. Polani, M. Prokopenko. Emergence of Genetic Coding: an Information-theoretic Model, in F. Almeida e Costa, L. M. Rocha, E. Costa, I. Harvey, A. Coutinho (eds). Advances in Artificial Life: 9th European Conference on Artificial Life (ECAL-2007), Lisbon, Portugal, September 10-14, Lecture Notes in Artificial Intelligence, vol. 4648, pp. 42-52, Springer, Berlin, 2007.
 
cab be d/loaded from

I should clarify that this paper is not intended as a statement on IDSO at all, but attempts to crudely model possible emergence of proto-symbols (rudimentary genes) that capture dynamics of an object (a rudimentary cell) within its environment.
 
Russ, you make an interesting distinction that "the genes themselves generally don't encode those regularities", but rather "just generate proteins that result in light skin, which correspond to certain facts about nature where those genes evolved". If, say, y is a gene, and h(y) is the protein that corresponds to some regularity in nature - let's ignore the trivialisation of the functional form h(.) for the moment - then wouldn't one be able to say that gene y itself also corresponds to this regularity?  ...and maybe even encodes it in some way?
 
Thanks,
Mikhail
 


From: Abbott, Russ
Sent: Thursday, 22 November 2007 3:41 PM
To: Boschetti, Fabio (CMAR, Floreat)
Cc: Prokopenko, Mikhail (ICT Centre, North Ryde); Lafusa, Antonio; IDSO-CSIRO
Subject: Re: starting IDSO discussions: weak and strong IDSO

Hi,

I can't resist jumping in here.

Mikhail's original statement was, I gather, in support, for example, of Adami's claim that "the evolutionary process extracts valuable information and stores it in the genes." That seems quite true. Genes do seem to do that sort of thing. That claim, though doesn't say that everything that happens in nature (or in man-made systems like sodaplay) involves the explicit encoding and passing of information.

Furthermore, it seems that even strong IDSO is an example of weak IDSO in the sense that genes do not store information about the world. Yes, one can say that certain genes evolved to correspond to certain regularities in nature. But the genes themselves generally don't encode those regularities. Genes for, say, light skin that evolved in higher latitudes don't say anything about the length of the day or the strength of the sun. They just generate proteins that result in light skin, which correspond to certain facts about nature where those genes evolved.

So I'd like to request that we attempt to clarify a bit what the issue is that's being discussed.

-- Russ



Parent Message unknown RE: Information as lingua franca

by Mikhail.Prokopenko :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi,
 
Sorry I hasn't replied sooner (too busy lately). I'll try to answer some of the questions, but first, let me mention again that it's important to identify the channel(s) over which information (flow) is to be maximised. For example, Adami argues that one channel may be between "genes" X and "environment" Z - hence I(X:Z) as a measure. In another example (Piraveenan et al. paper) we considered what happens to possible "genes" X when a coupled "phenotype" Y is affected by noise and shifts into Y' - hence, I(Y;Y') as a measure driving/mediating emergence of X.
 
In the following I assume that we are simply interested in how to compute I(X;Y) in the software example.
 
Any two code fragments can be same - it doesn't really matter. Although if all fragments are the same, the entropy H(X) would be zero. The better n fragments represent the total collection, the better is the analysis, but it's not necessary to represent the totality. Adami, for example, used the sequence length (e.g., if we deal with assembler code, that would be the max length of a program) to approximate max entropy, H_max(X). The same would apply to "results". If the "results" depend on the context, we need another variable (Z), and that's what "environment" in Adami's paper is, I think. It's hard to enumerate environments/contexts - Adami's paper shows one way to work around this.
 
Moving to the questions on how to compute entropies. These are mostly technical questions, and can be answered in many ways. One way is to use frequencies (how often fragments occur, etc.). Another way (parametric) is to assume a specific distribution, say Gaussian, compute entropies analytically, and estimate parameters from data. One may also use kernels. The same techniques apply to "behaviours/results" if the space is defined. We used a dynamical system to represent the "phenotype" level. H(Y) is computed independently of X, but to compute joint entropy, you'd need to consider pairs: which fragment results in which behaviour, and how often each pair occurs among all pairs, etc. In short, it comes down to enumerating possibilities (if one doesn't use parametric approaches) and building histograms :)
 
I'm not sure if we want to get into a technical discussion on how to compute entropies in this list - there are more interesting questions, I think, like the one Antonio asked about flows between levels, or the questions on explicit communication vs stigmergy, or the co-evolution of perception-action loops, related to your recent post on squirrels-and-rattlesnakes. 
 
Thanks,
Mikhail
 
 


From: Abbott, Russ
Sent: Tuesday, 27 November 2007 6:43 AM
To: Prokopenko, Mikhail (ICT Centre, North Ryde)
Cc: IDSO-CSIRO
Subject: Re: Information as lingua franca

So let's say we had n code fragments. Each one has a corresponding result when executed ("expressed").  Is it possible for two of the code fragments to be the same, or must they all be different from each other. Same question about the results. I'm assuming that the n code fragments must represent to total collection of code fragments. Is that true?  That is, we can't be interested in a subset of the code fragments. Or can we?  Same question about the results.  Although since the results depend on context, it's harder to say what the total collection of results are.  That is, a code fragment may have different results when executed (expressed) in different contexts.

However the preceding questions are answered, how would you go about computing H(X), H(Y), and H(X, Y)?  Do you need to know how often each of the code fragments appears in the overall population to compute H(X)? Do you need to know how often each of the results occurs in the lifetime of the population elements to compute H(Y).  To put this question another way, do you need to know how often each of the code fragments is expressed (executed) and thereby produces its result to compute H(Y)? How much of the above do you need to know to compute H(X, Y) and how is it computed?

As you can see, I'm not at all clear about the overall idea and am hoping for a simple concrete example to help me understand. I know I should probably read the papers, but you know how those things are. If I can get a shortcut intuitive explanation by reading my email, that's so much easier!

Thanks.

-- Russ

On Nov 26, 2007 12:29 AM, <Mikhail.Prokopenko@...> wrote:
 
OK. To compute I(X;Y), we'd need more examples of X - other code fragments, and corresponding examples of Y - functional outcomes. Then, having the samples (x1, x2, ..., xn) and (y1, y2, ..., yn), we can compute entropies H(X), H(Y) and H(X,Y).
 
Having one example of X - the three lines of code  x1 - does not allow to compute the entropy H(X). Adami's paper, on the other hand, does consider the software world with code fragments, etc. For example, if the line "x = y;" in your example is mutated away in a fragment x2, the functionality y2 would change, affecting entropies and mutual information. The way these quantities are affected depends on the entire population (the samples).
 
So the main point is that we consider statistical structure of the code-fragment space, the outcome space, and so on.
 
Mikhail
 


From: Abbott, Russ
Sent: Monday, 26 November 2007 7:14 PM

To: Prokopenko, Mikhail (ICT Centre, North Ryde)
Cc: IDSO-CSIRO
Subject: Re: Information as lingua franca

I was hoping my variable swapping 3-liner would be a simple concrete example for which you could illustrate how mutual information worked.

I was thinking of the program as the gene sequence (or the proteins they generate) and the function of reversing the variables as the function that the proteins perform.  So the three lines of code is the X (the gene sequence) and the reversal of the values in the variables is the Y (the functional behaviour ). How would you work out I(X:Y) in this case?

-- Russ

On Nov 25, 2007 11:47 PM, <Mikhail.Prokopenko@...> wrote:
Hi Russ,
 
The sentences in question have an "X" missing (sorry), and should read:
 
"one could attempt to measure mutual information I(X;Y) between *gene sequences* X and *functional behaviours* Y - without specifying or even assuming a precise relationship f between X ["X" was missing. MP] and Y. It could be said that genes (or the proteins they produce) encode *information* RATHER THAN specific regularities about the environment!"
 
I simply mean a black box f between input X and output Y. If we apply it to software, then different X's may, for instance, be different initial conditions to a problem-solver f, producing different solutions Y. Then, even if the relationship Y = f(X) is not precise - e.g., there is some random noise in f, we still can compute mutual information I(X;Y). That is, mutual information between initial conditions and solutions - not between initial conditions and the problem-solver.
 
A more involved software analogy is a process by which different queries/keys K to a database/memory X produce, as a result of some mapping f, different outcomes Y. In this case, one may want to analyse conditional mutual information I(Y;K|X) - how much a key is necessary to identify the output of the mapping, given the memory. In this crude analogy, K mimics "proteins" accessing the "DNA" X, and producing the "functionalities" Y.
 
I don't think I understand what are X, Y and f in your example, i.e. what are the respective analogies of genes, functionalities they produce, and the process that links them.
 
Not sure if the following is related to your question on the function that exchanges the values of two variables X and Y [please ignore it if it's not related]:
 
    There are some ways to compute information flow (not mutual information) between, say, temporal strings X = 0,1,0,1,0,1,.... and Y = 1,0,1,0,1,0 - dependent on whether there is a function f that connects them. For example, (causal) information flow would be 1 bit if the function f swaps x_i and y_i at every step i, yet the flow would be 0 if X and Y are independent of each other and each simply inverts previous value. Note that the results differ - but observationally, these two cases are equivalent - and neither mutual information nor transfer entropy would detect the difference. This is an example of interventionist (causal) approach:
    Ay, N., and Polani, D., (2007). Information Flows in Causal Networks. Advances in Complex Systems. Accepted; also Santa Fe Institute Working Paper 06-05-014
 
 
Mikhail
 


From: Abbott, Russ
Sent: Monday, 26 November 2007 5:09 AM

To: Prokopenko, Mikhail (ICT Centre, North Ryde)
Cc: IDSO-CSIRO
Subject: Information as lingua franca

Hi Mikhail,

You suggested:

One could attempt to measure mutual information I(X;Y) between *gene sequences* X and *functional behaviours* Y - without specifying or even assuming a precise relationship between f and Y ["f" and "between" switched. RA]. It could be said that genes (or the proteins they produce) encode *information* RATHER THAN specific regularities about the environment!

Would you be willing to elaborate on this? For example, if you apply this to software, would this imply that you can measure the mutual information between the code and the function it computes? Can you work out a simple example for me?  One of my favorites is the idiom that exchanges the values in two variables x and y.

temp = x;
x = y;
y = temp;

How would you express the mutual information in the code (above) vs. the function (the values in x and y were exchanged)?

-- Russ

On Nov 24, 2007 11:00 PM, <Mikhail.Prokopenko@...> wrote:
Hi Russ and all,
 
I agree, of course, that genotype-phenotype relationship is not a simple X = f(Y) relationship :)  And emergence plays a role in the process: I completely agree that "there is probably significantly more to be said than that genes (or the proteins they produce) encode specific regularities about the environment." 
 
Russ argued that
---------
Nature typically works that way; it builds new stuff on top of existing stuff.  So if one has a genetic mechanism that results in some functionality, that functionality may be incorporated into some other functionality if the new functionality turns out to be useful.

This view would propose to look at biological mechanisms as increasingly complex levels of abstraction. Some of those levels of abstraction produce functionality in the world. Others produce internal functionality. The overall effect result is that beings with certain functionalities survive better in certain environments. 
---------
 
This may be a good representation. If we are to learn from Nature (and I don't mean bio-mimetics) we have to start somewhere. Information Theory may be one good tool that transcends simple X = f(Y) relationships (read: simple correlations), and captures the [probably nested] relationships leading to functionality. For example, one could attempt to measure mutual information I(X;Y) between *gene sequences* X and *functional behaviours* Y - without specifying or even assuming a precise relationship f between and Y. It could be said that genes (or the proteins they produce) encode *information* RATHER THAN specific regularities about the environment! 
 
If we assume that a) environment has a "statistical structure", b) agents' genome has a "statistical structure", c) agent's dynamics/functionality in the environment have "statistical structure", then What's the information dynamics among these (and other conceivable) abstraction levels, is a valid question, I think.

In short, the view you advocated (increasingly complex levels of abstraction), as far as I understand, is not at odds with IDSO.
 
I'm curious if "neutral emergence" view (Andrew Weeks, Susan Stepney, Fiona Polack)
is related to this thread?  Susan, would you like to comment on this?
 
Cheers,
Mikhail
P.S. I'm still using IDSO but happy to switch to "information-mediated" rather than "information-driven" at any time when we reach a consensus :)
 


From: Abbott, Russ
Sent: Saturday, 24 November 2007 6:25 AM
To: Prokopenko, Mikhail (ICT Centre, North Ryde)
Cc: IDSO-CSIRO
Subject: Re: [IDSO] strong IDSO

Thanks, I'll have a look at the paper. 

Regarding your question:

If, say, y is a gene, and h(y) is the protein that corresponds to some regularity in nature ... then wouldn't one be able to say that gene y itself also corresponds to this regularity?

It seems to me that neither the gene nor the protein typically corresponds to some regularity in nature. In my original message I should have said "is advantageous given some regularity in nature" instead of "corresponds to some regularity in nature." 

Pardon the following fairly long discussion.

The example I used was skin color.  I freely admit to not being an expert in the field. The Wikipedia article (and I don't claim that Wikipedia is always a reliable source) paints a fairly complex picture of the relationship between genes, skin color, latitude, etc. One of the basic mechanisms (it claims) is that like chimpanzees, humans had light skin under our fur. When we lost our fur, dark skin became essential in those parts of the world where the sun was brightest as a way to protect against skin cancer. The genetic configuration for dark skin (lots of melanin) is apparently fairly specific. Any mutation results in significantly lighter skin, which tends to be disadvantageous for survival in the hot sun of Africa. According to this theory, the environment selected for dark skin, not light skin. When that environmental pressure wasn't present, mutation wasn't so disadvantageous, and lighter skin survived.  In addition, lighter skin is advantageous for the production of vitamin D.

So the cartoon version of this is that there is a gene for skin color.  One specific DNA sequence produces dark skin; all other versions produce lighter skin. Dark skin is advantageous where the sun is hot and strong (because of the skin cancer connection). Light skin is advantageous where the sun is weak (because of the vitamin D connection). Both of these connections are correlated with the strength of the sun. But neither of them is directly connected. That is, dark skin with weak sun is OK if you have another source of vitamin D (as the article says the Inuit do). Light skin with strong sun is OK if you have lots of fur (as our primate ancestors did).

So I think it's a stretch to say that either a gene or the protein it produces records information about the environment. It's much more complex than that.

And even if it were more directly connected (without the fur or the other sources of vitamin D), the gene controls the production of melanin. Assume that's all it does.  (It's probably even more complex than that.) The amount of melanin has the effect of making the skin "lighter" or "darker." But why does that matter? It matters because that affects how the skin will respond to sunlight. But isn't that backwards? Light colors reflect sunlight, and dark colors accept it. In this case, though melanin acts as a sun (UV) block rather than as a reflective color. So the more melanin, the less sun.  But:
  • Do we care how the specific configuration of nucleic acids produces lots of melanin and other configurations produce less? Or is that a level of abstraction that we can ignore?
  • Do we care about the mechanism whereby melanin acts as a sun block? Or is that another level of abstraction that we can ignore? 
  • And finally, do we care about what happens when UV rays are blocked or not blocked , i.e., the biological mechanisms that produce skin cancer and vitamin D? Or is that still  another level of abstraction we can ignore? 
As you know, I'm all in favor of understanding nature in terms of emergent phenomena.  So some of these mechanisms may be subsumed under the mantle of emergence, and we may be able to talk about them as levels of abstraction.  But even so, it seems to me that there is probably significantly more to be said than that genes (or the proteins they produce) encode specific regularities about the environment.

Having gone this far, I would say that it probably makes more sense to understand genes and the proteins they generate as part of a larger collection of mechanisms that working together produce useful (or not-so-useful) results.  Nature typically works that way; it builds new stuff on top of existing stuff.  So if one has a genetic mechanism that results in some functionality, that functionality may be incorporated into some other functionality if the new functionality turns out to be useful.

This view would propose to look at biological mechanisms as increasingly complex levels of abstraction. Some of those levels of abstraction produce functionality in the world. Others produce internal functionality. The overall effect result is that beings with certain functionalities survive better in certain environments.

For example, being able to extract oxygen from water is good if you live in the water, and being able to use oxygen in the atmosphere is good if you live in an atmosphere with oxygen. It's not so much that genes encode information about the environment; it's that genes provide certain functionalities, which are useful in certain environments.

Again, apologies for the length of this post. I got carried away.

-- Russ

On Nov 23, 2007 12:21 AM, <Mikhail.Prokopenko@...> wrote:
Hi Russ and all,
 
The paper I mentioned earlier
    M. Piraveenan, D. Polani, M. Prokopenko. Emergence of Genetic Coding: an Information-theoretic Model, in F. Almeida e Costa, L. M. Rocha, E. Costa, I. Harvey, A. Coutinho (eds). Advances in Artificial Life: 9th European Conference on Artificial Life (ECAL-2007), Lisbon, Portugal, September 10-14, Lecture Notes in Artificial Intelligence, vol. 4648, pp. 42-52, Springer, Berlin, 2007.
 
cab be d/loaded from

I should clarify that this paper is not intended as a statement on IDSO at all, but attempts to crudely model possible emergence of proto-symbols (rudimentary genes) that capture dynamics of an object (a rudimentary cell) within its environment.
 
Russ, you make an interesting distinction that "the genes themselves generally don't encode those regularities", but rather "just generate proteins that result in light skin, which correspond to certain facts about nature where those genes evolved". If, say, y is a gene, and h(y) is the protein that corresponds to some regularity in nature - let's ignore the trivialisation of the functional form h(.) for the moment - then wouldn't one be able to say that gene y itself also corresponds to this regularity?  ...and maybe even encodes it in some way?
 
Thanks,
Mikhail
 


From: Abbott, Russ
Sent: Thursday, 22 November 2007 3:41 PM
To: Boschetti, Fabio (CMAR, Floreat)
Cc: Prokopenko, Mikhail (ICT Centre, North Ryde); Lafusa, Antonio; IDSO-CSIRO
Subject: Re: starting IDSO discussions: weak and strong IDSO

Hi,

I can't resist jumping in here.

Mikhail's original statement was, I gather, in support, for example, of Adami's claim that "the evolutionary process extracts valuable information and stores it in the genes." That seems quite true. Genes do seem to do that sort of thing. That claim, though doesn't say that everything that happens in nature (or in man-made systems like sodaplay) involves the explicit encoding and passing of information.

Furthermore, it seems that even strong IDSO is an example of weak IDSO in the sense that genes do not store information about the world. Yes, one can say that certain genes evolved to correspond to certain regularities in nature. But the genes themselves generally don't encode those regularities. Genes for, say, light skin that evolved in higher latitudes don't say anything about the length of the day or the strength of the sun. They just generate proteins that result in light skin, which correspond to certain facts about nature where those genes evolved.

So I'd like to request that we attempt to clarify a bit what the issue is that's being discussed.

-- Russ





--
-- Russ Abbott
_____________________________________________
Professor, Computer Science
California State University, Los Angeles
o Check out my blog at http://russabbott.blogspot.com/



--
-- Russ Abbott
_____________________________________________
Professor, Computer Science
California State University, Los Angeles
o Check out my blog at http://russabbott.blogspot.com/

RE: Information as lingua franca

by Stanley N. Salthe :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Mikhail said:

>I'm not sure if we want to get into a technical  discussion on how to
>compute entropies in this list - there are more interesting  questions, I
>think,
     S: Entropies generally are uncertainties. But in natural situations
and occasions we can't know what all the uncertainties are.  Acquiring, via
inquiry, some certainties about a system will not reduce the overall amount
of uncertainty about it, and, indeed, the process of inquiring will itself
generate more uncertainties to propagate through the system, likely
disturbing our efforts.

like the one Antonio asked about flows between levels,
     S:  This depends upon how we construct the  levels.  In a scale
(compositional)hierarchy (e.g., [population [organism [cell]]], no
information passes unmediated across levels because they are dynamically
screened off from each other by rates of change (as mentioned in a previous
post).  In a specification (subsumption) hierarchy (e.g., {physical
dynamics {material connectivities {organismic activity}}}), information
flows freely between levels, and, in particular, regulation (integration)
from the higher levels directy affects the conformations of lower level
entities.

 or  the questions on explicit communication vs stigmergy, or the
co-evolution of  perception-action loops, related to your recent post on
squirrels-and-rattlesnakes.
     S: In this case mutual information between these systems has evolved
because of their 'interest' in each other (so they make up a supersystem).
But here too, squirrels will not 'know' anything about, e.g., where snakes
nest, and snakes will not know, e.g., what squirrels do in the treetops.
Information is always partial, except in scientific models, where
microsystemic details of statistical moments are unimportant, and which are
not formulated using fuzzy logic.

STAN

    Thanks, Mikhail    






Re: Information as lingua franca

by High Performance Coder :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Fri, Nov 30, 2007 at 08:26:14PM +1100, Mikhail.Prokopenko@... wrote:

>  
> I'm not sure if we want to get into a technical discussion on how to
> compute entropies in this list - there are more interesting questions, I
> think, like the one Antonio asked about flows between levels, or the
> questions on explicit communication vs stigmergy, or the co-evolution of
> perception-action loops, related to your recent post on
> squirrels-and-rattlesnakes.
>  
> Thanks,
> Mikhail
>  

It is probably useful to have a few simple examples of mutual
information calculation to help spice up the intutition of those
unfamiliar with the concept.

Has anyone checked out Wikipedia's article on the topic? There are
often clearer explanations of the concept there than something any of
us could come up with on the spur of the moment in the midst of our
busy lives.

Cheers

--

----------------------------------------------------------------------------
A/Prof Russell Standish                  Phone 0425 253119 (mobile)
Mathematics                        
UNSW SYDNEY 2052                 hpcoder@...
Australia                                http://www.hpcoders.com.au
----------------------------------------------------------------------------