Information Flows? A Critique of Transfer Entropies

Ryan G. James, Nix Barnett, and James P. Crutchfield

Phys. Rev. Lett. 116, 238701 – Published 9 June 2016

Abstract

A central task in analyzing complex dynamics is to determine the loci of information storage and the communication topology of information flows within a system. Over the last decade and a half, diagnostics for the latter have come to be dominated by the transfer entropy. Via straightforward examples, we show that it and a derivative quantity, the causation entropy, do not, in fact, quantify the flow of information. At one and the same time they can overestimate flow or underestimate influence. We isolate why this is the case and propose several avenues to alternate measures for information flow. We also address an auxiliary consequence: The proliferation of networks as a now-common theoretical model for large-scale systems, in concert with the use of transferlike entropies, has shoehorned dyadic relationships into our structural interpretation of the organization and behavior of complex systems. This interpretation thus fails to include the effects of polyadic dependencies. The net result is that much of the sophisticated organization of complex systems may go undetected.

Received 23 December 2015

DOI:https://doi.org/10.1103/PhysRevLett.116.238701

Physics Subject Headings (PhySH)

Entropy Network flow

Information theory

Networks

Authors & Affiliations

Ryan G. James^1,2,†, Nix Barnett^1,3,‡, and James P. Crutchfield^1,2,3,*

¹Complexity Sciences Center, University of California at Davis, One Shields Avenue, Davis, California 95616, USA
²Physics Department, University of California at Davis, One Shields Avenue, Davis, California 95616, USA
³Mathematics Department, University of California at Davis, One Shields Avenue, Davis, California 95616, USA