Complexity-based permutation entropy

Abstract: Entropy is a useful concept in many areas of physics and applied mathematics, primarily thermodynamics (where it originated), statistical mechanics, information theory, dynamical systems and data analysis. In the first part of our talk, we will review (i) the Shannon entropy, (ii) the permutation entropy, which is the Shannon entropy in the symbolic representation of real-valued time series via permutations, and (iii) the axiomatic definition of the entropy of probability distributions, which leads to the weaker concepts of generalized entropy, compossable entropy and group entropy.

In the second part, we will focus on the permutation entropy. Although this entropy has proven to be useful in data analysis, its theoretical aspects have remained limited to noiseless deterministic series (i.e., generated by dynamical systems), the main obstacle being the super-exponential growth of visible permutations with length when randomness (also in form of observational noise) is present in the data. To overcome this shortcoming, we take a new approach through complexity classes, which are defined by the asymptotic growth of visible permutations with length. Thus, deterministic processes belong to the exponential class, while usual noisy processes belong to the factorial class. For the processes of each possible class, we will construct a group entropy that is finite and coincides with the conventional permutation entropy on the exponential class. This construction is completely general and can be applied to other situations.