1. Introduction

Statistical entropy is a probabilistic measure of uncertainty, or ignorance about data. Information is a measure of a reduction in that uncertainty. But must be avoided according to Frank Lambert[39]the interpretation of entropy as disorder. So, instead proposing the notion as “dispersal of energy”.

We define I in terms of the probability p, by these

Properties of Information Measure, I:

1) I (p) ≥ 0; i.e. information content will be a non-negative quantity.

2) I (1) = 0; i.e. if an event has probability 1, we get no information from the occurrence of the event.

3) If two independent events occur, the information we get from observing the events is the sum of both information.

4) Information measure must be continuous, and also a monotonic function of the probability: slight changes in probability should result in slight changes in I.

From an advanced mathematical viewpoint[37,38,44], Kaufmann (1975) introduced the so-called “index of fuzziness” as a normal distance, using the functional form of Shannon´s entropy to define a discrete fuzzy set, A. Then, the Kaufmann´s entropy of A will be

Also must be mentioned[24-27] the very remarkable Iosifescu-Theodorescu Entropy (denoted IT-entropy, 1961), which, for instance, when it is defined on a stationary and homogeneous Markov Chain.For instance, we can obtainthe IT- entropy as a measure of the mobility for social processes as multiple Markov Chains. It would be justified, because the IT-entropy of a Markov Chain reaches a maximum when X(r) is independent of all their precedent states.

Yager (1979), and Higashi and Klir (1983), showed the entropy measure as the difference between two fuzzy sets. More concretely, it will be the difference between a fuzzy set and its complement, which is also a fuzzy set.

Pal, N. R., and Pal, S. K. (1989) introduce a definition for probabilistic entropy, in this case based on an exponential gainfunction, and used it as basis for defining this measure of fuzziness.

Bhandari and Pal (1993) use Renyi´s entropy to define a non-probabilistic entropy, or measure of fuzziness, for fuzzy sets.

2. Entropy

Given a discrete random variable, X, with an associated probability distribution, P (x), we will define[17,18] the Entropy of X by

H (x) = - p(x) ln p(x) =

= p(x) ln (1/p(x)) = E [ln (1/p(x)]

Therefore, such H is a measure of the quantity of information that we receive, when it is sent towards us.

The logarithmic base will be arbitrary. If b = 2, it is measured in bits. The more logical election, because the system of interest, is binary. If b = 10, will be in dits. And if b = e, will be in nats.

The Shannon Entropy is a measure of the average information content one is missing when one does not know the value of the random variable. These concepts proceed from the Shannon´s 1948 very famous paper and their derivations[46,47,49], i.e. the so-called “Mathematical Theory of Communication”. So, it represents an absolute limit on the best possible lossless compression of any communication, under certain constraints, treating messages to be encoded as a sequence of independent and identically distributed (i.i.d.) random variables. The information that we receive from an observation is equal to the degree to which uncertainty is reduced. So,

I = H (before) – H (after)

Among their main properties, we have the Continuity. The measure H should be continuous, in the sense that changing the values of the probabilities by a verysmall amount, should only change the H value by a small amount.

Maximality: The measure H will be maximal, if all the outcomes are equally likely, i.e. the Uncertainty is highest when all the possible events are equiprobable,

H_n (p₁, p₂, …, p_n) ≤ H_n (1/n, 1/n, …, 1/n)

And H will increase with the number of outcomes

H_n (1/n, 1/n, …, 1/n) < H_n+1 (1/n+1, 1/n+1, …, 1/n+1)

Additivity: The amount of entropy should be independent of how the process is considered, as being divided into parts. Such functional relationship characterizes the entropy of a system respect to their sub-systems. It demands that the entropy of every system can be identified, and then, computed from the entropies of theirsub-systems.

The notion of Intuitionistic Fuzzy Set (in acronym, IFS) was introduced by Atanassov (1983), and then developed by authors as Hung & Yang, among others. Recall that an Intuitionistic Set is an incompletely known set[3-5].

An IFS must represent the degrees of their membership and non-membership, with a degree of hesitancy. For this reason, they have widely used in many applied areas.

A very frequent measure of fuzziness is the entropy of Fuzzy Sets, which were first mentioned by Zadeh (1965). But recently two new definitions have been proposed by Szmidt&Kacprzyk (2001), and by Burillo&Bustince(1996). The first one is a non-probabilistic entropy measure, which departs on a geometric interpretation of an IFS. And by the second, it would be possible to measure the degree of intuitionism of an IFS.

Also we can to generalize from IFS to a Neutrosophic Set, abridgedly N-Set, concept due to Smarandache (1995). It is a set which generalizes[3-5] many existing classes of sets. In particular by the so-denoted, FS, and its first generalization, IFS.

Let U be a universe of discourse, and M a set included in U. An element x from U is noted, with respect to the set M, as x (T, I, F). And it belongs to M in the following way: it is T % in the set (membership degree), I % indeterminate (unknown if it is in the set), and F % not in the set (non-membership degree). Here T, I, F components are real standard/non- standard subsets, included in the non-standard unit interval,

]^-0, 1⁺[representing truth, indeterminacy and falsity percentages, respectively.

It is possible to define a measure of Neutrosophic H, or N-Entropy, as the summation of the respective entropies of three subsets, T, I and F, by

H_NS = H_T + H_I + H_F

The maximum entropy principle (expressed asMEP,by acronym) is a postulate about a universal feature of any probability assignment on a given set of propositions: events, hypotheses, etc.

Let some testable information about a probability distribution function be given. Consider the set of all trial probability distributions that encode this information. Then, the probability distribution that maximizes the information entropy is the true probability distribution with respect to the testable information prescribed.

E. T. Jaynessuggested[28-32] that the Entropy, denoted by S in Statistical Mechanics, and H (Information Entropy), are basically the same thing. Consequently, Statistical mechanics should be seen just as a particular application of a general tool of Logical Inference and Information Theory.

Given testable information, the maximum entropy procedure consists of seeking the probability distribution which maximizes the information entropy, H, subject to the constraints of the information. This constrained optimization problem will be typically solved using the analytical method well-known as Lagrange Multipliers.

Entropy maximization (max H) with no testable Information takes place under a single constraint: the sum of the probabilities must be one. Under this constraint, the maximum entropy probability distribution is the well-known as U (uniform distribution). The MEP can, thus, be considered as a generalization of the classical Principle of Indifference (due to Laplace), also known as the Principle of Insufficient Reason.

We may also consider the Metric Entropy, also called Kolmogorov Entropy, or Kolmogorov-Sinai Entropy, in acronym K - S Entropy[35,36].

In a dynamical system[6-10], the metric entropy is equal to zero for non-chaotic motion, and is strictly greater than zero for chaotic motion.

In Thermodynamics, Prigogine entropy[45] is a very often used term referring to the splitting of Entropy into two variables, one being that which is "exchanged" with the surroundings, and the other being a result of “internal” processes. The Entropy of a System is an Extensive Property, i.e.

If the system consists of several parts, the total entropy is equal to the sum of the entropies of each part, and

The change in entropy can be split into two parts.

3. Entropy on Networks

Graph entropywill be a functional on a graph, G = (V, E), with P a probability distribution on its node (or vertex) set, V. It will be denoted by GE. It was a concept initially introduced by Janos Körner, as solution of a coding problem formulated on IT. Because its sub-additivity, it has become a useful tool in proving some lower bounds results in Computational Complexity Theory[1,2].

The search for exact additivity has produced certain interesting combinatorial structures. One of such results is the characterization of perfect graphs by the additivity of GE. Such GE-function is convex. It tends to +∞ on the boundary of the non-negative orthant of Rⁿ. And monotonically to -∞ along rays from the origin. So, such minimum is always achieved and finite.

Recall that a Perfect Graph is one in which the chromatic number of every induced subgraphequalsthe size of the largest clique of that subgraph. Because in any graph the clique number provides a lower bound for the chromatic number.

We may distinguish now four of such structural models.

Thus, we can classify as Regular Networks, Random Net- works, Small-World Networks, and Scale-free Networks.

In the Regular Network, each node is connected to all other nodes. I.e. they are fully connected. Because of such a type of structure, they have the lowest path length (L), and the lowest diameter (D), being

L = D = 1

Also, they have the highest clustering coefficient. So, it holds C = 1. Furthermore, the highest possible number of edges

card (E) =[(n (n-1)) / 2] ∼n²

As related to Random Graphs (RGs), we can say that each pair of nodes is connected with probability p.

They have a low average path length, according to

L ≈ (ln n) / (ln) ∼ln n, for n >> 1

And it is because the total network may be covered in steps, from which

n∼exp L

Moreover, they possess a low clustering coefficient, when the graph is sparse,

C = ( / n) << 1such that the probability of each pair of neighbouring nodes to be connected is precisely equal to p.

The Small-World effect is observed on a network when it has a low average path length, i.e.

L << n, for n >> 1

Recallthat the now famous "six degrees of separation” is also known as the small-world phenomenon. This initially surprising hypothesis was firstly formulated by the writer Frygies Karinthy in 1929.Then, it was tested by Stanley Milgram in 1967. So, Small-World models share with Random Graphs some common features, such asPoisson or Binomial degree distribution, near to the Uniform distribution; network size: it does not grow; each node has approximately the same number of edges, i.e. it shows a homogeneous nature.Such models show the low average path length typical of Random Graphs,

L∼ln n, for n >> 1

And also such model gives the usual high clustering coefficient of Regular Lattices, givingC ≈ .75,for k >> 1. Therefore, the WS-models have a small-world structure, being well clustered.

The Random Graphs coincides on the small-world structure, but they are poorly clustered. This model (Watts- Strogatz) has a peak degree distribution of Poisson type. With reference to the above last model (Scale-Free Network), this appears when the degree distribution follows a Power- Law

P (k) ∼exp_k(- γ)

Whereγis a constant whose value is typically on thedomain

2 <γ< 3

In such a case, there exist a small number of highly connected nodes, called Hubs, which are the tail of the distribution.

On the other hand, the greatest majority of the sets of nodes have very few connections, representing the head of such distribution.Such a model was introduced by Albert- LászloBarabási, and simultaneously by Réka Albert (1999). But there exist certain historical precedents, as George Udny Yule (1925), Herbert Simon (1955), and Derek de Solla Price (1976), among others.

Some of their principal features may be these: Non- homogeneous nature, in the sense that some (few) nodes have many edges from them, and the remaining nodes only have very few edges, or links; as related to the network size, it continuously grows; and regarding to the connectivity distribution, it obeys a Power-Law distribution.

4. Modularity

A very notable example of Scale-Free Network may be the World Wide Web (WWW). It will be described by acollection of sub-networks. In such case, the nodes correspond to their pages, whereas the edges correspond to their hyperlinks. Also related to the Web graph characteristics, we may notice the Scale Invariance, as a very important feature of such notable complex network.

Community structure (also called Modularity) is a very frequent characteristic in many real networks. Therefore, it has become a key problem in the study of networked systems. Giving out its deterministic definition will be a nontrivial problem for the complexity of networks.

A concept of modularity Q can be used as a valid measure for community structure.Some current models have proposed to capture the basic topological evolution of complex networks by the hypothesis that highly connected nodes increase their connectivity faster than their less connected peers, a phenomenon denoted as PA (preferential attachment). So, we will found a class of models that view networks as evolving dynamical systems, rather than static graphs.

Most evolving network models are based on two essential hypotheses, growth and preferential attachment. The growth suggests that networks continuously expand through the addition of new nodes and links between the nodes. While the preferential attachment states that the rate with which a node with k links acquires new links is a monotonically increasing function of k.

The modularity function, Q, provides a way to determine if a partition is valid to decipher the community structure in a network.Maximization of such modularity function, over all the possible partitions of a network, is in fact a highly effective method.

An important case in community detection is that some nodes may not belong to a single community, and then placing them into more than one group is more reasonable.

The set of such nodes can provide a “fuzzy” categorization, and hence, they may take a special role[18], such as signal transduction in biological networks.

5. Conclusions

Therefore, we conclude for the moment our analysis on Entropy and another interesting Fuzzy Measures; in particular, when they are applied on Complex Networks. In such general vision we have showed some new entropy measures.