The Science of Diversity and Its
Relevance in a Fast Changing World
Diversity in the workplace, organization, even nation, has recently become a much touted goal. Our understanding has evolved to recognize that diversity includes more than ethnic variety, but also cultural and educational variety, in fact, any aspect of an individual that leads to a variety of approaches to problems, even personality. Yet, the scientific basis behind the role of diversity in social systems is largely unknown. Research at Los Alamos is providing support for our intuitive understanding of the importance of diversity in social systems.
Much of the past understanding about social diversity is motivated by the analogies to biodiversity in nature. The prevailing research on the role of biological diversity focuses on improvement of system performance (e.g., individual or species survival) by the selection from a pool of genetic diversity, in the process of natural selection or survival of the fittest. This process is fundamentally competitive between individuals or groups, with explicit winners and losers. This selection reduces diversity in its application - diversity is reduced by selection and must be replenished by mutation or migration. While a similar selection process may occur in social systems, this is neither an compelling justification for diversity within organizations, nor the only mechanism by which diversity contributes to better organizations or society.
Current research at Los Alamos has identified an alternative process for higher system performance, which does not involve competition or cooperation between the individuals, but instead stresses the importance of non-competitive self-organization.
Simulating collective problem solving
We wish to address the question: what is the most simple demonstration of increased global performance of a collective above that of the individual? The idealized system examined is the solution of a sequential problem (Insert in Figure 1), which has many optimal and non-optimal solutions, solved by agents that have identical capabilities. While this maze problem is quite simple, it serves as a representation of more complex problems encountered by individuals and organizations: the solution of a problem that has many decisions points and possible solutions and that has difficulty greater than solvable by one individual. Here, agents can represent individuals, groups or organizations within a greater system.
Because the agents have no global sense of the problem, they initially explore the problem until a solution is found. This "learned" information is then applied by the individual agent to solve the problem again, often with a shorter path as a consequence of eliminating unnecessary loops. Because the initial search is random, a collection of individuals shows a diversity of experience (regions of the maze), of preferences (preferred paths), and of performance (path lengths), even though they started with identical capability. (This source of diversity in performance by individuals of identical capability is a reminder how many of the advantages and disadvantages an individual has in social situations are a result of random events and not directly related to our "inherent" capability. Said another way, if we replayed or lived our own lives again, likely it would be quite different due to these random effects.)
Information for a collective of individuals is then constructed by a linear combination of the each individual's experiences. Then the same rules are used on this collective information to find a collective solution. As seen in Figure 1, the collective typically outperforms the average individual for larger collectives. In repeated solutions to a problem, we tend to remember only the information needed to solve a problem and forget extraneous information associated with unused paths. Here, the equivalent effect is for the individual to contribute to the collective only "established" information along paths used by individual, thereby "forgetting" unused paths. Both the learned and established information produce the same path for the individual agent. As seen in Figure 1, the solution using the established information performs better than the learned information. Furthermore, for collectives above 20, the optimal solution is found, even though nothing in the agent's rules seeks a minimal path length. Figure 2 shows one mechanism for the reason that the collective does better than any individual: individual information can be combined to indicate a shorter path for the collective.
Importance of Diversity
To better understand the role and importance of diversity in this simple model and gain insight into social systems, quantitative measures of diversity were examined. The first choice of a measure, the breadth of experience of a collective over all possible paths, was found to be uncorrelated with performance.
The best measure found defines diversity as the degree of unique information in a collection of agents. If all agents contribute the same information, even if it is for the entire domain, then this measure of diversity is low. If each agent contributes unique information not shared by others, then this diversity measure is high. Consequently, collectives contributing "established" rather than "learned" information exhibit higher diversity, even though less information is contributed by each agent. So we conclude that it is not how much information is important, but how the information contributed fits in to the other information known.
Not only does this measure of diversity correlate best with collective performance, it also indicates the degree of insensitivity to noise. The performance of a collective with low diversity is poor when valid information is randomly replaced with false information, a measure of the robustness of a solution. False information can lead to unexplored paths in a less diverse collective, and then the solution degenerates to a random search (imagine driving and finding yourself on an unfamiliar road - the solution is either to backtrack or to search randomly). Diverse collectives have contingency information that makes the groups highly insensitive to noise. The stabilizing effect of diverse groups is critical in dealing with difficult problems where false information can lead to unexplored paths. In research which focuses on rational individuals with perfect information, an evaluation of performance often does not consider the robustness to noise, hence, the reason that diversity is rarely considered important. Yet, robustness is a critical aspect of any modern organization.
If the effect of information exchange between individuals (a form of cooperation) is included in the above simulations, such that the individual while learning the maze can benefit from other agent's experiences, we find that improved individual performance can be achieved. But if the outside information is used too strongly, then there is the ultimate loss of diversity in the collective as every individual has the identical experience. In this case, the robustness of the collective can be severely degraded if the positive feedback or coherence is too great. We conclude that cooperation is actually a form of selection and reduces the expression of diversity (if I cooperate with you consistenly, then I don't cooperate with someone else). It is easy to see that the random associations that are beneficial to the group and individual above can be reinforced to the point of being exclusive. While this leads to more optimal performace, it also causes the groups to be less robust. (To see how this transition is part of a bigger view of how systems develop, see the paper titled Developmental Insights into Evolving Systems: Roles of Diversity, Non-Selection, Self-Organization, Symbiosis on the documents page. )
Social and Organizational Implications
The above study illustrates how diversity can arise with agents of identical capability from experiential differences within a system which contains multiple options. Just the existence of random options in the problem domain creates diversity. This is in contrast to the standard ecological arguments that diversity originates from competition to fill new niches. It is believed that this generation of random options and traits are a direct consequence of the system becoming more complex, a natural consequence of the development cycle of evolving systems (see the paper titled Developmental Insights into Evolving Systems: Roles of Diversity, Non-Selection, Self-Organization, Symbiosis on the documents page. )
Furthermore, higher system performance and robustness occurs by the simple combination of the experiences of individuals, even though each individual solves a problem from a limited perspective. Unlike the selectionist view of diversity in natural systems, this study indicates that even in the absence of direct competition between and the consequencial selection of individuals, a higher system performance can be achieved with an alternative mechanism: the non-competitive combination of information from a diversity of individuals. In fact, the idealized system exhibits lower performance or lower stability if any selection is made, either by eliminating participation or reducing their contribution. In some cases, it is even found that eliminating the contributions of the higher performers actually can improve the group solution!
The simulations also illustrate that improved collective performance can only be achieved with minimally skilled (or better) agents. If the agents gain experience randomly (have no "skill"), the collective shows no improvement over the average individual. We conclude that the performance of the whole is tied to the performance of the individual - as the individual learns, the non-competitive combination of diverse experiences increases the performance of the system as a whole.
These results argue for the importance of organizational environment that freely exchanges information for both the benefit of the individual, but also for the group or organization. Many economic and social models of human dynamics begin with the assumption of competitive agents seeking limited resources. Furthermore, they embody significant capability in their agents to explain higher global performance. In the simple model above, system performance is shown to be greater than the capability of the agents and to occur from essentially independent agents randomly sharing information. Mechanisms of competition or cooperation are not required. The results also show that if the collective dominates the learning of the individual, and thereby reduces the diversity, then group performance become less robust.
In modern times of organizations becoming more complex and facing problems of greater difficulty, centrally directed management of expert resources may not be an optimal approach to problem solving. For organizations to take advantage of increased performance from diversity, these studies suggest that, in addition to a skilled and diverse workforce, it is also necessary to encourage the expression of diverse views and to enable mechanisms for the exchange and processing of these views. The implications for all organizations is to create a work environment in which all employees are willing and able to contribute their knowledge and experience to solving the problems facing these organizations.
Publications | Talks | Interests: Past & Present | Symbiotic Intelligence Project
Contact | Science of Diversity | Home | Links | Diversity Documents | Norman Johnson
Infectious Idea Spread using Epidemiological Modeling | Collective Intelligence in the presence of bias and misinformation
Community, Coexistence and Pre-conflict Minimization Projects | The CISS (Coexistence in Suggestive Simulations) Project