The QualityCommons: Presentations
The QualityCommons: Presentations
Quality of research between preventing and making a difference 
Speaker : Petra Ahrweiler
Assessing research quality is a vibrant source of innovation generating new products, processes, services, jobs, and companies. This emerging assessment industry has to cater for a big market. Scientific quality concepts are negotiated in TV discussions, courts, evaluation committees of funding agencies, promotion boards of university management, government agencies for science and technology policy, or editorial boards of scientific journals. It is understandable that this heterogeneous community has to rely on obvious, manageable and transparent criteria, i.e. criteria everybody can oversee. Technical correctness of the production process and the countability of knowledge products (journal publications, patents, awards, projects) offer such a terrain for indicator-based quality assessment by everybody. However, to rely on the quality criteria of production standards and mainstream markets will generate mass products, following the lines of conformity – being scientifically correct but a little boring. What is science all about: is it about exploring and creating the new (trying to make a difference), or is it about meeting the requirements of standardised knowledge production measured by quantifiable indicators (trying to prevent any difference)? The control features ensure quality in terms of conformity to exclude free riders and black sheep. The question is whether the attempt to extinguish black sheep will only produce more intelligent black sheep, whether the control structure will suffocate the performance of the original target structure, whether the professional ethos of the profession will be irreversibly damaged, and worst of all: whether the actual contents-related quality discourse, which can only be performed by scientists, will be prevented by the parasite control discourse of bureaucratic rationality, which can be performed by everybody. There is a danger that the current quality assessment practices will produce what they actually try to prevent: worse quality.
Quality and Art Worlds
Speaker : Victoria Alexander
This paper looks at the issue of quality in the arts, drawing on the influential ideas of the sociologist Howard Becker. His work suggests that art is produced and given value within social networks he calls ‘art worlds’. Becker’s work provides a foundation for asking questions about the social construction of quality across different domains.
Quality versus fashion in collective behavior
Speaker : Alex Bentley
Classic human decision models often begin with individuals who select for quality (or fitness) as the basic behavior, with any social influence as a secondary add-on. This surely underestimates social influence among humans, whose brains have evolved to handle social relations. Alternative models represent social influence in virtually all its complexities. This represents much better the social nature of humans, but a large number of different parameters can make it difficult to test these models against real-world data. A better starting point in many cases may be to test between two simple extremes, namely whether people base their choices on (a) perceived quality versus (b) the decisions of others in an undirected manner. As captured by experiments and simple evolutionary drift models, undirected social influence introduces emergent patterns in collective behaviour that be used as a null hypothesis against which selection for quality becomes evident.
Food quality as a public good: cooperation dynamics and economic development in a rural community
Speaker : Riccardo Boero
The present work deals with an initiative that aims at creating and promoting rural local development systems. It is called “Presidia”, it has been started by the Slowfood movement, and here it is framed into the general context of rural economies; then its features are described along the results of its first years of application. At the core of the phenomenon here studied lies the fact that the quality of the food produced is the competitive advantage of the local development system. Finally, a case-based study is presented, based upon an empirical agent-based model. The results challenge the theory of voluntary public good provision to fully consider social structure and suggest some requirements for effective rural development policies.
Complexity and Quality in Mathematics and in Art
Speaker : John Casti
This talk examines the interrelationship between the objective, subjective and social complexity of mathematical theorems and works of abstract art. The latter two notions of complexity are related to the issue of "quality", while the first is basically syntactic. Our provisional conclusion is that there is a positive correlation between what people like (subjective complexity), what society values (social complexity) and what is actually complex (objective complexity).
The interplay of social links and user similarity in social media
Speaker : Ciro Cattuto and Alain Barrat
In social media, all shared entities are formed and evolve over the social network that connects individuals to each other. Understanding the concepts of alignment and influence, tracking the emergence of shared semantics, and determining the interplay of user similarity and social proximity are key open questions that require a careful definition of the metrics used to quantify such concepts. We investigated popular collaborative tagging systems and real-world social interactions to unearth the relation between proximity on the social network and similarity of interests. We show that defining null models is crucial for telling apart the actual signal from correlations that arise purely from the structural properties of on-line social networks. These findings have broad implications for the definition and interpretation of metrics of semantic cohesiveness in an on-line community, and highlight the crucial role of homophily in determining information flow in social media.
What is Quality?
Speaker : Edmund Chattoe-Brown
Using an analysis of data from Amazon reviews, this presentation examines what we do when talk about product quality and links this to more general issues about the ambiguous status of "quality" as an evaluative term. The analysis of data is then used to propose some possible directions for future data collection and interesting applications of computer simulation.
Dissagregating Quality Judgements
Speaker : Bruce Edmonds
Abstract will soon be available.
Open sourcing financial functions and institutions
Speaker : David Hales
Peer-to-Peer technology has created disruptive outcomes for both media distribution (bittorrent), knowledge creation and distribution (wikipedia) and money lending (zopa). Also open source software powers the web (apache). What binds these advances together is a commitment to decentralisation and opening of the social structures that produce the quality outcomes we all benefit from. Neither capitalist monopoly practices nor socialised central control are productive and hence peer production is taking an increasing role in our lives. The question I want to ask is "can we extend these trends to provide the functions of financial institutions?". Put simply could we envisage a P2P open source bank or money system? I would like to provoke a debate - is this a viable and desirable project and what would we need to do, now, to bring it about?
Scientific Collaboration, Publishing and Education in the Future
Speaker : Dirk Helbing
The way in which science is organized today has largely missed the opportunities of the information age, particularly Web2.0. I shall, therefore, discuss which parts of scientific knowledge creation and dissemination need to be reinvented, what tools are available, and which ones need to be created to implement the required changes.
Crowdsourcing Real-Time Impact Factors and a Semantic Research Database
Speaker : Victor Henning
Citation-based impact factors play an ever-increasing role in modern science. Several initiatives are now working on alternative, crowdsourced usage-based impact factors. Crowdsourcing has several advantages – most notably, article usage statistics can be collected in real-time as research is being done, instead of waiting for citations to be published and picked up by citation-based metrics. Moreover, crowdsourced data is segmentable by academic status, academic discipline, geographic region, and other socio-demographics. One of the aforementioned real-time research data initiatives is Mendeley – a desktop- and web-based bibliography manager and collaboration tool. Academics can use Mendeley to manage, read, annotate, share and cite academic literature. Every user’s documents and annotations are anonymously aggregated on Mendeley Web, accumulating more than 10m documents and 140m citations in less than 12 months since Mendeley’s launch (for comparison, Thomson Reuters Web of Science contains 40m documents and 700m citations). Mendeley's database is currently doubling every 12 weeks. Besides providing real-time usage data, researchers will be able to create semantic links between papers, e.g. “supports”, “contradicts”, “complements”, “uses same data as”, so that a crowdsourced, self-updating semantic research database will emerge.
Beyond Copying
Speaker : Paul Ormerod and Greg Wiltshire
This paper focuses on the emergence of shared conceptions of quality. Much of the agent based/network literature which focuses on the spread of ideas/behaviour, the emergence of norms etc. essentially involves ‘binary choice with externalities’ [Schelling 1973]. Heterogeneous agents are connected on a network and are in one of two states of the world (though this can be generalised to k states). Agents switch depending upon their individual threshold (propensity to switch) and the states of the world of their neighbours. This needs to be combined with an element of rational choice. So agents could have heterogeneous preferences on the weights they assign to ‘peer influence’ [copying] and ‘rational choice’ i.e. evaluating the attributes of the alternatives in some bounded rational way. Within the latter, we might also think of agents each having a propensity to persuade i.e. discuss with neighbours their own conception of quality and a propensity to be persuaded i.e. how willing are they to accept their neighbours ‘rational’ arguments. I propose to offer some initial results on how this affects patterns of percolation/adoption.
Is quality a normative notion? A socio-epistemological perspective
Speaker : Gloria Origgi
As David Hume wrote in his famous essay, Of Standard of Taste: “The great variety of Taste, as well as of opinion, which prevails in the world, is too obvious not to have fallen under every one's observation”. If this is the case, how can we construct coherent collective representations of quality? I will argue that quality is an intrinsically normative notion, not only in the domain of subjective-dependent representations such as aesthetics, but also in more objective domains such as science. The construction of collective representations of quality depends on a network of trust and reputation relations that enforce previously existing sets of evaluations – let us call them “traditions” – upon which the standards of quality are built and maintained.
Simulating cultural dynamics in peer production environment
Speaker : Mario Paolucci and Stefano Picascia
As the production of cultural goods moves towards a peer-based, non-market approach - with the internet as main platform of distribution - the formation and propagation of shared quality evaluations is increasingly mediated by the set of collective filters and social content platforms emerged to provide a shortcut to information in the overloaded web. The architectures differ from one another, but all have in common the following traits:
- The principle of rating. Implemented, explicitly or not, in the vast
majority of contexts - a set of algorithms which implement a digital flavor of one ancient
social artifact: reputation - the principle of ranking presiding over the presentation of information.
These filters and platforms play a fundamental epistemic role, in that they shape the way the symbolic production of society is accessed. In our view they need to be investigated both from a cognitive point of view and from a broader 'systemic' perspective.
To start inquiring the latter we propose to use agent-based simulation, to capture the complexity of the processes and mechanisms that underlie content publishing and accessing. In the present papers, we show a preliminary model of culture circulation in a peer production environment, using the meme metaphor. The model tests the different outcomes, in terms of memetic variation, of different filtering algorithms at work on the web. In the simulation, a population of agents play roles of producers and consumers of artefacts. Both the artefacts and the agents' mind contain memes, that compete for attention. The artefact produced are connected in a network of directed links, that simulates the world wide web. In this context, we experiment with different search algorithms, from pagerank-like to collective filtering.
Discerning The Good from The Bad and The Ugly: Issues of Quality in Computational Social Science
Speaker : Claudio Cioffi-Revilla
Social computing or computational social science is the modern descendant of several research traditions with roots in The Enlightenment and even earlier origins in Aristotle's comparative investigation of social systems. As a result, extant standards of scientific quality and excellence have been inherited through the philosophy of science in terms of basic principles such as formalization, replication, and dissemination. Valid and helpful as such classic standards of quality may be, the fact remains that social computing adds new scientific features (complex systems, object-oriented simulations, network models, nonlinear dynamics) that have yet to be included within new standards of quality.
The Social Dynamics of Quality Choice
Speaker : Andrzej Nowak
A lot of natural social dynamics is centered around quality. If you look at the content of magazines it looks like one of the crucial human concerns is about quality. Also quality seems to be one of the main topics of conversations among individuals. What makes a good car? which movies are good? Where you can eat well? What are the best spots for vacations? Which shoes are really comfortable, etc. Clearly quality is highly subjective and it provides an important dimension of social dynamics.
The standards of very high quality are associated with the belief that high quality cannot be very popular (popular=cheap). There are important elements of the minority game. One wants to be in the groups that established a new quality standards, but also want to be to be elite so the that the attractiveness of this group is somehow (perhaps nonlinearly ) related to its size and, as the group grows its membership, it loses attractiveness. People differ in how influential they are, so as the new members enter that group by adopting the quality standards the group is no longer attractive to the ones who have originally set up the trend (trendsetters), so they drop out to start a new trend. Their game is to start a new trend, but at the same time to be in a small majority.
One needs to be the leader in the quality game, i.e. have objects that are at the top of quality standards, i.e. the best car: Rolls Royce never breaks. These, who cannot compete are trying to set up new standards of quality that would allow them to assume dominance. Speed is crucial, (e.g. the speed is crucial: Ferrari), or comfort: (vans). If one cannot win at a game, one changes the game i.e. The quality standards.
The dynamics of quality standards of products not associated with prestige (i.e. washing powder or flashbulbs) is just trying to converge in a majority (social influence) game.
Only a few people are real innovators, trying to discover and establish standards of quality and the rest just copy their choices. When I need a camera, I found a friend who also was about to buy a camera. He has spent two weeks comparing all the models and trying to establish dimensions of quality. I simply asked him what camera he bought and I bought the same. Then my friend just asked me what camera I have and bought the same model. There is probably a social network of trend setters who really play the game by investing a lot of resources and the rest just copy their choices. The game for the trend setters is to be copied, and the game for the follower is to choose the best expert.
Quality of research between preventing and making a difference
Speaker : Petra Ahrweiler
Assessing research quality is a vibrant source of innovation generating new products, processes, services, jobs, and companies. This emerging assessment industry has to cater for a big market. Scientific quality concepts are negotiated in TV discussions, courts, evaluation committees of funding agencies, promotion boards of university management, government agencies for science and technology policy, or editorial boards of scientific journals. It is understandable that this heterogeneous community has to rely on obvious, manageable and transparent criteria, i.e. criteria everybody can oversee. Technical correctness of the production process and the countability of knowledge products (journal publications, patents, awards, projects) offer such a terrain for indicator-based quality assessment by everybody. However, to rely on the quality criteria of production standards and mainstream markets will generate mass products, following the lines of conformity – being scientifically correct but a little boring. What is science all about: is it about exploring and creating the new (trying to make a difference), or is it about meeting the requirements of standardised knowledge production measured by quantifiable indicators (trying to prevent any difference)? The control features ensure quality in terms of conformity to exclude free riders and black sheep. The question is whether the attempt to extinguish black sheep will only produce more intelligent black sheep, whether the control structure will suffocate the performance of the original target structure, whether the professional ethos of the profession will be irreversibly damaged, and worst of all: whether the actual contents-related quality discourse, which can only be performed by scientists, will be prevented by the parasite control discourse of bureaucratic rationality, which can be performed by everybody. There is a danger that the current quality assessment practices will produce what they actually try to prevent: worse quality.
Quality and Art Worlds
Speaker : Victoria Alexander
This paper looks at the issue of quality in the arts, drawing on the influential ideas of the sociologist Howard Becker. His work suggests that art is produced and given value within social networks he calls ‘art worlds’. Becker’s work provides a foundation for asking questions about the social construction of quality across different domains.
Quality versus fashion in collective behavior
Speaker : Alex Bentley
Classic human decision models often begin with individuals who select for quality (or fitness) as the basic behavior, with any social influence as a secondary add-on. This surely underestimates social influence among humans, whose brains have evolved to handle social relations. Alternative models represent social influence in virtually all its complexities. This represents much better the social nature of humans, but a large number of different parameters can make it difficult to test these models against real-world data. A better starting point in many cases may be to test between two simple extremes, namely whether people base their choices on (a) perceived quality versus (b) the decisions of others in an undirected manner. As captured by experiments and simple evolutionary drift models, undirected social influence introduces emergent patterns in collective behaviour that be used as a null hypothesis against which selection for quality becomes evident.
Food quality as a public good: cooperation dynamics and economic development in a rural community
Speaker : Riccardo Boero
The present work deals with an initiative that aims at creating and promoting rural local development systems. It is called “Presidia”, it has been started by the Slowfood movement, and here it is framed into the general context of rural economies; then its features are described along the results of its first years of application. At the core of the phenomenon here studied lies the fact that the quality of the food produced is the competitive advantage of the local development system. Finally, a case-based study is presented, based upon an empirical agent-based model. The results challenge the theory of voluntary public good provision to fully consider social structure and suggest some requirements for effective rural development policies.
Complexity and Quality in Mathematics and in Art
Speaker : John Casti
This talk examines the interrelationship between the objective, subjective and social complexity of mathematical theorems and works of abstract art. The latter two notions of complexity are related to the issue of "quality", while the first is basically syntactic. Our provisional conclusion is that there is a positive correlation between what people like (subjective complexity), what society values (social complexity) and what is actually complex (objective complexity).
The interplay of social links and user similarity in social media
Speaker : Ciro Cattuto and Alain Barrat
In social media, all shared entities are formed and evolve over the social network that connects individuals to each other. Understanding the concepts of alignment and influence, tracking the emergence of shared semantics, and determining the interplay of user similarity and social proximity are key open questions that require a careful definition of the metrics used to quantify such concepts. We investigated popular collaborative tagging systems and real-world social interactions to unearth the relation between proximity on the social network and similarity of interests. We show that defining null models is crucial for telling apart the actual signal from correlations that arise purely from the structural properties of on-line social networks. These findings have broad implications for the definition and interpretation of metrics of semantic cohesiveness in an on-line community, and highlight the crucial role of homophily in determining information flow in social media.
What is Quality?
Speaker : Edmund Chattoe-Brown
Using an analysis of data from Amazon reviews, this presentation examines what we do when talk about product quality and links this to more general issues about the ambiguous status of "quality" as an evaluative term. The analysis of data is then used to propose some possible directions for future data collection and interesting applications of computer simulation.
Dissagregating Quality Judgements
Speaker : Bruce Edmonds
Abstract will soon be available.
Open sourcing financial functions and institutions
Speaker : David Hales
Peer-to-Peer technology has created disruptive outcomes for both media distribution (bittorrent), knowledge creation and distribution (wikipedia) and money lending (zopa). Also open source software powers the web (apache). What binds these advances together is a commitment to decentralisation and opening of the social structures that produce the quality outcomes we all benefit from. Neither capitalist monopoly practices nor socialised central control are productive and hence peer production is taking an increasing role in our lives. The question I want to ask is "can we extend these trends to provide the functions of financial institutions?". Put simply could we envisage a P2P open source bank or money system? I would like to provoke a debate - is this a viable and desirable project and what would we need to do, now, to bring it about?
Scientific Collaboration, Publishing and Education in the Future
Speaker : Dirk Helbing
The way in which science is organized today has largely missed the opportunities of the information age, particularly Web2.0. I shall, therefore, discuss which parts of scientific knowledge creation and dissemination need to be reinvented, what tools are available, and which ones need to be created to implement the required changes.
Crowdsourcing Real-Time Impact Factors and a Semantic Research Database
Speaker : Victor Henning
Citation-based impact factors play an ever-increasing role in modern science. Several initiatives are now working on alternative, crowdsourced usage-based impact factors. Crowdsourcing has several advantages – most notably, article usage statistics can be collected in real-time as research is being done, instead of waiting for citations to be published and picked up by citation-based metrics. Moreover, crowdsourced data is segmentable by academic status, academic discipline, geographic region, and other socio-demographics. One of the aforementioned real-time research data initiatives is Mendeley – a desktop- and web-based bibliography manager and collaboration tool. Academics can use Mendeley to manage, read, annotate, share and cite academic literature. Every user’s documents and annotations are anonymously aggregated on Mendeley Web, accumulating more than 10m documents and 140m citations in less than 12 months since Mendeley’s launch (for comparison, Thomson Reuters Web of Science contains 40m documents and 700m citations). Mendeley's database is currently doubling every 12 weeks. Besides providing real-time usage data, researchers will be able to create semantic links between papers, e.g. “supports”, “contradicts”, “complements”, “uses same data as”, so that a crowdsourced, self-updating semantic research database will emerge.
Beyond Copying
Speaker : Paul Ormerod and Greg Wiltshire
This paper focuses on the emergence of shared conceptions of quality. Much of the agent based/network literature which focuses on the spread of ideas/behaviour, the emergence of norms etc. essentially involves ‘binary choice with externalities’ [Schelling 1973]. Heterogeneous agents are connected on a network and are in one of two states of the world (though this can be generalised to k states). Agents switch depending upon their individual threshold (propensity to switch) and the states of the world of their neighbours. This needs to be combined with an element of rational choice. So agents could have heterogeneous preferences on the weights they assign to ‘peer influence’ [copying] and ‘rational choice’ i.e. evaluating the attributes of the alternatives in some bounded rational way. Within the latter, we might also think of agents each having a propensity to persuade i.e. discuss with neighbours their own conception of quality and a propensity to be persuaded i.e. how willing are they to accept their neighbours ‘rational’ arguments. I propose to offer some initial results on how this affects patterns of percolation/adoption.
Is quality a normative notion? A socio-epistemological perspective
Speaker : Gloria Origgi
As David Hume wrote in his famous essay, Of Standard of Taste: “The great variety of Taste, as well as of opinion, which prevails in the world, is too obvious not to have fallen under every one's observation”. If this is the case, how can we construct coherent collective representations of quality? I will argue that quality is an intrinsically normative notion, not only in the domain of subjective-dependent representations such as aesthetics, but also in more objective domains such as science. The construction of collective representations of quality depends on a network of trust and reputation relations that enforce previously existing sets of evaluations – let us call them “traditions” – upon which the standards of quality are built and maintained.
Simulating cultural dynamics in peer production environment
Speaker : Mario Paolucci and Stefano Picascia
As the production of cultural goods moves towards a peer-based, non-market approach - with the internet as main platform of distribution - the formation and propagation of shared quality evaluations is increasingly mediated by the set of collective filters and social content platforms emerged to provide a shortcut to information in the overloaded web. The architectures differ from one another, but all have in common the following traits:
- The principle of rating. Implemented, explicitly or not, in the vast
majority of contexts - a set of algorithms which implement a digital flavor of one ancient
social artifact: reputation - the principle of ranking presiding over the presentation of information.
These filters and platforms play a fundamental epistemic role, in that they shape the way the symbolic production of society is accessed. In our view they need to be investigated both from a cognitive point of view and from a broader 'systemic' perspective.
To start inquiring the latter we propose to use agent-based simulation, to capture the complexity of the processes and mechanisms that underlie content publishing and accessing. In the present papers, we show a preliminary model of culture circulation in a peer production environment, using the meme metaphor. The model tests the different outcomes, in terms of memetic variation, of different filtering algorithms at work on the web. In the simulation, a population of agents play roles of producers and consumers of artefacts. Both the artefacts and the agents' mind contain memes, that compete for attention. The artefact produced are connected in a network of directed links, that simulates the world wide web. In this context, we experiment with different search algorithms, from pagerank-like to collective filtering.
Discerning The Good from The Bad and The Ugly: Issues of Quality in Computational Social Science
Speaker : Claudio Cioffi-Revilla
Social computing or computational social science is the modern descendant of several research traditions with roots in The Enlightenment and even earlier origins in Aristotle's comparative investigation of social systems. As a result, extant standards of scientific quality and excellence have been inherited through the philosophy of science in terms of basic principles such as formalization, replication, and dissemination. Valid and helpful as such classic standards of quality may be, the fact remains that social computing adds new scientific features (complex systems, object-oriented simulations, network models, nonlinear dynamics) that have yet to be included within new standards of quality.
The Social Dynamics of Quality Choice
Speaker : Andrzej Nowak
A lot of natural social dynamics is centered around quality. If you look at the content of magazines it looks like one of the crucial human concerns is about quality. Also quality seems to be one of the main topics of conversations among individuals. What makes a good car? which movies are good? Where you can eat well? What are the best spots for vacations? Which shoes are really comfortable, etc. Clearly quality is highly subjective and it provides an important dimension of social dynamics.
The standards of very high quality are associated with the belief that high quality cannot be very popular (popular=cheap). There are important elements of the minority game. One wants to be in the groups that established a new quality standards, but also want to be to be elite so the that the attractiveness of this group is somehow (perhaps nonlinearly ) related to its size and, as the group grows its membership, it loses attractiveness. People differ in how influential they are, so as the new members enter that group by adopting the quality standards the group is no longer attractive to the ones who have originally set up the trend (trendsetters), so they drop out to start a new trend. Their game is to start a new trend, but at the same time to be in a small majority.
One needs to be the leader in the quality game, i.e. have objects that are at the top of quality standards, i.e. the best car: Rolls Royce never breaks. These, who cannot compete are trying to set up new standards of quality that would allow them to assume dominance. Speed is crucial, (e.g. the speed is crucial: Ferrari), or comfort: (vans). If one cannot win at a game, one changes the game i.e. The quality standards.
The dynamics of quality standards of products not associated with prestige (i.e. washing powder or flashbulbs) is just trying to converge in a majority (social influence) game.
Only a few people are real innovators, trying to discover and establish standards of quality and the rest just copy their choices. When I need a camera, I found a friend who also was about to buy a camera. He has spent two weeks comparing all the models and trying to establish dimensions of quality. I simply asked him what camera he bought and I bought the same. Then my friend just asked me what camera I have and bought the same model. There is probably a social network of trend setters who really play the game by investing a lot of resources and the rest just copy their choices. The game for the trend setters is to be copied, and the game for the follower is to choose the best expert.
