posted 22 Jun 2010 in Volume 13 Issue 9
The power of structure
Jan Wyllie continues his series on emerging knowledge flows and usergenerated content with a look at algorithmic intelligence
My last article, ‘When two worlds collide’ identified the two principal characteristics of the emerging forms of knowledge – as flow rather than documents, and as user generated, and user ‘curated’ content, rather than professionally generated and ‘managed’ content.
The core assumption behind the design of software tools to help people to cope with these new forms of knowledge is that algorithms, typified by Google’s ‘relevance ranking’, are the best (indeed the only) means to deal with the unimaginable quantities of information available. Without these sorts of powerful algorithms and the immense processing power of vast server farms, people would drown in the torrent of information, or be lost forever in a huge and uncharted sea of data.
Unlike pure Boolean searching where people can understand how their queries work, the assumptions, logic and mathematics behind algorithmic solutions are completely unknown to users. They must have blind faith that the blackest of black boxes really is returning the results most pertinent to their query, because that’s what the software providers say. Of course, users can never really know because they have no idea of what knowledge is out there in the first place. They can have no idea of what the algorithm might have missed. This is not surprising since the algorithm cannot know what the user really wanted.
In addition to this radical uncertainty, software that calibrates relevance on the basis of aggregating and analysing users’ search behaviour carries the added risk of the tyranny of the crowd as popularity becomes of the arbiter of relevance. Nevertheless, the overwhelming bulk of research and development funding is going into the algorithmic computing culture.
Even the US Air Force is in on the act. According to April’s Wired magazine: “The Defense Department is continuing its push to reduce human thought and human action to a few lines of code.” The Air Force Office of Scientific Research1 is reported as, “looking to build ‘mathematical or computational models of human attention, memory, categorisation, reasoning, problem solving, learning and motivation, and decision making’.”
The social and intelligence consequences of this algorithmic approach have been enormous. Since the mysterious algorithms are supposed to do all the work, people have not had to learn the basic skills necessary for effective knowledge management (KM). Indeed, the costs of this profound ignorance must be even greater than all the money spent on the years of chasing the chimera of the intelligent machine.
Although most of the money has been spent on the development of algorithmic intelligence, the Web 2.0 phenomenon has produced a spontaneous flowering of the use of human intelligence to help make sense of the knowledge on the web. This includes tagging, folksonomies, systems enabling guidance by following others and their recommendations, or systems that measure the value and popularity of both knowledge items and knowledge flows.
What all these have in common is that they rely on people’s judgements about relevance, not algorithms. Another thing they have in common is that they were created by amateurs with little knowledge of the techniques of KM. The final thing these systems have in common is that they do not exploit the power of structure to open vistas on knowledge flows.
What if basic KM skills, tools and taxonomies were to be learned by large numbers of social media users? Wildcat, the original ‘knowmad’, puts it like this:
“Knowmads are developing a particular set of skills that will in time become the normative expression of extra ordinary cognition, I refer specifically to minds that are developing an exceptional capability of enhancing the health and wealth of an infocology by their choices.
These choices of what to publish and what to point, what to prune and what to emphasize are the key parameters of these intersectionists (hyperconnected individuals locating themselves at the intersection of feeds of information in advanced infocologies)2.”
This process is much more like music and dance, than an algorithm. Music and dance would be impossible without the structures of rhythms and harmonics and agreed rules for playing them. Like music and dance, the knowledge flows of infocologies unfold through time, rather being stuck and forgotten in past time.
Also just like music and dance, the play can only begin on the basis of some agreed conventions and structures. Think of the difference between dancing the Waltz and a
Meaning, whether in music, dance or knowledge flows, comes from imposing a structure on movement of the body or of information flows through time. The game is about channelling, monitoring and inferring significance from the flows. But before the play can start, there needs to be some agreed rules and taxonomies. This is the power of structure. Are people ready to embrace this knowledge en masse? Perhaps not, although it must be said that ideas whose time have come do spread like wildfire over the web.
Then knowledge managers would morph into knowmads channelling and adding value to the information. And users would become knowmads too by becoming proficient knowledge managers. And everyone would live happily ever after – except the relevance algorithms which weren’t alive in the first place.
Jan Wyllie is the founder of Open Intelligence. He is also the author of one of