[Visionaries] The Linking Problem Revisited

[Peter Minten]

Hi folks,

doing some thinking about the linking problem I came to the conclusion that it's
essentially unsolvable, it's only possible to reduce it's effect. The main
strategy for that is discrimination of data. An observation: the largest part of
the websearches targets just a small part of the web, the smallest part of the
websearches targets the biggest part of the web.

The theory behind proxy servers is just that, if you have the most common
searched stuff in cache the internet speeds up. This can also be done with RDF.

For RDF it's also useful to note that not every property of a node is searched
equally often. Thus you could divide properties into two classes: those that are
often searched (I call them search properties) and those that are not often
searched. Thus it makes sense for an RDF search engine to copy only the search
properties into it's database.

Proxies could use a similar model where the properties that are often used are
put in fast memory and the ones that are less often used are put in slow memory.

For searches it's important to note that some results come in later than others.
Abdabi already uses this quite elegantly by making the search async, results are
displayed when they come in.

Greetings,

Peter