Posts Tagged subject=performance
Schema-less Databases, Transactions and Eventual Consistency
Posted by admin in Infrastructure, Performance on March 11th, 2009
The value of schema-less databases seems to be a topic of emerging interest, see for instance:
Is the Relational Database Doomed?
Discusses some of the potential of key/value databases as compared to RDBs. The immediate answer to the inflammatory title is, of course, no. However, it seems increasingly clear that one can find a lot of company in suggesting that there is significant value in and adoption of schema-less database approaches.
See also:
How FriendFeed uses MySQL to store schema-less data
There are many responses to the above post, so some reading is required, but it may be worth it. I found it interesting though that there were no comments on triplestores. I’m not quite ready to jump in on that though. I’ll try and come back to it later and see what additional comments may have been made.
In any case, performance is still always an issue. Many approaches are taken to dealing with response time, and some form of replication is frequently involved, whether it be copying data into parallel systems, or storing it in alternate forms or formats that have different access versus update characteristics. This inevitably leads to a problem of maintaining consistency between the various manifestations of the data. The challenge of maintaining consistency across various forms of parallel systems is therefore a recurrent theme and one addressed in the following sources:
Eventually Consistent – Revisited
Discusses some of the problems managing reads and writes and keeping everything consistent.
Sesame 3.0 Preview: An Open Source Framework for RDF Data
From a recent DevX.com article. Mentions the concept of “eventual consistency” in the “Transactions” section.
In our case, end-user results and the process of achieving consistency depends on the order in which one updates:
- Files
- Triplestores
- Solr Indexes
The following references and quote are from an email exchange with Benjamin O’Steen [bosteen@gmail.com].
Writing to serialized data (files), and later updating Solr indices using JMS/AMQP [RAbbitMQ] enables ” indexes ‘eventually converging’ to the truth within seconds after the event (truth being whatever the data held on disc says is true.)”
Changes to an RDF document can be queued as a Talis changeset and later committed.
Note: This post originally addressed the topic of R/W Contention in triplestores and Solr as an approach, posed by Declan Fleming.
Recent Comments