This is a description of the "Percolator" technology behind Google's
new incremental "Caffeine" search index. It produces ~100x lower
latency than the previous MapReduce-based system, at about a 2x cost
in resources. It's a system for dependency propagation using observers
on BigTable columns. It also leverages BigTable transactional
semantics to provide snapshot isolation and other transaction
management services like straggler elimination.