relinkOwners() is an extremely expensive operation required when updating the structure

[thallium205]

After the addresses have been linked and owners built for the linked addresses, each edge from each affected owner needs to be updated. Currently, relinkOwners in GraphBuilder.java removes every edge, then rebuilds each one. This works well except for the super large nodes (mt.gox, satoshi dice, etc). Exploration into updating the values inside the edges instead of deleting them and recreating them might yield faster build times for nodes with high centrality.

[thallium205]

The reason this is a problem is that the API hangs incoming requests as this build process happens, essentially killing the website (sometimes an overflowed heap error occurs). A good experiment would be to stop pooling transactions in groups of 1000:

if (count > 1000)
{
    deleteTx.success();
    deleteTx.finish();  
    deleteTx = graphDb.beginTx();
    count = 0;
}

and instead create a transaction for each operation. Though this would drastically slow down the overall operation of relinkOwners by increasing the amount of disk writes, I suspect it would allow the database to manage other threads accessing the datastore (like the API) much better.

[thallium205]

That failed horribly...