Monday, January 20, 2014

blackbox: Bitcoin-enabled, decentralized cloud

Rethinking data centers, one node at a time

Bittorrent and bitcoin have proven the power of peer-to-peer, decentralized networking.  Amazon Web Services and Rackspace Cloud demonstrate the market desire for cloud storage and cloud processing. Why not combine these into a single package?

The storage system Tahoe LAFS demonstrates how the data center may be inverted, making the end user the ultimate provider of data storage services.  Continue this idea writ large, and apply it to cloud storage, cloud processing, and other cloud services.

Now, boil cloud services down to their primary components, and consider a user interface that anyone, even largely computer illiterate, could manage.

blackbox: core components

Ultimately, computers provide resources.  These resources may be enumerated, and resold on the free market.  For cloud services, these resources are
  • CPU cores
  • Temporary memory (RAM)
  • Persistent memory (SSD, hard drive)
  • Internet bandwidth
To make these resources usable, further meta-resources are required,
  • Amount of resource available
  • Time window when resource is available
  • Bitcoin address for payments
  • SIN or other decentralized identity
Given these few metrics, we can conceive of "blackbox" software that connects to TradeNet, offers the above resources on the free market, and receives payment when the resources are utilized.

The user experience is as simple as it gets:  Turn on your computer, run the blackbox, and get paid.

Automated service, cost routing

blackbox is not a new idea.  These sorts of ideas are decades old.  The new twist is adding an efficient digital currency (bitcoin) with online, automated, decentralized markets.

Computing jobs, be it 24/7 real-time services ("run my website") or periodic batch processing ("balance today's accounting and sales records") will be layered on top of blackbox's low level cloud services in precisely the same manner as cloud services are used today.

Based on your service needs, nodes will be selected based on cost, reliability, reputation, geolocation, speed, and many other factors.  Requests for service will be matched with offers for service.  Contracts will be agreed upon, perhaps lasting only 24 hours, or an hour, or 60 seconds.  Payment terms may be guaranteed through multi-signature transactions, with reputation, escrow and arbitration ensuring that all parties have incentives to deliver.

Keep us online and honest

The two primary weak points of a decentralized, generalized cloud network are resource accounting and network defense.

Resource accounting includes issues such as ensuring that participants X, Y, and Z all deliver their contractually-promised resources, or are highlighted for lack of delivery.  Service delivery fails for any number of reasons:  malicious action, hardware or software failure, human error.  Ultimately the reasons for service delivery failure are not important. Nodes will develop a reputation for good or bad service over time.

Network defense will involve associating and disassociating with various parties based on their reputation.  As in Tahoe-LAFS, most consumers of computing services will communicate with layers of software 1-2 layers removed from the low-level blackbox software itself.  This offers privacy for the provider, as well as shielding blackbox users themselves from many attacks.  Other defenses will depend more on the precise P2P technologies involved in TradeNet and blackbox.

Mesh network writ large

Mesh networks provide an example of a large, useful service collectively built from tiny components.  TradeNet takes the concept further, imagining a mesh of markets.  blackbox is simply the next logical extension of thought:  once easy, automated, anyone-can-join markets exist, it becomes easy to resell one's own computing resources using modern software that combines those computing resources into a greater collective whole.

Once these P2P cloud services are available, developers will create more useful software layered on top:  web hosting, email services, video processing, all the services one can perform in the cloud today.  Presumably, with the free market at work, costs would be lowered for many, while utilizing computing power that would otherwise be sitting around idle.

With blackbox, a user enters their bitcoin address, and tells the software to make use of the computer, when the owner is away.  Computers inside data centers will run blackbox 24/7.

blackbox 2.0

All of the above may be implemented, in theory, given today's technology.  Improving the system further involves additional data, permitting additional price discovery.

Recent hardware trends have included a focus on efficiency and power usage, a welcome change from decades of counting CPU cycles/second.  blackbox 2.0 would examine local electricity prices, and determine if contributing a resource is economically viable, considering the power usage involved.


  1. Sale of small, casual computing resources implies the practicality of small, casual payments.

    Satoshi pointed out the following problem with traditional payment processing systems: "The cost of mediation increases transaction costs, limiting the minimum practical transaction size and cutting off the possibility for small casual transactions".

    Until "small casual payments" (i.e., "micropayments" to anybody who is not a sleazy or delusional salesman) are a satisfiably solved problem, blackbox will only be possible on the computing resources of dedicated companies, anyway.

  2. Most of the pieces for this are already out there. It's just a matter of time until it all comes together. Network speeds will be the initial bottleneck but it's already possible for limited applications. Remember beowulf CPU clustering? How about a bootable XEN OS that uses namecoin/vpn enabled kerberos to create the platform?
    cpu=beowulf+scrypt hdd=bitsync ram=?not sure? network=meshnet+btc market=OpenTransaction

    Just thoughts...

    Ultimately it will be as you say, just boot it up and get paid. For me the ideal would be to have it automatically pay for each of the services I'm using out of the credits from services I've provided so that as long as I'm providing more than I'm using I'm banking creds. I might have a ton of ram that pays for my internet usage. I think each resource should have it's own 'coin' and have them autotrade built into the protocol.

  3. Forgot to add, this needs to also extend to electrical production. The trick seems to be how to meter it but if this could be solved it would open new horizons in sustainable energy production in an inherently decentralized way. There are some nice charge controllers available now that may enable this.

  4. Quick plug for, which is a project in this space. Essay about it here:
    Also it's a bit early in the peice but I'm starting to flesh a document here about how others in this space could go about building a faux-geonetwork

  5. It is worth noting that these decentralized systems have some core process that it aligns with. The internet has the TCP/IP, HTTP, and HTML at its core, while Linux is at the center of the OpenSource Movement. Skype, a good example of a decentralized cloud application, runs its own system. The very essence of what made this decentralized system work was that it allowed its users to use its core to solve their own problems.

    Ruby Badcoe @ WilliamsDataManagement