Monday, February 25, 2013

Bitcoin block size thoughts

As pasted from an IRC discussion today:
  •  (a) I once posted a patch to change max block size, so I thought about this long before forum readers ever woke up to the issue,
  • (b) I have since backed down from that radical position,
  • (c) it seems likely that max block size will change sometime in bitcoin's future,
  • (d) block size is VERY MUCH like bitcoin's 21M limit, so a lot of care must be taken when changing MAX_BLOCK_SIZE logic.  Block size is an economically limited resource whose production is tightly defined and controlled by algorithm, with an intentionally steady production rate (the 1MB limit). 
  • (d.1) Nonewithstanding the major impacts of a hard fork, in and of itself.
  • (e) I lean against solutions that are feedback-based (average of last 1000 block sizes, etc.), as they can be gamed too easily
  • (f) implementation will likely be:  if (now > chosen hard fork future date) { do it }
  • (g) my default rhetorical position will be to push back against changes, for now, since there is no demonstrated need for hard fork.
  • (g.1) Block sizes are nowhere near maximum, and
  • (g.2) Competition for space encourages efficient solutions, whereas a too-loose block size policy incentivizes the opposite: dumping into the block chain
  • (h) when I ran asic + p2pool, I set max block size at 900k and free tx at 300k.  That reflects my personal (not official dev team etc.) preferences.  I filtered out 1e8 outputs from mempool. We are nowhere near competing for block space at this point.
  • And very importantly, (i) it is a mistake to increase block size simply because people are too lazy to implement layers on top of bitcoin.  Bitcoin will forever be a zen balance of applications and layers that sit on top of the blockchain, and those that directly use the blockchain itself as their comm/functional layer (c.f. SatoshiDICE).

So I generally agree with a lot of gmaxwell's points, and it is important therefore to not arbitrarily increase blocksize, simply because that makes some apps easier -- because they free-ride on the blockchain itself. Maybe one future state of The Mainnet Blockchain is simply a high security merged mining root for several important chains, who can say?

Block size is a core economic resource, like the number of bitcoins itself. Not merely the number of transactions we can support... it influences fees and many other factors.

My off-the-cuff guess (may be wrong) for a solution was:  if (todays_date > SOME_FUTURE_DATE) { MAX_BLOCK_SIZE *= 2, every 1 years }  [Other devs comment: too fast!]  That might be too fast, but the point is, not feedback based nor directly miner controlled.

Conclusions:

  1. Block size does not need to change right now.
  2. If and when block size changes, implement a simple rule, and let the free market figure out the rest (which might include a more complex rule years later, when the picture is more clear).
Standard disclaimer:  speaking only for myself, not any other dev or organization.

P.S. That was more than I intended to type, about block size.  It seems more like The Question Of The Moment on the web, than a real engineering need. Just The Thing people are talking about right now, and largely much ado about nothing.

17 comments:

  1. Jeff, if and when it is changed, what do you think of my thoughts here: https://bitcointalk.org/index.php?topic=145754.msg1551072#msg1551072

    ReplyDelete
  2. The whole thing is just a distraction from more important objectives and waste of human resources.

    ReplyDelete
  3. I can definitely understand these sentiments. One thing I haven't seen explained, though, is how the services built on top of the blockchain might work.

    What I wouldn't want to see is bitcoin-backed payment systems run by some central authority. PayPal could simply take over that space and continue screwing people just the same as they do now. If that happened, bitcoin as a payment system has failed, hasn't it?

    So I think some of the core devs have something more distributed in mind but I'm not sure how that work work and haven't seen it explained anywhere yet. Is there a link I could read?

    ReplyDelete
  4. In Bitcoin, we have the one the most amazing and useful tools since the discovery of electricity, brought into existence and maintained by the gargantuan efforts of some of the best coders alive.

    If the block size has to change for the survival of Bitcoin, I don't see what the kvetching is about from people that DON'T EVEN CODE. I don't code but I understand the reason for the future change. Non-coders need to understand the technical reasons for the change rather than react emotionally to a non-emotional issue.

    jg: "it is a mistake to increase block size simply because people are too lazy to implement layers on top of bitcoin."

    ^ This a million times. ^

    A challenge to holders of 100 btc or more: donate 1% of your btc holdings to a development project that seeks to lessen the load on the blockchain. Ripple is a good idea that needs better implementation but it's a step in the right direction. If 100 different groups are trying for a solution, odds are that one or two will find a solution.

    I've seen a lot of energy wasted on this topic that could have been better spent on more constructive efforts. If we really want this thing to work, let's help each other out rather than freak out because the protocol might have to change someday. We have a rare and historic chance to pass a working model idea of economic cooperation onto the next generation. If we miss this chance, we'll be just another economic footnote, like the Worgl Experiment or the Liberty Dollar.

    Sorry for the rant.

    /soapbox

    ReplyDelete
  5. I don't care about the particular details of how or when MAX_BLOCK_SIZE is determined, as long as the promise that it will be raised when needed is kept before it holds back adoption.

    I'm not opposed to layers on top of Bitcoin but it's a mistake to assume that's what the users of Bitcoin want and need. I get the strong impression that some people who want to promote their preferred alternatives aren't sure those alternatives will be compelling enough for people to adopt them unless Bitcoin is prohibited from becoming too useful.

    Don't hold artificially hold back Bitcoin to create a market for somebody's pet project. Let Bitcoin become as useful as possible and those other layers can be adopted or not based on their own merits.

    ReplyDelete
    Replies
    1. I agree - it seems like a bad idea to deliberately allow bitcoin to remain crippled (in terms of maximum transaction volume) in the hopes that people will work around it, when there's still apparently *plenty* of growing room left before the transaction volume would actually overwhelm available network and storage resources.

      What if email had been replaced with a variant that required fees to send email as soon as bit became clear that spam would be a problem? That probably would have destroyed email. If people had to pay, they wouldn't have used it.

      I think if bitcoin requires all these extra hoops and complexity in order to send someone ten cents, then they aren't going to bother. To be honest it's difficult enough already.

      Adoption should trump all at this point in time.

      Yes, eventually bitcoin would crack under the load of micropayments. However that's a far better problem to have than for potential users to just walk away.

      However, as Jeff points out we still aren't close to the 1mb limit, so maybe by the time we get there, people won't be able to walk away so easily.

      Seems like raising the limit (one time only) would help guarantee that.

      Delete
    2. We are close to the limit in exponential terms.

      The number of transactions only needs to double 2 more times before the block size reaches 1 MB.

      Delete
  6. Open-transactions enables off-blockchain transactions, truly anonymous digital cash, and instant finality of settlement. And that is just the tip of the iceberg.

    https://github.com/FellowTraveler/Open-Transactions

    ReplyDelete
  7. (Doubling every year is way too fast: as an upper limit it should be pegged to the growth of storage density, i.e. double every 2-4 years.)

    ReplyDelete
  8. I think the fundamental problem is big sizes have a long term cost that the miners don't necessarily bear (i.e. part of the cost is an externality). They make it more expensive to store the block chain and more expensive to download it. The miner considering the reward and fees for the current block doesn't necessarily pay for all those costs.

    If we could somehow make miners pay the whole cost then the block size will be set by market forces. I don't have any idea as to how to do that though.

    ReplyDelete
  9. I think your thoughts are well rounded and rational. I'd only have a couple things to say...

    when you say block size is supposed to be an economic scarcity isn't that the same thing as saying that you want transactions to be an economic scarcity? That makes sense but it also presents a potential problem. The users want cheap transactions but at the same time we want a reason for miners to continue using their resources to secure the network so we need fees. With too many free transactions fees become less useful. But we don't know how scarce they should be and if there should come a point when the 1mb limit presents too much scarcity then it acts like a bottleneck. I would say the point where satoshi dice is forced off the chain is not yet a bottleneck.

    The other thing I was going to say... i don't think the block size limit would be a big deal at all if it didn't require a hard fork. If it didn't require a hard fork nobody would be worrying about it.

    ReplyDelete
  10. Jeff, if this constant is economically limited, let the transaction fees to expand it. We need a formula which will put maxblocksize into direct proportion to the current block fees.

    ReplyDelete
  11. I disagree with this. We can talk about how we're all supposed to be using "layers" on top of Bitcoin all we want, but the fact is that a major reason why a lot of people are interested in Bitcoin is specifically because you do not need to rely on third-party services to do anything - a simple Bitcoin client and the raw blockchain is all you need. The negligible fees are another major factor. We have BitcoinQt 0.8, so we have a standardized way for light clients to continue functioning with a blockchain going into the tens and hundreds of GB; there is no need to artificially enforce scarcity to prevent this. As for the argument that we need to keep transactions a scarce resource, once again, an emphatic no. The purpose of Bitcoin is NOT to make miners happy. The purpose of miners is to make Bitcoin users happy. And if millions of transactions a day is what Bitcoin users want, then so be it.

    I agree with Jeff above that "adoption should trump all".

    Sure, we could put this off until it's clear that we have to do something, but I'm not so optimistic that it'll be easy. With every passing month, the mining community is going to get more scattered and diffuse, and pushing through changes may get harder. It may not, but the future is uncertain. The present, on the other hand, is certain. It's better to do these things while we know we still can.

    ReplyDelete
  12. Why not use the number of fee-including transactions from previous blocks to determine future blocks size? Sure, it could still be gamed, but there would be a cost to do so. You would have to be careful to balance the costs against potential gains, but it seems to me that an appropriate balance could be struck. Free transactions would have no effect on blocksize, but could still tag along if there is available space.

    ReplyDelete
  13. Bitcoin does not necessarily have to be just about low fees and micro-transactions.

    The free market decides the BTC market price and the free market will decide transaction fees (based on desired transaction times). A state of no block size limits avoids the market mechanism for pricing transaction times. As Jeff says, it would be similar to tinkering with, or letting the market decide, the 21 million limit. So it must be adjusted with great care.

    If (and when) an increase is warranted, I would prefer a predictable increase path that is perhaps linked to the halving of the block reward. Those are checkpoints in time that can be anticipated and it also logically follows that allowing for a greater number of transactions at the same time as a block reward reduction would tend to equalize average transaction fees.

    ReplyDelete
  14. If Bitcoin becomes limited by transaction volume and has market driven fees, the most it could ever achieve is being a reserve currency. Forget the POS or virtual cash p2p exchange. Mind you, that'd still be an amazing feat, becoming a reserve currency, solving the outright theft or redistribution of wealth by quantitive easing and other inflation promoting measures.

    I believed bitcoin would be a currency for the people. Not just rich people but everyone. Even a farmer near Timbuktu can pay his workers their daily wages of $0.50 feasibly through bitcoin, for example. I also believed bitcoin would replace everyday banking, allowing the individual to decide whether they want their funds to be held in full reserve or fractional reserve.

    We have an inverse race condition. Bitcoin could be adopted more widely but since many are aware that the currency has limitations, adoption or investment is slower than it could/should be. People like yourself say, "We can fix this when we need to" but since adoption is slow, there won't be a need for a very long time. Fix it and they will come.

    I've heard others state the same; Make bitcoin all it *can* be and we will all benefit from a world-class *ideal* currency system.

    The problem, I see, is that the devs may have the skills but no vision to motivate them. If I had the expertise to patch bitcoin and the time, I'd get more involved myself. If I had a commercial interest in bitcoin, aside from holding a few coins, I'd be sponsoring a team of devs to fix these issues. If only...

    ReplyDelete
  15. I am a newbie into BTC in 3 days. Here's my question and I am sorry if any guru here find the question ignorant

    1. The Block Size, Transaction Fee incentives ( become bankers ) must be a dynamic variable depending on the scarcity with exponential function. You cannot set 4 years reduce from 50 BTC to 25 BTC over 4 years and 0.005BTC fee for miners payment as a fixed amount. I believe the rewards must be gradual and change according to demands. More transactions will offer higher fee to promote speed competition but agreed and shared equally by sender and receiver.

    Current transaction is a one way but not 2 way agreement. Even getting paid must have acknowledgement and lifespan. Imagine some send all the 21 million BTC to a wallet which disappear and dead. Does this mean the BTC cloud has no more BTC unit and forever hide in this wallet?

    I want the electronic money to introduce a penalty for dead capital which mean all BTC unit will disappear gradually is left dead and unused in the wallet and the speed grows exponentially. Which mean a BTC lost in a dead wallet should be deleted over a period of time unless transfer to a new wallet and suffer transactions fee as a consequence. This come into the time factor. There is no time factor in the BTC system.

    ReplyDelete