BlockSci v0.7.0

Version 0.7.0 is based on the development branch v0.6, but is not compatible with v0.5 or v0.6 parsings and requires a full reparse of the blockchain.

Notable Changes

• Python: New fluent interface

  A new fluent Python interface allows to execute many operations (such as filtering transactions) efficiently in C++, resulting in a major performance increase.

• Parser support for CTOR (Canonical Transaction Ordering Rule)

  BlockSci's parser has been updated to support arbitrary transaction ordering rules within a block (e.g., Bitcoin Cash's canonical transaction ordering).

• Transaction Input<->Output mapping

  Inputs now reference the output they are spending, and vice versa. These can be looked up using blocksci.Output.spending_input and blocksci.Input.spent_output

• New config files to store configurations for different blockchains

  Settings for blockchain parsings are now stored in a JSON config file. Find more information in the setup instructions

• Testing: Python test suite and CI have been added

  We've added a small test suite for the Python interface. It uses a special regtest blockchain created using our testchain generator. Current test coverage is limited, and we welcome contributions to extend it.

Important bug fixes

• v0.5 contains a bug that causes reused addresses to receive a new ID, rather than the previously assigned ID. If the address had been used multiple times before, subsequent occurrences would receive the old ID again, resulting in lookups that show only one transaction associated with the address. If the address had been used only once before, a lookup will miss this previous occurrence. Bitcoin parsings beyond block height 572072 are affected. You can find more information in this document. (Issue #272)
• The parser now builds the index to look up wrapping addresses. Previously, when retrieving the equiv address from a wrapped address (e.g., P2PK that is wrapped by a P2SH address), it would not include the wrapping address. (PR #402)
• v0.6 only: incorrect handling of compressed public keys resulted in multisig addresses not being correctly deduplicated (i.e. they would receive a new ID on reuse). (Fixed on 01/31/2020, PR #367)

Other changes and bug fixes

• Multisig addresses with invalid public keys are considered non-standard
• Updated dependencies (including range-v3, pybind11 and RocksDB)
• blocksci.Address.[ins|outs|in_txes|out_txes|txes] now return iterators
• The blocksci.heuristics.change.ChangeHeuristic interface has been rewritten.
  • Heuristics now return an blocksci.OutputIterator instead of a set of outputs.
  • ChangeHeuristic.unique_change now returns a new ChangeHeuristic object, allowing to use it to compose with other change heuristics.
  • A new None heuristic has been added and is also the default heuristic for change address clustering (effectively disabling it).
  • A new Spent heuristic allows to refine heuristics that return unspent outputs as potential change outputs.
  • Clustering no longer performs change address clustering by default. You can still specify a change address heuristic to enable it.
• Added a new tool to the parser (blocksci config.json doctor) that can detect a few common issues with the setup
• Added a new tool (blocksci_check_integrity) that computes a hash value over the BlockSci data produced by the parser
• Correctly handle Schnorr signatures on Bitcoin Cash with a length of 65 bytes (PR #395)
• Changed in(s)/out(s) in method/property names to input(s)/output(s) to avoid confusion with incoming and outgoing funds (PR #392)
• Fixed an inconsistency in recording the tx that first spends a script (PR #385)
• chain.cpp.filter_tx has been removed in favor of the new fluent interface (Issue #254)
• Recognize address formats that use more than one version byte (Issue #246)
• The parser will detect if another instance is already running on the same data directory (Issue #211)
• blocksci.cluster.ClusterManager.create_clustering now accepts a start and end height for clustering only a specific block range (does not apply to linking of wrapped with wrapping addresses) (Issue #118)
• Fixed rounding inconsistencies for values in Zcash (Issue #117)
• Added Witness Unknown address type support (Issue #112)
• Added transaction version numbers (Issue #92)

Known bugs and limitations

• Performance of directly accessing addresses and iterators/ranges in the Python interface is slower than in v0.5 (only noticeable when accessing them in large volumes)
• Iterating over an AddressIterator in pure Python causes a segfault. Use .to_list() to retrieve a list of the results over which you can iterate.