From 854f98fe439c6dc1901c850788e7fcc967a7e7b8 Mon Sep 17 00:00:00 2001 From: Braydon Fuller Date: Mon, 31 Aug 2015 09:00:00 -0400 Subject: [PATCH] Update Documentation to reflect Service Architecture --- LICENSE | 29 +-- README.md | 394 ++-------------------------------- docs/build.md | 92 ++++++++ docs/bus.md | 32 +++ docs/errors.md | 17 ++ docs/node.md | 41 ++++ docs/patch.md | 7 + RELEASE.md => docs/release.md | 6 +- docs/scaffold.md | 32 +++ docs/services.md | 56 +++++ docs/services/address.md | 35 +++ docs/services/bitcoind.md | 20 ++ docs/services/db.md | 34 +++ docs/testing.md | 52 +++++ 14 files changed, 445 insertions(+), 402 deletions(-) create mode 100644 docs/build.md create mode 100644 docs/bus.md create mode 100644 docs/errors.md create mode 100644 docs/node.md create mode 100644 docs/patch.md rename RELEASE.md => docs/release.md (97%) create mode 100644 docs/scaffold.md create mode 100644 docs/services.md create mode 100644 docs/services/address.md create mode 100644 docs/services/bitcoind.md create mode 100644 docs/services/db.md create mode 100644 docs/testing.md diff --git a/LICENSE b/LICENSE index d3e4d2e1..3f3b4df3 100644 --- a/LICENSE +++ b/LICENSE @@ -1,30 +1,7 @@ -bitcoind,js --------------------------------------------------------------------------------- +Copyright (c) 2014-2015 BitPay, Inc. -Copyright (c) 2014-2015, BitPay - -Permission is hereby granted, free of charge, to any person obtaining a copy -of this software and associated documentation files (the "Software"), to deal -in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is -furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included in -all copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN -THE SOFTWARE. - -bcoin --------------------------------------------------------------------------------- - -Copyright Fedor Indutny, 2014. +Parts of this software are based on Bitcoin Core +Copyright (c) 2009-2015 The Bitcoin Core developers Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal diff --git a/README.md b/README.md index 6b5df53c..3b56ca2c 100644 --- a/README.md +++ b/README.md @@ -1,398 +1,47 @@ Bitcore Node -======= +============ -A Node.js module that adds a native interface to Bitcoin Core for querying information about the Bitcoin blockchain. Bindings are linked to Bitcoin Core compiled as a static library. +A Bitcoin full node for building applications and services with Node.js. A node is extensible and can be configured to run additional services. At the minimum a node has native bindings to Bitcoin Core with the [Bitcoin Service](docs/services/bitcoind.md). Additional services can be enabled to make a node more useful such as exposing new APIs, adding new indexes for addresses with the [Address Service](docs/services/address.md), running a block explorer, wallet service, and other customizations. ## Install -Here is how you can you install and start your node: - ```bash -npm install -g bitcore-node@0.2.0-beta.4 +npm install -g bitcore-node@0.2.0-beta.5 bitcore-node start ``` -Note: For your convenience, we distribute binaries for x86_64 Linux and x86_64 Mac OS X. Upon npm install, the binaries for your platform will be downloaded. If you want to compile the project yourself, then please see the [Build & Install](#build--install) for full detailed instructions to build the project from source. +Note: For your convenience, we distribute binaries for x86_64 Linux and x86_64 Mac OS X. Upon npm install, the binaries for your platform will be downloaded. For more detailed installation instructions, or if you want to compile the project yourself, then please see the [Build & Install](docs/build.md) documentation to build the project from source. ## Configuration -Bitcore Node includes a Command Line Interface (CLI) for managing, configuring and interfacing with your Bitcore Node. At the minimum, your node can function with all of the features from Bitcoin Core running as a full node. However you can enable additional features to make your node more useful such as exposing new APIs, adding new indexes for addresses, running a block explorer and custom modules. +Bitcore Node includes a Command Line Interface (CLI) for managing, configuring and interfacing with your Bitcore Node. ```bash bitcore-node create -d mynode "My Node" cd mynode -bitcore-node add +bitcore-node add bitcore-node add https://github.com/yourname/helloworld ``` -This will create a directory with configuration files for your node and install the necessary dependencies. If you're interested in developing a module, please see the [Module Development Guide](#modules). +This will create a directory with configuration files for your node and install the necessary dependencies. For more information about (and developing) services, please see the [Service Documentation](docs/services.md). -## Build & Install +## Documentation -This includes a detailed instructions for compiling. There are two main parts of the build, compiling Bitcoin Core as a static library and the Node.js bindings. +- [Services](docs/services.md) + - [Bitcoind](docs/services/bitcoind.md) - Native bindings to Bitcoin Core + - [Database](docs/services/db.md) - The foundation API methods for getting information about blocks and transactions. + - [Address](docs/services/address.md) - Adds additional API methods for querying and subscribing to events with bitcoin addresses. +- [Build & Install](docs/build.md) - How to build and install from source +- [Testing & Development](docs/testing.md) - Developer guide for testing +- [Node](docs/node.md) - Details on the node constructor +- [Bus](docs/bus.md) - Overview of the event bus constructor +- [Errors](docs/errors.md) - Reference for error handling and types +- [Patch](docs/patch.md) - Information about the patch applied to Bitcoin Core +- [Release Process](docs/release.md) - Information about verifying a release and the release process. -### Ubuntu 14.04 (Unix/Linux) +## Contributing -If git is not already installed, it can be installed by running: - -```bash -sudo apt-get install git -git config --global user.email "you@example.com" -git config --global user.name "Your Name" -``` - -If Node.js v0.12 isn't installed, it can be installed using "nvm", it can be done by following the installation script at https://github.com/creationix/nvm#install-script and then install version v0.12 - -```bash -nvm install v0.12 -``` - -To build Bitcoin Core and bindings development packages are needed: - -```bash -sudo apt-get install build-essential libtool autotools-dev automake autoconf pkg-config libssl-dev -``` - -Clone the bitcore-node repository locally: - -```bash -git clone https://github.com/bitpay/bitcore-node.git -cd bitcore-node -``` - -And finally run the build which will take several minutes. A script in the "bin" directory will download Bitcoin Core v0.11, apply a patch (see more info below), and compile the static library and Node.js bindings. You can start this by running: - -```bash -npm install -``` -Once everything is built, you can run bitcore-node via: - -```bash -npm start -``` -This will then start the syncing process for Bitcoin Core and the extended capabilities as provided by the built-in Address Module (details below). - -### Fedora - -Later versions of Fedora (>= 22) should also work with this project. The directions for Ubuntu should generally work except the installation of system utilities and libraries is a bit different. Git is already installed and ready for use without installation. - -```bash -yum install libtool automake autoconf pkgconfig openssl make gcc gcc-c++ kernel-devel openssl-devel.x86_64 patch -``` - -### Mac OS X Yosemite - -If Xcode is not already installed, it can be installed via the Mac App Store (will take several minutes). XCode includes "Clang", "git" and other build tools. Once Xcode is installed, you'll then need to install "xcode-select" via running in a terminal and following the prompts: - -```bash -xcode-select --install -``` - -If "Homebrew" is not yet installed, it's needed to install "autoconf" and others. You can install it using the script at http://brew.sh and following the directions at https://github.com/Homebrew/homebrew/blob/master/share/doc/homebrew/Installation.md And then run in a terminal: - -```bash -brew install autoconf automake libtool openssl pkg-config -``` - -If Node.js v0.12 and associated commands "node", "npm" and "nvm" are not already installed, you can use "nvm" by running the script at https://github.com/creationix/nvm#install-script And then run this command to install Node.js v0.12 - -```bash -nvm install v0.12 -``` - -Clone the bitcore-node repository locally: - -```bash -git clone https://github.com/bitpay/bitcore-node.git -cd bitcore-node -``` - -And finally run the build which will take several minutes. A script in the "bin" directory will download Bitcoin Core v0.11, apply a patch (see more info below), and compile the static library and Node.js bindings. You can start this by running: - -```bash -npm install -``` -Once everything is built, you can run bitcore-node via: - -```bash -npm start -``` - -This will then start the syncing process for Bitcoin Core and the extended capabilities as provided by the built-in Address Module (details below). - -## Development & Testing - -To run all of the JavaScript tests: - -```bash -npm run test -``` - -To run tests against the bindings, as defined in `bindings.gyp` the regtest feature of Bitcoin Core is used, and to enable this feature we currently need to build with the wallet enabled *(not a part of the regular build)*. To do this, export an environment variable and recompile: - -```bash -export BITCORENODE_ENV=test -npm run build -``` - -If you do not already have mocha installed: - -```bash -npm install mocha -g -``` - -To run the integration tests: - -```bash -mocha -R spec integration/regtest.js -``` - -If any changes have been made to the bindings in the "src" directory, manually compile the Node.js bindings, as defined in `bindings.gyp`, you can run (-d for debug): - -```bash -$ node-gyp -d rebuild -``` - -Note: `node-gyp` can be installed with `npm install node-gyp -g` - -To be able to debug you'll need to have `gdb` and `node` compiled for debugging with gdb using `--gdb` (sometimes called node_g), and you can then run: - -```bash -$ gdb --args node examples/node.js -``` - -To run mocha from within gdb (notice `_mocha` and not `mocha` so that the tests run in the same process): -```bash -$ gdb --args node /path/to/_mocha -R spec integration/regtest.js -``` - -To run the benchmarks: - -```bash -$ cd benchmarks -$ node index.js -``` - -## Static Library Patch - -To provide native bindings to JavaScript *(or any other language for that matter)*, Bitcoin code, itself, must be linkable. Currently, Bitcoin Core provides a JSON RPC interface to bitcoind as well as a shared library for script validation *(and hopefully more)* called libbitcoinconsensus. There is a node module, [node-libbitcoinconsensus](https://github.com/bitpay/node-libbitcoinconsensus), that exposes these methods. While these interfaces are useful for several use cases, there are additional use cases that are not fulfilled, and being able to implement customized interfaces is necessary. To be able to do this a few simple changes need to be made to Bitcoin Core to compile as a static library. - -The patch is located at `etc/bitcoin.patch` and adds a configure option `--enable-daemonlib` to compile all object files with `-fPIC` (Position Independent Code - needed to create a shared object), exposes leveldb variables and objects, exposes the threadpool to the bindings, and conditionally includes the main function. - -Every effort will be made to ensure that this patch stays up-to-date with the latest release of Bitcoin. At the very least, this project began supporting Bitcoin Core v0.11. - -## Example Usage - -```js - -var BitcoinNode = require('bitcore-node').Node; - -var configuration = { - datadir: '~/.bitcoin', - network: 'testnet' -}; - -var node = new BitcoinNode(configuration); - -node.on('ready', function() { - console.log('Bitcoin Node Ready'); -}); - -node.on('error', function(err) { - console.error(err); -}); - -node.chain.on('addblock', function(block) { - console.log('New Best Tip:', block.hash); -}); - -``` - -## API Documentation - -Get Unspent Outputs - -```js -var address = '15vkcKf7gB23wLAnZLmbVuMiiVDc1Nm4a2'; -var includeMempool = true; -node.getUnspentOutputs(address, includeMempool, function(err, unspentOutputs) { - //... -}); -``` - -View Balances - -```js -var address = '15vkcKf7gB23wLAnZLmbVuMiiVDc1Nm4a2'; -var includeMempool = true; -node.getBalance(address, includeMempool, function(err, balance) { - //... -}); -``` - -Get Outputs - -```js -var address = '15vkcKf7gB23wLAnZLmbVuMiiVDc1Nm4a2'; -var includeMempool = true; -node.getOutputs(address, includeMempool, function(err, outputs) { - //... -}); -``` - -Get Transaction - -```js -var txid = 'c349b124b820fe6e32136c30e99f6c4f115fce4d750838edf0c46d3cb4d7281e'; -var includeMempool = true; -node.getTransaction(txid, includeMempool, function(err, transaction) { - //... -}); -``` - -Get Block - -```js -var blockHash = '00000000d17332a156a807b25bc5a2e041d2c730628ceb77e75841056082a2c2'; -node.getBlock(blockHash, function(err, block) { - //... -}); -``` - -You can log output from the daemon using: - -``` bash -$ tail -f ~/.bitcoin/debug.log -``` - -^C (SIGINT) will call `StartShutdown()` in bitcoind on the node thread pool. - -## Modules - -Bitcore Node has a module system where additional information can be indexed and queried from -the blockchain. One built-in module is the address module which exposes the API methods for getting balances and outputs. - -### Writing a Module - -A new module can be created by inheriting from `Node.Module`, implementing the methods `blockHandler()`, `getAPIMethods()`, `getPublishEvents()` and any additional methods for querying the data. Here is an example: - -```js -var inherits = require('util').inherits; -var Node = require('bitcore-node').Node; - -var MyModule = function(options) { - Node.Module.call(this, options); -}; - -inherits(MyModule, Node.Module); - -/** - * blockHandler - * @param {Block} block - the block being added or removed from the chain - * @param {Boolean} add - whether the block is being added or removed - * @param {Function} callback - call with the leveldb database operations to perform - */ -MyModule.prototype.blockHandler = function(block, add, callback) { - var transactions = block.transactions; - // loop through transactions and outputs - // call the callback with leveldb database operations - var operations = []; - if(add) { - operations.push({ - type: 'put', - key: 'key', - value: 'value' - }); - } else { - operations.push({ - type: 'del', - key: 'key' - }); - } - - // If your function is not asynchronous, it is important to use setImmediate. - setImmediate(function() { - callback(null, operations); - }); -}; - -/** - * the API methods to expose - * @return {Array} return array of methods - */ -MyModule.prototype.getAPIMethods = function() { - return [ - ['getData', this, this.getData, 1] - ]; -}; - -/** - * the bus events available for subscription - * @return {Array} array of events - */ -MyModule.prototype.getPublishEvents = function() { - return [ - { - name: 'custom', - scope: this, - subscribe: this.subscribeCustom, - unsubscribe: this.unsubscribeCustom - } - ] -}; - -/** - * Will keep track of event listeners to later publish and emit events. - */ -MyModule.prototype.subscribeCustom = function(emitter, param) { - if(!this.subscriptions[param]) { - this.subscriptions[param] = []; - } - this.subscriptions[param].push(emitter); -} - -MyModule.prototype.getData = function(arg1, callback) { - // You can query the data by reading from the leveldb store on db - this.node.db.store.get(arg1, callback); -}; - -module.exports = MyModule; -``` - -The module can then be used when running a node: - -```js -var configuration = { - datadir: process.env.BITCORENODE_DIR || '~/.bitcoin', - modules: [MyModule] -}; - -var node = new Node(configuration); - -node.on('ready', function() { - node.getData('key', function(err, value) { - console.log(err || value); - }); -}); -``` - -Note that if you already have a bitcore-node database, and you want to query data from previous blocks in the blockchain, you will need to reindex. Reindexing right now means deleting your bitcore-node database and resyncing. - -## Daemon Documentation - -- `daemon.start([options], [callback])` - Start the JavaScript Bitcoin node. -- `daemon.getBlock(blockHash|blockHeight, callback)` - Get any block asynchronously by block hash or height as a node buffer. -- `daemon.isSpent(txid, outputIndex)` - Returns a boolean if a txid and outputIndex is already spent. -- `daemon.getBlockIndex(blockHash)` - Will return the block chain work and previous hash. -- `daemon.estimateFee(blocks)` - Estimates the fees required to have a transaction included in the number of blocks specified as the first argument. -- `daemon.sendTransaction(transaction, allowAbsurdFees)` - Will attempt to add a transaction to the mempool and broadcast to peers. -- `daemon.getTransaction(txid, queryMempool, callback)` - Get any tx asynchronously by reading it from disk, with an argument to optionally not include the mempool. -- `daemon.getTransactionWithBlockInfo(txid, queryMempool, callback)` - Similar to getTransaction but will also include the block timestamp and height. -- `daemon.getMempoolOutputs(address)` - Will return an array of outputs that match an address from the mempool. -- `daemon.getInfo()` - Basic information about the chain including total number of blocks. -- `daemon.isSynced()` - Returns a boolean if the daemon is fully synced (not the initial block download) -- `daemon.syncPercentage()` - Returns the current estimate of blockchain download as a percentage. -- `daemon.stop([callback])` - Stop the JavaScript bitcoin node safely, the callback will be called when bitcoind is closed. This will also be done automatically on `process.exit`. It also takes the bitcoind node off the libuv event loop. If the daemon object is the only thing on the event loop. Node will simply close. +Please send pull requests for bug fixes, code optimization, and ideas for improvement. For more information on how to contribute, please refer to our [CONTRIBUTING](https://github.com/bitpay/bitcore/blob/master/CONTRIBUTING.md) file. ## License @@ -401,4 +50,3 @@ Code released under [the MIT license](https://github.com/bitpay/bitcore-node/blo Copyright 2013-2015 BitPay, Inc. - bitcoin: Copyright (c) 2009-2015 Bitcoin Core Developers (MIT License) -- bcoin (some code borrowed temporarily): Copyright Fedor Indutny, 2014. diff --git a/docs/build.md b/docs/build.md new file mode 100644 index 00000000..604d11de --- /dev/null +++ b/docs/build.md @@ -0,0 +1,92 @@ +## Build & Install + +This includes a detailed instructions for compiling. There are two main parts of the build, compiling Bitcoin Core as a static library and the Node.js bindings. + +## Ubuntu 14.04 (Unix/Linux) + +If git is not already installed, it can be installed by running: + +```bash +sudo apt-get install git +git config --global user.email "you@example.com" +git config --global user.name "Your Name" +``` + +If Node.js v0.12 isn't installed, it can be installed using "nvm", it can be done by following the installation script at https://github.com/creationix/nvm#install-script and then install version v0.12 + +```bash +nvm install v0.12 +``` + +To build Bitcoin Core and bindings development packages are needed: + +```bash +sudo apt-get install build-essential libtool autotools-dev automake autoconf pkg-config libssl-dev +``` + +Clone the bitcore-node repository locally: + +```bash +git clone https://github.com/bitpay/bitcore-node.git +cd bitcore-node +``` + +And finally run the build which will take several minutes. A script in the "bin" directory will download Bitcoin Core v0.11, apply a patch (see more info below), and compile the static library and Node.js bindings. You can start this by running: + +```bash +npm install +``` +Once everything is built, you can run bitcore-node via: + +```bash +npm start +``` +This will then start the syncing process for Bitcoin Core and the extended capabilities as provided by the built-in Address Module (details below). + +## Fedora + +Later versions of Fedora (>= 22) should also work with this project. The directions for Ubuntu should generally work except the installation of system utilities and libraries is a bit different. Git is already installed and ready for use without installation. + +```bash +yum install libtool automake autoconf pkgconfig openssl make gcc gcc-c++ kernel-devel openssl-devel.x86_64 patch +``` + +## Mac OS X Yosemite + +If Xcode is not already installed, it can be installed via the Mac App Store (will take several minutes). XCode includes "Clang", "git" and other build tools. Once Xcode is installed, you'll then need to install "xcode-select" via running in a terminal and following the prompts: + +```bash +xcode-select --install +``` + +If "Homebrew" is not yet installed, it's needed to install "autoconf" and others. You can install it using the script at http://brew.sh and following the directions at https://github.com/Homebrew/homebrew/blob/master/share/doc/homebrew/Installation.md And then run in a terminal: + +```bash +brew install autoconf automake libtool openssl pkg-config +``` + +If Node.js v0.12 and associated commands "node", "npm" and "nvm" are not already installed, you can use "nvm" by running the script at https://github.com/creationix/nvm#install-script And then run this command to install Node.js v0.12 + +```bash +nvm install v0.12 +``` + +Clone the bitcore-node repository locally: + +```bash +git clone https://github.com/bitpay/bitcore-node.git +cd bitcore-node +``` + +And finally run the build which will take several minutes. A script in the "bin" directory will download Bitcoin Core v0.11, apply a patch (see more info below), and compile the static library and Node.js bindings. You can start this by running: + +```bash +npm install +``` +Once everything is built, you can run bitcore-node via: + +```bash +npm start +``` + +This will then start the syncing process for Bitcoin Core and the extended capabilities as provided by the built-in Address Module (details below). \ No newline at end of file diff --git a/docs/bus.md b/docs/bus.md new file mode 100644 index 00000000..38e3a108 --- /dev/null +++ b/docs/bus.md @@ -0,0 +1,32 @@ +# Bus + +The bus provides a way to subscribe to events from any of the services running. It's implemented abstract from transport specific implementation. The primary use of the bus in Bitcore Node is for subscribing to events via a web socket. + +## Opening/Closing + +```javascript + +// a node is needed to be able to open a bus +var node = new Node(configuration); + +// will create a new bus that is ready to subscribe to events +var bus = node.openBus(); + +// will remove all event listeners +bus.close(); +``` + +## Subscribing/Unsubscribing + +```javascript + +// subscribe to all transaction events +bus.subscribe('transaction'); + +// only subscribe to events relevant to a bitcoin address +bus.subscribe('address/transaction', ['13FMwCYz3hUhwPcaWuD2M1U2KzfTtvLM89']); + +// unsubscribe +bus.unsubscribe('transaction'); +``` + diff --git a/docs/errors.md b/docs/errors.md new file mode 100644 index 00000000..eeb9b487 --- /dev/null +++ b/docs/errors.md @@ -0,0 +1,17 @@ +# Errors + +Many times there are cases where an error condition can be gracefully handled depending on a particular use. To assist in better error handling, errors will have different types so that it's possible to determine the type of error and handle appropriatly. + +```js +node.services.address.getUnspentOutputs('00000000839a8...', function(err, outputs) { + + if (err instanceof errors.NoOutputs) { + // the address hasn't received any transactions + } + + // otherwise the address has outputs (which may be unspent/spent) + +}); +``` + +For more information about different types of errors, please see `lib/errors.js`. \ No newline at end of file diff --git a/docs/node.md b/docs/node.md new file mode 100644 index 00000000..4cc9ef93 --- /dev/null +++ b/docs/node.md @@ -0,0 +1,41 @@ +# Node + +A node represents a collection of services that are loaded together. For more information about services, please see the [Services Documentation](services.md). + +## API Documentation + +- `start()` - Will start the node's services in the correct order based on the dependencies of a service. +- `stop()` - Will stop the node's services. +- `openBus()` - Will create a new event bus to subscribe to events. +- `getAllAPIMethods()` - Returns information about all of the API methods from the services. +- `getAllPublishEvents()` - Returns information about publish events. +- `getServiceOrder()` - Returns an array of service modules. +- `services..` - Additional API methods exposed by each service. The services for the node are defined when the node instance is constructed. + +## Example Usage + +```js + +var BitcoinNode = require('bitcore-node').Node; + +var configuration = { + datadir: '~/.bitcoin', + network: 'testnet' +}; + +var node = new BitcoinNode(configuration); + +node.on('ready', function() { + console.log('Bitcoin Node Ready'); +}); + +node.on('error', function(err) { + console.error(err); +}); + +// shutdown the node +node.stop(function() { + // the shutdown is complete +}); + +``` diff --git a/docs/patch.md b/docs/patch.md new file mode 100644 index 00000000..ef954572 --- /dev/null +++ b/docs/patch.md @@ -0,0 +1,7 @@ +# Static Library Patch + +To provide native bindings to JavaScript *(or any other language for that matter)*, Bitcoin code, itself, must be linkable. Currently, Bitcoin Core provides a JSON RPC interface to bitcoind as well as a shared library for script validation *(and hopefully more)* called libbitcoinconsensus. There is a node module, [node-libbitcoinconsensus](https://github.com/bitpay/node-libbitcoinconsensus), that exposes these methods. While these interfaces are useful for several use cases, there are additional use cases that are not fulfilled, and being able to implement customized interfaces is necessary. To be able to do this a few simple changes need to be made to Bitcoin Core to compile as a static library. + +The patch is located at `etc/bitcoin.patch` and adds a configure option `--enable-daemonlib` to compile all object files with `-fPIC` (Position Independent Code - needed to create a shared object), exposes leveldb variables and objects, exposes the threadpool to the bindings, and conditionally includes the main function. + +Every effort will be made to ensure that this patch stays up-to-date with the latest release of Bitcoin. At the very least, this project began supporting Bitcoin Core v0.11. \ No newline at end of file diff --git a/RELEASE.md b/docs/release.md similarity index 97% rename from RELEASE.md rename to docs/release.md index 0df761c4..0f1f4f71 100644 --- a/RELEASE.md +++ b/docs/release.md @@ -1,8 +1,8 @@ -## Release Process +# Release Process Binaries for the C++ binding file (which includes libbitcoind statically linked in) are distributed for convenience. The binary binding file `bitcoind.node` is signed and published to S3 for later download and installation. Source files can also be built if binaries are not desired. -### How to Verify Signatures +## How to Verify Signatures ``` cd build/Release @@ -15,7 +15,7 @@ To verify signatures, use the following PGP keys: - @kleetus: https://pgp.mit.edu/pks/lookup?op=get&search=0x33195D27EF6BDB7F - @pnagurny: https://pgp.mit.edu/pks/lookup?op=get&search=0x0909B33F0AA53013 -### How to Release +## How to Release Ensure you've followed the instructions in the README.md for building the project from source. When building for any platform, be sure to keep in mind the minimum supported C and C++ system libraries and build from source using this library. Example, Ubuntu 12.04 has the earliest system library for Linux that we support, so it would be easiest to build the Linux artifact using this version. You will be using node-gyp to build the C++ bindings. A script will then upload the bindings to S3 for later use. You will also need credentials for BitPay's bitcore-node S3 bucket and be listed as an author for the bitcore-node's npm module. diff --git a/docs/scaffold.md b/docs/scaffold.md new file mode 100644 index 00000000..f3eccf2d --- /dev/null +++ b/docs/scaffold.md @@ -0,0 +1,32 @@ +# Scaffold + +A collection of functions for creating, managing, starting, stopping and interacting with a Bitcore Node. + +## Create + +This function will create a new directory and the initial configuration files/directories, including 'bitcore-node.json', 'package.json', 'bitcoin.conf', install the necessary Node.js modules, and create a data directory. + +## Add + +This function will add a service to a node by installing the necessary dependencies and modifying the `bitcore-node.json` configuration. + +## Start + +This function will load a configuration file `bitcore-node.json` and instantiate and start a node based on the configuration. + +## Find Config + +This function will recursively find a configuration `bitcore-node.json` file in parent directories and return the result. + +## Default Config + +This function will return a default configuration with the default services based on environment variables, and will default to using the standard `~/.bitcoin` data directory. + +## Remove + +This function will remove a service from a node by uninstalling the necessary dependencies and modifying the `bitcore-node.json` configuration. + +## Call Method + +This function will call an API method on a node via the JSON-RPC interface. + diff --git a/docs/services.md b/docs/services.md new file mode 100644 index 00000000..a4c04bb9 --- /dev/null +++ b/docs/services.md @@ -0,0 +1,56 @@ +# Services + +## Available Services + +- [Bitcoin Daemon](services/bitcoind.md) +- [DB](services/db.md) +- [Address](services/address.md) + +## Overview + +Bitcore Node has a service module system that can start up additional services that can include additional: + +- Blockchain indexes (e.g. querying balances for addresses) +- API methods +- HTTP routes +- Event types to publish and subscribe + +The `bitcore-node.json` file describes which services will load for a node: + +```json +{ + "services": [ + "bitcoind", "db", "address", "insight-api" + ] +} +``` + +Services correspond with a Node.js module as described in 'package.json', for example: + +```json +{ + "dependencies": { + "bitcore": "^0.13.1", + "bitcore-node": "^0.2.0", + "insight-api": "^3.0.0" + } +} +``` + +*Note:* If you already have a bitcore-node database, and you want to query data from previous blocks in the blockchain, you will need to reindex. Reindexing right now means deleting your bitcore-node database and resyncing. + +## Writing a Service + +A new service can be created by inheriting from `Node.Service` and implementing these methods and properties: + +- `Service.dependencies` - An array of services that are needed, this will determine the order that services are started on the node. +- `Service.prototype.start()` - Called to start up the service. +- `Service.prototype.stop()` - Called to stop the service. +- `Service.prototype.blockHandler()` - Will be called when a block is added or removed from the chain, and is useful for updating a database view/index. +- `Service.prototype.getAPIMethods()` - Describes which API methods that this service includes, these methods can then be called over the JSON-RPC API, as well as the command-line utility. +- `Service.prototype.getPublishEvents()` - Describes which events can be subscribed to for this service, useful to subscribe to events over the included web socket API. +- `Service.prototype.setupRoutes()` - A service can extend HTTP routes on an express application by implementing this method. + +The `package.json` for the service module can either export the `Node.Service` directly, or specify a specific module to load by including `"bitcoreNode": "lib/bitcore-node.js"`. + +Please take a look at some of the existing services for implemenation specifics. diff --git a/docs/services/address.md b/docs/services/address.md new file mode 100644 index 00000000..63c4f0ff --- /dev/null +++ b/docs/services/address.md @@ -0,0 +1,35 @@ +# Address Service + +The address service builds on the [Bitcoin Service](bitcoind.md) and the [Database Service](db.md) to add additional functionality for querying and subscribing to information based on bitcoin addresses. + +## API Documentation + +Get Unspent Outputs + +```js +var address = '15vkcKf7gB23wLAnZLmbVuMiiVDc1Nm4a2'; +var includeMempool = true; +node.getUnspentOutputs(address, includeMempool, function(err, unspentOutputs) { + //... +}); +``` + +View Balances + +```js +var address = '15vkcKf7gB23wLAnZLmbVuMiiVDc1Nm4a2'; +var includeMempool = true; +node.getBalance(address, includeMempool, function(err, balance) { + //... +}); +``` + +Get Outputs + +```js +var address = '15vkcKf7gB23wLAnZLmbVuMiiVDc1Nm4a2'; +var includeMempool = true; +node.getOutputs(address, includeMempool, function(err, outputs) { + //... +}); +``` diff --git a/docs/services/bitcoind.md b/docs/services/bitcoind.md new file mode 100644 index 00000000..39b469f0 --- /dev/null +++ b/docs/services/bitcoind.md @@ -0,0 +1,20 @@ +# Bitcoin Service + +The bitcoin service adds a native interface to Bitcoin Core for querying information about the Bitcoin blockchain. Bindings are linked to Bitcoin Core compiled as a static library. + +## API Documentation + +- `bitcoind.start([options], [callback])` - Start the JavaScript Bitcoin node. +- `bitcoind.getBlock(blockHash|blockHeight, callback)` - Get any block asynchronously by block hash or height as a node buffer. +- `bitcoind.isSpent(txid, outputIndex)` - Returns a boolean if a txid and outputIndex is already spent. +- `bitcoind.getBlockIndex(blockHash)` - Will return the block chain work and previous hash. +- `bitcoind.estimateFee(blocks)` - Estimates the fees required to have a transaction included in the number of blocks specified as the first argument. +- `bitcoind.sendTransaction(transaction, allowAbsurdFees)` - Will attempt to add a transaction to the mempool and broadcast to peers. +- `bitcoind.getTransaction(txid, queryMempool, callback)` - Get any tx asynchronously by reading it from disk, with an argument to optionally not include the mempool. +- `bitcoind.getTransactionWithBlockInfo(txid, queryMempool, callback)` - Similar to getTransaction but will also include the block timestamp and height. +- `bitcoind.getMempoolOutputs(address)` - Will return an array of outputs that match an address from the mempool. +- `bitcoind.getInfo()` - Basic information about the chain including total number of blocks. +- `bitcoind.isSynced()` - Returns a boolean if the daemon is fully synced (not the initial block download) +- `bitcoind.syncPercentage()` - Returns the current estimate of blockchain download as a percentage. +- `bitcoind.stop([callback])` - Stop the JavaScript bitcoin node safely, the callback will be called when bitcoind is closed. This will also be done automatically on `process.exit`. It also takes the bitcoind node off the libuv event loop. If the daemon object is the only thing on the event loop. Node will simply close. + diff --git a/docs/services/db.md b/docs/services/db.md new file mode 100644 index 00000000..9a8d1eba --- /dev/null +++ b/docs/services/db.md @@ -0,0 +1,34 @@ +# Database Service + +An extensible interface to the bitcoin block chain. The service builds on the [Bitcoin Service](bitcoind.md), and includes additional methods for working with the block chain. + +## API Documentation + +Get Transaction + +```js +var txid = 'c349b124b820fe6e32136c30e99f6c4f115fce4d750838edf0c46d3cb4d7281e'; +var includeMempool = true; +node.getTransaction(txid, includeMempool, function(err, transaction) { + //... +}); +``` + +Get Transaction with Block Info + +```js +var txid = 'c349b124b820fe6e32136c30e99f6c4f115fce4d750838edf0c46d3cb4d7281e'; +var includeMempool = true; +node.getTransactionWithBlockInfo(txid, includeMempool, function(err, transaction) { + //... +}); +``` + +Get Block + +```js +var blockHash = '00000000d17332a156a807b25bc5a2e041d2c730628ceb77e75841056082a2c2'; +node.getBlock(blockHash, function(err, block) { + //... +}); +``` diff --git a/docs/testing.md b/docs/testing.md new file mode 100644 index 00000000..aba45658 --- /dev/null +++ b/docs/testing.md @@ -0,0 +1,52 @@ +## Development & Testing + +To run all of the JavaScript tests: + +```bash +npm run test +``` + +To run tests against the bindings, as defined in `bindings.gyp` the regtest feature of Bitcoin Core is used, and to enable this feature we currently need to build with the wallet enabled *(not a part of the regular build)*. To do this, export an environment variable and recompile: + +```bash +export BITCORENODE_ENV=test +npm run build +``` + +If you do not already have mocha installed: + +```bash +npm install mocha -g +``` + +To run the integration tests: + +```bash +mocha -R spec integration/regtest.js +``` + +If any changes have been made to the bindings in the "src" directory, manually compile the Node.js bindings, as defined in `bindings.gyp`, you can run (-d for debug): + +```bash +$ node-gyp -d rebuild +``` + +Note: `node-gyp` can be installed with `npm install node-gyp -g` + +To be able to debug you'll need to have `gdb` and `node` compiled for debugging with gdb using `--gdb` (sometimes called node_g), and you can then run: + +```bash +$ gdb --args node examples/node.js +``` + +To run mocha from within gdb (notice `_mocha` and not `mocha` so that the tests run in the same process): +```bash +$ gdb --args node /path/to/_mocha -R spec integration/regtest.js +``` + +To run the benchmarks: + +```bash +$ cd benchmarks +$ node index.js +``` \ No newline at end of file