Commit Graph

262 Commits

Author SHA1 Message Date
Matias Alejo Garcia c1d3f351f2 add address index to mempool + noBalance options 2016-02-08 13:21:53 -05:00
Braydon Fuller 83eba52657 Tests: Fix tests to use enable/disableRegtest 2016-02-04 17:26:17 -05:00
Braydon Fuller 98bd8ee560 DB Service: Include a version number for upgrading purposes 2016-01-28 12:08:05 -05:00
Braydon Fuller a2acc0c80f Address Service: Fixed test for max address limit 2016-01-26 13:09:31 -05:00
Braydon Fuller 45029030f1 Address Service: Sort after unconfirmed and confirmed 2016-01-18 16:16:53 -05:00
Braydon Fuller d4f2df5c51 Address Service: Sort mempool txids 2016-01-18 15:56:05 -05:00
Braydon Fuller 62934b4b66 Address Service: Removed event listeners prior to stopping 2016-01-18 15:00:55 -05:00
Braydon Fuller 687400eab2 Address Service: Added test for history `getAddressDetailsForTransaction` 2016-01-18 13:53:32 -05:00
Braydon Fuller 3d9b6d5532 Address Service: More tests for history 2016-01-18 12:59:49 -05:00
Braydon Fuller e79c00db10 Address Service: Updated tests and fixed various bugs 2016-01-15 18:04:32 -05:00
Braydon Fuller ead6c2f45f Address Service: Removed caching and added max query limits
Querying addresses that have millions of transactions is supported however
takes hundreds of seconds to fully calculate the balance. Creating a cache of
previous results wasn't currently working because the `isSpent` query is always
based on the current bitcoind tip. Thus the balance of the outputs would be included
however wouldn't be removed when spent as the output wouldn't be checked again
when querying for blocks past the last checkpoint. Including the satoshis in the
inputs address index would make it possible to subtract the spent amount,
however this degrades optimizations elsewhere. The syncing times or querying
for addresses with 10,000 transactions per address.

It may preferrable to have an additional address service that handles high-volume
addresses be on an opt-in basis so that a custom running client could select
high volume addresses to create optimizations for querying balances and history.
The strategies for creating indexes differs on these use cases.
2016-01-14 17:17:04 -05:00
Braydon Fuller 4fcec8755c Address Service: Fixed many bugs from tests
- Refactored getAddressSummary and added several tests
- Fixed bugs revealed from the integration regtests
- Updated many unit tests
2016-01-13 17:15:14 -05:00
Samuel Reed 3214390d4c
Fix #377; db must contain hash type, not just hash.
Prevents erroneous crediting of all transactions to both the
p2pkh and the corresponding p2sh address.
2016-01-11 09:47:00 -06:00
Patrick Nagurny 02620a5b47 Merge pull request #352 from braydonf/mem
Optimized address service mempool index size
2015-11-03 17:23:28 -05:00
Braydon Fuller 0ea035c4f0 Address: Fixed race condition with transaction event handlers 2015-11-03 17:12:13 -05:00
Braydon Fuller 378edd4186 Web: Changed to pass express subapp to services instead of the router. 2015-11-03 10:32:01 -05:00
Braydon Fuller c5c8e21c6c Address: Fixed bug with isSpent confusion with prevTxId and txid 2015-11-03 10:26:32 -05:00
Braydon Fuller a1bae366b3 Database: Removed `runAllMempoolIndexes` method
Replaced with using `tx` and `txleave` to manage the state of the mempool indexes.
2015-11-02 17:18:37 -05:00
Braydon Fuller 5ac3b1c61f Address: Added unit tests for new mempool index methods 2015-11-02 17:10:43 -05:00
Braydon Fuller 89ef28f0b7 Optimize mempool address index memory footprint
- Adds default to store a large portion of the mempool index in leveldb
- Includes an option to use memdown to have the mempool index in-memory
2015-10-30 18:36:16 -04:00
Braydon Fuller 086ba5fcfc Switch to remove items from mempool index as they leave asynchronously. 2015-10-29 19:16:56 -04:00
Braydon Fuller f0ec424161 Added bindings to be able to listen to tx leaving mempool. 2015-10-29 16:15:00 -04:00
Braydon Fuller bf97d7e70b Include node version in binary distribution. 2015-10-28 14:54:27 -04:00
Braydon Fuller 92382d69cc Add --testnet option to create command. 2015-10-21 15:28:28 -04:00
Braydon Fuller e046f7294e Changes to be able to run with only a configuration file.
- Adds parameter to cli methods to be able to specify the location of services modules. This is useful for packages that wrap bitcore-node to be able to pass along a node_modules directory with services.
- Adds another parameter for including additional services in the default settings.
- Will use the `process.env.HOME + '/.bitcore` as the default configuration location.
- There are now two `getDefaultConfig`, one that will instatiate a `~/.bitcore` directory with a default if it doesn't exist, and `getBaseDefaultConfig` that will return a basic configuration without additional services enabled.
- Changes logic to use the global install if a local node_modules version is not available, this would previously assume that it was a local install because of the existence of a configuration file.
2015-10-20 12:33:53 -04:00
Braydon Fuller 21e0aa28b5 Added a Node.js version check based on the package.json engine. 2015-10-18 22:50:29 -04:00
Braydon Fuller 1dec93edd4 bitcore -> bitcore-lib 2015-10-17 00:56:29 -04:00
Braydon Fuller d9b2a0969b Fixed bug with balance
There was a bug when getting unspent outputs that would include an output
that was spent in the mempool in addition to the new output with the change
address. This lead to a balance having an output counted twice towards the
end balance. The solution is to have the isSpent method for the address service
to also include if the output was spent in the mempool, as the isSpent
method exposed from bitcoind only includes if the output was spent in a block.
2015-10-12 09:29:49 -04:00
Patrick Nagurny 3e38d84d0b emit address/balance events for mempool transactions 2015-10-02 17:10:12 -04:00
Patrick Nagurny 9acb896f10 Wait for bitcoind to catch up if tip is not found 2015-10-02 13:35:42 -04:00
Braydon Fuller 539b263c67 Add spentTxId Index
- To be able to query an inputTxId and inputIndex that spends an outputTxId and outputIndex
- Extends the mempoolSpentIndex to include the inputTxId and inputIndex
2015-10-01 20:03:31 -04:00
Patrick Nagurny 26b27b292e update tests 2015-09-29 16:03:56 -04:00
Chris Kleeschulte af7966f94a Removed unneeded getTxOutSetInfo 2015-09-25 11:33:51 -04:00
Braydon Fuller 1183e0cae7 Service Documentation 2015-09-23 18:14:37 -04:00
Chris Kleeschulte d474d03cf2 Merge pull request #248 from braydonf/pubkeyout
Address compatibility with pay-to-publickey output scripts.
2015-09-21 14:39:42 -04:00
Braydon Fuller 7e82c933c0 Merge pull request #249 from kleetus/feature/getNextBlockHash
Next Block Hash Bindings
2015-09-21 14:30:30 -04:00
Chris Kleeschulte 402d01d7ac Next Block Hash Bindings
- Added bindings for next block hash
- Added the reg/unit tests
2015-09-21 14:07:14 -04:00
Braydon Fuller e3096d3646 Address compatibility with pay-to-publickey output scripts. 2015-09-21 13:49:48 -04:00
Patrick Nagurny b11ab97739 add tests 2015-09-18 14:06:26 -04:00
Patrick Nagurny 4a5031a917 Merge pull request #240 from braydonf/gettxoutsetinfo
Add binding for getting transaction output set information.
2015-09-18 13:20:46 -04:00
Braydon Fuller 68368397d3 Add binding to get the best block hash. 2015-09-17 18:11:47 -04:00
Braydon Fuller 4c674a8fbb Add binding for getting transaction output set information. 2015-09-17 17:52:23 -04:00
Chris Kleeschulte 0ba168e2a2 Fixed typo and added a test to prove that the inputs from the mempoolInputIndex are added on stream close. 2015-09-17 17:20:35 -04:00
Chris Kleeschulte c3654120ee MempoolInputIndex and MempoolOutputIndex
- Added in memory indexes for mempool input and 
output by address.
2015-09-17 14:29:11 -04:00
Braydon Fuller 4343b90de2 Merge pull request #219 from pnagurny/feature/timestamp-blocks
Get block hashes by timestamp range
2015-09-16 13:47:00 -04:00
Patrick Nagurny aa9504a402 Merge pull request #227 from braydonf/add-cmd-remote
Bitcore Node Add Command with Remote URLS
2015-09-16 13:34:24 -04:00
Patrick Nagurny a0be38f074 check for timestamp out of bounds 2015-09-16 12:04:44 -04:00
Patrick Nagurny 7e1d433781 go from highest timestamp to lowest timestamp 2015-09-15 18:23:06 -04:00
Patrick Nagurny 00d3a0ba67 binary encode key and value 2015-09-15 16:55:14 -04:00
Braydon Fuller bfba4a1b7f Address History: Include count in response results. 2015-09-15 13:17:15 -04:00
Patrick Nagurny eaee098cf0 add getBlockHashesByTimestamp 2015-09-15 12:56:44 -04:00
Patrick Nagurny 696759849f Merge pull request #223 from braydonf/history
Pagination for Address History
2015-09-15 12:34:26 -04:00
Braydon Fuller 25e801126f Bitcore Node Add Command with Remote URLS 2015-09-15 11:08:55 -04:00
Braydon Fuller 0b2d5f4891 Merge pull request #220 from kleetus/script_helpers
gitignore, integration test, env
2015-09-15 10:20:57 -04:00
Braydon Fuller 5fbd8991ed Address History: Combine addresses so that history is txid unique. 2015-09-14 21:04:54 -04:00
Braydon Fuller e7587564a7 Address History: Add getAddressHistoryCount to available API methods. 2015-09-14 17:29:43 -04:00
Braydon Fuller b1b40c892e Address History: Add getAddressHistoryCount method 2015-09-14 16:48:28 -04:00
Braydon Fuller a80b58e004 Address History: Fix sorting bug 2015-09-14 16:11:31 -04:00
Chris Kleeschulte 7b50f5ff3d Switched to using a flag instead of using a replacement. 2015-09-14 13:15:55 -04:00
Braydon Fuller f88eee5a1c Improve db storage efficiency by ~2 times
Encodes and decodes keys and values for leveldb storage more efficiently.
2015-09-14 00:30:21 -04:00
Braydon Fuller d3641f3b0a Pagination Optimization
- Sort and paginate before getting full transaction details.
- Only get detailed transaction information for items within the current page.
- Improves the performance with large sets of transactions.
2015-09-14 00:30:14 -04:00
Braydon Fuller 1cf34f2dd8 Address Block Handling Optimizations
- Changes to use ripemd160 hash directly instead of the base58check encoded values
- Speeds block handling performance by ~4 times
2015-09-12 19:56:13 -04:00
Braydon Fuller c205f781a5 Pagination for Address History
- Reindexing the bitcore-node database is required with this change.
- Address indexes are updated to include address and height in spent index so that both inputs and outputs can be queried by address and height using "start" and "stop" to limit the range of the query.
- Address history also now supports paginated results using "from" and "to" values that indicate an index in the array.
2015-09-12 19:44:10 -04:00
Chris Kleeschulte 51355c51f1 Multiple sigints sent to the node process
- Added a simple replacement function after the first sigint is handled.
- Added unit tests for the exitHandler, requiring it to be exposed from the module.
2015-09-11 16:27:24 -04:00
Chris Kleeschulte ef49264907 gitignore, integration test, env
- Added a very common way to launch a script using the env (saving keystrokes)
- Added a gitignore for the integration/data dir
- Added a bus integration test apart from the regtest and using a test service
2015-09-11 10:58:52 -04:00
Patrick Nagurny c4d28c3947 wait for db operations to complete before closing leveldb 2015-09-10 12:45:00 -04:00
Patrick Nagurny 8b0b401d52 inherit https options from node 2015-09-10 11:08:48 -04:00
Patrick Nagurny 5677964651 add https to web service 2015-09-09 16:49:36 -04:00
Patrick Nagurny 4682714c69 check for blockHandler on services 2015-09-09 15:04:07 -04:00
Braydon Fuller 2f6fc6e3e9 Merge pull request #202 from pnagurny/feature/main-chain
Add isMainChain method
2015-09-09 10:08:21 -04:00
Patrick Nagurny 0ff742c1b6 add isMainChain method 2015-09-08 17:38:59 -04:00
Braydon Fuller 60af86777f Start/Stop Improvements
- A Node will shutdown if there is an error starting a service as it would lead to cascading errors.
- `node.start()` needs to be called, and nolonger is called automatically when the instance is created.
- A service will only be added to node.services after it's started
- Stopping services that are not started will gracefully continue.
- Logging sync status of db will only apply if the service is started.
- Debug log about a service without a route will always include the service name
2015-09-08 10:17:57 -04:00
Braydon Fuller b4ed29eabe Merge pull request #193 from kleetus/reindex
Reindex logic
2015-09-08 10:15:34 -04:00
Chris Kleeschulte 94c345134e Daemon
- Added the daemon option to the cli start command
- Added the initialization routine to spawn the child process
- Added unit tests
2015-09-04 17:36:53 -04:00
Braydon Fuller 9aec734122 Merge pull request #195 from pnagurny/bug/transaction-fee
Handle getFee() error
2015-09-04 14:28:37 -04:00
Patrick Nagurny 79ae3bb412 handle getFee() error 2015-09-04 14:28:04 -04:00
Braydon Fuller 50925d1e0f Add tests for reorg improvement, and remove nolonger need code. 2015-09-04 13:52:59 -04:00
Chris Kleeschulte 730dd37318 Reindex logic
- If the reindex option is set in bitcoin.conf, then when start is called and onBlocksReady's callback is fired:
	- start's callback will not be fired until the reindex takes place.
	- along the way the sync percentage is display once per second
2015-09-04 11:39:38 -04:00
Patrick Nagurny a0e40ffd15 check for duplicate events 2015-09-03 17:29:28 -04:00
Patrick Nagurny 7ac429fbd2 remove name field from node 2015-09-03 16:59:43 -04:00
Patrick Nagurny 900f715a49 add tests 2015-09-03 16:38:23 -04:00
Braydon Fuller 52e80039d8 CLI fixes for installing and removing services. 2015-09-02 18:39:19 -04:00
Patrick Nagurny fd2790daab Merge pull request #170 from braydonf/coverage
Improve test coverage and cleanup.
2015-09-02 10:43:41 -04:00
Patrick Nagurny 1f94215aa2 timestamp should be a number not a string 2015-09-01 17:34:30 -04:00
Braydon Fuller d56986f281 Added unit tests for db.getMetadata 2015-09-01 16:42:27 -04:00
Braydon Fuller 63e71d7f0c Added unit tests for db.saveMetadata 2015-09-01 14:29:04 -04:00
Braydon Fuller efc770f7eb Bitcoind is already logging the status. 2015-09-01 13:55:51 -04:00
Braydon Fuller f4b24df024 Add tests for bitcoind proxy methods. 2015-09-01 13:18:44 -04:00
Braydon Fuller b6416409a2 Add unit tests for utils. 2015-09-01 12:48:08 -04:00
Braydon Fuller 96fa0920a4 Add unit tests for start. 2015-09-01 12:47:15 -04:00
Braydon Fuller 53adaa7e6a Improve test coverage and cleanup. 2015-08-31 16:32:25 -04:00
Patrick Nagurny 4ae6377548 fixes 2015-08-31 15:15:55 -04:00
Patrick Nagurny 895e46dcac pass config to services 2015-08-31 13:37:11 -04:00
Patrick Nagurny f1cac3da1d make webService into a real service 2015-08-31 13:06:49 -04:00
Patrick Nagurny 1b5f30a9a1 add tests 2015-08-31 12:57:49 -04:00
Patrick Nagurny a68cf4ecf8 Web service and other changes in preparation for insight-api 2015-08-31 12:54:28 -04:00
Braydon Fuller 7551f487f8 Rename modules to services. 2015-08-31 11:31:28 -04:00
Braydon Fuller 16eef1279c Combine chain with db module. 2015-08-31 11:25:30 -04:00
Braydon Fuller df9b62acca Convert db into a db module. 2015-08-31 11:15:51 -04:00
Braydon Fuller 96f6604222 Convert daemon into bitcoin module. 2015-08-31 11:11:16 -04:00
Braydon Fuller ea377c1251 Added CLI commands "remove" and "call" 2015-08-28 17:13:02 -04:00
Braydon Fuller 43ab4586e5 Bus module fix. 2015-08-28 16:16:51 -04:00
Braydon Fuller 1ed457504c Update unit tests with node.modules as an object. 2015-08-28 11:43:38 -04:00
Braydon Fuller 56ebf42403 Move Modules from DB to Node 2015-08-27 19:02:33 -04:00
Braydon Fuller 5361183a19 Use node as a reference in modules, db and chain. 2015-08-27 14:04:42 -04:00
Braydon Fuller ccab818d33 Use Bitcore Block/BlockHeader 2015-08-27 12:14:25 -04:00
Braydon Fuller bbc421a31e Migrate chainlib functionality into bitcore-node. 2015-08-26 17:24:10 -04:00
Braydon Fuller 81b2d2f2a3 Added tests for start command. 2015-08-25 13:57:54 -04:00
Braydon Fuller 7e174fd0cd Handle exit codes from npm install with the create command. 2015-08-25 13:15:30 -04:00
Braydon Fuller 5ea787b3a1 Added `add` command and tests. 2015-08-25 13:01:52 -04:00
Braydon Fuller be525b055d Replace ~/.bitcoin with process.env.HOME 2015-08-24 18:53:37 -04:00
Braydon Fuller 0b4af2757b Implement function to find configuration in the current path. 2015-08-24 18:51:07 -04:00
Braydon Fuller 67a2035365 Run `npm install` if the node is created locally. 2015-08-24 18:51:07 -04:00
Braydon Fuller 399d379ff5 More tests for config creation. 2015-08-24 18:51:07 -04:00
Braydon Fuller 00e4eac14a Layout test cases and files for cli commands. 2015-08-24 18:51:07 -04:00
Braydon Fuller e9763de594 Fix chain unit test. 2015-08-24 16:57:59 -04:00
Braydon Fuller 49cf3a9ca3 Fix reorg integration test after rebase. 2015-08-24 16:39:05 -04:00
Braydon Fuller f1642388af Fix unit tests for node. 2015-08-24 14:46:48 -04:00
Braydon Fuller e707b297d1 Fix unit tests for node. 2015-08-24 14:06:38 -04:00
Patrick Nagurny faf93a18a7 get regtest to work 2015-08-24 13:52:45 -04:00
Patrick Nagurny a9e5ee6f1a add tests 2015-08-24 13:29:42 -04:00
Patrick Nagurny a61e1d9b8f start and stop services 2015-08-24 13:29:42 -04:00
Patrick Nagurny d6d9c6a975 take out modifying behavior to estimateFee 2015-08-18 12:20:30 -04:00
Patrick Nagurny 43ec2d3b34 add more tests 2015-08-17 14:27:46 -04:00
Patrick Nagurny 738574d6b1 add fees to address history 2015-08-14 16:28:51 -04:00
Patrick Nagurny 78bc054bfa changes for BWS integration 2015-08-14 11:15:07 -04:00
Braydon Fuller 52cc91e1d7 Merge pull request #113 from kleetus/txindex_message
Blockchain state
2015-08-12 21:21:30 -04:00
Patrick Nagurny 223aebf98c Merge pull request #117 from kleetus/blockchain_synced
GetProgress, will report the GuessVerificationProgress for the blockchain
2015-08-12 15:32:09 -04:00
Chris Kleeschulte 8115d92de9 Removed verifyScript and related methods that we don't currently use. 2015-08-12 12:38:33 -04:00
Braydon Fuller dd91b75827 Fix test for bitcoindSync 2015-08-11 17:38:04 -04:00
Chris Kleeschulte 2b775af7a0 Added the badbitcoin.conf 2015-08-10 14:31:45 -04:00
Chris Kleeschulte fe7397fb0b Added a check for txindex enabled in the config. 2015-08-10 14:05:05 -04:00
Braydon Fuller 16f7ffa784 Process incoming transactions for the Address Module
- Transaction events are emitted when an address matches subscribers
- The Address Module listens for incoming transactions from the daemon
2015-08-07 16:49:45 -04:00
Braydon Fuller c21ff322b7 Merge pull request #97 from pnagurny/startnode
Simple RPC/websockets API
2015-08-04 17:33:35 -04:00
Patrick Nagurny e95d4c865f finish getting everything to work. add tests 2015-08-04 17:30:55 -04:00
Patrick Nagurny c0d28386ae Merge pull request #101 from braydonf/blkbench
Optimizations for address module blockhandler
2015-08-04 16:42:59 -04:00
Braydon Fuller dd59d5f64b Optimizations for address module blockhandler
- _scriptBuffer instead of reserializing the script
2015-08-04 16:19:06 -04:00
Braydon Fuller c88c873184 Update database directory for bitcore-node. 2015-08-04 15:06:00 -04:00
Braydon Fuller de77dbf99b Add tests around db.getTransaction and return NotFound error. 2015-08-03 18:11:11 -04:00
Patrick Nagurny afee19e1f7 remove console.log 2015-07-31 11:13:33 -04:00
Patrick Nagurny 2a55c900ce update tests 2015-07-31 11:13:33 -04:00
Patrick Nagurny 073353f895 write tests 2015-07-31 11:12:43 -04:00
Braydon Fuller 725120a24e Add tests for event bus. 2015-07-29 17:53:48 -04:00
Braydon Fuller 4f281dcefe Update chain prevHash cache. 2015-07-24 23:18:14 -04:00
Braydon Fuller 9890a4f0da Added tests for _syncBitcoindRewind 2015-07-24 13:45:31 -04:00
Braydon Fuller ab4addc82e Split out _syncBitcoindAncestor from _syncBitcoindRewind and added unit tests 2015-07-23 23:07:13 -04:00
Braydon Fuller 9eda30ae2b Handle forks with bitcoindSync
- added bindings to listen for tip updates from the daemon
- update the height during syncing for continuity
- call sync when there is a new tip
- added ability to rewind to a ancestor when there is a fork
2015-07-23 20:06:19 -04:00
Patrick Nagurny 9108b0f695 rename methods to getAPIMethods 2015-07-23 15:34:36 -06:00
Patrick Nagurny 8290d61a09 only pass db as option to module because other params aren't available until init 2015-07-23 13:30:36 -06:00
Patrick Nagurny 14e21fadb0 update tests 2015-07-23 09:34:10 -06:00