Change the block storage layer again, this time with multiple files
per block, but tracked by txindex.dat database entries. The file
format is exactly the same as the earlier blk00001.dat, but with
smaller files (128 MiB for now).
The database entries track how many bytes each block file already
uses, how many blocks are in it, which range of heights is present
and which range of dates.
Corrupt wallets used to cause a DB_RUNRECOVERY uncaught exception and a
crash. This commit does three things:
1) Runs a BDB verify early in the startup process, and if there is a
low-level problem with the database:
+ Moves the bad wallet.dat to wallet.timestamp.bak
+ Runs a 'salvage' operation to get key/value pairs, and
writes them to a new wallet.dat
+ Continues with startup.
2) Much more tolerant of serialization errors. All errors in deserialization
are reported by tolerated EXCEPT for errors related to reading keypairs
or master key records-- those are reported and then shut down, so the user
can get help (or recover from a backup).
3) Adds a new -salvagewallet option, which:
+ Moves the wallet.dat to wallet.timestamp.bak
+ extracts ONLY keypairs and master keys into a new wallet.dat
+ soft-sets -rescan, to recreate transaction history
This was tested by randomly corrupting testnet wallets using a little
python script I wrote (https://gist.github.com/3812689)
Before, opening a -datadir that was created with a new
version of Berkeley DB would result in an un-caught DB_RUNRECOVERY
exception.
After these changes, the error is caught and the user is told
that there is a problem and is told how to try to recover from
it.
Any problems seen during deserialization will throw an uncaught
exception, crashing the entire bitcoin process. Properly return an
error instead, so that we may at least log the error and gracefully
shutdown other portions of the app.
Satoshi's commits fdbf76d and c8ad9b8 (SVN import) removed the
DB_PRIVATE flag from the environment. In part, this enables processes
other than bitcoind to examine the active database environment.
However, this incurs a slight performance penalty versus working
entirely within application memory (DB_PRIVATE). Because bitcointools
and other direct-BDB-accessing tools are not used by the vast
majority of users, prefer to default with DB_PRIVATE with the option
of disabling it if needed via -privdb=0.
Cleans up and organizes several scattered functions and variables related to
the BDB env. Class CDBInit() existed to provide a
guaranteed-via-C++-destructor cleanup of the db environment.
A formal CDBEnv class provides all of this inside a single wrapper.
* This is safer than DB_TXN_NOSYNC, and does not appear to impact
performance.
* Applying this to the dbenv is necessary to avoid many fdatasync(2)
calls on db 5.x
* We carefully and thoroughly flush databases upon shutdown and
other important events already.
Add an option -detachdb (and entry in OptionDialog), without which no
lsn_reset is called on addr.dat and blkindex.dat. That means these
files cannot be moved to a new environment, but shutdown can be
significantly faster. The wallet file is always lsn_reset'ed.
-detachdb corresponds to the old behaviour, though it is off by
default now to speed up shutdowns.
This commit removes the dependency of serialize.h on PROTOCOL_VERSION,
and makes this parameter required instead of implicit. This is much saner,
as it makes the places where changing a version number can have an
influence obvious.
Since auto-remove-db-logs was enabled, each time a CTxDB was closed
outside of the initial download window, it causes a checkpoint + log
cleanup. This is overkill, so reduce the sync frequency to once per
minute at most.
Where possible, use boost::filesystem::path instead of std::string or
char* for filenames. This avoids a lot of manual string tinkering, in
favor of path::operator/.
GetDataDir is also reworked significantly, it now only keeps two cached
directory names (the network-specific data dir, and the root data dir),
which are decided through a parameter instead of pre-initialized global
variables.
Finally, remove the "upgrade from 0.1.5" case where a debug.log in the
current directory has to be removed.
