Running out of memory, how can I reduce the size of wallet.dat?

1

My wallet.dat is 260MB big. I am using 32bit bitcoind (bitcoin-0.10.2), linux.

Bitcoind is having memory problems:

 ["errors"]=>
  string(89) "EXCEPTION: St9bad_alloc       
std::bad_alloc       
bitcoin in ProcessMessages()       

Current memory allocation: 2.9 GB virt, 2.3 GB virt.

How can I resolve the memory problem? Can I delete old unused addresses? I did not find an API call to remove old addresses. Probably bitcoin-0.11.0 does not resolve the problem either, it has already nearly same values: 2.45 GB Virt, 2.1 GB Res.

Cahoiner

Posted 2015-10-15T16:48:58.367

Reputation: 41

Why's your wallet.dat so big? Are you automatically generating addresses?Nick ODell 2015-10-15T17:08:15.267

No i try to reduce the amount of newly generated addresses meanwhile. But this strategy does not help me because it is already too big. I want to reduce it, please help how can i do it?Cahoiner 2015-10-15T17:12:28.827

Is there a light weight implementation which can load the wallet.dat from the bitcoind?Cahoiner 2015-10-15T19:19:36.153

Are you certain that the size of wallet.dat is the cause of the problem? If you stop bitcoind, move wallet.dat out of the .bitcoin directory, and start bitcoind again, does it use much less memory? By the way, you can temporarily get it working (although perhaps slowly) by adding swap.Nate Eldredge 2015-10-15T19:59:59.567

There is no built-in way to delete addresses as far as I know - probably this was considered too dangerous. Roughly how many of the addresses in the wallet do you need to save?Nate Eldredge 2015-10-15T20:02:10.537

Hi there,SWAP does not help, the limit is under 32bit processes 4GB ram minus few MBs. Yes wallet.dat is the problem, without wallet.dat the process is using less than1 GB virt. As it sounds i need to create a new wallet.dat and import from the old one. And by this discarding unused addresses. But the transfer progress will be error prone. Sounds like huge problem, dam.Cahoiner 2015-10-15T20:39:23.907

Now after a restart with bitcoin-0.11.0 only 900M virt is used. Is it possible that someone attacked my wallet? I have read up with in memory transactions or something ... . Actually i saw many wallet conflicts warningCahoiner 2015-10-16T04:31:01.837

Is your wallet.dat is so big because of many privkeys/addresses or because of large number of transactions with them? These are two different problems. Each of them has own solution.amaclin 2015-10-16T08:07:37.283

dont know whats taking more memory, addresses or transactions. Addresses are about 150k addresses, the rest must be transactions. currently process is taking 1150MB virt, so it must be temporary data which has filled up the memory. because the 4GB virt limit was reached before restarting bitcoindCahoiner 2015-10-16T17:13:18.617

Did not have the problem anymore with bitcoind version bitcoin-0.11.2. Maybe it has been resolvedCahoiner 2015-12-13T10:48:42.890

Answers

1

Bitcoind uses a lot of virtual memory, due to having multiple threads, and maintaining several node-wide and per-peer caches of data (UTXO cache, mempool, relay cache, network buffers, known invs list, ...).

Until recently, many of these were effectively unbounded, depending on network conditions. The upcoming release of Bitcoin Core 0.12 has far better internal memory accounting, UTXO set limiting, mempool limiting, and more compact internal representations.

Pieter Wuille

Posted 2015-10-15T16:48:58.367

Reputation: 54 032

0

A bitcoind restart helps. The memory consumption grows over time, after a restart it will start growing again but you have some weeks of operation to go. memory growth rate depends on what is going on the bitcoin network.

Cahoiner

Posted 2015-10-15T16:48:58.367

Reputation: 41

0

The bitcoin version 0.11.1 (linux 32 bit) seems to deal better with the memory, so i amnot having the problem anymore

Cahoiner

Posted 2015-10-15T16:48:58.367

Reputation: 41