This is not about double spends and I believe that the related question is not about double spends ether!

On bitcoin's testnet3 I am seeing (using https://github.com/bitpay/bitcore-p2p) many tx change id

scenario:

I send some tBTC using http://testnetwallet.com
I see a utx with txid of XXXX broadcast onto the network with my inputs and going to my test address (everything looks cool)
I carry on monitoring looking at all broadcast-ed txs
a new block happens.
I look through the txids in the block but do not find my txid
I query my receiving address on some block explorer web service and see that my tx's txid got renamed to YYYY
I go and look at the block again and see YYYY is in its tx array!
I discontinue monitoring all broadcast-ed txs

I am starting to think that it (tx malleability) is happening when the miners are including the txs into the blocks!

I never see a tx with txid YYYY broadcasted NEVER (I am seeing every transaction!) this suggests that the txid is being altered!

I have been seeing this happen now for over 2 weeks and not found anyone talking about it apart for the first person today (4th april 2017) In the related question.

I have been trying many things in my code to stop relying on the txid to identify if my tx has 6 confirmations. But, I am finding it impossible to ...update my tx with its new txid!

I have been trying (many times in different ways) to ask peers (p2p peers) getdata for each unrecognizable txid in new blocks, but a lot of the time the peer seems to disconnect ....

My question is: What should I be doing in this situation?

Note: I'm trying not to use 3rd party services

also related: How did blockr.io see this?

UPDATE

for the last ten or so blocks today I have console.logged on each block

seen:1/52 by 73 peers seen:0/4 by 73 peers seen:0/35 by 73 peers seen:0/22 by 73 peers etc...

99% of all txs (everyones) on testnet3 are changing txid

I am listening to max peers 150

PROOF THAT THIS IS HAPPENING TO EVERYONE'S TXs

{ hash: '00000000000017f2a1d8ff44f149e7ead61a38e1a03217c33762ed9ace3c0c7d', validProofOfWork: true, validTimestamp: true, prevHash: '000000000000178c28b96e945965b9311ea4005a66e8c5c2701d30f928a498d8', txs: [ '775b7b0bf9345cbee62e3af57e1660a6fd92d0a089fd0694e4804a7190dd4a05', '1133ba34f4ef95af8a53c9698877cf8b7dc0e03d3095c371c0e2a1b8cd395ce2', '14df326c04bf2da2edbd4f9f04810916b95144eeef67570458c06449d3a56fc4', 'b969a4bdab4b6a5ea91500b4230c3eadc9aa807cca000ecce6686450d2676b42', 'fa5acc2feecae6a603673c6a817f79fa3a4dd1516d50c59014772c80500b7063', '0cd386804b65a3fb826557881dc3b50c1b298b7f0a0372da8faec97a2744d0e0', '24ca695bac05c4500a596415b70bd199c935308f7a90ccfc8ae01d7b96e6b115', '8125ed124dd4d0b3af98bc6028905fb329ff311c22f484be41bf9def8ea7b63e', '4f1e5a894be245d2a68c79e2b73dde54dbeaa8bb2e872b9b1115f40c0b93585b', 'aee4a615c617d71c544930d5dbacdb62d66fb089d556d84cbe92682295895d91', '545b2164e6bc2e764a702eac1bd69e625c6100c8cbeaa9528c16cd74eb144ecb', '450c86ef90858943f6b6ba284da3987191880b2b534749d3ae762d7c9060bb7c', '83d9107d90c36b0302af47f2b7e7bece30aaf20431dcdf251faa025d31f02fa8', '4785a7fb5b0a6106152dc80d2c028b2aa80cb9b4f0ce81985b8c672a34561703', '232a70178859760c5cd75ae054c503af9db4515d92a47b9da17ca5c81cee5d34', '5c218f9e957cfd7fa97dbbdd29a81673d4c91b3441279c23607a7d3889533855', 'b2472c88eb6c7a38476444c22cb560793624216e1e36354b406f15b09dd8316b', '5a7f3652282429eedf7bd2a6d0a8404e7844515f4139d14f8d7ed1e871efd373', '5147d190550b44eddad2d8ea6cdd022e47fe71a2517458b69c07c2424fd08bb2', '8e64ee1dcde8feb6b39ee043ecc43be719d0999ad181acee4d01e9d603976ff1', '0223ed8beb7d466d9d4026665d2ba58a97cbd1e6641be22f16d23be2b4574688', '18efc474f89835fb9eb1fa9ed1fc1dc296621aae3b727aab64f4f80ff76a098f', '8f38f6030a0471c7bef2b1e1757a6db0b63edf07aab3fe12a7e85d86b35c5eb5', '6b88744cba8f7661e64c67cd554d07a1b1981fe10614647a18b101520d289816', '6aee3deb7c0cb7247c2a85f14f96f8cb8447dee00e98a72ace80cbb850f5a709', '67de891a99a4c62fc90af91f5d880f9c2be80c1fc55bc444d10b74b63dd1ae1a', '62f955cf10d52f34a0b7bf2328c5095051d8dcf5009b65dd53632d54de4a515c', 'c42685086053a0b2d989453b6f0cb17a1b60642ac963acc782b5269c8e563b89', 'e2c2114e7cd15e47a84a4a48e7c1adbb29daff0f85afb5be6015a17271859909', 'cc834a0f20e43f4eb05638398e8cfc61bfaaa303dcd82705be2c9cbf80189ded', '10907000e1062dc63c817b995683a4a7433836f450f763c910fe6191aa555f24', 'a5bf7009e9c97b1b7652d3f5ed4067c1eeb40d1625e2fadb6ffb11ca18a27e74', '40c3ffce0140a9db634e4b869e79848f4316fd40528e48b0899e2b5606075ef2', '22b9db7c0ee380ec0c4eed97788dbf6f75fedbec569fb84a6c0d27bf36923cfa', 'f4906eda20607d733ae55caf3274fa91d5c2f20506881857af59be4d8bb0af25', '98dfd83a0506b7a56e7cc6b1c91623a38ecf504c9b78c658bb8a6e5c56f7af72', 'cd2711ab3400f03cb3830e6d7401058c99f5b3c0c438fe78180785582994b884', '566251b6b27cfbca202678f87ab6ebe5e6a71f306dea8e4035bca96207e630c1', '68eb0e868ebc4f0388c130bac9adcfb26fb526d8fdd7af6646ddcb9969e045df', '1cc856b20b1a66702782b4063caa1cce1cfd9a77600d53dbb727bc8d59cdeee3', 'd5312340a7bd04a732ea6da056fa4a32ef92558a30160cb63c457b9d72603823', 'd6cadd383605ab45e1cab2499349724d8973b160cd85e226115cbb30df383d44', '30a09d4abd5f3f52d47b7a7d2da05c5e0db2e40d93ae26a3d8b772227dffed48', 'efb6889c4507f2e7ccb781dc5982e14d111b696098acaa450a286b100322ca5d', 'cb7d6e3b904b49f01c85de319d4f3ed5ecc80aa3048dfbb97bb7300bc2fb4bd1', 'bb70bd40d0d9d2ceda243172a10c05b2f9099d4ea6c62414e6c04c0309e5f7d1', '7912e77605eddb2e18a5baac16326f5ee3f31ebee5ed5badec7f9327dda75a7e', '75255b0e2000c546b5f9e3836a8ed483b401280f993b5320f433507b7dad4981', '77af8d90ecca1b09f072b244a032775f025a82c148ae8393a56bdf2f055e687f' ] } look at the block: http://tbtc.blockr.io/block/info/1115996

next lets pick a random tx from the block that could be anyones tx:

http://tbtc.blockr.io/tx/info/a5bf7009e9c97b1b7652d3f5ed4067c1eeb40d1625e2fadb6ffb11ca18a27e74

you will see that this tx has two outputs lets look at one of the addresses an output leads to:

http://tbtc.blockr.io/address/info/mjSujFSUn1uCVHkD4bDj5cs3mZrp8ayec5

Oh what a surprise! it has 1 confirmed tx and one unconfirmed tx

unconfirmed: http://tbtc.blockr.io/zerotx/info/8d7778e2238db427149f0c7fa46a572e0ea5c3ae9eeefe78c0ec9e4b795f0d38

confirmed: http://tbtc.blockr.io/tx/info/a5bf7009e9c97b1b7652d3f5ed4067c1eeb40d1625e2fadb6ffb11ca18a27e74

the inputs and outputs are all the same on both!

I just pucked that out randomly from a random tx in a random block today happend a few minutes ago!

If you go through each of those txs then they will 99% all be like that!

Ben Muircroft

Posted 2017-04-04T18:49:40.583

Reputation: 408

testnet i suppose to act just like mainet. something is clearly broken with some/many miners on testnet... so testnet is currently useless for testing. – Ben Muircroft – 2017-04-04T19:10:24.410

1I would say just the opposite. Malleability is equally possible (and legal) on mainnet. It just so happens that nobody is doing it on a large scale right now, but they certainly could. So any application you design needs to be able to handle it appropriately. Whoever is doing this on testnet is doing us all a favor by forcing us to handle this important case that we might otherwise ignore or dismiss. – Nate Eldredge – 2017-04-05T17:22:33.510

i understand now – Ben Muircroft – 2017-04-05T17:34:47.970

Answers

Someone (let's call her Alice onwards) is messing up with testnet transactions, it happened to one of mines earlier today.

What's happening

Alice is changing the s representation of the EDCSA signature (See ECDSA signature generation algorithm) from at least one of the inputs of the transactions. By doing so, the signature is changing, but still valid, and thus the txid changed.

How is she doing so?

As explained in the BIP62, there are two valid representations of the s value, depending on how big the value is. Moreover, depending on the result the DER encoding may also be changed.

Alice is simply modifiying one of the the s values from one of the original transaction signatures, by performing a subtraction between the order n of the curve (that is FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFE BAAEDCE6 AF48A03B BFD25E8C D0364141 for the Secp256k1), and the value s.

Example:

Lets say that the s component of a signature from one of out transactions is s = 37c0af486123367bc7839d507894bd30cba3f61e0d1bbf54727e43199864452c, if we substract s to n:

s' = n - s = c83f50b79edcc984387c62af876b42cdef0ae6c8a22ce0e74d541b7337d1fc15

That is a valid representation of s.

Since s' starts by 0xc8, which is bigger than 0x80, the DER encoding will need a leading 0x00:

00c83f50b79edcc984387c62af876b42cdef0ae6c8a22ce0e74d541b7337d1fc15

Who is doing so?

It could be a miner, just as you suggested, but it could also be a normal node.

If its a miner, it could be doing the modifications just before including the transactions the block he is mining. On the other hand, if a normal node (better connected than you) receives the transaction, modifies it, and broadcast it, it could be included in a block instead of your original transaction. Notice also that, since you have been broadcasting the original transaction from your node, you are not likely to be aware of transactions trying to double spend it, since your neighbours (to which you have sent the original transaction) will drop the double-spending attempt and do not forward it to you.

What can you do?

You can forward the transaction from a different source that the one you are using to monitor the network, and do not rely transactions from the latter, in order to see what's going on. Moreover, you can monitor the network using the UTXO you are trying to spend (prev_txid + index) instead of the txid from the transaction you just created. If another transaction appears spending from the same source, you will know.

sr-gi

Posted 2017-04-04T18:49:40.583

Reputation: 2 382

I am seeing all txs change id not just mine! I am collecting every txid I see broadcasted and putting them in an array called seen, then when a block happens i look for each tx on the block in my seen array. seen:0/63, seen: 0/45, seen: 0/87 etc.... all txs on testnet3 are changing txid – Ben Muircroft – 2017-04-05T15:23:19.147

I know, as I said it happened to me also, I just considered all the possible cases. – sr-gi – 2017-04-05T15:39:10.100

i updated my question, and if you give me about 20 minutes to add further proof i think it will suprise you! – Ben Muircroft – 2017-04-05T15:47:01.653

I have posted indeniable proof in the question – Ben Muircroft – 2017-04-05T16:06:19.410

1Sorry, but what is the problem? Bitcoin transactions are malleable. The miner can use this technique and is using it now. Just live with this knowledge. – amaclin – 2017-04-05T16:14:22.907

@ben-muircroft I don't get the point, no one is telling you that you are not right, just explaining you why it is happening. – sr-gi – 2017-04-05T16:22:36.873

yeah i understand that. but i don't understand how I'm suppose to match a malleated txid in a block to the original tx – Ben Muircroft – 2017-04-05T16:24:12.603

1If what you want to do is just identify them, you can check which UTXO both transactions are spending, that is, which previous tx id and index both transactions are trying to spend. There will be two transactions spending from the same source, one in the block, and another one that you had received though the network. – sr-gi – 2017-04-05T16:27:29.033

but i never see the malleated txs. I only see the malleated txid in the block so i don't know what outupts the malleated tx has – Ben Muircroft – 2017-04-05T16:31:51.690

But you know the inputs of the transaction you have seen, you have to check with transaction in the new generated block is spending from the same previous output. – sr-gi – 2017-04-05T16:33:39.147

but seen as I never see malleated txs broadcated i don't have anything to check – Ben Muircroft – 2017-04-05T16:37:28.240

Let us continue this discussion in chat.

– sr-gi – 2017-04-05T16:39:19.023

you were correct also about there being extra tx info inside blocks – Ben Muircroft – 2017-04-06T11:24:10.657

The miner which confirms current blocks on testnet is malleating transactions. I can confirm this.

But what is the problem? Current consensus rules allow this. Trust me. I am an expert in malleability :)

amaclin

Posted 2017-04-04T18:49:40.583

Reputation: 5 763

Finally, thank you!! who do we report this to? – Ben Muircroft – 2017-04-05T16:08:51.640

Bitcoin is decentralized. There is no central authority to complain and send reports. This is normal behaviour. May be somebody is testing something. – amaclin – 2017-04-05T16:10:37.637

for the last 2 weeks? – Ben Muircroft – 2017-04-05T16:11:16.277

I do not see any problems here at all. – amaclin – 2017-04-05T16:12:06.960

the problem is that i never see the malleated txs broadcast over the network (i only see the original txs) – Ben Muircroft – 2017-04-05T16:13:13.300

It is your problem, not a bitcoin network problem :) nobody can help you because current clients do not relay HI-S transactions. So, they exist only on miner's node until confirmed in block – amaclin – 2017-04-05T16:17:06.733

Let us continue this discussion in chat.

– amaclin – 2017-04-05T16:20:59.993

testnet3 frequent tx malleability

UPDATE

PROOF THAT THIS IS HAPPENING TO EVERYONE'S TXs

Answers