blockchain checksig fail

i'm attempting to verify a checksig from a transaction in the live blockchain. it is failing on my debian jessie pc with openssl 1.0.1k, failing on my ubuntu 12.04 virtual machine with openssl 1.0.1 and validating correctly on my debian wheezy pc with openssl 1.0.1e. both are using the same code. the transaction is the first instance of a SIGHASH_NONE in the live blockchain. i calculate the "message" (tx) hash as:

hex: 2c7ecdcb2381e657228284398f2f66b2d7d9cf6aa1bd8e39a7300d0b3c8cfa5a
same thing in base58: 3zh5uvKWoRjFgKMSLRVqEWXB6YVLsuDKo97yCsCJZ2wB

the signature is:

hex: 30440220bb4fbc495aa23babb2c2be4e3fb4a5dffefe20c8eff5940f135649c3ea96444a022004afcda966c807bb97622d3eefea828f623af306ef2b756782ee6f8a22a959a2
same thing in base58: 381yXZvFc5V2dx81gRNNpj7ResKUgWA92mkTefemM9gM2kW37b3DGkV5v2BsFq94pSuH1E4jHq9ScHcp6T7Mfg41a7rbZCnd
note that this signature does not have the 02 hashcode appended to it

and the pubkey is:

hex: 04f1939ae6b01e849bf05d0ed51fd5b92b79a0e313e3f389c726f11fa3e144d9227b07e8a87c0ee36372e967e090d11b777707aa73efacabffffa285c00b3622d6
same thing in base58: SJa4B8CrqDs6St5KaXMm4KCCksaT7RDcCaRDyKSD6Mbu1WzSwokEq3JwGtfZRz7by6jYD5QmuVhgqsPYc6wTijQq
converts to bitcoin address: 145YPBBWRj4aquewvx59SAWNrSZFT5rvxr

could someone tell me if these parameters pass the ecdsa verification or not? also if anyone knows of an online validator that would be helpful.

i'm trying to verify this with the jsonrpc client but it doesn't work on any machine i have:

#!/usr/bin/env python2.7                                                        

from bitcoinrpc.authproxy import AuthServiceProxy, JSONRPCException
rpcuser = 1
rpcpassword = 1
rpc_connection = AuthServiceProxy("http://%s:%s@127.0.0.1:8332" % (rpcuser, rpcpassword))
message = "3zh5uvKWoRjFgKMSLRVqEWXB6YVLsuDKo97yCsCJZ2wB"                        
sig = "381yXZvFc5V2dx81gRNNpj7ResKUgWA92mkTefemM9gM2kW37b3DGkV5v2BsFq94pSuH1E4jHq9ScHcp6T7Mfg41a7rbZCnd"
addr = "145YPBBWRj4aquewvx59SAWNrSZFT5rvxr"                                     
res = rpc_connection.verifymessage(addr, sig, message)                          
print res

output:

False

maybe i'm encoding the message or signature wrong for the rpc client? i tried passing binary encoded data to it, but it didn't like that:

File "/usr/local/lib/python2.7/dist-packages/bitcoinrpc/authproxy.py", line 125, in __call__
    json.dumps(args, default=EncodeDecimal)))
  File "/usr/lib/python2.7/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/usr/lib/python2.7/json/encoder.py", line 200, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python2.7/json/encoder.py", line 263, in iterencode
    return _iterencode(o, 0)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xcd in position 2: invalid continuation byte

verifying with pybitcointools works on all machines:

from bitcoin import *
import binascii

msg = "2c7ecdcb2381e657228284398f2f66b2d7d9cf6aa1bd8e39a7300d0b3c8cfa5a"
# make sure to include the 02 hashcode on the end of the signature here!!
sig = "30440220bb4fbc495aa23babb2c2be4e3fb4a5dffefe20c8eff5940f135649c3ea96444a022004afcda966c807bb97622d3eefea828f623af306ef2b756782ee6f8a22a959a202"
pub = "04f1939ae6b01e849bf05d0ed51fd5b92b79a0e313e3f389c726f11fa3e144d9227b07e8a87c0ee36372e967e090d11b777707aa73efacabffffa285c00b3622d6"
msg = binascii.unhexlify(msg)
print ecdsa_raw_verify(msg, der_decode_sig(sig), pub)

output:

True

mulllhausen

Posted 2015-05-19T14:11:01.540

Reputation: 1 533

Answers

the fact that pybitcointools (a pure python implementation of ecdsa) is able to verify the message against the pubkey and signature indicates that it is a problem with openssl and not with the data itself being incorrect.

as per amaclin's comments under his answer, the der encoding for this signature is non-standard:

30440220bb4fbc495aa23babb2c2be4e3fb4a5dffefe20c8eff5940f135649c3ea96444a022004afcda966c807bb97622d3eefea828f623af306ef2b756782ee6f8a22a959a2

this is decoded as:

0x30 # constant placeholder
0x44 # total length = 68 bytes
0x02 # constant placeholder
0x20 # r-length = 32 bytes
0xbb4fbc495aa23babb2c2be4e3fb4a5dffefe20c8eff5940f135649c3ea96444a # r
0x02 # constant placeholder
0x20 # s-length = 32 bytes
0x04afcda966c807bb97622d3eefea828f623af306ef2b756782ee6f8a22a959a2 # s

note that the r value here starts with 0xbb = 1011 1011 - since the most significant bit (0x80) is set then this is a negative number. this would fail the IsValidSignatureEncoding check and be rejected ever since bip66. specifically, this line would cause the failure:

// Negative numbers are not allowed for R.
if (sig[4] & 0x80) return false;

here sig[4] is the first (leftmost) byte of the r value.

so this explains why some versions of openssl will fail this checksig and others will pass it - it is non-standard der encoding which is only compatible with certain versions of openssl.

mulllhausen

Posted 2015-05-19T14:11:01.540

Reputation: 1 533

Yes, you calculated digest 2c7ecdcb2381e657228284398f2f66b2d7d9cf6aa1bd8e39a7300d0b3c8cfa5a correctly.

Here is my piece of dirty code:

static void xxtest ( )
{
  const QByteArray t ( MyByteArray ( )
      .putInt32 ( TX_VERSION ) // 01000000
      .putInt8 ( 1 )           // 01 : one input
      .putArray ( QByteArray::fromHex ( "5f386c8a3842c9a9dcfa9b78be785a40a7bda08b64646be3654301eaccfc8d5e" ) )
      .putInt32 ( 1 )          // 01000000 : index
      .putPrefixed ( MyByteArray ( )
         .putInt8 ( OP_DUP )
         .putInt8 ( OP_HASH160 )
         .putPrefixed ( QByteArray::fromHex ( "21c43ce400901312a603e4207aadfd742be8e7da" ) )
         .putInt8 ( OP_EQUALVERIFY )
         .putInt8 ( OP_CHECKSIG ) )
       .putInt32 ( TX_SEQUENCE )
       .putInt8 ( 0 )          // no outputs
       .putInt32 ( TX_LOCK )
       .putInt32 ( SIGHASH_NONE ) );
  _trace ( QString ( t.toHex ( ).constData ( ) ) );
  const MyKey32 digest ( t.constData ( ), t.size ( ) );
  _trace ( QString ( "digest = %1" ).arg ( digest.toStringRev ( ) ) );
  const QByteArray pubkey ( QByteArray::fromHex ( "04f1939ae6b01e849bf05d0ed51fd5b92b79a0e313e3f389c726f11fa3e144d9227b07e8a87c0ee36372e967e090d11b777707aa73efacabffffa285c00b3622d6" ) );
  const QByteArray signature ( QByteArray::fromHex ( "30440220bb4fbc495aa23babb2c2be4e3fb4a5dffefe20c8eff5940f135649c3ea96444a022004afcda966c807bb97622d3eefea828f623af306ef2b756782ee6f8a22a959a2" ) );
  _trace ( QString ( "verify=%1 (expected value=1)" ).arg ( digest.verify ( pubkey, signature ) ) );
  // now test the same with wrong digest
  const MyKey32 temp ( "temp", 4 );
  _trace ( QString ( "verify=%1 (expected value=0)" ).arg ( temp.verify ( pubkey, signature ) ) );
}
//--------------------------------------------------------------
bool MyKey32::verify ( const QByteArray& pub, const QByteArray& sig ) const
{
  return 0 < ECDSA_verify ( 0, constPtr ( ), 32, (const quint8*)sig.constData ( ), sig.size ( ), EC_KEY_pub_key ( pub ) );
}

the output is:

"01000000015f386c8a3842c9a9dcfa9b78be785a40a7bda08b64646be3654301eaccfc8d5e010000001976a91421c43ce400901312a603e4207aadfd742be8e7da88acffffffff000000000002000000"
"digest = 2c7ecdcb2381e657228284398f2f66b2d7d9cf6aa1bd8e39a7300d0b3c8cfa5a"
"verify=1 (expected value=1)"
"verify=0 (expected value=0)"

Update: Here is small program for checking signatures with dependency only for OpenSSL lib:

#include <QByteArray>
#include <QDebug>

#include <openssl/ec.h>
#include <openssl/evp.h>
#include <openssl/ecdsa.h>
#include <openssl/sha.h>

static EC_KEY* EC_KEY_pub_key ( const QByteArray& pub )
{
  static EC_KEY* eckey = EC_KEY_new_by_curve_name ( NID_secp256k1 );
  const quint8* ppub = (const quint8*)pub.constData ( );
  o2i_ECPublicKey ( &eckey, &ppub, pub.size ( ) );
  return eckey;
}
//--------------------------------------------------------------
int main(int argc, char *argv[])
{
  const QByteArray data ( QByteArray::fromHex ( argv [1] ) );
  const QByteArray sign ( QByteArray::fromHex ( argv [2] ) );
  const QByteArray pubk ( QByteArray::fromHex ( argv [3] ) );

  quint8 tmp [32];
  ::SHA256 ( (const quint8*)data.constData ( ), data.size ( ), tmp );
  quint8 digest [32];
  ::SHA256 ( tmp, 32, digest );
  qDebug ( ) << "data=" << QString ( data.toHex ( ) );
  qDebug ( ) << "sign=" << QString ( sign.toHex ( ) );
  qDebug ( ) << "pubk=" << QString ( pubk.toHex ( ) );

  qDebug ( ) << "digest=" << QString ( QByteArray ( (const char*)digest, 32 ).toHex ( ) );

  const bool v ( ::ECDSA_verify ( 0, digest, 32, (const quint8*)sign.constData ( ), sign.size ( ), EC_KEY_pub_key ( pubk ) ) );
  qDebug ( ) << "result=" << v;

  return 0;
}

amaclin

Posted 2015-05-19T14:11:01.540

Reputation: 5 763

1excellent thanks. its very strange its failing to verify on two of my pcs! i will test it with the rpc client and also with pybitcointools. if the rpc client fails then it must be a problem with a particular version/configuration of openssl... – mulllhausen – 2015-05-21T00:53:10.080

oh, yes! the signature has non-standard der-encoding! Just look to the start of it: 30440220bb... - the proper encoding is 3044022100bb... because R-part can not be negative and starts with "bb". Some OpenSSL versions allow this, some others - not. See http://bitcoin.stackexchange.com/questions/2376/ecdsa-r-s-encoding-as-a-signature

– amaclin – 2015-05-21T14:10:05.610

1to be correct 3045022100bb... – amaclin – 2015-05-21T14:48:18.927

1no luck with the rpc client - see my updated op :p – mulllhausen – 2015-05-21T14:53:01.037

1I have never used rpc for verifying transaction signatures. Seems to me that this call is intended for something else. Drop me a mail to alistermaclin@mail.ru and I will send you Qt/C++ code for checking such things – amaclin – 2015-05-21T15:16:23.210

Yes, OpenSSL versions since Jan. 2015 check correct DER in signatures (DSA and RSA as well as ECDSA); that's 0.9.8zd+ 1.0.0p+ 1.0.1k+ and 1.0.2*. http://bitcoin.stackexchange.com/questions/35841/get-latest-blocks-with-bitcoind-on-ubuntu is the same problem.

– dave_thompson_085 – 2015-05-24T18:43:17.763

i came accross bip66 while googling and this addresses the problem.

– mulllhausen – 2015-05-27T01:05:52.733