Skip to content
This repository was archived by the owner on Aug 6, 2021. It is now read-only.

API Reverse Engineering

Jump to bottom
ChristianSch edited this page Aug 27, 2016 · 4 revisions

Requests

(uses the python module requests)

LOGIN

requests.post('https://api.numer.ai/login',
                          data={'email': username,
                                'password': userpass})

-> returns:

{'username': 'someusername',
'refreshToken': 'XXXXXXXXXXXXXXXXXXXXXXXX',
'id': 'XXXXXXXXXXXXXXXXXXXXXXXX',
'accessToken': 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'}

Fetch the competitions

requests.get('https://api.numer.ai/competitions?{%22leaderboard%22:%22current%22,%22end_date%22:{%22$gt%22:%222016-08-24T18:06:43.474Z%22}}').json()

the query params are build by the following js code:

"/competitions?" + JSON.stringify({
    end_date: {
        $gt: new Date(Date.now() - 55296e5)
    },
    leaderboard: "best25"
}

the data returned is quite verbose and includes the current and upcoming round at the time I checked:

upcoming round:

{"end_date": "2016-09-01T00:00:00.000Z", "title": "Round 16", "dataset_id": "57bd1a0c95c9fe518ccf18a8", "_id": "57bd19cb95c9fe518ccf18a7", "start_date": "2016-08-25T00:00:00.000Z", "leaderboard": []}

current round:

{"end_date": "2016-08-25T00:00:00.000Z", "title": "Round 15", "dataset_id": "57b4899195c9fe518ccf18a6", "_id": "57b4895595c9fe518ccf18a5", "start_date": "2016-08-18T00:00:00.000Z", "leaderboard": [{"username": "wsw", "originality": {"public": 0}, "earned": 6868.460000000002, "created": "2016-08-18T21:35:28.952Z", "rank": {"public": 1}, "logloss": {"public": {"current": "0.64714", "best": "0.64390"}}, "earnings": {"public": 738.4069145590257}}, {"username": "xirax", "originality": {"public": 0}, "earned": 8197.57, "created": "2016-08-23T15:11:16.460Z", "rank": {"public": 2}, "logloss": {"public": {"current": "0.66387", "best": "0.65538"}}, "earnings": {"public": 281.65942964506223}}, …]}

Past Submissions

Public:

requests.get('https://api.numer.ai/user/rogue').json()

-> result:

{u'username': u'rogue', u'created': u'2016-06-22T08:01:18.000Z', u'followers': 0, u'earnings': 0, u'_id': u'576a45ce1bf7c40f0063a961', u'submissions': [{u'accuracy_score': u'0.69478', u'_id': u'57bde0a4d996970f0027c7d7', u'metric': u'LOGLOSS', u'created': u'2016-08-24T18:00:04.755Z'}, {u'accuracy_score': u'0.69390', u'_id': u'57a49bd1c5ddad0f004b873f', u'metric': u'LOGLOSS', u'created': u'2016-08-05T13:59:48.696Z'}, {u'accuracy_score': u'0.69227', u'_id': u'5770de688f555e0f00e9c11b', u'metric': u'LOGLOSS', u'created': u'2016-06-27T08:06:02.242Z'}, {u'accuracy_score': u'0.69328', u'_id': u'5770de3b8f555e0f00e9c119', u'metric': u'LOGLOSS', u'created': u'2016-06-27T08:05:17.204Z'}, {u'accuracy_score': u'0.69227', u'_id': u'576aac771bf7c40f0063a994', u'metric': u'LOGLOSS', u'created': u'2016-06-22T15:19:20.842Z'}, {u'accuracy_score': u'0.69851', u'_id': u'576a6a43a5834b0f006de82f', u'metric': u'LOGLOSS', u'created': u'2016-06-22T10:36:53.358Z'}, {u'accuracy_score': u'0.69184', u'_id': u'576a64f30892270f002284f2', u'metric': u'LOGLOSS', u'created': u'2016-06-22T10:14:12.531Z'}], u'badges': []}

Private:

requests.get('https://api.numer.ai/user/rogue', headers={'Authorization':'Bearer XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'}).json()

-> result:

{u'username': u'rogue', u'rewards': [{u'start_date': u'2016-08-04T00:00:00.000Z', u'end_date': u'2016-08-11T00:00:00.000Z', u'earned': 0}], u'followed': False, u'created': u'2016-06-22T08:01:18.000Z', u'withdrawals': [], u'notifications': True, u'followers': 0, u'earnings': 0, u'balance': {u'claimed': 0, u'unclaimed': 0, u'withdrawal': 0}, u'_id': u'576a45ce1bf7c40f0063a961', u'submissions': [{u'filename': u'24_08_2016_Bernoulli_NB__a_0_64__bin_0_23.csv', u'accuracy_score': u'0.69478', u'_id': u'57bde0a4d996970f0027c7d7', u'metric': u'LOGLOSS', u'created': u'2016-08-24T18:00:04.755Z'}, {u'filename': u'tpot_nb_alpha_64_bin_0_23.csv', u'accuracy_score': u'0.69390', u'_id': u'57a49bd1c5ddad0f004b873f', u'metric': u'LOGLOSS', u'created': u'2016-08-05T13:59:48.696Z'}, {u'filename': u'predicted_gbc.csv', u'accuracy_score': u'0.69227', u'_id': u'5770de688f555e0f00e9c11b', u'metric': u'LOGLOSS', u'created': u'2016-06-27T08:06:02.242Z'}, {u'filename': u'predicted_rfc_500_0_5.csv', u'accuracy_score': u'0.69328', u'_id': u'5770de3b8f555e0f00e9c119', u'metric': u'LOGLOSS', u'created': u'2016-06-27T08:05:17.204Z'}, {u'filename': u'predicted_gbc.csv', u'accuracy_score': u'0.69227', u'_id': u'576aac771bf7c40f0063a994', u'metric': u'LOGLOSS', u'created': u'2016-06-22T15:19:20.842Z'}, {u'filename': u'predicted_rfc.csv', u'accuracy_score': u'0.69851', u'_id': u'576a6a43a5834b0f006de82f', u'metric': u'LOGLOSS', u'created': u'2016-06-22T10:36:53.358Z'}, {u'filename': u'predicted_gbc.csv', u'accuracy_score': u'0.69184', u'_id': u'576a64f30892270f002284f2', u'metric': u'LOGLOSS', u'created': u'2016-06-22T10:14:12.531Z'}], u'email': u'XXX@XXX.XXX', u'badges': [], u'last_logged_in': u'2016-08-24T18:33:10.742Z'}

Upload

Note: the Amazon S3 part seems to look a lot like this: https://gist.github.com/lukasz-madon/d723e9f4369c54724f62. Not sure where this is from.

  1. Authorize Upload
res = requests.post('https://api.numer.ai/upload/auth', data={'filename': 'XY', 'mimetype': 'text/csv'}, headers={'Authorization':'Bearer XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'}).json()

(probably text/csv as mimetype)

-> returns

{u'url': u'$AWS_URL', u'signedRequest': u'$SIGNED_REQUEST', u'filename': u'$FILENAME'}
  1. Upload to AWS bucket. (Not working yet, receiving either SignatureDoesNotMatch or NotImplemented errors.)
with open('/Users/foobar/Documents/Development/numerflow/24_08_2016_Bernoulli_NB__a_0_64__bin_0_23.csv', 'rb') as fp:
    res = requests.put(req['signedRequest'], data=fp, headers={'Referer': 'https://numer.ai', 'Origin':'https://numer.ai', 'x-amz-acl': 'public-read', 'Content-Type': 'text/csv'})
    print res.text

-> result

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>XXXXXXXXXXXXXXXXXXXX</AWSAccessKeyId><StringToSign>PUT

text/csv
1472071578
x-amz-acl:bucket-owner-full-control
/numerai-production-uploads/rogue/XXXXXXXXXXXX-24_08_2016_Bernoulli_NB__a_0_64__bin_0_23.csv</StringToSign><SignatureProvided>XXXXXXXXXXXXXXXXXX/XXXXXXXX=</SignatureProvided><StringToSignBytes>50 55 54 0a 0a 74 65 78 74 2f 63 73 76 0a 31 34 37 32 30 37 31 35 37 38 0a 78 2d 61 6d 7a 2d 61 63 6c 3a 62 75 63 6b 65 74 2d 6f 77 6e 65 72 2d 66 75 6c 6c 2d 63 6f 6e 74 72 6f 6c 0a 2f 6e 75 6d 65 72 61 69 2d 70 72 6f 64 75 63 74 69 6f 6e 2d 75 70 6c 6f 61 64 73 2f 72 6f 67 75 65 2f 68 37 49 39 4c 45 61 6b 78 45 6c 46 2d 32 34 5f 30 38 5f 32 30 31 36 5f 42 65 72 6e 6f 75 6c 6c 69 5f 4e 42 5f 5f 61 5f 30 5f 36 34 5f 5f 62 69 6e 5f 30 5f 32 33 2e 63 73 76</StringToSignBytes><RequestId>XXXXXXXXXXXXXXXX</RequestId><HostId>XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/XXXXXXXXXXXXXXXXXXXXXX=</HostId></Error>

I'm not quite sure why this fails, as the following, simple curl example shows it can work:

curl -X PUT --upload-file data/24_08_2016_Bernoulli_NB__a_0_64__bin_0_23.csv  'https://numerai-production-uploads.s3-us-west-1.amazonaws.com/rogue/XXXXXXXXXXXX-24_08_2016_Bernoulli_NB__a_0_64__bin_0_23.csv?AWSAccessKeyId= XXXXXXXXXXXXXXXXXXXX&Expires=1472072086&Signature=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&x-amz-acl=bucket-owner-full-control' -v
*   Trying 54.231.236.20...
* Connected to numerai-production-uploads.s3-us-west-1.amazonaws.com (54.231.236.20) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate: *.s3-us-west-1.amazonaws.com
* Server certificate: DigiCert Baltimore CA-2 G2
* Server certificate: Baltimore CyberTrust Root
> PUT /rogue/XXXXXXXXXXXX-24_08_2016_Bernoulli_NB__a_0_64__bin_0_23.csv?AWSAccessKeyId= XXXXXXXXXXXXXXXXXXXX&Expires=1472072086&Signature=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&x-amz-acl=bucket-owner-full-control HTTP/1.1
> Host: numerai-production-uploads.s3-us-west-1.amazonaws.com
> User-Agent: curl/7.43.0
> Accept: */*
> Content-Length: 741965
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< x-amz-id-2: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
< x-amz-request-id: XXXXXXXXXXXXXXXX
< Date: Wed, 24 Aug 2016 20:54:23 GMT
< x-amz-version-id: 8NsSB.0UX0dLaRKCuc4_mx07jzexzxTm
< ETag: "da9d431035bd68cf7e483317dd8b1266"
< Content-Length: 0
< Server: AmazonS3
<
* Connection #0 to host numerai-production-uploads.s3-us-west-1.amazonaws.com left intact

Header Comparison

request:

> Access-Control-Allow-Origin: *
> Access-Control-Allow-Methods: PUT
> Access-Control-Max-Age: 500
> Vary: Origin, Access-Control-Request-Headers, Access-Control-Request-Method
> x-amz-request-id: 4B29B267AD887391
> x-amz-id-2: uengmV8JZSwQ4eaWSoTuzW54NaH60KGUppCuEH/BbDluv9WbeOp/U/TfU0vqj6M5SLdJJIrGY1s=
> Content-Type: application/xml
> Transfer-Encoding: chunked
> Date: Sat, 27 Aug 2016 10:30:01 GMT
> Connection: close
> Server: AmazonS3

curl:

> PUT /rogue/XXXXXXXXXXXX-24_08_2016_Bernoulli_NB__a_0_64__bin_0_23.csv?AWSAccessKeyId= XXXXXXXXXXXXXXXXXXXX&Expires=1472072086&Signature=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&x-amz-acl=bucket-owner-full-control HTTP/1.1
> Host: numerai-production-uploads.s3-us-west-1.amazonaws.com
> User-Agent: curl/7.43.0
> Accept: */*
> Content-Length: 741965
> Expect: 100-continue

The second result:

>>> with open('/Users/nexus/Documents/Development/numerflow/24_08_2016_Bernoulli_NB__a_0_64__bin_0_23.csv', 'rb') as fp:
...     res = requests.put(req['signedRequest'], data=fp)
...     print res.text

->

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>NotImplemented</Code><Message>A header you provided implies functionality that is not implemented</Message><Header>Transfer-Encoding</Header><RequestId>XXXXXXXXXXXXXXXX</RequestId><HostId>g0qROMYORHDueGfcUGbxXY6vdIzrivLTyu9J7w2x+TX1Kv5qRwuJkrX1TkWogamvtifwZyl/Z08=</HostId></Error>

Solution: Don't send any headers besides Content-Length for the request to S3. See ApiController.py for more information.

  1. Submit predictions
requests.post('https://api.numer.ai/submissions', data={'competition_id': 'X', 'dataset_id': 'X…X', 'filename': 'foo.csv'})

this is how it looks in the js:

JSON.stringify({
    competition_id: o._id,
    dataset_id: o.dataset_id,
    filename: i.filename
})

Clone this wiki locally