API: add scrape endpoint.

[josecelano]

Relates to: #708

I would like to improve the importation of statistics in the Index.

Currently, the Index is getting statistics from a single torrent at the time using the endpoint to get the torrent info:

/torrent/info-hash

I would like to add a new endpoint to import statistics in batches.

/tracker/scrape?info-hash=3b245504cf5f11bbdbe1201cea6a6bf45aee1bc0,info-hash=...

You can specify up to 74 torrents at the same time.

The normal HTTP tracker response for a scrape request is (converted to JSON format):

{
  "090c6d4fb3a03191c4ef1fda6236ef0efb2d5c10": {
    "complete": 1,
    "downloaded": 1,
    "incomplete": 0
  }
}

I think we can use a structure that is better for JSON, matching the Rust struct so that it's easier to generate the in-memory representation from the JSON object.

This is the proposed JSON format for the new scrape endpoint. It would follow the scrape specification but in JSON.

[
  {
    "info_hash": "090c6d4fb3a03191c4ef1fda6236ef0efb2d5c10",
    "complete": 0,
    "downloaded": 0,
    "incomplete": 0
  }
]

In Rust the struct would be:

#[derive(Debug, Serialize, Deserialize, PartialEq)]
pub struct TorrentInfo {
    pub info_hash: String,
    pub complete: i64,
    pub downloaded: i64,
    pub incomplete: i64,
}

Current implementation in the Index

The current struct used by the Index for the torrent details endpoint:

#[derive(Debug, Serialize, Deserialize, PartialEq)]
pub struct TorrentInfo {
    pub info_hash: String,
    pub seeders: i64,
    pub completed: i64,
    pub leechers: i64,
    pub peers: Vec<Peer>,
}

Notice, we are also importing the peer list and we don't need it in the Index.

Torrent endpoints

List of torrents:

http://127.0.0.1:1212/api/v1/torrents?token=MyAccessToken

[
  {
    "info_hash": "090c6d4fb3a03191c4ef1fda6236ef0efb2d5c10",
    "seeders": 1,
    "completed": 1,
    "leechers": 0
  }
]

Torrent info (the one currently used by the Index to import statistics):

http://127.0.0.1:1212/api/v1/torrent/090c6d4fb3a03191c4ef1fda6236ef0efb2d5c10?token=MyAccessToken

{
  "info_hash": "090c6d4fb3a03191c4ef1fda6236ef0efb2d5c10",
  "seeders": 1,
  "completed": 1,
  "leechers": 0,
  "peers": [
    {
      "peer_id": {
        "id": "0x2d71423030303030303030303030303030303031",
        "client": null
      },
      "peer_addr": "0.0.0.0:17548",
      "updated": 1709916034742,
      "updated_milliseconds_ago": 1709916034742,
      "uploaded": 0,
      "downloaded": 0,
      "left": 0,
      "event": "Completed"
    }
  ]
}

Extra considerations

This endpoint will be useful even if we decide to import statistics only for the torrents that are being loaded in the views in the Index. See torrust/torrust-index#469 (comment). Becuase, in the torrent list page the Index also shows the seeders and leechers:

So we need to get statistics for more than one torrent at the same time (for each result page)

[josecelano]

In the end, I decided to use the torrents endpoint. It contains the same information as the tracker "scrape" request. Currently, the torrent list resource only contains the torrent's stats:

[
  {
    "info_hash": "9c38422213e30bff212b30c360d26f9a02136422",
    "seeders": 1,
    "completed": 0,
    "leechers": 0
  },
  {
    "info_hash": "2b66980093bc11806fab50cb3cb41835b95a0362",
    "seeders": 1,
    "completed": 0,
    "leechers": 0
  }
]

I've only added a new query parameter info_hash which is an array. You can specify directly the list of torrents you want to get.

http://127.0.0.1:1212/api/v1/torrents?token=MyAccessToken&info_hash=9c38422213e30bff212b30c360d26f9a02136422&info_hash=2b66980093bc11806fab50cb3cb41835b95a0362