Overhaul clients

[josecelano]

This is an epic issue for tracking progress with clients.

• Overhaul UDP Tracker client.
  • UDP Tracker client: Print unexpected responses in JSON #670
  • UDP Tracker client: Print unrecognized responses #671
• Overhaul HTTP Tracker client.
  • HTTP Tracker client: Print unrecognized responses in JSON #672
    • Serialization from Bencode to JSON #673
• Overhaul Tracker Checker.
  • Tracker Checker: Ouput in JSON #674
  • Tracker Checker: Support different types of tracker addresses #675
  • Tracker Checker: gracefully handle timeouts #677
    • Tracker Checker: handle UDP Tracker timeouts #682
      • Refactor bit_torrent::tracker::udp::client::UdpClient to return errors instead of panicking #681
    • Tracker Checker: handle HTTP Tracker timeouts #679
      • Refactor: bit_torrent::tracker::http::client::Client to return errors instead of panicking #680
    • Tracker Checker: handle Health Check timeouts #678
  • Tracker Checker: handle HTTP Tracker error ConnectionRefused #683
  • Tracker Checker: Improve errors. More concrete errors #676
  • Tracker Checker (UDP): Don't stop on the first fail #1040
  • Tracker Checker (HTTP): Improve error message when the JSON config is not well-formatted #1042
• De-duplicate Integration Testing and Tracker Client Code #767
• Merge clients into a single Tracker Client. See Should we merge testing binaries into a single `Tracker Client`? #660

NOTES:

• The list order is the recommended order of implementation.
• If you want to contribute to one task but there is no issue yet or want more info feel free to ask for more information.

Introduction

I've recently added three new tools to the repo. These three console commands can be used for testing and debugging. Since I started working on this project, I've missed tools that can help developers and sys admins detect and fix trackers' problems. The new three console commands are:

HTTP Tracker Client

This client allows making announce and scrape requests to HTTP trackers.

Announce:

cargo run --bin http_tracker_client announce http://127.0.0.1:7070 9c38422213e30bff212b30c360d26f9a02136422

{
  "complete": 1,
  "incomplete": 0,
  "interval": 120,
  "min interval": 120,
  "peers": []
}

Notice: all parameters except the info_hash are hardcoded. This client is not intended (yet) to be used with a full request from inputs. It's only used for testing so it generates a sample request. It could be changed to accept all parameters as a json object in the future. It depends on this.

You could use any browser for the HTTP Tracker Client, but the main advantage is getting the response in JSON instead of Bencoded format.

Scrape:

cargo run --bin http_tracker_client scrape http://127.0.0.1:7070 9c38422213e30bff212b30c360d26f9a02136422

{
  "9c38422213e30bff212b30c360d26f9a02136422": {
    "complete": 1,
    "downloaded": 0,
    "incomplete": 0
  }
}

UDP Tracker Client

This client allows making announce and scrape requests to UDP trackers.

Announce:

cargo run --bin udp_tracker_client announce http://127.0.0.1:6969 9c38422213e30bff212b30c360d26f9a02136422

{
  "transaction_id": -888840697,
  "announce_interval": 120,
  "leechers": 0,
  "seeders": 1,
  "peers": []
}

Scrape:

cargo run --bin udp_tracker_client scrape http://127.0.0.1:6969 9c38422213e30bff212b30c360d26f9a02136422

{
  "transaction_id": -888840697,
  "torrent_stats": [
    {
      "seeders": 1,
      "completed": 0,
      "leechers": 0
    }
  ]
}

Notice: this is fully implemented as the only parameter for the scrape request is the info_hash.

This client is very convenient because there is no easy way to build and send UDP packets. And some BitTorrent clients don't show you the Tracker response. (https://www.biglybt.com/ does it).

Tracker Checker

You can use this command to test running services on trackers. It can check:

• UDP trackers: it makes an announce and scrape request.
• HTP trackers: it makes an announce and scrape request.
• HTTP endpoints (health check endpoints): it makes a GET request to the provided endpoint and checks that the endpoint returns a 200 HTTP status code.

You can run it for the local dev env with:

TORRUST_CHECKER_CONFIG='{
    "udp_trackers": ["127.0.0.1:6969"],
    "http_trackers": ["http://127.0.0.1:7070"],
    "health_checks": ["http://127.0.0.1:1313/health_check"]
}' cargo run --bin tracker_checker

or for the live demo with:

TORRUST_CHECKER_CONFIG='{
    "udp_trackers": ["144.126.245.19:6969"],
    "http_trackers": ["https://tracker.torrust-demo.com"],
    "health_checks": ["https://tracker.torrust-demo.com/health_check"]
}' cargo run --bin tracker_checker

Output:

Running checks for trackers ...
UDP trackers ...
✓ - Announce at 144.126.245.19:6969 is OK
✓ - Announce at 144.126.245.19:6969 is OK
HTTP trackers ...
✓ - Announce at https://tracker.torrust-demo.com/ is OK
✓ - Scrape at https://tracker.torrust-demo.com/ is OK
Health checks ...
✓ - Health API at https://tracker.torrust-demo.com/health_check is OK

Considerations

This code is still in beta, some parts were moved from testing code to production code. For example the UdpClient. That means it's not covered by tests. We should add tests while we implement all these new features. I merged as it is because I wanted to use the Checker in the testing workflow (E2E tests). I wanted to enable this feature asap to avoid regressions when we merge PRs, since a lot of the bootstapping of the app is not covered by unit tests.