feat(http_client): move to opendht's builtin HTTP proxy - #34
Conversation
sim590
changed the title
| {"data", encoded_data}, | ||
| {"utype", Node::DPASTE_USER_TYPE} | ||
| }.dump(); | ||
| req.setOpt(curlpp::Options::PostFields(body)); |
There was a problem hiding this comment.
Why use curlpp::Options::PostFields instead of keeping curlpp::options::HttpPost? You also create a temporary body variable instead of simply create the form parts in-place as we were doing before with curlpp::FormParts::Content which was taking care of creating the dictionary of values by itself. It's not a dealbreaker, but why not use the same approach that we were using before?
There was a problem hiding this comment.
Proxy POST requires raw JSON PostFields, not multipart HttpPost.
| std::stringstream response, oss; | ||
| req.setOpt<curlpp::options::Url>(HTTP_PROTO+ | ||
| host+"/"+dht::InfoHash::get(code).toString() | ||
| +"?user_type="+dpaste::Node::DPASTE_USER_TYPE |
There was a problem hiding this comment.
You removed the user_type spec from the URL. That makes it so the get operation on the DHT node service's side won't have this spec and it will then pull more values than it should. This usertype spec is used in the DHT protocol to filter out values so requests are as light as possible and also per-application specific.
I know that you are filtering values out afterwards, but at this point it's too late, network traffic was wasted for no reason.
Let's keep this network level optimization.
There was a problem hiding this comment.
The problem is the migration to OpenDHT’s DhtProxyServer: its new GET /key/<hash> handler ignores query parameters entirely. Adding ?user_type=dpaste back would look correct but provide no DHT-level filtering.
| d.decode(iss, oss); | ||
| std::istringstream lines(response.str()); | ||
| std::string line; | ||
| while (std::getline(lines, line)) { |
There was a problem hiding this comment.
How to be sure that parsing line by line the output of the service will lead to valid JSON content? We can't know. The server's response should rather be parsed in its entirety.
There was a problem hiding this comment.
Because we are reading NDJSON files. By definition, each line is a valid JSON document.
2 participants
This pull request introduces significant improvements to how
dpasteinteracts with OpenDHT, providing a more robust and efficient DHT proxy setup. The main changes include adding support for an optional persistent local OpenDHT proxy via a systemd user unit, updating documentation and configuration to reflect this, and refactoring HTTP client interactions to use a new JSON-based API with base64 encoding. Additionally, installation scripts are updated to install the new systemd unit.DHT Proxy Integration and Systemd Unit:
dpaste-dhtnode.servicesystemd user unit to run a persistent local OpenDHT proxy, and updated installation scripts (CMakeLists.txt,Makefile.am) to install this unit. [1] [2] [3] [4]README.mdanddoc/dpaste.1to describe how to enable and use the systemd DHT proxy, and clarified the configuration file'shostandportsettings. [1] [2]Configuration and Documentation Updates:
config/dpaste.confto clarify that thehostandportrefer to the OpenDHTDhtProxyServerendpoint.Codebase Changes for Proxy Support:
Binclass insrc/bin.cpp/src/bin.hto only start a local DHT node if the proxy is unavailable, avoiding unnecessary node startups. IntroducedensureNodeRunning()and anode_running_flag. [1] [2] [3] [4] [5]HTTP Client Refactoring:
HttpClientinsrc/http_client.cppto use a new JSON API for the proxy, including base64 encoding/decoding for data transmission, and improved error handling and compatibility with the proxy server. [1] [2] [3]These changes collectively improve performance, reliability, and usability for users who wish to run a persistent DHT proxy, and modernize the client-server communication protocol.