Day 20: Striving for equality

Yesterday I began adding an interface so that Séance could reach out to the network via an HttpGet action. At the time, the action was defined like this:

#[derive(Debug, Clone, PartialEq, Eq)]
enum Action {
    // ...
    HttpGet(HttpGetId, Url)
}

with a corresponding completion message defined similarly:

#[derive(Debug, Clone, PartialEq, Eq)]
enum Message {
    // ...
    HttpGetComplete(HttpGetId, Result<Vec<u8>, ???>)
}

I mentioned then that I was still figuring out how I should represent HTTP errors. I didn’t want to expose the errors returned by the underlying HTTP client (reqwest in this case) because that would couple the implementation to what should be an interchangeable detail.

Today I realized a more fundamental shortcoming of the design: HTTP responses are not Vec<u8>s! I was imagining that all I had to do when I got a successful response was parse it, but there may also be useful information in the HTTP headers that I care about. For example, the Content-Type header might indicate that the page is actually HTML. This would indicate an error in the config, or possibly an error with the web service (maybe it’s a Cloudflare error page, for example). In either case, I would want to surface this information to the person who requested the synchronization.

At this point, I remembered the existence of the http crate, which aims to be a series of pseudo-standard library type definitions for the HTTP protocol. Notably, it doesn’t contain any actual HTTP logic (other than things like parsing URLs), with the intent being that web services or clients could couple themselves to this common definition but still be able to change the backing library that uses the types. This sounded perfect!

I drafted a new version of the HTTP action and response message. Because http defines a generic Request struct which covers all methods, I would also update them to be generic across methods rather than specific to GET. That looked like this:

#[derive(Debug, Clone, PartialEq, Eq)]
enum Action {
    // ...
    HttpRequest(HttpRequestId, http::Request<Vec<u8>>)
}

#[derive(Debug, Clone, PartialEq, Eq)]
enum Message {
    // ...
    HttpRequestComplete(HttpRequest, Result<http::Response<Vec<u8>>, HttpError>)
}

However, this didn’t work. The rust-analyzer began complaining with the following error:

binary operation `==` cannot be applied to type `&http::Request<Vec<u8>>`
the foreign item type `http::Request<Vec<u8>>` doesn't implement `PartialEq`

A similar error was shown for http::Response. In other words, the types don’t implement the Eq, trait so Rust was no longer able to derive a definition for Message or Action types in turn. I want to have these because they will make equality-based unit testing easier (ie. I can check that the actions created by the code are equal to those I expect).

This seemed strange to me; shouldn’t equality be well defined on HTTP messages? Sure, it’s more complicated than strict string equality, since you would have to ignore case for the case-insensitive headers, etc., but complicated doesn’t mean impossible. And finding a way to reflect the nuances of some specification in the type system is one of the favourite activities of the standard Rust developer. So what gives?

I decided to track this mystery down to its source. Literally, the source code (and generated documentation). Indeed, Request is a simple composite of multiple Parts, such as a Method, HeaderMap, Url, etc. As I’d expected, each of these interior types implemented Eq themselves. All except for one: extensions, which is supposed to represent non-standard extensions to the HTTP protocol.

My best guess then is that the http crate wants to be able to represent non-standard extensions, but doesn’t know the equality characteristics of those extensions. Therefore, the core Request and Response types can’t implement Eq either. This is unfortunate, because it removes a useful feature because of needing to support what I have to assume is a rare edge case. For now, I’ve decided to implement the HTTP types myself, limiting them only to those that I actually need. But if I end up needing most of the API that http already exposes, I may instead create a newtype like this:

struct StandardRequest<T>(http::Request<T>)

This type will ensure that the request contains no extensions during construction. I can then implement equality on it in terms of the existing equality definitions for http::Request’s other fields extensions