Recent comments in /f/technology

nitori OP wrote

Both Basic and Digest access authentication are improved to provide a better native-looking browser-based experience than form-based authentication.

Oh how I wish we got Cookie-based authentication implemented straight in HTTP itself instead of having to use forms...

The spec has been updated with a new set of accepted headers - and in a break with past tradition, any header not in the list of accepted headers is to be rejected by a compliant server.

Wait that just breaks backwards compatibility with HTTP/1.1, how can this joke protocol be 1.2 lol

2

nitori OP wrote

Actually perhaps we might not need compression for the response headers even, but some sort of ETag.. There'd be like a Headers-ETag for the unique value and Headers-ETag-Names (I'm not satisfied with this name but can't think of something better) for the list of redundant headers to not be repeated in subsequent requests

2

nitori OP wrote (edited )

as i understand it, this mode is never going to happen under normal browsing, though.

I don't think any of my scenarios are normal at all lol :P

HTTP_1_1_REQUIRED (0x0d)

I definitely did not know about this until now, thanks! And searching online it seems like curl does retry its request in HTTP/1.1 if it encounters this. Personally I think it still would've made more sense for the HTTP/2 authors to extend 505 instead, especially since they kept the 1.1 response codes from 2xx-5xx (except 426) anyway, and you can explain to the user why you can't support HTTP/2 for the request in a 505's body... But glad to know there's an error code that can signal to the client to downgrade

3

emma wrote (edited )

but rereading the spec, you're supposed to send a 505 in the representation used by the major version requested by the client

GET / HTTP/2.0 is parsed with http/1 semantics, so i think it makes sense to give any >= 2.x version the HTTP/1.1 505 treatment.

An HTTP/2 request can be sent without negotiation; this is how h2c (HTTP/2 over cleartext) reliably works for me (for some reason I couldn't get Upgrade from http/1.1 to h2c working). It's called "prior knowledge", and curl supports this.

yeah, but as the name implies, you somehow know in advance that the server's gonna accept HTTP/2 if you send those. i suppose 505 here would make sense, if the HTTP/2 support was ever removed. as i understand it, this mode is never going to happen under normal browsing, though.

No, 505 wouldn't be useful because an HTTP/2 request to an HTTP/1-only server would only result in the client just closing the connection itself. You can see this by using nghttp or curl --http2-prior-knowledge against a server that only supports HTTP/1

An HTTP/2-only client (which those two commands earlier are) would not bother to process an HTTP/1 response (if it even gets one) whether that'd be a 505 or 200.

i meant a hypothetical http/2 that's more http/1-like, not the actual http/2 that came into existence which made it very hard to accidentally use the wrong protocol.

anyway, the solution to your woes is apparently to send an error packet or whatever:

HTTP_1_1_REQUIRED (0x0d):
The endpoint requires that HTTP/1.1 be used instead of HTTP/2.

it sounds like it does what you want, but i have no idea if this applies on the stream or the connection level or what.

3

nitori OP wrote (edited )

Hmm I don't think nginx is correct to send a 505 in that case. I actually thought as well before that it was correct, but rereading the spec, you're supposed to send a 505 in the representation used by the major version requested by the client. But nginx does it in 1.1's representation instead of 2.0:

GET / HTTP/2.0

HTTP/1.1 505 HTTP Version Not Supported
Server: nginx
[...]

A more appropriate response might be 400 or 500, since HTTP/2 obviously isn't plain text, and the client is trying to do a HTTP/2 request in HTTP/1 format which is wrong.

Having said that..

http/2+ requests are sent after negotiation, at which point it's established they are accepted. this obsoletes the need for a 505.

An HTTP/2 request can be sent without negotiation; this is how h2c (HTTP/2 over cleartext) reliably works for me (for some reason I couldn't get Upgrade from http/1.1 to h2c working). It's called "prior knowledge", and curl supports this.

Even if negotiation becomes strictly required (which Google and Mozilla wanted by requiring TLS) in all of HTTP/2, I don't think 505 is obsolete. If for some reason you want to sunset HTTP/2 and have users use HTTP/5+, while not leaving those still stuck with HTTP/2 in the dark, how would you signal to them that you refuse to support HTTP/2? A 505 would be able to fulfill that role, and indeed this is one of its intended purposes when it was first proposed.

505 would have been useful for a future where http/2 requests might be preemptively sent to http/1-only servers.

No, 505 wouldn't be useful because an HTTP/2 request to an HTTP/1-only server would only result in the client just closing the connection itself. You can see this by using nghttp or curl --http2-prior-knowledge against a server that only supports HTTP/1

An HTTP/2-only client (which those two commands earlier are) would not bother to process an HTTP/1 response (if it even gets one) whether that'd be a 505 or 200.

since you very much have to opt in for http/2+, incompatibilities with it can be resolved by just not enabling it, and the use cases where one would want partial http/2 support on any given host are extremely contrived

Heh, perhaps. :D Maybe in an earlier time where a webmaster really wants (or needs, because maybe some impatient stockholder is forcing their client to deploy h2 even if they're not fully ready) the benefits of HTTP/2 (multiplexing is pretty cool after all) as soon as possible but have parts of their website not yet ready for the new version, this could've been pretty relevant...

2

emma wrote

I mean the same applies if an HTTP/1 response is a 505, right..?

no, since http/1 requests are sent preemptively without knowing if the server accepts them. http/2+ requests are sent after negotiation, at which point it's established they are accepted. this obsoletes the need for a 505.

505 would have been useful for a future where http/2 requests might be preemptively sent to http/1-only servers. if i send GET / HTTP/2.0 (or any non-1.x version) to nginx, it indeed responds with that status code. but as things turned out, the negotiation mechanism in http/2+ just sidesteps this problem altogether, so 505 ends up being little more than a relic from a time when people didn't know what the future of http held.

since you very much have to opt in for http/2+, incompatibilities with it can be resolved by just not enabling it, and the use cases where one would want partial http/2 support on any given host are extremely contrived, i would argue it's a good thing that support for it is declared on the connection level. it's one less special case for clients to deal with.

3

nitori OP wrote (edited )

I mean the same applies if an HTTP/1 response is a 505, right..? If the intent is to completely unsupport an HTTP version then the appropriate response is none at all. But I don't think that's what 505 is for, or at least that's not its main purpose (it can be a way to slowly remove HTTP/1 for whatever reason while not leaving users in the dark). All a 505 is is a signal to the client that it will not produce a response for the requested HTTP version other than for this 505. And indeed that's what the spec says:

The 505 (HTTP Version Not Supported) status code indicates that the server does not support, or refuses to support, the major version of HTTP that was used in the request message. The server is indicating that it is unable or unwilling to complete the request using the same major version as the client, as described in Section 2.5, other than with this error message. The server SHOULD generate a representation for the 505 response that describes why that version is not supported and what other protocols are supported by that server.

I think my (questionable usecase) example link fits within the "unwilling" and "other than" parts, and the requested path (which is / in my example) is a part of the "request message".

I can also think of a reasonably valid reason why a server would choose to 505 in a HTTP/2 response instead of not offering support at all. Perhaps one is using a third-party nginx module that doesn't support HTTP/2 (though why one would still use it in 2024 is a good question), and the webmaster is trying to find a way to disable HTTP/2 just for a specific location where that module is used. It's not actually possible without putting it in another server block with a different hostname (or in older versions of nginx, listening to another IP address!). The only other workaround right now is to put the location behind a proxy where you can force HTTP/1; that way the whole site can still use HTTP/2 and the user wouldn't even notice it in devtools. But tbh even if it does result in a nice and transparent HTTP/2 for the user, it still would be simpler if nginx just returned a 505 for HTTP/2 requests and the client automatically retried the request in HTTP/1.

I just wish the HTTP/2 and HTTP/3 spec writers found the time to write another MAY or SHOULD for 505 to make it a hint for the client to downgrade its HTTP version... Or even add a new header like Version-Allow as in the Allow header required by a 405 to indicate in a machine-readable format of what major HTTP versions it supports (instead of just putting it in the body which is ultimately not required)

3

nitori OP wrote (edited )

Btw if you try to set the HTTP version to something absurd like 1.6, the server will still return 200 OK. I really read the spec. ;) 505 is only meant for major versions, and the HTTP version that appears in the server's response can also signal the maximum HTTP minor version it can support (so if you sent a HTTP/1.0 request and the server responds with HTTP/1.1 before the status code, it's telling you that it can support HTTP/1.1, and vice versa)! HTTP/1 is really ahead of its time

3

emma wrote

I'm not much of a fan of ditching plain text for binary, since it makes debugging more complex

I don't think this always holds true, like there was one time at work where an outgoing http request was failing in a strange way, and it took us hours to discover that the environment variable holding the URL in production contained a trailing newline, which the client library didn't pick up on. So this resulted in the following request:

POST /some/shit
HTTP/1.1
X-Some-Header: etc

some payload

If the length of the URL was known ahead of time, as would be typical with a binary protocol, the server would have known the newline was part of it, and handled it accordingly. It wouldn't be friendly as a plain text protocol, but it would make parsing the request very unambiguous and robust.

On the other hand, we see things like http/2 support in curl on Debian 12 being just broken, and the maintainer being too scared to merge the fixes from upstream due to http/2's complexity. So this cuts both ways, I suppose.

Oh, you can write a server that doesn't implement keepalive (while doing everything else 1.1) and still be 1.1-compliant? Well that's neat I suppose!

Yeah, you can just ignore the client's wish for keep-alive and send Connection: close, according to RFC 7230. I imagine this has to be terrible if the client attempts pipelining.

This might be a cursed opinion but I do actually want all websites to be root/path-agnostic. So if you wanna host Postmill for example but you already have a separate service running in port 80/443, and can't do it in a separate domain (which would require another host in this reality) or port which would have its own root, then I should be able to put it in like /postmill instead.

I believe Postmill supports this, but I haven't tested. I think a lot of devs just ignore the possibility you'd want to host something a subpath, unfortunately.

2

nitori OP wrote (edited )

Excellent write-up as always emma :D

I'm not much of a fan of ditching plain text for binary, since it makes debugging more complex (compared to 1.1 where you can just telnet lol), though I do realize that it's necessary if multiplexing is going to be a thing. Idk, is all of this added complexity really worth it just to shave off probably just the same as pipelining would do? In an ideal world where pipelining would only help the websites that really need it even with so many optimizations already applied and considered and where pipelining implementations in servers, clients, and proxies are perfect, I don't think so. But we don't live in that world, and frustratingly I suppose multiplexing is the way to go...

Idk I just wish that for every performance improvement we make, I can just be excited and not think about how webdevs are just going to ruin everything and add so much shit on top of the shit that became a non-factor due to those improvements that the improvements become meaningless again. Instead of "hmm how do we make the web go back to square one >:)" we just go "wow this is amazing we've reached peak I think :D"

Anyway I do wholeheartedly agree that pipelining is fundamentally wrong (even though it does work if it works), it just looks like a silly hack lol.

the server isn't required to support these

Oh, you can write a server that doesn't implement keepalive (while doing everything else 1.1) and still be 1.1-compliant? Well that's neat I suppose!

If virtual hosts didn't exist, I reckon we'd just see as much stuff shoved onto the same host as possible, and more extensive use of the path parameter in cookies to achieve the same stuff we have separate virtual hosts for in this reality.

This might be a cursed opinion but I do actually want all websites to be root/path-agnostic. So if you wanna host Postmill for example but you already have a separate service running in port 80/443, and can't do it in a separate domain (which would require another host in this reality) or port which would have its own root, then I should be able to put it in like /postmill instead.

Like think about it, CDNs like Cloudflare centralizing every damn website like we have right now wouldn't just be feasible without IPv6. Anycast is out of the question and each website under the CDN would require its own IP. The only way for this to go wrong is if every ISP just sold all of their address spaces to the CDNs and NATed the hell out of IPv4 that our own CG-NATs would sweat in fear of what we have created. But that's so ridiculous pessimistic imo that I don't think it will just happen. Well, hopefully.. :P

This exists because some http responses are produced before there's a known content length, thus the content-length header cannot be sent. It wouldn't be necessary if one connection handled a single request, though.

Oh yeah this is actually good lol, silly me :P

Looking into it more it seems like in HTTP/1.0 when there's no Content-Length, the client just assumes the transfer is successfully complete when the connection is closed. Which isn't good because we don't actually know whether the transfer was actually successful or it just got interrupted. 1.1's chonk stuff seems to be for that :D (EDIT: Actually maybe not but still neat regardless)

5

emma wrote

ok so like, i've made things for the web for a very long time, including at a time before http/2 and spdy, and http/1.1 has a bunch of very annoying limitations that http/2 solved. i've also written my own http/1.1 and fastcgi servers, to give you an idea of my level of experience. while we can all agree that http/2 is a shitty protocol, and should not have been Like That, it did solve some real problems.

The big one is lack of multiplexing. your html and stylesheets and scripts and images and fonts and other assets get loaded one after the other with http/1.1, and the burden was placed on the developer to figure out the bottlenecks and speed up page loading by placing the assets on separate hosts. We had entire services dedicated to pushing your site through them to try and spot these bottlenecks, and spent a lot of effort trying to fix them. Abominable ideas like shared CDNs for javascript libraries and server-side "compilation" of css and javascript largely stem from trying to work around the lack of multiplexing. Now we can simply serve all and many assets from the same host without thinking too much about it.

FastCGI (a pseudo-http protocol for application backends) is a binary protocol and had multiplexing since it was introduced in the 90s. It is reasonably simple to implement (and I much prefer working with binary <data size> <data> protocols to http/1.x's plain text protocol), and http/2 really ought to have just been a version of it.

While it's true that pipelining can improve performance without the need for http/2, it was always, fundamentally, the wrong solution. On top of it, i doesn't help that http/1.1's rules for when pipelining requests is allowed are surprisingly complex, and we ended up with a bunch of buggy implementations that led to pipelining being disabled in new stuff.

Mandatory keepalive when you don't send a Connection header?

As you've already discovered, keepalive is actually useful, so I won't go too deep into that. The opt-out mechanisms are very simple (request HTTP/1.0 or send Connection: close), and the server isn't required to support these, so I don't think this is a big deal.

Virtual hosts? If the spec writers just knew how their little hack would ultimately spell doom for IPv6 quickly replacing IPv4 for everyone they would've gotten second thoughts on it.

I don't think virtual hosts are the reason for IPv6's slow adoption. We have like 1 year old companies pretending they have technical debt from before IPv6's introduction. If virtual hosts didn't exist, I reckon we'd just see as much stuff shoved onto the same host as possible, and more extensive use of the path parameter in cookies to achieve the same stuff we have separate virtual hosts for in this reality.

Chunked transfer encoding? Ummmmmm, FTP? (Tbh I haven't really familiarized myself in this part lol)

This exists because some http responses are produced before there's a known content length, thus the content-length header cannot be sent. It wouldn't be necessary if one connection handled a single request, though.

6