cucked transfer encoding

IDK I think there is a use for keeping the same stream open if you're a big website serving a lot of clients tbh. Each TCP handshake takes three packets minimum (unless you use TCP fastopen which is its whole own thing), and then if you want SSL on top of that there's even more latency, especially for slow connections, plus the computation, which is small per request but if you're a big site serving a lot of people it adds up. Even if you're not jamming your page full of ten trillion google ads it can add up.

Using the same connection again if you expect the client to make another one pretty soon makes a lot of sense.

I don't do web dev tho so what do I know.

I think TCP FO should be the way to go since it's more elegant imo than keeping a connection open, though unfortunately ossification means it will take a very long while to get all TCP-based services and clients to support it.. There's also privacy issues with its cookies

As for SSL, if we just had tcpcrypt or any other opportunistic encryption we wouldn't need Let's Encrypt or any free TLS lol (I feel like TLS has been abused too much, it should've been more about identity verification than encryption). I'm actually hopeful for Yggdrasil since it's an IPv6 mesh network where end-to-end encryption between IPs is the norm and each IP is a public key

I changed my mind a bit about keepalive. I think that's necessary for reverse proxies when connecting to their upstream lol

But I still question its use in a real server-client model

dont know what this one means. hope everybody is having a great time in this thread

ok so like, i've made things for the web for a very long time, including at a time before http/2 and spdy, and http/1.1 has a bunch of very annoying limitations that http/2 solved. i've also written my own http/1.1 and fastcgi servers, to give you an idea of my level of experience. while we can all agree that http/2 is a shitty protocol, and should not have been Like That, it did solve some real problems.

The big one is lack of multiplexing. your html and stylesheets and scripts and images and fonts and other assets get loaded one after the other with http/1.1, and the burden was placed on the developer to figure out the bottlenecks and speed up page loading by placing the assets on separate hosts. We had entire services dedicated to pushing your site through them to try and spot these bottlenecks, and spent a lot of effort trying to fix them. Abominable ideas like shared CDNs for javascript libraries and server-side "compilation" of css and javascript largely stem from trying to work around the lack of multiplexing. Now we can simply serve all and many assets from the same host without thinking too much about it.

FastCGI (a pseudo-http protocol for application backends) is a binary protocol and had multiplexing since it was introduced in the 90s. It is reasonably simple to implement (and I much prefer working with binary <data size> <data> protocols to http/1.x's plain text protocol), and http/2 really ought to have just been a version of it.

While it's true that pipelining can improve performance without the need for http/2, it was always, fundamentally, the wrong solution. On top of it, i doesn't help that http/1.1's rules for when pipelining requests is allowed are surprisingly complex, and we ended up with a bunch of buggy implementations that led to pipelining being disabled in new stuff.

Mandatory keepalive when you don't send a Connection header?

As you've already discovered, keepalive is actually useful, so I won't go too deep into that. The opt-out mechanisms are very simple (request HTTP/1.0 or send Connection: close), and the server isn't required to support these, so I don't think this is a big deal.

Virtual hosts? If the spec writers just knew how their little hack would ultimately spell doom for IPv6 quickly replacing IPv4 for everyone they would've gotten second thoughts on it.

I don't think virtual hosts are the reason for IPv6's slow adoption. We have like 1 year old companies pretending they have technical debt from before IPv6's introduction. If virtual hosts didn't exist, I reckon we'd just see as much stuff shoved onto the same host as possible, and more extensive use of the path parameter in cookies to achieve the same stuff we have separate virtual hosts for in this reality.

Chunked transfer encoding? Ummmmmm, FTP? (Tbh I haven't really familiarized myself in this part lol)

This exists because some http responses are produced before there's a known content length, thus the content-length header cannot be sent. It wouldn't be necessary if one connection handled a single request, though.

Excellent write-up as always emma :D

I'm not much of a fan of ditching plain text for binary, since it makes debugging more complex (compared to 1.1 where you can just telnet lol), though I do realize that it's necessary if multiplexing is going to be a thing. Idk, is all of this added complexity really worth it just to shave off probably just the same as pipelining would do? In an ideal world where pipelining would only help the websites that really need it even with so many optimizations already applied and considered and where pipelining implementations in servers, clients, and proxies are perfect, I don't think so. But we don't live in that world, and frustratingly I suppose multiplexing is the way to go...

Idk I just wish that for every performance improvement we make, I can just be excited and not think about how webdevs are just going to ruin everything and add so much shit on top of the shit that became a non-factor due to those improvements that the improvements become meaningless again. Instead of "hmm how do we make the web go back to square one >:)" we just go "wow this is amazing we've reached peak I think :D"

Anyway I do wholeheartedly agree that pipelining is fundamentally wrong (even though it does work if it works), it just looks like a silly hack lol.

the server isn't required to support these

Oh, you can write a server that doesn't implement keepalive (while doing everything else 1.1) and still be 1.1-compliant? Well that's neat I suppose!

If virtual hosts didn't exist, I reckon we'd just see as much stuff shoved onto the same host as possible, and more extensive use of the path parameter in cookies to achieve the same stuff we have separate virtual hosts for in this reality.

This might be a cursed opinion but I do actually want all websites to be root/path-agnostic. So if you wanna host Postmill for example but you already have a separate service running in port 80/443, and can't do it in a separate domain (which would require another host in this reality) or port which would have its own root, then I should be able to put it in like /postmill instead.

Like think about it, CDNs like Cloudflare centralizing every damn website like we have right now wouldn't just be feasible without IPv6. Anycast is out of the question and each website under the CDN would require its own IP. The only way for this to go wrong is if every ISP just sold all of their address spaces to the CDNs and NATed the hell out of IPv4 that our own CG-NATs would sweat in fear of what we have created. But that's so ridiculous pessimistic imo that I don't think it will just happen. Well, hopefully.. :P

This exists because some http responses are produced before there's a known content length, thus the content-length header cannot be sent. It wouldn't be necessary if one connection handled a single request, though.

Oh yeah this is actually good lol, silly me :P

Looking into it more it seems like in HTTP/1.0 when there's no Content-Length, the client just assumes the transfer is successfully complete when the connection is closed. Which isn't good because we don't actually know whether the transfer was actually successful or it just got interrupted. 1.1's chonk stuff seems to be for that :D (EDIT: Actually maybe not but still neat regardless)

i am going to write a protocol which uses TCP for sending syntactically-correct tags and media in a page, but all the context is indexed from an array which comes in through a custom checksumless UDP. let me see just how many errors the IETF really saved me from

I'm not much of a fan of ditching plain text for binary, since it makes debugging more complex

I don't think this always holds true, like there was one time at work where an outgoing http request was failing in a strange way, and it took us hours to discover that the environment variable holding the URL in production contained a trailing newline, which the client library didn't pick up on. So this resulted in the following request:

POST /some/shit
HTTP/1.1
X-Some-Header: etc

some payload

If the length of the URL was known ahead of time, as would be typical with a binary protocol, the server would have known the newline was part of it, and handled it accordingly. It wouldn't be friendly as a plain text protocol, but it would make parsing the request very unambiguous and robust.

On the other hand, we see things like http/2 support in curl on Debian 12 being just broken, and the maintainer being too scared to merge the fixes from upstream due to http/2's complexity. So this cuts both ways, I suppose.

Oh, you can write a server that doesn't implement keepalive (while doing everything else 1.1) and still be 1.1-compliant? Well that's neat I suppose!

Yeah, you can just ignore the client's wish for keep-alive and send Connection: close, according to RFC 7230. I imagine this has to be terrible if the client attempts pipelining.

This might be a cursed opinion but I do actually want all websites to be root/path-agnostic. So if you wanna host Postmill for example but you already have a separate service running in port 80/443, and can't do it in a separate domain (which would require another host in this reality) or port which would have its own root, then I should be able to put it in like /postmill instead.

I believe Postmill supports this, but I haven't tested. I think a lot of devs just ignore the possibility you'd want to host something a subpath, unfortunately.