HTTP2 server push in depth with node.js

HTTP/2 or HTTP/2.0, the upgraded version of the HTTP network protocol is derived from an experimental protocol, named SPDY, developed by Google. It’s the first revision of HTTP protocol since HTTP1.1 and contains many useful features in it. The most interesting and promising feature of HTTP/2 is server push. This article will give you a low level view of how server push works and how to implement it using node.js in the server. You can pickup any other server side language as well; we’re choosing node as majority of our users are from JavaScript ecosystem.

HTTP/2 server push

At a high level server push is as simple as giving you a soda bottle and a glass when you asked for a whiskey in a wine retail shop.
When the browser hits a particular route of your server and it serves an html file; you know it very well that in a few moment the browser will again ask for some css or JavaScript files (if not cached). So why not send those files along with that html file itself somehow?
The entire idea of server push is based on this. A server should be able to send or push extra files/streams even if there was no request from the client/browser.

The mechanism

  • Client requests for index.html to the server.
  • Server knows this file will need style.css and script.js soon, so he should forecast for these.
  • Server will send Header response in the stream created by the client (for index.html) notifying the client that he is willing to send two more data streams.
  • Server sends content for index.html.
  • Server sends first header and then content in two different streams for style.css and script.js.
  • Client saves the two new (and extra) stream responses in a temporary zone.
  • After rendering index.html, when it will feel the need of style.css and script.js, it will check if anything received by server push is similar to the requirement, and gets the response from there.

How frame wise communication happens

At the protocol level, it’s all about frames (group of bytes), which are part of streams. There are different kinds of frames available in HTTP/2, whereas to understand the server push, we should be good by knowing only four of them. HEADERS, PUSH_PROMISE, RST_STREAM and DATA.
The HEADERS frames carries the http headers and works like a notifier or messenger. Client can send these frames to server which makes the server understand that a request has been made and if sent by the server to the client; it means a response to the previous request (or push) is being sent.
PUSH_PROMISE, as the name suggests, is the most important frame of server push mechanism. It works like a pre-header frame for pushed contents. It is used to notify the client in advance of streams the server intends to initiate (the new streams for style.css and script.js as per our previous example). It is sent along with the header information of the about to be pushed contents. PUSH_PROMISEs are sent before the content of any file (even the initiater, which is index.html as per our example).
There are two big benefits of this approach. First, it will avoid the race condition of the same resource getting requested again by the client (cause if the client don’t have a info on what the server is going to push, it may request for style.css after index.html is rendered and push is not fully finished). Secondly, if the about to be pushed files are already there on the client due to previous cache, the client can refuse to accept the pushed streams (or any other stream) by sending RST_STREAM frame.
After all these settlements, DATA frame comes into the picture to send the actual content of those files.

Server push implementation with node.js

Node is always in active development and one of the fastest growing tools out there. Previously there used to be an npm package spdy which was generally used to implement server push; but since node 8.4.0 onwards started supporting http/2 natively (in experimental mode with --expose-http2 flag), we will do it in the native way. So, you should have node.js 8.4.0 or above installed.

Generate ssl keys

Server push doesn’t mandate a secure server; but majority of the browsers will not support server push unless done from a secured server.
You can refer our node.js ssl server or just run the following command (if you have openssl) in your terminal or command prompt, navigating to your project’s root directory. You will find two new files privateKey.key and certificate.crt has been created.

Create a secure server

First of all scaffold your node project (may be by using npm init) and create a server.js file with the code below to make a secure server.

After completing this step, to check whether the experimental http2 is working or not you should start your node server using the command node --expose-http2 server.js. You should be able to visit https://localhost:3000 now. If you find something like the image below in your browser; click on advanced and then proceed.

node-ssl-localhost

Server push example

You can refer our git repo to get the project structure. It’s very simple. Just have added a public folder containing the files to be served. Rest you already know about the certificate, private key etc. Once you’ve set the same in your end, just modify your server.js like the following to make your server, push the stylesheet and javascript files whenever client requests for index.html.

Now again run your node server just like you did previously with node --expose-http2 server.js and go to localhost’s post 3000 keeping your developer’s tool open. You will find something like the following there.

node-http2-server-push

If you do the exact thing without server push

node-non-http2-push

This image represents what happens when there is no server push and the call for the stylesheet and javascript file was initiated from the client.
You can clearly see the green bars, which are nothing but the waiting time. This waiting time is not there in case of server push (refer previous image). You may not find a huge benefit in overall time to render in this example, cause it’s very small. But in case of a real webpage where there will me lots of resource calling; reducing the waiting time will give you a big performance benefit and as well as a great user experience.

Won’t it unnecessarily push with repeated requests?

A very common question comes in people’s mind is, what will happen when the client again make a request for another html page of the same website (considering the entire website uses same css and js files)?
Will it again push those files? Cause server doesn’t know if it’s a new request or old. So will it hamper the bandwidth?
The answer is NO!

While describing the http/2 push communication, I’ve said that before sending any DATA frame (which has the content of a file) to the client the server sends the PUSH_PROMISE frame. And PUSH_PROMISE also contains headers.

Once the PUSH_PROMISE is received in the clients end; the client checks the headers to determine in the file about to be pushed is already cached to his side. If yes, the client sends a RST_STREAM to confirm the rejection of the to be pushed file/files.
This way, the cached files will not be re-pushed and your bandwidth will not be killed.

What about server side operations?

As I’ve just described that cached files will not be re-pushed; but all these decisions are being handled by the server and client internally with no intervention from the developer. If I consider our demo, if the client again asks for index.html, the server’s code of push() function will run; even though the files will not be actually transferred.
I think this is also a performance hit. For this small example it’s a small push() function, but for someone else there could be more complicated and time consuming operations. So for him, even though the bandwidth problem won’t occur, but the CPU consumption will still increase.
But that’s how it is for now. You can probably drop a cookie in your client to determine whether he is a returning user or new and do your complex operations basing on that. If you have another way to bypass this scenario, kindly add that to comment section.
So keep pushing and stay happy 🙂

About This Author

Hello! I am Paul Shan, a JavaScript Expert, Full Stack and DevOps Engineer cum Consultant based out of Bengaluru, India.

  • Alberto Esposito

    nice post 🙂 line 22 of first example contains a small mistake: the arrow function got lost, instead we got some UTF8 garbage 🙂

    • Paul Shan

      Thanks for notifying. Corrected 🙂

    • Alberto Esposito

      also the example doesn’t work for me. reqPath is always undefined, but if I hardcode it, it works. Still trying to troubleshoot the issue, I’m on latest chrome + ubuntu

    • Paul Shan

      reqPath is nothing but req.path. What is your request object?
      You can check the repo here https://github.com/voidcanvas/http2-server-push