ENESFRITEO
Racket: Running http, server-sent events and websockets on the same port

I have been using racket for a few years now and at some point I discovered the web-server library. I started to play with it and I have been learning some html, css and javascript along with it. This blog is a racket servlet (in fact I created a package for it, you can find a description here and the code here).

At some point I wanted to be able to update a webpage dynamically and I discovered server-sent events (SSE's) and websockets. There was already a websockets package, it is called rfc6455 (written by Tony Garnock-Jones, code here) and there wasn't one for SSE's.

I decided to create an implementation of SSE's for racket. I quickly realized that the serve/servlet function (provided by web-server/servlet-env) I had been using for serving webpages could not be used for my project since it must receive a request in order to give a response. In the first version of the library I built everything on top of a tcp connection (code here). After I finished on the one hand I was happy I had achieved my goal but on the other hand I felt my implementation was unsafe since there are so many security issues I am sure I wasn't considering since I was starting from such a low level (a tcp connection).

Both my SSE implementation and the rfc6455 package run on their own port apart from the http server. But I noticed the websockets implementation from other languages allows you to run http and websockets in the same port (for example the shiny package from R uses websockets to update elements in a webpage and it only requires one port). So I got curious about how that could be done in racket. Someone could ask why? why bother if you can achieve the same results with two ports and there is no shortage of ports in a computer? Let us not get distracted with these useless mundane practicalities.

I did not want to continue building on top of bare tcp connections so I decided to study the code of rfc6455 to see how things were done in there. That is when I discovered dispatchers. I found the documentation of that library difficult to follow, but by going back and forth between the documentation and how it's functions are used in rfc6455's code I started to make some progress. That's when I stumbled upon the excellent blog post The Missing Guide to Racket's Web Server written by Bogdan Popa and then everything was clear. I knew what I had to do.

I rewrote the SSE library using dispatchers (code here) and below I give examples on how to use them to run a http server and websockets or SSE's on the same port. If you are not familiar with dispatchers I recommend you to read first The Missing Guide to Racket's Web Server, it has everything you need to understand what follows.

The idea is the same in all the examples. One first gets a dispatcher for each service and then filters and a sequencer are used to serve different protocols on different urls (instead of different ports).

Http and Server-sent Events

The SSE library provides the function sse-dispatcher which gets the dispatcher used to serve the corresponding SSE. In the following example the message "hello world" is added to a text area after receiving that string from a SSE running on the url http://localhost:8080/sse .
#lang racket/base

(require SSE
	 (prefix-in sequencer: web-server/dispatchers/dispatch-sequencer)
	 (prefix-in filter: web-server/dispatchers/dispatch-filter)
	 web-server/web-server
	 web-server/http/xexpr
	 web-server/servlet-dispatch)


(define (start-page req)
    (response/xexpr
        `(html
          (head
          (meta ([http-equiv "content-type"] [content "text/html; charset=utf-8"] ))
          (meta ([name "viewport"] [content "width=device-width"] ))
          (title "Events test"))

          (body
          (h1 "Welcome to the Events test")
          (textarea ([readonly ""]))
          (script
          "
          var evtSource = new EventSource(\"/sse/\");
          var textArea = document.getElementsByTagName(\"textarea\")[0];

          evtSource.onmessage = function(e){
              textArea.value =  e.data + \"\\n\" + textArea.value ;
              console.log(e.data);
          }")))))


(define a-sse (make-sse))

(serve #:dispatch (sequencer:make
		   (filter:make #rx"^/sse/" (sse-dispatcher a-sse))
		   (dispatch/servlet start-page))
       #:port 8080)

;; Send message
(sse-send-event a-sse #:data "Hello world" #:id #t)

;; Don't close the program
;; Necessary if you run the script in the command line    
(displayln "Press enter to terminate app")
(read-line)

Note that in line 24, a connection is made to the events server in the url /sse, while in line 36 a filter is used to serve the events on that same url. The purpose of (read-line) at the end is to prevent the server from exiting when the code above is run as a script.

Http and Websockets

The websockets package supports two interfaces (documentation here). The legacy interface which uses the ws-serve function and a the new interface that has support for different url's and websockets subprotocols which uses the ws-serve* function.

Legacy Interface Example

A dispatcher for the legacy websockets interface can be created by using the function make-general-websockets-dispatcher provided by net/rfc6455/dispatcher

In the following example there is originally a webpage with an empty text area. Then a websocket connection is started in which the client sends the message "Hello". The server takes that message, appends " world" and sends it back to the client which adds it to the text area.

#lang racket/base

(require (prefix-in sequencer: web-server/dispatchers/dispatch-sequencer)
	 (prefix-in filter: web-server/dispatchers/dispatch-filter)
	 net/rfc6455	
	 net/rfc6455/dispatcher;; provides make-general-websockets-dispatcher
	 web-server/web-server
	 web-server/servlet-dispatch
	 web-server/http/xexpr)

(define (start-page req)
  (response/xexpr
   `(html
     (head
      (meta ([http-equiv "content-type"] [content "text/html; charset=utf-8"] ))
      (meta ([name "viewport"] [content "width=device-width"] ))
      (title "Websocket test"))

     (body
      (h1 "Welcome to the websockets test")
      (textarea ([readonly ""]))
      (script
       "\"use strict\"
      var websocket_url;
      var textArea;
      var socket;

      websocket_url = 'ws://' + window.location.host + \"/ws_test\";
      textArea = document.getElementsByTagName(\"textarea\")[0];

      socket = new  WebSocket(websocket_url);
      socket.onopen = function(){
	  socket.send(\"Hello\");
      }

      socket.addEventListener('message',
			      function (event){
				  var message = event.data;
				  textArea.value = event.data + \"\\n\" + textArea.value;
			      });
        ")))))

(define (a-conn-dispatch c s)
  (define message (ws-recv c))
  (ws-send! c (string-append message " world!\n")))



(serve #:dispatch (sequencer:make
		   (filter:make #rx"^/ws_test" (make-general-websockets-dispatcher a-conn-dispatch ))
		   (dispatch/servlet start-page))
       #:port 8080)

(displayln "Press enter to terminate app")
(read-line)

Modern Interface Example

The modern interface uses the function ws-service-mapper to support url path matching and websocket subprotocols. The package provides the function make-service-mapper-dispatcher (provided by net/rfc6455/service-mapper ) to convert the output of ws-service-mapper into a dispatcher.

The code example below, implements the same example but now using the modern websockets interface.

#lang racket/base

(require (prefix-in sequencer: web-server/dispatchers/dispatch-sequencer)
	 (prefix-in filter: web-server/dispatchers/dispatch-filter)
	 net/rfc6455	 
	 net/rfc6455/service-mapper;; for make-service-mapper-dispatcher
	 web-server/web-server
	 web-server/servlet-dispatch
	 web-server/http/xexpr)
	 

(define (start-page req)
  (response/xexpr
   `(html
     (head
      (meta ([http-equiv "content-type"] [content "text/html; charset=utf-8"] ))
      (meta ([name "viewport"] [content "width=device-width"] ))
      (title "Websocket test"))

     (body
      (h1 "Welcome to the websockets test")
      (textarea ([readonly ""]))
      (script
       "\"use strict\"
      var websocket_url;
      var textArea;
      var socket;

      websocket_url = 'ws://' + window.location.host + \"/ws_test\";
      textArea = document.getElementsByTagName(\"textarea\")[0];

      socket = new  WebSocket(websocket_url);
      socket.onopen = function(){
	  socket.send(\"Hello\");
      }

      socket.addEventListener('message',
			      function (event){
				  var message = event.data;
				  textArea.value = event.data + \"\\n\" + textArea.value;
			      });
        ")))))



(define a-service-mapper
 (ws-service-mapper
  ["^/ws_test"
   [(#f) (lambda (c) (ws-send! c (string-append (ws-recv c) " world!\n")))]
   ]))


(serve #:dispatch (sequencer:make
		   (filter:make #rx"^/ws_test" (make-service-mapper-dispatcher a-service-mapper))
		   (dispatch/servlet start-page))
       #:port 8080)

(displayln "Press enter to terminate app")
(read-line)


In all the examples we have run and http server and either a websocket or an SSE on the same port. But I think it is clear how to generalize to running all the three on the same port, even though I am not sure about the use of it (and I know what you are thinking: "are you the one playing the practicality card now?" but hey, who isn't full of contradictions?).
Reveni al la ĉefa paĝo