I have been using racket for a few years now and at some point I discovered the web-server library. I started to play with it and I have been learning some html, css and javascript along with it. This blog is a racket servlet (in fact I created a package for it, you can find a description here and the code here).
At some point I wanted to be able to update a webpage
dynamically and I discovered server-sent events (SSE's) and
websockets. There was already a websockets package, it is called
rfc6455
(written by Tony
Garnock-Jones, code
here) and there wasn't one for SSE's.
I decided to create an implementation of SSE's for racket. I
quickly realized that
the serve/servlet function
(provided
by web-server/servlet-env) I
had been using for serving webpages could not be used for my
project since it must receive a request in order to give a
response. In the first version of the library I built everything
on top of a tcp connection
(code
here). After I finished on the one hand I was happy I had
achieved my goal but on the other hand I felt my implementation
was unsafe since there are so many security issues I am sure I
wasn't considering since I was starting from such a low level (a
tcp connection).
Both my SSE implementation and the rfc6455 package run on their own port apart from the http server. But I noticed the websockets implementation from other languages allows you to run http and websockets in the same port (for example the shiny package from R uses websockets to update elements in a webpage and it only requires one port). So I got curious about how that could be done in racket. Someone could ask why? why bother if you can achieve the same results with two ports and there is no shortage of ports in a computer? Let us not get distracted with these useless mundane practicalities.
I did not want to continue building on top of bare tcp connections so I decided to study the code of rfc6455 to see how things were done in there. That is when I discovered dispatchers. I found the documentation of that library difficult to follow, but by going back and forth between the documentation and how it's functions are used in rfc6455's code I started to make some progress. That's when I stumbled upon the excellent blog post The Missing Guide to Racket's Web Server written by Bogdan Popa and then everything was clear. I knew what I had to do.
I rewrote the SSE library using dispatchers (code here) and below I give examples on how to use them to run a http server and websockets or SSE's on the same port. If you are not familiar with dispatchers I recommend you to read first The Missing Guide to Racket's Web Server, it has everything you need to understand what follows.
The idea is the same in all the examples. One first gets a dispatcher for each service and then filters and a sequencer are used to serve different protocols on different urls (instead of different ports).
Http and Server-sent Events
The SSE library provides the functionsse-dispatcher
which gets the dispatcher used to serve the corresponding SSE. In
the following example the message "hello world" is added to a text
area after receiving that string from a SSE running on the url
http://localhost:8080/sse
.
#lang racket/base
(require SSE
(prefix-in sequencer: web-server/dispatchers/dispatch-sequencer)
(prefix-in filter: web-server/dispatchers/dispatch-filter)
web-server/web-server
web-server/http/xexpr
web-server/servlet-dispatch)
(define (start-page req)
(response/xexpr
`(html
(head
(meta ([http-equiv "content-type"] [content "text/html; charset=utf-8"] ))
(meta ([name "viewport"] [content "width=device-width"] ))
(title "Events test"))
(body
(h1 "Welcome to the Events test")
(textarea ([readonly ""]))
(script
"
var evtSource = new EventSource(\"/sse/\");
var textArea = document.getElementsByTagName(\"textarea\")[0];
evtSource.onmessage = function(e){
textArea.value = e.data + \"\\n\" + textArea.value ;
console.log(e.data);
}")))))
(define a-sse (make-sse))
(serve #:dispatch (sequencer:make
(filter:make #rx"^/sse/" (sse-dispatcher a-sse))
(dispatch/servlet start-page))
#:port 8080)
;; Send message
(sse-send-event a-sse #:data "Hello world" #:id #t)
;; Don't close the program
;; Necessary if you run the script in the command line
(displayln "Press enter to terminate app")
(read-line)Note that in line 24, a connection is made to the events server
in the url /sse, while in line 36 a filter is used to
serve the events on that same url. The purpose of
(read-line)
at the end is to prevent the server from exiting when
the code above is run as a script.
Http and Websockets
The websockets package supports two interfaces
(documentation
here). The legacy interface which uses
the ws-serve function and a
the new interface that has support for different url's and
websockets subprotocols which uses
the ws-serve* function.
Legacy Interface Example
A dispatcher for the legacy websockets interface can be created
by using the function
make-general-websockets-dispatcher provided
by net/rfc6455/dispatcher
In the following example there is originally a webpage with an empty text area. Then a websocket connection is started in which the client sends the message "Hello". The server takes that message, appends " world" and sends it back to the client which adds it to the text area.
#lang racket/base
(require (prefix-in sequencer: web-server/dispatchers/dispatch-sequencer)
(prefix-in filter: web-server/dispatchers/dispatch-filter)
net/rfc6455
net/rfc6455/dispatcher;; provides make-general-websockets-dispatcher
web-server/web-server
web-server/servlet-dispatch
web-server/http/xexpr)
(define (start-page req)
(response/xexpr
`(html
(head
(meta ([http-equiv "content-type"] [content "text/html; charset=utf-8"] ))
(meta ([name "viewport"] [content "width=device-width"] ))
(title "Websocket test"))
(body
(h1 "Welcome to the websockets test")
(textarea ([readonly ""]))
(script
"\"use strict\"
var websocket_url;
var textArea;
var socket;
websocket_url = 'ws://' + window.location.host + \"/ws_test\";
textArea = document.getElementsByTagName(\"textarea\")[0];
socket = new WebSocket(websocket_url);
socket.onopen = function(){
socket.send(\"Hello\");
}
socket.addEventListener('message',
function (event){
var message = event.data;
textArea.value = event.data + \"\\n\" + textArea.value;
});
")))))
(define (a-conn-dispatch c s)
(define message (ws-recv c))
(ws-send! c (string-append message " world!\n")))
(serve #:dispatch (sequencer:make
(filter:make #rx"^/ws_test" (make-general-websockets-dispatcher a-conn-dispatch ))
(dispatch/servlet start-page))
#:port 8080)
(displayln "Press enter to terminate app")
(read-line)Modern Interface Example
The modern interface uses the
function ws-service-mapper to
support url path matching and websocket subprotocols. The package
provides the
function make-service-mapper-dispatcher
(provided by net/rfc6455/service-mapper )
to convert the output
of ws-service-mapper into a
dispatcher.
The code example below, implements the same example but now using the modern websockets interface.
#lang racket/base
(require (prefix-in sequencer: web-server/dispatchers/dispatch-sequencer)
(prefix-in filter: web-server/dispatchers/dispatch-filter)
net/rfc6455
net/rfc6455/service-mapper;; for make-service-mapper-dispatcher
web-server/web-server
web-server/servlet-dispatch
web-server/http/xexpr)
(define (start-page req)
(response/xexpr
`(html
(head
(meta ([http-equiv "content-type"] [content "text/html; charset=utf-8"] ))
(meta ([name "viewport"] [content "width=device-width"] ))
(title "Websocket test"))
(body
(h1 "Welcome to the websockets test")
(textarea ([readonly ""]))
(script
"\"use strict\"
var websocket_url;
var textArea;
var socket;
websocket_url = 'ws://' + window.location.host + \"/ws_test\";
textArea = document.getElementsByTagName(\"textarea\")[0];
socket = new WebSocket(websocket_url);
socket.onopen = function(){
socket.send(\"Hello\");
}
socket.addEventListener('message',
function (event){
var message = event.data;
textArea.value = event.data + \"\\n\" + textArea.value;
});
")))))
(define a-service-mapper
(ws-service-mapper
["^/ws_test"
[(#f) (lambda (c) (ws-send! c (string-append (ws-recv c) " world!\n")))]
]))
(serve #:dispatch (sequencer:make
(filter:make #rx"^/ws_test" (make-service-mapper-dispatcher a-service-mapper))
(dispatch/servlet start-page))
#:port 8080)
(displayln "Press enter to terminate app")
(read-line)