Manage expiration of cached assets with Service Worker caching

Posted on January 05, 2019 in Dev • 4 min read

My Cygnal app uses OSM map tiles to render the map on which the reports are shown. It is meant to be used in realtime, on your mobile phone, mounted on your bike and there can often be some network issues in this setup.

I was looking for a way to cache the map tiles locally, so that if the tile has already been used recently, the mobile phone would not download it once again (then sparing some network bandwidth and displaying the map even with network issues).

The most basic option is to rely on the HTTP headers for cache control. Sadly, these are set by the server and the client side has no easy control on them and the caching strategy. Additionally, the browser may not serve the cached file if the server end is not reachable (in case of network issue).

Another approach is to have a look at Service Workers which have a Caching API. This is the way I decided to use and I will now detail it. I will not cover the basics of Service Workers and Caching API, which are already well covered on the web, but instead describe my particular setup to cache responses and control cache expiration.

The main issue we have to face with the Caching API is that caching is done indefinitely and it is the app’s role to delete expired items from the cache. Additionally, there seems to be no way of accessing the date at which an entry was put in the cache, so we have to find a trick to keep this info.

Here is my fetch handler in my sw.js Service Worker file:

// Name of the cache
const CACHE_NAME = "cache";
// Caching duration of the items, one week here
const CACHING_DURATION = 7 * 24 * 3600;
// Verbose logging or not
const DEBUG = true;

global.self.addEventListener('fetch', (event) => {
    const { request } = event;

    // ...

    event.respondWith(global.caches.open(`${CACHE_NAME}-tiles`).then(
        cache => cache.match(request).then(
            (response) => {
                // If there is a match from the cache
                if (response) {
                    DEBUG && console.log(`SW: serving ${request.url} from cache.`);
                    const expirationDate = Date.parse(response.headers.get('sw-cache-expires'));
                    const now = new Date();
                    // Check it is not already expired and return from the
                    // cache
                    if (expirationDate > now) {
                        return response;
                    }
                }

                // Otherwise, let's fetch it from the network
                DEBUG && console.log(`SW: no match in cache for ${request.url}, using network.`);
                // Note: We HAVE to use fetch(request.url) here to ensure we
                // have a CORS-compliant request. Otherwise, we could get back
                // an opaque response which we cannot inspect
                // (https://developer.mozilla.org/en-US/docs/Web/API/Response/type).
                return fetch(request.url).then((liveResponse) => {
                    // Compute expires date from caching duration
                    const expires = new Date();
                    expires.setSeconds(
                        expires.getSeconds() + CACHING_DURATION,
                    );
                    // Recreate a Response object from scratch to put
                    // it in the cache, with the extra header for
                    // managing cache expiration.
                    const cachedResponseFields = {
                        status: liveResponse.status,
                        statusText: liveResponse.statusText,
                        headers: { 'SW-Cache-Expires': expires.toUTCString() },
                    };
                    liveResponse.headers.forEach((v, k) => {
                        cachedResponseFields.headers[k] = v;
                    });
                    // We will consume body of the live response, so
                    // clone it before to be able to return it
                    // afterwards.
                    const returnedResponse = liveResponse.clone();
                    return liveResponse.blob().then((body) => {
                        DEBUG && console.log(
                            `SW: caching tiles ${request.url} until ${expires.toUTCString()}.`,
                        );
                        // Put the duplicated Response in the cache
                        cache.put(request, new Response(body, cachedResponseFields));
                        // Return the live response from the network
                        return returnedResponse;
                    });
                });
            })
        )
    );
});

The trick here is to recreate a new response with extra HTTP headers (SW-Cache-Expires) to keep trace of the cache expiration date. We must be careful here not to use an HTTP header name which could conflict with a real HTTP header (or the information sent by the server would be lost).

When fetching a new item, we first try to match it with the cache. If a response is already cached, we check its expiration datetime and eventually return it. Only if no matching response (or an expired one) is found in the cache, we fetch from the network.

Note: We could use the same strategy to actually enforce a Cache-Control or Expires HTTP header and let the browser handle all the caching for us. However, with this setup, we would not be able to have full control of the cache strategy and enforce the browser to actually serve the local cached response instead of trying to fetch the online response when there are network issues.

Finally, we have to manage the cache expiration manually. This can be easily done at startup of your app using a message.

global.self.addEventListener('message', (event) => {
    console.log(`SW: received message ${event.data}.`);

    const eventData = JSON.parse(event.data);

    // Clean tiles cache when we receive the message asking to do so
    if (eventData.action === 'PURGE_EXPIRED_TILES') {
        DEBUG && console.log('SW: purging expired tiles from cache.');
        global.caches.open(`${CACHE_NAME}-tiles`).then(
            cache => cache.keys().then(
                keys => keys.forEach(
                    // Loop over all requests stored in the cache and get the
                    // matching cached response.
                    key => cache.match(key).then((cachedResponse) => {
                        // Check expiration and eventually delete the cached
                        // item
                        const expirationDate = Date.parse(cachedResponse.headers.get('sw-cache-expires'));
                        const now = new Date();
                        if (expirationDate < now) {
                            DEBUG && console.log(`SW: purging (expired) tile ${key.url} from cache.`);
                            cache.delete(key);
                        }
                    }),
                ),
            ),
        );
    }
});

This can then be called from your client code at startup. For instance, you can use, in your main Vue.JS component

mounted() {
    // Service worker is for caching only here, so it needs both SW support
    // and caching API support.
    if ('serviceWorker' in navigator && 'caches' in window) {
        navigator.serviceWorker.register('/sw.js').then(
            // Clean expired tiles from the cache at startup
            () => navigator.serviceWorker.controller.postMessage(JSON.stringify({
                action: 'PURGE_EXPIRED_TILES',
            })),
        ).catch((error) => {
            console.log(`Registration failed with ${error}.`);
        });
    }
},

The full code of this Service Worker can be found here.

As a bonus, here is a little snippet based on expired to get the time before expiration of a response, according to HTTP headers:

// Get duration (in s) before (cache) expiration from headers of a fetch
// request.
function getExpiresFromHeaders(headers) {
    // Try to use the Cache-Control header (and max-age)
    if (headers.get('cache-control')) {
        const maxAge = headers.get('cache-control').match(/max-age=(\d+)/);
        return parseInt(maxAge ? maxAge[1] : 0, 10);
    }

    // Otherwise try to get expiration duration from the Expires header
    if (headers.get('expires')) {
        return (
            parseInt(
                (new Date(headers.get('expires'))).getTime() / 1000,
                10,
            )
            - (new Date()).getTime()
        );
    }
    return null;
}