Polymer+Node+K8s = All Kinds of Awesome
Building and deploying a modern, scalable PWA (Part 1)
In this series of posts, we’re going to be looking at a collection of technologies and techniques that can be deployed to create Progressive Web Application, containerise it, and serve it from a Kubernetes cluster. This allows us to provide a great experience to our users, whilst having a simple to maintain, and highly scalable back end for our DevOps team to look after.
We’re going to be creating a number of moving parts, but they’re all quite small and simple to organise, so lets start by taking a look at the collection of technologies employed in this stack.
Polymer
Polymer is a front-end project created by a team at Google. It’s main aim is to simplify the creation of Web Components, which is where it’s most commonly deployed. It can, however, be used to great effect to create single-page applications and PWA’s, so whilst it’s often overlooked in favour of larger, better known frameworks like Angular, React and Vue.js, it could be a great choice for your next project and is well worth spending some time getting to know. You can learn more about the Polymer Project at https://www.polymer-project.org/
Node
Node is the platform that took JavaScript from being just a front-end technology to being a solid choice for the back-end of system too. It allows the creation of highly efficient and responsive services which can be run almost anywhere. It’s also extremely easy to containerise Node services using Docker, which combined with the non-typed nature of JavaScript makes it a good choice when working in a micro-services style architecture. In our stack we’ll be using Node and several open source Node packages to create a lightweight, efficient service to serve up our PWA. You can find out more about Node, find the official distributions and read all of the documentation at https://nodejs.org/en/.
Docker
Docker, probably the most popular containerisation technology today, has been around for a number of years. However, the rise in popularity of Continuous Delivery/Deployment, the DevOps culture etc. has seen widespread adoption of containerisation and an explosion of interest in Docker. Using Docker we can both package up our own software into simple to run containers, and also leverage the wide array of containers available for running other software, such as database engines, web servers etc. In this series we’ll be specifically using Docker to host our web server, and using a clever Docker pattern along the way to ensure we have the most optimised containers we can get. Given the popularity of Docker in modern architectures, there’s a wealth of information available; you can find the official site at https://www.docker.com/, and checkout official container images for a variety of software on the Docker Hub at https://hub.docker.com/.
Kubernetes
Once the use of containers for running software became w widely used pattern, it became obvious that some way of wrangling complex systems composed of sometimes hundred or thousands of containers was needed; they became to big and intricate to be managed manually.
This lead to a number of projects for container orchestrators, including Apache Masos and Swarm amongst others. In 2014 Google open-sourced their Borg orchestrator as the first release of Kubernetes, and in the ensuing years this has become probably the pre-eminent orchestrator in production use. We’ll be building a Kubernetes cluster in GCP, the Google Cloud Platform, to run our containers in. We’ll take a look at how it monitors the health of services, scales services, and how we can get internet traffic in and out of the cluster to serve public websites. You can find out a lot more about what Kubernetes can do on the official site, at https://kubernetes.io/.
Tooling
Obviously we’re going to need to get the right tooling in place to build all of this. Actually not much is needed, it’s all open source so freely available (with the exception of a GCP account, but you get $300 of credit with a new account so that’s all good), and simple to install. Here’s what you’ll need:
Node and NPM — start by installing the latest LTS version from the NodeJS website.
The Polymer-CLI — The Polymer team produce a handy command line toolkit for creating and working with Polymer projects, which can be globally installed using NPM.
Docker — or the Docker Toolkit if you’re running on Windows. Docker is designed to run on, and produce Linux containers. It can be installed natively on Linux systems, on Windows the Docker Toolkit automatically spins up a Linux VM with everything you need to run Docker.
A good code editor — Personal choice is key here, I’m a fan of VS Code, but Atom, Sublime or Notepad++ will do the job. Anything that can edit text files!
MiniKube — Optional. A good choice for running a small, single node K8s cluster on your dev box, for testing before moving into the cloud.
There’s nothing to trick in installing any of these, and there’s plenty of guides and blogs out there, so I won’t reinvent the wheel here.
The Polymer Project
At it simplest, the Polymer Project (or just Polymer) is a JavaScript and HTML library designed to aid in the creation of web components. But I think that sells it a bit short, and doesn’t do justice to how it’s goals are fundamentally quite different to other front end frameworks such as React or Vue.
“Use the Platform.” — Polymer Team
Polymer’s goal is encapsulated right there, in that one quote. The platform in this case is the browser, and its ever evolving and growing ecosystem of features and API’s. Whist other front end frameworks seem to grow and grow, Polymer takes a radically different, and almost diametrically opposite path. By continually leveraging the latest relevant changes in browser, as well as working to shape future browser features; rather than growing, Polymer is instead shrinking, and has the aim of eventually disappearing altogether. It’s aim is eventually self-obsolescence, as the browser (platform) winds up being able to do everything the library does. Wow, that really is a very different out look to the framework view of the world!
Polymer was first release by Google in early 2015, I started playing with it later that year, and we released our first web components in early 2016. We used it to create some ‘Lego brick’ style GUI components that wrapped a complex API we’d created, so that they could be easily leveraged by customers with very limited development skills and resource. From that perspective Polymer worked brilliantly, delivered just what we needed, as was well liked by our development team.
Later that year I started playing the the ‘app-’ element family the that Polymer Team had released, and realised that the same framework we’d initially assumed was only really for building simple front end widgets could, in fact, scale and extend to creating full blown Single Page Applications, and was a great platform for getting into the new world of the PWA.
Progressive Web Apps
Widespread adoption of smart devices — from smart phones to tablets, introduced new ways for users to interact with applications. Early attempts at using browser-based apps on such devices proved largely fruitless, as users preferred the more fluid interfaces, and richer features that a native app could offer. What users liked less was the friction involved in installing, and managing apps, over the simplicity of browsing to a URL.
In 2015, developers at Google released work that showed how the features offered by modern browsers, such as API’s to access sensors, Bluetooth and cameras, as well as the ability to work off line, could be used to give a ‘native app like’ experience to a browser based application. They coined the term Progressive Web App, and put forward a set of requirements for what they felt was needed to fulfil the role.
- Progressive — Work for every user, regardless of browser choice because they’re built with progressive enhancements as a core tenet.
- Responsive — Fit any form factor: desktop, mobile, tablet, or forms yet to emerge.
- Connectivity independent — Service workers allow work offline, or on low quality networks.
- App-like — Feel like an app to the user with app-style interactions and navigation.
- Fresh — Always up-to-date thanks to the service worker update process.
- Safe — Served via HTTPS to prevent snooping and ensure content hasn’t been tampered with.
- Discoverable — Are identifiable as “applications” thanks to W3C manifests and service worker registration scope allowing search engines to find them.
- Re-engageable — Make re-engagement easy through features like push notifications.
- Installable — Allow users to “keep” apps they find most useful on their home screen without the friction of an app store.
- Linkable — Easily shared via a URL, and do not require complex installation
It’s taken a while, but as of 2018 PWA’s are fully supported on the latest versions of all of the common browsers, and on many older versions through the use of polyfils.
Creating your first PWA
So now we know what we’re building, or at least starting to build, and the library we’re going to use to build it. The great news is that the folks on the Polymer Team at Google have made getting started really easy for us with the polymer-cli, which includes templates for both simple web components, but also a basic PWA which can be used as a starting point, and also a full demo web store to show what the library can do.
Assuming you have the polymer-cli installed, lets build a basic PWA, see it in action, and then take a high level tour of the various bits. In a shell window, create a directory for your PWA source, and in that directory run the shell command
polymer init
After selecting the polymer-2-application template you’ll see the tool do a few things; first it creates the application from it’s template, and then (assuming you have Bower installed) it runs a bower update to ensure all of the dependencies are installed. And that’s it, you now have a simple PWA ready for you to start customising. Lets see what it looks like.
The Polymer cli includes a development server that you can use to quickly spin up your Polymer projects, so let’s do that now. In the root of your new project, issue the command
polymer serve -o
At this point your browser should kick into life, and you’ll see a very simple single page app, with a title bar, a couple of views, and sidebar navigation. You’ll notice it’s also fully responsive out of the box, and based on Google’s popular Material Design language. It’s pretty basic, doesn’t do much, but is all set for you to turn into the next killer app. But, how much of a PWA is it?
Google Chrome offers a PWA audit facility, powered by Lighthouse. It’s very simple to use; open the website in Chrome, open the developer tools, and on the ‘audits’ tab run an audit. Lighthouse will audit a number of elements of the site and it’s performance, including how ‘PWA’ like it is. What you should spot is that right now, our PWA has a lot of failing audits — which is odd as it’s straight out of the box. So lets see why, and what we can do about it.
The Polymer Build
Let’s start by taking a tour of what the CLI has provided us. First thing to notice is the bower_components directory. If you’ve not used Bower before, it’s a front end package management tool, and this directory is where it stores all of the packages that our PWA needs (Polymer, various UI elements etc.). Generally you’ll use Bower to install packages you need, and reference them from here; you shouldn’t go in and modify the files in here, as a Bower re-install would overwrite your changes. Bower is still maintained, but tools like Yarn and WebPack are starting to take over, that said a great many front end projects still support Bower, so it’s worth taking some time to get to know. You can find out a whole lot more at https://bower.io/.
The src directory contains more of interest to us. First, and most obviously there are three view files, each representing each of the pages of the app that we saw when we ran it. Each ‘view’ is actually a Web Component, created with Polymer, and containing additional Web Components to render and control the UI. This is a key concept when working with Polymer, and Web Components; it gives us the ability to create Lego brick like elements that can be endlessly re-used and combined to create a rich user experience.
The src directory also contains an app element, which acts as a shell for our application. This is another Polymer element, and controls rendering of toolbars, navigation, contains the views, and ensures the correct view is shown for the state of the application.
In the root of the project we find a number of files, most of which can be ignored for the sake of this exercise (for example, linting files, bower files, git ignore files etc.). The root contains the index.html file for the app, which loads up and renders the app-shell we mentioned above, to bootstrap the entire application, but you’ll notice does little else, the majority of the app is contained in the app-shell we looked at above.
Of particular interest at this point are two additional files. First is the service-worker.js file which implements the JavaScript service worker, which we use to allow the app to run offline, and when the browser is closed. If you take a look at this right now, you’ll see it just contains a boilerplate message that a service worker will be created when the project is built. This might be surprising, but makes sense as having a service worker running during development and debugging is troublesome, so Polymer helps us out as we’ll see in a minute.
The second file of interest is the polymer.json file. This is critical when we build the application, as it controls the Polymer build process, determines what builds are produced (more on that in a second), and ensures the correct files are included in the build. As you work on your app, and add/rename views for example, you’ll need to ensure that they are referenced from this file so that the build includes them — otherwise you’ll be getting a lot of 404’s in production!
Now we’ve had a quick look around, lets build our app; it might seem a bit weird at first to be building HTML and JavaScript, but it’ll all make sense. From the root of the project issue the command
polymer build
Then sit back, and await the end result.
What’s happening while your waiting is that a build tool set behind the scenes is taking all of the sites files, and using the polymer.json file building the production versions of the app ready to be served.
Wait, did that say versions? Well, yes. If you look in the builds directory that’s been created, you’ll see three sub-directories, each containing a version of the app. What the Polymer build is doing is taking a vanilla PWA, creating the service worker for it, and then packaging up the app in a variety of way to allow it to optimally serve different browser technologies.
For older browsers we have a es5-bundled version, using older JavaScript constructs, polyfills, and bundled/minified code. For newer browsers we have an es6-bundled build, featuring less polyfils and newer JavaScript syntax but the same bundling and minification. For the king-of-the-hills browsers we have an es6-unbundled version using the latest and greatest JavaScript features, and no bundling of the files; instead it’s set up to use HTTP push to optimise delivery of the app.
So that’s awesome, we write code once, and then have a tool chain build it optimised ready to deliver to our users in the most optimum way for the device and browser they chose to use. The only downside is that in order to serve the correct version, we need to understand the capabilities they have, which adds complexity to our server solution. But that’s fine too, the Polymer team have thought of that and have a solution that’s perfect for the job as we’ll see in Part 3 of the series.
For those who want more control over their builds, don’t worry either. Polymer makes sure that you have access to all of the tooling they use through Gulp, so if you need to customise the build process, or you’re looking to integrate it with an existing build process you’re still covered.
That’s it for Part 1; in this article we’ve explored Polymer and PWA’s, learned how to use the polymer-cli to create a simple PWA that we can build on, and learned a bit about the tool set that the CLI gives us for debugging and building our code. In Part 2 we’ll take a look at NodeJS, and see how we can use it to be build a scalable web server to handle serving our PWA to clients.
Byteconf React is a free React/JavaScript conference, streamed on Twitch. Join us on August 31, 2018 to hear from the best React developers and teachers from around the world. Join our mailing list and follow us on Twitter to stay up to date!
