Scalable Analytics in Plain Java with Keen IO and Spring Boot
Introduction
In this tutorial we are going to create a web application that collects events and shows statistical information on a dashboard. All this will be done in plain Java without the use of JavaScript or HTML and without installation or configuration of any web servers or SQL/NoSQL databases.
To give you an idea of what we are going to implement, take a look at the following screenshot of the finished application:
The application simulates the last step of an online shopping process: the confirmation that the order has been received. As a gesture of gratitude, this hypothetical last step allows the user to select a “gift”. We won’t implement any extra business logic to actually process any orders; we will limit the UI to merely show a confirmation message. However, we are going to use Keen IO to store the selected option and display the population data on the dashboard shown in the following screenshot.
Remember that we will be coding in Java only!
Prerequisites
- Have Java 8 installed. Follow installation guidelines here
- Install Maven and make sure it works.
- Use your favorite IDE
- Obtain a Keen IO account. If you haven't officially used Keen.IO, sign up for free.
We are going to use Spring and Vaadin frameworks as well, but you don’t really need to have any previous experience with these frameworks to follow this tutorial.
Create a Spring Boot project
Spring Framework is a popular enterprise application framework that allows you to create high-performing, easily testable, and reusable code. Although Spring Framework has many modules, we will use it as an Inversion of Control container. One of the easiest ways of creating a web application with Spring Framework is by using Spring Boot and Spring Initializr.
Go to Spring Initializr to generate a project with the configuration shown in the following screenshot:
Make sure you added the Vaadin dependency. Click the Generate Project button and extract the generated zip file. You should retrieve a Maven project that you can import into your favorite IDE:
Create a web UI in plain Java
Time to start coding! Let’s begin by creating the first screen. Create the following Java class next to the DemoApplication class:
1package com.example;
2
3import com.vaadin.annotations.Theme;
4import com.vaadin.server.VaadinRequest;
5import com.vaadin.spring.annotation.SpringUI;
6import com.vaadin.ui.Notification;
7import com.vaadin.ui.UI;
8import com.vaadin.ui.themes.ValoTheme;
9
10@SpringUI // this allows us to inject other Spring beans into instances of this class
11@Theme(ValoTheme.THEME_NAME)
12public class WebUI extends UI {
13
14 // the entry point of the web application
15 @Override
16 protected void init(VaadinRequest request) {
17 Notification.show("It works!");
18 }
19}This is already a web application, and, thanks to Spring Boot, we don’t need to worry about configuring any web server. Just compile the application using your IDE or the command line and run the Spring Boot application:
1mvn clean install
2mvn spring-boot:runMost IDEs allow creating run configurations. You may want to create one to run the Spring Boot Java Application directly in your IDE. Alternatively, you can start the application by running the jar file created in the target directory. In the end, a Spring Boot application is simply a Java standalone application. You can find the standard main method defined in the DemoApplication class.
Once you have executed the Java application, Spring Boot will configure and start a Tomcat server on port 8080, so you can access the application at http://localhost:8080 and see the “It works” notification.
How does it work? The previous class is just a minimal implementation of a web UI using Vaadin. It should extend org.vaadin.ui.UI and thus implement the init method. The init method is the starting point of the web application, similar to the main method in DemoApplication class.
The WebUI class also includes two annotations. @SpringUI allows Spring Framework to create instances of this class for us. We don’t need to instanciate it manually. Instead, it will be automatically discovered and new instances will be created when a new browser or browser tab points to the application’s URL. The @Theme annotation changes the overall look-and-feel of the application to the Valo theme. There are other themes available, such as reindeer, chameleon, and runo, but those are not as modern and attractive as valo.
There are tons UI components you can use to create a web UI in a similar way to how you would implement a desktop application using Swing or JavaFX. For example, you can use the TextField, ComboBox, CheckBox, and Grid classes and arrange them into a VerticalLayout or HorizontalLayout.
Let’s use more UI components to implement UI we want. Change the implementation of the WebUI class to the following:
1package com.example;
2
3import com.vaadin.annotations.Theme;
4import com.vaadin.server.FontAwesome;
5import com.vaadin.server.VaadinRequest;
6import com.vaadin.spring.annotation.SpringUI;
7import com.vaadin.ui.*;
8import com.vaadin.ui.themes.ValoTheme;
9
10@SpringUI // this allows us to inject other Spring beans into instances of this class
11@Theme(ValoTheme.THEME_NAME)
12public class WebUI extends UI {
13
14 // all components are going to be added into this layout
15 private VerticalLayout layout = new VerticalLayout();
16
17 // the entry point of the web application
18 @Override
19 protected void init(VaadinRequest request) {
20 layout.setMargin(true); // add space around the layout
21 layout.setSpacing(true); // add space between the components inside the layout
22 setContent(layout); // set the content of this web UI to be the layout
23
24 Label title = new Label("Your order is ready!"); // a Label is just static text on the page
25 title.addStyleName(ValoTheme.LABEL_H1); // make the text big
26 layout.addComponent(title); // add the label into the layout
27
28 Label instructions = new Label("We want to thank you for your purchase by giving you a gift. Please select one option:");
29 instructions.setWidth("270px"); // adjust the width to get some word-wrapping.
30 layout.addComponent(instructions); // add the label into the layout
31
32 // add some "gift" options
33 addGiftOption("Prepaid card", FontAwesome.CREDIT_CARD, ValoTheme.BUTTON_PRIMARY, 12);
34 addGiftOption("Voucher", FontAwesome.TICKET, ValoTheme.BUTTON_FRIENDLY, 14);
35 addGiftOption("Surprise", FontAwesome.GIFT, ValoTheme.BUTTON_DANGER, 16);
36 }
37
38 // creates a button with the given configuration
39 private void addGiftOption(String name, FontAwesome icon, String styleName, int value) {
40 Button button = new Button("$" + value + " " + name, icon);
41 button.setWidth("270px");
42 button.addStyleName(ValoTheme.BUTTON_LARGE);
43 button.addStyleName(ValoTheme.BUTTON_ICON_ALIGN_TOP);
44 button.addStyleName(styleName);
45 layout.addComponent(button);
46
47 // call the giftSelected method when the button is clicked
48 button.addClickListener(event -> giftSelected(name, value));
49 }
50
51 // this gets called when any of the "gift" buttons are clicked
52 private void giftSelected(String name, int value) {
53 Notification.show("Thank you! Your gift will be sent soon!");
54 }
55}You may have to rebuild the application and run it again if you want to see the changes in action.
Although most of the code is self-explanatory, let’s review some key parts:
setContent(layout): This is a method in theUIclass we are extending. This is how you set the component that is going to be rendered in this UI.title.addStyleName(ValoTheme.LABEL_H1): This simply adds a CSS class to the component. The Valo theme comes with many ready-to-use styles we can use to change the appearance of the components.layout.addComponent(title): This is how we can add components into layouts. In this case,layoutis aVerticalLayoutmeaning all the contained components will be aligned, well, vertically. There are other kinds of layouts, such asHorizontalLayoutandGridLayout.button.addClickListener(event -> giftSelected(name, value)): We can add click listeners to buttons in order to respond to click events on them. In this case, we are using a lambda expression to define theClickListenerimplementation as a call to thegiftSelectedmethod.
Configure Keen IO
Now that we have a web UI implemented. We can start collecting events any time a button is clicked. The first step of course, is configuring Keen IO. Go ahead and add the Keen IO Java API into the dependencies section of the pom.xml file:
1<dependency>
2 <groupId>io.keen</groupId>
3 <artifactId>keen-client-api-java</artifactId>
4 <version>4.0.0</version>
5</dependency>We need to configure Keen IO by using Spring Framework. Create the following Config class:
1package com.example;
2
3import io.keen.client.java.JavaKeenClientBuilder;
4import io.keen.client.java.KeenClient;
5import io.keen.client.java.KeenProject;
6import org.springframework.beans.factory.annotation.Value;
7import org.springframework.context.annotation.Bean;
8import org.springframework.context.annotation.Configuration;
9
10@Configuration
11public class Config {
12
13 @Value("${keen.projectId}")
14 private String projectId;
15
16 @Value("${keen.writeKey}")
17 private String writeKey;
18
19 @Value("${keen.readKey}")
20 private String readKey;
21
22 @Bean
23 public KeenProject keenProject() {
24 return new KeenProject(projectId, writeKey, readKey);
25 }
26
27 @Bean
28 public KeenClient keenClient(KeenProject keenProject) {
29 KeenClient keenClient = new JavaKeenClientBuilder().build();
30 keenClient.setDefaultProject(keenProject());
31 return keenClient;
32 }
33}This class will be detected by Spring Framework, which in turn will create a bean of type KeenProject and a bean of type KeenClient. Beans are created and managed in Spring IoC (Inversion of Control) Containers. We will inject the KeenClient bean later in the WebUI class and use it to send events to Keen IO.
Notice the @Value annotations on the projectId, writeKey, and readKey fields. With this annotation we can specify a property name defined in the application.properties file and Spring Framework will set the corresponding value accordinly. This means we have to add the three properties in the application.properties file as follows:
1keen.projectId=YOUR_PROJECT_ID
2keen.writeKey=YOUR_WRITE_KEY
3keen.readKey=YOUR_READ_KEYLog into your Keen IO account, create a new project, and get your project ID and keys from the Settings --> API Keys:
Send events to Keen IO
We are ready to use the Keen IO API directly in the WebUI class. First, inject the KeenClient bean in the WebUI class as follows:
1package com.example;
2...
3import io.keen.client.java.KeenClient;
4import org.springframework.beans.factory.annotation.Autowired;
5...
6public class WebUI extends UI {
7
8 @Autowired
9 private KeenClient keenClient;
10 ...
11}Notice how we used the @Autowired annotation to tell Spring Framework that it should inject an instance of this bean. Notice that there’s no new operator but we won’t get any null pointer exceptions.
With this in place, we can add the event-sending logic inside the giftSelected method of the WebUI class as follows:
1...
2import java.util.HashMap;
3import java.util.Map;
4...
5public class WebUI extends UI {
6 ...
7 private void giftSelected(String name, int value) {
8 ...
9 // send the event to Keen IO
10
11 Map<String, Object> event = new HashMap<>(); // create a map with the data to send
12 event.put("name", name);
13 event.put("value", value);
14 // send the event asynchronously so the user doesn't have to wait to see the notification
15 keenClient.addEventAsync("gifts", event);
16 }
17}
18...As you can see, we can send key/value pairs to Keen IO using a map. We don't have to define any schema prior to start sending events. Also notice we are using the addEventAsync method which sends the event asynchronously, so that the user doesn't have to wait until this event is sent. You can also use the addEvent method if you need to wait until the event is sent.
In this example application, it makes sense to use the asynchronous version because we are just gathering data for statistical calculations only. Notice how we used the "gifts” string as the name of the collection. Think of this collection as a way of grouping the same kind of events (a user clicked a gift option). The collection name will be used later when we create queries.
Now you should be able to use the application and start collecting events. Try rebuilding the project, starting the application again, and clicking the buttons. Although the application looks the same as before, we are now collecting events.
Create a Dashboard in plain Java
It’s time to analyze and visualize the data we are collecting! One way to quickly see something interesting on the screen is by defining a query directly in https://keen.io. You shouldn’t have any trouble exploring the website and creating some queries and even making a dashboard by yourself! Go ahead and try it if you want.
However, this tutorial will focus on creating a dashboard using the Java programming language.
First off, we need to add the following two dependencies into the pom.xml file:
1<dependency>
2 <groupId>io.keen</groupId>
3 <artifactId>keen-client-api-query</artifactId>
4 <version>4.0.0</version>
5</dependency>
6
7<dependency>
8 <groupId>org.vaadin.keen.charts</groupId>
9 <artifactId>keen-charts</artifactId>
10 <version>1.1</version>
11</dependency>With this two dependencies in place, we can now define Keen IO queries and pass them to KeenCharts objects. But first, we need to add a link in the web application UI to show the dashboard. Modify the WebUI class to include the following:
1...
2public class WebUI extends UI {
3
4 @Autowired
5 private Dashboard dashboard;
6 ...
7 protected void init(VaadinRequest request) {
8 ...
9 // add a Button to visualize the dashboard
10 Button viewDashboard = new Button("View dashboard");
11 viewDashboard.addStyleName(ValoTheme.BUTTON_LINK); // make the button look like a link
12 layout.addComponent(viewDashboard); // add the button to the layout
13
14 viewDashboard.addClickListener(event -> showDashboard()); // call showDashboard() when the button is clicked
15 }
16 ...
17 private void showDashboard() {
18 setContent(dashboard); // sets the content of this UI to be the dashboard instead of the layout
19 }
20}Of course, this won’t compile because we haven’t implemented the Dashboard class. So let’s implement this class as follows:
1package com.example;
2
3import com.vaadin.spring.annotation.SpringComponent;
4import com.vaadin.ui.GridLayout;
5import com.vaadin.ui.HorizontalLayout;
6import com.vaadin.ui.Label;
7import com.vaadin.ui.VerticalLayout;
8import com.vaadin.ui.themes.ValoTheme;
9import org.springframework.context.annotation.Scope;
10
11import javax.annotation.PostConstruct;
12
13@SpringComponent
14@Scope("prototype")
15public class Dashboard extends VerticalLayout {
16
17 @PostConstruct
18 public void init() {
19 setSizeFull();
20 setMargin(true);
21 setSpacing(true);
22
23 Label title = new Label("Dashboard");
24 title.addStyleName(ValoTheme.LABEL_H1);
25 addComponent(title);
26
27 GridLayout chartsLayout = new GridLayout(2, 2);
28 chartsLayout.setSizeFull();
29 chartsLayout.setSpacing(true);
30 addComponent(chartsLayout);
31 setExpandRatio(chartsLayout, 1);
32 }
33}This is an empty dashboard right now. We are just extending a UI component, VerticalLayout and marking the class with the @SpringComponent annotation to allow Spring to manage this class. The @Scope("prototype") tells Spring to create a new instance of this class any time we inject a bean of this type.
Also note how we defined the init method by marking it with the @PostConstruct annotation. Spring will call this method after creating the instance. But let’s not worry too much about definitions and instantiation; you can find dozens of good tutorials and documentation about Spring on the web. Instead, let’s add some Keen charts to the dashboard!
You need to define a query per each chart. How do you do this? Super-easy:
1Query query = new Query.Builder(QueryType.SUM)
2 .withEventCollection("gifts")
3 .withTargetProperty("value")
4 .withTimeframe(new AbsoluteTimeframe("2016-08-01T00:00", "2016-08-31T23:59"))
5 .build();As you can see, there’s a Builder class that allows you to easily create a new query. In this query, we are summing (new Query.Builder(QueryType.SUM)) all the values in the "values" property (withTargetProperty("value")) of the "gifts" collection (withEventCollection("gifts")) that were created between the specified times (withTimeframe(new AbsoluteTimeframe("2016-08-01T00:00", "2016-08-31T23:59"))).
Using your own ID and Read Key, pass this query to a KeenChart object as follows:
1KeenChart chart = new KeenChart("YOUR_PROJECT_ID", "YOUR_READ_KEY", query);This chart instance is a UI component you can add into any layout. Let’s go ahead and put some charts into the dashboard. Modify the Dashboard class as follows:
1package com.example;
2...
3import io.keen.client.java.AbsoluteTimeframe;
4import io.keen.client.java.Query;
5import io.keen.client.java.QueryType;
6import org.springframework.beans.factory.annotation.Value;
7import org.vaadin.keen.charts.KeenChart;
8
9...
10public class Dashboard extends VerticalLayout {
11
12 @Value("${keen.projectId}")
13 private String projectId;
14
15 @Value("${keen.readKey}")
16 private String readKey;
17
18 @PostConstruct
19 public void init() {
20 ...
21 KeenChart total = new KeenChart(projectId, readKey, new Query.Builder(QueryType.SUM)
22 .withEventCollection("gifts")
23 .withTargetProperty("value")
24 .withTimeframe(new AbsoluteTimeframe("2016-08-01T00:00", "2016-08-31T23:59"))
25 .build());
26
27 KeenChart distribution = new KeenChart(projectId, readKey, new Query.Builder(QueryType.SUM)
28 .withEventCollection("gifts")
29 .withTargetProperty("value")
30 .withTimeframe(new AbsoluteTimeframe("2016-08-01T00:00", "2016-08-31T23:59"))
31 .withGroupBy("name")
32 .build());
33
34 KeenChart value = new KeenChart(projectId, readKey, new Query.Builder(QueryType.SUM)
35 .withEventCollection("gifts")
36 .withTargetProperty("value")
37 .withTimeframe(new AbsoluteTimeframe("2016-08-01T00:00", "2016-08-31T23:59"))
38 .withGroupBy("name")
39 .build());
40
41 KeenChart count = new KeenChart(projectId, readKey, new Query.Builder(QueryType.COUNT)
42 .withEventCollection("gifts")
43 .withTimeframe(new AbsoluteTimeframe("2016-08-01T00:00", "2016-08-31T23:59"))
44 .withInterval("daily")
45 .build());
46 ...
47 chartsLayout.addComponents(total, distribution, value, count);
48 }
49}Build, run the application, and click the View dashboard link. You should see some nice-looking charts now!
Summary
Congratulations! You’ve just developed a web application with scalable analytics features using only Java. Thanks to Keen.IO, you should now be able to send millions of events without having scalability issues. And thanks to Vaadin, you can develop the web application in plain Java. And last, but not least, thanks to Spring, you can develop high-performing, easily testable, reusable code.
You can find the full source code of the application on GitHub.
If you are planning to develop a dashboard application with the technologies described here, you may want to take a look at the following websites:
- Keen IO APIs: https://keen.io/docs
- Vaadin documentation: https://vaadin.com/docs
- Spring documentation: https://spring.io/docs