Spring Cloud remote config tutorial

Introduction

In the big applications where there are multiple moving parts ( microservices ) there is a problem of configuration management. Microservices must be disposable applications that can be started and killed frequently, so would it just be awesome to hold configuration at one place and allow microservices to obtain the configuration from this single location.

We can easily manage all configurations on one place and all that is required for the microservices is to know what is the profile it is running in, for example PRODUCTION, STAGE …

In this post I will cover the Spring Cloud Remote Config using three config sources

  1. Native, where configuration is stored on the hard drive
  2. Git, where configuration is stored in git repository
  3. Vault, this is my favorite all secrets are stored in vault
Setting the project

To get things started first we need to create two applications:

  • Config Server
  • Client

Config server serves the configuration to the clients and has REST API that is used by the client apps that want to obtain configurations.

To get things started here is the pom.xml I used to create config server for brevity I removed all unimportant parts of it.

Because this is Spring Boot application we need Application class that is annotated with @EnableConfigServer and @SpringBootApplication.

@EnableConfigServer is very powerful annotation as it is all that it takes to have config server running.

If you run ConfigServerApplication you will not get far because you will get error stating:

You need to configure a uri for the git repository

This is because config server default profile is git and we did not specify the config server profiles and configuration yet. So lets create one application.properties file and add next lines of code that will specify port, config name and profiles. For now we will have only native profile and latter we will add git and vault here as well.

Spring Cloud Config native profile

What is native profile?

Native profile allows that configuration files are located on the hard drive of the server. Not very good for production but is awesome as a starting point.

We are going to have one application that feeds the configuration from config server and it is going to have most awesome name ever configclient. It is important to memorize app name as it will be important in the next paragraph.

We can split configuration into two separate parts for each application:

  • Default configuration
  • Configuration for specific profile

Default configuration will go in the file configclient.properties  or {APPLICATION_NAME}.yml and settings for specific profile ( in our case production) will to go into configclient-production.yml or configclient-{PROFILE}.yml. All the files can be located in location specified by configuration  ( you may investigate the class NativeEnvironmentRepository to see all the default locations):

configclient.yml:

configclient-production.yml

As you can see here configuration for production and for default overlap in rs.pscode.value .

Now if you start the config server application and hit the URL  localhost:8888/configclient/default response is

and if you hit the  localhost:8888/configclient/production

response will be :

Awesome, this was easy. Lets move on to git profile.

Spring Cloud Config git profile

In previous section we enabled native profile and now we will enable git profile, to do this we need to add git to the profile list :

Remember you do not need to leave native if you do not want to.

Next we need to specify the git repository url with username and password if required like so.

You will notice that search-paths is used , this is because configuration is not located in the root of the project it is located in config folder.

There are numerous options for git, like multiple repositories and so on …  you can check this out in the reference and looking at the implementation of EnvironmentRepository for git  MultipleJGitEnvironmentRepository .

You may wonder because there are two profiles native and git who will win ? Well last specified profile will override previously specified.

Spring Cloud Config Vault

What about those configurations that are supposed to be secrets, you know things that only few know? We can not put those in the git repository or store them on the hard drive of the server, well no.

For this problem spring vault comes to rescue, make sure to check the vaults website https://www.vaultproject.io/ and their get started pages.

This part of the post is going to require some docker knowledge, but nothing big. First we are going to create two containers vault and vault-ui , then we will add two secrets in vault and configure our config application to point to vault server.

Step 1: Create vault container.

I did specify the root token here, do not do that in production.

Step 2: Enter the vault container

Step 3: Authenticate to vault and configure the vault client properly

If you see the logs of the started container you will see something like:

So as it says we need to enter the container using

then execute ( observe myroot is used as token specified when container is started)

Step 3: Save some secrets

You should have two response Success! messages like this

so the secret names are composed of secret/{ApplicationName}-{profile}.

Step 4: Configure the config server to talk to Vault

In the application.properties add and append( or just set) profiles with vault

Step 5: Get URL  localhost:8888/configclient

And it does not work  it shows the error :

What the hell is this you may ask , well the client must provide with especial header

X-Config-Token with value myroot .

 

Vault-ui is a really nice application that allows you to manage vault using great UI. Find the page of vault-ui on docker hub and start it ( https://hub.docker.com/r/djenriquez/vault-ui/)

Then open the page and enter the url of the vault server. Here you may have multiple docker configurations, so if you have docker for mac you can :

1: Obtain the ip of the host computer by entering the vault

and executing

Then in the vault-ui enter the IP address printed by the upper command like so:

and enter the token myroot to see this

As you see in the image are the secrets we entered using console.

Spring Cloud Config Client Part

Now when we know how to setup config server we need to use the values using the client application. Here is the ConfigClientApplication main class. It is really simple except it has @RefreshScope annotation.

Configuration is loaded on application startup and if we update any of the configurations in our config server client application will now know. This is where @RefreshScope comes handy, it marks the beans that are refreshed once /refresh REST API is called. Awesome right?

Here is the important part of the pom.xml for the client app

If you read the spring cloud reference you will notice that there are two contexts application and bootstrap context as a parent of the previous one. This means that configuration related to config server has to be put in bootstrap.properties for the bootstrap phase …

  • spring.application.name is the name of the application that maps to config file names
  • spring.cloud.config.uri is the url to the config server
  • spring.cloud.config.token is the vault token, remember the Missing required header: X-Config-Token
  • management.security.enabled , I skipped security
  • spring.profiles.include, current profile configuration

Now for the closing words lets call the localhost:8080/test . And here it is one value from vault, one from git.

 

Please do find the code on my git hub repository here:

https://github.com/savicprvoslav/Spring-Cloud-Config

 

 

 

 

Swagger Code Gen Retrofit2 library

Generating code using swagger-codegen is an easy way to get your client up and running using different languages and libraries. In this tutorial I will show how to create clients in java using retrofit2 library.

Basics

First you need to download swagger-codegen library to your computer using next command. You might want to recheck the current version in the github swagger-codegen repository here.

Next you want to execute command for generating the code. Here there are several mandatory parameters you need to add:

  • url to the api-docs
  • target language
  • library to be used
  • api and model package
  • output folder

java -jar swagger.jar generate -i http://localhost:8080/v2/api-docs  -l java –library=retrofit2 -DmodelPackage=rs.pscode.start.model,apiPackage=rs.pscode.start.api -o spring-boot-starter-genereated

Retrofit2

Generating retrofit 2 client code can be done using next command

Here we have all necessary parameters like URL to the api-docs, language, library, packages and output folder.

 

 

Spring boot conditionals

Spring boot allows you to create components conditionally,  already seen condition is profile condition. You can just say @Profile(“dev”) and have different implementation of the bean.

What if you want to configure some especial condition, like enable or disable a feature? Here the @Conditional comes to play.

We want to create a service only if feature is configured to be enabled. Enabled status of the feature is specified using especial property in application.properties

To accomplish this we need to do 3 things

  1. Create Condition Implementation
  2. Create Condition Annotation
  3. Annotate the Class

 

Condition implementation is super simple, you just implement Condition interface and return true or false. Here I check if property is set to true.

Custom annotation that is annotated by especial annotation @Conditional with value set to condition implementation

Using FirebaseCondition I annotate my service that depends on this condition

This is it! Now FirebaseService will be wired only when application.property is correctly filled.

Firebase and Spring boot integration

Firebase can be integrated into the backend so that clients that are authenticated with firebase can call backend REST APIs.

To accomplish this few key parts must be added to the backend

  1. Maven Dependencies
  2. Filter
  3. Authentication provider
  4. Token parsing
  5. Registration

GitHub repository is here.

To add firebase dependencies one library must be added

Every request to the backend must have especial header that is used by FirebaseFilter. Filter checks if header named “X-Authorization-Firebase” exists, if does not this means that request should not be processed by this filter. When token is found it is parsed using firebaseService.parseToken(xAuth) and correct Authentication is created FirebaseAuthenticationToken that is put in the securityContextHolder. Job of the filter is here done and authentication process is being taken over by authentication provider.

Authentication provider uses the FirebaseAuthenticationToken to map the token to database user and set its roles.

Firebase backend configuration is done in one Configuration class. Here I have added firebase reference bean and initialization.

Initialization of firebase depends of two properties , configuration path and database url.

This is it. Be sure to checkout the GitHub repository here.

https://github.com/savicprvoslav/Spring-Boot-starter

 

Spring boot starter repository

Intro

Spring boot makes it easy to get started but putting together features like security, jpa, spring-data, user authentication on both mysql and embedded database , roles and profiles can sometimes take your time away. Main goal is to help start creating backend application to be used as REST API for multiple client applications, like android, iOS , angular and other clients.

GitHub

Included spring boot starters

There are several included spring boot starters

  • spring-boot-starter-web
  • spring-boot-starter-data-jpa
  • spring-boot-starter-security
  • spring-boot-starter-actuator
  • spring-boot-devtools

Additional libraries used are

  • mysql-connector-java
  • hsqldb
  • org.modelmapper.modelmapper

Web

As a example one resource is created and named TestResource that demonstrates how to

  • use @RestController
  • inject services
  • map methods to URLs
  • inject configurational properties
  • specify the http status values

Exception Handler annotated with @ControllerAdvice is added to handle exceptions with custom response.

For generating the output for client applications modelmapper is used. It converts for example TestEntity to TestJson. Here is the word or two from official website http://modelmapper.org:

“Why ModelMapper? The goal of ModelMapper is to make object mapping easy, by automatically determining how one object model maps to another, based on conventions, in the same way that a human would – while providing a simple, refactoring-safe API for handling specific use cases.”

Using modelMapper is super easy to use, here is how to convert List to List.

REST api is split into 3 big sections

  • /api/open
  • /api/client
  • /api/admin

Each of the sections is secured so that users with correct roles can use them

JPA

All of the data means nothing if it can not be persisted, for this problem JPA is used with JpaRepositories enabled. This allows you to quickly create DAO layer. Initially only one entitiy is created the TestEntity, it is super simple with just generated ID of the entity.

Spring JpaRepositories are awesome way for creating DAO layer quick and easy, here is how to create JpaRepository for TestEntity that has CRUD abilities.

To utilize the repository all you need to do is inject it

Additionally two more entities are used UserEntity and RoleEntity, so that spring security can be setuped and users are allowed to login.

Transactions are enabled by default so now you can add @Transactional annotations freely

Spring Security

Securing the data is important, for this important task spring security is used. Initially there are 3 roles in the system

  • Admin Role
  • User Role
  • Anonymous Role

REST API is secured so that

  • Administrators can call only /api/admin
  • Users can call only /api/client
  • Anonymous users can call only /api/open

Method security is enabled with @EnableGlobalMethodSecurity(securedEnabled = true), this means that you can protect your service layer by annotating the public methods like this

Users are authenticated against the database, to setup this service UserService is created, here is the relevant part of the setup where user details service is set to the AuthenticationManagerBuilder.

In the dao layer two entities are created as mentioned above UserEntity and RoleEntity

All that remains is to query the database and load the user and prefill the roles if user has some

Spring actuator

Actuator is an interesting part of the app, it allows administrators to use REST api and test if the backend is working and what is the status of it. Here are some examples

To demonstrate how to create custom health indicator there is PomodoroHealthIndicator as a demonstration.

Spring actuator allows you to monitor the application by creating counts. In the TestService every time when Pomodoro is created counter service is called like this

Now when we call /metrics we can see how many pomodors are created.

Devtools

Devtools can help you develop, one feature is currently used directly so that you do not need to stop start the app all the time during development
spring.devtools.restart.enabled=true

Profiles

There are two profiles that are enabled in the system

  • prod
  • dev

For each profile application-{env}.properties file is created. In each file you can add your properties that depend of the profile that is enabled. Enabling profile is done in the applicatin.properties file

application-prod.properties has mysql access specified but application-dev.properties has no mysql access specified, this is a nice way to tell spring to use embedded database for dev environment. Here is the application-prop.properties, this tells spring boot that we want to have datasource that connects to mysql.

Swagger 2

Swagger 2 is used to allow developers to view your REST API is a simple and understandable way.

Maven dependencies:

 

Enabled only in ‘dev’ profile.

Because of swagger especial configuration is added in spring security section and WebSecurityConfigurerAdapter

Swagger UI is accessible on http://localhost:8080/swagger-ui.html

 

GitHub Repository

Spring boot Configuration properties

Spring boot is awesome and it allows you to configure your app using external properties file (application.properties) with no problem at all. Here is the example of the application.property file located in the class path.

Injecting myValue is super easy, it can be done using @Value annotation like this

Same would go for other awesomeConfiguration values and here is the problem. We have a number of items that all belong to the group awesomeConfiguration it would be great if we could load all the values into the spring bean. Configuration properties to the rescue!

ConfigurationProperties

Configuration properties is set using 3 steps

1: Create POJO object where all property values are going to be loaded and specify @ConfigurationProperties with prefix to control what values are used.

2: enable Configuration properties in your spring config class using annotation @EnableConfigurationProperties

3: Inject AwesomeConfiguration

Done.