Introduction
In the big applications where there are multiple moving parts ( microservices ) there is a problem of configuration management. Microservices must be disposable applications that can be started and killed frequently, so would it just be awesome to hold configuration at one place and allow microservices to obtain the configuration from this single location.
We can easily manage all configurations on one place and all that is required for the microservices is to know what is the profile it is running in, for example PRODUCTION, STAGE …
In this post I will cover the Spring Cloud Remote Config using three config sources
- Native, where configuration is stored on the hard drive
- Git, where configuration is stored in git repository
- Vault, this is my favorite all secrets are stored in vault
Setting the project
To get things started first we need to create two applications:
- Config Server
- Client
Config server serves the configuration to the clients and has REST API that is used by the client apps that want to obtain configurations.
To get things started here is the pom.xml I used to create config server for brevity I removed all unimportant parts of it.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
<properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding> <java.version>1.8</java.version> <spring-cloud.version>Dalston.SR1</spring-cloud.version> </properties> <dependencies> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-config-server</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> </dependencies> <dependencyManagement> <dependencies> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-dependencies</artifactId> <version>${spring-cloud.version}</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement> </project> |
Because this is Spring Boot application we need Application class that is annotated with @EnableConfigServer and @SpringBootApplication.
|
1 2 3 4 5 6 7 |
@SpringBootApplication @EnableConfigServer public class ConfigServerApplication { public static void main(String[] args) { SpringApplication.run(ConfigServerApplication.class, args); } } |
@EnableConfigServer is very powerful annotation as it is all that it takes to have config server running.
If you run ConfigServerApplication you will not get far because you will get error stating:
You need to configure a uri for the git repository
This is because config server default profile is git and we did not specify the config server profiles and configuration yet. So lets create one application.properties file and add next lines of code that will specify port, config name and profiles. For now we will have only native profile and latter we will add git and vault here as well.
|
1 2 3 |
server.port: 8888 spring.config.name=configserver spring.profiles.include=native |
Spring Cloud Config native profile
What is native profile?
Native profile allows that configuration files are located on the hard drive of the server. Not very good for production but is awesome as a starting point.
We are going to have one application that feeds the configuration from config server and it is going to have most awesome name ever configclient. It is important to memorize app name as it will be important in the next paragraph.
We can split configuration into two separate parts for each application:
- Default configuration
- Configuration for specific profile
Default configuration will go in the file configclient.properties or {APPLICATION_NAME}.yml and settings for specific profile ( in our case production) will to go into configclient-production.yml or configclient-{PROFILE}.yml. All the files can be located in location specified by configuration ( you may investigate the class NativeEnvironmentRepository to see all the default locations):
|
1 |
spring.cloud.config.server.native.searchLocations=classpath:/config |
configclient.yml:
|
1 2 3 4 |
rs: pscode: value: This is Default default: Some Value for all profiles |
configclient-production.yml
|
1 2 3 |
rs: pscode: value: This is PRODUCTION |
As you can see here configuration for production and for default overlap in rs.pscode.value .
Now if you start the config server application and hit the URL localhost:8888/configclient/default response is
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
{ "name": "configclient", "profiles": [ "default" ], "label": null, "version": null, "state": null, "propertySources": [ { "name": "classpath:/configclient.yml", "source": { "rs.pscode.value": "This is Default", "rs.pscode.default": "Some Value for all profiles" } } ] } |
and if you hit the localhost:8888/configclient/production
response will be :
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
{ "name": "configclient", "profiles": [ "production" ], "label": null, "version": null, "state": null, "propertySources": [ { "name": "classpath:/configclient-production.yml", "source": { "rs.pscode.value": "This is PRODUCTION" } }, { "name": "classpath:/configclient.yml", "source": { "rs.pscode.value": "This is Default", "rs.pscode.default": "Some Value for all profiles" } } ] } |
Awesome, this was easy. Lets move on to git profile.
Spring Cloud Config git profile
In previous section we enabled native profile and now we will enable git profile, to do this we need to add git to the profile list :
|
1 |
spring.profiles.include=native, git |
Remember you do not need to leave native if you do not want to.
Next we need to specify the git repository url with username and password if required like so.
|
1 2 3 4 |
spring.cloud.config.server.git.uri=https://github.com/savicprvoslav/Spring-Cloud-Config spring.cloud.config.server.git.search-paths=config #spring.cloud.config.server.git.username=myUserName #spring.cloud.config.server.git.password=myPasword |
You will notice that search-paths is used , this is because configuration is not located in the root of the project it is located in config folder.
There are numerous options for git, like multiple repositories and so on … you can check this out in the reference and looking at the implementation of EnvironmentRepository for git MultipleJGitEnvironmentRepository .
You may wonder because there are two profiles native and git who will win ? Well last specified profile will override previously specified.
Spring Cloud Config Vault
What about those configurations that are supposed to be secrets, you know things that only few know? We can not put those in the git repository or store them on the hard drive of the server, well no.
For this problem spring vault comes to rescue, make sure to check the vaults website https://www.vaultproject.io/ and their get started pages.
This part of the post is going to require some docker knowledge, but nothing big. First we are going to create two containers vault and vault-ui , then we will add two secrets in vault and configure our config application to point to vault server.
Step 1: Create vault container.
|
1 |
docker run -d -p 8200:8200 -e 'VAULT_DEV_ROOT_TOKEN_ID=myroot' -e 'VAULT_DEV_LISTEN_ADDRESS=0.0.0.0:8200' vault |
I did specify the root token here, do not do that in production.
Step 2: Enter the vault container
|
1 |
docker exec -i -t vault sh |
Step 3: Authenticate to vault and configure the vault client properly
If you see the logs of the started container you will see something like:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
==> WARNING: Dev mode is enabled! In this mode, Vault is completely in-memory and unsealed. Vault is configured to only have a single unseal key. The root token has already been authenticated with the CLI, so you can immediately begin using the Vault CLI. The only step you need to take is to set the following environment variables: export VAULT_ADDR='http://0.0.0.0:8200' The unseal key and root token are reproduced below in case you want to seal/unseal the Vault or play with authentication. Unseal Key: Km2zFCAF8ADgilGNxnGk7ud1Sz02y+gjo08QWhZ8lgw= Root Token: myroot |
So as it says we need to enter the container using
|
1 |
docker exec -i -t vault sh |
then execute ( observe myroot is used as token specified when container is started)
|
1 2 |
export VAULT_ADDR='http://localhost:8200' vault auth myroot |
Step 3: Save some secrets
|
1 2 |
vault write secret/configclient rs.pscode.value="This is From vault" vault write secret/configclient-production rs.pscode.value="This is From vault production" |
You should have two response Success! messages like this
|
1 2 |
Success! Data written to: secret/configclient Success! Data written to: secret/configclient-production |
so the secret names are composed of secret/{ApplicationName}-{profile}.
Step 4: Configure the config server to talk to Vault
In the application.properties add and append( or just set) profiles with vault
|
1 2 3 |
spring.profiles.include=native, git, vault spring.cloud.config.server.vault.port=8200 spring.cloud.config.server.vault.host=localhost |
Step 5: Get URL localhost:8888/configclient
And it does not work it shows the error :
|
1 2 3 4 5 6 7 8 |
{ "timestamp": 1497892661534, "status": 400, "error": "Bad Request", "exception": "java.lang.IllegalArgumentException", "message": "Missing required header: X-Config-Token", "path": "/configclient" } |
What the hell is this you may ask , well the client must provide with especial header
X-Config-Token with value myroot .
Vault-ui is a really nice application that allows you to manage vault using great UI. Find the page of vault-ui on docker hub and start it ( https://hub.docker.com/r/djenriquez/vault-ui/)
Then open the page and enter the url of the vault server. Here you may have multiple docker configurations, so if you have docker for mac you can :
1: Obtain the ip of the host computer by entering the vault
|
1 |
docker exec -i -t vault-ui sh |
and executing
|
1 |
/sbin/ip route|awk '/default/ { print $3 }' |
Then in the vault-ui enter the IP address printed by the upper command like so:
|
1 |
http:172.17.0.1:8200 |
and enter the token myroot to see this
As you see in the image are the secrets we entered using console.
Spring Cloud Config Client Part
Now when we know how to setup config server we need to use the values using the client application. Here is the ConfigClientApplication main class. It is really simple except it has @RefreshScope annotation.
Configuration is loaded on application startup and if we update any of the configurations in our config server client application will now know. This is where @RefreshScope comes handy, it marks the beans that are refreshed once /refresh REST API is called. Awesome right?
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
@SpringBootApplication @RestController @RefreshScope public class ConfigClientApplication { public static void main(String[] args) { SpringApplication.run(ConfigClientApplication.class, args); } @Value(value = "${rs.pscode.value:Hello default}") private String value; @Value(value = "${rs.pscode.default:All Profiles}") private String valueDefault; @GetMapping(value = "/test") public ResponseEntity<Map<String, String>> test() { Map<String, String> map = new HashMap<>(2); map.put("value", value); map.put("valueDefault", valueDefault); return ResponseEntity.ok(map); } } |
Here is the important part of the pom.xml for the client app
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
<dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-config</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency> </dependencies> <dependencyManagement> <dependencies> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-dependencies</artifactId> <version>Dalston.SR1</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement> |
If you read the spring cloud reference you will notice that there are two contexts application and bootstrap context as a parent of the previous one. This means that configuration related to config server has to be put in bootstrap.properties for the bootstrap phase …
|
1 2 3 4 5 6 7 8 |
spring.application.name=configclient spring.cloud.config.uri=http://localhost:8888 spring.cloud.config.token=myroot management.security.enabled=false #spring.profiles.include=production |
- spring.application.name is the name of the application that maps to config file names
- spring.cloud.config.uri is the url to the config server
- spring.cloud.config.token is the vault token, remember the Missing required header: X-Config-Token
- management.security.enabled , I skipped security
- spring.profiles.include, current profile configuration
Now for the closing words lets call the localhost:8080/test . And here it is one value from vault, one from git.
|
1 2 3 4 |
{ "value": "This is From vault", "valueDefault": "Some Value for all profiles from Git" } |
Please do find the code on my git hub repository here:
https://github.com/savicprvoslav/Spring-Cloud-Config