Blog
Spring Boot: Multiple MongoDB connections with repositories
Typical situation for developers when you need to connect to multiple DB instances and it should be done in efficient way. Not so long time ago I have been faced the problem during migration quite big project from Spring MVC to Spring Boot as the part of that we had to change our DB connection configuration. We have architecture when one set of repositories linked to DB_1 and other set of repositories linked to DB_2 and so on…
So… we got a question… How to configure Spring Boot application to use several Mongo databases with different sets of repositories?
Figure 1: General connection to DB graph
It’s important to mention that we are using *.yml files for storing config properties instead of *.properties files there are no specific reason for that simply YML easer to read for me.
First things first… by default Spring Boot gives for us possibility to configure one DB using pre-defined configuration parameters right out of the box without any extra config but for all other DBs we will need our own configuration.
Note: All default configuration properties could be found in Spring Boot docs called Appendix A https://docs.spring.io/spring-boot/docs/current/reference/html/common-application-properties.html
According to the Appendix A following configuration fields may be used for setting up MongoDB:
|
1 2 3 4 5 6 7 8 |
# MONGODB (MongoProperties) spring.data.mongodb.authentication-database= # Authentication database name. spring.data.mongodb.database=test # Database name. spring.data.mongodb.field-naming-strategy= # Fully qualified name of the FieldNamingStrategy to use. spring.data.mongodb.grid-fs-database= # GridFS database name. spring.data.mongodb.host=localhost # Mongo server host. spring.data.mongodb.password= # Login password of the mongo server. spring.data.mongodb.port=27017 # Mongo server port. spring.data.mongodb.repositories.enabled=true # Enable Mongo repositories. spring.data.mongodb.uri=mongodb://localhost/test # Mongo database URI. When set, host and port are ignored. spring.data.mongodb.username= # Login user of the mongo server. |
From theory to practice
Goal: I will create small app where two DB connections and one set of repositories will be linked to primary DB(core_db) and second set of repositories will do queries to secondary DB(history_db). “core_db” – will store costumer data, “history_db” – will store data about shopping history. And simple one HTML page where result can be displayed.
Step 1: Create empty Spring Boot project. I hope that everyone know how to do it, or am I wrong? =)
Step 2: Open gradle.build file and ensure that you have all dependencies that are listed in my build file.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 |
buildscript { ext { springBootVersion = '1.3.3.RELEASE' } repositories { mavenCentral() } dependencies { classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}") } } apply plugin: 'java' apply plugin: 'eclipse' apply plugin: 'spring-boot' jar { baseName = 'multiple-mongodb-connector' version = '0.0.1-SNAPSHOT' } sourceCompatibility = 1.8 targetCompatibility = 1.8 repositories { mavenCentral() } dependencies { compile('org.springframework.boot:spring-boot-starter-actuator') compile('org.springframework.boot:spring-boot-starter-data-mongodb') compile('org.springframework.boot:spring-boot-devtools') compile('org.springframework.boot:spring-boot-starter-thymeleaf') compile('org.springframework.boot:spring-boot-starter-web') compile (group: 'org.springframework.boot', name: 'spring-boot-starter-jetty') { exclude group: 'org.eclipse.jetty.websocket' } compile('org.projectlombok:lombok:1.16.6') testCompile('org.springframework.boot:spring-boot-starter-test') } eclipse { classpath { containers.remove('org.eclipse.jdt.launching.JRE_CONTAINER') containers 'org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-1.8' } } |
From Code 2 snippet most important three dependencies:
- ‘spring-boot-starter-web’ – will create web application project with end points and all web parts
- ‘spring-boot-starter-thymeleaf’ – need for HTML templating (replacement for old JSP if someone doesn’t know)
- ‘spring-boot-starter-data-mongodb’ – all Mongo elements with Mongo driver
I’m simply more used to Jetty server instead of TomCat that is not principle to use any of them.
Step 3: Where to place configs for DBs such as password, URI or port? Well… due to the fact that we will use two DBs we will need to have to configuration sets. For primary DB make sense to use default configuration fields from “Code 1” but for second DB we will need our custom set of parameters. Go to <your_project> -> src -> main -> resources -> application.yml and place this code:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
spring: data: mongodb: uri: mongodb://localhost/core_db username: password: port: 27017 host: localhost datasource: history: mongodb: uri: mongodb://localhost/history_db username: password: port: 27017 host: localhost |
First set of parameters from spring.data.mongodb fields automatically will be resolved by Spring Boot as the primary DB configuration. And second set datasource.history.mongodb we will use for second DB configuration.
Step 4: Finally! We are ready for Java DB configuration.
My file structure looks like on the Screenshot 1
Screenshot 1: File structure.
Shortly about packages that I used to locate my config files:
- config – place for files responsible for DB connection.
- controller – place for Rest controller and our business logic.
- repo/core – repositories for core(primary) DB and data object.
- repo/history – repositories for history(secondary) DB and data object.
config package
Spring Boot already has in memory MongoProperties @Bean which contains spring.data.mongodb parameters and we don’t need to create it but we need to create same object for secondary DB. File content really small Code 4:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
package com.artgr.mongoconnector.config; import lombok.Data; import org.springframework.boot.autoconfigure.mongo.MongoProperties; import org.springframework.boot.context.properties.ConfigurationProperties; import org.springframework.stereotype.Component; @Data @Component @ConfigurationProperties("datasource.history") public class DataSourceHistoryDBProperties { private MongoProperties mongodb = new MongoProperties(); } |
Here is two main elements in this file. First one are @ConfigurationProperties defines property block from application.yml that should belong to this file. And second element are MongoProperties mongodb = new MongoProperties(); means that content from application.yml file should be written to the mongodb variable with MongoProperties type. If application.yml file does not have parameters specified in @ConfigurationProperties system will use default data that is in or case equal to configuration of primary DB. Also if type of the data does not equal to MongoProperties object you will get errors during start-up.
DB connection done in files:
- CoreDBConfiguration – config for connecting to primary DB.
- HistoryDBConfiguration – config for connecting to secondary DB.
- AbstractRepoConfig – abstraction for first two files.
Files CoreDBConfiguration, HistoryDBConfiguration contains similar logic but most interesting parts are: CoreDBConfiguration(Code 5):
|
1 2 3 4 5 6 7 8 9 10 11 |
… @EnableMongoRepositories(basePackageClasses = CoreDBMarker.class, mongoTemplateRef = "mongoOperations") … @Bean @Primary public MongoProperties primaryDataSource() { return new MongoProperties(); } … |
This file (Code 5) says to Spring Boot establish connection to DB specified in MongoProperties and use repositories from package where CoreDBMarker.class located. When this file executed and new config created all repositories from CoreDBMarker.class package automatically will do queries to primary DB. Same situation with HistoryDBConfiguration.class but it uses HistoryDBMarker.class and properties from the DataSourceHistoryDBProperties.class file. As the result after executing HistoryDBConfiguration.class all repos from HistoryDBMarker.class package will do queries to secondary DB.
Finally lets do queries to our DBs:
To test our repos I will create controller that will return index page with MODEL with all data from two DBs. Java controller looks like that (Code 6):
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 |
package com.artgr.mongoconnector.controller; import com.artgr.mongoconnector.repo.core.PersonRepo; import com.artgr.mongoconnector.repo.history.ShoppingHistoryRepo; import lombok.val; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Controller; import org.springframework.ui.Model; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestParam; /** * Created by artemgryn on 04/05/16. */ @Controller public class IndexController { @Autowired private PersonRepo personRepo; @Autowired private ShoppingHistoryRepo shoppingHistoryRepo; @RequestMapping(value = "") public String getIndexPage(@RequestParam(value="name", required=false) final String name, final Model model) { val person = personRepo.findByFullName(name); // will go to core_db if (person != null) { val history = shoppingHistoryRepo.findAllByPersonId(person.getId()); // will go to history_db model.addAttribute("name", name); model.addAttribute("person", person); model.addAttribute("history", history); } return "index"; } } |
IndexController (Code 6) defines end-point and returns “index.html” page with model. End-point example “http://localhost:8080?name=John” . In this file personeRepo and shoppngHistoryRepo autowired and will do queries to different DBs. Index.html (Code 7)
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"/> <title>Mongo DB test</title> </head> <body class="main"> <p/>For user: <span th:text="${name}" /> <p/>User Db record: <span th:text="${person}" /> <p/><hr/> <p/>User shopping history: <span th:text="${history}" /> </body> </html> |
Add initial/test data to DBs:
In the core_db created one collection called person and added one element (Screenshot 2)
Screenshot 2: Core DB content
In the history_db created one collection called shoppingHistory and added two elements (Screenshot 3)
Screenshot 3: History DB content
Test!
Start application with gradle bootRun and open URL “http://localhost:8080?name=John”. And you will see the result:
Result