Integrate Spring Boot With Amazon DynamoDB
In this post, let’s see how to integrate Spring Boot with Amazon DynamoDB. I have been exploring the option of creating the REST API using Spring Boot backed by DynamoDB. I will share the information that I have learned during the development process with a simple project named Tasklet API.
Let’s build our project on top of the example we have used in the spring boot starter post. The complete project we are going to develop in this post can be downloaded from here or from the downloads section available at the end of this post.
Prerequisite tools
-
- Java 1.8 or higher
- Eclipse IDE
- Maven
DynamoDB library for Java
To make the integration between Spring based applications and DynamoDB easier, we can make use of the community module Spring Data DynamoDB. I assume that you have imported the project used in the spring boot starter post.
Add the below dependency for spring-data-dynamodb
in pom.xml
1 2 3 4 5 |
<dependency> <groupId>com.github.derjust</groupId> <artifactId>spring-data-dynamodb</artifactId> <version>5.0.2</version> </dependency> |
Using the Tasklet API, users should be able to add, read, edit and delete the Tasks.
As next step we need to create the data model class which represent the Task itself.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 |
package com.rayfocus.api.tasklet.model; import java.util.List; import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBAttribute; import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBHashKey; import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTable; import com.fasterxml.jackson.annotation.JsonCreator; import com.fasterxml.jackson.annotation.JsonProperty; @DynamoDBTable(tableName = "Tasks") public class Task { private String taskId; private String taskName; private String taskStatus; private List<SubTask> taskContent; private String userId; private String createdOn; @JsonCreator public Task(@JsonProperty("task_id") String taskId, @JsonProperty("task_name") String taskName, @JsonProperty("task_status") String taskStatus, @JsonProperty("task_content") List<SubTask> taskContent, @JsonProperty("user_id") String userId, @JsonProperty("created_on") String createdOn) { this.taskId = taskId; this.taskName = taskName; this.taskStatus = taskStatus; this.taskContent = taskContent; this.userId = userId; this.createdOn = createdOn; } public Task() { // default constructor } /** Getters */ @DynamoDBHashKey(attributeName = "task_id") public String getTaskId() { return taskId; } @DynamoDBAttribute(attributeName = "task_name") public String getTaskName() { return taskName; } @DynamoDBAttribute(attributeName = "task_status") public String getTaskStatus() { return taskStatus; } @DynamoDBAttribute(attributeName = "task_content") public List<SubTask> getTaskContent() { return taskContent; } @DynamoDBAttribute(attributeName = "user_id") public String getUserId() { return userId; } @DynamoDBAttribute(attributeName = "created_on") public String getCreatedOn() { return createdOn; } /** Setters */ public void setTaskId(String taskId) { this.taskId = taskId; } public void setTaskName(String taskName) { this.taskName = taskName; } public void setTaskStatus(String taskStatus) { this.taskStatus = taskStatus; } public void setTaskContent(List<SubTask> taskContent) { this.taskContent = taskContent; } public void setUserId(String userId) { this.userId = userId; } public void setCreatedOn(String createdOn) { this.createdOn = createdOn; } @Override public String toString() { return "Task [taskId=" + taskId + ", taskName=" + taskName + ", taskStatus=" + taskStatus + ", taskContent=" + taskContent + ", userId=" + userId + ", createdOn=" + createdOn + "]"; } } |
We have to represent the sub tasks as well for which another Model class SubTask
to be defined.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 |
package com.rayfocus.api.tasklet.model; import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBAttribute; import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBDocument; import com.fasterxml.jackson.annotation.JsonCreator; import com.fasterxml.jackson.annotation.JsonProperty; @DynamoDBDocument public class SubTask { private String taskItem; private String status; @JsonCreator public SubTask(@JsonProperty("task_item") String taskItem, @JsonProperty("status") String status) { this.taskItem = taskItem; this.status = status; } public SubTask() { // default constructor } /** Getters */ @DynamoDBAttribute(attributeName = "task_item") public String getTaskItem() { return taskItem; } @DynamoDBAttribute(attributeName = "status") public String getStatus() { return status; } /** Setters */ public void setTaskItem(String taskItem) { this.taskItem = taskItem; } public void setStatus(String status) { this.status = status; } } |
DynamoDB Config
To access the DynamoDB from Spring Boot application, we need to create and register Spring beans for DynamoDB client.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
package com.rayfocus.api.tasklet.db.config; import org.socialsignin.spring.data.dynamodb.repository.config.EnableDynamoDBRepositories; import org.springframework.beans.factory.annotation.Value; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import com.amazonaws.auth.AWSCredentials; import com.amazonaws.auth.BasicAWSCredentials; import com.amazonaws.services.dynamodbv2.AmazonDynamoDB; import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient; @Configuration @EnableDynamoDBRepositories(basePackages = "com.rayfocus.api.tasklet.db.repository") public class DynamoDBConfig { @Value("${amazon.aws.accesskey}") private String awsAccessKey; @Value("${amazon.aws.secretkey}") private String awsSecretKey; @Bean public AmazonDynamoDB amazonDynamoDB(AWSCredentials awsCredentials) { @SuppressWarnings("deprecation") AmazonDynamoDB amazonDynamoDB = new AmazonDynamoDBClient(awsCredentials); return amazonDynamoDB; } @Bean public AWSCredentials awsCredentials() { return new BasicAWSCredentials(awsAccessKey, awsSecretKey); } } |
To complete the configuration, add the entries for AWS access key and secret key in the application.properties
.
1 2 |
amazon.aws.accesskey=[AWS_ACCESS_KEY] amazon.aws.secretkey=[AWS_SECRET_KEY] |
Note: When deploying the application into Amazon EC2 instance or Amazon ECS, we can make use of the IAM roles to access the other AWS services like DynamoDB or S3, instead of storing credentials in properties file. At this point of time, to develop this project let’s make use of credentials.
IAM Policy for accessing DynamoDB
For this project, I have created a IAM user named TaskletAPI and attached the below custom IAM policy. I have restricted the IAM policy to have access only for DynamoDB operations on a specific table named Tasks
.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
{ "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": [ "dynamodb:BatchGetItem", "dynamodb:BatchWriteItem", "dynamodb:PutItem", "dynamodb:DeleteItem", "dynamodb:Scan", "dynamodb:Query", "dynamodb:UpdateGlobalTable", "dynamodb:UpdateItem", "dynamodb:CreateTable", "dynamodb:DescribeTable", "dynamodb:DescribeGlobalTable", "dynamodb:GetItem", "dynamodb:CreateGlobalTable", "dynamodb:UpdateTable" ], "Resource": "arn:aws:dynamodb:us-east-1:<aws_account_id>:table/Tasks" }, { "Sid": "VisualEditor1", "Effect": "Allow", "Action": "dynamodb:ListTables", "Resource": "*" } ] } |
Repository interface for CRUD operations
Since we have spring-data-dynamodb
in our classpath, we can make use of CRUD repository for our database operations. With @EnableScan
annotation, the repository implementation will be registered as a component in the Spring container.
1 2 3 4 5 6 7 8 9 10 11 12 13 |
package com.rayfocus.api.tasklet.db.repository; import java.util.List; import org.socialsignin.spring.data.dynamodb.repository.EnableScan; import org.springframework.data.repository.CrudRepository; import com.rayfocus.api.tasklet.model.Task; @EnableScan public interface TaskRepository extends CrudRepository<Task, String> { List<Task> findByUserId(String userId); } |
REST controller
Okay. Now we are all set to write our controller class for handling the incoming requests. Let’s write handler methods for writing/fetching user task to/from DynamoDB. Our controller class looks as shown in below listing.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
package com.rayfocus.api.tasklet.controller; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.http.ResponseEntity; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.PathVariable; import org.springframework.web.bind.annotation.PostMapping; import org.springframework.web.bind.annotation.RequestBody; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestParam; import org.springframework.web.bind.annotation.RestController; import com.rayfocus.api.tasklet.http.HttpResponse; import com.rayfocus.api.tasklet.model.Task; import com.rayfocus.api.tasklet.service.TaskletService; @RestController @RequestMapping("/tasklet") public class TaskletController { @Autowired TaskletService taskletService; @PostMapping("/task") public ResponseEntity<HttpResponse> saveTask(@RequestBody Task newTask, @RequestParam(value = "version", required = false) String apiVersion) { return taskletService.saveTask(newTask); } @GetMapping("/task/{userid}") public ResponseEntity<HttpResponse> getTasksByUserId(@PathVariable("userid") String userId, @RequestParam(value = "version", required = false) String apiVersion) { return taskletService.getTaskByUserId(userId); } } |
@PostMapping
annotation indicates that the HTTP POST request with URI http://<domain>:<port>/tasklet/task
will be handled and the data will be saved to database. The call is then delegated to service class from the controller.
Similarly, @GetMapping
annotation on method getTaskByUserId
indicates that HTTP GET request with URI http://<domain>:<port>/tasklet/task/{userid}
will be handled and the data will be returned after fetching from database.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 |
package com.rayfocus.api.tasklet.service; import java.util.List; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.http.HttpStatus; import org.springframework.http.ResponseEntity; import org.springframework.stereotype.Component; import com.rayfocus.api.tasklet.db.repository.TaskRepository; import com.rayfocus.api.tasklet.http.HttpResponse; import com.rayfocus.api.tasklet.model.Task; import com.rayfocus.api.tasklet.model.TaskMetaData; @Component public class TaskletService { @Autowired TaskRepository taskRepository; public ResponseEntity<HttpResponse> saveTask(Task newTask) { Task savedTask = taskRepository.save(newTask); TaskMetaData taskMetaData = null; if (savedTask == null) { taskMetaData = new TaskMetaData(newTask.getTaskId(), newTask.getTaskName()); return new ResponseEntity<HttpResponse>( new HttpResponse("500", "ERROR", "Task creation failed", taskMetaData), HttpStatus.INTERNAL_SERVER_ERROR); } taskMetaData = new TaskMetaData(savedTask.getTaskId(), savedTask.getTaskName()); return new ResponseEntity<HttpResponse>( new HttpResponse("200", "SUCCESS", "Task created successfully", taskMetaData), HttpStatus.OK); } public ResponseEntity<HttpResponse> getTaskByUserId(String userId){ List<Task> tasks = taskRepository.findByUserId(userId); if (tasks == null) { return new ResponseEntity<HttpResponse>( new HttpResponse("500", "ERROR", "Error fetching task details", tasks), HttpStatus.INTERNAL_SERVER_ERROR); } return new ResponseEntity<HttpResponse>(new HttpResponse("200", "SUCCESS", "Task Details", tasks), HttpStatus.OK); } } |
HttpResponse and TaskMetaData are helper classes to form the ResponseEntity.
HttpResponse.java
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
package com.rayfocus.api.tasklet.http; public class HttpResponse { private final String statusCode; private final String statusMessage; private final String responseDesc; private final Object responseData; public HttpResponse(String statusCode, String statusMessage, String responseDesc, Object responseData) { this.statusCode = statusCode; this.statusMessage = statusMessage; this.responseDesc = responseDesc; this.responseData = responseData; } public String getStatusCode() { return statusCode; } public String getStatusMessage() { return statusMessage; } public String getResponseDesc() { return responseDesc; } public Object getResponseData() { return responseData; } } |
TaskMetaData.java
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
package com.rayfocus.api.tasklet.model; public class TaskMetaData { private final String taskId; private final String taskName; public TaskMetaData(String taskId, String taskName) { this.taskId = taskId; this.taskName = taskName; } public String getTaskId() { return taskId; } public String getTaskName() { return taskName; } } |
Managing API versions
If we need to manage the API versions going further, we can do that using Query params in the request URI. For example, if we have this application running on port 8080, then issue the request as below:
with query param
http://localhost:8080/tasklet/task?version=2
In this case, we can route to the appropriate service method for processing the request.
without query param
http://localhost:8080/tasklet/task
If we define the query param as optional, then by default we can route to the latest api version service method if the version
param is not present in request.
Unit testing
We have completed developing our Tasklet
service. Let’s write some unit tests to ensure that the application is working as expected.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 |
package com.rayfocus.api.tasklet; import static org.junit.Assert.assertTrue; import java.util.ArrayList; import java.util.Arrays; import java.util.List; import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.http.ResponseEntity; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; import com.amazonaws.services.dynamodbv2.AmazonDynamoDB; import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapper; import com.amazonaws.services.dynamodbv2.model.ProvisionedThroughput; import com.amazonaws.services.dynamodbv2.util.TableUtils; import com.amazonaws.services.dynamodbv2.util.TableUtils.TableNeverTransitionedToStateException; import com.rayfocus.api.tasklet.http.HttpResponse; import com.rayfocus.api.tasklet.model.SubTask; import com.rayfocus.api.tasklet.model.Task; import com.rayfocus.api.tasklet.model.TaskMetaData; import com.rayfocus.api.tasklet.service.TaskletService; @RunWith(SpringJUnit4ClassRunner.class) @SpringBootTest public final class TaskletServiceTests { private final Logger logger = LoggerFactory.getLogger(getClass()); @Autowired private AmazonDynamoDB amazonDynamoDB; @Autowired private TaskletService taskletService; private static boolean taskTableExist = false; @Before public void setUp() throws TableNeverTransitionedToStateException, InterruptedException { if( ! taskTableExist ) { if(TableUtils.createTableIfNotExists(amazonDynamoDB, new DynamoDBMapper(amazonDynamoDB).generateCreateTableRequest(Task.class) .withProvisionedThroughput(new ProvisionedThroughput(2L, 2L)))) { logger.info("~~~ Waiting for Tasks table to be active ~~~"); TableUtils.waitUntilActive(amazonDynamoDB, "Tasks"); } else { logger.info("~~~ Tasks table already exists ~~~"); } taskTableExist = true; } } @Test public void testSaveTask() { logger.info("Saving the taskable items"); List<SubTask> subTasks = new ArrayList<SubTask>( Arrays.asList(new SubTask("Vegetables", "INCOMPLETE"), new SubTask("Fruits", "COMPLETE"))); Task newTask = new Task("456", "Buy Groceries", "Active", subTasks, "users@taskletAPI.com", "11-JUN-2018"); ResponseEntity<HttpResponse> response = taskletService.saveTask(newTask); assertTrue("Task saved successfully.", response.getBody().getStatusCode() == "200"); assertTrue("Task Id matches", ((TaskMetaData) response.getBody().getResponseData()).getTaskId() == "456"); logger.info("Save task done"); } @SuppressWarnings("unchecked") @Test public void testGetTaskByUserId() { logger.info("Fetching the task items"); ResponseEntity<HttpResponse> response = taskletService.getTaskByUserId("users@taskletAPI.com"); assertTrue("Task fetched successfully.", response.getBody().getStatusCode() == "200"); assertTrue("User Id matches", ((List<Task>) response.getBody().getResponseData()).get(0).getUserId() .equals("users@taskletAPI.com")); logger.info("Get task done"); } } |
Testing with Postman API client
Before testing with Postman client, make sure the application is up and running. It should run on the default port 8080 since we haven’t explicitly configured the server’s port.
Launch the Postman client and make a HTTP POST request to http://localhost:8080/tasklet/task?version=1
. Input the below sample request.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
{ "created_on": "11-JUN-2018", "task_content": [ { "status": "INCOMPLETE", "task_item": "Vegetables" }, { "status": "COMPLETE", "task_item": "Cakes & Fruits" }, { "status": "COMPLETE", "task_item": "Get Rice" } ], "task_id": "123", "task_name": "Buy Groceries", "task_status": "Active", "user_id": "users@taskletAPI.com" } |
You should see the response similar to the one shown in the below listing.
Conclusion
We have seen how to integrate Spring Boot with Amazon DynamoDB . Finally, we have written unit tests to test our Tasklet API service. We will add more features of Spring Boot and Spring Cloud to our application in the upcoming posts. Stay tuned!
The source code we have worked on so far is available in the downloads section. If you have any questions, feel free to post it in the comments section. Thank you.
Download source code : spring-boot-with-dynamodb.zip
10 Comments
Nisar · October 9, 2018 at 6:46 am
Vignesh – The article is awesome.
Just a quick question.
I want to know if it makes sense to just go with aws sdk (“aws-java-sdk-dynamodb” – see the url below) as opposed to “spring-data-dynamodb”:
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/GettingStarted.Java.html
I know I’ll miss all the spring related eco-system capabilities if I chose “spring-data-dynamodb”. However, just using the aws sdk could be a reasonable option in my spring boot application?
Vignesh M · October 12, 2018 at 6:14 am
Nisar,
The main advantage that
spring-data-dynamodb
brings is simplifying the data access layer with DynamoDB. If we use the aws-java-sdk-dynamodb, we have to deal with certain activities ourselves..like we will createUpdateItemSpec,GetItemSpec
to perform operations on dynamoDB table.In short, if we need fine-grained control on the table operations, we have to go with aws-java-sdk-dynamodb library, otherwise spring-data-dynamodb should serve our purpose.
Nisar · October 13, 2018 at 4:12 am
Thank you for your feedback!
Rajib · October 9, 2018 at 9:05 am
I have used this code as same to same and changed access key and secret access key.but i am getting below error in postman.
“timestamp”: “2018-10-09T08:59:36.116+0000”,
“status”: 500,
“error”: “Internal Server Error”,
“message”: “The security token included in the request is invalid. (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: UnrecognizedClientException; Request ID: OOM30F157ERCRU4J4FHTQ2QUH7VV4KQNSO5AEMVJF66Q9ASUAAJG)”,
“path”: “/tasklet/task”
Vignesh M · October 12, 2018 at 6:47 am
Rajib,
I have tested the code again, with valid credentials replaced. And it’s works.
However, I was able to replicate your issue by making my credentials
Inactive
in the IAM console.Can you check whether that is the case for you? if your credentials are Inactive, you would get this error.
Daniel Marcos · November 20, 2018 at 3:10 pm
Hi Rajib,
Thanks for your article!! It’s what I was searching for and everything works great. But I have to said that define AWS credentias it’s necesary in local just to let the dynamodb connection find them. I created a file in my local machine (~/.aws/credentials) with empty credentials and now your project runs perfect.
Thanks.
Regads.
Rao · December 9, 2018 at 4:49 pm
Hi Vignesh,
Interesting facts are there in this post and thanks for your effort .
my requirement is , lets consider using same example, I want to develop project with AWS lambda function with CRUD operation, and I will create Rest endpoints using AWS gateway which will invoke lambda functions, possible could you please share example if you have any..
Thanks
Rao
Vignesh M · December 19, 2018 at 5:30 am
Thanks Rao for your comments. You might have to look into the serverless framework which helps in orchestration and deployment of lambda functions and other AWS resouces you need.
Introduction to that framework is here: https://blogs.rayfocus.com/serverless-framework-with-aws-example/
Mayur · February 6, 2019 at 2:01 pm
Hi Vignesh,
I am new for working with DynamoDB. Your article was really helpful for me to implement my services. I want to implement REST OATH2 protocol implementation for my REST application. Could you please help me if you have any example of it.
Thanks
Vignesh M · February 12, 2019 at 3:33 am
Hi Mayur,
Glad that you find this post helpful.
For example using OAuth2, you can refer the article: https://blogs.rayfocus.com/secure-your-spring-rest-api-using-oauth2/
Hope it will also be helpful to you.