This is an Ecommerce microservices project. In microservices architecture, each service is a separate entity and can be developed, deployed, scaled and maintained independently. This project is a simple example of an Ecommerce microservices project. It consists of the following services:
Product Service, Order Service, User Service, Inventory Service, API Gateway. Eureka Server (Service Discovery), Notification Service, Payment Service (to be added later), Config server, The choice of communication between services will be RESTful API. The options are:
RestTemplate (deprecated), Feign Client(Declarative HTTP Client: Provides a more concise and easier way to call other services) WebClient (Reactive & Recommended: Supports asynchronous, non-blocking calls). Asynchronous and non-blocking calls are calls recommended for microservices architecture.
Asynchronous calls: These allow a program to initiate a task and move on to other tasks before the initial task is completed. This helps in utilizing resources more efficiently and improving the overall performance of the application.
Non-blocking calls: These ensure that the program does not wait for the task to complete before moving on to the next task. This is particularly useful in I/O operations where waiting for a response can be time-consuming.
In the context of microservices, asynchronous and non-blocking calls are recommended because they help in handling multiple service requests concurrently without blocking the execution of other tasks, leading to better scalability and performance
Eureka Server (Service Discovery: Used to register and discover services).
IMPORTANT NOTE: The project will be implemented using the web client. When microservices need to call each other dynamically, they register with Eureka server and use Eureka client with for service discovery.
API Gateway: This is a single entry point for all the services. It is a reverse proxy that routes requests to the appropriate service.
Steps:
Add dependencies (cloud-starter-netflix-eureka-client, Starter WebFlux (For WebClient), spring actuators) in the pom.xml file for the following services: PRODUCT-SERVICE: provides the product details (product id, name, description, price, etc.) to the inventory service and order service. When new products are added, the product service will notify the inventory service to initialize the stock for the product. ORDER-SERVICE: provides the order details (order id, product id, user id, quantity, etc.) and calls the product service to get the product details. It will also call the inventory service to check if the product is available in stock and update the stock after placing the order. It will also call the user service to get the user details. POTENTIAL ISSUE: When an order is updated, the original ordered qauntity is not returned to the inventory service. The new quantity is simply deducted without considering the previous deduction. This leads to an incorrect stock count/level. SOLUTION: Track the original ordered quantity in the order service and return it to the inventory service when updating the stock. This way, the inventory service can correctly adjust the stock level. NOTIFICATION-SERVICE: will be added later. It will be used to send notifications to the users when an order is placed or updated. It will call the user service to get the user details and send the notification. It will also call the order service to get the order details and send the notification. It will also call the product service to get the product details and send the notification. USER-SERVICE/AUTHENTICATION SERVICE:
-
This is a service that will be used to manage the users of the application. -
It will provide APIs to create, update, delete and get the user details. -
It will also provide APIs to authenticate the users. -
It will call the order service to get the order details and user details.
INVENTORY-SERVICE: Manages the stock of the products, tracks available inventory and ensures orders can be fulfilled. It will call the product service to get the product details and update the stock. It will also call the order service to get the order details and update the stock. API-GATEWAY: implemented using Spring Cloud Gateway. It will be used to route the requests to the appropriate service. It will also be used to provide a single entry point for all the services. TODO: It will also be used to provide security for the services. It will also be used to provide rate limiting for the services. It will also be used to provide load balancing for the services. NOTIFICATION-SERVICE:
-
This is a service that will be used to send notifications to the users when an order is placed or updated. -
It will call the user service to get the user details and send the notification. -
It will also call the order service to get the order details and send the notification. -
It will also call the product service to get the product details and send the notification.
PAYMENT-SERVICE:
-
This is a service that will be used to process the payments for the orders. -
It will call the order service to get the order details and process the payment. -
It will also call the user service to get the user details and process the payment.
SHIPPING-SERVICE:
-
This is a service that will be used to manage the shipping of the orders. -
It will call the order service to get the order details and manage the shipping. -
It will also call the user service to get the user details and manage the shipping.
CART-SERVICE:
-
This is a service that will be used to manage the cart of the users. -
It will call the product service to get the product details and manage the cart. -
It will also call the user service to get the user details and manage the cart.
REVIEW/RATING-SERVICE:
-
This is a service that will be used to manage the reviews and ratings of the products. -
It will call the product service to get the product details and manage the reviews and ratings. -
It will also call the user service to get the user details and manage the reviews and ratings.
ANALYSIS/REPORTING-SERVICE:
-
This is a service that will be used to analyze the data and generate reports. -
It will call the product service, order service, user service and inventory service to get the data and generate reports. -
It will also call the API gateway to get the data and generate reports.
EUREKA-SERVER:
-
This is a service discovery server that allows microservices to register themselves and discover other services. -
It will be used to register the product service, order service, user service and inventory service. -
It will also be used to discover the product service, order service, user service and inventory service.
CONFIGURATION-SERVER:
-
This is a service that provides configuration properties to the microservices. -
It will be used to provide configuration properties to the product service, order service, user service and inventory service. -
It will also be used to provide configuration properties to the API gateway.
Add the @EnableEurekaClient annotation in the main class of the services. Configure the application.yml file with the Eureka server details. Create a new spring boot project for Eureka server and add the dependency( spring-cloud-starter-netflix-eureka-server) in the pom.xml file. Add the @EnableEurekaServer annotation in the main class of the Eureka server. Configure the application.yml file with the Eureka server details. Implement the WebClient in the order service to call the product service. 6.1 create a config class to initialize the WebClient.Builder bean. 6.2 Modify the OrderService class to call ProductService via Eureka Discovery 6.3 Expose an api in order controller to get the product details. Run the Eureka server and the services. TODO: Switch from Spring Data JPA to Spring Data R2DBC by updating your pom.xml org.springframework.boot spring-boot-starter-data-r2dbc io.r2dbc r2dbc-postgresql
maven command for cleaning and building the project: mvn clean install -DskipTests To highlight text in the editor: File → Settings → Editor → TODO→ Add a new pattern and choose a color.
POTENTIAL MEMORY LEAKS:
When using WebClient, ensure that you are not creating a new instance of WebClient for each request. Instead, create a single instance of WebClient and reuse it for all requests. This will help in reducing memory leaks and improving performance. Use the WebClient.Builder to create a single instance of WebClient and reuse it for all requests. This will help in reducing memory leaks and improving performance. #What was observed during integration testing: The web application [ROOT] appears to have started a thread named [reactor-http-nio-*] but has failed to stop it. This is very likely to create a memory leak: is a very common issue when using Netty-based servers (like Spring Webflux or reactive client like WebClient) in integration tests, especially when tests are run inside Spring boot + Tomcat(Servlet-based) environment. For now, it is being suppressed by setting the following property in the application-test.yml file with: logging: level: org.apache.catalina.loader.WebappClassLoaderBase: ERROR API DOCUMENTATION:
Use Swagger or Spring REST Docs to document the APIs. Add the dependency in the pom.xml file for Swagger or Spring REST Docs. Configure the application.yml file with the Swagger or Spring REST Docs details. Expose an API to get the API documentation. In this project, we will use springdoc-openapi-ui for API documentation (springdoc.org). Automatically generate doc in JSON/YML and HTML formats APIs. Check more from the link: https://springdoc.org STEPS TO ADD SPRINGDOC-OPENAPI-UI: - Add the following dependency in the pom.xml file for springdoc-openapi-ui. - change path in the application.yml: e.g., http://localhost:port/swagger-ui.html - Add the @OpenAPIDefinition annotation in the main class of the services. - Configure the application.yml file with the springdoc-openapi-ui details. - Expose an API to get the API documentation. - Access the API documentation at http://localhost:port/swagger-ui.html
dependency in the pom.xml file for springdoc-openapi-ui in HTML format. org.springdoc springdoc-openapi-starter-webmvc-ui 2.8.9 dependency in the pom.xml file for springdoc-openapi-ui in JSON/YML format. org.springdoc springdoc-openapi-starter-webmvc-api 2.8.9 command to run docker compose file from the common-utils folder: ****docker compose -f docker-compose.yml up --build command to run docker compose file from the root folder: *****docker compose -f common-utils/docker-compose.yml up --build Check actuator endpoints in the browser: *****http://localhost:port/actuator/health TESTING KAFAKA:
PRODUCER: docker run -it --rm --network common-utils_microservices-network confluentinc/cp-kafka:latest kafka-console-producer --bootstrap-server kafka:9092 --topic test-topic
After running the above command, you can type messages in the console and press Enter to send them to the Kafka topic. Press Ctrl+C to exit the producer.
CONSUMER: docker run --rm --network common-utils_microservices-network confluentinc/cp-kafka:latest kafka-console-consumer --bootstrap-server kafka:9092 --topic ecommerce-topic --from-beginning --max-messages 5
This will show the first 5 messages in test-topic. After running the above command, you will see the messages sent to the Kafka topic in the console. Press Ctrl+C to exit the consumer.
#OR
echo "Hello, Kafka!" | docker run --rm --network common-utils_microservices-network -i confluentinc/cp-kafka:latest kafka-console-producer --bootstrap-server kafka:9092 --topic test-topic Checking specific kafka broker versions:
exec into the kafka container: docker exec -it kafka kafka-broker-api-versions --bootstrap-server localhost:9092 Rebuilding docker image for a specific service with no cache:
docker build --no-cache
docker-compose up -d To run all images in a docker compose file in detached mode:
docker compose up -d --build To stop all images in a docker compose file:
docker compose down To stop a specific service in a docker compose file:
docker compose stop To start a specific service in a docker compose file:
docker compose start To view logs of a specific service in a docker compose file:
docker compose logs -f
In case, service is not behaving same in docker as in local: *mvn clean install -DskipTests *then rebuild the docker image for the service with no cache: *docker build --no-cache *docker compose down -v: to remove all volumes
*to remove specific service volume: *docker volume rm common-utils_-data *docker-compose up -d
summary: Remove all containers, volumes, and images (use with caution as this will delete all your data): *docker rm -f
Unable to render expression.
docker ps -aq → lists all container IDs (running + stopped).
docker rm -f ... → force-removes all those containers.
docker volume rm $(docker volume ls -q)
docker volume ls -q → lists all volume names/IDs.
docker volume rm ... → deletes all volumes.
docker rmi -f $(docker images -q)
docker images -q → lists all image IDs.
docker rmi -f ... → force-removes all images.
&& chaining
Ensures that the next command only runs if the previous one succeeds.
SETTING UP A CUSTOM CONTAINER FOR KAFKA FOR TESTING PURPOSES: @Container public static final GenericContainer<?> kafka = new GenericContainer<>("confluentinc/cp-kafka:7.8.0") .withExposedPorts(9092, 29092) .withEnv("KAFKA_BROKER_ID", "1") .withEnv("KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR", "1") .withEnv("KAFKA_LISTENERS", "PLAINTEXT://0.0.0.0:9092") .withEnv("KAFKA_ADVERTISED_LISTENERS", "PLAINTEXT://localhost:9092") .waitingFor(Wait.forLogMessage(".started.\n", 1));
Firebase Cloud Messaging (FCM) for sending notifications:
Add the dependency in the pom.xml file for FCM. Create a service class to send notifications using FCM. Call the notification service from the order service when an order is placed or updated. More details: https://firebase.google.com/docs/cloud-messaging Example for java: FileInputStream serviceAccount = new FileInputStream("path/to/serviceAccountKey.json"); FirebaseOptions options = new FirebaseOptions.Builder() .setCredentials(GoogleCredentials.fromStream(serviceAccount)) .build(); FirebaseApp.initializeApp(options);
Reasons why .env content gets loaded locally when services run *When there is spring boot dev tool dependency in the pom.xml file. *IDE Automatic .env loading: *Some IDEs (like IntelliJ IDEA) automatically load environment variables from a .env file when running a Spring Boot application.This feature is often enabled by default to facilitate development and testing.
Command to show dependency, for example: com.vaadin.external.google:android-json
- mvn dependency:tree -Dincludes="com.vaadin.external.google:android-json"
How to convert secret json file to base64 encoded string:
- This is useful when you want to store the secret in an environment variable or a secret manager.
- *You can use the following command to convert the json file to base64 encoded string:
- *Navigate to the directory where the serviceAccountKey.json file is located.
- *Run the following command to convert the json file to base64 encoded string:
- Linux/Mac: cbase64 firebase-service-account.json > firebase-service-account.json.b64
- Windows (PowerShell): $base64String = [Convert]::ToBase64String([System.IO.File]::ReadAllBytes("firebase-service-account.json")) Set-Content -Path "firebase-service-account.json.b64" -Value $base64String
- Open the firebase-service-account.json.b64 to get the base64 encoded string.
- NOTE: Make sure the json file is in the project root folder (if running from there) or provide the correct path to the file.
- *Check other details on how to use the secret in GitHub Actions here: ONE NOTE