Saturday 9 January 2021

Microservices - Transactional Messaging and handling 2 phase commit problem

 

In a microservices architecture, when data is updated in one microservice, another service may also require to reflect the updated state or the services may be involved in a SAGA pattern for completing a transactional outcome.

But updating the data in the database and publishing the message is a 2 phase commit problem. The database update could be successful whereas the publishing of the message could fail or vice versa. If one think of storing and republishing the messages only on failure, then this can result in messages going out of order.

The typical way to handle this issue is to use transactional outbox pattern, i.e, store the message to be published in a table every time the data is updated. This would mean that the entire operation can be committed in a single transaction and no 2 phase commit problem is involved.

Now, this message needs to be published from the transactional outbox table to the mesaging broker for the other microservices to take corresponding actions. This can be done via one of the following approaches:

1. Polling publisher: There can be a Polling publisher which frequently checks this transactional outbox table and picks the data in the timeCreated order and publishes the messages in the same order. If the messaging broker is down or not reachable while trying to publish the message, the Polling publisher can do finite retries and quit for the next iteration to publish the message. The message publishing can get delayed slightly with this approach based on the frequency of the Polling publisher. So the frequency of the Polling publisher needs to be adjusted based on the use case. The polling publisher need to delete the data from Transactional Outbox table once the message is published.

2. Transaction Log tailing: There are various connectors available with the messaging brokers like kafka for tailing a source database and publishing the messages and even to a target sink database. Such connectors can be used for tailing the transactional outbox table and the connector can be used for publishing the messages. This approach is reliable but is database specific and a bit complex. So if the minor delay is fine for the use case, polling publisher could be the go to solution.

But Transaction Log tailing may be a requirement for a NOSQL database to handle this scenario. The NOSQL database typically will not be having a transaction and hence to store the messages in a single transaction to the database, the generated message may also be required to be stored in the same table as the one in which domain object is stored. Now the challenge will be to identify the messages that needs to be published for a polling publisher. Using transactional log tailing will help to generate messages for only the newly created/updated data eliminating this challenge. 


Connector frameworks:

Debezium

Linkedin Databus

Dyanamo DB streams

Confluent connectors






Friday 8 January 2021

Microservices and Software Architecture Styles

  Software Architecture Styles

1. Layered Architecture - Application is classified into multiple layers like presentation layer, business layer, persistence layer in a 3 tier architecture.

2. Hexagonal Architecture - Business logic is kept independent of other layers with communications happening via inbound/outbound adapters (Interfaces).

3. Microservices Architecture - The application is divided into multiple services based on domain/maintainability etc and other factors, each service of a microservice architecture typically follows hexagonal architecture.


Advantages of microservices

1. Enables continous delivery and deployment of large complex application via independent services.

2. Independent scaling

3. Fault isolation

4. Independent technology stack and can easily update to latest tech stack

5. Independent CI/CD pipeline and faster deployment/release

6. Smaller services are be easily maintainable

7. Enables teams to be autonomous

Drawbacks of microservices architecture

1. Finding the right set of services is challenging

2. Features spanning multiple services requires careful co-ordination:

 a. how to handle transaction across services (Saga)

 b. how to query data from multiple microservices (CQRS/API Composition)

 c. may require careful rollout plan with involved microservices to not break backward compatibility etc.

3. Testing and deployment can be difficult when multiple services are involved.


SOA Vs microservices

SOA uses SOAP based webservices where as the microservices use lighter REST based or gRPC based calls.

SOA used to integrate large monolithic applications where as services are broken down as multiple micro services in a microservice architecture.

SOA based architecture have single shared database for the entire application where as microservices have its own database.


Thursday 7 January 2021

Write a program to print longest sequence of each numbers in a given array

 

Sample input array: {1,1,2,2,2,3,3,2,2,2,2,3,3,3,4,1,1,1}


Expected output:

1 occurred 3 times max in sequence

2 occurred 4 times max in sequence

3 occurred 3 times max in sequence

4 occurred 1 times max in sequence


https://github.com/prasune/Algorithms/tree/master/src/main/java/com/test/algorithm


package com.test.algorithm;

import java.util.HashMap;
import java.util.Map;

public class MaxRepeatingOccurrence {

public static void main(String[] args) {
int[] nums = new int[]{1, 1, 2, 2, 2, 3, 3, 2, 2, 2, 2, 3, 3, 3, 4, 1, 1, 1};

printMaxRepeatingOccurrences(nums);
}

private static void printMaxRepeatingOccurrences(int[] nums) {
Map<Integer, Integer> numOccurrenceMap = new HashMap<>();
int currentSequenceCount = 0;
for (int i = 0; i < nums.length; i++) {
currentSequenceCount = currentSequenceCount + 1;
if (i == nums.length - 1 || nums[i] != nums[i + 1]) {
if (numOccurrenceMap.get(nums[i]) == null || currentSequenceCount > numOccurrenceMap.get(nums[i])) {
numOccurrenceMap.put(nums[i], currentSequenceCount);
}
currentSequenceCount = 0;
}
}
for (Map.Entry<Integer, Integer> numOccurrenceEntry : numOccurrenceMap.entrySet()) {
System.out.println(numOccurrenceEntry.getKey() + " occurred " + numOccurrenceEntry.getValue() + " times max in sequence");
}
}
}

Write a program to print combination of numbers in an array that gives a target sum

 

A set of unique sorted numbers are given as an input like - {1,2,4,6,9,10}

Write a program to print all combination of numbers that gives a target sum, say 15

In this case, the answer should be:

10 4 1

9 4 2

9 6


https://github.com/prasune/Algorithms/tree/master/src/main/java/com/test/algorithm/recursion


package com.test.algorithm.recursion;


public class FindSumCombinations {

public static void main(String[] args) {
int[] nums = new int[]{1,2,4,6,9,10};
int target = 15;

printCombinations(nums, target);
}

private static void printCombinations(int[] nums, int target) {
for (int i = 0; i < nums.length; i++) {
if (nums[i] <= target && printCombinations(nums, target - nums[i], i+1)) {
System.out.print(nums[i]);
System.out.println();
}
}
}

private static boolean printCombinations(int[] nums, int target, int startIndex) {
for (int i = startIndex; i < nums.length; i++) {
int newTarget = target - nums[i];
if (newTarget > 0) {
if (printCombinations(nums, newTarget, i+1)) {
System.out.print(nums[i] + " ");
return true;
}
} else if (newTarget == 0) {
System.out.print(nums[i] + " ");
return true;
} else {
return false;
}
}
return false;
}
}