General Topics for Distributed System

 

What is Service Mash ?

Google File System

Map Reduce

Fallacies of Distributed Systems

1. The network is reliable;
2. Latency is zero;
3. Bandwidth is infinite;
4. The network is secure;
5. Topology doesn't change;
6. There is one administrator;
7. Transport cost is zero;
8. The network is homogeneous.

https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing 

Software Quality
https://asq.org/quality-resources/software-quality 

Lambdas & Streams

 

https://docs.oracle.com/javase/8/docs/api/java/util/function/package-summary.html

https://www.linkedin.com/learning/java-lambdas-and-streams 

Time Complexity Calculation

 Time Complexity Calculation

Code Review Interview

When doing a code review interview or even looking in the code review for the team the following steps can be used 

• Look into the main logic first
• Understand the complete code
• Check for Time and Space complexity. Can we make it better ?
• Check if alternate data structure can be used ?
• Can the Dependency Injection and Inversion of Control be used ?
• Is there code redundancy or code duplication ?  
• Will the code work with concurrency ?
• Naming Conventions
• Readability of the code
• Check if the Logging (Error, Debug, Info) is properly done?
• Check if the Constants can be moved ?
• Can we apply any design patterns to make the code more readable
• Check if it is easy to write the unit test for the code ?
• Unit testing 
• Comments added to the code ?
• IO calls are expensive. Use caching, request collapsing and batch queries to reduce load.

Google Code Review Standards

https://github.com/google/eng-practices/blob/master/review/reviewer/index.md

Videos to watch before a Code Review Interview :

Good Reads :

https://blog.tryexponent.com/how-to-ace-a-code-review/

Probabilistic Algorithms

Bloom Filter

Calculate the memory required and the number of hash functions required here

https://hur.st/bloomfilter/ 

Bloom filter can test with 100% confidence if the value is not present and with lower confidence if the value is present.

Efficiently count with BitMap
https://www.youtube.com/watch?v=8ZgRW0DNus4

Count Min Sketch
The algorithm is used for data which is coming as a stream and we want to have a fast processing. We will have multiple hash functions and will have a table where we store the output of the values hashes. 
If we need to find the occurrence of the value we run the values through the hash, get the stored value and then take the minimum of those values.

https://link.medium.com/Tr0RDZZt2qb 

Sharding

 Sharding

Sharding helps in spreading out the data in different machines in case the data volume to too high. Sharding has its own disadvantages like increase maintenance, query result aggregation, increased infrastructure cost.

Before considering sharding we should consider if vertical scaling will solve the issue. If yes then should go for vertical scaling rather than Sharding.

Advantages of Sharding

• Increase read/write throughput
• Increased storage capacity
• Data Locality

Sharding Types

• Range/Dynamic Sharding
• Algorithmic / Hash Sharding
• Entity/Relationship Sharding
• Geography Based Sharding

References

https://www.mongodb.com/features/database-sharding-explained

https://www.mongodb.com/basics/sharding

Microservices Design Patterns

If you are preparing for an interview or just trying to get started with a Microservice then you should be aware of some of the design patterns which can be used to make your microservices resilient and useful.

Principles Used to Design Microservice Architecture

The principles used to design Microservices are as follows:

1. Independent & Autonomous Services
2. Scalability
3. Decentralization
4. Resilient Services
5. Real-Time Load Balancing
6. Availability
7. Continuous delivery through DevOps Integration
8. Seamless API Integration and Continuous Monitoring
9. Isolation from Failures
10. Auto -Provisioning

 These are some of the Microservices Design Patterns

1. Retry Pattern
   In this pattern we try to retry to request to another microservice for certain number of time in case there is a failure before return a failure status to the client.
   
   
2. Circuit Breaker Pattern
   In this pattern the microservices will keep track on the number of times a service returns failure. In case the number of failures goes above a threshold then the circuit opens and starts returning error for the calling service. The circuit breaker will keep on checking the failing service. Once we get a success then the circuit will close again so that the traffic can go to the service which was failing previously.
   
   
   
   
   
   
   
   
   
3. Bulkhead Pattern
   The patterns helps in isolating the services within a microservice from failure.
   
   
   
   
   
4. Aggregator Pattern
   The pattern is used to get the response from various microservices and the aggregate the result together.  
   
   
5. API Gateway
   The request first goes to the API gateway which routes the request to respective service based on the API Gateway.
   
   
6. Chained or Chain of Responsibility
   Chained or Chain of Responsibility Design Patterns produces a single output which is a combination of multiple chained outputs. So, if you have three services lined up in a chain, then, the request from the client is first received by Service A. Then, this service communicates with the next Service B and collects data. Finally, the second service communicates with the third service to generate the consolidated output. 
   
   
7. Asynchronous Messaging
   This pattern is used for asynchronous messaging between different microservices. We can add a messaging queue between the services. This pattern is helpful if we have a lot of requests coming and the service might need some time to process the request. 
   
   
8. Event Sourcing
   The event sourcing design pattern creates events regarding the changes in the application state. Also, these events are stored as a sequence of events to help the developers track which change was made when. So, with the help of this, you can always adjust the application state to cope up with the past changes. You can also query these events, for any data change and simultaneously publish these events from the event store.
   
   
9. Command Query Responsibility Segregation
   According to this pattern, the application will be divided into two parts: Command and Query. The command part will handle all the requests related to CREATE, UPDATE, DELETE while the query part will take care of the materialized views. 
   
   
10. Branch
    Branch microservice design pattern is a design pattern in which you can simultaneously process the requests and responses from  two or more independent microservices.
    
    
11. Saga Pattern
    This pattern helps with implementing a business transaction that spans multiple services. A saga is a sequence of local transactions. Each local transaction updates the database and publishes a message or event to trigger the next local transaction in the saga. If a local transaction fails because it violates a business rule then the saga executes a series of compensating transactions that undo the changes that were made by the preceding local transactions.
    

    There are two ways of coordination sagas:

• Choreography - each local transaction publishes domain events that trigger local transactions in other services
• Orchestration - an orchestrator (object) tells the participants what local transactions to execute
  https://medium.com/javarevisited/distributed-transaction-management-in-microservices-part-2-saga-pattern-53808a55e641
12. 2 Phase or 3 Phase Commit
    https://medium.com/javarevisited/distributed-transaction-management-in-microservices-part-1-bb7dc1fbee9f 
13. Backend for Frontend (BFF) design pattern
    BFF is a variant of the API Gateway pattern, but it also provides an additional layer between microservices and each client type separately. Instead of a single point of entry, it introduces multiple gateways
    Video : https://www.droidcon.com/2021/11/10/backend-for-frontend-the-secret-of-a-great-mobile-project/

Reference:

• https://medium.com/aspnetrun/microservices-resilience-and-fault-tolerance-with-applying-retry-and-circuit-breaker-patterns-c32e518db990 
• https://www.edureka.co/blog/microservices-design-patterns
  
• https://microservices.io/patterns/data/saga.html
• https://medium.com/trendyol-tech/saga-pattern-briefly-5b6cf22dfabc#:~:text=2PC%20works%20as%20a%20single,makes%20the%20data%20eventually%20consistent.
• https://medium.com/mobilepeople/backend-for-frontend-pattern-why-you-need-to-know-it-46f94ce420b0

Algorithms

Sorting Algorithm

 https://www.geeksforgeeks.org/radix-sort/

https://www.geeksforgeeks.org/heap-sort/

Dynamic Programming - Optimization of the solution.

1. Greedy Algorithm  - In a greedy algorithm the decision is taken once and the result is calculated based on the same decision. Like Dijastra's Algorithm where we always take the shortest vertex to find the shortest path. Other example Minimum cost spanning tree - Always select the minimum cost edge.

2. Dynamic Programming - The decision is taken at every step to get to an optimal solution.

Memoization  is a technique used to optimize the time complexity. In the technique we store the result of the calculation so that we can use them again. It is a top down approach. This is used in Recursion.

Tabulation is a technique which is a bottom up approach. We start with the smallest value and calculate the bigger problems. It is used as part of the iterative method

Software Architectural Patterns

 

Architectural Patterns

1. Layered pattern
2. Client-server pattern
3. Master-slave pattern
4. Pipe-filter pattern
5. Broker pattern
6. Peer-to-peer pattern
7. Event-bus pattern
8. Model-view-controller pattern
9. Blackboard pattern
10. Interpreter pattern

Hexagonal Architecture

Onion Architecture

Layered Architecture

Domain Driven Design

Domain-Driven Design is about creating shared understanding of the problem space that is reinforced ubiquitously via conversations, code and diagrams.
Things to know in Domain Driven Design 

1. What is Bounded Context ?
   Bounded context defines tangible boundaries of applicability of some sub-domain. It is an area where a certain sub-domain makes sense, while the others don't. It can be a talk, a presentation, a code project with physical boundaries defined by the artifact.
   
   
2. What is Ubiquitous Language ?
   A Bounded Context is an explicit boundary within which a domain model exists. Inside the boundary all terms and phrases of the Ubiquitous Language have specific meaning, and the model reflects the Language with exactness.

Event Driven Design

Reactive Architecture

https://www.lightbend.com/blog/understand-reactive-architecture-design-and-programming-in-less-than-12-minutes

Reference

• https://towardsdatascience.com/10-common-software-architectural-patterns-in-a-nutshell-a0b47a1e9013
• https://medium.com/@edamtoft/onion-vs-clean-vs-hexagonal-architecture-9ad94a27da91
• https://www.codeguru.com/csharp/understanding-onion-architecture/

Network Layer

 Websockets vs HTTP

WebSockets, on the other hand, allow for sending message-based data, similar to UDP, but with the reliability of TCP. WebSocket uses HTTP as the initial transport mechanism, but keeps the TCP connection alive after the HTTP response is received so that it can be used for sending messages between client and server.

https://sookocheff.com/post/networking/how-do-websockets-work/#:~:text=WebSockets%2C%20on%20the%20other%20hand,messages%20between%20client%20and%20server.

Network Layer