Monday, May 30, 2022

Capacity Estimation and Planning

 1 Byte = 8 bits

1 byte = 1 ASCII character

2 bytes = 1 UTF-16

UTF-8
UTF-8 is based on 8-bit code units. Each character is encoded as 1 to 4 bytes.


2^32  = 4,294,967,296

1 KB = 1000 Bytes

1 MB = 1000 KB

1 GB = 1000 MB

1 TB = 1000 GB

1 PB = 1000TB 


1 Million = 1,000,0000 = 10^6

1 Billion = 1,000,000,000 = 10^9

1 Trillion = 1,000,000,000,000 = 10^12


Request Calculation per second

1 M req per day = 1 M req / 86400 seconds = 12 req per second

1.5 M req per day = 1.5 * (1 M req / 86400 seconds) = 18 req per second

10 M req per day = 10 M req / 86400 seconds = 120 req per second

1 B req per day = 1B req / 86400 seconds = 12000 req per second


Redis can handle 81000 req per second

Number of users
Twitter users = 300 million active monthly users

Youtube = 2 Billion active monthly users
Number of videos watched per day = 5 billion

Facebook users 3 billion users out of which 2 billion are daily active users

Pastebin users = 17 million

Bitly = 600 million bitly link created per month.


Name
Length
char1 byte
short2 bytes
int4 bytes
long4 bytes
float4 bytes
double8 bytes
long double16 bytes

Note that on AIX® and Linux® PPC a long double is 8 bytes.

pointer4 bytes
ptrdiff_t4 bytes
size_t4 bytes
time_t4 bytes
clock_t4 bytes




Videos :


Design Principles and Design Pattern

 Design Principles

  • Encapsulate what varies
  • Favor Composition over Inheritance
  • Loose Coupling
  • Program to interfaces
  • SOLID Principle
    • Single Responsibility Principle
    • Open/Closed Principle
    • Liskov's substitution principle
    • Interface Segregation Principle
    • Dependency Inversion Principle.

   Design Patterns

  • Creational Design Pattern
    • Factory Pattern
      Let the factory do the job of creating the objects rather than instantiating the object using New
      https://java-design-patterns.com/patterns/factory/ 
    • Abstract Factory Pattern -
      Creating a Factory from list of factories.
      https://java-design-patterns.com/patterns/abstract-factory/
    • Builder Pattern
      The builder pattern is an object creation software design pattern with the intentions of finding a solution to the telescoping constructor anti-pattern. Telescoping constructor anti-pattern means that the constructor is getting bigger and bigger due to addition of a lot of parameters.
      https://java-design-patterns.com/patterns/builder/
    • Prototype Pattern - Clone the existing object and then make modification to the object rather than creating the object from scratch.
      https://java-design-patterns.com/patterns/prototype/
      In java can be done by implementing the Cloneable  and overriding the Clone() method. But Clonable makes a shallow copy. We need to override the clone method for the child objects and the clonable method should be updated
    • Singleton Pattern

Sunday, May 29, 2022

Restful Services


https://www.edureka.co/blog/microservices-design-patterns

Design Steps
  • Identify object model
  • Create Model URIs
  • Determine Representations
  • Assign HTTP Methods

JAX-RS 2.0 (Java API for RESTful Services).
JAX-RS stands for JAVA API for RESTful Web Services

From <https://www.tutorialspoint.com/restful/restful_jax_rs.htm>


From <https://restfulapi.net/create-rest-apis-with-jax-rs-2-0/>

Caching
Has the following attributes in the header
  • Date
  • Last Modified
  • Cache Control - Public, Private, no-cache/no-store, max-age, must-revalidate
  • Expires
  • Age

  1. Scope microservices using Bounded Context
    1. Domain Driven Design- Bounded Context
    2. Ubiquitous Language
  2. N+1 problem - https://restfulapi.net/rest-api-n-1-problem/
  3. Asynchronous Web Services - Event Based
    1. Competing Worker Pattern
    2. Fanout Pattern
    3. Async API Call
  4. Architecture of API Based Microservice
    1. Functional Requirement
      1. Loosely coupled
      2. Independently changeable
      3. Independently deployable
      4. Support and honor contracts
      5. Technology Agnostic
      6. Stateless API
    2. API Architectural Styles
      1. Basic REST Implementation
        1. Following the rest constraints
        2. HTTP Protocol is followed
        3. Inherits the advantages of the Web
        4. Uses JSON/XML to send and receive data
        5. Uses Resource End point to communicate
        6. Uniform Resource Identifier
        7. Stateless
        8. Cacheable
        9. Example
      2. Pragmatic REST
        1. Not all resources would do CRUD Operation
        2. Some resources takes actions/Task
        3. Name these resources as Verbs instead of nouns
      3. HATEOAS REST
      4. RPC
      5. SOAP
    3. API Architectural Pattern
      1. Façade Pattern
      2. Proxy Pattern
      3. Stateless service pattern
      1. No State information maintained on server
      2. Clients maintain state
      3. State is send as part of the request
      4. Advantages: Help in scalability, performance and availability
  1. Composing Microservices
    1. Composition Patterns
      1. Broker Composition Pattern
      2. Aggregate Composition Pattern
      3. Chained Composition Pattern
      4. Proxy Composition Pattern
      5. Branch Composition Pattern

Data Consistency
  1. Two Phase Commit
  2. Saga Pattern-Saga Execution Coordinator
  1. Eventual Consistency : Eventually the Data would be consistent across the systems. It is a practice rather than pattern. It chooses Availability over Consistency.

API Gateway




Open Source API Gateway

Sunday, May 22, 2022

Google Interview Guide

 Engineering Manager Interview Guide

https://igotanoffer.com/blogs/tech/google-engineering-manager-interview 


Google Behavioural Questions Guide

https://igotanoffer.com/blogs/tech/google-behavioral-interview

https://igotanoffer.com/blogs/tech/google-engineering-manager-interview

Google Interview Guide
https://cdn.shopify.com/s/files/1/0417/7869/files/GoogleManager.pdf?v=1590400847


Some useful Links for preparation.


  • Code Review

  • Google File System

    • https://www.youtube.com/watch?v=eRgFNW4QFDc
    • https://ai.google/research/pubs/pub51

    • What is your mentorship style? Give a example of a employee that you helped them grow
    • Have you handled any arrogant and stubborn employee ? Give Example.
    • How do you keep your team motivated ?
    • Have you taken an unpopular decision and how did it work out ? examples 
    • Have you worked on a project with vague requirement ? How did you go about the same?
    • How have you contributed to team and stepped up when the team needed me.
    • How do you create visibility for the team?
    • Tell me about a time you led a team through a difficult situation
    • How would you build a diverse and inclusive team
    • Tell me a time when you failed or made a big mistake

    General

    • Why are you an effective R&D leader?
    • Tell me about yourself
    • Why Google?

    People management

    • How do you handle conflicts?
    • How do you handle people who are not team players?
    • How do you deal with low performers?
    • How do you deal with high performers? 
    • Tell me about a time you developed and retained team members
    • How do you manage your team’s career growth?
    • Tell me about a difficult employee situation that you handled well/not so well
    • What would you do with someone that had stayed at the same level for too long?
    • How do you recruit good engineers?
    • Give an example of how you helped another employee
    • Tell me about a time you had a conflict with your supervisor and how you resolved it

    Project management

    • As a manager, how do you handle trade-offs?
    • Describe how you deal with change management
    • Describe in detail a project that failed
    • Describe a project in the past that was behind schedule and provide concrete to steps that you took to remedy the situation
    • Tell me how you would balance engineering limitations with customer requirements
    • What was the largest project you've executed?
    • Tell me about a time you needed to deliver a project on a deadline but there were multiple roadblocks and constraints to deliver. How did you manage that situation?
    • Tell me about a project, product or system you worked upon. What were the design and technical problems you faced? How did you solve them?

  • Thursday, October 10, 2019

    Designing Distributed Systems Useful Links

    Distributed Systems Useful Links
    Below are some of the links to the different technologies which are used to build a distributed system. It is necessary to know the different technologies available and how they can be used to build a distributed system.

    Topics to Discuss for System Design Interview
    Whenever you are Designing a System the following things should be discussed.
    1. What is the problem Statement
    2. Requirement and Goal of the System?
    3. Functional Requirements
    4. Non Functional Requirements
    5. Some Design Considerations
    6. Capacity Estimation and Constraints
      • Traffic Estimate
      • Storage Estimate
      • Bandwidth Estimate
      • Memory Estimate
    7. High Level Design
    8. System API
    9. Database Design
    10. Load Balancer
    11. Caching
    12. Disaster Recovery plan
    Keep the following things in Mind and call the following things during the interview to show that you have a deep understanding of the concepts
    • Load Balancer and usage
    • CAP theorem - Will you choose Consistency or Availability ?
    • Multiple instances of the service
    • Software Architectural pattern for used
    • Microservices Architectural pattern used
    • Discuss Databases
      • Type of database used - NoSQL vs SQL and why ?
      • Do we need DB Sharding?
      • Do we need Consistent Hashing ?
      • Master - slave architecture
      • Will Read and Write happen from same DB ?
    • Discuss Caching
      • Caching strategy - DBs that can be used for Caching
      • Cache eviction policy
    • Discuss Queueing mechanism
      • Kafka, RabbitMQ
    • Discuss Disaster Recovery
      • How multiple instances could help 
      • How DB backups can be made

    Service Registry

    Rabbit MQ:


    Video : 



    KAFKA vs RabbitMQ



    JMS vs RabbitMQ:







    Capacity Estimation

    Design a Job Schedular


    Design a Job Schedular


    A job scheduler to schedule M jobs on N threads on a single machine. The input contains the following data:
    1. Job Name
    2. Duration: time taken for job completion
    3. Priority: priority of the job. P0>P1>P2
    4. Deadline: Expiry time after which job should not be run (The clock starts from 0 and deadline is the actual clock time)
    5. UserType: Type of user who has initiated the job, Precedence of users Root>Admin>User
    Shortest Job First – SJF : Shortest job first (SJF), is a scheduling policy that selects the waiting process with the smallest execution time to execute next. In case of tie choose the job according to the following order-
    1. Priority(higher priority job gets scheduled first)
    First Come First Serve -FCFS : Jobs are executed on first come, first serve basis. Take the input as the order of jobs need to be scheduled.
    Fixed Priority Scheduling -FPS : Each process is assigned a priority. Process with highest priority is to be executed first and so on. In case of tie choose the job according to the following order-
    1. User Type
    2. Longest Job First
    Earliest Deadline First – EDF : Next job will be searched on the basis of job which is closest to its deadline. In case of tie, choose the job according to the following order-
    1. Priority(higher priority job gets scheduled first)
    2. Duration(lesser duration job gets scheduled first) In case we cannot schedule a job such that it completes before its deadline then it should be ignored.
    You would be given a list of jobs (refer example below for format) and number of threads as input. You are expected to print the order of jobs scheduled for each algorithm on each thread as output.
    Input
    Threads= 2
    JOB NAME
    DURATION
    PRIORITY
    DEADLINE
    USER TYPE
    J1
    10
    P0
    10
    Root
    J2
    20
    P0
    40
    Admin
    J3
    15
    P2
    40
    Root
    J4
    30
    P1
    40
    User
    J5
    10
    P2
    30
    User
    Output
    SJF
    Thread 1 - J1, J3, J4
    Thread 2 - J5, J2
    FCFS
    Thread 1 - J1, J3, J5
    Thread 2 - J2, J4
    FPS
    Thread 1 - J1, J4, J5
    Thread 2 - J2, J3
    EDF
    Thread 1 - J1, J2
    Thread 3 - J5, J4

    From <https://www.geeksforgeeks.org/flipkart-interview-experience-sde-1-experienced/>


    AWS Useful Links