Skip to content

Architecture Decision Records

Discuss

Gitter

Crowdsourcing platform: Eliminate direct RDBMS access

  • Status: APPROVED
  • Driver(s): Rajat Singhal
  • Approver(s): Pramod Verma, Vivek Raghavan
  • Impact Area: High Level Architecture
  • Date created: 2021-03-24
  • Date closed: 2021-03-26

Technical Story: Eliminate direct RDBMS access

Context and Problem Statement

Currently, the application service directly talks with RDBMS to fetch the sentences shown while doing Contributions and to fetch contributions while doing the Validations.

When the application will scale and more features are added, there may be some contention at RDBMS layer.

See current architecture

Solutions

  • Add cache layer over RDMS
  • Write to topics asynchronously

Tech Choices

  • For cache : AWS ElasticCache - Redis, Apache Ignite : ADR
  • For queue: AWS Kafka, AWS Kinesis : ADR

Proposed Architecture

ADR

  • Caching and messaging queue layer can be added with small imcremental changes

Positive Consequences

  • RDBMS will be off loaded from frequent reads and writes
  • Users will experience lower latency
  • System will be able to scale easily at data layer
  • Audio processing like 'automatic validation', 'SNR' etc can be done in 'Contributions processor'
  • Contribution processor can be in any other langauge like Python which is more suited for audio processing
  • If in future the analytics need to be moved to separate data store, rich application domain events can be used rather than crude CDC events.
  • Additional domains can be on boarded easily like 'Likho India' by adding new domain topics, processors and services
  • Can be easily evolved into event driven microservices architecture assuming more features will come onto this platform

Negative Consequences

  • Overhead of maintaining cache and queue system
  • The user experience may change (may be for go) due to async behaviour of the system

Decision Outcome

WIP

Choose Cache above RDBMS

  • Status: PROPOSED
  • Deciders: Rajat Singhal, Heera Ballabh, Soujyo Sen
  • Impact Area: Tools and frameworks
  • Date: 2021-03-24

Technical Story: []

Context and Problem Statement

Options

AWS ElasticCache(Redis), Apache Ignite

Solutions

Decision Outcome

not yet decided

Choose messaging queue system

  • Status: PROPOSED
  • Deciders: Rajat Singhal, Heera Ballabh, Soujyo Sen
  • Impact Area: Tools and frameworks

  • Date: 2021-03-24

Technical Story: []

Context and Problem Statement

Options

Apache Kafka, AWS Kinesis

Solutions

Decision Outcome

not yet decided

Continuous Intelligent Data Pipeline

  • Status: PROPOSED
  • Deciders: Rajat Singhal, Pramod Verma, Vivek Raghavan
  • Impact Area: High Level Architecture
  • Date: 2021-03-24

Technical Story: Continous Intelligent Data Pipelines

Context and Problem Statement

Solutions

Decision Outcome

Choose a UI framework

  • Status: PROPOSED
  • Deciders: Rajat Singhal, Sunny Goel
  • Impact Area: Tools and frameworks
  • Date: 2021-04-17

Technical Story: []

Context and Problem Statement

Options

Vue, React, Custom framework

Solutions

Decision Outcome

not yet decided