Log Stream

Harness the power of real-time log data management with ease and efficiency.

Architecture

Prerequisites

Technologies Used

Installation and Setup

1. Clone this repository.

   git clone https://github.com/OmAvhad/log-stream.git
   

2. Navigate to the project directory.

   cd log-stream
   

3. Run the following command to build and start the application:

   docker-compose up -d --build
   

4. Make migrations for the database.

   docker-compose run --rm flask flask db upgrade
   

Producer

• Topics

  • auth
  • database
  • email
  • payment
  • server
  • services

• Produce logs (Publish logs to a topic)

  docker-compose run --rm flask python producer/producer.py --topic TOPIC_NAME
  

• View logs (Consume logs of a topic)

  docker exec -it kafka kafka-console-consumer --bootstrap-server localhost:9092 --topic TOPIC_NAME --from-beginning