High-load storage mechanism
The business goal
The goal of the project is enhancing a current functionality of the game service infrastructure. This module serves for analyzing game logs.
An application is able to analyze hundreds of gigabytes of text logs with minimum time delays showing only those logs that meet necessary criteria. Logs could be stored on a distributed basis on different machines on a local network. The software application works in 2 ways: it filters logs from a previous time period or monitors logs in real time. It provides monitoring and analyzing of log files with distributed architecture.
From the technical side we used Python and its web framework Flask. We also use RQ and Redis.
The result of the project was decreasing a timeout for service by about 15%. The delivered application also helped make operations for the game simpler.