“Chroma, a project Delivery Hero have been actively developing for over 2 years, is a platform for collectors. It allows detailed cataloguing and categorization of what they call “figures”: Photographic records of any collectable, with visual and textual indexing for search and retrieval.
Each category in Chroma represents the child tree underneath itself and the category cover photos are calculated dynamically, based on your current search and browsing context. If you are browsing categories of a specific user or place, we provide the latest photo of that category for that user or place. On top of this, there are additional requirements such as custom collections and private figures or users. This requires a tremendously complicated dynamic calculation system that needs to operate at sub-300 millisecond speeds” – Tuna Vargi –
How did they accomplish this?
In the beginning, they architected everything on MongoDB. There are three reasons they used only MongoDB. It is very easy to use and adapt RDBMS to your project. It also easy to scale. Lastly, JSON like data structure was highly usable to keep the high detail information data of categories and photo.
However, MongoDB became hard to scale as their figure count grew more and more so they decided to use graph database such as Neo4j. Neo4j platform is developed by Neo Technology and it is the most popular graph database engine. It is comfortable to express every data as intuitive visualizing data because Neo4j have a great UI/UX for users and used Cypher query language. Of course, Mongo DB also has good features for simple querying and storing data as a document, but the aggregation function and recursive queries in Neo4j is behind the function of MongoDB.
They had kept using the platform implementing a graph database using PostgreSQL as core database. It was impressive for the ease of querying and usability.
In spite of the fact that it is very effectiveness and amazing, they gave up maintaining two databases at the same time because Neo4j is a great but it gets slower with increased data scale. In addition, the main reason they gave up using two databases for the project is that sync issues between Neo4j and MongoDB occurred rather frequently.
They were spending too much time to write complicated queries and figure out sync problems. In the end, they cannot help giving up Neo4j platform. PostgreSQL is almost perfect to use aggregation function and complex queries except performance impacts such as speeding of queries and visualization data.
Although they gave up maintaining two databases due to a lot of issues, Tuna Vargi, the senior engineer of Delivery Hero, makes sure the database implementing a graph database based on PostgreSQL is very useful if it does not have any sync issues and the speed issue with high volume of data. Many IT company such as Bitnine and Neo Technology are developing the performance observably in every release. By improving the data storing features and complex querying performance, graph database based on PostgreSQL like Agens become a primary database in DBMS market.
Vargi, T. (2016, July). Our journey from Graph Databases to PostgreSQL. Retrieved November 14, 2016, from http://engineering.hipolabs.com/graphdb-to-postgresql/
BITNINE GLOBAL INC., THE COMPANY SPECIALIZING IN GRAPH DATABASE
비트나인, 그래프 데이터베이스 전문 기업