Migrating ToolJet server from Ruby on Rails to Node.js
We finally decided to rewrite ToolJet server using TypeScript!
ToolJet has two main components, the client and the server. ToolJet client is a ReactJS application and ToolJet server is a Ruby on Rails API-only application. Whenever a new application is built using ToolJet, the frontend ( client ) generates the definition of the app in JSON and the server persists the versioned definitions on a PostgreSQL database. The server also acts as a proxy that run queries on top of the data sources ( data sources in ToolJet includes databases like MongoDB and API based services like Google Sheets ).
Ruby accounts for only 14.5% of our codebase. When we built ToolJet, we chose Ruby on Rails as the backend because of its ability to quickly prototype and iterate due to its strong adherence to ‘convention over configuration’.
Node.js has a very active and robust community. Stackoverflow reported that Node.js takes the top spot in terms of usage with more than half of the respondents using Node.js. With our shift to 100% Node.js, developers from this growing community will be able to use and extend ToolJet easily.
Other benefits of migrating to Node.js
a) Unopinionated: with fewer restrictions, there is the freedom to adopt conventions as per the need of developers.
b) Faster execution: non-blocking I/O system allows simultaneous processing of requests resulting in faster performance.
undefined is not a function. We decided to use TypeScript as it helps us to avoid mundane errors and to improve the maintainability of our codebase.
In the Rails world, everything is already decided for you. But in the Node.js world, there is an endless possibility of file structures, naming conventions and tooling. For example, in Rails, we used ActiveRecord but in Node.js, we get to ( or have to) choose ORM ( if any ). When we started looking for a suitable Node.js backend framework. A huge range of frameworks was available to choose from. This can lead to Choice Paralysis but on the bright side, having too many options at our disposal helps us choose what's best for our use case.
Choosing the framework
Since we are migrating from Ruby on Rails, SailsJS was the obvious choice because of its similarity with Rails. We decided not to use SailsJs when we came across issues raised by developers related to the built-in ORM, waterline.
Express is a very minimal yet powerful framework. Express, being an un-opinionated framework, doesn't come with an error handler, body-parser etc. This gives the developer a lot of freedom but with a lot of freedom comes a lot of responsibilities to choose the right way to do something. We did not want to spend a lot of time discussing what framework to use for every single requirement. Hence, we decided not to use Express.
Meteor is a powerful full-stack Node.js framework. We did not go ahead with Meteor as it doesn't support PostgreSQL and migrating the database to MongoDB wasn't something we want to spend our time on. ( We came across meteor-postgres but as their documentation says, it is still a work in progress ).
NestJS has everything that we were looking for in a backend framework. NestJS is a slightly opinionated framework but gives some level of flexibility by allowing the usage of other libraries. For example, NestJS uses Express under the hood but it can be configured to use Fastify as well.
The new architecture for the server
We decided to go ahead with NestJS because:
a) It fully supports TypeScript
b) Database agnostic: we can directly use any Node.js database integration libraries or ORM. NestJS documentation explains in detail integrating TypeORM and Sequelize.
c) Excellent documentation: everything is explained well.
We started looking for an Object Relational Mapper (ORM) because we do not want to spend our time building and debugging SQL queries. TypeORM and Sequelize were the most mature choices. We chose TypeORM because it is a matured ORM available for TypeScript.
In the coming days, we will be working on migrating the Ruby on Rails endpoints and query services to Node.js in a way that the users will not have to change the existing data in their PostgreSQL database.
We would love you to check out ToolJet on GitHub: https://github.com/ToolJet/ToolJet/.