Don’t Use Sails (or Waterline)

Posted on

The Shyp API currently runs on top of the Sails JS framework. It's an extremely popular framework - the project has over 11,000 stars on Github, and it's one of the top 100 most popular projects on the site. However, we've had a very poor experience with it, and with Waterline, the ORM that runs underneath it. Remember when you learned that java.net.URL does a DNS lookup to check whether a URL is equivalent to another URL? Imagine finding an issue like that every two weeks or so and that's the feeling I get using Sails and Waterline.

The project's maintainers are very nice, and considerate - we have our disagreements about the best way to build software, but they're generally responsive. It's also clear from the error handling in the project (at least the getting started docs) that they've thought a lot about the first run experience, and helping people figure out the answers to the problems they encounter trying to run Sails.

That said, here are some of the things we've struggled with:

  • The sailsjs.org website broke all incoming Google links ("sails views", "sails models", etc), as well as about 60,000 of its own autogenerated links, for almost a year. Rachael Shaw has been doing great work to fix them again, but it was pretty frustrating that documentation was so hard to find for that long.

  • POST and PUT requests that upload JSON or form-urlencoded data sleep for 50ms in the request parser. This sleep occupied about 30% of the request time on our servers, and 50-70% of the time in controller tests.

  • The community of people who use Sails doesn't seem to care much about performance or correctness. The above error was present for at least a year and not one person wondered why simple POST requests take 50ms longer than a simple GET. For a lot of the issues above and below it seems like we are the only people who have ran into them, or care.

  • By default Sails generates a route for every function you define in a controller, whether it's meant to be public or not. This is a huge security risk, because you generally don't think to write policies for these implicitly-created routes, so it's really easy to bypass any authentication rules you have set up and hit a controller directly.

  • Blueprints are Sails's solution for a CRUD app and we've observed a lot of unexpected behavior with them. For one example, passing an unknown column name as the key parameter in a GET request (?foo=bar) will cause the server to return a 500.

  • If you want to test the queries in a single model, there's no way to do it besides loading/lifting the entire application, which is dog slow - on our normal sized application, it takes at least 7 seconds to begin running a single test.

  • Usually when I raise an issue on a project, the response is that there's some newer, better thing being worked on, that I should use instead. I appreciate that, but porting an application has lots of downside and little upside. I also worry about the support and correctness of the existing tools that are currently in wide use.

  • Hardcoded typos in command arguments.

  • No documented responsible disclosure policy, or information on how security vulnerabilities are handled.

Waterline

Waterline is the ORM that powers Sails. The goal of Waterline is to provide the same query interface for any database that you would like to use. Unfortunately, this means that the supported feature set is the least common denominator of every supported database. We use Postgres, and by default this means we can't get a lot of leverage out of it.

These issues are going to be Postgres oriented, because that's the database we use. Some of these have since been fixed, but almost all of them (apart from the data corruption issues) have bit us at one point or another.*

  • No support for transactions. We had to write our own transaction interface completely separate from the ORM.

  • No support for custom Postgres types (bigint, bytea, array). If you set a column to type 'array' in Waterline, it creates a text field in the database and serializes the array by calling JSON.stringify.

  • If you define a column to be type 'integer', Waterline will reject things that don't look like integers... except for Mongo IDs, which look like "4cdfb11e1f3c000000007822". Waterline will pass these through to the database no matter what backend data store you are using.

  • Waterline offers a batch interface for creating items, e.g. Users.create([user1, user2]). Under the hood, however, creating N items issues N insert requests for one record each, instead of one large request. 29 out of 30 times, the results will come back in order, but there used to be a race where sometimes create will return results in a different order than the order you inserted them. This caused a lot of intermittent, hard-to-parse failures in our tests until we figured out what was going on.

  • Waterline queries are case insensitive; that is, Users.find().where(name: 'FOO') will turn into SELECT * FROM users WHERE name = LOWER('FOO');. There's no way to turn this off. If you ask Sails to generate an index for you, it will place the index on the uppercased column name, so your queries will miss it. If you generate the index yourself, you pretty much have to use the lowercased column value & force every other query against the database to use that as well.

  • The .count function used to work by pulling the entire table into memory and checking the length of the resulting array.

  • No way to split out queries to send writes to a primary and reads to a replica. No support for canceling in-flight queries or setting a timeout on them.

  • The test suite is shared by every backend adapter; this makes it impossible for the Waterline team to write tests for database-specific behavior or failure handling (unique indexes, custom types, check constraints, etc). Any behavior specific to your database is poorly tested at best.

  • "Waterline truncated a JOIN table". There are probably more issues in this vein, but we excised all .populate, .associate, and .destroy calls from our codebase soon after this, to reduce the risk of data loss.

  • When Postgres raises a uniqueness or constraint violation, the resulting error handling is very poor. Waterline used to throw an object instead of an Error instance, which means that Mocha would not print anything about the error unless you called console.log(new Error(err)); to turn it into an Error. (It's since been fixed in Waterline, and I submitted a patch to Mocha to fix this behavior, but we stared at empty stack traces for at least six months before that). Waterline attempts to use regex matching to determine whether the error returned by Postgres is a uniqueness constraint violation, but the regex fails to match other types of constraint failures like NOT NULL errors or partial unique indexes.

  • The error messages returned by validation failures are only appropriate to display if the UI can handle newlines and bullet points. Parsing the error message to display any other scenario is very hard; we try really hard to dig the underlying pg error object out and use that instead. Mostly nowadays we've been creating new database access interfaces that wrap the Waterline model instances and handle errors appropriately.

Conclusion

I appreciate the hard work put in by the Sails/Waterline team and contributors, and it seems like they're really interested in fixing a lot of the issues above. I think it's just really hard to be an expert in sixteen different database technologies, and write a good library that works with all of them, especially when you're not using a given database day in and day out.

You can build an app that's reliable and performant on top of Sails and Waterline - we think ours is, at least. You just have to be really careful, avoid the dangerous parts mentioned above, and verify at every step that the ORM and the router are doing what you think they are doing.

The sad part is that in 2015, you have so many options for building a reliable service, that let you write code securely and reliably and can scale to handle large numbers of open connections with low latency. Using a framework and an ORM doesn't mean you need to enter a world of pain. You don't need to constantly battle your framework, or worry whether your ORM is going to delete your data, or it's generating the correct query based on your code. Better options are out there! Here are some of the more reliable options I know about.

  • Instagram used Django well through its $1 billion dollar acquisition. Amazing community and documentation, and the project is incredibly stable.

  • You can use Dropwizard from either Java or Scala, and I know from experience that it can easily handle hundreds/thousands of open connections with incredibly low latency.

  • Hell, the Go standard library has a lot of reliable, multi-threaded, low latency tools for doing database reads/writes and server handling. The third party libraries are generally excellent.

I'm not amazingly familiar with backend Javascript - this is the only server framework I've used - but if I had to use Javascript I would check out whatever the Walmart and the Netflix people are using to write Node, since they need to care a lot about performance and correctness.

*: If you are using a database without traditional support for transactions and constraints, like Mongo, correct behavior is going to be very difficult to verify. I wish you the best of luck.

Liked what you read? I am available for hire.

45 thoughts on “Don’t Use Sails (or Waterline)”

  1. Barney Davis

    Wow. I’m building an app now, and was strongly considering and researching Sails. This greatly concerns me though, because the point of a framework (imo) is to know one is working with well designed and tested code. This seriously makes me consider going a different route. Thanks for the article.

    Reply ↓
  3. David

    What alternatives in Node you propose? Meteor for me is an options because is a very solid project, but is hard work with it if you used work with ORM or something like that

    Reply ↓
    1. kevin Post author

      doesn’t meteor only work with Mongo or something? to be honest I would just use vanilla express with `node-postgres` and write the queries. not sure I trust other libraries tbh.

      Reply ↓
  4. Nick Vlku

    Yikes. I’ve been dipping my toes into NodeJS for a side project and ran screaming from Sails after a few hours. (It had a weird “Rails developers building something like it in Node” feel to it).

    Ended up playing with Koa. Absolutely gorgeous… Uses ES6 generators and yields to make asynchronous code look… well synchronous. Strongly recommend playing with it.

    That being said, if I was just going to try and “build something to get it out quick for customer feedback” I’d still go with the tried and true Django + Postgres. Quick, easy and scales well. Only difficulty is persistent connections, but Swampdragon handles that pretty gracefully.

    For longer term problems, Dropwizard in Java makes a lot of sense. I’ve seen lots of orgs start with something like Python/Django, Ruby/Rails and move on to a JDK powered language with Dropwizard when they needed more performance.

    A caveat, Dropwizard for Scala is kind of dead — most orgs that use it have forks of it internally — but its not being maintained in the same way dropwizard-java is. Still, not a bad framework, that and all.

    Reply ↓
  5. Noland

    Just wanted to point out that Sails is in version 0.11. Note the zero in front.

    When I see versions like that, it tells me the framework is not yet optimized for performance and there’s going to be a few holes. I would assume the only reason you’re using it is because you want to ride the front edge of the Node wave, not find a stable production server.

    No need to run away screaming if you go into it knowing it’s a pre-1.0 version.

    Reply ↓
    2. Benoît Hubert

      If you apply the same reasoning to Node…

      Well Node stayed for a good while at version 0.10.x, 0.11.x, and so on… Yet quite a few companies already were using it for serious, production projects.

      So I don’t think version number is a reliable indicator of project maturity here.

      Reply ↓
  6. Albert

    I’ve been using Sails for writing production apps for the past year.

    It started off great. The blueprint API helped me get running quickly, and was great as long as the application was small. However, once the application started to get larger the glaring deficiencies of Sails began to surface. I can safely say that I will never ever recommend this framework to anyone – not even a beginner. The issues you encounter are not worth it, and development/improvements on the framework are so slow you are better off going with Express from the start.

    Flat file structures for models/controllers/services? Uh no thanks.

    Global variables littered everywhere?

    Having the most-in-demand features be barely touched for several years (deep populate anyone?).

    Not to mention everything posted in the article.

    Great write up. I will be following your posts.

    Reply ↓
  7. Diego

    That’s a shame. The Node.js community needs serious MVC and ORM frameworks. In order to convince enterprises that there’s a migration path from well-established Java and .Net frameworks based on convention over configuration such as Spring and Hibernate. I’ve also run into issues when trying to run an insert against a custom Postgres schema. I haven’t received an answer from the community yet. I’m currently trying to fix the issue on my own and submit the fix.
    https://github.com/balderdashy/sails-postgresql/issues/207

    Reply ↓
  8. Sai Nageswar Satchidanand

    Yup.. exactly.. Sailsjs is great work, however, it is still in its initial stages.. Used it to develop a monitoring app which has very high network calls with lots of aggregation… Having lots of memory leaks… :( :(

    Reply ↓
  9. wmehanna

    This arcticle focus on cons only of SailsJS., no pros.

    Also, PHP frameworks, .NET MVC and others have their pros/cons.

    I’ve been developping under SailsJS for Backend and AngularJS for frontend and the projects i’ve been working on are working perfectly fine.

    Reply ↓
    1. kevin Post author

      Thanks, I’m aware other frameworks are not perfect, though the amount of unexpected behavior seems to be pretty large in this case.

      In 3 months when your tables grow to have hundreds of thousands of rows and count() crashes the server, be sure to update this comment!

      Reply ↓
  10. Rich

    While there are many valid points here, I think there’s a little fear-mongering going on as well.

    There’s no need to include past bugs in a post like this, because it’s a moot point. The bug has been fixed, life goes on.

    Also, some of these could be solved by submitting a pull request to the project without much hassle. I’m looking at the hardcoded typo in the command, and wondering why this wasn’t either pointed out to the team or submitted as an easy fix.

    That said, there are good points and Sails isn’t the solution for everything. I find it to be a great starting point for small to medium applications, and I haven’t found anything that’s as good or as easy to use. There’s a bit of a learning curve since most of Sails is configuration-driven, but once you master that you can move on to making great apps.

    For the default behaviors, you can disable most of them easily. I’d suggest maintaining a “bootstrap” configuration that sets the defaults that YOU want, and use that as the basis of your apps moving forward.

    Reply ↓
    1. mwkaicz

      Absolutely agree,
      there is few good points but many debateble points … and many of them have easy solution (f.e: default routes … show me someone who builds something bigger on sails and still uses default routes for controller)

      Don’t breake a stick over Sails.js, it’s pretty hard to find something better and more complex than Sails.js !

      Reply ↓
  11. Michael

    Great article. I did check a lot of your links to the problems, and a bunch seem to be fixed. It would be nice if you could strike out these bullet points so others who don’t do their own research don’t see this is current.

    Reply ↓
        1. kevin Post author

          The updated versions continue to have showstopping bugs. I don’t have time to do the full amount of research, but here’s a partial list:

          https://github.com/balderdashy/waterline-sequel/issues/82
          https://github.com/balderdashy/sails/issues/3682#issuecomment-205328393
          https://github.com/balderdashy/sails-postgresql/issues/236
          https://github.com/balderdashy/waterline-sequel/issues/86

          This is from the last 5 days, only, and a two minute scan of my inbox. Why would I update, new versions are likely to be filled with more problems.

          Reply ↓
  12. Kevin Whinnery

    Large frameworks like Sails go against the grain of Node’s design, which is to focus on small modules that do one thing well (Unix philosophy). When Node is used in this way, the inner workings of your system are not magical, and you have a lot of flexibility in how you accomplish a task. The downside, of course, is that batteries are not included for most Node projects – you end up assembling your entire stack from smaller components.

    Server-side JavaScript has a lot of benefits, among them orchestration of multiple services, a rich module ecosystem (of varying quality, of course, and ORM tech is the biggest laggard IMHO), great front-end tools, and a shared development paradigm on both client and server. Sails is not the framework I would choose, but I would push back on your assertion that the fix is to use Python or Go. For an API server, using Express or Hapi is likely to produce better results.

    Reply ↓
  13. muslim

    I am also working on a test project on sails and encountered issues:
    1) no embedded docs
    2) no upsert
    3) no guide to handle M-M cases

    may be I am wrong but I am a mongoose user waterline is far behind from mongoose for handling mongodb

    I prefer using sequelize.js for sql databases and mongooses for mongodb
    and use vanilla express for my projects

    Reply ↓
  14. Amreesh Tyagi

    I was a die heart fan of SaisJs. I have used sails in most of my small apps. But now going to move on from it. What do you think about StrongLoop’s Loopback framework? I am planning to create an enterprise application, which can have 1000 or may be 10,000 concurrent users in future. The only concern of loopback is it’s licensing , which I am not able to understand e.g. licensing of basic passport component, https://github.com/strongloop/loopback-component-passport/blob/master/LICENSE.md

    Reply ↓
    2. kevin Post author

      i don’t know anything about it, in general i’m wary of the javascript “flavor of the month” approach to packaging and releasing. many of these libraries don’t have a large user base or one that’s particularly motivated by correctness

      Reply ↓
    1. kevin Post author

      I don’t think any cross-database ORM can be good, you end up with the lowest common denominated feature set, and the error handling isn’t great, because it’s usually specific to a backend (e.g. Postgres constraint failures). I’m also worried about the “flavor of the month” nature of Javascript packages, I’d rather lean on a few things that lots of people are using and testing in production.

      I’d probably just write SQL using node-postgres and maybe a few helper functions around it.

      Reply ↓
      1. Benoît Hubert

        Yeah, I couldn’t agree more, after having used Sails/Waterline with mongoDB, MySQL and Postgres, I feel like writing queries with a few helper functions as you say would be just as good.

        Reply ↓
    2. kevin Post author

      just scanning the readme, here’s an example of why I don’t trust things like this. any method named “updateOrCreate” is vulnerable to a race, since two updates can complete on different threads, and then one of the creates can error and usually the handling is bad. you need to do it in the reverse order – try both creates first and then try the update if create throws a failure. https://github.com/biggora/caminte/issues/73

      Reply ↓
  16. Dimitri Roche

    Thanks for this write up. Many people have misinterpreted popularity (github stars!) with quality when it comes to Sails js.

    Reply ↓
  18. PC

    Definitely appreciate the honest review. Sharing the pitfalls of a library is valuable for people considering using it.

    In the spirit of open source, why did you (Kevin Burke / Shyp) fix the problems in the Postgres adapter? We all want something for nothing, but given the opportunity to contribute back to the community, I am an advocate of giving a little in return for so much.

    Reply ↓
    1. kevin Post author

      It’s little frustrating that you’ve written this, since we have contributed *a ton* of open source code, spent a ton of time clearly reporting issues, and documentation to improving the libraries we use:

      – https://github.com/brianc/node-postgres/pull/1006
      – https://github.com/goodeggs/librato-node/pull/32
      – https://github.com/Shyp/pg-transactions
      – https://github.com/balderdashy/waterline-sequel/commit/ae8dfd05751c0794c68b16e2a3efa52c32f0efa8
      – https://github.com/balderdashy/waterline-sequel/issues/64
      – https://github.com/balderdashy/sails/issues/3693
      – https://github.com/mochajs/mocha/pull/1848
      – https://github.com/balderdashy/sails-postgresql/pull/180
      – https://github.com/shyp/lusitania (Fork, with much clearer error messages on validation failures)
      – https://github.com/balderdashy/sails-postgresql/issues?utf8=%E2%9C%93&q=is%3Aissue+author%3Akevinburkeshyp
      – Not to mention the 10,000 lines we’ve removed/patched/improved in our sails fork: github.com/Shyp/sails

      As I wrote in the post, part of the problem is that Waterline/Sails has to cater to everyone, which significantly hampers their ability to merge/make improvements.

      Reply ↓
  19. Gadi

    Thanks for the post. I wrote a small sails app a few months back, and was very frustrated by the ORM support. It is very difficult to join tables and create select conditions. I know its due to the noSQL support, but its not worth it for my app.

    I’m starting to re-write in django. I just don’t get the advantage of server side JS over python.

    Reply ↓
  20. Nicholas Rempel

    I strongly agree with you on this Kevin. We sunk a lot of effort into using Sails.js and it became unwieldy shockingly fast.

    We switched to Adonis JS and couldn’t be happier.

    Thanks for the thorough writeup.

    Reply ↓
  21. davidrogers

    I’ve been using sails for years on production apps and love it.

    In my opinion, just because something isn’t the right solution for you, doesn’t mean that it’s not the solution for others. Making statements like “don’t use X” when people put a lot of thought and hard work into it, in my opinion, is just being a jerk.

    Reply ↓
  22. Mike

    Kevin, thank you for your article. It’s refreshing to hear specific points as to why something shouldn’t be used, rather than just saying it shouldn’t be used (or only having 1 or 2 points). I especially appreciated your PostgreSQL observations.

    Seeing how your article is now 2 years old (and that could be a lifetime in Open Source) I’d be interested to hear a follow-up to your articles; especially, what are still the outstanding problems.

    I have read a little about how there are still problems with Waterline, but as I understood Sails, it was meant to be customized by default. It was created to get up and running but the default configuration wasn’t supposed to be the best to begin with. That said, there are other ORM packages, such as Sequelize, that can replace Waterline in Sails. Have you had a chance to look into these alternatives? I know someone like me may not have the traffic or the conditions that would present these pitfalls immediately, so I appreciate learning from experiences of someone like yourself.

    Having not done much with SailsJS yet, I am evaluating it and would be interested to see a more up-to-date article. Given the time nature, I’m not willing to dismiss it just yet. I especially would like to see someone review their 1.0b version that is coming.

    Thanks for all the hard work, both in patching and communicating issues. I hope instead of deterring people away, it has instead encouraged more support from the community.

    Reply ↓
    1. kevin Post author

      to be honest – I quit the company with the Sails framework, after helping to significantly dismantle it – we forked it here: https://github.com/Shyp/sails, and removed a lot of the pieces, slowly – the config framework, the logging framework, etc.

      there are so many good options for building web frameworks nowadays that I’m not even sure it’s worth bothering with, even if it’s better.

      Reply ↓
      1. Anon

        Does this mean your team refactored, Kevin? Mind giving some insight into that? We are about to tackle this… I apologize, I am not a developer but curious for our business which runs on sailsjs framework. We need to scale and we CANT. it’s frustrating.

        Reply ↓
  23. Dennis Bauszus

    Saddens me to say so but I have to agree with your observations. At the time of writing (2018) the PostgreSQL implementation is still a disaster. Connection pooling may either not work or is poorly documented. Whichever it is I am now forced to write my own connector which will probably take less time than bug fixing the sails-postgresql adapter.

    Reply ↓

Leave a Reply

Your email address will not be published. Required fields are marked *

Comments are heavily moderated.