MEAN+ Organization 04 Jul 2013

MEAN+

Although I’ve been using Ruby for some small tasks lately, NodeJS and the MEAN stack comprise the core of my day to day development. I’ve been using MEAN heavily for the past 7 months, and since I’ve discovered NodeJS I’ve developed a project structure that seems to scale well enough. AngularJS is a recent addition to this, but it fits nicely. Although I tend to make adjustments with each new project, the core of it all has stayed the same for some time now.

To begin with, what is the MEAN stack? It’s composed of the following four technologies/frameworks/things:

I also use LESS and CoffeeScript for all CSS/JS code, SocketIO for realtime comms, Grunt to handle building, Mocha for testing, and C9’s Architect module to structure everything. So technically this is the MEAN + LCGMA Stack, but since that isn’t a very elegant name, I’ll just call it the MEAN+ stack. I’ll go over each component in turn outlining how they fit into the general structure presented in this post, and what benefits they bring to a project.

MEAN(+) - Architect

At the heart of my structure is C9’s Architect module. Architect allows you to structure your application as a series of “modules”, which have access to one another through interfaces that they choose to expose. This allows you to clearly separate the different parts of your app into distinct groupings of functionality, and properly emulates private variables and methods. A module needs two things to exist, a package.json describing it, and a source file implementing the setup method. For example, a module that does nothing useful:

package.json

 1 {
 2   "name": "doesnothing",
 3   "version": "0.0.1",
 4   "main": "logic.js",
 5   "private": true,
 6   "plugin": {
 7     "consumes": [],
 8     "provides": ["useful"]
 9   }
10 }

logic.coffee

 1 setup = (options, imports, register) ->
 2 
 3   # Perform module initialization, access other modules, etc
 4 
 5   # Register your exposed method(s)
 6   register null,
 7     "useful":
 8 
 9       log: (m) -> console.log m
10 
11 module.exports = setup

Then in another such module, you can access that method by including “useful” in its package.json “consumes” array, and grabbing the object with:

1 setup = (options, imports, register) ->
2 
3   useful = imports["useful"]
4   useful.log "So useful it's unreal"
5 
6   ...

“Stop right there!”, I hear you say, “You named the logic file logic.js in the package.json, but yet you are writing it in CoffeeScript! Blasphemy!” I’ll address that in a bit when I get to Grunt :) Before that, let’s see how architect modules work together to create an awesomely scalable application. To keep this post from getting ridiculously long, I’ll take a screenshot of my Architect-specific folder structure:

MEAN+ Architect Folders

I break my modules out into two main folders. One named “line”, which is shared among all of my projects, and another aptly named “modules”. Each module gets its own folder (not sure there is any way around this, not that I’ve tried), with its own logic.coffee and package.json. This is an exceedingly clean architecture, and makes for a great dev experience. How do we actually pull all the modules together though? That’s what the architecture.coffee is for. To illustrate:

architecture.coffee

 1 module.exports = [
 2   "./line/express",
 3   "./line/mongodb",
 4   "./line/snapshot",
 5   "./line/socketio",
 6 
 7   "./modules/core/init",
 8   "./modules/core/views",
 9 
10   "./modules/pages/about",
11   "./modules/pages/home",
12   "./modules/pages/settings"
13 ]

Our modules are loading in the order they are specified; this leads to things like two separate initialization modules, one called before anything else, and another at the very end. To keep things simple though, we’ll pretend we only need to perform initialization before loading everything else :) Launching the application at this point is simple:

mean+.coffee

1 architect = require "architect"
2 
3 app = architect.createApp architect.loadConfig(__dirname + "/architecture.js") , (err, app) ->
4   if err then throw err
5 
6   # Application running

(M)EAN+ - MongoDB

Any useful application needs database access. I’ve worked with MySQL in the past with PHP, but as soon as I started using NodeJS I became an instant fan of MongoDB. I’ve set up a common structure for modules that I re-use with little modification, and it works great. It is composed of two things, an Architect module that serves as a wrapper around Mongoose, and a folder containing application models.

Each model gets its own file (User.coffee, Message.coffee, etc), which is broken out in index.coffee:

exports.Thing = require "./Thing.js"

Models look very similar, with the core structure being:

 1 mongoose = require "mongoose"
 2 
 3 schema = null
 4 model = null
 5 
 6 exports.createSchema = ->
 7   schema = new mongoose.Schema
 8     name: String
 9     type: String
10     modified: { type: Boolean, default: false }
11     # ...
12 
13 exports.createModel = ->
14   model = mongoose.model "Thing", schema
15 
16 exports.getModel = -> model
17 exports.getSchema = -> schema

The mongodb architect module takes care of initializing everything, and provides a helpful fetch function, which can execute multiple queries and return the results to a callback. I’ve found it invaluable in preventing nesting/callback hell. It’s a bit long to post here (~120 lines), so I made a Gist of it.

Using the module, db queries are as simple as:

 1   ...
 2   # In the setup function of a module
 3   db = imports["core-mongodb"]
 4 
 5   db.models.Thing.getModel().findOne
 6     name: "Stack"
 7   , (err, thing) ->
 8     ...
 9 
10   db.fetch [ "Thing", "Thing" ], [{ name: "Stack" }, { name: "MEAN" }], (data) ->
11     stack = data[0]
12     mean = data[1]
13 
14     ...

M(E)AN+ - ExpressJS

The usage of Express is quite straightforward, I’m sure you’ve seen some incarnation of it before. You create your server, then declare route handlers of the form:

app.get "/", (req, res) ->

All I’ve done is written an architect module that handles the creation of routes, middleware, and the instantiation of the server itself. Since you need to create the server before you add handlers too it, and you need to set it to listen on a port after adding handlers (not verified), I’ve broken the module up into two main functions. Also made a Gist out of it

registerRule() - Call to register middleware, passing a function taking (req, res, next) as parameters

registerPage() - Call to register routes in a friendly manner. registerPage() is actually a bit special, so I’ll go on a tangent for a moment and explain it. It takes 4 parameters, (route, view, args, logic). Route is what it claims to be (“/”, “/about”, etc), and view is the name of the view to render, with the other arguments being optional. Args is an object that is passed through to the view, and logic is the interesting part. Normally, you register basic pages with:

server.registerPage "/", "home.jade"

If you need to do anything fancy before rendering the page though (99% of the time), you can pass in a logic function, which takes a single argument named “render”. A code listing is worth a thousand words:

1 server.registerPage "/", "home.jade", {}, (render) ->
2 
3   db.fetch "Thing", { name: "ImportantThing" }, (thing) ->
4     render thing

‘render’ is a function provided by the ExpressJS architect module, which takes an object as a single parameter. The object is then passed down to the view, instead of the args object passed to registerPage(). This way, you can perform your DB queries and processing and whatnot, then render the view with a single function call.

Of course, you can do this with using ExpressJS by itself, the only reason why I wrote the module is to standardize error messages and handling. That + to integrate Express with the architect way of doing things.

MEAN(+) - Socket.io

Not much to say here really, just another architect module wrapping SocketIO and providing a friendly way to register listeners in modules. Gist, and an example:

1   sio = imports["core-socketio"]
2   sio.addListener "some-trigger", (data) ->
3     ...

MEAN(+) - LESS and static/views organization

I keep all static/public files in a folder named “static” in the root directory of projects. This contains images, scripts, fonts, and whatever else needs to be served. I keep my LESS in a subfolder named ‘less’, organized in the standard manner. Grunt takes care of building it for me when modifications are detected. Inside of the ‘less’ folder (and ‘js’), I have a subfolder named “vendor” for keeping all 3rd-party code.

I compile all LESS into a single stylesheet, static/css/style.css, which is included in the default layout.

On the topic of views, I keep those in a root-level folder named “views”, mixed in with any layouts. I further organize the views folder when necessary, with layout and functionality-specific folders, etc.

ME(A)N+ - AngularJS

This is the real meat of it all, although a very recent addition. I first used Angular while helping out on a friend’s project (Boombox), and I’ve been using it for nearly every project since. Keeping in mind that that only means ~3 weeks worth of time, I’ve learned plenty to share. In terms of architecture, I keep all Angular-related code in a folder named “coffee”, in the static directory.

Currently this only contains a file named “app.coffee”, and various controllers in a folder named “controllers”. app.coffee defines the routes, and performs global DOM manipulation and whatnot. An example structure:

 1 # Set up SocketIO
 2 socket = io.connect "", { secure: false }
 3 
 4 window.Thing = angular.module "Thing", []
 5 
 6 # Routing
 7 window.Thing.config ($routeProvider, $locationProvider) ->
 8 
 9   # HTML5 Urls
10   $locationProvider.html5Mode true
11 
12   $routeProvider.when "/dashboard", {
13     controller: "dashboard",
14     templateUrl: "/views/dashboard"
15   }
16   $routeProvider.when "/settings", {
17     controller: "settings",
18     templateUrl: "/views/settings"
19   }
20   $routeProvider.otherwise({ redirectTo: "/dashboard" })
21 
22   true
23 
24 $(document).ready ->
25   # Ba bammm....

Each controller then is of the form:

window.Thing.controller "dashboard", ($scope, $http, $route) ->
  # ...

I currently keep AngularJS “views” mixed in with application views, and dedicate an architect module to registering their routes. I’ll be moving them into their own view folder in the static directory, to remove the need for own routes.

MEAN(+) - Grunt

I’ve been using Grunt for even less than I’ve been using AngularJS, but again, it has proven invaluable. Grunt automates common tasks, such as building your CoffeeScript/LESS, testing, and anything else you want it to do.

I like to build my project in a “build” folder, and keep a nodemon instance up to restart my application automagically whenever something changes. This, coupled with grunt-contrib-watch, means you can check changes nearly instantly after making them.

Since it is quite long, I’ve made a Gist of the Gruntfile I use as a base for my projects. Although it is definitely rough around the edges, it does its job well, and can be easily extended.

Example useage:

  # Generate build/static/css/style.less
  grunt less

  # Build all coffee files
  grunt coffee

  # Build on changes in views, LESS, and coffee files
  grunt watch

  # Re-create build dir, run "npm install", and build everything
  grunt rebuild

Conclusion

I won’t go into unit testing with Mocha since I’m a complete noob when it comes to that. I will dedicate a blog post to it once I write unit tests for my active projects, and get a better grasp of them myself.

Overall, this application structure is clean and makes for a really simple development process. Add your controllers to static/coffee/controllers, set up backend methods with a new module or two, interface with them using Socket.io calls, and scale smoothly in complexity. I use spew throughout my code to emit clean log messages for a variety of listeners. I wrote it, go check it out! (npm, github)

Here’s a repo containing a MEAN+ boilerplate

A final screenshot illustrating project structure with all of the above in place:

MEAN+ Organization