We believe magic happens at the intersect of code and design. On this techblog we share the details of our approach, the technical challenges we face, the software and technology we create, the events we visit or host, and presentations we give.
nodeday is coming to Amsterdam.
Luca Maraschi speaking at the event.
Pascal Dufour will give a talk about our Open Sourced Library, Minosse
IceMobile hosting the October DevOps
Special guests speaking.
IceMobile sponsoring the Afterparty
Luca Maraschi speaking at the event.
RSVP via event website
Desmond sharing insights on Minosse: Common steps for testing api's using Cucumber.js.
Hosted at Publitas in Amsterdam.
Hosted at IceMobile HQ, speakers: Node Summit Recap, Jasper Spoel (Black Orchid) & Jelle Herold
Luca Maraschi is Panel speaker on designing Node.js Applications
Luca Maraschi will be mentor one of the tracks on Day-Zero of the Node Summit
Bert Rijsdijk and Valerio Barrila speaking about API Testing in cloud environments
Hosted at IceMobile HQ, speakers: Luca Maraschi (IceMobile) & Hongli Lai (Phusion)
Saturday October 10th the GDG Devfest, one of the most iconic events in Amsterdam for the Android community, took place once more at the Science Park. And of course, IceMobile didn’t miss the date. As a company, we always try to engage with the community of developers and experts in our fields not only as a sponsor but as collaborative partners.
After the Devfest was announced, we thought about ways in which we could participate. The most obvious ones were the same as last year: bringing beer as part of our IceMobrew initiative and preparing a presentation for the conference. Giving the great feedback that we received from GDG Devfest in 2014 we prepared both of them again. What's more, in order to secure success we brought even more beer to the event: up to 100 bottles. Then we thought carefully about what to present since last year’s presentation made by one of our tech leads was one of the most successful ones.
Why make a presentation about android performance patterns and profiling tools?
In IceMobile we usually say that magic happens at the intersection of technology and creativity. In every app that we develop we put plenty of effort in creating the best possible user experience combining both. Taking this into account, the only way to achieve a user experience that will take your app to the next level is if it is provided without any impediment. There is where performance enters the equation.
If your app feels laggy or choppy to its final users all that effort put in design will not impress them. Just thinking about how material design is build upon animations and driving the user through motion, performance in our apps is now more important than ever.
Devfest 2015, better, stronger and bigger
Thanks to the organizators for an incredible event. The Google Developers Group in Amsterdam is one of the best communities for learning and discovering new google related technologies. As well as last year, they prove themselves one of the best event making people among the industry in the Netherlands.
Find the presentation of via this link: Presentation on Slideshare
At IceMobile we're using Cucumber for a lot of our tests. When we started working with Cucumber, we had testers writing the scenario's and developers implementing the steps needed to make them run. To cut dependencies across the team we wrote Minosse, a library of reusable gherkin step definitions for testing REST API's. This set of steps basically forms a small programming language and with it testers are almost completely self-sufficient in writing test scenarios. Minosse was a great success and greatly increased the productivity and agility of our teams.
Obviously, by limiting our features to using some predefined steps, we limit the expressiveness of our feature files. We were okay with that, because we chose for Cucumber to be more productive in writing tests, not out of a desire to go full BDD. A more pressing but related problem for us was that we had no way of hiding recurring sequences of steps behind a single step. This resulted in very long, hard to understand test scenarios.
Today we present our solution to both of these problems: Gherkin Definitions. A Gherkin Definition allows one to define a test step as a series of other teststeps. A script reads definitions and features together and produces output of plan features in which all the definitions have been expanded. As an example, look at the following feature file containing a defnition:
This will be expanded into:
Of course it's possible to put your definitions into separate files and in fact we recommend you do this. For instance, we usually have a definitions file accompanying each feature file, plus a separate definitions file for common definitions. In fact, as the name suggest, we consider Gherkin Definitions to be step definitions in their own right, but instead of being forced to know some other programming language Cucumber has support for, you can keep using the Gherkin format. Keeping features and step definitions separate makes sense regardless of what language the step definitions are written in.
We believe Gherkin Definitions gives us the best of both worlds: We can use generic libraries of test steps, like Minosse for testing REST API's, that give testers a great deal of autonomy in writing test scenarios. And thanks to Gherkin Definitions, this doesn't come at the cost of making those scenarios harder to understand.
A long time ago, when we started writing automated integration tests at IceMobile, we had a workflow that looked something like this: A tester would write down some test scenario's for a piece of functionality in a text document. Then a developer would take that document and code the testcases described there. We soon shifted to using Cucumber.js, but the process remained more or less the same: The tester would write a feature-file and then the developer would implement the necessary steps. It was a pretty time consuming process and we ended up writing quite a number of very similar teststeps.
When we started working on a new large project, consisting of a number of rest API's, we saw our chance to make a change. We decided to flip the process on its head. Instead of starting with writing features, we'd start by writing a library of common steps for testing REST API's. Then our testers would use, as much as possible, that library of existing steps. We named the library Minosse and open sourced it on Github.
As an example, let's take a look at a hypothetical API for getting data on mountain peaks:
It is true that by limiting ourselves to a predefined set of steps, we limit the expressiveness of our feature files. We weren't attempting to do BDD so this wasn't a deal-killer for us. Nonetheless we did build another tool which allows us to full flexibility in writing very clear feature files, while still using Minosse steps to power them. If you are interested, stay tuned to this blog since we will write on this topic in the near future.
Thanks to Minosse, we've become really fast at writing automated integration tests. This approach has been such a success that we're now working on writing similar libraries for testing websites and apps. We're hoping others will find it as useful as we do.
You can Fork Minosse on Github.
Finally, like all the good Hollywood movies, the panel talk I was part of at the NodeSummit 2015 is out, available for everybody! To refresh your memory, I was invited to talk at the keynote panel of the NodeSummit 2015 in San Francisco. Speaking and sharing my experience in designing Node.js applications in enterprise as we do at IceMobile.
An inspiring and instructive conversation with an “elite” of node.js veterans. Click the image below to be directed to the NodeSummit video page. Enjoy!
A month ago, for the second time, we organized the Amsterdam Node Meetup at our office in Amsterdam. The evening was filled with three speakers, beer, pizza's and some 40 enthusiastic Noders. Theme of the evening was Mathematical modeling and performance. Thanks to the speakers for your contribution to the evening!
Find the presentations of the evening via the links below:
Presentation Luca Maraschi, NodeSummit 2015 Recap:
Presentation Jasper Spoel, Process Orchestration, a petri net perspective:
Presentation Jelle Herold, Gulp, Types, Purescript, Reuse:
Did you miss it? No worries we are starting preparations for an April Amsterdam Node Meetup edition so stay tuned! Updates will be announced via our Twitter and Facebook account or just join the AMS NodeMeetup group.
See you in April!
Technology is moving so fast that just 1 release of node makes a huge difference. The NodeSummit 2015 unveiled new features as well as new perspectives how to run Node.js at scale in production. Scale is not just a technology factor but also a social effect happening inside the fences of enterprise. I was very lucky to attend and speak during the keynote panel of the Node Summit conference, sharing the experience we have at IceMobile, moving to a full Node.js environment for our production stack.
Highlights of the Node Summit
- Node.js 0.12 was finally released one year after the original announcement
- The announcement of the Node.js Foundation, where Joyent continues to be an active member of
- The ancient debate of “server vs client” rendering was still alive on stage, having two engineers from PayPal talking on stage about the move from Node.js/Dust to a full client side rendering done via AngularJS (if you want to know more on this subject you should definitely check out the talk of Peter Marton from RisingStack
- The announcement that DTrace and MDB are now available on LX-branded zones on SmartOS, changing the way we can access low level tracing and debugging of Node.JS applications running on *nix environments (you MUST check out the talk from Bryan Cantrill about it)
Learnings of the Node Summit
- Docker was highlighted as the only way to efficiently deploy Node.js applications at scale
- If you are building an API use Restify
- Dtrace and mdb, the two pearls that are going to save you from sleepless nights when something happens in your Node.js (and not only!) production’s application
- Don’t take connection flights if you don’t want to end up in Houston for a day!
All events and interviews are recorded by the Node Summit, as soon as the paneltalk is available online we will share it with you!
Our first hosted Amsterdam Node Meetup is a fact! On Tuesday 16th of December about 40 Noders came to our office in Amsterdam where we had two speakers, the usual beers and of course the techtalk between people. The theme for the evening was Development environments & numbers and was of such interest that we even had someone flying in from Vienna! (Nice having you Matthias, see you at a future occasion!)
Find both presentation on our SlideShare page via the links below:
Presentation Luca Maraschi, the numbers magic:
Presentation Hongli Lai, Phusion Passagner:
Next Amsterdam Node Meetup will be February 24th again at our office in Amsterdam, hope to see you all there!
One of the most important learning from our “migration” from JAVA to node.js, is definitely the acknowledge that in the node ecosystem there is no “golden rule”, but a bunch of guidelines and practices, not unified in a single framework; on the other hand JAVA provides all of these in several frameworks. We soon recognized that one of the key elements for the success of this migration, was providing these “tools” to all our developers, creating a common ground of discussions. Let the hack begin.
Virgilio was born after few sketching sessions on the board, starting defining the key user of it: the developer. From that moment on every design, tweak, feature has been designed with the developer in mind: our main question was always “what as a developer I would like to code…how would it make my life easier?”. One of the problems that we wanted to solve was definitely to distribute linearly and flat all the module dependencies that we have: npm has the bad habit of chain dependencies in a tree structure making the requiring necessary to make use of the modules. We actually wanted and needed something different, like namespaces in .NET or JAVA, distributing functionalities (functions) across different modules and places of the application, keeping as a fundamental principle the stateless nature of these functions: the context is the only medium to distribute informations across all the module, without using any singleton storage or global object to serve this purpose (even if Virgilio itself is a global object...but nothing different than a Restify/Express app instance!). These reasons drove us to conclude that the concept of class and instances of a class were useless to serve our idea, functional and stateless became then the keywords in our design.
We immediately recognized, particularly when moving to node.js, that conforming to lean methodologies was at the core of an effective development: reducing the time to setup a project, ease of change and progressive improvements on it turned to be the keys of an efficient and consistent growth of the application in a structured and distributed manner.
Working daily with node.js and all the modules that can be found in npm, we quickly experienced that there is no uniformed way of designing any of these: if we wanted to be efficient we should have created a pattern to help people to build and consume Virgilio's modules. Like they say "time is money" and the necessity of a way of shelve different functionalities/modules became more clear with the increase number of libraries that we needed to integrate for the development of our products (Bright Shopper, Bright Stamps, Bright Promo, etc.): Virgilio's modules (or simply plugins) are the way we "hack" and add functionalities on top of Virgilio and its existing functionalities.
In an effort to make Virgilio as easy to work with as possible, we started out writing examples demonstrating how we'd like to use it. Then we started building the interface defined in those examples, while the examples themselves turned into our Acceptance Criteria and test suite.
Have a tiny, hackable core
A tiny framework is easy to understand and relatively easy to make really stable. A tiny core, if it is to stay tiny, needs to be very easy to extend. So make it hackable, and think about supporting extensions from the start.
Make it easy to restructure
Creating a small test app should trivial, and scaling from there to a big project should be seamless. So setting up a Virgilio app requires only a single line of code. Its easy to move code around the project and your project configuration is available everywhere. Namespaces can be used to organise the code.
When writing business logic, boilerplate distracts from what's really going on. So we favor the use of promises for the same reason: they allow for very concise asynchronous code.
Virgilio itself has been running beautifully for some time. Our current focus is on extending the plugin ecosystem for Virgilio and defining generic interfaces for certain types of plugins. That should make it possible, for instance, to swap one caching engine for another without any changes in the code.
Also, we're looking into making it possible for Virgilio to communicate across processes, perhaps even machines. This should make it very easy to, for instance, isolate a particularly processor intensive action.
Interested in taking Virgilio to the next level? Virgilio can be found on Github, so please fork us!.
Welcome to this new IceMobile blog focused purely on technology. Here we'll share our perspectives, decisions and challenges regarding the software and technology we create at IceMobile.
But first a little history...
A couple of years back IceMobile decided to make a transition from a mobile agency, building apps for a large variety of brands, to a product development company building white label products for the food retail industry. This was quite a journey, a lot of decisions had to be made, both strategic and technical. We decided to build a digital customer loyalty platform for our Bright Stamps product with Java and Oracle but after our first product implementation discovered there was room for radical improvement. And so our new but already established Java/Oracle focused development organization turned itself, after a few proof of concepts, into a Node development organization. In a little less than a year we managed to transform our technical infrastructure to Node, a great achievement by the development teams! We are very happy where we are now and look confident at the future for this new digital loyalty platform. We couldn’t have done it without Node and especially not without the community that supports and actually makes Node into what it is. So the time came for us to contribute and give back to the Node community.
Our first step towards reaching this goal was open sourcing our code, so we did. (Find out all about Virgilio on http://github.com/icemobilelab/virgilio. More of our stuff will be open sourced soon!).
As a next step being Amsterdam-based company we want to facilitate the further evolution of the Amsterdam Node community, we decided to start this by participating in the Amsterdam Node Meetup. The Amsterdam Node Meetup group was founded in January 2012 by Bottlenose. Bottlenose is an early adopter of Node and in 2012 already build their whole backend in Node. At that time there existed no local community for this technology in the Netherlands so the guys of Bottlenose decided to start the Amsterdam Node Meetup group. Organizing around 8 Meetups, the last event in this series was the Node.js on the Road organized together with Joyent.
After this last event things got a little bit quiet... A nice opportunity for us to step in and host a number of events starting with the Node Meetup Amsterdam last December at our office in Amsterdam. One of the major steps we see in contributing back to a community of which we benefited so greatly the last year.
The next step in this sequence is this blog. We intend to use this blog to share the details of our approach, the technical challenges we face, the software and technology we create, the events we visit or host, and presentations we give. We envision it to be a tool for prospective employees and fellow engineers in our industry to understand IceMobile's take on technology issues. We hope we can help you achieve and enjoy your work better as you once did for us!
Thank you for reading and participating in the conversation, hope to see you at the next Amsterdam Node Meetup on February 24th!.
Roelof Blom, IceMobile Technology Director