How collapsing the database stack improves developer productivity

Software developers are looking at new generations NoSQL database stacks that provide better correlation between the application coding languages and the database. This reduces coding time for new applications and to adapt an application for new clients. Developers are finding that this approach makes it easier to focus on new algorithms for creating business value rather than tweaking their applications for scalability and performance.

This approach enabled Adfenix to launch a novel new advertising service in Scandinavia in only 4 months with a team of 3 developers, said André Hegge, CEO of Adfenix. The service helps classified advertising services create new business models that leverage social media like Facebook, Twitter, and Instagram. The team leveraged the Starcounter NoSQL platform to implement the database logic directly within the application.

In many cases the rewriting of code turns out to be a successful and stepwise simplification.

Dan Skatov, Starcounter

The average Adfenix customer is one who has a busy site that buys and sells goods, with high traffic volumes being the norm. Products and services are marketed, promoted, advertised and sold on the site, and Adfenix works to match the behavior and characteristics to the individual to deliver promotions both on-site and on social networks like Facebook. This increases conversion rates. The service easily comes up in several hundred million page views per month, and processes thousands of transactions per second.

A new programming style 

Hegge said, "Starcounter makes it easier to do more sophisticated algorithms. I don’t have to worry as much about optimizing the database, which allows us to develop much faster. Previously I used tools like  NHibernate, which uses an additional layer between the application and database. But the complexity of adding applications that connect to NHibernate and the database takes a lot of time. In Starcounter, the classes in the code are database objects. When you are dealing with your classes in the applications, you are simultaneously dealing with the applications."

It took Hegge a few weeks to get used to this new style of programming. But he now finds that it difficult to go back to traditional development paradigms. Hegge said that this mode of programming is based on object oriented techniques, but provides better abstractions between application logic and database logic. He said, “Changes to the application are propagated to the database as well. It is a very intuitive way of developing applications when you get used to it.”

Rethinking the development stack

One of the challenges developers face today lies in navigating the trade-offs between system complexity and scalability. The need for scalability implies multi-tier architectures with different sorts of servers (business logic, web, database, presentation tiers) to process more users and data. In addition developers also need to think about multi-node data redundancy, caches and data grids to increase performance.

This affects time to market for new releases and, in result, the business itself. Dan Skatov, head of development at Starcounter said, "In practice, mix-load (read/write) applications are hard to predictably scale." The decision to use scale-out architecture can lead to drop in performance, reliability and data consistency. This can be difficult to solve just by adding more hardware. Even if a scaled-out solution has no complex logic and heavy computations inside, the amount of required hardware to run it with reasonably low UX response time can be large.

Multi-tier scale-out data-excessive architectures can work very good in many cases like social networks, storing historical data and business analytics. But Skatov said they can have issues with ERP and line-of-business apps. For software, collapsing the stack means eliminated complexity and simplified system architecture coupled with increased flexibility and scalability. For business, it means improved agility, competitiveness and reduced total cost of ownership.

Skatov said that  "Collapsing the Stack means virtualization of data management and business logic tiers, which leads to dramatic increase in simplicity, performance and ownership costs. A code which effectively looks like a single-threaded, non-concurrent, pure-logic plus business-objects application, becomes a full-featured line-of-business application after running it in the platform."

Many approaches to collapsed stacks

Starcounter’s collapsed stack solution supports multiple language platforms (.NET, Node.JS, Java) with its VMDBMS technology.  Other approaches for collapsing the database and development stack include SAP’s S4/HANA in-memory platform and Oracle’s eBusiness line of products. Smaller niche adopters of collapsed stack development  tools include vendors like GridGain and GigaSpaces.

The open-source movement in collapsing the stack is concentrated around client side and client-server. As an example, modern implementations of JSON Patch RFC 6902 specification brings small footprint and high performance communication between server presentations and client views. Web Components and Polymer from Google allow implementing modern, thin and highly customizable clients.

New generation language platforms like the Go Programming Language focus on the back end to create highly-performant applications in a clean. Event based programming like Node.JS aims to improve performance by changing an architecture style that programs are being created with. Skatov said, "We clearly start to notice strong parallels in patterns and practices being evolved inside originally different stacks like Node.JS and .NET. More adoption of declarative style, optimal concurrency models, REST-based communication and writing less code with bigger outcome is a today’s trend in modern software development."

A large base of legacy code can present a challenge to moving to a collapsed stack development paradigm. But Skatov points out that rewriting these applications using better programming models could lead to a reduction in the code. He said, "In many cases the rewriting of code turns out to be majorly a successful and stepwise simplification."

What kinds of tools and practices are you leveraging to collapse the stack used for database applications? Let us know.