Spryker: rethinking e-commerce architecture with Project A

On my quest to find the right technology for my rethought e-commerce architecture I am always happy to see more radical approaches. And this is why I was very excited to hear that Project A is releasing its e-commerce framework Yves & Zed for merchants with very high demands in flexibility and especially pace for driving their business models.

I have already mentioned Yves & Zed in my main architecture post and why I like their architecture approach. By that time this technology was only available for ventures incubated by Project A. This will change very soon.

Spryker is an e-commerce framework, which enables launching and scaling individual e-commerce businesses at a very fast pace. And here is what happened before I’ve actually started to believe this:

To get some brief insights about this project I had a call with Alexander Graf (@supergraf), who is driving the company behind Spryker: Spryker Systems. A couple of weeks after that I sat together with Fabian Wesner (@FabianWesner), CTO of Project A Ventures and the technical brain of Spryker. He provided me with deeper insights into the core of Spryker: the software architecture, the components, source code and the reasoning behind important technology decisions.

The following characteristics of Spryker distinguish it from currently available solutions on the market:

1. Developer friendly core technology instead of endless and mostly useless features

A very important fact to get straight right at the start: Spryker is not supposed to be a plug and play e-commerce software with a lot of ready to use.

Its main purpose is to be a powerful tool and proven base for development teams with the task to build a solution for very ambitious merchants or brands. These merchants and brands have already years of experience in digital commerce and have very well defined expectations and requirements. Usually they are currently stuck on their current technology stack, which consists of heavily customized software products.

Spryker is addressing this exactly by not shipping with a long list of features. Instead it provides a framework and a core architecture with separated components to guide the development team and make it very efficient.

2. Deeply integrated own data warehouse as the single point of truth for business decisions

Most of the products on the market are not covering this essential component at all. This leaves the clients stumbling in the dark when it comes to analysing the efficiency of marketing campaigns and spending their budget with more success in the future. And this is only one example.

Spryker has a data warehouse at its heart and it is the single point of truth for all decisions, because it pulls in web analytics data, order processing data (returns, payment defaults, etc.) and offers an easy way for developers to create reliable reports.

3. No page caching for predictable application behaviour instead of cache layers of maintenance hell

The front end of Spryker is build to serve requests very fast by accessing the needed data from fast key-value-storage systems and search engines like ElasticSearch. There are no requests to relational databases or web services. Only when customer data is requested (e.g. at the login) or an order is placed the backend systems are called via an API.

This allow Spryker to scale without whole page or page fragment caching which becomes quickly inefficient when you begin to service customers individually and can be also very painful to maintain.

4. Different and separated application components for the right job instead of all-in-one monsters

Stryker consists of the main software components front end, backend and data warehouse. They are separated and built on different technology to master their designated tasks better than an all-in-one solution.

The front end handles presentation, is easy to extend, has no own data storage and allows very fast release cycles. The backend covers order processing, stores operational data and provides web services for the front end. The data warehouse collects all data about customer engagement, operations and finance and offers reports.

All these components are built on PHP if you exclude software like ElasticSearch or Saiku. This enables the team to work on the complete backbone of the solution for the client.

Summary

Considering the facts above I will definitely keep an eye on the further development of Spryker. After what I have learned about it I’m convinced that development teams can indeed work fast and efficient with Spryker. And without all the overhead merchants do not get stuck in a slow and expensive development process.

For all of you who like to get information about Stryker at first hand and meet the team: there are already announcements for events on http://spryker.com/events/.

Digital Commerce Technology Update – Sep. 2014

Back from my holiday and after catching up on a lot of things I want to give a brief update on what happened in the last month and what I think is very important for rethinking E-Commerce architecture. But before I do that I want to thank all of you for the great feedback and mentions I have received from you about my thoughts. This is very encouraging for me to move forward!

Commercetools was acquired by Rewe Digital

Commercetools is the company behind sphere.io, an e-commerce platform-as-a-service provider, and was recently acquired by Rewe Digital. sphere.io is a very interesting product as it fills the role of the component which stores the essential commerce data and offers a very performant API. This enables to use very lean front end applications which do not have keep essential commerce data redundantly in an own persistence layer. In fact they don’t need a persistence layer at all.

Rewe Digital belongs to the REWE Group, which is a very large German retail group operating in Europe with over 50 billion euro annual turnover. Why REWE thinks it is a good idea to own such technology? I personally think that it was a smart move and will write some deeper thoughts about this soon.

eBay Enterprise announces the Commerce Marketing Platform

eBay Enterprise has introduced its Commerce Marketing Platform to enable merchants to optimize their conversations with individual customers. To do so, the Commerce Marketing Platform offers a number of different applications. This nice marketing video shows, where the journey is heading to: http://vimeo.com/106129707

I’m very excited if the Commerce Marketing Platform services enable us to personalize our conversations with customers on the level shown in the video and if we have to give Ebay our precious data in exchange. As soon as I have deeper insights I will give you an update!

Project A Ventures announces Cryllo

I have mentioned Yves & Zed in my previous post. Yves & Zed is a an e-commerce platform which from what I know more or less follows the four axioms in its core architecture. It was developed by Project A Ventures, a German startup incubator, which has announced to release Yves & Zed under the name Cryllo (the article is in German).

On it’s page Cryllo is listing it’s main advantages and the most important one for me is that it is “customer centered” and sees the catalog and the ERP system as pure suppliers only. Can’t wait to get my hands on it! I’ll keep you posted.

Rethink e-commerce architecture

You haven’t read posts from me since a couple of years. My last post was about the development of Magento under the full control of Ebay and this is over 2 years ago. A lot of things have happened since then: new Magento projects and events like Meet Magento or the Developer Paradise where I always enjoy to meet the Magento community. Also, I enjoy a lot to discuss with e-commerce leaders from retailers and brands about their challenges and goals. Especially retailers with brick-and-mortar stores are facing a very quickly growing competition with strong digital pure players like Amazon. I like to read german e-commerce blogs Exciting Commerce by Jochen Krisch and Kassenzone by Alexander Graf which are covering these challenges quite well.

The question “how to enable my clients to successfully deal with this situation?” has bugged me a lot during these years. I believe that the only way to go is to focus on the customer: his needs, his experience connected to the brand (yes, a retailer is brand too), to surprise him in a positive way and never let him feel lost. I also think, that most of the e-commerce solutions out there are not capable of providing a sufficient technical platform for that!

My experience with online stores can be summed up with these examples: usually the online stores serve the same content for everybody, no matter if this is my first visit or if I am a loyal customer for years. The same content for female customer as for male customers. And let’s not forget mobile: most mobile store are just a view on the content and functionality you see when you use a desktop/laptop or tablet. This means I get the same content when I am walking on the street with a coffee to go in one hand and my smartphone in the other as when I’m sitting comfortably on my couch with my tablet. It’s just fitted to the smaller screen size. It feels soulless.

And since I have been designing and building online stores since nearly 10 years I know why this is the status quo: online shop solutions like Magento, Shopware, Oxid (to name a few) try to serve too much functionality from one solution and are crafted around static data models: category trees, page hierarchies, catalogues. There are more solutions which I don’t know good enough like Hybris or Demandware, but I think they share more or less the same core issues in their architecture approach. They are all violating following axioms I’ve started to believe in stronger and stronger the in last months:

  1. store front ends have to be as lean as possible
  2. essential data like products, customers, orders etc. mustn’t be kept redundantly
  3. all interactions of the customers have to be tracked and stored in one central system
  4. content must be connectable depending on dynamic contexts and not only in static category trees and page hierarchies

Sounds like a lot of theory? Let’s go a little bit deeper:

  1. Lean front ends: to have a lot of flexibility, fast release cycles and no risk of affecting business-critical processes applications which are responsible for presentation of content (products, branding, etc.) must be decoupled from applications which are responsible for processes like order handling, payment processing, risk management, complex price calculation and so on. This is why Project A  has built their own solution Yves & Zed with separate components for front end and backend.
  2. Keeping the essential data in one place and avoiding duplication and synchronization is something which minimizes the effort to build and maintain these processes and also to be able to act and react much faster on new insights instead of waiting for the next nightly full import. This has been a pain in the ass for years for me and usually nothing could be done about it because a lot of the legacy IT infrastructure couldn’t be replaced for political and financial reasons.
  3. By all interactions of the customers with the brand I mean all interactions: every visit and click in every front end application, every visit and purchase in a brick-and-mortar store, every call in the customer service, every post on social platforms like Facebook and Twitter and every return in the fulfillment has to be gathered in one place. Only then the call center agent can serve the customer quickly without asking too many unnerving questions. And only then the front end applications can serve the customer with relevant content and support him to fulfill his needs.
  4. It became already quite complex, but we shouldn’t forget dynamic contexts: we want to address customers according to their current interest and needs. A mother of two kids should see a different version of a homepage of a groceries store than a male college student. I would also be surprised in a positive way when I as a loyal customer receive a free umbrella with my order because the system knows that it will rain in my hometown on the next day. And when the parcel actually arrives on the next day😉

I think that these are the reasons why the successful players are investing huge amounts of money in their own IT infrastructure. Because there is no solution on the market which covers all these requirements. Strong digital pure players are designing and building their own e-commerce architecture which are based more or less on the axioms above. There are data warehouse systems which collect and aggregate customer related data to be able to answer queries like: list me all customers which are interested in running and have shown interest (e.g. visited related products, searched for relevant terms) recently in running wear for cold temperatures because it will be cold in their hometowns in the next months. All of them receive a discount for sports apparel if they are active customers with revenue in the last years. And this is not only sent to them by email, but also presented in the online store. Their front end application are very fast and scalable. They do not use caching. Forget caching: there are no equal pages anymore expect your terms&conditions!

I will designate a lot of my time in the next months to think about e-commerce architecture and to search for suitable software components which follow the axioms and are designed and crafted around the customer needs and not the ERP system!

For all of you who haven’t stopped reading until now: you will find more posts about e-commerce architecture approaches, shop front end solutions, data warehousing and other topics covered in this post in the future. Let’s rethink e-commerce architecture!

Magento and Ebay: 9 months later

It has been nearly 9 months ago since the Magento Developers Paradise 2011 and the announcement that Ebay has acquired 51% of the remaining Magento shares. You might think that 9 months is an unusual timeframe for a recap? Or that some kind of new idea or product has just been born? The answer is no! I’m writing this because I feel that I have to.

At first I’d like to sum up the positive events which happened during this period of time:

X.commerce Innovate Developer Conference 2011

I have attended the X.commerce Innovate Developer Conference 2011 in San Francisco and had a great time meeting new folks but also having a chat with people I haven’t seen for quite a while. I’ve also enjoyed some great and inspiring talks, especially the presentaion of Tom’s Shoes by Blake Mycoskie. Magento had it’s own track with sessions about Magento 2, Magento’s role in X.commerce, development, etc. During the keynote it became clear to me, that Magento will be one of the numerous parts of X.commerce. At this moment I was not sure if this will be positive or negative for the further development of Magento as an ecommerce platform.

The result of the event was that I’ve been planning to have a deeper look on the X.commerce fabric which was made public to download, but I haven’t done this until now!

Magento Developer Certification

A couple of weeks before the Innovate Conference Magento announced the beta phase of the Magento Developer Certification. This is something I was waiting for since 2009. It is a very important instrument to make the quality and development capacity of a solution partner transparent to the public (clients, developers seeking for a place to work with a good team, etc.). It was possible to take part at the exam right on the conference and I’ve received a voucher on site, so I gave it a shot and signed for the Developer Plus exam. The Developer Plus exam covers also the Enterprise Edition modules while the normal Developer Certification focuses on the Community Edition only. The exam was not easy, but working with Magento since the end of 2007 made it possible for me to pass without any preparation.

This accelerated my plan to set up an internal training program for my developers at the company I work for and also for a sibling company. Since last month I’m currently preparing 14 developers to take the Developer Plus exam!

Magento University

Offering training courses for developers is another crucial service to enable and support solution partners to deliver in time and in quality to their clients. I remember the days when you had to figure out everything by yourself by digging the code. And by everything I mean everything: even the architecture and how to write a proper module which doesn’t mess in the core and keeps the solution upgradeable. The lack of such services led to numerous failed projets or messed up solutions which didn’t satisfy the client. All of this is bad for the reputation of Magento.

With the Magento U Magento offers on site courses. The most important one is the course “Fundamentals of Magento Development” with a duration of 5 days. This is the perfect start for any developer who wants to work with Magento. I wish that there was a course like this in 2007!

In addition to the on site courses there are on demand video trainings which are a nice addition to my training program.

Magento Tests Automation Framework

This is the most recent publication: the Magento Tests Automation Framework. It is using PHPUnit to run about 1000 Selenium testcases to ensure if the core functionality of Magento is still working properly along with your customizations. Also it offers you to extend the tests to cover your customizations. Another thing I’ve been waiting for since a long period of time.

The lack of such tool leads to high testing effort: either you build such suite of testcases by yourself or you keep testing your projects manually. Both require high effort and under budget pressure this leads to poorly tested shops and again disappointed customers.

I didn’t have the time to tests the Magento Tests Automation Framework, but if the results will be good I will consider to implement it into our quality assurance process instead of our Selenium smoke test suites which we use at the moment.

Interlude

Sounds pretty good so far, doesn’t it? If there was not one more thing: Magento 2! Magento 2 is a very important project for Magento. It contains crucial architecture refactoring, decoupling of modules, performance enhancements, etc. The list is long and I’ve already written about it. Just releasing some new features, bigfixes and small improvements on indexing and stuff is not enough!

The release date was announced to be the end of 2012, so there is still some time left. My main concern is the lack of transparency. The last time I heard something about Magento 2 was in October 2011 at the Innovate Conference.

Lack of Transparency

Magento announced in June 2011 at the Magento Developers Paradise 2011, that the development of Magento will be performed using the Atlassian software stack which will also give the Magento community a better insight in the progress and development process.

So let’s see, what we’ve got:

I would say that there could be some more transparency than that! At least we still have GitHub: https://github.com/magento/magento2. Unfortunately, the development is not based on GitHub, there are only a couple of merges into GitHub. The last one was a couple of days ago, the commit before that was in the end of December 2011!

In fact, this is the reason why I’m writing this post. I’ve looked into the GitHub project about 4 or 5 days ago and until then the last commit was still from 2011 which concerned me a lot!

Conclusion

I think that Magento should meet its announcement to provide more transparency in the development process of Magento 2. I’m happy to read (after digging around for this information) that the public alpha will be completed in Q1 2012 and will contain the first version of the visual design editor, a modular admin application (so not everything is in Mage_Adminhtml !) and jQuery as the primary JavaScript library! It’s nice to announce that on the Magento Imagine, but you have to remember that the Magento community is not a bunch of salesmen which are happy to hear about the new stuff on conferences.

We are developers and like to be in touch with all the details! With every commit which might contain a fix for an issue reported by us! There are only 6 reported issues on GitHub, I think there could be way more of them if there would be more announcements and information feeds. Of course I know, that the majority of the community is working with Magento 1 and is very busy. But I think that Magento should enable it to deal with Magento 2 much earlier. That’s very important to enable a smooth migration to Magento 2. Which client will want to spend money to implement a store on Magento 1 in one year, when Magento 2 will hopefully be available as a stable release? You have to be prepared for that!

Magento should remember how the development process of Magento 1 was driven by the huge participation of the community which reported thousands of issues before Magento 1.0 was released. This is something which will be crucial for Magento 2 as well! For Magento and the community!

Magento 2 Preview

A couple of days ago I wrote my first blog post about my experience at the Magento Developers Paradise 2011 on Ibiza and promised to dig deeper into the details of Magento 2, which I noted while listening to the talks by Yoav Kutner, Dmitry Soroka and Anton Makarenko. I think that this project is crucial for Magento to keep and develop its most precious asset: the Magento community. The Magento 2 project will show what the acquisition of Magento by Ebay really means for us. The release date is planned for mid/end of 2012.

Transparency

The first thing I want to mention is that Magento wants the development process be more transparent to the community. Therefore Magento will soon provide view access to the development tool stack, which consists of following Atlassian products:

  • Jira – issue and project tracking
  • Confluence – wiki for documentation
  • Bamboo – continuous integration
  • Crucible – code review tool
  • Fisheye – repository browser (thanks for the screenshot by Andreas von Studnitz in his blogpost about Magento 2)

Jira should give us a much better overview of the currently developed release and the planning of further releases. Confluence will provide a wiki with the Magento documentation. Bamboo should show metrics like broken builds ratio, unit tests count and code coverage percentage. Making Crucible public will provide a completely transparent code review process. I can’t really believe that this will be made public, because it is very intimate😉. And Fisheye will make browsing the code and looking on diffs between version smoother than now (actually you can only browse the directory tree in SVN and just look into files).

It was also interesting to hear that the Magento core team consist of 6 developers and is using Scrum.

Software Quality

Magento was and is often critisized that Magento lacks of automated tests and proper documentation. And in deed this leads to unstable releases and code which is hard to understand. At the moment maintaining Magento solutions is more expensive as it should be. The time to qualify a PHP developer to be able to work with Magento is quite long (approx. 6 months which is not only my opinion) and expensive. This makes it harder for Magento solution partners to scale their development capacity. In addition, the lack of automated tests makes it harder to develop and maintain customizations and even the core itself. Clients expect a stable solution and low maintenance costs from their Magento solution partner. Magento 2 is supposed to address these issues:

  • there are already approximately 1900 automated unit, integrity and performance tests for Magento 2
  • they already cover 40-50% of the code
  • the documentation process (inline and also external in Confluence) is implemented in the development process of Magento 2
  • there won’t be features which are released without automated test coverage and documentation
  • Magento tries to cut the time to qualify a developer by 50%
  • Magento considers porting the tests down to Magento 1 (I’m doubtful about this one)

This is nice to hear, but I recommend Magento to fulfil these announcement for Magento 2. I remember when I have visited the Magento office in Kiev for the first time in the mid of June 2009. I’ve discussed lots of topics with the team around Dima, Micha and Anton and we also talked about unit tests.

I'm the guy on the left, Anton, Micha and Dima from Varien from the right (2009)

A couple of weeks after the trip I saw this tweet: http://twitter.com/magento/status/2506514713 and was very excited about this news, but still we don’t have automated tests today.

The external documentation in Confluence is also a very good point. There is no official publication about the Magento architecture in form of a book. The first literature I’ve discovered was a book written by Mark Kimsal in 2008: php/architect’s Guide to E-Commerce Programming with Magento. It covers some basic architecture and digs into certain important modules. This was not really it, but it was better than nothing back then in 2008. In 2010 O’Reilly has published a German book called “Magento – Das Handbuch für Entwickler“, which I have co-authored. It went deeper into the core architecture and provided much more examples of how to apply the knowledge to practical tasks and issues. Unfortunately analyzing the book reviews today show, that people still have issues transferring the theory into real Magento development. Until now I haven’t heard of any other books which go deep into the core of the code. Most of the books concentrate on the installation, configuration and guiding admin users through the Magento Admin Panel functionality for the shop setup and the daily business of a Magento store after the Go Live.

Long story short: I’m very excited to see the official Magento documentation and hope that developers can benefit from it and that it can be used to educate Magento developers.

Frontend

I’ve enjoyed hearing that Magento 2 will throw out Prototype and extJs and will use jQuery only. My team at next commerce recently completed a project which target was to create a frontend theme which is using jQuery only and doesn’t contain any inline Javascript and CSS in the templates. It will be a component of our D2C software core and will be used in future D2C projects.

Magento 2 will have unlimited theme fallback: this means that you can build an unlimited theme inheritance tree which allows avoiding code redundancy in the templates, CSS and JS files completely. In Magento 1 there are only three levels of inheritance.

Example for multiple theme inheritance in Magento 2

A GUI design editor will provide functionality like moving blocks in layouts via drap&drop and it also will be able to export pages as one template file which will consist of all block templates and all states.The designer can edit that template instead of having to perform changes in lots of templates scattered in lots of different modules.

Architecture

Since Magento 2 is first of all a refactoring release, there will be a lot of changes to the architecture to improve scalability, extendability and quality of Magento.

In Magento 2 there won’t be any magic getters and setters. This means that the interfaces to all objects in Magento have to be strictly declared in the code. Also the data model definition will be declared, so that the database is no more the source of the data model definition of objects (this will actually have an impact on EAV). Together with the stricter documentation policy we should have finally a proper API documentation for Magento 2.

Currently, Magento objects like products, orders and customers are contaminated with things like controlling the persistance of their data (load, save, delete). There will be a strict separation of domain models (define which characteristics objects like products have) and service models (perform operations on objects).

If you install two Magento extensions which are rewriting the same model, block, helper, etc. from the core, one of the extensions won’t work, because the first extensions wins when it comes to rewriting. In Magento 1 the only solution for that is to modify one of the modules to build an inheritance chain, so that the modules both work again (of course this only works if the extensions change different characteristics of a class). Magento 2 will allow to configure inheritance chains of models from additional extensions without code changes, which makes it easier to upgrade these extensions. This is a feature which OXID eSales already has.

To be able to disable functionality easier, Magento 2 will have less coupling between modules. Magento 2 introduces components, which consist of tightly coupled modules: Sales (Sales, Checkout), Catalog (Catalog, CatalogIndex, CatalogSearch, …), CMS. These components can be disabled without breaking the rest of the system, so it will be possible to disable Sales to have just a browseable catalog and static content (CMS) or just only static content on the site, if you disable Sales and Catalog.

Modules in Magento 2 will contain all files which belong to the module in one folder: app/code/<pool>/<space>/<model>. Not only PHP classes, but also the templates and skin files will be in this path. The filesystem structure will be rearranged: there will be a folder called “pub”, which will contain all files which should be accessible to the public (jslib, media, skins). The other directories (app, dev, lib, var) should not be delivered by the webserver directly. See a screenshot in Alexander von Studnitz post about Magento 2.

Objects (models, helpers, blocks, etc.) will be instantiated using this pattern: createObject(‘<class name>’) (e.g. createObject(‘Mage_Catalog_Model_Product’)) and not with Mage::getModel(‘catalog/product’). The method is of course still a factory which looks into the configuration for rewrites. Also the getTable-method will be changed: you will have to use getTable(‘catalog_product_entity’) instead of using the alias getTable(‘catalog/product’). The method will return the table name which might have a table prefix.

These changes have an impact on the compatibility of Magento 2 to Magento 1: Magento 1 extensions won’t work with Magento 2. It will be required to migrate extensions to be compatible with Magento 2. Magento will provide a documented migration path for that. I hope there will be a database migration tool, too.

Persistence

Magento 2 will provide support for the RDBMS MySQL, Oracle and MS-SQL.The refactoring of the resource model is already announced for the upcoming CE1.6 and EE1.11 release and will make following changes to the architecture of the persistence layer of Magento:

Refactoring of the resource model layer in Magento 1

The advantage of the refactoring is that the non-RDBMS-specific code is in the Resource Model and everything which is RDBMS-specific is removed and put into a helper for each supported RDBMS. This makes it easier to introduce support for additional RDBMS with nearly no code redundancy. The helpers are only needed when it is required to go beyond the limits of the database abstraction layer provided by Zend_Db for better performance or usage of additional functionality which other RDBMS do not support. Also the install and upgrade scripts will be abstract (upgrade-0.0.1-0.0.2 instead of mysql4-upgrade-0.0.1-0.0.2) and only when it is required, Magento 2 will still support mysql-*, oracle-*, etc. scripts for RDBMS-specific code. This refactoring will be backwards compatible.

A breaking news for me was that Magento is considering to remove EAV and only provide the EAV database schema for the catalog as an optional mode. This would improve the performance significantly, because there will be less JOINs in the queries and therefore less temporary tables which have to be created to sort and filter such results in MySQL. Also Magento would require less indexing which makes data imports and modifications of data faster.

EAV was build in since the start to support the modification of data models for products, categories, orders, quotes, customers and customer addresses without changes to the database schema. So basically it was possible to change the data model of these entities during the runtime of Magento in the Admin Panel. I was involved in a couple of medium- to big-size ecommerce projects on Magento in my career and I can say, that such kind of functionality is not needed and it also can be very dangerous for the stability of the Magento solution. Changes to the data model have to be part of a release and have to go through a proper release process which consists of QA activities such as testing the changes on a staging environment. This requires to perform such kind of changes by using upgrade scripts to ensure that the data model of all entities is the same on all systems (dev, acceptance, staging, production). And since the changes are part of a release and have to be deployed it is not a big deal that there are changes to the database schema.

Everybody who has watched the evolution of Magento 1 has also noticed, that EAV was removed step by step from Magento to gain better performance: first the quotes were stored in flat tables instead of the EAV schema, then the orders followed. Order items were flat right from the start (if I remember correctly) to be able to generate sales reports without too much pain. The flat catalog was introduced to remove the overhead which slowed down the rendering of category and product pages. The flat catalog is a special kind of an index, where each store has its own category and product table.

For me the logical next step is to remove the EAV for customer and customer addresses and last but not least from the catalog.

Web API

Although Magento has often announced to extend the API to cover more functionality, the API is still way behind the possibilities you have in the Admin Panel. Magento 2 will address this issue by including the extension of the API into the development process of new functionality. So all new features will be released together with new API calls covering the new functionality. Also the API will support JSON-RPC and REST in addition to SOAP and XML-RPC which are supported in Magento 1.

Maintenance

There are a couple of things which should improve the maintenance of running Magento sites.

The upgrade process will have a better workflow: the upgrade will be performed on a staging copy of the store (code and database). This staging environment can be used for tests before deploying the upgrade on the production environment. Magento solution partners should all do this already now with own deployment scripts or manually on staging environments. If not: shame on them!

The Magento profiler will output profiling entries in a hierarchy and not as a plain list like in Magento 1. This makes the profiler output easier to read. Also the output format can be configured to support Firebug and other browser plugins.

Wrap Up

At first I was expecting Magento 2 to go with the latest trends in the world of web applications. So called post-modern web applications (e.g. GMail) leave the classical request-response principle for loading complete HTML pages. They are designed like a native client-server application which has a powerful view running on the client and just fetches raw data from the server. I watched a nice presentation on the next09 by Malte Ubl in 2009, who was a Technical Director at SinnerSchrader at that time and now has joined Google in the end of 2010, with the title “Building post-modern Web Applications”. It covers this topic and you can watch it here: http://video.nextconf.eu/video/923175/building-post-modern-web.

Today, I think that the approach of Magento to make a major refactoring release is more pragmatic than changing the architecture of Magento completely. It is possible to release earlier and remove a lot of urgent issues outlined above faster to help the community to deliver better solutions for clients, better extensions for the community and better feedback for Magento. The benefits of this will form a better foundation for a big step to a modern web application design. The big question is how the X. Commerce strategy will influence the evolution and especially the revolution path of Magento. Will Magento be able to perform such big steps being a part of X. Commerce?

Magento Developers Paradise 2011 Wrap Up

Last week on the 4th of June 2011 I’ve started my trip from Hannover to Ibiza to attend the annual Magento Developers Paradise(MDP). I was already looking forward to the event because I have enjoyed my time at the Magento Developer Paradise on Mallorca in October last year. The atmosphere at this kind of locations is very nice and relaxed, everybody is in a good mood and enjoys the sun and free drinks during the breaks and after the talks.

Unfortunately my plane was late, so that I’ve missed my connection to Ibiza, had to spend some hours on the airport in Palma de Mallorca and missed the conference kickoff dinner. But on a MDP this is not a problem: even if you join at 11pm you still meet most of the attendees at the pool bar. So did I and had a chat with Dmitriy Soroka, lead system architect at Magento Inc., Thomas Fleck, CEO of Netresearch and host of the event, and lots of other people. Until 2am I’ve been chatting about Magento topics and what I expect of the upcoming Magento 2.

The next day started with having a breakfast together with Yoav Kutner, CTO of Magento Inc., and some other Magento core developers. After this at 9am Yoav held his keynote about the current status of Magento and what topics will be covered in the upcoming talks. The most interesting was of course Magento 2 which will be released mid/end of 2012. The development has already started, but not all decisions are final. The release will be a major refactoring release which will focus on software quality and performance. To achieve this, a lot of changes to the architecture will be performed, so that Magento 2 won’t be backwards compatible with Magento 1 extensions.

Yoav Kutner is helding the keynote for MDP11

Right after the keynote Andrey Tserkus from Magento Inc. held his presentation “Breaking news: Magento starts running on multiple RDBMS”. This feature will already be available in the releases Magento EE1.11 and CE1.6 next month. Magento will focus on the support of Oracle and MSSQL first, but will add more RDBMS in the future. This feature contains the refactoring of the resource model of the Magento core: in current releases there are only “Mysql4” resource models which contain MySQL specific code but also abstract database access by using Zend_Db functionality which allows using multiple RDBMS. In EE1.11/CE1.6 the resource models will be cleaned up and will only contain non-RDBMS-specific code. The RDBMS-specific code will be moved into resource model helpers. This enables to add support for other RDBMS with less effort since most of the code is in the abstract resource model. Only when it is required to leave the DB abstraction layer e.g. for achieving better performance you need to write code in the resource model helper for the specific RDBMS. The refactoring of the resource model is backwards compatible, so that custom Magento modules still will work using MySQL.

I have also enjoyed the following talks about checkout customization by Vitaliy Korotun (Magento Inc.) and extending Magento hosting capabilities by Philippe Humeau (nbs System). In the afternoon everybody was invited to join a boat trip to Formentera, a small and beautiful island next to Ibiza. We were on board of a catamaran so it was possible to go swimming and we had a delicious dinner on it, too. In the evening we were brought back to our hotel and you could meet plenty of attendees at the pool bar afterwards. Some of them went out to hit the bars and the clubs at night. The nightlife on Ibiza is amazing!

Catamaran trip

The next day had very exciting talks about Magento 2 by Dmitriy Soroka, lead system architect at Magento Inc., and “MVC – model lessons learned” by Anton Makarenko, member of the Magento core team at Magento Inc.. I will only sum up the, in my opinion, most important news on these topics. I will dig deeper into them and write down my thoughts in a follow up post about Magento 2.

  • transparent development process using the atlassian software stack and making that available for the community (continious integration, documentation, issue tracking)
  • automated unit, integrity and performance tests are available for Magento 2; there are already 1900 tests
  • the documentation will be included in the development process, so it is always complete and up to date
  • jQuery will be used in the frontend, no Prototype and no extJs
  • unlimited theme fallback, you can create unlimited theme inheritance trees
  • Magento considers the removal of EAV and only offer it as an optional mode for the catalog
  • domain structural integrity: there will be a strict separation between domain models and service models
  • removal of magic getters and setters enforcing strict data model definition and interface declaration for models
  • configurable inheritance chains to be able to resolve incompatibilities between modules; OXID eSales has this feature already
  • the Core API will be extended in parallel to the development of new features, so it will cover all functions; it will support SOAP, XML-RPC, JSON-RPC and REST
  • less coupling between modules: Magento will introduce components, which consist of modules which are tightly coupled; these components (e.g. Sales, Catalog) will be loosly coupled to the rest of Magento and can be disabled

I didn’t mention lots of things and also was not able to write everything down during the talks, but I think this gives a good overview of all improvements which are planned for Magento 2. If Magento will be successful in realizing them, this will be a huge jump forward in software quality and transparency and this should distinguish Magento from other open source shop platforms.

At 8pm everybody gathered in the lobby for the night event. We went by bus to the historic city of Ibiza and had a guided tour to the fortification and the historic city of Ibiza, called Dalt Vila. After the tour we went to the famous Zoo Bar and had some drinks before getting back to the hotel. Like on the previous day, a large group of people bought club tickets and had an amazing night in the Privilege club.

Great view from the fortification in the historic town of Ibiza

On the last day you could chat to people sitting at the pool bar and spending some time before taking a cab to the airport. You could hear most of them discussing the news about the acquisition of Magento by Ebay and that Magento will be at the core of the new X.Commerce platform. Some people were sceptical, some were optimistic. I’m excited of how this will influence the Magento products and the community.

In the late afternoon I finally took a cab together with the team of Netresearch (we had the same flight from Ibiza to Mallorca) and had a good trip back home.

To sum it up: it was an amazing event with lots of good talks, interesting people and a very nice and relaxed atmosphere for networking. I can recommend this event and already plan to attend next year! I also wanted to thank Alice Patzer, Andrea Panajotowa, Annemarie Tappert and Thomas Fleck for organizing this event and Rico Neitzel for shooting lots of nice photos. You can see them here: http://www.flickr.com/photos/34764379@N04/sets/72157626906928668/