Rethink e-commerce architecture
August 14, 2014 14 Comments
You haven’t read posts from me since a couple of years. My last post was about the development of Magento under the full control of Ebay and this is over 2 years ago. A lot of things have happened since then: new Magento projects and events like Meet Magento or the Developer Paradise where I always enjoy to meet the Magento community. Also, I enjoy a lot to discuss with e-commerce leaders from retailers and brands about their challenges and goals. Especially retailers with brick-and-mortar stores are facing a very quickly growing competition with strong digital pure players like Amazon. I like to read german e-commerce blogs Exciting Commerce by Jochen Krisch and Kassenzone by Alexander Graf which are covering these challenges quite well.
The question “how to enable my clients to successfully deal with this situation?” has bugged me a lot during these years. I believe that the only way to go is to focus on the customer: his needs, his experience connected to the brand (yes, a retailer is brand too), to surprise him in a positive way and never let him feel lost. I also think, that most of the e-commerce solutions out there are not capable of providing a sufficient technical platform for that!
My experience with online stores can be summed up with these examples: usually the online stores serve the same content for everybody, no matter if this is my first visit or if I am a loyal customer for years. The same content for female customer as for male customers. And let’s not forget mobile: most mobile store are just a view on the content and functionality you see when you use a desktop/laptop or tablet. This means I get the same content when I am walking on the street with a coffee to go in one hand and my smartphone in the other as when I’m sitting comfortably on my couch with my tablet. It’s just fitted to the smaller screen size. It feels soulless.
And since I have been designing and building online stores since nearly 10 years I know why this is the status quo: online shop solutions like Magento, Shopware, Oxid (to name a few) try to serve too much functionality from one solution and are crafted around static data models: category trees, page hierarchies, catalogues. There are more solutions which I don’t know good enough like Hybris or Demandware, but I think they share more or less the same core issues in their architecture approach. They are all violating following axioms I’ve started to believe in stronger and stronger the in last months:
- store front ends have to be as lean as possible
- essential data like products, customers, orders etc. mustn’t be kept redundantly
- all interactions of the customers have to be tracked and stored in one central system
- content must be connectable depending on dynamic contexts and not only in static category trees and page hierarchies
Sounds like a lot of theory? Let’s go a little bit deeper:
- Lean front ends: to have a lot of flexibility, fast release cycles and no risk of affecting business-critical processes applications which are responsible for presentation of content (products, branding, etc.) must be decoupled from applications which are responsible for processes like order handling, payment processing, risk management, complex price calculation and so on. This is why Project A has built their own solution Yves & Zed with separate components for front end and backend.
- Keeping the essential data in one place and avoiding duplication and synchronization is something which minimizes the effort to build and maintain these processes and also to be able to act and react much faster on new insights instead of waiting for the next nightly full import. This has been a pain in the ass for years for me and usually nothing could be done about it because a lot of the legacy IT infrastructure couldn’t be replaced for political and financial reasons.
- By all interactions of the customers with the brand I mean all interactions: every visit and click in every front end application, every visit and purchase in a brick-and-mortar store, every call in the customer service, every post on social platforms like Facebook and Twitter and every return in the fulfillment has to be gathered in one place. Only then the call center agent can serve the customer quickly without asking too many unnerving questions. And only then the front end applications can serve the customer with relevant content and support him to fulfill his needs.
- It became already quite complex, but we shouldn’t forget dynamic contexts: we want to address customers according to their current interest and needs. A mother of two kids should see a different version of a homepage of a groceries store than a male college student. I would also be surprised in a positive way when I as a loyal customer receive a free umbrella with my order because the system knows that it will rain in my hometown on the next day. And when the parcel actually arrives on the next day 😉
I think that these are the reasons why the successful players are investing huge amounts of money in their own IT infrastructure. Because there is no solution on the market which covers all these requirements. Strong digital pure players are designing and building their own e-commerce architecture which are based more or less on the axioms above. There are data warehouse systems which collect and aggregate customer related data to be able to answer queries like: list me all customers which are interested in running and have shown interest (e.g. visited related products, searched for relevant terms) recently in running wear for cold temperatures because it will be cold in their hometowns in the next months. All of them receive a discount for sports apparel if they are active customers with revenue in the last years. And this is not only sent to them by email, but also presented in the online store. Their front end application are very fast and scalable. They do not use caching. Forget caching: there are no equal pages anymore expect your terms&conditions!
I will designate a lot of my time in the next months to think about e-commerce architecture and to search for suitable software components which follow the axioms and are designed and crafted around the customer needs and not the ERP system!
For all of you who haven’t stopped reading until now: you will find more posts about e-commerce architecture approaches, shop front end solutions, data warehousing and other topics covered in this post in the future. Let’s rethink e-commerce architecture!
About Dimitri Gatowski
Dimitri Gatowski 
Reblogged this on ecomPunk and commented:
Some interesting e-commerce architecture thoughts from our friend Dimitri!
Pingback: This Week in Shop Tech (TWiST #19) | Shop->Tech->Blog
Reblogged this on ecomhack.com – ecommerce thoughts and commented:
Nice post from Dimi – definitely worth reading!
Pingback: Magento-Neuigkeiten der Wochen 33/34 2014
Since we were already building solutions that follow your axioms with OXID (though using external systems and a tight ERP coupling), I know what you mean. But I see several problems with what you postulate.
1-> no arguing about that. The #1 task in a project should always be to reduce the fat frontend (which has to be fat so every customer sees everything he wants in his shop) to the Frontend the retailer wants.
2-> I know this is against everything you learn in university and it not a clean design, but sometimes redundancy gives you the performance you need to avoid caching. Also this contradicts your mobile approach a bit. You can’t assume that every customer roaming around with a mobile phone always has a connection fast enough to feed the shop frontend (no matter how lean it is). Most of the time you can’t even assume he has a connection at all. If you want those customers in your shop (and hey… you want EVERY customer in your shop that might buy something), you might need to duplicate data to his device and think about synchronizing. Else he will go away. (I do.)
3-> no doubt. But this is not the genuine task of the shop, but of the ERP or CRM system. This is one reason why we never build larger installation without a tight coupling of the shop to one or more of these systems. You were talking about lean and fast systems. So leave the tasks the systems were made for to the specialists. You do not want to build and manage a shop system that also does customer management, callcenter agent support, return handling, be the frontend on the register and so on. In an optimal shop installation those are all interconnected though.
4-> I can only talk for OXID, but categories are not in static trees there. It is more like a dynamic net. And there are (I guess also for other systems) a lot of modules that allow for steering content, offers and all that from customer data (either profile of that customer, a profile comparison to other customers and what they bought or the order history). So the frontend of current systems do not have to be as static as they usually are. The shop system allows that. (Oh and as always: There is a reason that the shop system does not do this out of the box: There are partners that are experts in this on how to do this without scaring away customers, violating laws or invade the customers privacy while keeping up the performance)
Hi Sven,
thanks for your thoughts!
Regarding the redundancy in #2: I know that I can’t get rid of it completely, but you should invest more in avoiding it. And client side redundancy is not such a big issue than server side redundancy since the client has already received personalized content.
Collecting all customer interactions (#3) can’t be handled by shop systems as we know them. I don’t talk about a new “shop system” in my architecture approach, instead I describe criteria for a system landscape. The tightly coupling you are describing is the right direction, e.g. if you use the ERP for order processing and avoid to store orders in the shop system.
I will post some further thoughts along with a diagram to visualize the idea soon.
And last but not least #4: most shop systems allow that, but they don’t encourage it. Especially those which rely on multiple cache layers to be able to scale.
External providers like recommendation engines are not a solution, because they don’t have enough data in one place to solve these complex queries. And they also shouldn’t have it, because we are talking about such sensitive data you don’t want to give away to external parties.
Both (post an the comment) are a great thought-provoking impulse. From my distinct view, as we compare shop software in a technical and functional based detailled manner, I totally agree with Sven, saying:
“There is a reason that the shop system does not do this out of the box: There are partners that are experts in this on how to do this without scaring away customers, violating laws or invade the customers privacy while keeping up the performance)”
But there are other shop systems, which in their target group of online merchants (e.g. below ten million in revenue) pursue another approachs, e.g.: Aggregating essential functionality (e.g. plentymarkets, powergap, etc.) Of course the modular plugin approach of shop systems is promoted these days as the more promising. But clients of these, in their way specialized shop systems, are satisfied in the majority. What I am trying to say in a brief way: Every e-commerce project is (often very) different and thus a large number of specialized shop software has its right to exist.
To come back to the main statement of the above post, my assessment of the current and future evolution of shop systems is slightly different. I don’t see the battle of functions and technique based shop system progression vs. the user focused future ecommerce.
The mentioned scenario of “every user sees the same products” can nowadays be solved buy recommendations engines like yoochoose, as Sven states.
Approaching an visitor depending on he is already a loyal customer or not can also be achieved by specialised third party functionality.
But the different approach concerning the visitors location is certainly one for the future, with google going hunting. Nevertheless, it seems natural, that this demand for tracking the users location within an onlinestore or the storefront will be served by the market for third party services for shop systems WITHIN THE EXISTING SHOPSOFTWARE-ECOSYSTEM (oxid, magento, shopware, speed4trade, plentymarkets, seoshop, etc.) as better shop system search was served by findologic or the demand for recommenations was served by yoochoose (just examples).
I believe there are several companies working on serving these and future demands. On the other side, relatively new ecommerce solutions as sphere.io or shopcloud.io could also put the future ecommerce on an new user focused level.
The ecommerce evolution stays exciting. Looking forward to future posts!
I just estimate the number of specialized shop software at about (number of customers)*(number of target markets) 😉
In my personal opinion the “good enough” approach (having one software that more or less fits the customer needs so that he does not need to adapt more than his logo and some colours) will not really lead to successful shops.
And if so, they will probably be fine with configurator solutions offered by larger hosting companies.
Of course (number of customers)+(number of target markets) ist provoking 😉
“In my personal opinion the “good enough” approach (having one software that more or less fits the customer needs so that he does not need to adapt more than his logo and some colours) will not really lead to successful shops.
And if so, they will probably be fine with configurator solutions offered by larger hosting companies.”
I absolutely agree!
I was rather aiming at solutions with scalable rental paying-models (e.g. seoshop, sphere.io (Service-oriented architecture)), which are or develop towards an ecommerce plattform (modular design) where sophisticated third party plugins can provide the demanded merchants’ needs for quick and cheap extensive custimization.
It is right that our service also compares configurator solutions offered by large hosting companies (or epages reseller shop systems).
But the statement ist a completely different:
Nobody should make the “good enough” approach! But too many merchants are on this track because of poor market review! One of the major goals is to make the lack of functionality, smart backends or automation visible, so the difference between shop systems gets clear and manageable.
Above all, just a view compressed thoughts, no profound essay.
Outsourcing of functionality means that you have to replicate and keep in sync a lot of data. And it also means that you have to give away very precious data, especially customer data. This is more or less the only asset a merchant has nowadays.
I don’t want my business logic and my data to be scattered around at X different companies. That’s pretty ambitious, but that’s the spirit 🙂
Definitely, that’s a major point. Retaining control over customer data is getting more and more difficult.
On the other hand: The “best of breed” discussion and combination of these services in e-commerce projects is still very popular. The future will show the companies main focus.
Pingback: Doch keine Konsequenzen durch die EU-Verbraucherrechte-Richtlinie? [5 Lesetipps]
Pingback: Digital Commerce Technology Update – Sep. 2014 | Dimitri Gatowski
Pingback: Spryker: rethinking e-commerce architecture with Project A | Dimitri Gatowski