Level: Introductory Benoit Marchal (bmarchal@pineapplesoft.com), Consultant, Pineapplesoft
09 Sep 2003 While excellent solutions are available for large corporations that want to implement XML, few solutions exist for smaller organizations. In this article, Benoit Marchal launches a new project for the Working XML column: an XML client for e-commerce, born out of his experience with B2B e-commerce over the last couple of years. Share your thoughts on this article with the author and other readers in the accompanying discussion forum. (You can also click Discuss at the top or bottom of the article to access the forum.)
I have spent a significant part of the last 10 years working on electronic commerce projects. I originally worked on a technology known as Electronic Data Interchange (EDI), graduated to online shops, and am now working on XML and Web services. Yet the most significant shift I've witnessed is not the technology but the expectations.
Ten years ago, an electronic commerce project was a major undertaking. I am referring to a time before the Internet was widespread and the projects depended on so-called value-added networks (for instance, X.25 or X.400), unique file formats (EDI formats such as X12 and UN/EDIFACT), and very specialized software. With so many custom-designed components, it's no wonder the projects were expensive, slow, and difficult.
Today, the technology has matured, which has led to different expectations. Nearly everyone has access to a cheap network (the Internet), fewer file formats (such as HTML and XML) ensure worldwide compatibility, and plenty of cheap software helps you build and deploy Web applications. It's only natural to expect electronic commerce projects to be painless, fast, and cheap.
By and large, that's true for online shops. Mom and pop shops turn to Yahoo! Store or eBay, while larger businesses buy packaged solutions such as IBM WebSphere Commerce (see Resources).
Only in the business-to-business (B2B) field -- the direct connections between business partners -- do expectations still fall short. Those projects still tend to be fairly expensive and difficult to pull off. In this new series for the Working XML column, I want to reflect on experiences from the last couple of years and highlight promising solutions.
Some context
The major difference between a typical B2B project and a typical online store project is the need for integration on both ends. An online store is an interactive piece of software where the shopper clicks his way through the purchasing process. Along the way, he has to provide a lot of information -- including name, address, and delivery instructions -- by filling in forms. This works well because, while a store may have many customers, each one usually shops infrequently.
In most B2B scenarios the transactions are more frequent (it is common for a company to send several dozen orders to a supplier every day) so you want to automate the transactions as much as possible. For example, instead of asking the user to fill in forms or click on choices, the software should make the decision automatically based on a local database. Automating streamlines the process, reduces the workload (allowing one employee to do more work), and reduces the risk of errors (typing mistakes are common). The challenge is to achieve this level of automation cheaply and effectively.
Here are just a few examples of how businesses can benefit from electronic B2B transactions:
- Electronic catalogs can save a lot of money because they give buyers a more integrated view of the products. For example, they may allow buyers to select a cheaper but less known product. As a prerequisite, the suppliers
must agree to provide electronic copies of their product lists and update them regularly.
- Direct shipping allows retailers to close many warehouses (saving on stock and real estate) by asking suppliers to ship
directly to the final user. However, this increases the number of orders tremendously (instead of one order for 100 boxes sent to the warehouse, there are now 100 orders for 1 box sent to different addresses) so it makes sense to automate them.
- Real-time delivery information wins new customers and is a strong competitive argument. Yet it may require
pulling information from different systems, such as the company-owned warehouse and the carrier tracking system. This is impossible to do in real-time unless fully automated.
- Banking is such an essential part of any business that it makes a lot of sense to download the statements straight to an accounting package.
Two alternatives
Most, if not all, B2B projects are initiated by large organizations; that's where the most transactions are, so that's where it pays the most to automate. Yet, since most of their business partners are small or medium-sized organizations, they have to find ways to interface with partners of different sizes.
Integration solutions broadly fall into two categories: Either they provide some sort of data-entry package (a Web site or a specialized client) and ask the partners to key in all the data, or they provide a simple solution for the supplier to integrate with.
The first option (a data-entry package) is the most popular one because it builds on familiar technology. Indeed, a simple Web site will do, and you can probably build one with the tools you've used for the online store. It's cheap and easy to deploy quickly. Yet it is not a perfect solution for at least three reasons:
-
It shifts the burden onto the partners. Most have an accounting package (such as QuickBooks or Peachtree) where they keep a record of the transactions. It makes little sense for them to enter the same data into a different system. Few operations like the
double-duty and they frequently renegotiate prices accordingly.
-
It's slow. Partners will update their own accounting packages promptly, but they may delay updating the online system until a quiet period of the day or week, which could cost them money.
-
It's error-prone. It's easy to make a typing mistake, especially when you're blindly copying data from one system to another.
In short, a data-entry package negates most of the benefits of a B2B electronic relationship: It may affect prices, it may slow things down, and it may cause more errors.
A more promising solution is to take advantage of the accounting package database, which holds a copy of the transactions. By integrating with the package, it is possible to retrieve the information and automate much of the transaction. However, few small and medium-sized organizations have the expertise for this integration, so it's up to the larger partner to provide assistance.
Project notes and reality check
One of the most interesting aspects of these projects is the difference in culture between different-sized organizations. This is particularly reflected in the different tools that they use. Large organizations have dedicated IT staff and can afford to invest in custom-built software. Smaller corporations tend to buy off-the-shelf packages and their IT staff may be limited to network administrators. Large organizations also tend to follow rigid procedures, while smaller organizations are more flexible and tend to select software packages that are not so rigid.
It is with those thoughts in mind that I was initially interested in XML. In 1997, the XML/EDI Group promised to explore ways that XML could deliver B2B solutions for small- and medium-sized organizations.
As I said earlier, the reality check is that B2B projects remain somewhat painful and expensive. It is not trivial to interface with an accounting package. Still, by harnessing open source projects, you can succeed. This new series is based on my experiences over the last couple of years.
Keep it simple . . . sweetie
Although integrating with an accounting package is the most attractive solution, it is not very popular with IT staff because it has a reputation for being difficult and expensive. They know it takes an integration server, such as IBM Branch Transformation Toolkit for WebSphere, and a fair amount of consulting to lift it off the ground. Integration is a perfect solution for a large corporation, but it is rarely viable for smaller partners.
Those smaller partners need a middle ground solution. A tool that is simpler than an integration server, but less taxing than a data-entry package. The middle ground is an XML client -- a simple tool for preparing XML messages as automatically as possible, which you can then forward to the integration server.
The difference is in these four words: as automatically as possible. A server is designed for 24/7 availability and to process thousands of transactions unattended. It offers advanced logging, user and application partitioning, and clustering (maybe) -- perfect if you have thousands of transactions per hour; otherwise, it's overkill.
In my experience, it is worth automating even for just a few hundred transactions per day or less (sometimes much less), but you have to use the right tools. With low volumes, it is perfectly reasonable to ask the user to initiate and monitor the transactions if it reduces the costs. Also, the XML client is as automated as possible, but it is not unreasonable to expect the user to provide a missing piece of information.
Such an XML client is in a completely different league from an integration server. My experience is that an XML client complements but does not compete with integration servers.
Requirement analysis
Where do you find such an XML client? Unfortunately, there are no packaged solutions that I know of. A few good kits are available, but since you have to develop anyway, why not build your own client with open-source components, such as an XSL processor and a SOAP library?
The specifications for the XML clients are:
-
Message preparation: Prepares an XML message from data exported by a standard accounting package and vice-versa. This is the heart of the XML client: It prepares XML messages from data exported by an accounting package. Conversely, it feeds the accounting package with XML messages.
-
Server interface: Sends and receives messages to and from an integration server through a
standard XML protocol, such as SOAP. Again, the client is complementary to the server. It is designed for those partners who have no need for a full-blown integration server.
-
Ease of use: If the user is familiar with off-the-shelf packages, the XML client should look familiar. An
e-mail client is a good metaphor for the user interface.
-
Provides lots of user information: B2B e-commerce is new for most users so they need to build trust into such a system. My experience is that the best approach is to provide lots of information.
-
Good logging: Almost by definition, e-commerce relationships are not limited geographically, so you need great logging to be able to assist a user remotely.
I have deliberately left one piece out of these specifications: the ability to edit messages before routing them to the server. The need arises because most accounting packages were not designed with e-commerce in mind. They may not store all the information needed or they may not be able to export all the information needed (my experience is that some of these low-cost packages are less open than high-end packages).
No solution is perfect when data is missing. The most realistic option is to retrieve as much information as possible from the accounting package and ask for clarifications from the user. Again, this would be unacceptable in a high-end integration server, but it makes practical sense for an XML client.
Although the ideal XML client would include an XML editor, I choose not to include one in the context of this column. I am confident that, in the near future, I will plug an open source XForm component into this project, but at the time of writing, the only good editors are commercial projects. Since I don't want to tie the column to a specific product, I will leave the editor as an exercise for the reader.
Some changes to XI
How should you prepare the XML messages? Not all accounting packages recognize XML, but all have the option to export and import data in a variety of formats. Two of the most popular formats are comma separated values (CSV) and fixed-length fields. If the package does not explicitly offer an export function, look for an integration with Microsoft Office. In most cases, it is implemented through CSV files.
To convert these files in XML, you can use XML Import (XI), a project I developed previously in this column. As you might remember, XI uses regular expressions to parse text documents and import them to XML.
What about export? <xsl:output method="text"/> is too limited for practical use. A better solution is to add a text generator to XI. The text generator uses a special XML vocabulary to describe the textual document. I have defined only three tags:
-
flat:root is the root of a text document
-
flat:field represents one text field
-
flat:br represents the end-of-line character on the current system (\n under UNIX, \n\r under Windows)
flat:field accepts the following attributes:
-
width represents the width of a field in characters; the text will be truncated or padded up to the width
-
align indicates whether the text should be padded on the left or the right
-
padding sets the padding character
You can set the default values for these attributes on the flat:root element through the default-width, default-align, and default-padding attributes.
To invoke the text generator from XI, you need to set the output method to xi:flat, as in <xsl:output method="xi:flat"/>. The generator (as well as updates to the XI user interface that uses it) are available for download.
While I was working on XI, I abstracted the regular expression processor so that it is no longer limited to the JDK. A small benefit is that XI now runs under JDK 1.3.
Till next time
This article marks the beginning of a new project for the "Working XML" column. The goal is to write an XML client around the existing XI engine. Take a moment to share your thoughts on the project in the Working XML forum (click
Discuss at the top or bottom of the article to access the forum).
Resources
About the author
Rate this page
| 
