From structured processes to chaotic AI ‘processes’

Mostly, this 3-4 decade ‘wave’ helped corporations scale – allowing businesses to grow bigger and bigger. That’s all good but we are now in a new ‘wave’ where the structure of the past is actually crippling, keeping businesses too slow for the pace of innovation we have been experiencing in the past decade.

And, to boot, there is a new generation of models, methods and indeed technologies that are emerging as a new paradigm for businesses in the 21st century – which are made for the new normal.

Structured… and slow

Let me show you with a simple example:

Imagine a manufacturer who developed a new product and needs to buy new raw materials from new suppliers. They need to start placing PO’s. Let’s see what they have to do before they can do that in a structured processing world:

Buyers at the manufacturer have conversations with suppliers, likely via email. Information is gathered about suppliers and materials. That information is not structured – it comes as conversations, which are not columns and rows.

Now, the buyers have to configure that information into structured data: vendor codes, vendor records, material codes and attributes, etc. Then, they have to enter that data into a system through screens with little boxes for each field of data. These fields go into a relational data base, made of tables with data.

Now, they can place a PO. They go into screens with little boxes where they enter what they wish to order in a very specific, prescribed sequence of data fields.

This process – from conversational, unstructured information, to master data, to structured transactions – repeats for everything in that operation: products, product recipes, manufacturing processes, labour and equipment data, customers, etc., etc.

Most companies don’t think about it but a large portion of their staff are doing just this: translating the natural form of information in the real world into a very structured, prescribed collection of fields.

Ok, now – what happens when supply chain managers want to track orders and evaluate the service provided by those new vendors?

The data is structured inside a relational database, right? So, now, somebody has to write a program to produce reports or dashboards. The format of those reports or graphics has to be very prescribed. Supply chain managers have to know exactly in what form they will want to see data about vendors – even though, in the real world, they don’t know exactly what they will need to see every time because they don’t know what problems will emerge. Yet, everybody lives with this state of things and doesn’t even question it.

The following diagram summarizes the current paradigm of structured data and processing.

The most critical element you need to see in this picture are the words “real world”. Before business people can do anything, the real world has to be translated into an internal, very structured picture; and then, very structured data has to be translated back to the real world.

It is this structured zone inside the operation that makes businesses too slow to react and to interact with the volatile reality out there. It is because of all that volatility that businesses need a very different way of operating, in order to compete with nimble start-ups that don’t carry this heavy, slow legacy.

Operating in the real world

I don’t know if you noticed but this entire ‘story’ began with one invention: the relational database. Well, a new paradigm is emerging based on a different form of data: Big Data.

Big Data means: unstructured data in multiple media forms, stored in a data lake.

Let’s say that, instead of going from the real world to a structured world and back to the real world, you could just be in the real world all the time. Imagine the productivity gain! And the speed of doing business with either suppliers or customers. And the flexibility to handle a reality that is changing constantly.

In the Big Data paradigm, data is accepted in its natural form: natural language in text – social media streams, email threads, web pages, etc. – pictures, video and voice. Data lakes are data storage technologies that don’t require the structure of relational databases; and can accommodate very large volumes of data. (Imagine, for instance, the volume of data that comes from sensors attached to manufacturing equipment.)

The companion to Big Data is unstructured analytics and AI. AI platforms can actually make sense of unstructured information in their natural form. NLP (natural language processing) enables technology to interact with the real world without the need for translation into structured data by a human.

Then, there’s Blockchain technology which is basically a distributed database that retains any form of data (you have all heard of NFT’s and smart contracts, right?).

Now think that the world of business operations, of people who work in a business, has changed completely from:

Real world:: stop:: get organized:: respond to the real world;

To;

Real world:: respond to the real world.

Hence, the vast majority of the time, people are reacting to stuff happening in the real world at a very high speed. There’s no time to “stop and get organized”. The world isn’t prescribed or predictable: it’s chaotic.

This basic reality requires two things simultaneously:

• Know what’s happening;
• Know what’s going to happen.

Big Data analytics and AI are made for this. With a combination of good visualization technology and AI models, we now can do two things very well:

• Visualize trends in behaviour and phenomenon – suppliers who are late; where is a PO; are there natural disasters happening in the regions of our suppliers; are there issues with transportation – closed borders, harbours, strikes, blockades;
• Predict: will suppliers be on time; is there going to be a hike in demand; is there going to be a shortage of raw materials; should we expect problems of reliability; how soon should we order; should we accelerate purchasing before there is a strike or because of a natural disaster; etc.

And – both of these two at the same time.

In addition, with a new approach to business that not only accepts but is centred on the volatility of the real world, we can analyze and make predictions across a vast spread of reality, instead of segment by segment.

For example: in classical supply chain management, demand, production and purchases from suppliers are managed in sequence and separately. First you forecast demand; then you plan production; and then you plan suppliers.

In the Big Data/AI paradigm, all of these are just one thing: the supply chain! All these segments are viewed and analyzed together; and the predictions are made across the entire supply chain – not one segment at a time.

It is worth noting that the vast majority of data about the product of one company is not inside that company; it is outside and it is in unstructured forms: in social media streams, emails, web pages of regulatory entities or competitors or client web sites, in financial statements of public companies, news web sites, YouTube channels, etc., etc. That’s where you can find the real world.

In addition, external data is not specific to that company; it is not just about the company and it’s not private or confidential to the latter. The Big Data one may collect about one’s supply chain and market is useful to one’s suppliers and customers. Thus, whatever one comes up with in terms of insights and predictions can be useful to one’s suppliers or customers.

This opens the door to dynamic collaboration across the supply chain, which makes things even faster and more flexible.

So, back to our example:

When Buyers at the Manufacturer talk to suppliers through either voice or (most likely) text, data about suppliers and materials are already there – in that Big Data.

You can pass a PO simply by describing what you want in natural language – again, via email or voice or voice and video.

Natural Language Processing (NLP) tech can actually be used to extract the required structured data from those conversations, structure it and send it to the structured information systems of the 20th century. (It’s not pragmatic to fathom that all the operational IT infrastructure that took 4 decades to build is going to be retired any time soon.)

The best part, though, is this:

• Learning what’s happening with our supply chain – be it PO’s or conditions in general – can be done by inquiring Big Data lakes using NLP.
• AI can help ‘connect dots’ in the data to help with analysis: for instance, suppliers that use parts from plants in Mexico, in the winter, are usually late with their orders. Or – if there is an epidemic in Asia, orders from Europe will be late. Or – if the whether was great on the weekend and it was Canada Day, we’re likely to see quality issues in production over the next week; etc.
• Graph databases can be used on top of data lakes to make searches faster, by storing connections between datasets that are found frequently.
• AI can predict the impact of current events or expected events on our supply plans. AI can basically replace all planning processes.
• AI has the advantage that it doesn’t need fixed parameters to make predictions. It basically changes the ‘assumptions’ in its models as reality changes.

Blockchain technology also has a big part to play in the new world, particularly with supply chains.

Everything that moves in the economy entails a supply chain. Everything. Even in the Service sector. When a Bank manages a mortgage or a financial advisor manages a portfolio, there is a supply chain, even though there are no materials involved. For government to offer services – be it health care, immigration, education or defense – there is a supply chain.

There is a supply chain anytime that, in order to provide a product or a service to a citizen, multiple organizations are involved.

Interactions between organizations are transactions. They’re usually buy or sell transactions – of all kinds of things, from parts to expertise. A blockchain is the perfect place to store and execute those transactions.

The most common object used in blockchains for purposes of inter-organization trades is the smart contract. They’re fantastic because the transactions don’t consist of two (or more) separate parts – one in each of the organizations involved; they exist only once for all parties, in a common, irrefutable instance, inside the blockchain.

Let me summarize:

• The business world is evolving from a stop and go; step 1, 2, 3 operational format; to a real-time model where interactions between organizations and their suppliers and customers occur at the same time as the operations of the providing business move.
• Every business should be investing in their Big Data-AI-Blockchain frontier, rather than more relational database centred software.
• The fragmented picture of roles in an organization – where there are separate roles for separate, specialized steps in a process – needs to be completely replaced by broad, end-to-end roles.
• 20th century organizations with structured hierarchies and bureaucracy, can’t survive this decade. They need to be replaced by Agile models without hierarchies or bureaucracy.
• When you see a business problem, a missing capability, a new challenge in a changing market, a new service you need to create – do not think of an information system or a report as the solution.
• The technology for the new paradigm exists in abundance and is fairly cheap to access. Start-ups of the past 10 years are already using it. It’s easy for a start-up to create a Bank (e.g., Sterling Bank in the UK), a Pharmaceutical company (e.g., Moderna), a payment company (e.g., Stripe or Square) or a Space company (Space X), precisely because anyone can access these technologies at Amazon or Microsoft or IBM, among others.