AI Shouldn't Book Your Orders Automatically

Imagine this. A customer sends an email with a purchase order. An AI agent reviews the document, identifies the buyer, finds the relevant products, verifies the quantities, and prepares the order. Everything appears to be correct.

The AI then sends the order directly to the enterprise resource planning system.

No human review. No draft. No approval process.

At first, this sounds like the ideal version of no touch or zero touch order processing.

But it also creates an important problem.

What happens if the information processed by the AI cannot be fully trusted?

In B2B eCommerce, automation involves much more than having AI enter orders automatically. Businesses need assurance that AI will not create incorrect orders because it misunderstood the information in a purchase order or was manipulated by the information it received.

This is where a draft checkout becomes important.

Instead of allowing AI to directly finalize the transaction, the system can create a draft checkout containing the extracted and validated order information. The appropriate business user can then review and approve it before the transaction is finalized.

This provides the benefits of automation while keeping an important control point between AI interpretation and the final business transaction.

Zero-Touch Doesn't Mean Zero Controls

There is also a significant difference between:

Eliminating manual data entry is a goal of good automation.

Eliminating human or system controls, however, can create a significant security and business risk.

A purchase order may contain information such as:

• Product description
• Customer part number
• Specific instructions
• Price
• Shipping information
• Comments
• Text embedded within the document
• Scanned content

When an AI agent interprets this information, it needs to determine:

What information represents actual business data?

Are there any instructions within that information that could influence or control the behavior of the AI agent?

This distinction is critical when designing AI based B2B order processing systems. The AI should interpret the document, but it should not be allowed to override the business rules and controls that govern the transaction.

What Is Prompt Injection?

Prompt Injection is a type of attack against an AI system where untrusted data attempts to alter how the AI behaves.

If we look at order processing in a B2B environment, we may have a document containing sentences such as:

A typical document parser would extract that sentence from the document and identify it simply as text.

An AI agent, however, could potentially interpret the same sentence as an instruction rather than as content within the document. Whether this happens depends largely on how the AI agent and its workflow have been configured.

This ultimately represents the core security concern.

The document contains data. Therefore, it must never be granted automatic authority.

Your Purchase Order Is an Untrusted Input

One of the most important principles when designing B2B order automation using artificial intelligence is that any input coming from outside your controlled system should be treated as untrusted input by the AI.

This includes:

• Email attachments
• Customer uploaded PDFs
• Scanned documents
• Free text order notes
• External product descriptions
• Any other customer provided content

The AI can interpret these inputs and use the information to support the transaction. However, these inputs should never have the authority to change the rules governing the transaction on their own.

The AI should interpret the data, while the business systems remain responsible for enforcing the rules, validations, and controls that determine whether a transaction is valid.

The Dangerous Architecture

Consider the following workflow:

In this architecture, the AI has a direct path from input to action.

The AI itself may not be malicious. The problem is the architecture.

By giving the AI a direct path from untrusted input to a business transaction, the system gives the AI too much latitude to interpret information and take action without sufficient controls.

The Safer Architecture: Draft Checkout

Replace:

with:

In both approaches, the hard work is still handled by the AI.

The AI can read the purchase order, identify the customer, extract product information, map SKUs to products, determine the required quantity for each product, calculate the total amount of the proposed transaction, and flag any uncertainty about whether the transaction can be completed.

The key difference is what happens after the AI has completed that work.

Instead of sending the transaction directly to the business system, the AI creates a Draft Checkout.

The draft checkout provides a controlled point where the extracted information can be validated and approved before the transaction becomes a final order.

What a Draft Checkout Should Contain

A draft transaction should show what the AI has understood about the transaction and how it has interpreted the information in the purchase order.

The system can then highlight any important exceptions that require attention:

This allows the AI to do the work of interpreting and preparing the transaction while ensuring that exceptions are clearly identified before the order is finalized.

Draft Checkout Creates a Security Boundary

The main architectural advantage is the separation between the AI system and the transactional system.

This creates a much stronger separation of duties. The AI can interpret information and propose what should happen, but it does not automatically get access to perform whatever action it interprets.

AI Should Never Be the Source of Truth

Your existing ERP and commerce systems already provide authoritative business data, such as:

• Customer account
• Catalog SKU
• Contract pricing
• Inventory levels
• Credit status
• Payment terms
• Shipping addresses
• Tax rules

The AI should never have the authority to replace this data simply because a document states something different.

What Happens When the AI Gets It Wrong?

Let's assume the purchase order states:

The AI then identifies two potential product matches in the catalog:

Instead of the AI selecting one of these products for the salesperson to approve or reject, it should flag the issue:

Once the salesperson selects the appropriate item, the draft order is updated with the correct product. The order can then proceed through the remaining validation and approval steps.

Draft Checkout Isn't Just a Security Feature

A simple example is what happens when a sales representative receives 50 purchase orders.

Before automation, the salesperson is effectively acting as an order entry clerk. After automation, the salesperson becomes an exception manager while continuing to focus on their primary role as a customer-facing seller.

Adobe Commerce and Magento Can Fit This Model

Instead of allowing an AI service to bypass the commerce layer and directly create a completed transaction, the AI service can use the shopping cart, quote, or draft order functionality provided by the commerce system within a controlled environment.

The same approach can be applied when integrating with other commerce platforms and ERP environments.

What About Fully Automated Orders?

There will be situations where it makes sense to completely automate a process.

For example, a consistent customer order with known variables such as the customer, SKU, and price, and with no potential issues, could proceed automatically as long as it passes established business rule validation.

However, this does not give the AI total authority over the order submission process.

There are ways to allow an AI system to execute autonomously while still maintaining control over what happens when certain exceptions occur.

In other words, controlled autonomy.

A Better Definition of Zero-Touch

A mature automation system will still have checks and controls in place. However, the level of human involvement should be conditional on the level of risk and uncertainty.

Build the Approval Logic Around Exceptions

The Architecture Matters More Than the Model

We may be tempted to think that as AI models become smarter, they will also become better at providing secure solutions for enterprises.

While better models can certainly improve accuracy and reliability, there are many other factors to consider beyond how intelligent the model is.

Even the most sophisticated AI models can still fail because of:

• Ambiguous information
• Inaccurate or incorrect source data
• Maliciously designed content
• Unforeseen document structures
• Conflicts within business rules
• Lack of context

The goal is not to assume that AI will always make the right decision. The goal is to design the system so that when AI is wrong, the appropriate controls are already in place to prevent that error from becoming a harmful business transaction.

The Principle: Never Give Interpretation Direct Execution Authority

Each step has a specific function and responsibility:

• Interpret: The incoming data is analyzed and interpreted by the AI.
• Validate: The company's systems verify whether the proposed transaction is permissible based on the established business rules.
• Draft: The proposed transaction is created as a draft that can be viewed and reviewed before it is finalized.
• Approve: The transaction is approved either by the appropriate person or by predefined business rules.
• Execute: The commerce platform or ERP generates the actual transaction.

This separation of responsibilities is how zero-touch automation can be implemented while keeping the transaction under control.

The Future Isn't "AI Places Every Order"

A more realistic future for AI-enabled B2B order processing would look like this:

AI processes all orders. After processing each order, the system determines whether it can be completed automatically or whether there are any risks or exceptions associated with the transaction.

If the order does not present any significant risk and meets all predefined business rules, it can proceed automatically.

If the order presents a risk that cannot be resolved through automated validation, it is escalated for human review.

The goal is not to give AI complete control over business operations. The goal is to reduce the amount of human interaction required based on the level of risk involved.

Final Takeaway

AI agents can dramatically reduce the amount of time spent manually processing orders for B2B customers. However, simply connecting an AI agent directly to a company's ERP system and instructing it to "complete the order" would be an unsafe architectural model.

A safer architectural model:

In this design, the AI does what it is best suited to do: interpreting messily constructed documents and preparing the information needed to create a transaction.

Meanwhile, the eCommerce platform and ERP systems do what they are designed to do: enforce business rules, validate authoritative data, and complete the required transactions.

When an exception or issue requires actual business judgment, the transaction is routed to a human for review and approval.

This is what a controlled Zero-Touch B2B order processing model can look like.