Integrating Spring AI with LLMs — An Enterprise E-Commerce Example

Introduction

The rise of Large Language Models (LLMs) has fundamentally changed how we think about building enterprise applications. From intelligent search to conversational customer support, LLMs unlock capabilities that were once relegated to research labs. But integrating these powerful models into a production-grade Spring Boot application has historically been a fragmented, boilerplate-heavy affair.

Spring AI changes this. It provides a first-class, Spring-idiomatic abstraction layer for connecting to any LLM — OpenAI, Anthropic Claude, Google Gemini, Ollama (local models), Amazon Bedrock, Azure OpenAI, and more — with a consistent API. Think of it as what Spring Data JPA did for databases, but for AI models.

In this post, we’ll build an enterprise-grade e-commerce AI assistant that can:

1. Recommend products based on natural language queries
2. Answer customer questions about orders, returns, and policies
3. Use Function Calling to look up real-time order status from backend services

All powered by Spring AI and swappable across any LLM provider.

---

Why Spring AI?

Before we dive in, let’s understand why Spring AI is the right choice for enterprise Java teams:

Feature	Spring AI	Raw SDK (e.g., OpenAI Java SDK)
Provider Portability	Swap LLM providers with config changes	Tightly coupled to one vendor
Function Calling	Declarative @Bean registration	Manual JSON schema wiring
RAG Support	Built-in VectorStore + DocumentReader	Build from scratch
Observability	Micrometer metrics out of the box	Manual instrumentation
Prompt Management	Template-based with PromptTemplate	String concatenation
Spring Ecosystem	Integrates with Security, Data, Web	Standalone

The key design principle: code to the ChatModel interface, not to a vendor.

---

Architecture Overview

Here’s what we’re building:

┌──────────────────────────────────────────────────┐
│                  Frontend (React / Angular)       │
│            Customer asks: "Where is my order?"    │
└─────────────────────┬────────────────────────────┘
                      │ REST API
                      ▼
┌──────────────────────────────────────────────────┐
│              Spring Boot Application              │
│                                                   │
│  ┌─────────────┐  ┌────────────────────────────┐ │
│  │ ChatController│ │     EcommerceAssistant      │ │
│  │  (REST API)  │──▶  (Service + Prompt Logic)  │ │
│  └─────────────┘  └────────────┬───────────────┘ │
│                                │                  │
│                     ┌──────────▼──────────┐       │
│                     │    Spring AI         │       │
│                     │   ChatModel          │       │
│                     │  (OpenAI / Claude /  │       │
│                     │   Gemini / Ollama)   │       │
│                     └──────────┬──────────┘       │
│                                │                  │
│              ┌─────────────────┼──────────────┐   │
│              ▼                 ▼               ▼  │
│     ┌──────────────┐ ┌──────────────┐ ┌────────┐ │
│     │ OrderService  │ │ ProductService│ │VectorDB│ │
│     │ (Function     │ │ (Function     │ │ (RAG)  │ │
│     │  Calling)     │ │  Calling)     │ │        │ │
│     └──────────────┘ └──────────────┘ └────────┘ │
└──────────────────────────────────────────────────┘

---

Step 1: Project Setup

Start with Spring Initializr and add the following dependencies:

<dependencies>
    <!-- Spring AI Core -->
    <dependency>
        <groupId>org.springframework.ai</groupId>
        <artifactId>spring-ai-starter-model-openai</artifactId>
    </dependency>

    <!-- Spring Web -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>

    <!-- Spring Data JPA (for product/order data) -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-jpa</artifactId>
    </dependency>

    <dependency>
        <groupId>com.h2database</groupId>
        <artifactId>h2</artifactId>
        <scope>runtime</scope>
    </dependency>
</dependencies>

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-bom</artifactId>
            <version>1.0.0</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

Swapping LLM Providers: To use Anthropic Claude instead of OpenAI, simply replace spring-ai-starter-model-openai with spring-ai-starter-model-anthropic. The ChatModel interface stays identical.

---

Step 2: Configuration

# application.yml
spring:
  ai:
    openai:
      api-key: ${OPENAI_API_KEY}
      chat:
        options:
          model: gpt-4o
          temperature: 0.3   # Lower = more deterministic for e-commerce

For Ollama (local LLM — zero cost, full privacy):

spring:
  ai:
    ollama:
      base-url: http://localhost:11434
      chat:
        options:
          model: llama3
          temperature: 0.3

The beauty of Spring AI: your Java code doesn’t change — only the YAML configuration and the starter dependency.

---

Step 3: Domain Models

Let’s define our e-commerce domain:

@Entity
public class Product {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;
    private String name;
    private String description;
    private String category;
    private BigDecimal price;
    private Integer stockQuantity;
    private Double rating;
    
    // Constructors, getters, setters omitted for brevity
}

@Entity
public class Order {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;
    private String customerEmail;
    private String status;          // PLACED, SHIPPED, DELIVERED, RETURNED
    private String trackingNumber;
    private LocalDateTime orderDate;
    private LocalDateTime estimatedDelivery;
    
    @OneToMany(cascade = CascadeType.ALL)
    private List<OrderItem> items;
}

@Entity
public class OrderItem {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;
    private String productName;
    private Integer quantity;
    private BigDecimal unitPrice;
}

---

Step 4: The AI-Powered E-Commerce Assistant

This is the heart of our application. We create a service that uses Spring AI’s ChatClient — a fluent API over the ChatModel — to handle customer interactions.

@Service
public class EcommerceAssistantService {

    private final ChatClient chatClient;

    public EcommerceAssistantService(ChatClient.Builder chatClientBuilder) {
        this.chatClient = chatClientBuilder
            .defaultSystem("""
                You are an AI shopping assistant for "NovaMart", a premium 
                e-commerce platform. Your responsibilities:
                
                1. Help customers find products based on their needs
                2. Answer questions about orders, shipping, and returns
                3. Provide accurate pricing and availability information
                4. Follow NovaMart's return policy: 30-day returns for 
                   unused items with original packaging
                
                Always be professional, concise, and helpful. If you don't 
                know something, say so honestly. Never fabricate order 
                statuses or tracking numbers — use the provided functions 
                to look up real data.
                
                When recommending products, consider the customer's budget 
                and preferences. Format product recommendations clearly 
                with name, price, rating, and a brief reason for the 
                recommendation.
            """)
            .defaultFunctions("getOrderStatus", "searchProducts", 
                              "getReturnPolicy")
            .build();
    }

    public String chat(String userMessage) {
        return chatClient.prompt()
            .user(userMessage)
            .call()
            .content();
    }

    public Flux<String> chatStream(String userMessage) {
        return chatClient.prompt()
            .user(userMessage)
            .stream()
            .content();
    }
}

Notice how clean this is. The ChatClient.Builder is auto-configured by Spring AI based on which starter you included. The .defaultFunctions() call registers the tools the LLM can invoke — we define those next.

---

Step 5: Function Calling — The Enterprise Game Changer

Function Calling (also called Tool Use) is what elevates LLM integration from a chatbot to an enterprise agent. Instead of hallucinating order statuses, the LLM calls your actual backend services.

@Configuration
public class AiFunctionConfig {

    @Bean
    @Description("Look up the current status and tracking information " +
                 "for a customer order by order ID or email address")
    public Function<OrderStatusRequest, OrderStatusResponse> getOrderStatus(
            OrderRepository orderRepository) {
        
        return request -> {
            Order order = orderRepository
                .findByIdOrCustomerEmail(request.orderId(), request.email())
                .orElseThrow(() -> new OrderNotFoundException(
                    "Order not found"));
            
            return new OrderStatusResponse(
                order.getId(),
                order.getStatus(),
                order.getTrackingNumber(),
                order.getEstimatedDelivery(),
                order.getItems().stream()
                    .map(item -> item.getProductName() 
                         + " x" + item.getQuantity())
                    .toList()
            );
        };
    }

    @Bean
    @Description("Search for products by name, category, or description. " +
                 "Can filter by price range and minimum rating.")
    public Function<ProductSearchRequest, List<ProductSummary>> searchProducts(
            ProductRepository productRepository) {
        
        return request -> {
            List<Product> results = productRepository
                .searchProducts(
                    request.query(),
                    request.category(),
                    request.minPrice(),
                    request.maxPrice(),
                    request.minRating()
                );
            
            return results.stream()
                .map(p -> new ProductSummary(
                    p.getName(),
                    p.getDescription(),
                    p.getPrice(),
                    p.getRating(),
                    p.getStockQuantity() > 0 ? "In Stock" : "Out of Stock"
                ))
                .limit(5)
                .toList();
        };
    }

    @Bean
    @Description("Get the return and refund policy details for NovaMart")
    public Function<PolicyRequest, PolicyResponse> getReturnPolicy() {
        return request -> new PolicyResponse(
            "30-day return window for unused items",
            "Items must be in original packaging",
            "Refunds processed within 5-7 business days",
            "Free return shipping for defective items",
            "Contact support@novamart.com for return authorization"
        );
    }
}

How it works under the hood:

1. Customer asks: “Where is my order #1042?”
2. Spring AI sends the message + function schemas to the LLM
3. The LLM responds with a function call: getOrderStatus({orderId: 1042})
4. Spring AI intercepts this, invokes your @Bean, and sends the result back to the LLM
5. The LLM formats a natural language response using the real data

The request/response records:

public record OrderStatusRequest(Long orderId, String email) {}

public record OrderStatusResponse(
    Long orderId,
    String status,
    String trackingNumber,
    LocalDateTime estimatedDelivery,
    List<String> items
) {}

public record ProductSearchRequest(
    String query,
    String category,
    BigDecimal minPrice,
    BigDecimal maxPrice,
    Double minRating
) {}

public record ProductSummary(
    String name,
    String description,
    BigDecimal price,
    Double rating,
    String availability
) {}

public record PolicyRequest(String topic) {}

public record PolicyResponse(
    String returnWindow,
    String condition,
    String refundTimeline,
    String shippingPolicy,
    String contactInfo
) {}

---

Step 6: REST Controller

@RestController
@RequestMapping("/api/chat")
@CrossOrigin
public class ChatController {

    private final EcommerceAssistantService assistantService;

    public ChatController(EcommerceAssistantService assistantService) {
        this.assistantService = assistantService;
    }

    @PostMapping
    public ResponseEntity<ChatResponse> chat(
            @RequestBody ChatRequest request) {
        String response = assistantService.chat(request.message());
        return ResponseEntity.ok(new ChatResponse(response));
    }

    @PostMapping(value = "/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
    public Flux<String> chatStream(@RequestBody ChatRequest request) {
        return assistantService.chatStream(request.message());
    }
}

public record ChatRequest(String message) {}
public record ChatResponse(String reply) {}

---

Step 7: Putting It All Together — Real Conversations

Let’s see the assistant in action with realistic e-commerce interactions:

Scenario 1: Product Discovery

Customer: "I'm looking for wireless headphones under $100 
           with good noise cancellation for my daily commute"

AI Assistant: Here are my top recommendations from NovaMart:

1. **SoundCore Pro X** — $79.99 ⭐ 4.6
   Excellent active noise cancellation with 30-hour battery life. 
   Perfect for daily commutes. ✅ In Stock

2. **AudioPeak NC-200** — $89.99 ⭐ 4.4
   Premium ANC with transparency mode. Lightweight and foldable 
   for portability. ✅ In Stock

3. **BassWave Quiet** — $59.99 ⭐ 4.2
   Budget-friendly option with decent noise cancellation and 
   20-hour battery. Great value. ✅ In Stock

Would you like more details on any of these, or shall I help 
you with something else?

The LLM internally called searchProducts({query: "wireless headphones noise cancellation", maxPrice: 100.00, minRating: 4.0}) and formatted the real database results.

Scenario 2: Order Tracking

Customer: "Can you check the status of order #1042?"

AI Assistant: Here's the status of your order #1042:

📦 **Status**: Shipped
🚚 **Tracking**: FX-9847201834
📅 **Estimated Delivery**: July 9, 2026

**Items in this order:**
- SoundCore Pro X Headphones x1
- USB-C Charging Cable x2

Your package is currently in transit. You can track it in 
real-time using the tracking number above at fedex.com. 
Is there anything else I can help with?

Scenario 3: Return Policy Inquiry

Customer: "I received a damaged item. What's the return process?"

AI Assistant: I'm sorry to hear about the damaged item! Here's 
how NovaMart handles returns:

🔄 **Return Window**: 30 days from delivery
📋 **Condition**: For damaged/defective items, no need for original 
   packaging
🚚 **Shipping**: Free return shipping for defective items
💰 **Refund**: Processed within 5-7 business days after we receive 
   the item

**Next Steps:**
1. Email support@novamart.com with your order number and photos 
   of the damage
2. You'll receive a prepaid return label within 24 hours
3. Ship the item back and your refund will be processed automatically

Would you like me to look up your order to get the details started?

---

Swapping LLM Providers

One of Spring AI’s most powerful features is provider portability. Here’s how easy it is to switch:

To use Anthropic Claude:

<!-- Replace OpenAI starter with Anthropic -->
<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-model-anthropic</artifactId>
</dependency>

spring:
  ai:
    anthropic:
      api-key: ${ANTHROPIC_API_KEY}
      chat:
        options:
          model: claude-sonnet-4-20250514

To use Google Gemini:

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-model-vertex-ai</artifactId>
</dependency>

To use Ollama (fully local, no API costs):

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-model-ollama</artifactId>
</dependency>

Zero Java code changes required. Your EcommerceAssistantService, function definitions, and controllers remain identical.

---

Enterprise Considerations

For production deployment, consider these additional patterns:

1. Conversation Memory

public EcommerceAssistantService(ChatClient.Builder builder, 
                                  ChatMemory chatMemory) {
    this.chatClient = builder
        .defaultSystem("...")
        .defaultAdvisors(new MessageChatMemoryAdvisor(chatMemory))
        .defaultFunctions("getOrderStatus", "searchProducts")
        .build();
}

This enables multi-turn conversations where the LLM remembers context like “I mentioned headphones earlier — show me the cheapest one”.

2. Guardrails & Content Filtering

this.chatClient = builder
    .defaultSystem("...")
    .defaultAdvisors(
        new SafeGuardAdvisor(List.of(
            "competitor pricing",
            "internal employee information",
            "credit card numbers"
        ))
    )
    .build();

3. Observability

Spring AI integrates with Micrometer out of the box:

management:
  endpoints:
    web:
      exposure:
        include: health, metrics, prometheus
  metrics:
    tags:
      application: novamart-ai-assistant

Monitor token usage, latency, error rates, and function call frequency — critical for cost management with LLM APIs.

4. Rate Limiting & Cost Control

@Bean
public ChatClient.Builder chatClientBuilder(OpenAiChatModel chatModel) {
    return ChatClient.builder(chatModel)
        .defaultOptions(ChatOptions.builder()
            .maxTokens(1024)         // Cap response length
            .temperature(0.3)        // Reduce randomness
            .build());
}

---

Testing

Spring AI makes testing straightforward:

@SpringBootTest
class EcommerceAssistantServiceTest {

    @Autowired
    private EcommerceAssistantService assistantService;

    @Test
    void shouldRecommendProducts() {
        String response = assistantService.chat(
            "Show me laptops under $1000"
        );
        
        assertThat(response).isNotBlank();
        assertThat(response.toLowerCase()).contains("laptop");
    }

    @Test
    void shouldLookUpOrderStatus() {
        String response = assistantService.chat(
            "What's the status of order #1042?"
        );
        
        assertThat(response).isNotBlank();
        // The function call should return real order data
        assertThat(response).containsAnyOf(
            "Shipped", "Delivered", "Processing"
        );
    }
}

For unit testing without LLM calls, mock the ChatModel:

@MockBean
private ChatModel chatModel;

@BeforeEach
void setup() {
    when(chatModel.call(any(Prompt.class)))
        .thenReturn(new ChatResponse(List.of(
            new Generation("Mocked AI response")
        )));
}

---

Key Takeaways

1. Spring AI abstracts the LLM provider — code to the ChatModel interface and swap providers via configuration
2. Function Calling turns chatbots into agents — the LLM invokes your real backend services instead of hallucinating data
3. The Spring ecosystem advantage — Security, Data, Observability, and Testing all integrate seamlessly
4. Start local with Ollama, validate with a hosted model, and deploy with your enterprise’s preferred provider
5. Prompt engineering is your business logic — the system prompt defines the assistant’s personality, boundaries, and capabilities

Spring AI brings the maturity, testability, and enterprise patterns that Java teams expect — to the world of generative AI. For e-commerce teams, this means building intelligent, production-ready AI features without leaving the Spring ecosystem.

---

The complete source code for this example is structured as a standard Spring Boot application and can be extended with RAG (Retrieval-Augmented Generation) using Spring AI’s VectorStore abstraction for product catalog search over embeddings.