<dependency>
    <groupId>dev.langchain4j</groupId>
    <artifactId>langchain4j</artifactId>
    <version>0.28.0</version> </dependency>
<dependency>
    <groupId>dev.langchain4j</groupId>
    <artifactId>langchain4j-openai</artifactId>
    <version>0.28.0</version> </dependency>
<dependency>
    <groupId>dev.langchain4j</groupId>
    <artifactId>langchain4j-spring</artifactId>
    <version>0.28.0</version> </dependency>
For Gradle (build.gradle):

Add the following inside the dependencies { ... } block:

Gradle

implementation 'dev.langchain4j:langchain4j:0.28.0' // Check LangChain4j GitHub for latest version
implementation 'dev.langchain4j:langchain4j-openai:0.28.0' // Match core LangChain4j version
// Optional: If you plan to use LangChain4j with Spring Boot specifically
implementation 'dev.langchain4j:langchain4j-spring:0.28.0' // Match core LangChain4j version
Important Note: Always check the official LangChain4j GitHub repository (https://github.com/langchain4j/langchain4j) for the very latest stable version. The version 0.28.0 used here is an example.

3.3. Basic Spring Boot Application Structure
Your main application class will look like this:

Java

package com.javayou.llmjavaapp;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class LlmJavaAppApplication {

    public static void main(String[] args) {
        SpringApplication.run(LlmJavaAppApplication.class, args);
    }
}
4. Building Your First LLM Application with OpenAI and LangChain4j
Now, let's create a simple service that interacts with OpenAI's powerful GPT models.

4.1. Configure OpenAI API Key
First, you need an OpenAI API key. You can get one from https://platform.openai.com/account/api-keys.

It's best practice to store sensitive keys securely, for example, in src/main/resources/application.properties (or application.yml) or as environment variables.

src/main/resources/application.properties:

Properties

openai.api-key=YOUR_OPENAI_API_KEY
Replace YOUR_OPENAI_API_KEY with your actual key.

4.2. Create an AI Service
Let's define a simple Spring component that uses LangChain4j to communicate with OpenAI.

Create a new package, e.g., com.javayou.llmjavaapp.service, and add AIService.java:

Java

package com.javayou.llmjavaapp.service;

import dev.langchain4j.model.openai.OpenAiChatModel;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;

import javax.annotation.PostConstruct; // For Spring Boot 2.x
// import jakarta.annotation.PostConstruct; // For Spring Boot 3.x if using Jakarta EE

@Service
public class AIService {

    @Value("${openai.api-key}")
    private String openaiApiKey;

    private OpenAiChatModel chatModel;

    // Use jakarta.annotation.PostConstruct for Spring Boot 3+ if you face issues
    @PostConstruct
    public void init() {
        chatModel = OpenAiChatModel.builder()
                .apiKey(openaiApiKey)
                .modelName("gpt-4o-mini") // Or "gpt-3.5-turbo", "gpt-4" depending on your needs
                .temperature(0.7)
                .logRequests(true)
                .logResponses(true)
                .build();
    }

    public String chat(String userMessage) {
        // Simple chat interaction
        return chatModel.generate(userMessage);
    }

    public String generateJavaCode(String prompt) {
        // A more specific use case: generating Java code
        String fullPrompt = "Generate a Java code snippet for the following request: " + prompt + "\n\nProvide only the code.";
        return chatModel.generate(fullPrompt);
    }
}
Note on PostConstruct: If you are using Spring Boot 3.x and encounter errors related to javax.annotation.PostConstruct, change the import to jakarta.annotation.PostConstruct;.

4.3. Create a REST Controller
Now, expose this AI service via a REST endpoint.

Create a new package, e.g., com.javayou.llmjavaapp.controller, and add AIController.java:

Java

package com.javayou.llmjavaapp.controller;

import com.javayou.llmjavaapp.service.AIService;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/api/ai")
public class AIController {

    private final AIService aiService;

    public AIController(AIService aiService) {
        this.aiService = aiService;
    }

    @GetMapping("/chat")
    public String getChatResponse(@RequestParam String message) {
        return aiService.chat(message);
    }

    @GetMapping("/generate-code")
    public String getGeneratedCode(@RequestParam String prompt) {
        return aiService.generateJavaCode(prompt);
    }
}
4.4. Running Your Application
Open your terminal or command prompt in your project's root directory (llm-java-app).

Run the application:

Bash

mvn spring-boot:run
or

Bash

./gradlew bootRun
Once the application starts (usually on http://localhost:8080), you can test your endpoints:

Chat: Open your browser or use Postman/curl: http://localhost:8080/api/ai/chat?message=What is Java?

Generate Code: http://localhost:8080/api/ai/generate-code?prompt=A simple 'Hello, World!' method

You've just built your first Java LLM application using LangChain4j and Spring Boot, integrating with OpenAI!

5. Local LLM Integration with Ollama: Privacy, Performance, and Control
While cloud-based LLMs like OpenAI are incredibly powerful, there are compelling reasons to explore local LLMs:

Data Privacy: Your data never leaves your infrastructure.

Cost Efficiency: No per-token charges after the initial model download.

Offline Capability: Run models without an internet connection.

Customization: Fine-tune models locally for specific use cases.

Performance: Lower latency for certain applications, especially if deployed on powerful local hardware.

Ollama is a fantastic tool that simplifies running open-source LLMs locally. It provides a straightforward way to download, manage, and interact with models like Llama 2, Mistral, Gemma, and more via a simple API.

5.1. Install Ollama
Download Ollama: Go to https://ollama.com/download and download the installer for your operating system (Windows, macOS, Linux).

Install Ollama: Follow the installation instructions. Ollama will run in the background as a service.

Download a Model: Open your terminal and download a model, for example, Mistral:

Bash

ollama run mistral
This command will download the mistral model (it might take some time depending on your internet connection) and then launch an interactive chat. You can type a message and ollama will respond. Type /bye to exit.

You can browse other available models at https://ollama.com/library.

5.2. Integrate Ollama with LangChain4j
LangChain4j provides specific support for Ollama.

First, add the Ollama dependency to your pom.xml or build.gradle:

For Maven (pom.xml):

XML

<dependency>
    <groupId>dev.langchain4j</groupId>
    <artifactId>langchain4j-ollama</artifactId>
    <version>0.28.0</version> </dependency>
For Gradle (build.gradle):

Gradle

implementation 'dev.langchain4j:langchain4j-ollama:0.28.0' // Match core LangChain4j version
5.3. Update Your AI Service for Ollama
Now, you can either create a new service or modify AIService.java to conditionally use Ollama based on configuration. For simplicity, let's add an Ollama-specific chat method.

Modify AIService.java:

Java

package com.javayou.llmjavaapp.service;

import dev.langchain4j.model.ollama.OllamaChatModel; // Import Ollama model
import dev.langchain4j.model.openai.OpenAiChatModel;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;

import javax.annotation.PostConstruct; // For Spring Boot 2.x
// import jakarta.annotation.PostConstruct; // For Spring Boot 3.x if using Jakarta EE

@Service
public class AIService {

    @Value("${openai.api-key:#{null}}") // Make OpenAI key optional for Ollama-only use
    private String openaiApiKey;

    @Value("${ollama.base-url:http://localhost:11434}") // Default Ollama URL
    private String ollamaBaseUrl;

    @Value("${ollama.model:mistral}") // Default Ollama model
    private String ollamaModel;

    private OpenAiChatModel openAiChatModel;
    private OllamaChatModel ollamaChatModel; // Declare Ollama model

    @PostConstruct
    public void init() {
        if (openaiApiKey != null && !openaiApiKey.trim().isEmpty()) {
            openAiChatModel = OpenAiChatModel.builder()
                    .apiKey(openaiApiKey)
                    .modelName("gpt-4o-mini")
                    .temperature(0.7)
                    .logRequests(true)
                    .logResponses(true)
                    .build();
        }

        // Initialize Ollama model
        ollamaChatModel = OllamaChatModel.builder()
                .baseUrl(ollamaBaseUrl)
                .modelName(ollamaModel)
                .temperature(0.7)
                .build();
    }

    public String chatWithOpenAI(String userMessage) {
        if (openAiChatModel == null) {
            return "OpenAI model not configured. Please provide openai.api-key.";
        }
        return openAiChatModel.generate(userMessage);
    }

    public String chatWithOllama(String userMessage) {
        return ollamaChatModel.generate(userMessage);
    }
}
5.4. Update Your Controller for Ollama Endpoint
Modify AIController.java to add a new endpoint for Ollama:

Java

package com.javayou.llmjavaapp.controller;

import com.javayou.llmjavaapp.service.AIService;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/api/ai")
public class AIController {

    private final AIService aiService;

    public AIController(AIService aiService) {
        this.aiService = aiService;
    }

    @GetMapping("/chat-openai")
    public String getOpenAIChatResponse(@RequestParam String message) {
        return aiService.chatWithOpenAI(message);
    }

    @GetMapping("/generate-code-openai")
    public String getOpenAIGeneratedCode(@RequestParam String prompt) {
        // You'd need a separate method in AIService for OpenAI code gen now
        return aiService.chatWithOpenAI("Generate Java code for: " + prompt);
    }

    @GetMapping("/chat-ollama")
    public String getOllamaChatResponse(@RequestParam String message) {
        return aiService.chatWithOllama(message);
    }
}
5.5. Running with Ollama
Ensure Ollama is running in the background (it usually starts automatically after installation).

Ensure you have downloaded the model you specified in application.properties (e.g., ollama run mistral or ollama pull mistral).

Start your Spring Boot application: mvn spring-boot:run or ./gradlew bootRun.

Test the Ollama endpoint: http://localhost:8080/api/ai/chat-ollama?message=Tell me a short story.

This setup allows you to seamlessly switch between or even combine powerful cloud-based LLMs and efficient local models, giving you immense flexibility in your Java Generative AI applications.

6. Advanced Concepts & Next Steps
You've built a foundational LLM application. The journey doesn't stop here! Consider exploring:

Retrieval-Augmented Generation (RAG): Integrate vector databases (like Chroma, Weaviate, Pinecone, or even simple in-memory vector stores with LangChain4j) to allow your LLM to answer questions based on your private, proprietary data, drastically reducing hallucinations and improving relevance.

Chains & Agents: Design more complex workflows where LLMs can use tools (like searching the web, calling external APIs, or executing code) to achieve multi-step tasks.

Memory & Conversation History: Implement different types of memory (e.g., chat message history, summary memory) to enable more natural and context-aware conversations.

Prompt Engineering: Master the art of crafting effective prompts to elicit the best responses from LLMs.

Model Fine-tuning: For highly specific tasks, fine-tuning smaller open-source models (with tools like Ollama and more advanced frameworks) can yield superior results and efficiency.

Deployment Strategies: Beyond local development, learn about deploying your Spring Boot AI applications to cloud platforms (AWS, Azure, GCP) or Kubernetes.

Conclusion: Java's Bright Future in Generative AI
The integration of Generative AI into Java applications is no longer a distant dream but a tangible reality. With powerful frameworks like LangChain4j and the robust ecosystem of Spring Boot, Java developers are incredibly well-equipped to build scalable, reliable, and innovative AI-powered solutions. Whether leveraging the might of cloud APIs like OpenAI or harnessing the privacy and control of local models via Ollama, Java is proving its enduring relevance at the cutting edge of software development.

Embrace this exciting new frontier. Your Java skills are not just foundational; they are your gateway to building the next generation of intelligent applications. Happy coding!