Published on Tue Oct 06 2026 00:00:00 GMT+0000 (Coordinated Universal Time) by Miguel Gonzalez Herrera

Using Claude or ChatGPT to search case law is convenient, but it has a cost that matters in a law firm: everything you type or attach in the chat is sent to the model. If you paste a contract or a claim to ask for related rulings, that text leaves your computer. In this guide you will set up the same thing with a language model that runs on your own machine: Ollama as the model, AnythingLLM as the chat window and the Apify MCP to search CENDOJ, the Spanish judicial documentation database.

⚠️ It’s experimental. It works, but a small local model uses tools worse than Claude or ChatGPT: sometimes it picks the wrong tool, fills in the wrong parameters or makes up part of the answer. Always verify the rulings in the original source before using them.

🔒 Why a local model?

• Privacy: your conversations and the documents you share in the chat stay on your computer. They are not sent to OpenAI or Anthropic.
• Cost: no subscription and no Claude or ChatGPT tokens to pay for.
• Independence: the model doesn’t depend on any company or on an internet connection, only on your machine.

So what leaves your computer? Only the search terms the agent sends to Apify and CENDOJ. The results are public court rulings. Just don’t put personal or confidential data in the search itself.

🧩 What we are building

You ──► AnythingLLM (chat and agent) ──► Ollama + gemma4:e4b (local)
            │
            └──► Apify MCP (cloud) ──► legaltech/cendoj ──► CENDOJ

• Ollama runs the language model on your machine. We’ll use gemma4:e4b.
• AnythingLLM puts a chat window on top of it, so day to day you use it just like ChatGPT or Claude.
• The Apify MCP is the connector that gives access to the legaltech/cendoj actor, which searches rulings, orders and agreements in CENDOJ and lets you filter them by jurisdiction, court, type of decision and dates.

💻 What you need

• RAM: 16 GB minimum; 32 GB recommended.
• Processor / GPU: Apple Silicon (M1 or later) or an NVIDIA GPU with at least 8 GB of VRAM.
• Free disk space: about 15 GB. The model alone is about 9.6 GB.
• System: macOS 13+, Windows 10/11 or Linux.
• An Apify account. If you don’t have one yet, create it here.

Without a GPU it works, but very slowly: it can take minutes per answer.

1️⃣ Step 1: Install Ollama and the model

This is the only step where you need the terminal: a window where you type commands instead of clicking buttons. It can feel daunting if you’ve never opened it, but you just copy and paste one command.

1. Download and install Ollama from ollama.com/download.

2. Open the terminal:

   • Mac: press Cmd + Space, type Terminal and press Enter.
   • Windows: press the Windows key, type Terminal (or PowerShell) and press Enter.

3. Copy this command, paste it into the terminal and press Enter. It downloads the model, so it may take a while:

   ollama pull gemma4:e4b

🎉 Congratulations! You now have a language model running on your own computer.

2️⃣ Step 2: Install and configure AnythingLLM

1. Download AnythingLLM Desktop from anythingllm.com/download and install it.
2. Go to Settings → AI Providers → LLM and choose:
   • Provider: Ollama
   • Base URL: http://127.0.0.1:11434 (usually detected automatically)
   • Model: gemma4:e4b
3. Create a workspace (for example, legaltech). In its settings, under Agent Configuration, also select Ollama and gemma4:e4b as the agent model.

3️⃣ Step 3: Copy your Apify token

Go to Apify Console → Settings → API & Integrations and copy your Personal API token (it starts with apify_api_...).

🔐 Treat it like a password: don’t share it and, if it leaks, revoke it and create a new one from the same screen.

4️⃣ Step 4: Connect Apify to AnythingLLM

1. In AnythingLLM, go to Settings → Agent Skills → MCP Servers and click the book icon. A text file with the configuration opens.

2. Delete whatever it contains and paste this. Replace YOUR_APIFY_TOKEN with your token from step 3, keeping the word Bearer and a space in front. Save the file.

   {
     "mcpServers": {
       "apify": {
         "type": "streamable",
         "url": "https://mcp.apify.com/?tools=actors,docs,runs,storage,tasks,legaltech/cendoj,legaltech/hacienda,legaltech/tribunal-constitucional,legaltech/tjue,legaltech/fiscalia,legaltech/catastro,legaltech/ine,legaltech/aepd",
         "headers": {
           "Authorization": "Bearer YOUR_APIFY_TOKEN"
         }
       }
     }
   }

3. Go back to Settings → Agent Skills → MCP Servers and click Refresh. “Apify” should show as On, with its list of tools.

5️⃣ Step 5: Enable only the tools you will use

The Apify connector comes with about 30 tools, and each one’s description is sent to the model with every message. With all of them enabled, a small model has no room left to think: it hangs or makes up tools.

In Settings → Agent Skills → MCP Servers → Apify, leave only these three enabled:

• ✅ legaltech--cendoj: search rulings and extract their text.
• ✅ get-actor-run: check whether a long search has finished.
• ✅ get-dataset-items: collect the results of that search.

Disable all the others. If you later want another source (Catastro, Constitutional Court, CJEU, INE…), enable only its legaltech--… tool and disable the CENDOJ one: one at a time works best.

6️⃣ Step 6: Search for rulings

Open a new chat in your workspace and start your message with @agent, which is what turns on tool use. Try one of these examples (CENDOJ is a Spanish database, so the legal terms stay in Spanish):

@agent find 5 rulings on cláusula suelo

@agent find 5 rulings on gastos hipotecarios

@agent find 5 rulings on IRPH

@agent find 5 rulings from the social (labour) jurisdiction on despido improcedente

When the agent uses the connector you’ll see a notice like “MCP server: apify:legaltech—cendoj completed successfully”. Click the > arrow to see which parameters the model sent to CENDOJ.

✅ Best practices

• Open a new chat every few searches. Results take up a lot of space and the model gets lost.
• Ask for few results (5–10).
• Always verify. Look up the ROJ on poderjudicial.es or open the PDF. The model can summarise badly or mix up data.
• If the model falls short, try a bigger one with the same setup (for example, qwen3.6:35b-a3b if you have 32 GB of RAM or more). It’s slower, but more accurate.

🧭 Local, or Claude and ChatGPT?

A local model doesn’t replace Claude or ChatGPT: it is less accurate with tools and needs a powerful computer. But when confidentiality is what matters, it lets you work with client documents without them leaving the firm, and search case law from the same window.

You’ll find this same setup, step by step and with copy buttons, in the connector installation guide. And if you get stuck or have any question, write to us at legaltechnologybootcamp@gmail.com.

Written by Miguel Gonzalez Herrera