Skip to content
Skip to this exploration
All explorations

Practical self-hosting · intermediate

22 · The plug fits, but does the conversation?

Test API compatibility as observable behavior.

10 min 5 sectionsv1

What you can take away

  • Build a contract test matrix for a model client.
  • Use fake responses or an isolated test endpoint

Your trail through this idea

Checking saved progress…

Ian’s LocalAI/LocalAGI linkage questions expose a common trap: two systems can advertise compatible APIs while disagreeing on the features your application actually uses.

Prerequisite: complete the preceding lesson or bring its worksheet.

Mark sections complete when you choose. Reading alone does not mark them.

Understand the system

Ollama documents support for a subset of the OpenAI API. The word 'compatible' should therefore trigger a feature checklist, not a blanket assumption. Authentication, model names, message roles, streaming, structured output, tool calls, error responses, and cancellation are separate behaviors. Test the subset your client requires against the exact server version.

A gateway can normalize some differences, but every normalization is another behavior to validate. Silent fallback is particularly dangerous when it changes privacy, cost, or tool permissions. A rejected request can be safer than an undisclosed route to a different provider.

Checking saved progress…

Work through the case

Original contract matrix: a basic text request succeeds; a streaming response ends correctly; an invalid model returns a documented error; a tool call preserves its identifier; an interrupted request releases resources. Give each row an expected result and a captured sanitized observation. Do not mark the whole API compatible because the first row passed.

Checking saved progress…

Make something useful

Try this

Original exercise

Use fake responses or an isolated test endpoint. Write five tests covering success, malformed input, unsupported capability, timeout, and cancellation. Keep prompts synthetic and cap output. If running real requests, record server version and the model identifier without exposing endpoint addresses or keys. Revert any temporary gateway configuration after the test.

Runnable contract lab

This Python 3 lab uses fake responses, no network, credentials or model server. Save it as lesson_contract.py and run python3 lesson_contract.py. Python's unittest documentation explains the assertions. The fake contract is deliberately smaller than any real inference API.

import unittest

def read_text(payload):
    if not isinstance(payload, dict):
        raise ValueError("response must be an object")
    choices = payload.get("choices")
    if not isinstance(choices, list) or len(choices) != 1:
        raise ValueError("exactly one choice required")
    choice = choices[0]
    if not isinstance(choice, dict):
        raise ValueError("choice must be an object")
    message = choice.get("message")
    if not isinstance(message, dict):
        raise ValueError("message required")
    text = message.get("content")
    if not isinstance(text, str):
        raise ValueError("text content required")
    return text

class ContractTests(unittest.TestCase):
    def test_valid_text(self):
        self.assertEqual(read_text({"choices": [
            {"message": {"content": "Synthetic answer"}}]}), "Synthetic answer")

    def test_missing_choice(self):
        with self.assertRaises(ValueError):
            read_text({"choices": []})

    def test_nontext_content(self):
        with self.assertRaises(ValueError):
            read_text({"choices": [{"message": {"content": None}}]})

if __name__ == "__main__":
    unittest.main()

Expect three passing tests and OK. Change the valid fixture's content to a list and observe the failure. That is the point: a response can be valid JSON while violating the client's required contract. Restore the fixture afterward. This does not test streaming, authentication, tool calls or actual server compatibility; those need separate fixtures and integration tests. Remove the practice script for cleanup.

Checking saved progress…

Check your reasoning

A quick check · for your own learning

What is an honest API compatibility claim?

Choose one answer

A self-study prompt. You can retry; this is not an exam.

Checking saved progress…

Keep the result and follow the sources

Success is a compatibility statement with a scope: 'the tested behaviors passed on this version.' That is more useful than a universal compatibility badge.

Sources and scope

  • Primary documentation or archival source, reviewed 14 November 2026. Additional direct links appear beside the claims they support.
  • Editorial basis: Ian's historical self-hosting questions and current repository inspection informed the case selection. Historical requests do not establish current deployment. Private chat records, credentials and host details are excluded.
  • Examples and exercises are original educational synthesis. External sources support the stated technical or historical context, not every hypothetical scenario.
Checking saved progress…

These checks are for reflection and practice. Your answers and reflection notes are not sent anywhere.

Back to all explorations